The other posters didn't mention the good falafel places DOWNTOWN like you asked.
I can recommend two:
1) Mystic Muffin at Richmond and Jarvis. Falafel's are good but the owner who makes them is hilarious.
2) Ghazale on Bloor St next to the Bloor Theatre between Spadina and Bathurst. I think these are the best in the city.
They are both reviewed on my website.
Although my recommendation for great falafel isn't downtown, I would recommend the trip to this place. The place is called "What a Bagel" (Note - if you're there, check out their bagels/other baked goods; they're great!). The place is located in Thornhill, at Bayview and Laureleaf, in the Longo's plaza (on Bayview, on light south of John).