Just moved back to Toronto from Queen's!
Depends on the cuisine you're looking for, this site is a good index: http://www.themenusite.ca/index.php
I would suggest Windmills for a more traditional and solid lunch. If you're looking for more upscale, I would recommend either Chez Piggy or Aquaterra (in the Radisson Hotel). Amadeus is also quite nice if you're looking for something German.
Hope this helps!