Smokeless Joe's (John St., south of Richmond)... the menu is limited, but the food is very good. The beer selection is great. And the atmosphere is nice and cozy with a sort of secret-Toronto, you-found-where-the-locals-know feel.
I like Bar Wellington (portland & wellington)... about 5-10 minute walk from where you'll be. The only time to avoid it is late on Thursday-Saturday nights, when it gets rowdy from people who can't get into the bars on King St. W.