Not sure what brunch items you're looking for but Aroma in the Annex and L'Espresso at Bloor and St. George open early and have breakfast items on their menus. I would think parking should be fine for both, though not sure if there may be no street parking before 9:30 on at least one side of Bloor during the week.
Although Lola's Commissary is east of Yonge, I'll mention it anyways, since it is located within a 5 minute walk of Yonge & Bloor. Street parking is available nearby. While Lola's doesn't offer the complete brunch menu during the week, some of the weekend brunch items are available on the weekday lunch menu. Lola's menu, available from 11 am-3 pm on weekdays: http://www.lolascommissary.ca/menus.html
The Coffee Mill (which is located a 5 minute walk north of Bloor) and Holt's Cafe (which is located on Bloor, on Holt Renfrew's second floor) are both located east of Yonge, and both are open at brunchtime/lunchtime Mon-Fri.
99 Yorkville, Toronto, ON M5R3K5, CA
50 Bloor St W, Toronto, ON M4W, CA
634 Church St, Toronto, ON M4Y 2G3, CA