You could also try the Peartree on Parliament St. just above Carlton, they do a great brunch. You might want to phone ahead for a reservation, they are likely to be very busy. On and off street parking and easy to get to.
Edward Lévesque's Kitchen does a great brunch (and lunch and dinner) on Queen St. East, north side, between Leslie and Greenwood (right in front of the Beer Store). You may want to check if he's open on Easter though (I know he is on Good Friday).