How far are you willing to walk? Personally I think the best brunch options are in the West 80s but I used to live up there, so I'm biased.
The Neptune Room (my favorite)
Sarabeth's
Good Enough to Eat
Nice Matin
Popover Cafe
Popover Cafe and Neptune Room will probably have the shortest waits.