Balthazar. Make a reservation. Get the waffles, french toast, bacon, and/or le panier bread basket.
I have also heard good things about Hundred Acres and Aquagrill's brunch but haven't been. Aquagrill is great for dinner, too, and you can get a zillion varieties of oysters if that interests you.