Where are you staying? If you want to dine in the FQ go to Antoine's or Arnaud's for Sunday Jazz Brunch. I like Antoine's the best. Be sure to check out Hermes Bar at Antoine's or French 75 Bar at Arnaud's. Mr. B's has an outstanding Sunday Jazz Brunch. It's across from Hotel Monteleon.
In Mid City near City Park is Ralph's which has a fine SB and outdoor dining. Redemption is a neat restaurant in an old church in Mid City. If you go to the GD or Uptown go to Coquette or Patois for Sunday Brunch. Cafe Atchafalaya has a Sunday Jazz Brunch. I really like CA. It's a unique neighborhood restaurant near the Irish Channel W of the GD.
Dante's is Uptown and has Sunday Brunch.
Antoine's @ 713 Saint Louis St., New Orleans, LA. 504 - 581 - 4422.
Arnaud's @ 813 Bienville Street, New Orleans, LA 504 - 523 - 5433.
Mr. B's Bistro @ 201 Royal St., New Orleans, LA. 504 - 523 - 2078.
Muriel's @ 801 Chartres St., New Orleans, LA. 504 - 568 - 1885.
Cafe Adelaide @ 300 Poydras St., New Orleans, LA 504 - 595 - 3305.
Ralph's on the Park @ 900 City Park Ave., New Orleans, LA. 504 - 488 - 1000.
Redemption Restaurant @ 3835 Iberville Street New Orleans, LA 504 – 309 – 3570.
Cafe Degas @ 3127 Esplanade Ave., New Orleans, LA 504 - 945 - 5635.
The Columns Hotel @ 3811 Saint Charles Avenue, New Orleans, LA 504 - 899 - 9308.
Coquette @ 2800 Magazine St., New Orleans, LA. 504 - 265 - 0421. http://www.coquette-nola.com/index.html
Patois @ 6078 Laurel St., New Orleans, LA 504 - 895 - 9441. http://www.patoisnola.com/
Cafe Atchafalaya @ 901 Louisiana Ave., New Orleans, LA. 504 - 891 - 9626.
Dante's Kitchen @ 736 Dante St., New Orleans, LA. 504 - 861 - 3121.
Here's a link to Sunday Brunch restaurants in NOLA.
The best brunch is on the Jazz fest grounds. No need to go anywhere else with the variety of food at fest. I think the grounds open at noon and the last performance is around 5 ish. But my memory maybe wrong on the times. Regardless, why take 2 hours out of fest to go to a brunch?
If you're trying to eat enough to fill you up while enjoying yourself at the fest, it has to be on local's day. Otherwise you just spend your day standing in line. Grabbing lunch/brunch before going and snacking at the fest is a much more reasonable thing to do.
I'd probably go somewhere like Atchafalaya where you can make a reservation.
Jazz Fest is from 11:00 am to 7:00 pm. If you arrive early ready to eat, you will have no trouble having enough to eat before the lines get long, and again in late afternoon the lines go down. Even when the food areas are very crowded, the lines aren't really that long. I've been in much longer lines at event like the N. O. Wine and Food Experience.