Two places spring to mind. Casual and no buffets. The Front Porch in the Penguin Hotel on Ocean Drive, and Balans on Lincoln Road, both of them offering a full bar so you can indulge in Mimosas or Bloody Marys if you wish and both having a well-balanced menu without being frou-frou or over-the-top expensive. The cred comes from being popular with the locals.
One thing I've noticed about Miami, it's not a brunch city. We could use some more offerings.
Nemo. If you are going in gentler weather, the patio is lovely but the restaurant is great inside as well. I had the best croissants here. The buffet is lovely and nary a food service type tray in sight. (I personally think many chef's leave the Sunday buffet to Sysco or Orval Kent).
Had a lovely brunch at Baleen but it's been awhile for me so I don't know how it is currently. Plan to find out in the next month though. LMF