I haven't been in a while but I've been there around 3 pm on a weekday once or twice and didn't have to wait at all. It was more crowded than I would have expected at that time, but there were seats available.
I've had the same experience at those times. Obviously 1pm is more crowded than 3. If you need more than a two-top there is a small chance you'll have to wait a bit.
You're also at an advantage this time of year because they have outdoor seating.