Fancy Cocktails before Babbo
How close do you need it to be to Babbo? The places that come to mind immediately could be crowded and may be further than you want but I'll throw these out there: Little Branch, Employees Only, and Pegu Club.
I just realised you need this for 5 people so that's probably a deal breaker.
Most fancy cocktail places will be very busy on a Saturday night so it might be a stretch to get a table/booth for a party of 5 around 8pm. I'd aim for a bit earlier, 6:30 pm or 7pm.
Pegu and Little Branch are probably the best in the surrounding neighborhood, assuming you want specifically a cocktail lounge/bar.
For Little Branch, they don't open until 7pm IIRC so if you get there a little beforehand and line up, you might be OK and get a booth (it's kind of small). Pegu Club opens much earlier, 5pm I think.
Greenwich Project is REALLY close (around the corner), and doesn't really seem like it would get as crowded as the others in the area. What I tried, however, wasn't nearly as good as Little Branch or Pegu Club. But given your size constraints, might be the best option?
Melibea and Wallflower are both new restaurants w/ serious cocktail programs and are a short walk away on Greenwich Ave. I haven't tried either yet. Since they're both new, you might get in easily since they're not on a lot of people's radars yet.
All of these are about a 10 minute walk away except for Greenwich Project.
I might start with Melibea or Wallflower (just to see how crowded their bar areas are), and use Greenwich Project as a backup.
GP is also on OpenTable but they might assume you're there for dinner and not just drinks if you make a reservation. When we were there, we just walked in & sat at the bar.
With a crowd that size on a Saturday at that time I'd recommend two wine bars. Eighth Street Wine Bar and Amalie are within steps of each other, close to Babbo and have full cocktail menus. You won't get the custom-crafted cocktails you'll get at Employees Only or Greenwich Street Project but you'll probably get seating for your whole party and good cocktails and outstanding wines by the glass.
So basically, Eighth St. Send a scout to check out Greenwich St Project, Eighth St. Wine, Amalie, and Pour George (across the street from Eighth St., decent cocktails) and have everyone gather at the place with the most seats.