The Modern or Perry Street?
I've eaten at and really liked both. At The Modern, we did a pretty elaborate tasting menu, so it seemed like a more special meal. The food and setting were terrific. But I also really enjoyed Perry Street. It's a little more casual, but I had some outstanding dishes (although dessert was a little disappointing). Tough choice.
I can't speak for the Modern, but I did have dinner at Perry St. about 4 months ago with a party of 10. I remember the food being good, but nothing was memorable (and for the prices JGV charges I think the food should be memorable). The service was also very slow - almost painfully slow towards the end of the meal.
Overall we had a good time and the food was fine, but I would look for something else if you are expecting a memorable meal.