First time in New Haven
Everyone's suggestions are great. Try Ibiza for the incredible flavors that come out of their food. Also, wine list is excellent. Barcelona has a good wine list, but especially good if you know spanish wines. You can find great values if you know what you are looking for.
Let me add my recommendation for Ibiza, recognized as one of the very best Spanish restaurants in the US.with a very good Spanish-heavy wine list.
Certainly, the other dining "destination" (other than what has already been noted) are:
Union League Cafe
I know that many contributors here like Foster's, but my experiences there have been decidedly uneven.
One other suggestion: Nini's Bistro - very well priced prix fixe menu and BYOB. I love being able to bring really top shelf wines here that I would not generally order at restaurant mark ups. We ate there again last Sat night and had wonderful meals.
Bentara & Central Steakhouse on Orange st have great food and New Haven's best wine list. They have the same owners so their wine list are almost interchageable. You could even dine at Central's wine cellar downstairs. Definetly two of New Haven "musts". Nini's Bistro, also on Orange St has great food and is a BYOB. Have a great weekend in NH