I like doing Italian for those types of occasions. Some antipasti to share, a warm atmosphere that lends itself to you leaning closer (or leaning back and just having fun), etc...
Crispo's got good food, it's affordable, and it's saved from making a strong romantic statement by being bustling and a bit loud. Pasta (carbonara and I've liked the mushroom paparadelle special) and the risotto balls there are good.
It's on 14th btw Seventh and Eighth Ave on the south side of the street
You're not giving me too much to go with here, as far as information. Not quite sure what you mean by either a "great atmosphere" or "won't break the bank". However, I used to use a couple places in the area as first dates spots. The food at each is good, not great, but good, fine, few bad surprises, sometimes quite tasty. At none of them have I ever spent so much as $100 for two for dinner and neither ever costs me less than $40 something. Neither are overly romantic, but I really never wanted such an atmosphere until I was on at least a fourth or later date. Finally, you give no clue as to if you are a man or woman, which would put different requirements onto how much you lead the way.
My best dates in this area would include dinner and then getting up, and going somewhere else for coffee and/or dessert.
The simplest would be to pick Trattoria Pesce & Pasta (Bleeker west of 6th, near Cornelia, I think), a good, reasonable and unremarkable Italian place with enough atmosphere for a date, but has also been fine for dinner with a grad school friend. My standard date plan here would be to order a cold antipasta plate of grilled veggies, etc. to share, and entrees, then direct my date to wait and order coffee and/or dessert across the sreet at Bruno's or Rocco's (same block as the restaurant toward 6th Ave.), both pastry places that serve coffee, espresso, etc. This plan worked so many times, without a single failure, that it just became my standard first date and/or the one that I fell back onto when other plans blew up.
For dessert one could also go to the ice cream place between this restaurant and 7th Ave, I think it is Cones.
Another similar choice invovled eating at Cafe Asean (10th St. just west of 6th Ave.), same type of price range, same quality of food, about the same atmosphere (tending toward a date place, but acceptable for other things). I might also walk to one of the same places for dessert (I have done so). One of my exes, particularly liked this place, because she liked the atmosphere and food. It has a back porch area too, but I prefer inside.
If I wanted something more expensive than this, I would probably opt for Mary's Fish Camp, walk up, give my name and cell number, then walk to Lederhosen for a drink. It's certainly upscale, with an atmospehre that feels like a cross between New England clam shack and chi-chi lounge. Afterward, maybe then one of the same places for dessert.
All of the above have worked for me as dates, but could work in the "I don't know if it is a date" category.