I grew up in the Princeton area and most of the places I remember most fondly are of the casual variety.
There is also a longer post on these boards which lists a lot of the best places in Princeton.
For ice cream it was Thomas Sweet on Palmer Square and Nassau Street ..
For a yummy breakfast/casual was a big fan of PJ's Pancake House on Nassau Street ..
For yummy wings .. Chuck's on Witherspoon Street.
The place I crave most though (and its a stand up sandwich place) is Hoagie Haven on Nassau Street which is one of the best sandwich shops anywhere and a place people who move away crave.
I would look at the boards for fancier places, but did have fond memories of the Ferry House.