Near Madison Square Garden
People's view of what "not too expensive" means depends entirely on their pocketbooks. That said...
The Pub Room at Keens (36th, b/t 5th & 6th) is less expensive than the main dining room
A few months ago, Uncle Nick's opened a satellite location on the corner of 8th & 29th. Same tasty Greek food as the original.
Two other possibilities:
Biricchino - http://ww.biricchino.com
Seven - http://www,sevenbarandgrill.com
Tir Na Nog is right by MSG and isn't too bad for traditional and modern Irish pub fare. It's one of the few places in town that does a proper sheperd's pie.
A brisk walk or cab ride east will give you access to Korean barbecue, south Indian, and all the options on / near Madison Square Park: Artisinal (cheese heaven / bistro), Eleven Madison Park (expensive place often recommended for special occasions), and that fusion / Indian-influenced restaurant with really good crab cakes whose name I'm blanking on at the moment.