best steak place close to newburgh is in new windsor, schlesinger's steakhouse:
caveat: i haven't been there in 5+ years, but it was very good then and i keep meaning to go back.
italian - you sure you want to have italian in this area? i'd say boston would blow things away.
i haven't been to this one, but remember hearing good things about it (it's in newburgh):
not in newburgh, but fishkill, is il barilotto which is one of the better ones in the area. that one i have been to before and liked a bunch.