Grand Centeral Market has a decent selection. Also, Kalusyians on 28 and 3rd has a wall of hot sauces. The homemade hot sauce at waterfront ale house on 3nd and 29th is hot and has great flavor. Definitely pick some up.
123 Lexington Ave, New York, NY 10016
Waterfront Ale House
540 2nd Ave, New York, NY 10016