Relatively cheap classic New York
Best rarely = Cheap. What do you mean by "Cheap?"
I agree with most of the recs.
Katz's and Pastrami Queen pastrami sammich is excellent but not cheap I.M.H.O. unless of course you share a sammich. They are both huge!!
I am a lover of Junior's strawberry cheese pie. Eileen's is also a favorite.
In order of greatness: Ess-A-Bagel, Tal Bagel and Bagel Boss (In the old Vic's Bagel Bar location next to soon to be reopened Sarge's)
New School--South Brooklyn Pizza
knish--Maybe I have never had a good one but I think they're gross.
bagel--Like Kathryn says, freshest is usually the best.
Bagels have a short life. Which ever bagel store you go to ask for whatever is just out of the oven. A piping hot fresh bagel with butter is a wonderful thing. For Fish, Russ n Daughters, Sables, Barney GreenGrass and Zabar's all sell the good stuff.
hot dog--I am not really a hot dog person but a Grays Papaya/Papaya King is a very NYC thing to do.
doughnut--Don't eat donuts
cheesecake--S&S is my favorite cheesecake and they sell to many major restaurants and are available in whole cake form at Zabar's. For a slice Little Red Hens makes a very nice version.
1. Best slice: Many of the famous and notable NYC pizza joints don't do slices. Only whole pies. There is more also more than one type of indigenous NY style pizza. So you need to decide what you want. Coal oven or gas oven. Slice or whole pie.
Best NY style pizza:
2. Best pastrami:
I really enjoy Katz's Deli (from When Harry Met Sally). I hope you like fatty beef. Their bread leaves something to be desired and there have been complaints about their security guards on nights/weekends. YMMV.
Unfortunately with Sarge's being closed right now, it's not like we have a ton of great options to choose from (and 2nd Ave Deli, while having other good items, isn't as strong in the pastrami dept). Carnegie is alright, not great, in my book.
Mile End is also famous but they don't do NY style, it's Montreal style.
Here is a long discussion thread on best pastrami:
3. Best knish -- unfortunately, it's not like there's a lot of competition in knishes these days either.
Yonah Schimmel is the most famous, probably, but I've heard that the 2nd Ave Deli and Zabar's ones are better, at least in Manhattan.
4. Best bagel -- first, read this:
Summary: the freshest bagels are the best; bagels don't age well at all.
I think a lot of long time NYers would also say that our bagel scene isn't as good as it used to be, either.
Focus on the smoked salmon instead. Preferably at Russ & Daughters, so you can get a taste of what an appetizing store is like. Do a search on Youtube for when Louis CK visited, to get a glimpse of what it's like.
5. Best hot dog -- kind of a misleading category. Most vendors use the same recipe Sabrett beef dog around the city. If you are just interested in an everyday "typical NY" hotdog experience, then Gray's Papaya. It's not gourmet, at all, though. Cheap, filling eats, and fun for people watching late at night, I guess.
Here's an in-depth discussion:
6. Best doughnut -- go to another city, honestly. The West Coast and Chicago have us beat.
7. Best cheesecake -- I defer to the judgment of Serious Eats:
See also: Eats Unique to NY:
If you are interested in other foods that are specific to NY and hard to find elsewhere, you also need to be looking at William Greenberg's black and whites, egg creams from Gem Spa or Ray's, the Pickle Guys, the Halal Guys chicken/lamb over rice cart, Sigmund's pretzels.
Best pizza is way too subjective. (There are whole threads as was noted above.)
Best pastrami, every New Yorker agrees Katz's has the best pastrami. It is also a great NY experience and puts you a few doors from Russ and Daughter where you can get the best "fish." (Fish here referes to smoked/preserved type fish for bagels.) [NOTE: I was sarcastic about EVERY New Yorker agreeing on Katz's pastrami. It is really just every New Yorker who is correct....]
Criff Dog is great (and the speakeasy through the phone booth has bacon infused bourbon.) Katz's also has a really good dog. But neither of them will still be warm by the time you get to sit on the steps of Metropolitan Museum of Art and eat them. So I would do that and get them from a cart.
Cheesecake I vote Juniors. Unless you are going to be in Brooklyn anyway, get it at the Grand Central location.
Donut classic- I'm a big fan of Donut Pub on W14th Street.
Gourmet Donut- Donut Plant on Grand St. (lower east side) Around the corner is the Pickle Guys. Go There! Get a lot. Get the pineapple.
Since Yonah Shimmel went downhill (a few more doors east on Houston from Katz's/Russ & Daughter) I've read the "only halfway decent knish" left in NYC is in Queens. I'll defer but if Yonah Schimmel is open, try one.