Oh jeez, eating is the national pasttime in Taiwan. There is good food on literally every corner. Most people/tourists hit Ding Tai Fung for shiao leung bao and then turn the corner for mango ice at Ice Monster. Both places are at Yong Kang Jie. There is a cheaper alternative to Ding Tai Fung across the street from the CKS Memorial Hall but you'll have to sit on the sidewalk. The food is still great.
Night markets are also great destinations for eats. Shi Lin is the biggest. Ma la guo AKA numb spicy hot pot is also a must.
There's just too much. I will consult my list of business cards when I get home. How long will you have and where are you staying?
I second the night markets. Stick with cooked and popular and you should be fine.
Ji Pai is a favorite of mine - Taiwan style fried chicken. Also good are the green onion pancackes, dumplings of all shapes and sizes, grilled squid, octopus balls, these little fish that are battered and deep fried and oyster omlettes. A lot of the other stuff I'm not sure about the names - I usually just follow my nose (except in the case of stinky tofu, where my nose shuts down in self defense).
Eating out is cheap too. It's actually cheaper for me to eat out than it is to cook for myself, but I like to cook so I do it anyways.