There are lots of good places to choose from. I wouldn't miss Sunny Point in West Asheville, Flying Frog Cafe, Laughing Seed (all veg. Their sesame dipping sauce gives me shivers it's so good). Tupelo Honey is good.
As previously noted, it depends on what you're after. Downtown is really pretty walkable, though, so you have plenty of choices.
Middle eastern food Rezzazz is great. Try Sunny Point for breakfast best in town. Best meal is Gabrielles fancy Fixe price. Salsas and Doc Chey's are great for lunch and Melas (indian cuisine) has great atmosphere. I'd steer clear of the Grove Park's restaurants as they are pricey and food can be variable at best
Also check out Noodle Shop on Pack Square for good Asian - In my opinion better than Doc Chey...
Mamacitas for a great burrito!
Marco's for truly stellar pizza (I've lived in NYC and Italy)
Jack of the Wood for a nice pub atmosphere and good pub grub n' beer (non smoking - good music)
Although it is not fine dining, we had some of the best pancakes ever outside Asheville in Maggie Valley on our way to the Great Smoky Mountains. Joey's Pancake House. Their waffles were good, too. In fact, I ordered 2 pancake mixes for my husband for Christmas from there and he was so excited.