We spent three nights in Reims in February and enjoyed it very much. Though, to be honest, the restaurant scene in Reims leaves a bit to be desired. Yes, there are the crazy nice places like Assiette Champenois, where we stayed and ate. But if you want to go downscale from there, it's slim pickings.
We had a nice, relaxing lunch at Les Cornichons in downtown Reims. It was not the most amazing meal, but it was generally well prepared, the service was good, and the menu was nice. I'd suggest going with the menu of the day -- it was a nice piece of fish when we visited. That was the best deal and the tastiest item. It was crowded with locals who all seemed to know each other -- maybe they came from City Hall across the street? -- but we only had to wait five minutes to get a table for a weekday lunch.
Le Petit Comptoir was a fine, competent restaurant, but after eating in so many similar restaurants in Paris, this one just did not stack up.
The champagne tours/tastings in the Reims area are astounding. See one of the major houses in Reims, but I strongly recommend visiting some of the tiny producers in the small towns south of Reims. Cattier, R&L Legras, Bergeronneau, Daniel Dumont, etc. -- all of those are great places to visit and will forever change your view of champagne producers and varieties.
The restos mentioned are all pretty fancy. If you want something easy for a jetlagged tummy I would try a Louise -15 Rue Marx Dormoy, Reims - 03 26 78 00 61. Great crepes and side salads for a healthy little dinner. Restaurant de La Paix is pretty solid for tartare. If you want to go to a Microbrewery and eat regional food from Alcase region (Stratsbourg, France) go to Le 3 Brasseurs and try some local beer, eat some choucoute or Flammeküche (pizza like). I'm American but lived in the region for a year if you want to email me for advice email@example.com