Taillevent or Guy Savoy
Sorry to correct you both. Men are required to wear a jacket at Taillevent, even for lunch. In fact, they will provide your husband with one if he doesn't wear one. But Guy Savoy wants to be a more relaxed and modern experience and there really is no dress code there, even at night. So: jacket for Taillevent, no jacket for Savoy.
Oh, and I would chose Savoy, especially for a gentleman who does not want to wear a jacket. Taillevent is for those who want to eat at Ratatouille's Gusteau (nothing wrong with that!).
Why not? He will feel more comfortable. For Taillevent there is no specific dress code, but in my experience people generally dress well. It is popular with business executives at lunchtime so a lot of people are in suits. In France people do tend to dress well to go out to eat, it shows respect for the meal, and I suspect you get treated better.