Unless your Sunday museum time is near r. Cler (Musee Branly), I would not make a special trip. Except for the excellect Cantin, Lenotre and the bakeries at St. Dominque, I don't find the Cler open market to be anything special. I agree with the above post that the organic market at bd Raspail is excellent and also more convenient on a Sunday if you are doing the D'Orsay, Rodin and even the Lourvre.
monday is when food shops are almost all closed in Paris. Sunday morning (til 1pm top) has many markets open, including rue Cler. But some shops, in particular Cantin (cheese) are closed on sundays.
and sunday morning is when there is the organic market bd raspail: great ingredients and French stars shopping.