Ethnic Grocery Stores...
You can pretty much find anything you are looking for at Kalyustan's on 28th and Lexington. It is technically an Indian Market but they carry spices and food products from all over the world.
There is also The International Market on 38th and 9th ave. Small place with every spice and grain you could want along with other specialty ingredients. Homemade sheeps milk Greek yogurt, hummus, and feta. Also some of the best olives I have ever had. Down the street from there is Manganaro's Italian Groceria which is ok but not always great.
And while not European if you are looking for harder to find asian goods, Kam Man in chinatown on canal around Mott street is excellent.
I've been to both, but Di Palo really is my favorite. They are different places though - what I love about Di Palo, aside from the quality of the products and the selection, is the knowledgeable customer service. That comes with a price - time waiting in line at peak times, whereas Buon Italia is more amenable to browsing.
In terms of French - I can't think of a French speciality store, but certainly many "gourmet" stores carry many French ingredients. If you (OP) post further about what you are looking for, I'd be happy to make further suggestions.
For Spanish - there is Despana on Broome.