Grande Case has a plethora of dining spots for fresh fish but regretfully many serve Parrot Fish and other non sustainable fish. Please avoid Grouper and head to where they serve Mahi Mahi. Le Petite Auberge de Isles used to be nice.
While you can purchase seafood in places like Grand Marche or Market Garden (2 American style supermarkets), you might want to check out Skip Jack's in Simpson Bay. It is a restaurant that also sells fish retail.