Well, it depends on your family. Ten minutes from Santa Monica is Mariscos Chente, which is the darling of the Chowhound boards. It's not got fancy decor but every single time I've been in there there's been a party of 8 or 10 or 15 having the best seafood ever.
Or, if you want fancy shmancy, go to the Lobster on Ocean Avenue.
For the best Mexican style seafood try Las Serenata downtown. Check our their menu
For fancy and pricey, there's the Water Grill downtown. In Santa Monica there's ICugini which specializes in Italian seafood. www.icugini.com. And for very casual, there's Captain Kidds at the Pier in Redondo is said to be good.
There's Ocean Ave Seafood right there in Santa Monica - but I'm not sure it's as good as it once was.
KING FISH -in calabasas is tops...a few years back was voted best seafood by la magazine.. very fresh ..i tried chinnook salmon that was fabulous... mac/cheese yummy...has a crunchy top layer that fuses flavors.. there are also patios as well... give it a shot ..you will see why this place is packed...
I would suggest you look into Tlapazola on Lincoln. It is not billed as a "fish" restaurant but they serve great fish dishes, particularly their specials of the day.
The room is small but cosy and very well decorated.
There is also the original Tlapazola on Gateway.
Some of the best "fish restaurants" on the westside in my book are Japanese. Which doesn't always mean everyone has to have raw fish (the usual deal breaker for larger parties with mixed tastes) . But you know what's best for this particular gathering...