Really casual local favorite is monterey fish house:
Sunday open 5-9pm, sorry linked to yelp but they don't have a website! Ha! Really super fresh simple seafood, reasonable prices
Also +1 for passionfish rec, but make a reservation-great wine list there too
It might be a hard reservation to get, but right next to Monterey, just a few blocks away, is the small town of Pacific Grove. There is a lovely restaurant on Lighthouse Ave. called Passionfish. They only serve sustainable seafood, have a creative, constantly changing menu, and if I'm not mistaken, have won a James Beard award. Otherwise, I would recommend Restaurant 1833, or if you don't mind corporate, the Chart House on Cannery Row has wonderful seafood, and the best view of the water in town. I lived there for two years. Watch out for the overly touristy places. Hope this helps.