For non-soft shells:
In the city, I like Galway Bay (interestingly enough they have good crab cakes), El Torro Bravo for casual Mexican. If your infant is young and pretty sleepy you might even try Carrol's Creek or Hellpoint Seafood. McGarvey's also has good crabcakes, but at night it gets more bar-ish.
Out at the mall Punk's Backyard Grill is very good.
Agree with above poster for crabs. Mike's isn't bad either.
Carrol's Creek Cafe
410 Severn Ave., Annapolis, MD 21403
1525 S Philadelphia Blvd, Aberdeen, MD 21001