Neither are on the sound, but I like FISH in Portchester and love Rowayton Seafood in Rowayton. Rowayton is definately the "nicer" of the two with a great balance of fine dining in a casual atmosphere. The food is great and it is on the scenic 5 mile river. There is a place to have cocktails out closer to the water before and after your meal as an alternative to the often crowded bar.
F.I.S.H. in Port Chester is very good, although not as elegant at L'Escale, it is still upscale and attractive. Also, there is a good fish restaurant in Stamford, (where Rusty Scupper used to be - can't remember the name). We went there last year and thought it was also very good.
The one in Stamford is called "Saltwater Grille". Had a very nice dinner on the deck recently. Seems like last year's 20-something crowd must have moved on since the deck was not full on a really nice Friday night.
Since others may read this thread I thought I would mention our fav waterfront restaurant. It is NOT appropriate for the type of dinner you are planning, but for a great casual meal try the Sunset Grill in Norwalk.
the jfood went to sunset grill at calf pasture beach a couple of weeks ago.
just a head's up on the food.
The pastas are very good, the artichoke appetizer is also good. AVOID the fried belly clams, they are inedible and are probably frozen at delivery and just thrown in the hot oil. On a scale of 1-10 they do not even get an integer. This would be jfood's guess on all the fried fish/seafood.
But the place is in a great location and the pastas are very good.
If you want VERY casual, fair service, Cape Cod fried type place, try SONO seafood as well. They have great fried clams and a nice deck overlooking the inlet. Not fancy at all, and service is QSR caiber, but the fish is very good (avoid the salad here though, blech).
Both of these places are at the other end of the fancy spectrum from L'Escale for full disclosure.
Then you have Rowayton Seafood. Food is fantastic and upper casual in demeanor. If you can grab a reservation here, that would be jfood's recommendation.