I am a huge fan of the Portsmouth Brewery. Their Thai mussels and salmon are my favorites. Great beer too. Even their burgers are great.
If that's too casual for you, I like the Riverhouse. Get a cup of chowder, it's amazingly good. And then there is also the Oar House, which is a little pricey but the food is good.
All of these places have online menus to check out.
56 Market St, Portsmouth, NH 03801
The Oar House
55 Ceres St., Portsmouth, NH 03801
53 Bow St, Portsmouth, NH 03801