In La Jolla, there are two restaurants I enjoy. The Cottage and Barbarella's. They both feel very Southern California and the food is good. The Cottage is in the heart of La Jolla and has great breakfasts and lunches at a good price. Barbarella's is in La Jolla Shores and has good brunches and dinners - also good prices.
El Prado in Balboa Park is a great restaurant. You can dine indoors or out. The outside patio is very romantic with twinkling lights and lots of plants. The food is excellent...the thick pork chops taste great.
It's a beautiful area to stroll around and admire the architecture and romantic setting.
If by special you mean you are willing to lay down some bucks, I could recommend either George's at the Cove, where Trey Foshee does some inventive work using, among other great ingredients, the full glory of the produce of the Chino farm; or Parallel 33, which pays tribute to the cuisines of our latitude.
In what part of San Diego will you be staying and will you have a car? San Diego is pretty easy to get around in, but it's not exactly compact, a car may be essential. Also what price range do you want.
If worst comes to worst, just have someone drop you off at the intersection of 4th or 5th and University. Walk 3 or 4 blocks in any direction and you will find more restaurant choices than you could possibly eat in in a month. Most of them are adequate to good, covering the gamut in price range and cuisine choices.