My main suggestions right off the bat would be La Tasca, and The Rusty Scupper
La Tasca is a spanish tapas place on the 2nd floor of one of the little mall/galleries on the inner harbor. While I know some people aren't always a big fan of it, I personally really like their food, it's small plates which are perfect for sharing and since the sizes are small you can try a number of different things for a very reasonable price. I've never had any complaints nor have any of the people I've brought there.
The second suggestion would be the Rusty Scupper, again it's a place that I know a number of people who may not be a fan but I've always enjoyed it. Especially for lunch it's perfect, they have great salads and sandwiches, it's definitely a good price, and the service is good.
Both places are pretty quick, both places have good food, both places have water front views of the inner harbor, and both are within walking distance of the Aquarium.
Yeah, you're not going to find a lot of Rusty Scupper fans here.
If it were me I'd walk across President Street and up a couple blocks and go to Attman's, the best deli in the area. Get the corned beef or pastrami on rye, or a hot dog topped with bologna.
Another good suggestion is Isabella's in Little Italy. Very good pizza and amazing sandwiches.
Both are inexpensive and both very chow-worthy.
No offense Hungry Fox, but both of your suggestions are chains and I think you can do better within easy walking distance. If you head east from the Aquarium over the foot bridge, you'll soon reach Eastern Avenue and President Street. On Eastern is Cafe Gias, which is inexpensive decent Italian (especially for lunch).