We need more information to give you recommendations that will best meet your needs; otherwise we'd just be rattling off names of some of our hundreds of good restaurants.
Where will you be in downtown Chicago? The areas collectively known as "downtown Chicago" cover a wide area, three miles from the Gold Coast to the South Loop. Let us know what hotel (or major intersection) you'll be traveling to and from, so that we can recommend places close by.
What kind of foods do you like? Do you like unusual ethnic foods? Creative contemporary cuisine? Steaks and seafood for the "meat and potatoes crowd"? Italian, French bistros, etc?
How casual? Really casual, like shorts and beach attire? Or are places with business casual attire okay (i.e. slacks and shirts/sweaters with collars)?
Any budgetary constraints? By any definition, "casual" rules out the big bucks places, and everything else probably wouldn't go over $30-35/entree or $100/person with wine/alcohol and tax/tip at most. If that's still significantly too much, say so, and we'll stick to cheaper places.