I live in the area and have eaten at Whym many times. The food is pretty good, though not outstanding. There is a weekday lunch special and the lunch crowd tends to be a bit older than the dinner crowd, at least in my experience. But I am in my mid 20's and have taken my parents, who are older than you, for both brunch and dinner there and neither of them complained that they felt like senior citizens (and that is a frequent complaint they have of restaurants that I generally choose to take them to).
I guess it does have a young vibe, but I don't think you will feel like senior citizens there, it's definitely not immature. The food is good, I was especially impressed by the desserts - a pear bread pudding I think for my dining companion and I just had some strawberries with fresh whipped cream that was nice and simple.
I was there a few weeks ago, and I would say probably 3/4 of the patrons were in their 20s or early 30s, while the other 1/4 was older. I would, however, highly reccomend going. I do not think you will feel like senior citizens at all! The food was very good - I had a delicious chopped salad with scallops for the main, and I also enjoyed their specialty cocktails. In comparing it to Eatery, its sister restaurant, I would say it actually feels more "adult" though their prices are slighly higher than Eatery's. So, yes I would say go!