I was going to suggest Kings and Whole Foods as well...Gary's is a good bet, as is C'est Cheese on South Street in Morristown or the Summit Cheese Shop. If you want to take a slightly longer ride (Bridgewater), I'm sure Wegmans will have it as well.
Kings Supermarkets also carries it, although you may have to special order it if it is not during the holiday season. (there is a Kings in both Chatham and Morristown, so which ever is closer to you in Madison)
Try Gary's Wine & Marketplace on Main St. in Madison. They have a really impressive cheese counter and a large gourmet foods area. I can't say for certain that they carry caviar, but I would be surprised if they didn't.