If it's a savory dish, prepare a mirepoix and refrigerate that. Cut up all the non-perishable ingredients and cover them to store in the fridge. When you want to assemble your casserole, bring all the non-perishable ingredients together with the eggs, milk, etc. and bake. If you need some time for the mixture to meld you can mix all the ingredients the night before and bake it the next day. 24 hours (covered) in the refrigerator shouldn't cause any problems as long as you keep your culinary hygiene under control.
I wouldn't worry about the safety issue, assuming you're planning on cooking it fully at some point, but the quality of the casserole. Some do well left overnight to cook, eg. stratas are far better when they've had a chance to meld before baking but a lasagna tastes better cooked and then reheated. What are you planning on making?
I don't have an exact answer, but I would consider that eggs, once broken, do have bacteria in them that needs to be killed by cooking. That bacteria could be something harmless or as dangerous as salmonella. Breaking eggs and mixing them with food stuffs and keeping them uncooked that way for hours, even in a refrigerator, may just be giving that bacteria a medium to grow in. On that basis, yours truly, a neurotic admittedly, would probably recommend keeping it cold and in the fridge for no more than an hour or two. Less neurotic people may say days, but I'll put my degree and expertise in microbiology up against their judgment on this one. Is there some reason you can't cook ahead and then reheat? A texture thing, perhaps?