So why not just use a plain old lightbulb? Why complicate it with multiple pieces of equipment that could fail? No need for relays and the the like. Just a regular wall outlet wired only to grid power. If it's lit then you know.
That and if anything goes wrong with the grid power you'll only be risking that one light.
If you want an audible signal to go with it then how about something festive like using a string of Christmas holiday lights? One that starts automatically and plays a tune. Yes, it's an oddball way to do it but probably pretty cheap to obtain in the off-season now. Or maybe just a radio tuned to a local station. I've used beat-to-hell clock radio as a way to test AC circuits. Put it in a room, turn it all the way up and then walk back to the breaker panel. Flipping on/off, listening for when the radio is playing.
The trouble with most audible alarms is they get on your nerves pretty fast. Beep-beep-beep, ugh. Something that played a little tune is less likely to bug you as much.
Also consider whether or not the power is going to come back on while you're asleep.
Auto-transfer panels aren't outrageously expensive for what they offer. And that's an engineered solution to handling what can be very dangerous amounts of voltage and current. When you start creating solutions of your own you're asking for trouble, the consequences of which involve burning down the structure and potentially risking lives. Just raising the point.
-Bill