Raspberry Pi Connectivity Alert

I have a recurring issue where I’ll walk by my shelf of Raspberry Pi’s and knock the cord out of something. Many times, I won’t know about it for days. So I made them blink when I do this.

My original solution deployed TICK Stack, with each individual Pi feeding stats into InfluxDB, and Kapacitor configured to alert on sudden loss of metrics from a host. This worked, until I knocked the cord out of the Pi running TICK Stack. I could conceivably also knock the cord out of the mail server, and not receive any alerts from TICK at all.

I considered using the audio interface of the Pi’s to play back a sound, with each Pi playing back a different sound to identify itself (side note: if you haven’t seen Andon in action, I highly recommend going on a plant tour at TMMK - their tour of the production floor is fascinating). This would have required keeping several pairs of amplified speakers running, and with the entire cluster coming in at 27 watts today, it seemed excessive.

This is a solved problem, of course. Ethernet cables have keys. But my Ethernet cables are old, and some are missing the keys. But they’re perfectly good otherwise, and replacing them would’ve been wasteful.

Solution

Since I have the Pi cluster in eyeshot several times per day, I wrote a minutely cron that responds to several possible error states:

1. If the local network and Internet are reachable, the LEDs keep their normal function: red for power, green for SD access.
2. If the local network is reachable, but a ping to my ISP’s DNS server fails, turn green off and fast blink the red LED.
3. If the local network and Internet are both unreachable, fast blink both LEDs.

The separate checks help determine the source of the fault: did I knock the cable out of the Pi, or out of the switch?

The code is available as a GitHub Gist.