Rebooting radio in Santa Cruz mountains once a week via GitHub script?

If we wanted to automatically reboot our radios once a week, should we do it with a GitHub script on the radio or do you know of a better way?

We're on a classic small WISP outfit (2-man operation, maybe 100 homes) in the Santa Cruz mountains, where, whenever we call for technical support, they *always* tell us to reboot the radio - where - shockingly - often - the performance goes up (don't ask me why or how - it just does).

So let's not argue the fact that our WISP Internet speeds are better (for whatever reason) after we reboot our rooftop radios (which most of you would call a "modem" but we call them transceivers, or just radios, for short).

Let's just ask the question.

If we wanted to automatically reboot our radios once a week, can we do it with a GitHub script on the radio or do you know of a better way?

For example, how does this GitHub script look to you Linux coders like Marek Novotny and William Unruh?

Reply to
harry newton
Loading thread data ...

If you're just looking for a periodic reboot then the simplest way would be a 7 day timer with a poewr-down at (say) 03:45 Thursday morning. Alternaitvely, you could shut thing down Mon, Wed, and Sat.

If you're looking for "on demand", then there are numerous options available using cell phones.

Reply to
danny burstein

looks like it could work, if you've changed the admin passwords on your radios you'll need to update the script. if not you probably should.

one reason why rebooting may help is if you skipped the firmware release early this year, your radios may be being subverted by IOT malware. (there was a more recent patch for wifi insecurity - that should be applied too.

Reply to
Jasen Betts

I use a AC lamp timer to power cycle everything: Pick your flavor, they all work. I've been using these: The main problem is that the tiny button batteries don't last long enough.

The problem is that I don't trust software solutions to fix a problem caused by an operating system or application that has gone insane. The reset has to be done by something that won't be trashed, hung, or become too busy, which means an external device or independent "heartbeat" timer.

Before you complain about crashing your system by killing the power, please note that if your stuff can't handle a power glitch or failure, it doesn't deserve to be run unattended.

Reply to
Jeff Liebermann

Hope you really have another admin user/password than ubnt.

If you want the reboot to occur periodically, then a crontab job would be simplest.

If you want more control, then have periodical checks of a site, and if the value changes on the requested page say from 0 to 1, then reboot (then have a grace period when it will not check the page, or else it will keep rebooting until you change the value back to 0).

If making the page with some kind of logic, then each client sends a hash unique for them to you, if the value is in your database, then send value 0, if not send 1 and each time you want all units to reboot you empty the table.

Sure you could use nrpe to do the same thing as your ssh connection, but with the difference that you wouldn't need to know the password of the device, lock it with ip restriction. This will work quite nicely till someone does an arp poisoning and make it look for the device that you would be sending the nrpe request while it's someone else. I have used nrpe where the client been allowing request from a dynamical ip server, this has the small drawback that you have a minute while you can't access the nrpe service, but I do rather use it than ssh which requires you to configure username/password in nagios/icinga.

Reply to
J.O. Aho

Klugey approach?

Set a WatchDog Timer reboot.

Specify an IP that won't respond to pings, set up the WatchDog timer to ping it every 24*60*60 seconds, with a fail count of 7. (or suitable numbers that the GUI will accept).

Reply to
Blake Snyder

On Wed, 15 Nov 2017 07:22:29 -0000 (UTC), Blake Snyder wrote in response to Blake Snyder

You could even have it ping a machine on your network somewhere that you could take off-line when you wanted to reboot everything.

Reply to
ATANARJUAT

Do you know of a reasonably cheap hardware device that can monitor something on the network, and powercycle it when needed? A watchdog that acts on a hung device, say.

I know one or two, but they are expensive. A timer reboot is too aggressive when a reboot is not needed.

Reply to
Carlos E.R.

Use a Raspberry

Reply to
Peter Köhlmann

He who is Peter K+APY-hlmann said on Wed, 15 Nov 2017 14:26:20 +-0100:

The idea of using a Raspberry Pi would work!

It's actually probably perfect in that it's highly programmable, and it has all the necessary WiFi, Ethernet, and USB connections!

I did try out the WatchDog feature of all Ubiquiti radios. Here's my Rocket M5, for example, which has 28 dBm of transmit power into a

30 dBm dish which means I can pickup or throw my Internet for ten miles.

The Ping Watchdog facility was unused when I looked just now:

I turned the Ping Watchdog on, and set it to 86,400 seconds (1 day), and then set the ping failure number to 7 days.

The question is what IP address to ping that doesn't exist on the net? It wouldn't take 192.168.1.256

So I picked an IP address that doesn't exist on my net of 192.168.1.254

This only works if you have the login to the radio, which most people on WISP do not have access to though. So the WISP admin has to set it up.

Reply to
harry newton

It's spelled kludge. In the not so distant past, I helped maintain a series of mountain top weather stations. Service calls were expensive and best avoided. As an added bonus, this was in an environment full of RF pollution.

Sure, if the watchdog timer is independent of what it's monitoring. Long ago Kantronics KPC-2 TNC (packet radio modem) had a built in watchdog timer. Too bad it was all software and located in the same chip it was monitoring. When the KPC-2 hung, the watchdog timer also hung. In later models, they simply removed the watchdog timer.

Roll forward a few years, and I'm maintaining servers in a big server farm. Remote reboot via ethernet was problematic. It was quite common to arrive at the ISP and find a message declaring that the OS refuses to reboot until some obscure process agrees to die gracefully. The customer got tired of paying me to reboot his servers, and I got tried of driving 50 miles to flip a switch. So, I install a paging receiver and decoder to initiate a reboot. That was quite a challenge as server farms are full of RF interference.

However, even that didn't quite work. It seems that most servers have a "feature" called WOL (Wake on LAN) that allows me to remotely power on the server. In order to do that, it needs to have the power left on to the LAN card(s) even when the server is turned off. (Note: WOL is mostly used for desktops, but at the time was also appearing in servers). Sometimes, the ethernet card would hang. If I reboot the machine, the LAN card would remain hung. If I flipped the power on/off switch on the server, it still would remain hung. Of course, with no connectivity, I couldn't do a remote reboot in software. Compaq later introduced a server management card that provided a secondary management channel, but it was too expensive. The only good solution was to pull the plug on the server.

For server farms, I eventually went to SNMP managed remote power switches. I still have a bunch of APC AP9211 switches in service. Primary control is via ethernet, but some had a secondary control channel via the serial port.

I've tried other schemes and solutions. Some worked, but all had a surprise hidden somewhere.

Won't work and may I humbly suggest that you think about this a bit more. The problem is that in any ISO layer cake device, it is possible to have the lower layers working, while the upper layers are hung or stuck in a spin look making the lower layers too busy to respond. I've seen machines that respond quite nicely to ICMP pings where the main function (email or web server) is totally hung. For these things, you need to test the higher level services and not rely on the lower levels.

In this situation, there's one big advantage that MIGHT make such a simple ping work with a wireless link. All wireless connectivity is done at layer 2 (MAC layer). The IP layer is only involved in managing the device. If one pings by IP address, it's fairly good assumption that the underlying layers are working. Some services such as SNMP, SMTP for email fault notification, and the usual internal web server might be hung, but with layer 2 still working, the wireless bridge would likely still be doing its job.

Now, back to the original problem. Heartbeat timers and timed reboots are a kludge. They're needed because the manufacturer of the wireless bridge radios didn't do a decent job of keeping his hardware up and running 24x7. The failure might be in software, susceptibility to power glitches, susceptibility to DoS attacks, crappy components, or environmental issues (overheating). If the wireless is bridge is so unreliable that it has to be rebooted once per week, I suggest you look into the cause of this unreliability, and not apply a band-aid.

Reply to
Jeff Liebermann

One of these daze, I should read the Ubiquiti instructions. I didn't know it had a built in watchdog reboot feature. Same warning as with the Kantronics TNC. If the watchdog feature lives in the same hardware that it's trying to monitor, then chances are good that if one hangs, so will the other.

I've also had the reboot timer in DD-WRT screw up on me a few times, but that was long ago.

Bah Humbug (T'is almost the season).

Reply to
Jeff Liebermann

Yes, that's one of the methods I'm considering for when I have the time.

Reply to
Carlos E.R.

I used transitor driven relay wired to the RTS line of a serial port. open the serial device and the relay switches on, I used the normally- closed contacts to interrupt the DC supply to the flakey device (said DC supply also powered the relay)

A cron job would check the status of the internet and do "sleep 10 < /dev/ttyS0" when a hard reboot was needed.

Reply to
Jasen Betts

He who is Jeff Liebermann said on Thu, 16 Nov 2017 16:03:27 -0800:

For what I need, the watchdog feature "should" work fine, since if it's actually hung, I can always manually pull the POE for a minute (or put it on a weekly or daily timer).

PS: The rains are coming.

Reply to
harry newton

I've got a server on another continent that I need to do an operating-system reinstall on. it's a 3 year old supermicro with the IPMI having a java-webstart based remote console. at some level it uses RDP, but seems to only work via the java web app.

the IPMI allows viewing the console, turing power on and off, hard reset, turnging the identity LED on and off, mounting network files as boot media. etc.

I'm going to try a PXE boot of Debian install medis from a server in the same rack. (the virtual medis boot is poorly documented)

All this remote admin stuff is pretty insecure and, so runs on LAN addresses, not puclic IP addresses.

The moral of this story, spend extra for server grade SSDs, don't use laptop drives.

Reply to
Jasen Betts

True, but I don't have any computer on that room.

Have a look at these:

formatting link
formatting link

They are the exact thing, but expensive for my needs.

There are domotic switches, but those I find are controlled remotely via a phone app - however, as the device that hangs in my house is precisely the router, there is no Internet and those switches would not work remotely. I need something that acts on its own.

I could use an arduino, but I'd have to learn how - which might be interesting, anyhow.

The best and cheaper seems to be switches than have the chip "ESP8266 sonoff" and can be reflashed. But again, I need to learn the procedure and the programming.

Reply to
Carlos E.R.

Or query the SNMP "time since restart" and force a rest when the monitored device has been restarted?

On a cisco router (and lots of other infrastructure tin) you can set "reboot after xxx minutes" - very useful when reconfiguring remotely something that may affect the in band link the control traffic arrives on - as long as you remember to cancel it when you are done......

however a timer with a real power cycle is going to catch "stuff" that a reset may not - some hardware lock ups have been known to need a physical power cycle to recover.....

Reply to
Stephen

Yeah right. A few days ago, it was suppose to rain Sunday and Monday. Yesterday, the prediction was Monday only. Today, it's no rain. See the "NOAA Weather Prediction Center" Mouse over the "Day 1, Day 2, Day 3" just above the weather map. Notice how the southern boundary of the predicted rainfall stops just south of San Francisco Bay.

Must be a conspiracy. I want my rain.

Reply to
Jeff Liebermann

southern half of the lower 48.

Reply to
Neill Massello

Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.