Mac Pro network drops

As I already mentioned, the Mac Pro 2008 (10.10.5) I’m running the Retrospect server on, dropped its network connection for the first time in nine years (that I’m aware of) after installing Retrospect. Since then, it’s happened more than ten times, three times just during the last 24 hours.

It certainly seems as if Retrospect has something to do with it, but on the other hand, it puts a lot of load on the network that exceeds what I normally would expose it to, so it doesn’t have to be something wrong with Retrospect itself. Also, you’ll find a lot of reports on the net about Macs of all kinds losing network connections like this, without any mention of Retrospect. So I don’t know. The worst part is that it’s not at all certain that replacing the Mac Pro with a new machine would solve it. Or updating beyond OSX 10.10 either (which I can’t do anyway; 10.10 is the last supported version of OSX on the 2008 Mac Pro).

(Update 2017-09-18: it clearly has nothing to do with Retrospect directly. I turned off the Retrospect engine and the “Instant Scan”, started copying a massive amount of data from the Mac Pro to the Synology using Finder, and I got network drops every ten minutes or so. Heavy network traffic is enough to trigger it. Switching to the other interface didn’t help either.)

I did, however, pick up a script someone has written which toggles the network to fix the problem when it occurs. I took that script and simplified it greatly. I’ve lost the link to the original. Sorry about that.

The original script was meant to test connectivity by pinging every 90 seconds, then took about 10-20 seconds to verify that the network was down before toggling the interface. This meant up to two minutes of downtime; long enough to interrupt backups or long-running network processes like copy or move of entire directories. I slimmed down the script and made it react and restore the network connection in less than 10 seconds, which seems short enough to preserve open network connections.

#!/bin/bash
# check for lack of ping, restarts the en0 if necessary

while [ 1 ] 
do
  IS=`/sbin/ping -c 5 10.55.66.1 | grep -c "64 bytes"`
  if (test "$IS" -gt "2") then
    echo "Your internet connection appears to be working. Code" $IS
  else
    echo "There is a problem with your internet connection. Attempting to fix by ethernet restart... "
    ifconfig en0 down
    sleep 5
    ifconfig en0 up
    sleep 5
    sendemail -f scripts@example.com -t martin@example.com -u "Internet Connection Down on $(date '+%Y-%m-%d @ %H:%M:%S')" -m "Connection restarted" -s smtp.example.com:587 -xu scripts@example.com -xp <password>
  fi
done

The network interface should be the one you’re using, of course. On the Mac Pro, it’s en0 or en1.

Note that the IP you ping in the script should be on your local net, preferably the upstream gateway. If you reach out to the Internet, there’s a risk you’ll trigger the toggle far too easily.

To send out an email you need to get the sendemail utility. Easiest is through homebrew:

brew update
brew install sendemail

You start this script through launchctl, but I’m taking the easy way out by using Lingon to set it up to run “always” as root, which means that if it for some reason exits, it will automatically restart. It must be run as root to be able to toggle the interface.

When run that way, the two “echo” statements do nothing you can see, of course, but they’re useful when debugging through the command line.

Since running this script, I’ve had a number of network outages, but none that interrupted ongoing backups or other network processes. It’s quick enough to fix the network on time.