If you’ve got one or more Macs backing up to Time Machine via WiFi, how many times have you seen this?
Backing up is the essential activity to protect the integrity of your data, so how did Apple’s famed backup technology become so unreliable?
Allow me to regale you with a short history lesson, and it involves a big cat, an apocryphal tale of Steve Jobs and the stagnation of what once seemed like a clever idea.
The Birth of Time Machine
When Snow Leopard was announced and Apple’s Scott Forstall revealed that finally OS X would be getting backups, people theorised as to how this new feature would work. Some initially thought that Apple would implement a smart file system such as ZFS, which would snapshot instances of the disk state at the file system level.
Sadly, the reality was less exciting. Apple opting not to adopt a more modern file system stuck with their notoriously unreliable HFS+, and (in my opinion) slowly edged open the doors to backup hell.
The story goes that Apple were close to adopting ZFS at one point, until Steve Jobs’s ego was infringed upon. So in the end Apple built on the shaky foundation of HFS+ to implement an intelligent yet crude system of hard links to support backup snapshots.
Here’s how it works. When you make your first Time Machine backup, essentially it’s just a direct copy of all your files from your Mac to the backup volume.
The clever part happens when you make subsequent backups. Time machine makes a copy of the previous backup consisting entirely of those hard links I mentioned. It then copies over just the files which are new or changed, overwriting the relevant hard links, and removing the hard links for deleted files. This is why later backups take a fraction of the time of the initial backup, but you still get a complete snapshot of your file system.
Now, when you think about it this made sense back in the middle-life of OS X. Most people had an external hard drive attached to their Mac via USB, which they used for their Time Machine backup. You could always rely on the connection to the drive being uninterrupted, unless there was a power or hardware failure.
Jump forward to today, and a lot of people backup over their WiFi networks, and here in lies the problem: Time Machine was intended to work by mounting an external disk, backing up and then cleanly un-mounting the volume; A process that requires a 100% reliably uninterruptible connection.
But when you consider flakiness of a typical WiFi connection, and combine that with the protocol used to mount the drive (apparently always AFP for Time Machine, even though SMB2 is the current standard in the OS), you have a recipe for data corruption. This can be due to many factors, but often caused by the WiFi connection dropping during a file transfer.
As a result, when Time Machine completes the backup and then later verifies it, it finds corruption and deems to discard the entire backup, snapshots an’ all. The only way forward is to begin afresh with a brand new backup.
When starting a fresh backup means transferring upwards of 500Gb from scratch over WiFi, you’re looking at up to 24 hours of backing up time, with no guarantee of data integrity at the end.
More so, the longer the backup takes, the greater the risk of connectivity issues, and consequently an increased risk of corruption.
And to rub even more salt in the wound, you lose all your past snapshots each time this happens.
Pretty useless in my not-so-humble opinion.
Mitigation
The best way to mitigate the risk is to ensure your Mac is connected to your router via Ethernet rather than WiFi. Of course, this means added wires and the assumption the your Mac is close to your router.
Or, you can just go back to old reliable USB, which is a problem if you want to backup multiple Macs to one drive, or if you like to move your laptop around your house.
A Better Fix?
This year at WWDC, Apple announced a new file system called APFS. It won’t make it into the current macOS release (10.12 Sierra) except for developers to play around with, but it does promise to revamp how Time Machine works by providing proper snapshotting. Ars Technica’s yearly macOS review contains a fantastic read covering the more technical aspects of this file system, and it’s really something to get excited about.
Suffice to say, Apple has let the ball drop on Time Machine and their badly ageing HFS+ file system for too long, but the future is looking a lot brighter.
Until then, Apple users will keep having to play the game of digital Russian roulette with their backups.
Daniel's Blog 