Are you really backed up?

I saw this article on the SANS diary this evening, and it reminded me once again how important both backing up is AND how important it is to verify your backups. You're not just defending against hackers and hardware failure, you're defending against stupid or malicious employees as well. Here's the gist of SANS article, but you'll have to go to the SANS page to read the entire thing, which has links to supporting stories.

So.. Are all of the bad guys really on the outside?
Published: 2008-01-25,
Last Updated: 2008-01-26 00:29:06 UTC
by Deborah Hale (Version: 1)

Today Fox News reported that a 41 year Jacksonville, FL women who thinking that she was about to be replaced, deleted 7 years of drawings, blueprints and other files from her employers server. Estimated damage $2.5 million. The company owner said that they managed to recover the files by using an expensive data recovery service.

...

As I read both of these stories I have to ask myself, "Where were the backups?" In the first story, they had 7 years worth of data, as they indicate $2.5 million worth, why on earth would they not have a backup? What if it had been a drive failure that took out the drive instead of an employee purposely taking it out? What would they have done then? And in the case of the cable company, even if you don´t routinely backup the mail boxes, when doing maintenance such as this one, best practice is, "BACK IT UP". When doing any type of maintenance my experience says "Murphy´s Law Prevails".

A little while ago I got a phone call from a friend of mine. They own a small business and have one computer in the shop that contains all of their customer records, financials, and receivables. I setup a backup for them and it was set to run every night after the shop closed. She called and said that when she turned the computer on this morning it won´t load Windows, it says "insert system disk". I explained to her that it sounded like the hard drive had failed, that we would have to replace the hard drive and then restore from the last backup. She said "well, that may be a problem". I asked her why and her reply was "well, we needed some space on the computer table to layout some papers, so I unhooked the backup drive and forgot to plug it back in. Guess what, she wants to know if I can help get the data back....

So, all three of these "loss of data" situations were caused by insider errors. One accidental, one on purpose and one.... Well, let´s not go there...

Another situation that popped up in the last couple of weeks was also caused by an insider. Not on purpose but none the less it happened.

...

Therefore, I again say "The bad guys aren´t always on the outside".

Thanks to one of our reader's Dan Jones I am updating the diary with a link to
an incredible piece of wisdom.

taobackup.com/

The TAO of Backup is a good read -- I read it years ago and it's good to be reminded of it.

Just last week I was moving a hard drive from an old computer at home to the new one, intending to hook it up using an external USB-to-IDE cable and the drive dropped to the tile floor. The drive is now useless -- it just clicks when hooked up, and none of the systems I plugged it into recognized it as a drive. I chatted electronically with a data-recovery technician, and they estimated $500 to $2,500 to recover data from the drive, and they didn't guarantee success. Fortunately I had backed up the drive across the network just a few days before so the data loss was tolerable -- just a few days of home e-mail, nothing critical. I was lucky, though -- the system hadn't been backed up for months before I did the backup in anticipation of the move to a new system.

It isn't enough just to back up your systems, you have to have backups off-site where they can't be damaged by the same [person|fire|hacker] who takes out the primary system. Last fall, Francis Ford Coppola lost not only his computers but his backup server, which was stolen at the same time. He lost 15 years of digital photography that he may never get back.

Over the past few days I've been setting up several backup servers, one for the office and one for the house, and I will be linking them across my VPN so they mirror each other. I'm also trying out a remote backup service that uses Amazon's S3 storage. I'll blog about those when I get the system up and running and I'm happy with it.