About a week ago, my Apple G5 tower suffered a fatal crash. Initially it just froze up - upon restarting, it froze again in several minutes. Sadly, earlier that day, the hard drive I backup all of my business files to also crashed so at that point I had to get the information off my G5 tower as quickly as possible. Luckily I was able to get the information off and all of my important data before the G5 tower wouldn’t start up at all.
I find it quite amazing how unreliable hard drives are - and I seem to have some of the worst luck of any photographer I know when it comes to faulty hard drives. As a result I back everything up on a minimum of three devices. For my images they are always on at least two hard drives and burned to DVD for offsite storage. My computer is also backed up completely to an external hard drive each night. And every few days I back it up to a third hard drive just in case. I have had four hard drives crash on me this year alone.
I have used several methods of backing up my main computer - LaCie’s Silverkeeper, Superduper and others. The point is to find something that works for your system and be vigilant. I covered backing up Lightroom catalogs in a previous post. You can check that out here.
Because I back up my entire computer’s hard drive I have not been using the Lightroom back up solution. My Lightroom catalogs are already backed up each and every night when I run SilverKeeper. And since I have many catalogs - sometimes one catalog for a single set of images it would be a pain to have to go through and let Lightroom back up all of the catalogs individually.
Whatever your systems, I would say backup your images and important information to a minimum of three locations. And whatever your backup solution, whether you use Lightroom’s very good backup option or a different method make sure it is robust and can withstand a fairly severe catastrophe. If your office or home burns down do you have your images and important information somewhere else?
This week’s blog post is just a cautionary tale for those of you out there that have never had a hard drive fail. It happens much more often than hard drive makers like to talk about. Luckily, with my many backups, I was up and running within a few hours on my laptop and will have my tower either repaired or replaced this week.
That’s it for this session. See you next week.
Adios, Michael Clark
I agree...it is nuts to have to restore/rebuild an OS to restore data. The idea of a backup is to provide a quick (relative term) way to recover from a failure.
The idea of just backing up data and having to reinstall a system and applications, let alone settings, cookies, passwords... it's insane.
----
Rand
Here's the product page for the Smart-UPS series:
http://www.apc.com/products/family/index.cfm?id=165
(Note the "sine wave output" in the specifications list).
@michael: That's a bit like asking a builder how much a house costs, it depends. Obviously at the end of day, from a business perspective the numbers have to add up, and give the risk/benefit ratio. Though you won't go wrong with APC's Smart-UPS series (the full versions, not the cheaper SC versions).
Another interesting option for off-site backup is Amazon's S3 service. It's geographically distributed, and the pricing is excellent. For the Mac you can use Panic's Transmit FTP client or there's also a product called Jungle Disk that mounts the S3 storage space as a disk in the finder. For windows and unix there's a number of different products, including a firefox addon.
I could not agree with you more....having diversity in the back up locations (at least three) will save you every day. With disk prices being so cheap it is easy/cheap to set up technology to serve most of our needs.
Matt -
Do you have any recommendations as to what is the best UPS set up commercially available?
It's also worth noting that the filtering in most ups's is no where near as good as a dedicated unit. Most of the cost of a ups is in the batteries. This is especially true for the cheaper end of the market. Though Michael is right on about cooling, that'll kill a good drive faster than anything else.
Great discussion and feedback from everyone here - thank you! I do have my computers and hard drives connected to UPS systems as New Mexico in general has shady electrical reliability.
My hard drives do get worked a lot as I am constantly accessing images for clients. Interestingly enough my internal Seagate drives have been bomber while the external drives have really been the ones to fail on me. My advice is when looking for a drive - if it doesn't have a fan or cooling system - avoid that model. For internal drives they are cooled by the computer so that works well.
Great topic Michael and some interesting comments (especially from Matt). I spend most of my time in South East Asia and have always assumed that my electrical source is bad, so I have always used UPS and Surge Protectors and very rarely experience any drive failures (none in over ten years).
I have one friend who is always reformatting and optimizing his hard disk drives and experimenting with file back-up software, he has a lot of drive failures.
I perform all my file uploads, downloads and back-ups manually, it may not be the reason I have so few problems but it makes me feel better.
A recommendation I follow is to connect external drives and computer to an uninterruptible power supply (UPS) with voltage regulation, to shield computer and drives from power surges. In my experience such surges can set the stage for subsequent disk failures.
As someone that's worked in the IT industry for over 10 years, I'm surprised everyone here seems to take data corruption and loss as a given, and that there's no mention of the causes of hardware failure and data corruption and what can be done to prevent it.
I owned many computer systems both manufacturer built and built myself and too many hard drives to count, and have never had a system or hard disk crash. However, I've never connected any of my equipment to a wall outlet. Bad power is the number one cause of data corruption and incremental hardware failure, specifically brown outs for the former. Computers work with 1's and 0's that are internally represented as the difference between two voltages, as computer systems have become faster the voltages inside memory subsystems and CPU cores have got lower and the gap smaller.
With unfiltered dirty utility power it's all too easy for line noise (someone next door turning a high current device; dryer, vacuum cleaner, industrial machinery) to cause a small dip that turns a 1 into a 0, and then that error most likely gets written out to disk unnoticed. These small dips and surges (which aren't stopped by a surge protector, since they only handle big ones) slowly decrease the life span of the equipment, and make hardware failure more likely.
Considering how cheap power conditioners are these days, it's surprising how few people invest in one. My original APC Line-R 600 cost me $250 about 8 years ago, yet they're as little $50 USD now at amazon.com. These units aren't a UPS, as they don't have any batteries, but instead contain large capacitors that are used a generate a clean feed via induction. In doing so your equipment receives clean power at exactly the correct voltage.
http://www.amazon.com/APC-LE600-Automatic-Voltage-Regulator/dp/B00009RA5Z
UTILITY POWER IS FINE FOR YOUR TOASTER, BUT I'D NEVER ATTACH PRECISION EQUIPMENT DIRECTLY TO IT.
Also you should protect all points of entry, this includes ethernet and modem jacks. This is especially important for laptop's since these ports are usually soldered directly to the machine's motherboard. I use one of these (below) with my laptop when travelling, and connect it my my Line-R in the office, since it does both basic surge protection and protects either your ethernet connection or modem line. They're also 100-240V international voltage/phone system capable, so great if you go overseas a lot.
The laptops with 3 pin plugs (toshiba, etc):
http://www.amazon.com/APC-PNOTEPROC6-100-240V-Protector-Notebook/dp/B0002RSPFS
The laptops with 2 pin plugs [no earth pin] (sony, etc):
http://www.amazon.com/APC-SurgeArrest-Notebook-Pro-suppressor/dp/B00066ISFM
The number two cause of the data corruption is bad memory. Always buy high quality ram, it may cost more but it's worth it. Dodgy memory is almost impossible to diagnose in day to day use, and the faults are often intermittent. If you suspect a ram problem, test and replace it. If you're running a PC or intel Mac, download memory test ISO from the site below and burn it to cd. I'm sure there's something out there for PPC users as well, but i don't use that platform myself.
http://www.memtest.org/
Third is heat, which is more of a problem in some parts of world than others. But always try to keep your equipment as cool as possible. Also it goes without saying to never drop your external drives either, they don't like that ;-)
Having said all this, I always have at least two copies of everything, usually three with one stored off-site. As I've seen too many clients reduced to the tears after losing 2,3 or more years of data due to thief, fire, flood or lightning strike, where both the original and their backups were either destroyed or stolen. oh and... TEST YOUR BACKUPS!! It's very easy to forget an important folder or find that your while it appeared to be working, it wasn't. So always try a test restore from time to time (to a temporary folder obviously ;-)
Hope this was helpful,
Matt.
Davide: I agree that incremental backup to a remote host is a good idea. (I'm also an amateur, but I have 10x your amount of image data at the moment. Fortunately, DreamHost has been generous ;-).
BTW, if you only perform incremental backups (the default with SuperDuper! I think), you may not notice bit rot: Images that haven't actually been fully accessed for a while may be corrupt because of hardware decay. It's unusual for the same files to suffer from bit rot across all backup sets, but to be sure I think it's a good idea to periodically (e.g., monthly) perform a complete bit-by-bit copy (from each disk).
I agree, backup the entire drive, every day and you're covered. I use SuperDuper! every day, day 1 on one drive, day 2 on another drive, day three overwrite drive 1, etc. I run it no matter what I've done during the day and no matter where I am (I bring a portable drive with me when I travel). It's not just a matter of backing up images or data, it's a matter of having a bootable hard disk that you can connect to a new computer in 1 minute, start up from it and everything is as it was yesterday: user settings, email, files, the works.
The idea of just backing up data and having to reinstall a system and applications, let alone settings, cookies, passwords... it's insane.
I've put off buying and upgrading to Leopard for two reasons:
1. I'm waiting for the Lightroom update
2. TimeMachine isn't compelling to me as I already have a workable backup system.
Also purchase Hard drives from different stock/manufacturers, to ensure some diversity. If they all have the same MTBF (mean time between failure) then they are more likely to all fail at the same time.
I just finished to backup all my images on www.mozy.com. I have only 30 GB because I'm an amateur. But it feels very good to know my data now it also in another continent!
It takes a while to upload that with a 50KB/s bandwith but from now it will faster because it's an incremental backup.
Have you considered RAID or something like Drobo? The software RAID that comes with MacOS X is plenty good (I also use it on a G5), and has the advantage that when one of you main drives fail, you can continue working until you've got a convenient time to swap the faulty drive. I recently added a Drobo to my setup: It's even easier, but the throughput is more limited.
I additionally use SuperDuper! to keep periodic backups, but I see those primarily as dealing with software glitches that corrupt files, or with my own clumsiness (when I command-line-erase a subtree that I shouldn't). If Leopard's Time machine improves to work on networked partitions, I may switch to that (or add it to my data integrity strategy). However, I think those solutions aren't as great for drive failures: Redundancy is preferred for those.
Oh, and I sympathize: For a while it seemed that every FireWire drive I bought was dying within 6 months no matter which machine it got attached to. (The latest few drives seem to fare better.)
Geoff -
I use SilverKeeper which basically just adds new and modified files to an already existing backup to make sure they match. Thus, it isn't like writing full backups everyday. And that is only to one drive.
Are you doing 'full' backups every night or differential? Full backups that regularly will certainly reduce the lifespan of your disks having to read and write every sector every day. I opt for full once a week (sometimes two) with daily differential. I also use a program called Acronis to take a complete image of the whole disk every so often. Acronis also will allow you to re-image onto a disk of a different setup incase you have to rebuild to whole machine.
G
Always good to back-up, but with 4 HD crashes in a year i would start to look at other components in your system that keeps Frying your harddisks.
Good luck,
Asprine