Since January, I’ve been spending most of my time away from home. This means that while I’ve got all the hard drive space I need at home, I’ve been getting a bit creative to keep my images near to me while on the road. If I were just away for a week at a time, it’d be easy, but when I spend 4 or 6 weeks away from home, I need to be able to dig back through my archives and fetch out images when a client comes calling for it. This seems to happen every few weeks at this point, and I’d be loosing out on licensing opportunities if I didn’t keep my library accessible. To accomplish this, I’ve been using a portable 750GB portable hard drive. For a while, this worked out great and I could easily take all of my images everywhere I went.
During the last few months, however, space has been getting very tight. I’ve been creeping up on the capacity of my drives and have been thinking about what to do next. Once you get above 700GB of image data, this turns into a tough problem when you want to stay mobile and not cart around huge heavy RAID arrays. Of course, the real solution is to leverage the network. I’ve been investigating various options such as PhotoShelter, Amazon S3, Strongspace, and several other places to park a copy of my entire archive in a place where I can get to it whenever I want to. However, I haven’t yet finished up this evaluation process.
Unfortunately, time caught up with me and I’ve now run out of space. So, while I finish up the processing from last week’s WWDC shoot and this weeks TOC shoot, I’m juggling data around so that I can get everything done and still keep backups of all my data. This means I’ve been moving fragments of my library around and that I’ve needed to let Lightroom know where my folders have been moving. And I’ve needed to do this more than a few times.
There are a lot of ways to do this, but I’ve found the hardest—at least for me—is to use the question mark icon that appears next to a missing file in grid view. The easiest is to simply right-click the folder in the Library view and use the “Locate Missing Folder” menu item. Here’s a screenshot:
This makes the process of relocating your image files easy. Too bad it took me more than a few tries at it before I discovered this little gem in the contextual menu. Now that I’ve found it, it’ll be getting a bunch of use in the next few weeks as I migrate to my next Library setup. After, of course, figuring out what solution I’m going to buy into. But that’s next week’s job. First, there’s a few thousand pictures to sort through, rank, develop, export, and deliver to my clients.
Steve: That is indeed one of the possible solutions, and it may be a core component of where I go next. But it won't be the full solution. There's the details of offsite backup to factor in as well. I'll probably start working on this a bit more in a few days once I catch up from the photoshoots of the last few weeks.
Maybe I'm missing something, but how about this as a solution. You have a normal Lightroom structure on an internal hard drive, with a Files directory alongside the Lightroom database.
When your internal drive is getting full, you move all of the raw image files from Files onto an external drive and mount the drive back into the filesystem in Files. Lightroom carries on as before.
When your external hard drive is full, you unplug it, plug in another, mount it under Files and carry on as before.
Sure, Lightroom can only see a few hundred Gb of originals at a time, but it works.
Kim: Yah. I've been working really hard this year. It's been zip zip zoom from one conference to the next. When I get a handle on my backup solution, I'll either post it here or do a big write up on my own blog, blog.duncandavidson.com, and link to it from here.
The drobo is indeed cool, but its USB support turned me off at first. Something like that should have a FW800 port on it. :) But the way it works with multiple drives is extremely compelling. One thing I've considered with the drobo is to hook it up to one of the new Airport Extreme base stations and use it as both a backup of photos as well as a target for Time Machine backups with Leopard ships. Even so, I like the idea of an offsite backup as well, so.
Mariano: Yes, the initial upload part is a big hurdle. I don't mind updates of several GB to take a few hours as those can run while I sleep. But to do that best, in my mind, requires rsync access to the remote filesystem. The tools that require a web browser or their own upload client are more problematic for my use I think. Of course, I'll be digging into that a bit more as I go. But, the old school sysadmin in me really wants rsync access.
As far as the initial upload, yah. That's the doozy. Whoever I use, I'll either make sure I can send disks or get access to fast bandwidth at a number of places I have access to. Barring anything else, I'll drop by a data center I still have access to that has big pipes to the net. :)
Hi Duncan,
nice.
I also looked into online backups and have to say that it doesn't work for me. I am using ADSL and the "A"-part really makes it impossible to upload gigabytes of images. And looking at your amount of data it would take weeks just to upload the data initially and then hours for every shot.
I believe to remember though that you can send disk drives to photoshelter or photobucket and they will do the initial upload for you.
Duncan, Duncan, Duncan. You must be working too hard homie. ;) I found that feature fairly early on while nailing down my setup.
I'm super-curious which solution you pick for a library setup because I'll be looking for a solution later this year. Something I can access from any computer (which is tough b/c LR currently won't let you store your library on a networked drive). My wife and I will have two or three computer to access photos from. I thought I struck gold when Leo Laporte Patrick Norton & Dvorak were talking about the Drobo. But, it's not a NAS device, just DAS and USB2 only. So, back to the drawing board.
Even if the folder is still in place, you can activate Locate Missing Folders by Ctrl/Opt (PC/Mac) clicking the folder.
One idea is to move one file in Finder to the new location, fix that in Lightroom. Now with the new location in the Folders panel, drag all the other subfolders in Lightroom.
The behavior where you had to locate the images in a subfolder one-by-one is exactly what I ran into when I tried to do it on an image by image basis. However, using the contextual menu shown above does the trick for me every time and avoids the one-by-one gruesome process.
Unfortunately, I've found, if you move a subfolder and try to locate that, Lightroom forces you to identify the location of the images in the subfolder one by one. Has anyone else noticed this?