Copying of iPhoto library going very slowly

After reading samking71's post I did some more research and found the following information on Experts Exchange (Thanks serial band!):

"You should use rsync instead of cp for copies of large folders. rsync also has options to show the progress. It will also synchronize the files. In case you interrupt the copy, you won't have to start over. If your cp has already completed, your future copies will go much faster with rsync. You could run rsync anytime.

rsync -avz /Users/missy/Pictures/iPhoto\ Library /Volumes/Missy/iPhoto\ Library

That -v option means verbose and you'll see each file and folder.

rsync -azv --progress /Users/missy/Pictures/iPhoto\ Library /Volumes/Missy/iPhoto\ Library

The additional --progress option will show you the percent completed, which is only really useful if you have large files.

You can also redirect the output to a log file.

rsync -avz /Users/missy/Pictures/iPhoto\ Library /Volumes/Missy/iPhoto\ Library > rsync.log

Or output to screen and a log file simultaneously.

rsync -avz /Users/missy/Pictures/iPhoto\ Library /Volumes/Missy/iPhoto\ Library | tee rsync.log

The initial copy might be faster with cpio or cp -a, but if it gets interrupted, you don't really want to start over from the beginning, so rsync is especially better once you have a copy, and it's better overall. "

With rsync -azv --progress i was able to backup 300GB of content from the new Photos Library to an external drive overnight. The issue i had with cp was i couldn't see that it was working. With rsync -azv --progress i was able to see files being transferred. I then used "Get Info" to see the progress based on file size.

That sounds like there's a problem on your system somewhere or the external hard drive. When you are doing a copy of the iPhoto library, open your the application called Activity Monitor which is found in your Applications/Utilities folder. Look at the disk activity tab. Also, run the console application which is also found in yourApplications/Utilites folder. Both of these programs should help you figure out what is going on. If you are close to an Apple Store, you might want to check in at the Genius Bar and see if they can help you debug the problem.

Yesterday I purchased a brand new 1TB firewire 800 external drive. It's correctly formatted for my Mac.

What format is that?

Forgot about this, did you reformat with the Disk Utility after you bought it? It could be stuck in FAT32 format which is really bad. Always reformat an external drive when you buy it, because they always come pre-formatted with the FAT32 file system.

The problem is that the iPhoto library is actually a "package" with thousands and thousands of files. Apparently this overwhelms the Time Capsule file server.

The solution for me was to create a disk image sparse bundle and copy to that. This apparently circumvents the above problem since the files are first assembled on the computer then copied in "bands" to the disk image. It went very quiclky when I did this.

This was not my idea - learned it here: http://help.bombich.com/discussions/problems/1377-backup-of-iphoto-library-to-no n-apple-nas-extremely-slow

If you only need to back up your photos and not all of the iPhoto Library package, you may just select show package and open the folder system inside the library. Then, you only select the originals... or the modified file Folder (there is a link to the folder somewhere and therefor it's easy to find it) and just copy that to your external hard disk or Time Capsule... It will be copied there fast like a normal file ;-) ...In my system i needed a day to copy my library if dragged all of it to the external disk... while doing as i described.. it only needs 46 minutes! 😉

The photos there are organized by year and by date so it is pretty cool to browse them all.

Well no you don't if you have any value in your Library. Accessing the Library via the Finder or any other app is dangerous and can lead to corruption and dataloss. If you are going to advise people to do this you should at least warn them of the possible outcomes.

Further, if you copy over the original then all you get are the photos as downloaded from your camera. No added metadata, no edits, only the the files in folders named for the date and time they were imported.

There is no longer an "modified" folder. There is a Preview folder. That contains lower quality versions of your edited photos. These too are missing a lot of the metadata - they don't even have all the metadata the originals have.

So there's two instances of dataloss already, not to mind the further risk if you accidentally disrupt the Library.

The Supported way to access the files is to Export them

File -> Export

Here you can export all the Original files to a folder scheme of your choice and naming style.

You can export your edited photos to a variety of formats, including all the metadata from the original and any you've added since - places, keywords, captions. You can select the size and quality of the output shot as well. Again, you can export to a folder scheme of your choice and naming style.

And none of this involves any dataloss or danger to your Library.

This User Tip

https://discussions.apple.com/docs/DOC-4921

has details of the options in the Export dialogue.

That's why i first wrote to do so, only " If you only need to back up your photos and not all of the iPhoto Library package" ... Just copying the raw photo material was what i just needed recently so i wanted to share that... Thank you for your interesting, detailed post! 🙂

I found a fast soution 45 Gb Library copied in ~30 min. over USB 2.0

I did a semi-manual copy, by opening the iPhoto Library - "Show package contents" (in right click menu) then copying the contents to a new iPhoto libaray "Package"

Instead of copying all at once copy the small files/folders in one batch, wati to complete, then copy the larger files folders. You can copy the package contents one folder at a time or a few at a time. I found that copying the Library files in one batch, then copying the Masters, Previews, & Thumbnails in a second batch seemed to work well without choking up.

Here's the steps

1. Connect your hard drive or Mac in Target disk mode
2. navigate to the folder containing the iPhoto Library you wish to copy
3. 'Right click'

   Show Package Contents

4. You'll see apx 19 files - select all but exclude 3 folders: Masters, Previews, & Thumbnails you'll come back for theses later
5. Copy these files to an empty folder on your destination drive/Mac (Later you'll use iPhoto to make a new libary "Package" then you'll put these files inside it) The copy should take a few minutes, high volume + tiny files
6. When the first copy is finished, go back and copy the remaining 3 folders Masters, Previews, & Thumbnails. This is the opposite as previous step, lower volume + larger files. Copy speeds should be normal ie 30 min. for 45 Gb over USB 2.0.
7. Once all these files and folders are copied you need to put them into an empty iPhoto Library container. An easy way to do this is
   1. Launch iPhoto holding the Option key
   2. Choose 'New Library' (choose an apprpriate name)
   3. Quit iPhoto
   4. in Finder go to the new Library you just created (User/Pictures/) - be sure its the one you just made
   5. Right Click (Control + Click) the iPhoto Libraray
   6. Choose - Show Package Contents (now you're looking at the contents that makup an iPhoto library)
   7. Select all these files and folders and delete them to the trash

      Open the folder that has your freshly copied files and put them in this empty iPhoto Library package.

   8. Close finder windows (disconnect your hard drive or Mac)
   9. Launch iPhoto (you can hold the Option Key while launching iPhoto to choose the correct Library if you have more than one).
   10. Viola. You should have a functional copy from your old Mac/External hard drive.

REMEMBER!

The easiest way to mess up your iPhoto Library is to move things around inside the "Package contents" of your iPhoto Library. Its hidden in there for a reason, so don't go mucking about under the hood. Best practice is to touch nothing inside the "Show Package Contetns" of your iPhoto library. As described above, we're merely just speeding up a file transfer and copying files so they mirror their source in a new destination. Good luck.

Ok - but extremely complicated and very risky and hard to see any real advantage over just copying the iPhoto library intact and letting it finish - it it takes a few minutes longer but so what - it is much safer that mucking around with the components of the library as you are suggesting)

Your ten step process with many manual interventions vs a single drag and drop and off to the movies while it copies

If you are going to do this mucking around inside the iPhoto library approach to potentially save a few minutes of copy time, be really sure that you have a good, current backup first

LN

I stand behind this process is valuable if you're having trouble with copy errors and 1,2,3 day estimated copy time.

its not always a perfect world Larry, if you remember you threw in your worthless 2¢ yesterday on this thread https://discussions.apple.com/thread/4983653?tstart=0 1. I My time is valuable. 2. After 2 days of trying, this library wouldn't copy, it had error and wouldn't copy from drag and drop. I tried iPhoto First Aid, I rebuilt it with iPhoto Library Manager, tried cp/ditto in Terminal, I pulled the hard drive, Tried CCC etc etc. This 7 step process finally worked for me so I'm contributing it to the community. I don't know what you're doing because telling people they're wrong just to feel right is a waste of keystrokes IMHO.

Alternative method:

Open up terminal (under application: utility)

enter the following:

cp -r [drag the iphoto source from finder] [drag the destination folder from finder]

Or, for folks who aren't comfortable in the Terminal, use any of the dozens of apps that put a GUI on rsync.

maybe just leave it to get on with it?

Ok I just had this problem - I was trying to copy a 48Gb iphoto library as part of a safety back up for someone's computer before trying to solve the other problems with it. It started very slowly and before long was predicting it would take "about a day".... but while researching other solutions and discussing progress things speeded up drastically, and eventually it finished the process in less than a hour without my doing anything. My impression is that the first 300mb or so were very very slow to copy - but the rest went fine - but I was not watching closely enough to see when exactly the change occurred.