Aperture Updating Library... 126,637 of 6,959,575

Question

Level 2

189 points

Aperture Updating Library... 126,637 of 6,959,575

Hello all,

I upgraded to Aperture 3.2.3 and got an unexpected "Recovering Library" when it first started. Well, it seemed to have worked, and I did some importing in Aperture. It also started doing facial recognition in the background. I let all the background stuff finish, and then yesterday I quit Aperture.

At that time, it popped up a panel/sheet saying Updating library... 1 of 6,959,575 (commas added by me). It had done Updating Library in the past so I didn't think much of it.

However, now, a day and a half later, it is only up to 126,767 of those nearly 7 million. It seems to be going at a rate of one per second. At this rate, it will take around 80 days to finish. Given that my computer is a MacBook connected to an external DroboPro, that means I'm stuck with a non-portable computer that I have to hope has absolutely no crashes nor power failures nor any other hiccoughs for three months. Clearly, this is unacceptable.

Does anyone have any ideas what's going on here and what I could do, please?

Thanks,

Doug

MacBook Pro (15-inch 2.53 GHz), Mac OS X (10.7.3), Aperture 3.2.3

Posted on Apr 27, 2012 2:56 PM

Reply

Answer 1

Apr 27, 2012 4:52 PM in response to Doug F.

From what version are you upgrading?

How is your Library set-up?

Are any Aperture files -- Library or Masters -- on the Drobo? Problems with Aperture and Drobo's are regularly reported here. Take a look.

Reply

Answer 2

Doug F. Author

Level 2

189 points

Apr 27, 2012 5:50 PM in response to Kirby Krieger

I was upgrading from the previous point version, 3.2.2, I suppose (I always keep Aperture up to date per Software Update, but I don't upgrade until I'm going to use it). It is also my first time using Aperture under Lion 10.7.3, the previous time having been under 10.6.8.

My library is all stored on the Drobo. Aperture manages all the files. I probably have around 75,000-80,000 photos, the vast majority of which are Canon Raw formats in 20D and 5D II, but including some other Canon RAW formats and some JPGs from other manufacturers, iPhone/iPad, etc.

During this process, I'm seeing 100% CPU usage (i.e., full usage of one core of my quad-core, HT CPU) but insignificant disk access (almost none), per iStat Menus.

I will note that OS X Lion does not seem to do core affinity on processes, so that 100% CPU usage is spread over all 4 cores for some reason.

Thanks,

Doug

Reply

Answer 3

e2photo

Level 2

176 points

Apr 28, 2012 7:45 AM in response to Doug F.

I had lots of problems when my library was on the Drobo (Referenced Library to Masters stored on Drobo). Once I moved the library to my internal drive and just pointed to the masters on the Drobo, it has been ok for quite some time. I actually lost a library residing on a Drobo, but fortunately I had a back up.

You say you have 75-80,000 photos but it is updating nearly 7 million? Sounds strange to me. If I had a current backup of the library, I would do a force quit on Aperture, move the library to my internal drive (or a non raid external drive if you must), rebuild the library and relocate the master files to the Drobo, move the library to the internal drive and NEVER put the library again on the Drobo for regular use again. If that does not give you a good library, then I would resort to my back up, but with a similar process in mind.

Good luck

Reply

Answer 4

Doug F. Author

Level 2

189 points

Apr 28, 2012 7:47 AM in response to e2photo

Thanks for the suggestion. I would have absolutely no idea how to relocate just the library to a different place than the photos. In the meantime, it just passed 200,040 of 6,959,575, so it doesn't address the current problem.

I don't have a current backup of the library. I back up only every month or two because it takes so long (it's almost 2T).

Cheers,

Doug

Reply

Answer 5

Apr 28, 2012 10:13 AM in response to Doug F.

As sheriff Beuford T. Justice is so fond of saying, "You in a heap of trouble(s), boy!"

Let's start with the Drobo, Lion, and Aperture.

-- It appears that Lion significantly tightened up on the use of Aperture files on other than directly connected Mac HFS+ formatted drives. If your Drobo is anything other, then you have a problem that you may not be able to fix.

-- Drobo speed - by all accounts, there isn't any. And Aperture hates having the Library on a slow drive. The good news is that, in my experience, the speed of the drives that holds your Masters is not a big factor. If you have been living with a Managed Library on a Drobo, then relocating your Masters to the Drobo and moving the Library to your fastest, emptiest, internal drive should give you a real speed boost. Especially if you have over 4 GB of RAM. This link will give you the info you need: http://photo.rwboyer.com/wp-content/uploads/2010/08/Aperture3Organization.pdf

-- You back up only every month!! On a Mac!! Shame on you! Really, with Time Machine, Vaults, and products like SuperDuper!, you should never be more than one hour or one day down on backups. Period. Head out to your nearest Best Buy, Costco, or whatever and immediately buy yourself two cheap and nasty USB drives, each the same size as your entire system. Better yet, replace the Drobo with a decent FW800 drive and use the Drobo for Time Machine. Speed is not an issue and the RAID function of the Drobo would actually be useful.

Seriously, if you have any photos of any value and you do not have at least two backups, ideally three, and one off site, then you are simply waiting to lose 'em. There are many threads in this Forum discussing back up strategies but in simplest terms, Time Machine is so easy to use that Mac owners have no excuse to ever lose data.

Best wishes!

--

DiploStrat 😉

Reply

Answer 6

Apr 28, 2012 10:26 AM in response to DiploStrat

Well put -- "heap o' trouble" sadly sums it up.

One small point for the OP. If you follow this excellent advice:

If you have been living with a Managed Library on a Drobo, then relocating your Masters to the Drobo and moving the Library to your fastest, emptiest, internal drive should give you a real speed boost.

... move your Library first, then relocate your Masters. In your (OP's) case: copy the Library package to your system drive (or at the least a FW800 locally-mounted drive) using Finder, then in Aperture use "File→Relocate Masters" to move your Masters to another drive (does not have to be speedy, but works better is locally mounted).

Storing your Masters outside of your Library is simple. With Aperture you can move them in or out of the Library at any time. You Library should be on your fastest drive (almost always your system drive), and that drive should have at least 20% free space. Masters, as Diplostrat says, can be on "slow" drives.

At this point I think your only choice is to Force Quit, make a back-up copy of the Library, move the Library to a local drive, and rebuild the Library. If that fails, try to open the Library in an older version of Aperture, and rebuild there.

Reply

Answer 7

Doug F. Author

Level 2

189 points

May 2, 2012 8:52 AM in response to Doug F.

Thanks to all for opinions and ideas. Some responses:

1. My Aperture library is almost 2T. Hence, I am unable to copy it to my internal drive. (I do have a modified MacBook Pro with an SSD and a rotating drive, but either way it wouldn't fit.) So, those solutions won't work for me, but see #4 below for an alternate idea.

2. I do indeed do regular backups. My two internal drives are backed up via a perpetually connected FW800 3T external drive. I also back up my Aperture library regularly (as I said, about monthly). I only work in Aperture once or twice a month so despite the infrequent backups, it's usually up to date. (I use Synchronize Pro X.) In this case, however, the failure occurred just after I had imported a month's worth of videos and photos (from 3 cameras) but before I had finished organizing things, and then kicked off the usual backup. Unfortunately, I did choose the "eject and erase" options on the photos after having imported them. (I don't see why people need to lecture me on backups when that was entirely off-topic.)

3. I eventually gave up and force quit the Aperture that was counting up to 7 million after a few days. I then re-ran it, and it opened just fine. Unfortunately, it later crashed again, and then when I went to open it again, it decided to rebuild the Aperture library. During this time, Aperture decided to crash at 56% rebuilding. I haven't opened Aperture again.

4. I have ordered a Promise R6 12T Thunderbolt RAID which I hope to get next week and set up in RAID 6 (so, around 8T usable). I will copy the Aperture library to that from the iSCSI-connected DroboPro, and then see what happens with Aperture.

5. Is there any way to relocate the library and leave the masters in place? That's the only way I could see being able to put the library on my SSD. Is there a way to see how large the library would be if this is done? Space is perpetually at a premium in my local drives. That's why I had the DroboPro with 20T of drives internal to it in the first place (12T usable with dual-drive redundancy)!

Thanks,

Doug

PS: All in all, I'm extremely disappointed with Lion. My system is much more unstable (Mail regularly stops retrieving mail, Aperture fails, the DroboPro seems more unreliable, Time Machine took several days to start backing up again, etc.) and the user interface paradigms and colors are less user-friendly. If it weren't for the fact that they required you to move to Lion to migrate from MobileMe to iCloud, I never would have done it. Shame on Apple. (Or, alternatively, shame on me for not

Reply

Answer 8

May 2, 2012 11:29 AM in response to Doug F.

If you have not read and undertstood this, you should: http://photo.rwboyer.com/wp-content/uploads/2010/08/Aperture3Organization.pdf

This will tell you how to open the Library Package and determine how large your Masters, Thumbs, and Previews are.

Some other thoughts:

How full is your system drive on your MacBookPro? If it does not have between 25 and 30% free space, then that is the first thing you have to fix. Similarly, I assume that you have run Disk First Aid or Diskwarrior against your system drive, just be be sure that it is not, for any reason, hosed. From your comments about Lion, I would guess that:

-- Your OSX installation has real problems; problems that probably rate a visit to the Genius Bar; there should be one near you in New York.

-- Your Aperture rebuild may be having problems with latency - possibly because the catalog of your system disk is damaged or your Lion install is damaged. Couple that with the slow speed of the Drobo and things may be timing out. I will admit that this is a complete shot in the dark.

Assuming that your 2 TB Library is Managed, that is that the 2 TB includes your masters, and that you do not have 2 TB of free space in your MacBookPro, then:

-- First relocate your Masters out of the Managed Library and put them somewhere on the Drobo. There re many, many threads on this Forum discussing strategies for doing this.

-- Move the Library (now without Masters) onto your fastest, emptiest internal drive.

-- Resync everything. A quick pass with iDefrag wouldn't hurt after this much file movement.

Why lecture about backups?

-- Well, first of all you said that you had no backups.

-- Secondly, we are trying to diagnose a problem at a distance, on hardware that we cannot see. We can't really tell if your problem is with your Mac, Lion, the Drobo, or Aperture. Under those circumstances we could easily give really bad advice. The cyber version of the Hippocratic Oath (First, do no harm.) is to assure that backups are available.

Good luck and best wishes.

--

DiploStrat 😉

Reply

Answer 9

e2photo

Level 2

176 points

May 2, 2012 12:38 PM in response to Doug F.

1. Ouch.....

2. You can check your internal drive with a small program called 'Smart Utility' Might be worth it if you have having lots of problems with Lion. Disk First Aid may or may not be able to identify a problem.

3. I would be very reluctant to do anything with your library on any kind of RAID system.

4. I would move it to a 4 TB drive (non-raid) and see if I could repair it.

5. If it is repairable, I would understand the difference between managed and referenced pictures and once understood, move all of the pictures out of the library using the relocate masters command and place them in appropriate folders (these could be placed on your DroboPro). If successful, your library will become quite small and will likely fit nicely on your internal drive. As example only, my 2011 Library is 56.15 GB in size (internal drive) and it points to slightly more than 1TB worth of pictures on external thunderbolt drive from Western Digital.

I have been working with D800 images (36MP) without it being particularly slow.

Reply

Answer 10

May 4, 2012 11:02 PM in response to Doug F.

Sorry to not really have a solution but slow leads to problems with data handling

a) Macbooks except the best 2011+ are slow and

b) Drobos are slow and problematic.

I just hope you can escape from your current predicament with data intact.

Lose the Drobo and connect to a large FW 800 drive. RAID0 frequently manually backed up would be great.

HTH

-Allen

Reply