What is BIG, what is F A S T ?

Question

Level 2

345 points

What is BIG, what is F A S T ?

Aperture is a rather complex application and image files are getting larger. All of this can lead to slow downs and other problems. And this, in turn can lead to a bit of thrashing by folks who don't fully understand how Aperture (and Lightroom) works.

Aperture is a data base application. An Aperture application starts out as two files/folders:

-- The Aperture application, normally installed in your Applications folder. This folder, of just under 900 MB, is a "Package" full of goodies that actually runs the application. It belongs on your fastest drive so that it can load into RAM when you start the application. No mysteries here.

-- The Aperture "Library." This is another Package, that is, a folder that does not normally open. Open it, by right clicking and you will see a list of folders. The most interesting, the only ones that you can really control, are labled "Previews" and "Masters." The largest single item in the Library will be your Masters, followed, normally, by your Previews and, finally, your Thumbnails.

So how can you make your Library smaller?

-- Take fewer photos. 😁

-- Shoot lower resolution Masters, ie., JPG as opposed to RAW. 😢

-- Reduce the size/resolution of your Previews. 😕

When you are using Aperture, you spend most of your time looking at Previews and Thumbs, not Masters. In fact, the only time that I am sure that you are actually using the Master is when you export, print, and when you view at full resolution. (Others may be able to clarify this.)

So I find the following to be reasonable:

-- Set the size of Previews at or one size smaller than the resolution of your largest monitor.

-- Set the quality as low as you can accept, generally 6 - 8.

-- Rebuild your old Previews and Thumbs at a lower size/resolution. There was a problem with one release of Aperture 3 that caused it to create bloated Previews and Thumbs. If this happened to you, then deleting your Previews and Thumbs will get you back some space. (Be sure you backup first and have ample time and disk space.)

That is about it. You can make slight reductions by limiting your metadata, or not using Faces (Faces create tiny thumbnails) but none of these amounts to much.

So what's next? RAM,RAM, and more RAM! Why. Because during any given Aperture session it is going to read the application, scroll the Thumbs, read the Preview, read the Versions, and, finally, read the Master. And then it will rewrite various of these as you adjust. If everything is in RAM, this will be fast. If not, then you will have to page to disk and this will be slower. And if your hard disk (HD) is full, much slower.

ALL HD slow as they fill. So, for best speed, you want to keep your HD as empty as possible, the sweet spot being somwhere between 50% and 75%. (Bigger HD can get fuller.) Why? Because Mac OSX is constantly making work files and rearranging data for the best fit and speed. The more free space available for this, the better.

So, after you buy all of that RAM, spring for the multi-TB HD.

Posted on Mar 26, 2012 6:14 AM

Reply

Answer 1

DiploStrat Author

Level 2

345 points

Mar 26, 2012 6:48 AM in response to DiploStrat

But what if you have a MacBook or a Mini and can't just add a drive? This is where Referenced Masters can be VERY powerful.

Sierra Dragon and others have provided detailed and accurate descriptions of excellent Referenced Master layouts and and workflows, so I will limit myself to some pedantic observations, which are, I hope useful.

First some overview. The term "Library" can be a bit confusing. It can mean the physical "Package" or the entireity of your Aperture installation. And you can have several Libraries.

Apple uses the terms "Managed Library" to describe a Library where the Master images are stored inside the Library Package. This is the default. A "Referenced Library" is one where some or all of the Masters are stored outside the Library Package.

What are the implications of storing Masters outside of the Library?

-- The biggest and most important is that you can unload your system HD. A Managed Master Library on a single HD that is loaded to over 75% is a disaster waiting to happen. (Imfamously, most of those disasters occurred during the transition from Aperture 2 to Aperture 3 when many users simply ran out of disk space and suffered crashes and other mishaps.)

-- Now some good news. As Aperture is a database and not a pixel editor, per se, Masters behave in some interesting ways, ways that you can turn to your advantage. The main thing is that Masters, once written to disk, do not move or change. (Technically, they are write once/read many.) This means that if you have enough RAM, the speed of the disk that holds your masters is not very important as you will typically read the Master only once per editing session.

-- For those of us who have sprung for solid state drives (SSD) looking for speed, this can be a life saver as SSD are still pricey, and most of us cannot afford a SSD large enough for our entire library. Relocating most of your Masters to another HD as "Referenced Masters" makes the use of a SSD quite practical. In my case, my numbers are as follows:

* 12,000 images

* 175 GB of Masters (Mostly RAW and scanned slides as 100 MB TIFF)

* 17 GB of Previews/Thumbs, etc.

So, allowing 50 GB for Mac OSX and all of your Applications, and 25 GB for your Aperture Library, many users can fit onto a 120 GB SSD very comfortably - if most of their Masters are relocated to a second HD. And the good news is that that disk holding your Masters could be an old USB 2.0 drive.

Notice that relocating your Masters does not make your database any smaller, it merely reduces the size of your Library package. Wherever you place them, Aperture still has to maintain the same links between the Previews, Thumbs, Versions, Metadata, Masters, etc. The good news is that UNIX is very happy to make those links across multiple HD. (And across networks, although it appears that Apple imposes some limits here.)

So, what can go wrong? Lots. With your Masters outside of the Library Package, Vaults become much less useful. I don't consider this a real obstacle as I use Time Machine, SuperDuper! and Crashplan to keep two local and one remote backups at all times. But if you were relying on Vaults for your backups, you will have to rethink.

The greatest "danger" is that if you use the Finder to physically move a Master, you risk breaking the link. This can usually be avoided by moving Masters only within Aperture with the "Relocate" and "Consolidate" commands. There are also relink tools which work as well.

Finally, YOU will have to develop some form of default layout for your Masters now that Aperture is no longer doing this for you. And do remember that that arrangement will never change, no matter what changes you make inside of Aperture. (But you can do a Consolidate/Relocate cycle if this matters to you. Rob Boyer pointed out on his website that even those who prefer a Referenced Master system might be well advised to suck everything into Aperture first, organize/rename, and then relocate. People forget that Aperture is quite capable of doing most of the tasks previously done with programs like Photo Mechanic.

Hopefully, all of this is both clear and accurate. Corrections welcomed.

--

DiploStrat 😉

Reply

Answer 2

Sep 4, 2012 10:56 PM in response to DiploStrat

Nice description.

I'm trying to fit my library on my Macbook Air doing what you said.

Faces just won't work on my Aperture for some reason. On top of that, for just about every photo, it makes another 3-4 thumbnails of the faces. I'd call this huge bloat for my 50k photo library.

Is there any way to get rid of all these face thumbnails? I'm guessing I'll get back at least 2gb which would make a big difference for me.

Reply

Answer 3

patH72

Level 1

34 points

Sep 4, 2012 11:16 PM in response to DiploStrat

One problem with referenced files is that, in certain situations, Aperture can "forget" where the orginal file is. In other situations, it attempts to find the file it cannot locate and fastens on to another file with no relation to the original image file itself.

It's also a problem if you move your masters to a new disk. Correct me if I'm wrong, but if I move all of my referenced files from a volume called "DiskA" to a new hard drive called "DiskB", there's no simple way to reconnect all of my references. You have to go through the tedious reconnect process, folder by folder, which Aperture provides. You would think there would be a simple way to tell Aperture to just substitute "DiskB" for "DiskA" in all the references but I don't think there is. If someone knows a simpler way to handle this I'd like to know about it.

Reply

Answer 4

Yer_Man

Level 10

162,242 points

Sep 5, 2012 1:29 AM in response to patH72

The trick is to do it with Aperture.

File -> Relocate Masters

Reply

Answer 5

Sep 5, 2012 2:06 AM in response to Yer_Man

Anyway, no tips on nuking all the Faces thumbnails? If I turn off Faces, then delete the thumbnails directory and then rebuild the thumbnails (yet another overnight operation) will it do the trick, or is it impossible to prevent the Faces from working.

Just trying to slim down my Aperture library to fit on my 128GB Macbook Air.

Reply

Answer 6

Sep 5, 2012 3:25 AM in response to patH72

I think this misleads:

patH72 wrote:

One problem with referenced files is that, in certain situations, Aperture can "forget" where the original file is. In other situations, it attempts to find the file it cannot locate and fastens on to another file with no relation to the original image file itself.

It's also a problem if you move your masters to a new disk. Correct me if I'm wrong, but if I move all of my referenced files from a volume called "DiskA" to a new hard drive called "DiskB", there's no simple way to reconnect all of my references. You have to go through the tedious reconnect process, folder by folder, which Aperture provides. You would think there would be a simple way to tell Aperture to just substitute "DiskB" for "DiskA" in all the references but I don't think there is. If someone knows a simpler way to handle this I'd like to know about it.

When you import an image-format file into Aperture you commit to using Aperture (and only Aperture) for all file management. That includes moving your Originals (formerly "Masters") to any new location, including a new HD.

(As Terence said, use "File➞Relocate Masters".) It's simple, and it works.

I have had well over half a million Images under management by Aperture. This spring, in a worrisome "cluster", I had five drives fail in as many weeks. Between keeping things running, restoring backups, and creating new backups, I moved millions of files. Not one Image or Original was lost, or "forgotten", or "fastened" wrong.

Over several years, several computers, and many drives, I have lost two Originals. I never found out why. I assume user error. IME, Aperture is a robust and reliable image and file manager.

Reply