How to best store images with Aperture's Library?

Question

Level 1

31 points

How to best store images with Aperture's Library?

I do not understand, after all this time, how the Aperture library works. I've done a little research online but I cannot see anything which satisfies me. The problem is that I have a 500GB HDD and its getting full, with Aperture taking up roughly 300GB of it!

I find that I have a library with 317.91GB in it and the folder has 47k images, but my library has over 100k, so are the rest inside the library? Am I doubling up the amount of imagery with them organised in my Photos folder and a big Aperture library?

I'm not sure if I need the two or wether I can amalgamate the two, so if anybody know please let on thanks.

MacBook (white), Mac OS X (10.6.5), 4GB 667 MHz DDR2 SDRAM

Posted on Jan 2, 2011 11:37 AM

Reply

Answer 1

Mr Endo

Level 4

1,010 points

Jan 2, 2011 2:16 PM in response to Hamper

Hamper,

To what are you referring when you write of "The two"? I'm not entirely sure what you're asking.

In general, though, if you have Aperture manage your photos, it will make a copy of each photo in its library from wherever you imported it. So, if you copy your camera's flash card contents to your hard drive, and then import "Managed" masters into Aperture, then you do have two instances of the same file on your computer. One is where you put it, and the other is managed in Aperture's library. If this is the case, you don't need that original, since you told Aperture to copy the photos into its library.

nathan

Reply

Answer 2

Hamper Author

Level 1

31 points

Jan 2, 2011 3:05 PM in response to Mr Endo

By two I meant the files in the photo folder and the library. I've just noticed I have the import option to the Photos folder, so if I set that to the Aperture library will it run faster? I take it then that the images copy to the library file anyway, so I could delete the photos in the Photo folder, how can I check they exist in the two places?

I worried about removing the photos from the photo folder and they'll be deleted from Aperture, is there a way to merge everything into the library file and be done with it, without causing duplicates?

Thanks!

Reply

Answer 3

Mr Endo

Level 4

1,010 points

Jan 2, 2011 3:24 PM in response to Hamper

Hamper,

It sounds like you are currently set up to have "referenced" masters. That may not have always been the case, though, so you may have a mixture of referenced and managed masters. That is not a problem at all, but rather a demonstration of the flexibility of Aperture.

The location of your photos only affects the speed of Aperture if you are storing them on a faster or slower medium. A network disk would be slower (and potentially problematic). Another location on your hard drive shouldn't affect the speed. Aperture looks up the photo in one place versus another place on the same hard drive. Reading from the storage device is the biggest hit in the speed of your library, not figuring out where to read it from.

Go to the top level of your Aperture library and display based on a filter (The magnifying glass in the upper right corner). Make a filter based on "file status" is "referenced" and you will see what pictures Aperture is getting from outside its library -- that is, pictures that may be in your Photos folder.

If you have no referenced images, then you have no worries about removing the photos from the Photo folder, *except that you don't know if you have all of those pictures in Aperture yet*!

One of my favorite techniques (depending on how many photos we're talking about) is to move the photos I'm worried about to a different directory and start up Aperture and see if things behave right. This is of course a fallable technique. Asking Aperture where it's storing things (through the above mentioned filter) is probably the best bet.

If you import your entire Photo folder, and ensure that you are not importing duplicates, you probably will get the "merge" you want. Make sure you use your brain here, though. Take a look at a good sample of photos that Aperture presents for import and make sure you're looking what you think you're looking at.

nathan

Reply

Answer 4

Hamper Author

Level 1

31 points

Jan 2, 2011 3:47 PM in response to Mr Endo

One project has me with about 600 in the library and 37k outside, based on that filter. With about 4GB to spare on my disk I'm wondering if it will be possible to do a big import, because it would be a move function wouldn't it?

Searching for managed I see those 600 files but when I click "Locate referenced Files..." I get the message "The selection does not have any referenced files" - so what is that? The "Show in Finder" option does not exist for these ones.

I've also got about 300 "offline" images which cannot be found in Finder but do Reference, although the text is red. The remainder of the 37k are listed as "online".

Missing images are some 300 also, which I think could be the Offline ones. What is going on??

Lets say I add the 37k photos from my images folder, if I wanted to see them in Finder would it let me go into the library folder? I doubt it! so how to I get access to the file, do I go via the referenced route?

Next I was wondering how I might preserve the exact metadata I had when importing a file if its been appended after I added the file. So, by importing the original from the photo folder and it not containing the extra metadata I added after I added the file would I lose my metadata? Would I have to export all 37k files in order to preserve my metadata?

Thanks.

Reply

Answer 5

Mr Endo

Level 4

1,010 points

Jan 2, 2011 6:07 PM in response to Hamper

Hamper wrote:
One project has me with about 600 in the library and 37k outside, based on that filter. With about 4GB to spare on my disk I'm wondering if it will be possible to do a big import, because it would be a move function wouldn't it?

Not sure what you are measuring or how you are measuring it. "600 in the library and 37K outside?" 600 pictures that are referenced based on the filter? And you were able to isolate those 600 pictures to look at through Finder "outside?"

With only 4 GB of space, you're probably playing with fire to do a big import. Yes, I believe Aperture will move the files, and the individual photos won't take up any more space, but there's overhead for Aperture's library, no matter if it's referenced or managed.

Searching for managed I see those 600 files but when I click "Locate referenced Files..." I get the message "The selection does not have any referenced files" - so what is that? The "Show in Finder" option does not exist for these ones.

So did you search for managed files (as this paragraph implies) or referenced files (as your first paragraph implies)? If you searched for managed files, then of course there are no referenced files in your selection -- Aperture is showing only managed files, and it will not let you peer into its library to see them in Finder. Export files if you need to use them in Finder.

I've also got about 300 "offline" images which cannot be found in Finder but do Reference, although the text is red. The remainder of the 37k are listed as "online".

Missing images are some 300 also, which I think could be the Offline ones. What is going on??

Your offline files may be deleted referenced files. Can you look in Spotlight for the file names corresponding to some of your offline images? If they aren't on your hard drive, you're probably out of luck. Somehow you made a master that is referenced to something that is either stored on a different drive or that you already deleted.

Lets say I add the 37k photos from my images folder, if I wanted to see them in Finder would it let me go into the library folder? I doubt it! so how to I get access to the file, do I go via the referenced route?

If you add the 37K photos as Managed photos, you do not need to see them via Finder. That's the essence of Aperture's library. Aperture is managing them for you. Finding things in the Aperture library is a bad idea. If you need a particular photo, you export it to somewhere new (like a temporary folder on your desktop), do with it what you need, and then delete that exported file.

Next I was wondering how I might preserve the exact metadata I had when importing a file if its been appended after I added the file. So, by importing the original from the photo folder and it not containing the extra metadata I added after I added the file would I lose my metadata? Would I have to export all 37k files in order to preserve my metadata?

Sounds like the topic for a new thread in this discussion group, but remember you can always export your managed master from Aperture and get exactly what you started with. You've got about 3 or 4 questions in the above quoted paragraph, so posting a new thread would be good.

nathan

Reply

Answer 6

Jan 2, 2011 6:24 PM in response to Hamper

Yours is a common problem, even typical when folks use Managed Masters. You are no doubt currently using Aperture's (unfortunately default) Managed-Masters setup, which invariably leads to overfilled hard drives on single-drive boxes.

Working with only a few GB of free hard drive space is a recipe for disaster. Once you switch to the more appropriate *Referenced-Masters Library* Aperture will help you keep from overfilling the internal drive.

First off, all *hard drives should be less than 70% full.* That is important for speed and stability. Don't try to do anything until <70% full is achieved for all drives. To get to less than 70% full move data off of the internal drive to external drives.

The Aperture Library should live on the internal drive for best speed.

Best solution (after you achieve minimum 30% free space on all drives), especially with mobility-hardware computers like laptops and iMacs, is to reconfigure your image Masters to have a *Referenced-Masters Library* on the internal drive with the Masters referenced on external drives via FW800 or (preferably) eSATA; however Referenced Masters usually do still work adequately with Masters accessed on externals via slower methods like FW400 or USB. After a slow reconfiguration process it will work fine, your speed and operation will be recovered and the Referenced-Masters Library will remain a reasonable size, not again grow to overfill the internal drive.

Note that Vaults back up the Library but a Referenced-Masters Library does not have the Masters in it, so Masters must be separately backed up. *I very strongly recommend that back up of original Master files be performed BEFORE importing into Aperture.*

Aperture 3 has improved methodology for handling Libraries. I suggest that your steps should be:
• Back up your current Library with its Masters to a Vault on an external drive.
• Move data off of the internal drive to external drives to get to less than 70% full.
• Set Aperture 3 to use your existing Library on the internal drive.
• Use Aperture 3's tools to reconfigure your Library to be a *Referenced-Masters Library* on the internal drive with the Masters referenced on external drives.
• In the future back up Masters before import into Aperture or any other images management application. Then with the new Masters on external drives select Store Files: "In their current location" from the import window when importing into Aperture. This will maintain a *Referenced-Masters Library*. Routinely create Vaults to back up the Library.

HTH

-Allen Wicks

Reply

Answer 7

Hamper Author

Level 1

31 points

Jan 3, 2011 10:34 AM in response to SierraDragon

Thanks SierraDragon for your thorough reply, but I can't be using Managed-Masters because 95% of my Pictures are outside of the library and referenced, I have since found out. I wasn't under the impression the default was to store inside the library, I have always saved to the Pictures folder.

I know about the 70% HDD issue (although I thought it was 50%), but what can I do, I can't split my Aperture Library, I wan't everything in one place. Storage on an external drive is a no no because I use a MacBook, also I have only FW400 and USB2, so I know internal storage is best and of course SSD would solve a lot of my problems.

Mr. Endo is advocating the use of Managed-Masters while you keep telling me to rely on a Referenced-Masters library, but I think Managed is better because everything is in one place. Also, I'm had an internal HDD failure before and although I'd made a recent backup with Vaults (I have backed up with vaults since v.2 by the way) and Time Machine, the process of piecing back together the library was arduous to say the least - it was all manual!! If it were all contained inside the library file It would have been simple, right?

As my library is already 317.91GB I am left a little unsure as to why, because I know I have most of my images stored in the Pictures folder (122.64GB) but not sure why its so big just from Reference images and metadata (which is basically text!). I base this on the "File Searches" (Managed/Referenced) which Mr Endo advised me to do and with only 7321 images as Managed the rest (126,422) are not inside, so why so big, my library being mostly JPEG and consisting of 133,743 images??

How can the Referenced-Masters Library remain a reasonable size if placed on an external source like you say? surely it will be the same size anywhere?! You mean my local HDD will be kept under control, well sure, but tethering an external HDD is no fun with a MacBook as I use it on my lap. I think that storing imagery inside the Library is better, I'm not one of those who touches the folder structures in the Pictures folder and work exclusively inside Aperture, it seems like those who use referenced masters are worried about that.

I base my decision on centralisation as, having read the following, I get the impression that Managed will allow Vaults to back everything up, even though I have a Time Machine its easier this way. The only part of the document which I think is opinionated, rather than factual, is the last bit which mentions:

"As a point of comparison; most hobbyists use Managed, most pros use Referenced."

ref: http://aperture.maccreate.com/2010/01/12/tip-0-referenced-vs-managed-files/

..I must be a hobbyist because I don't shoot weddings and edit on the road.

Another site had most people using Managed because its easier to handle imagery and back up, while those who used a reference library did so because of the restrictions of space on a portable and bad habits from the past. Yet I believe that I can manage my library on one HDD and don't fancy the idea of tethering external storage from a MacBook.

The only things I am concerned with now are the the "Offline" and "Missing" discrepancies in my library; which come to 1277 images, I don't know how this happened but I should be able to get them back with my Time Machine backup shouldn't I?

So, if I move to a Managed Library with the built in tool (set to move, not copy) I've been told I might get duplicates, hmm, this is to be avoided, but what on earth is taking 317.91GB of space when 95% of my library is outside of it and in a folder which totals to 122.64GB?? Further, is it an all or nothing affair when engaging in this change or can I do selected imagery at a time?

Thanks.

Reply

Answer 8

Hamper Author

Level 1

31 points

Jan 3, 2011 10:36 AM in response to Mr Endo

Thanks Mr Endo, but a lot of those questions of mine are redundant now as I've gotten to grips with the Managed and referencing terms for Aperture now.

Reply

Answer 9

141FD

Level 1

0 points

Jan 3, 2011 11:48 AM in response to Hamper

I also prefer managed masters though I do have another master backup just like we use as referenced masters. So I maintain and work under both structures. I first started with managed then after reading all the discussions here, especially the long referenced vs managed thread, I moved to referenced but now I am back to managed and think its Aperture's beauty to maintain masters.

I have SSD on my notebook and my drives space on mac pro doesn't really matter at all to managed masters. I still have some referenced masters though for purposely but I enjoy working under managed masters than referenced masters. I don't think its a matter of pro using referenced and hobbyists use managed. It's a matter of the way you think.

Reply

Answer 10

Hamper Author

Level 1

31 points

Jan 3, 2011 12:36 PM in response to 141FD

Whats Aperture like with an SSD?

Reply

Answer 11

141FD

Level 1

0 points

Jan 3, 2011 12:46 PM in response to Hamper

I will never go back to HDD if that tells you something, would be an answer.

Reply

Answer 12

Jan 3, 2011 1:36 PM in response to Hamper

Outside the Aperture Library and Referenced is ignoring the fact that they are referenced to the Pictures folder on the same boot volume, and thus competing for space on the save drive.

In my book, Referenced only makes sense if the referenced Masters are on a completely different disk drive.

For my purposes, I favor a Managed Library where the complete Aperture Library resides on a different drive than my boot volume. But with my Mac Pro, that other drive is still an internal drive, and from what I have read having the application on one drive and the Library on another internal drive is the fastest configuration. That cannot be said of an external drive, however.

I modeled my configuration after the advice from Apple for configuring Final Cut Pro, where the video assets are on a drive or drives totally separate from that where the application is running. I have also read that Apple originally planned Aperture to only use a Managed Library, but that was not as practical for use on laptops of most consumers.

Ernie

Reply

Answer 13

Jan 3, 2011 1:40 PM in response to Ernie Stamper

Ernie Stamper wrote:
Outside the Aperture Library and Referenced is ignoring the fact that they are referenced to the Pictures folder on the same boot volume, and thus competing for space on the save drive.

In my book, Referenced only makes sense if the referenced Masters are on a completely different disk drive.

+1

Reply

Answer 14

Wes W.

Level 2

255 points

Jan 3, 2011 3:34 PM in response to 141FD

Tested the fast Crucial C300 (~350MB/s) SATA 6 SSD for two days with my managed Aperture library. The SSD was a little faster, but not worth the trade-offs, IMHO. Reverted back to the Seagate "nearline" drives in RAID 1 which I've been very happy with and don't miss the SSD.

Don't compare a SSD to a slow HDD...that won't even be close. Unfortunately, most laptop-class hard drives are pitifully slow, so a SSD might make more sense there if you can afford enough capacity.

Reply

Answer 15

Hamper Author

Level 1

31 points

Jan 3, 2011 3:59 PM in response to Wes W.

Well I've got a 5200RPM HDD and its only got 4GB left on a 500GB HDD, so I know anything will be faster. But at least it hasn't overheated like the 7200RPM version I got, which broke before! I am not touching SSD until the transfers hit 1GB each way, no point in upgrading.

Reply