14 Replies Latest reply: Nov 20, 2013 11:17 AM by e2photo
Sarge_ Level 1 Level 1 (0 points)

I presently use a Mac Pro with 4TB SATA drives, but my old MBP just died and I need to replace it.

 

The problem with new Macbooks is the very limited storage space. In my old MBP, I had two 1 TB drives (one in main bay, other in the optical drive bay via the OWC adapter).

 

Unfortunately, the new Retina Display Macs have insufficient storage options. I need lots of drive space when I travel (had 2TB, and that was on a 5 y/o MPB), and I really loathe having to cart around sloppy external drives which need auxiliary power, etc.

 

I know OWC makes this product which allows for a bit of extra storage (use the 768 GB drive as storage and a new 480 GB internal for the boot drive). But this solution is sloppier (external plugins) than the simple old setup with two internal drives.

 

I've also never really had a great system for syncing libraries between the MBP and the MP tower, it was always cumbersome.

 

So, are there any other pro photogs out there who have a good system for location shoots and syncing with large libraries?

 

I'm contemplating getting rid of the Mac Pro tower all together, but making a MBP work well seems problematic.

 

Also, FWIW, I shoot HDR, architecture mainly, and can fill a 64GB card in a day or two of shooting with my Nikon D800. That's why I need a lot of laptpp storage; When I travel, I may be gone for a month, and can easily shoot 500GB+ of content. If I use a new MBP as my only computer (and sell the MP), then I need a lot of drive space for email and other docs. Currently, but home folder has about 1.3 TB in it, and that's without any photo libraries (I have itunes and some video in there).

 

Any insight or ideas appreciated...


Mac Pro, OS X Mountain Lion (10.8.5), 3.33 Ghz 6-core, SSD Boot Drive
  • 1. Re: Large library workflow with Retina Display MBP?
    Ernie Stamper Level 8 Level 8 (37,475 points)

    My rMBP running 10.8.5 handles Aperture much better than my Mac Pro which can only run Lion.  I am loving to use it to process the images I shoot with my D800, but keep my main Library (currently 2 TB) on a 4 TB Thunderbolt drive.  I look forward to getting a new Mac Pro when it is released later this fall.  Since I will also use Thunderbolt drives with it, it suits my purpose to do so now.

     

    However, via Smart Update with SuperDuper, I easily keep a full clone on a USB3 drive (works very well, and perhaps as well as the TBolt), and also use SuperDuper Smart Update to keep a full clone uptodate on my Mac Pro from which I print.  I have always preferred a dedicate drive to hold a Managed Library, but do put some temporary libraries on my rMBP's internal SSD.  My temporary needs are not as great as you have indicated yours are, however.

     

    But I am very weary of not having both USB3 and TBolt interface on my old MP, and can hardly wait for the new one.

     

    Ernie

  • 2. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    There is a very harsh reality.  If you shoot pictures, you will eventually run out of internal drive hard space. 

     

    The solution is simple.  Keep your application on your internal drive, accept the inevitable and learn to store all of your pictures on an external drive.  You can also put the library there as well.  When it is full, you can simply purchase a new external drive.

     

    I do this all the time with my MacBook Pro Retina Display (probably have about 6 2 TB of portable external storage (USB 3.0).  I use Western Digital Passport drives (max size is 2TB) that do not require an external power supply.  I hold the library and pictures on the external drive.  I works, it is fast.  I even do Final Cut Pro X video work with the image files on the external Passport drives and the application on my SSD.

     

    I have a 500 GB SSD for my applications.

  • 3. Re: Large library workflow with Retina Display MBP?
    léonie Level 9 Level 9 (51,720 points)

    I've also never really had a great system for syncing libraries between the MBP and the MP tower, it was always cumbersome.

    I don't even try to sync libraries between macs

    When I am travelling, I start a completely new library on my MBP and import that one into my my main library when I return home. I'll keep a copy of the new library as well, so it doubles as a backup of the new photos.

  • 4. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    I divide my projects into to groups:  working projects and completed projects.  My completed projects are 'archived' on to 4-8 TB Western Digital My Book drives connected to my desktop.  My working projects are stored on Western Digital Passport drives.  I plug the Passport drives into which ever computer I am working on and I have my working projects with me!  I am very careful to make sure each computer has the same version of softwares.

     

    I backup my Passport drives regularly as well as my 'archive' drives.

     

    In this strategy, there is no syncing of libraries, but I work on the same library across several different Macs.

  • 5. Re: Large library workflow with Retina Display MBP?
    ontravel Level 1 Level 1 (10 points)

    How do you physically attach your multiple drives to your Mac? Do you have 6 USB sockets or play DJ and use them individually?

     

    Do you travel and if so, how do your handle your workflow?

  • 6. Re: Large library workflow with Retina Display MBP?
    Kirby Krieger Level 6 Level 6 (11,945 points)

    I'm not e2photo, but I use multiple Libraries and multiple drives in multiple places.

     

    Keep it simple.  Buy drives in triplicate.  One holds your working Library.  The other two are for back-ups.  Name them and label them clearly.  Never have all three in the same physical location.

     

    If your Library+Referenced Originals is larger than 2 TB, split them with the Library on one drive and the Referenced Originals on the other.  That's _six_ drives total for that Library and the drives to back it up.  (If you have need to store 2 TB of work, you shouldn't even hesitate at the cost of three external drives.)

     

    I clone every Library after every use.  I use SuperDuper!

     

    My rMBP has two Thunderbolt and two USB-3 ports.  My external drives are mostly USB-3, with a few FW800 drives still around.  The FW drives can be chained.  I use a FW<-->MiniDisplayPort adapter to connect the chain to one of my Thunderbolt ports.  (I connect my external monitors to the other Thunderbolt port.)  The USB-3 drives get connected to my USB-3 ports.  I keep two USB-3 hubs at work.  One has seven ports, the other has four.  When I know that I will need more than two ports outside my studio, I take the four-port hub with me.  It does require external power.  None of my USB-3 drives require external power.

     

    In practice, I've found the above easy to use and to maintain.

     

    Message was edited by: Kirby Krieger

  • 7. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    I think you are asking two different questions.  For the desktop situation:

    1. Thunderbolt drives and FW drives are daisy chained on my iMac

    2. USB 3.0 drives (Thunderbolt port to the Belkin Thunderbolt Express-->Belkin USB 3.0 Hub off the Thunderbolt Express)

    3. USB 2.0 drives (thru the USB 2.0 port to a USB 2.0 hub)

     

    When traveling with my retina display I carry 3 USB 3.0 Western Digital Drives (I typically use the 2TB passport drives as they don't need an external power supply) and at least two different USB 3.0 card readers.

     

    Drive 1: this contains two partitions.  One partition is sized to hold a bootable backup of my SSD, the remainder is reserved as a backup partition (about 1.5 TB or so).   Once I return and have secured the pictures on other drives, I will reuse the backup partition for future travels

     

    Drive 2: usually just a single partition.  My usual strategy is to import the pictures off the cards into individual Aperture Libraries for each day of travel.  I label them as a 6 digit number YYMMDD and the project contained in the library will have the same name.  I usually do this right before I go to bed.  As soon as I have imported them into the library, I close the library and eject the card reader.  I then plug in Drive 1 and do an incremental SuperDuper backup of Drive 2 to the backup partition on Drive 1.  At this point, I have two copies of the pictures and I try to keep the two drives a part from each other (eg. one in my camera bag and one in a second carry on bag.

     

    I may or may not continue to use the cards until they are full, just pulling the pictures for the particular day or I may reformat the cards to use them again.  Since I shoot with D800's I also have a duplicate SD card to preserve images with if necessary (that would give me a third copy)

     

    In my last trip to Peru, I generated 221 GB of digital data.  Pictures and video.  I store both in Aperture and other than just doing a casual check, I don't do much editing, rating and etc.  I am much to busy typically.

     

    Once I return home, I will generate a backup of Drive 2 to a desktop My Book drive before I do anything.  Once done, I will consolidate all of the individual libraries into a single larger library (consisting of project for each day.)  Typically, the first day Project I relabel with a date range YYMMDD(first day of travel)-MMDD(last day of travel and then generate an album labeled YYMMDD and move all the pictures from the first day into that album.  I then move the pictures into the first project and place them into an album YYMMDD (second day of trip).  This consolidates the pictures into a single project, but individual days are held in individual albums. 

     

    This approach helps me then work on the full set of travel pictures within a single project.

  • 8. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    Not sure the next to last paragraph is all that clear.  My final travel library is labeled YYMMDD(first day)-MMDD(last day).  Inside this library will be one project labeled

     

    YYMMDD(first day)-MMDD(last day)

    Album YYMMDD(first day)

    Album YYMMDD(second day)

    Album YYMMDD(third day)

    ...

     

    My Peru trip library was 221 GB

    My Isla Mujeres trip was 189 GB

     

    Hope this helps.  If not feel free to ask questions.

  • 9. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    One last note

     

    I also do what Kirby does.  I buy drives in triplicate for my iMac (one working drive, one backup drive and one archive drive that is stored off site).  I use SuperDuper to generate the copies and it has worked very well. 

     

    On my iMac, everything is referenced.  I do not store the pictures in the library (managed pictures)  I have a file structure using folders that allows me to get to an image file very quickly without going into Aperture, if need be.  That mean if for some reason the library fails, gets corrupted, I still have easy access to the primary pictures.

     

    When traveling and when I am actively working on the project, however, I keep the pictures in the Aperture Library (managed).  These libraries are typically on Western Digital Passport drives (now 2TB). 

     

    Once I am done with a project, I import the library into my yearly master library on my iMac and immediately relocate the masters to an external folder on a Thunderbolt drive.  This allows me to keep the library relatively small.  I have backup and archive copies of my iMac drive and backup and archive copies of the Thunderbolt drive.  Eventually I will move the yearly Aperture Library to the drive that holds the referenced pictures (drag and drop).  The links remain intact.

  • 10. Re: Large library workflow with Retina Display MBP?
    ontravel Level 1 Level 1 (10 points)

    Thanks, I like your strategy with 3 drives and I might follow this.

     

    Incidentally I have just bought the WD My Passport for Mac as it had USB 3. Another model had a metal casing, but only had 2 Firewires and a USB 2!  I will start the daisy chaining task when I return from my next trip. 

     

    I wish to store all photos on an external drive as you do. For that I need to move them from my Mac. Is this a problem with any edits and tagging? My collection goes back some time and when I first started with Mac I played with iPhoto, which I Iike (for some things) to this day. I still import all data into iPhoto, which I use for sorting shoots. For editing, I then open Aperture.

     

    So my question is, how can I move all photos and all their relevant data to an external drive so that I don't loose the "Event" (iPhoto) / "Project" (Aperture) names and naturally not any META data either?

  • 11. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    Hi,

    I can not advise you on iPhoto since I have not used it for about 4 years.  All my pictures are linked to Aperture in some way.

     

    With respect to Aperture and Aperture Libraries I have several configurations on my iMac, depending on the life cycle

     

    A bit about structure

    For each year, I generate a nested folder structure that looks like this:

    2013 Pictures

        January 2013

        February 2013

        thru to December 2013

        Misc Pictures 2013

        Misc Libraries 2013

        photo.net to hold pictures upload to this site

        facebook to hold pictures upload to this site

       and etc

     

    Inside each monthly folder there will project date folders that have this general structure (YYMMDD) so inside the January 2013 folder one might the following

       January 2013

           130106

               130106 NEF (the folder that holds my RAW files)

               130106 JPEG (things that I upload or transmit to clients)

           130110 

              130110 NEF (the folder that holds my RAW files)

              130110 JPEG (things that I upload or transmit to clients)

     

    and etc.  For any images related to the project, I usually store in their respective folder.

     

    In fact I have 2011, 2012 and 2013 pictures on a single 6 TB Thunderbolt drive.  The 2011, 2012, and 2013 Aperture libraries reside on my internal iMac drive.  This libraries are referenced and point to the 6 TB Thunderbolt drive.  This drive is getting full so when 2013 is over, I will move all of the Aperture Libraries over to the Thunderbolt drive.  Those libaries already know where the pictures are so typically I don't have to do much more than drag and drop the libaries to the Thunderbolt drive.

     

    I will pick up a 3 8TB Thunderbolt drives and start building a new collection starting with 2014 pictures.  I will likely last through 2016 or 2017. These new Aperture libraries will reside on my internal iMac drive since I have moved older libraries to the external drive.

     

    Earlier this year, I consolidated my pictures from 2001-2010 on to a single 8 TB Thunderbolt drive (called 2001-2010 Pictures).  They were spread out over 5 or 6 different disks.  I still have those disk just in case, but I only had to do the following:

     

    I dragged and dropped each years picture folder to the 2001-2010 Picture drive.  So 2001 Pictures, 2002 Pictures, 2003 Pictures and etc.  Then I dragged and dropped each yearly Aperture Llibrary over to the 2001-2010 Picture drive: 2001 Aperture Library, 2002 Aperture Library and etc.  At this point, the Aperture Libraries do NOT know where the pictures are located since I have moved the pictures from their original drive to a new drive.  BUT, open a particular years Aperture Library, go to File: Locate Referenced Files and point the library to one correct picture (I may have 30-40,000 pictures in a particular year) in the appropriate Pictures folder and select connect all.  Aperture will do all the work to reconnect all of the referenced images back to their master pictures. 

     

    One last step: Verify that all of your pictures are reconnected.  Go to the search icon in the Aperture Library and 'Add Rule:File Status' and make sure all of pictures are reconnected to the library.  Occasionally, I will find a picture that is not connected but will a little bit of work, I can find the original and move it into the library so they can be connected again.

     

    AND BACKUP AND ARCHIVE AS YOU GO!

     

    So when I want to find a picture from the time frame 2001-2010, I mount the 2001-2010 Picture drive, open the relevant Aperture library and I have my picture of interest.  This works pretty well, if you keyword your pictures as you go along. 

     

    Lastly, I have not had any difficulty running Aperture Libraries (either referenced or managed) on external drives (USB 2.0, USB 3.0, FW or Thunderbolt).  In fact, nearly all of my editing is on USB 2.0 or USB 3.0 passport drive, before I bring them into the master yearly library.

     

    As I said at the beginning, I don't have recent experience with iPhoto, but I would assume similar strategies would work.  But I suspect that using both iPhoto and Aperture makes your work flow more complicated than it needs to be.  What functionality does iPhoto give you that can not be found in Aperture?

     

    Good luck and remember: backup, backup, backup.

  • 12. Re: Large library workflow with Retina Display MBP?
    ontravel Level 1 Level 1 (10 points)

    Sincere thanks, you spent a lot of time just to compose that reply to me

     

    Following your comments, I don't think there will be any problems, but I will do some tests when I get back from travels.

     

    I do make backups but rarely find the time to verify their restorability... who does?

     

    > What functionality does iPhoto give you that can not be found in Aperture?


    iPhoto is of course a consumer product, but I found it very useful during my initial 'migration' to the Mac platform. I used and still use iPhoto Manager to create various libraries, mostly private, but a few for business. There, I have some set-ups for easily creating an attractive collection of photos with captions, which I can 'print' as a PDF and send to anybody. These functions are possibly available in Aperture, but my set-up has worked sofar, so I have not spent the time to investigate.

     

    Thanks again

  • 13. Re: Large library workflow with Retina Display MBP?
    Kirby Krieger Level 6 Level 6 (11,945 points)

    ontravel wrote:

     

    I do make backups but rarely find the time to verify their restorability... who does?

    I put Libraries and referenced Originals on external drives (or volumes), and clone the drives (or volumes).  Verification of some use is easy: mount the drive, launch Aperture, check a smart Album to make sure no Image's are missing Originals, take a quick scroll through Projects, load one, close Aperture, unmount the drive.

     

    I do this more out of curiosity than for rigor, but I did once find that SuperDuper was failing during my overnight auto-backups (faulty USB port -- not SuperDuper's fault), so I try to check regularly, if not often.

     

    Easy verification is one of the advantages of cloning drives (or volumes) instead of using a back-up program such as Time Machine or Aperture's Vault (though Vaults can -- afaik -- simply be renamed and used in place).

  • 14. Re: Large library workflow with Retina Display MBP?
    e2photo Level 2 Level 2 (165 points)

    I verify my backups to make sure all the files are attached to the library, as Kirby indicated.  But, every once in a while an individual file will have a problem and I don't check for that necessarily.  In other words, I don't look at each picture to know that it is ok.