Uploading photos to iCloud stalls unless the Photos app on Mac is active

Question

Level 1

39 points

Uploading photos to iCloud stalls unless the Photos app on Mac is active

I have a library of over 100,000 photos stored on an external Firewire drive which I've recently re-attached to my iMac, and have designated as the system library. The iMac is running Sierra 10.12.2. I've enabled iCloud photo sync, and am subscribed to 1 TB of iCloud storage which is sufficient to contain this library. In the past, I've been able to use this library with iCloud successfully.

At the moment, I'm stuck on re-syncing this library to iCloud. The Photos app has decided to traverse the entire library and is in the process of uploading it back to iCloud (or technically, I suppose that it is transmitting information about it so that it finds out if there are any differences between the local and cloud copies, which apart from a handful of recent photos, there will not be any at all). Specifically, it says that it is uploading X of Y photos.

Now, the problem is with respect to waiting for it to finish. The bizzarre thing is that the Photos app is only doing something when the app is running in the foreground on my Mac. It literally starts transmitting MB's of data when I click on its application window to bring it to the foreground. I know, because I am using iStat to show network activity. It transmits a few MB's, then after a few minutes, it stops again. I have disabled sleep on my Mac. But I have to return to my Mac and move the mouse before Photos starts transmitting again and making progress on the upload (which I can see counting down, slowly). If I switch to another app, or close Photos, then it will stop for good.

I know that normally, with a large library, that it can take a week or more to sync, which is not great but acceptable. However, in my case, I have to literally be working within the Photos app for the entire day to see any progress. I was under the impression that I can simply leave my Mac running and that it would eventually finish, and that Photos could even be closed and that a background process would do the job. But in my case, nothing happens. I don't see any sync related errors in my system log, either. It does make progress, but it is just painfully slow.

Is anyone seeing this? It's driving me crazy. By the way, I have tried turning iCloud photo sync off and then back on, but the same thing happens.

iMac, MacBook Pro, iPhone 3GS, iPod nano, Mac OS X (10.7), Apple TV, Time Capsule, Airport Extreme & Express

Posted on Jan 2, 2017 6:05 AM

Reply

Answer 1

Best reply

andrerc

Level 1

4 points

Jan 10, 2017 6:11 PM in response to Krakatos

I'm glad I found someone with the same problem as mine.

I have a mid-2011 27" iMac, macOS Sierra 10.12.2, with a ~100k photo library. I'm trying to upload it to iCloud for the first time.

I've experimented quite a lot with it and the only way I've managed to keep a steady upload rate is by:

disabling computer sleep;
disabling display sleep;
disabling put the hard disk to sleep (not sure if this had any influence on the outcome though);
disabling the screensaver;
using Network Link Conditioner to limit upload speed, as recommended here;
leaving Photos on the foreground.

If I left Photos on the background for 15 minutes, upload would suddenly stop.

I still have 280 GB out of 390 GB to go.

Reply

Answer 2

léonie

Level 10

194,761 points

Jan 2, 2017 6:16 AM in response to Krakatos

By the way, I have tried turning iCloud photo sync off and then back on, but the same thing happens.

Toggling iCloud on and off will just start the whole upload over again from the beginning.

You may have items in your Photos Library that cannot be processed and Photos is getting stuck on these items. Videos in an incompatible format can be causing this.

Have you checked, if the library contains items that cannot upload to iCloud? YOu can test this with a smart album with the rule "Photo is unable to upload to iCloud Photo Library".

Reply

Answer 3

LarryHN

Level 10

106,110 points

Jan 2, 2017 6:18 AM in response to Krakatos

Wha version of the OS and Photos do you have?

1 - Photos has not decieded to upload the library - you caused this by disconnecting the library and reconnecting it

I've recently re-attached to my iMac,

The only way Photos can determine if iCloud.com is correct to to upload and examine all photos -

2 - a week may be to short - I took 10 days to upload a 7000 photo library - yours will be a long time for 100,000 photos

3 - As far as I know the upload continues as long as the Mac is powered and awake - the upload has always been a background process

4 - everytime you turn it off and back on you restart the process form the beginning so if you keep doing that it will never finish - that is the worst thing you can do

I have tried turning iCloud photo sync off and then back on

LN

Reply

Answer 4

Krakatos Author

Level 1

39 points

Jan 2, 2017 6:37 AM in response to léonie

Thanks for the tip, but it must be something else. I've created the smart album to display items that cannot be uploaded, but there are none. There are no videos in my collection. Only JPEG and RAW images supported on the Mac. It's not that it's getting stuck at a particular point. It's that progress is really, really slow. Maybe a few hundred images a day, and again, only when I keep prodding the Photos app to be awake.

Reply

Answer 5

léonie

Level 10

194,761 points

Jan 2, 2017 7:37 AM in response to Krakatos

Is there any more general problem with the Mac? For example, are you low on storage on the system drive? Are other processes hogging the RAM or using all processing power?

Does your Firewire drive occasionally dismount itself? Have you checked the drive with Disk Utility?

Reply

Answer 6

Krakatos Author

Level 1

39 points

Jan 2, 2017 3:37 PM in response to léonie

There's nothing unusual as far as I can tell. It's a mid-2011 iMac, but it's equipped with 16 GB RAM. The external drive storing the photos has about 700 GB left. Nothing else is running in the system. It's actually an external RAID drive but is not suffering from disconnects, and it has no errors.

I just find it bizarre that I have to click on the Photos app to bring it to the foreground, for it to suddenly start uploading photos to iCloud. I can see a cloudd process running that is uploading to the network, and then the photos start counting down. Then it stops doing so after a few minutes, and network activity stops, even though the iMac is not set to sleep. The screen turns off eventually though. When I move the mouse again and the screen wakes up, it starts uploading photos again. Is no one else experiencing this? I don't see evidence of a background process doing all the work as I expected.

Reply

Answer 7

Krakatos Author

Level 1

39 points

Jan 2, 2017 3:42 PM in response to léonie

Actually, the only unusual thing that I can think of is that the drive is an encrypted volume. Maybe not many people use encrypted volumes. Maybe the background sync process doesn't work well with an encrypted drive?

Reply

Answer 8

Krakatos Author

Level 1

39 points

Jan 2, 2017 7:38 PM in response to LarryHN

Yes, it is extended (journaled) but it is also encrypted (by OS X). Maybe it's a red herring, but I only considered that it's encrypted once I was asked about the drive itself. I wonder if the background process that syncs photos does not work on an encrypted volume. I can't test it myself now without re-starting the sync, which I obviously don't want to do.

It is still syncing, it's just that it keeps pausing unless I'm actively using the Photos app, which drastically slows down the whole process.

Reply

Answer 9

léonie

Level 10

194,761 points

Jan 3, 2017 7:57 AM in response to Krakatos

It is still syncing, it's just that it keeps pausing unless I'm actively using the Photos app, which drastically slows down the whole process.

The encryption may be causing this problem. It should not, and I have not seen previous examples, where the encryption caused sync problems in Photos, but there have occasionally been problems with aperture libraries, where Aperture could not access the referenced originals, if they were on an encrypted drive. It was a timing problem. If the part of the drive that was needed next had already been unencrypted, it worked well, but again and again the originals could not be accessed.

Could you make a test with removing the encryption for a few days and see what happens?

Reply

Answer 10

Krakatos Author

Level 1

39 points

Jan 3, 2017 8:24 AM in response to léonie

Sorry, I made a mistake, I checked again and it turns out that the drive was actually *not* encrypted after all. I turned on the encryption just now and it exhibited the same behaviour. It doesn't appear that encryption makes a difference in my case.

Perhaps the external RAID drive is going to sleep, even though I have drive sleep turned off in the Mac OS settings. It's just a wild guess, but that's the only other thing that I can think of.

Reply

Answer 11

léonie

Level 10

194,761 points

Jan 3, 2017 8:55 AM in response to Krakatos

Are you using a wired connection to the internet? I had a much better download performance after I connected the MacBook Pro directly to the router and did not use the Wi-Fi. Occasional Wi-Fi drop-outs seemed to cause the download to pause.

Reply

Answer 12

Krakatos Author

Level 1

39 points

Jan 10, 2017 6:48 PM in response to andrerc

Thats interesting. After writing my original post, I coincidentally did the exact same steps that you listed, with the exception of the Network Link Conditioner. All with the goal of keeping the app actively running in the foreground. Frankly I think it's just designed this way, when doing a full resync, so that theres no impact on other applications. Most people don't have such large collections and won't even notice.

THis way, I made progress at about five GB per day. However, I have hit another snag. At around the halfway point, it went back to saying that it's uploading the full 100,000 or so photos again. And this time it doesn't appear to be uploading anything nor making progress. Id like to leave it running overnight but I cant because its in my room and I have to shut it off each night. Im still monitoring to see what happens.

Reply

Answer 13

May 10, 2017 10:38 PM in response to andrerc

I'd just like to add that I'm seeing exactly the same problem and accelerated the upload process by taking the same steps as recommended above, except without the Network Link Conditioner. I run on WiFi too.

Now I am stuck with it trying to upload the last 283 items, and only 10 photos apparently are 'unable to be uploaded to iCloud' using the Smart Album check. Nice tip by the way.

Reply

Answer 14

havarton

Level 1

4 points

Jul 24, 2017 10:31 AM in response to Krakatos

Can someone from apple ring in on this!! This has not been solved.

Macbook 12" 2016 Photos library stored externally.

The app Bandwidth+ Shows immediate stopping of upload as soon as Photos is not in the foreground.

Almost doesn't upload anything in the background...

I travel and am trying to back up video to the cloud when I get a chance to connect to wifi. I have other things to do as well during that time, not just sit with my mac on Photos and use it for nothing else!!!

Apple, where are you?

Reply

Answer 15

swandy

Level 4

2,545 points

Jul 24, 2017 10:59 AM in response to havarton

Unfortunately no one from Apple is allowed to "officially" respond in these forums. They do monitor them, but to get any type of support from Apple for your issue you have to contact Apple Tech Support directly. We are only users here.

I do know that my system Photos Library is around 3500 photos, and when I had to rebuild the library and do the upload thing a while back, it did take a while - and fortunately I was not trying to upload anywhere near the photos that some here are. I did not leave the Photos app in the foreground, just minimized, but I did turn off the settings for computer/hard drive/monitor to go to sleep.

Reply