Installation stuck at "Retrying http://swcdn.apple.com/.../InstallESDDmg.pkg after 1 failure(s)"

Question

JinsungL Author

Level 1

6 points

Installation stuck at "Retrying http://swcdn.apple.com/.../InstallESDDmg.pkg after 1 failure(s)"

Spec

Hardware: MacBookPro12,1 @ 2.70 GHz (x 4), 8192 MB RAM

Running OS Build: Mac OS X 10.13 (17A365)

I'm using FileVault.

Backgrounds

I was using Sierra, tried to upgrade through Appstore, successfully downloaded, launched installer.

It seemed working fine. Installer required me to restart my macbook and I did. Black screen with apple logo appeared and the progress bar was telling me that everything was working fine at that time.

But then suddenly a message showed up. "something is wrong blabla .... you should diagnose with holding D when booting up .... or just restart."

So I restarted it, now there's a question mark folder in black screen. I tried Safe mode, same screen.

Now, I'm on probably 7th try to install it on Recovery mode but it's keep failing.

Current State of The Problem

I know there are 3 different options for Recovery mode when you're trying to reinstall your macOS. How to install macOS - Apple Support

Command + R - Install the latest macOS that was installed on your Mac, without upgrading to a later version.
Option + Command + R - Upgrade to the latest macOS that is compatible with your Mac.
Shift + Option + Command + R - Install the macOS that came with your Mac, or the version closest to it that is still available.

First, I tried the first option. Apparently, at the time, the latest macOS that was installed on my macbook was High Sierra. Maybe because there are some progresses that were made even though it stopped due to error or crash or whatever. Anyway I tried reinstall, didn't work. Here's my recent logs.

Retrieving 7 packages (5.162 GB)
No native mechanism to verify AppleDiagnostics.dmg
Retrieved package com.apple.dmg.AppleDiagnostics (http://swcdn.apple.com/content/downloads/04/61/091-34298/almpfkbhyxnsgbxxqhoqo7sb40w3uip0wk/AppleDiagnostics.dmg)
No native mechanism to verify InstallInfo.plist
Retrieved package com.apple.plist.InstallInfo (linkfile:///var/tmp/OSISPredicateUpdateProductTemp/InstallInfo.plist)
No native mechanism to verify AppleDiagnostics.chunklist
Retrieved package com.apple.chunklist.AppleDiagnostics (http://swcdn.apple.com/content/downloads/04/61/091-34298/almpfkbhyxnsgbxxqhoqo7sb40w3uip0wk/AppleDiagnostics.chunklist)
Retrying http://swcdn.apple.com/content/downloads/04/61/091-34298/almpfkbhyxnsgbxxqhoqo7sb40w3uip0wk/InstallESDDmg.pkg after 1 failure(s)

And nothing after that line. It stuck. Saying "About 24 minutes remaining" for hours.

My trials

I tried the second option with Option + Command + R but the result is the same.

I check the disk utility, ran First Aid, it says my disks are OK.

45.97 GB / 120.47 GB are Free.

Note: it says my disk format is AFPS (Encrypted). don't know why.

I was able to navigate the folder named "/macOS Install Data" in which all of "AppleDiagnostics.chunklist", "AppleDiagnostics.dmg" etc are located. No "InstallESDDmg.pkg" found but there was "InstallESDDmg.pkg.partial".

Thankfully I could navigate "/Users/myname" and all my files seem to be ok.

I've tried wating for 2 hours, nothing has changed so far.

I haven't tried the third option because I'm afraid to lose my data while trying to downgrade to my old macOS.

Help me!!

I can't figure out why it can not download "InstallESDDmg.pkg" when it seems like I can download it using Safari, Chrome in other laptop. Although it says it will take 50m to complete download. That's why I tried to wait for 2 hours.

So I think it doesn't make sense that there's a network issue.

According to the image posted in this article: https://eclecticlight.co/2017/09/25/macos-high-sierra-installation-and-issues/

my next step should be "Starting SKDaemon...".

But I've never seen that line showing up.

Please help me out.

MacBook Pro with Retina display, macOS High Sierra (10.13)

Posted on Sep 27, 2017 3:57 AM

Reply

Answer 1

JinsungL Author

Level 1

6 points

Sep 28, 2017 11:28 AM in response to JinsungL

Short Answer

I made instructions. Follow them if you’re experiencing the same problem.

https://github.com/Jinsung-L/install-macos-high-sierra

Updates

After posting this discussion, another 5+ hours passed, nothing has changed.

So I tried different approach. Remember I could access to my Macintosh HD volume though Safari’s file navigator? I tried to find the exact location of ‘macOS Install Data’ on terminal so that I could inspect more information.

I found out that they are located at ‘/Volume/Macintosh HD/macOS Install Data’ and was able to confirm that all the essential files for OS update were located in there.

Then I thought I could just download ‘InstallESDDmg.pkg’ manually using curl then move it to the folder. I tried, didn’t work. There are two important notes here.

A. Downloading stops when it hits around 326MB. I don’t know why this is happening nor if I’m the only one who’s suffering from this issue.

B. Downloaded files get removed when you start the installation.

Fact A can be solved easily, you just try multiple times with curl’s -C option. You fail like 30 times but eventually you’ll get the complete download. Look carefully if the ‘current download speed’ drops to 0. Then you’ll have to cancel the process and restart it again.

Fact B was a huge problem since even if you could somehow place the complete file at the right location, it will be removed.

So my solution to resolve this situation was;

1. Create a server that contains and serves pre-downloaded files you need but when it doesn’t have one, it will stream the original file from Apple server ‘swcdn.apple.com’.

2. Download all the files at the right directory of your server. You have to match the path of urls.

3. Modify ‘/etc/hosts’ file of your macbook.

4. Update will work now.

I created #1 using flask, downloaded all files including ‘InstallESDDmg.pkg’. Also I configured my server to stream missing files from ‘swcdn.apple.com’ if it doesn’t contain one. Later, I missed some essential files and this feature saved the installation process from errors.

I uploaded the source code at github.

#3 Didn’t work. Apparently you can’t modify ‘/etc/hosts’ when you’re in Recovery Mode. I guess.

I tried every possible method I knew including chmod, chown, chflags but all of them didn’t work.

So I had to decide to create a DNS server to make this work without modifying ‘/etc/hosts’ file. I created one using dnsmasq on ubuntu. Then I configured it to redirect ‘swcdn.apple.com’ to my custom file server, except from its own.

I’ve actually done this by redirecting ‘swcdn.apple.com’ to <my server ip> with no exception, then making my server to use the ip address of ‘swcdn.apple.com’ with header ‘Host: swcdn.apple.com’. But you can do this either way. The key point is to make your mac to lookup your files server IP address instead of Apple’s one but your file server to lookup the original Apple Server's IP.

Don’t forget to configure your router to use your DNS. Then you’re good to go.

Still I had to try this several times because you can’t figure out which files are needed to be downloaded unless you try the installation step. – So don’t forget to turn on the debug mode and check the log.

But eventually this method worked.

I made this process easier for someone who’s experiencing the same issue with me.

https://github.com/Jinsung-L/install-macos-high-sierra

After the installation is done, you’re system will get rebooted. At this point it seems OK to turn off your file server and DNS server. But I wasn’t 100% sure so I just left it turned on.

After the reboot another serious problem occurred. I couldn’t unlock the disk using my password.

This problem seems quite common and I could find a working solution for my case.

Disk Password Error

I followed this instruction. https://forums.developer.apple.com/message/242694#242694

But with some modifications.

My ‘Macintosh HD’ disk wasn’t mounted on ‘/’ instead ‘/Volume/Macintosh HD’. Also the directory ‘/var/db/dslocal/nodes/Default’ didn’t exist in ‘/Volume/Macintosh HD’. But I could find the same files in '/Volume/Macintosh HD/Recovered Items/private'

So I tried

diskutil apfs updatePreboot -od /Volume/Macintosh HD/Recovered Items/private/var/db/dslocal/nodes/Default

and it worked. Don’t forget to put ‘\’ before spaces.

After you see the line “overall error = 0” you’re OK to reboot. When you’re system gets rebooted the login screen will be present with your account profiles. Enter your password, then it will begin to unlock your disk.

Then it gets rebooted again. This time it will continue the installation progress. It took about 30 m to be completed in my case.

After the installation, there’s one final reboot and then you’re in the new macOS High Sierra!

Thoughts

I’ve never imagined that this kind of thing would ever happen to me while just upgrading macOS. Unbelievable. Still don’t know what caused the problem but only guess is unstable internet connectivity. So never have unstable internet connectivity while upgrading your macOS. But who knew that I had one? My internet works fine for everything except ‘swcdn.apple.com’. Odd. Maybe it’s an issue from Apple’s server status. Who knows.

I’m very impressed of the system stability of macOS. I thought macOS is so stable that I would never ever strike to this kind of problem. Well… surprise! Never say never. I just wasted my 2 days.

Also I don’t understand why Apple let me do this. Apparently installation process uses http protocol which can be easily spoofed like what I did there. Maybe the installer is verifying the validity of installation files? But if not, you can basically hack someone’s Mac using this method if you have control of your local network. So don’t upgrade your macOS on public network like Work, School if you concern about hacking. Although I’m pretty sure there’s some kind of protection.

Reply

Answer 2

1313a

Level 1

9 points

Oct 10, 2017 11:43 PM in response to JinsungL

Specs:

Running Mountain Lion (freshly installed), not yet using FileVault
MBP Retina early 2015 (2.7Ghz, 8GB RAM)

Background:

Unable to download installation files for High Sierra using small downloader/installer from App Store
Always got stuck with the app unable to retry/continue; every new attempt started to download again and got always ended up the same:

Oct 8 20:37:05 MacBook-Pro osinstallersetupd[453]: Retrying http://swcdn.apple.com/content/downloads/30/55/091-36857/e08asjpjpbflt33p43ufqmu v6b39x8pa10/InstallESDDmg.pkg after 1 failure(s)

After trying different (faster) Internet connection and with the same failure, I went to download the files manually one by one and then served them locally to the downloader/installer
I did NOT have to go into the recovery console, all has been done from normal run of Mountain Lion
Now updated, but haven't yet tested everything. Follow only at your own risk!

What worked for me:

AppStore did download successfully the "Install High Sierra macOS"downloader/installation package (˜14MB)
When started, opened Log and after failure exported it
The Log pointed at the catalog (your results may vary, I was updating from Mountain Lion):

https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountain lion-lion-snowleopard-leopard.merged-1.s…

The Log also pointed at the specific catalog part with all the necessary files (in my case InstallESDDmg.pkg)
Looked up the rest of the critical files from the catalogue (according to the path ../30/55/091-36857/e08asjpjpbflt33p43ufqmuv6b39x8pa10/..) and ran cURL in Terminal (replace with the problematic files that you cannot download, for me it was the largest one):

cURL -O -C - http://swcdn.apple.com/content/downloads/30/55/091-36857/e08asjpjpbflt33p43ufqmu v6b39x8pa10/InstallESDDmg.pkg

Every time the speed dropped to 0, I cancelled the download (Ctrl+C) and started the same again, which (thanks to "-C -" parameters) resumed the download instead of downloading it again from scratch.
After getting all necessary files, I started simpleHTTPserver in my Downloads folder, by running the following command in Terminal

sudo python -m SimpleHTTPServer 80

In the Downloads folder, I created the same folder structure as in /content/downloads/30/55/091-36857/e08asjpjpbflt33p43ufqmuv6b39x8pa10/
Put all downloaded files into this folder, to be specific I downloaded all of the following into this folder

AppleDiagnostics.chunklist

AppleDiagnostics.dmg

BaseSystem.chunklist

BaseSystem.dmg

InstallAssistantAuto.smd

InstallESDDmg.pkg

InstallESDDmg.pkm

InstallInfo.plist

OSInstall.mpkg

RecoveryHDMetaDmg.pkg

RecoveryHDMetaDmg.pkm

Then changed hosts file to point at my local http server (add this line under 'su' user and make sure to save it properly

0.0.0.0 swcdn.apple.com

When all configured, started the small downloader/installation app "Install High Sierra macOS"
Because all files were hosted locally, I went fairly quickly to reboot and finishing the installation

So am now updated with High Sierra. Still have to check whether there aren't any issues during regular run.

Thanks to JinsungL for pointing me in the right direction. All it required was a bit of playing around 🙂. And I couldn't agree more, that Apple using http-only to download the necessary packages only implies possible security risks (not sure if the app itself doesn't check something inside the files, etc..).

Reply

Answer 3

Sep 27, 2017 5:05 AM in response to JinsungL

I have the same problem:

Sep 27 09:28:14 fs-Mac-Pro osinstallersetupd[727]: Retrying http://swcdn.apple.com/content/downloads/04/61/091-34298/almpfkbhyxnsgbxxqhoqo7s b40w3uip0wk/BaseSystem.dmg after 1 failure(s)

After that, there is no indication of anything happening.

Reply

Answer 4

Oct 6, 2017 8:07 AM in response to John Hall3

Apparently Apple recently had some problems with their cloud services including the Mac App Store. Possibly this was also affecting downloads of High Sierra and in this case the InstallESDDmg.pkg

According to Apple status page here Apple - Support - System Status those problems have now been resolved and therefore things should be back to normal.

However as Apple also use a CDN - Content Delivery Network sometimes that CDN is caching broken downloads for a while and therefore there may still be a delay before it propagates properly to you.

Reply

Answer 5

Oct 6, 2017 6:38 AM in response to JinsungL