App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

"This is partly out of stubbornness as if the 2 incidents are related then I'll look like less of an idiot on the other forum.

"...Given that some Macs can't run Mojave and people reinstall their OS for many reasons, locking the users of such macs out of the App Store won't be an experience Apple will want to be in place even if these users are using very old Macs."

About that other forum – I was distressed at the response to this problem by a person who I thought was being unnecessarily dismissive (someone I've known of, and interacted with, for many years on yet a third forum.) I didn't think you looked like an idiot at all. Actually, he kept insisting that High Sierra is officially unsupported, but I couldn't confirm that. In its typical opaque way, Apple never really officially unsupports any OS; they just quit issuing updates for it, which is not the case with HS. So I'm very glad you started posting over here.

Of course at some point one has tell one's client that either they can't do what they want to do, or they're going to have to buy a new computer. We aren't to that point yet.

"...Note that I do appreciate that many users are not interested in a root cause analysis and just want their Macs to work."

I have a foot in both worlds – I just want my client's Mac to work, but I am interested in a root cause analysis even though a lot of it is beyond my depth.

Thanks for you reply. Actually things don't appear as bad as they did earlier with looks toward a fix. I downloaded all the updates from v0 to v146 (not anybody else doing this with wget and no special options note they'll be brotli compressed) and noticed that they are often about 5MB big. So it may be the case that a full update vs an incremental one would be no problem after all. Sometimes mobile carriers don't charge customers for certain traffic and probably Apple have made them do this for traffic to valid.apple.com which in many cases will be a CDN server in the carriers own network anyway. So on that score I think we can push Apple to send a full database, unless of course these databases are supposed to hold some data unique to each machine after all and sending a full database would end up deleting that stuff.

So going back to looking for the correction update that I called vY. bgrep makes light work of this

$ bgrep CE057691D730F89CA25E916F7335F4C8A15713DCD273A658C024023F8EB809C2 ~/Downloads/v*

I shows the v126 was the last update to talk about the Apple Worldwide Developer Relations Certification Authority by searching for it's SHA-256 fingerprint.

We can then open up v126 into its component plists and rerun bgrep. It shows that the first (00) and 6th (05) plists were the ones with the SHA-256 fingerprint in it. 00 is more interesting though and here it is.

This shows an update command asking for a row in the issuer_hash table matching SHA-256 fingerprint of Apple Worldwide Developer Relations Certification Authority to be removed I believe. That also means to cascade down through the hierarchy and remove certificates (e.g stop them being considered revoked) signed by this CA and that would include our friend with serial 04CA81F77D5E33F7 Apple Mac OS Application Signing.

I bet if I look at v125 it will be what I called vX above. The blobs at the end of the plists might have a date in them, I can't remember and if there is one in there I bet it will be Nov 12 2020.

I did try the Wayback machine just in case but they don't appear to index these files.

So timestamps are optional and Apple don't put them into their signatures of these updates so it will be pretty hard to say exactly when they signed v126 before sending it on its way. Luckily I found a simple tool for windows that would show the signature for me. I'm sure there is one for the Mac too but this was the quickest route to see what was inside.

Ignore the fact it says the signature is invalid, I didn't give it the data that was signed as input.

I've just spotted an error in the file downloaded as v126. It reports inside that it is v146. This may well have caused havoc.

The near empty version with a url of /g3/v146 might be some kind of fix. So the updates coming from Apple have been quite mucked up.

The one in /System/Library/Security/Certificates.bundle shouldn't really be touched it's what was current at the time the OS was built and is supposed to get updated to a later version. I'd leave that one alone as it's part of the OS itself for all intents and purposes.

Applying the security patch 006 should be OK as I've done it a few times now but yes, backups are always a good idea.

Copying your DB to your client's machine should be fine, if you trust that your client won't start digging into it. Though I suspect that they wouldn't be employing you in the first place if they were the sort of person that would do that. We haven't ruled out if these databases could in some way perhaps expose past user activity. I don't think they do but can't say 100% they don't.

I've written up a summary of the whole thing here, and I haven't included hex numbers or commands in it just human readable names of things

https://drive.google.com/file/d/1Ngyty19xiUYkqCXbrgvcMocnUqqm-UtS/view

You could probably read that against what I've posted here and join the dots as in here I've posted how I got all of this information. I could probably write up a more detailed version of that PDF but I think this is enough for you to explain to your client that it's very likely that Apple had problems and that they'll likely still working out how to fix them.

No worries. The client is a close friend and would have no clue about any of this stuff. She just wants her Pages to open.

What I worry about more is people finding random solutions via an Internet search and blindly following terminal or other technical instructions, a situation with which I have years of experience. Job security for people like me!

Hello, i have the same problem, when I install a fresh High Sierra OS in a SSD. But I have an HDD with High Sierra upgraded from Sierra and I don't have this problem. Sadly, the command only works few hours

Here's all that above as picture which explains why this is happening on fresh High Sierra installations.

There is a bug which isn't understood but it looks like it was mitigated by Apple in the past and the mitigations are currently being avoided for another reason which again we don't know yet.

guillermo316 wrote:

Hello, i have the same problem, when I install a fresh High Sierra OS in a SSD. But I have an HDD with High Sierra upgraded from Sierra and I don't have this problem. Sadly, the command only works few hours

On your High Sierra on the spinner that is working, can you confirm that you downloaded that High Sierra installer after November 12?

HFS+ / APFS, HDD or SDD, 2020-004, 2020-05 or 2020-6, all of this won't make a difference and the issue will be present no matter what.

Any workarounds posted here are temporary and meant to keep your installation secure at the same time.

I can confirm all my apps run and the update page in the App Store is showing the install status correctly. Note that I resisted touching the db since reinstalling HS 2 weeks ago off my trusty USB installer. I thought I’d let Apple do their thing. Definitely shows as 147.

Oh yeah I forgot about them.

It does not look like all of those issuer hashes are deleted through trustd after all. Perhaps only those without anything to say at the levels below are removed. 232 appear to stick around. I think trying to mimic the update that Apple would do may be more trouble than it's worth and could compromise security TBH.

macos-10:~ johnno$ sqlite3 /Library/Keychains/crls/valid.sqlite3 < Desktop/issuer_hashes.txt | wc -l

231

macos-10:~ johnno$ sqlite3 /Volumes/macOS\ Mojave/Library/Keychains/crls/valid.sqlite3 < Desktop/issuer_hashes.txt | wc -l

232

macos-10:~ johnno$ sqlite3 /Volumes/macOS\ Catalina\ -\ Data/Library/Keychains/crls/valid.sqlite3 < Desktop/issuer_hashes.txt | wc -l

232

macos-10:~ johnno$ sqlite3 /Volumes/macOS\ Big\ Sur\ -\ Data/private/var/protected/trustd/valid.sqlite3 < Desktop/issuer_hashes.txt | wc -l

232

Also 147 is another empty update like 146 was (at least when I download it via Chrome it is and I presume it doesn't give different versions based on user agent). Last time when 146 came out people reported things working for a while and then we reverted back. I also checked 43 again and reports as 147 now. I thought that Apple might have kept the run from 43-46 in this changeset.

I still don't think this is the final fix from Apple but perhaps just a kick, perhaps necessary for another reason.

Likely the same reason why all the data is not following protocol (according to my understanding/expectation of the protocol).

I don't understand why an empty update would make things work for a while other than perhaps clearing a cache layer above or something silly like that.

I'm willing to be wrong, but I think the problem will come back ...

Pleasant surprise! ...

Right now I have downloaded apps that I had bought from the App Store and having updated my iMac (mid 2010) to High Sierra (V.10.13.6) they have started working as usual.

Apple has fixed it without the user having to do anything! ...

Thanks to all of you who have worked to offer us solutions outside of Apple.

Cheers

It may have been overkill, but I followed your instructions to the letter. Saw that the valid.sqlite3 HAD been deleted, on reboot it was there so I now have a file that has not been edited and apps seems to run.

While many of your's & johnno's posts were kinda hard to follow, I DO appreciate that you took the time to dig into this! My expectation is that this is finally put to bed.

My working, pre-existing (July) install of High Sierra is still showing v42 for /Library/Keychains/crls/valid.sqlite3. I'll give it a few days and see if it updates to 147. I could always delete it (I have backups) and see if the resulting rebuild works – when I did that before, apps were broken.

And I'm with you, I'm also very curious whether Apple paid any attention to users' investigations or not. Of course we will probably never know.