App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

Note: I did get a very key part of how the protocol works significantly wrong.

What is put at http://valid.apple.com/g3/vX is not version X but a complete update to get a machine that from version X right to the current version in one hit. So it's the delta from X to T(oday).

So this explains that it is normal for them all to include version T inside of them and to have the same digests and all be replaced when T moves on. It's also a much more sensible protocol to avoid out of date machines having to cycle through versions, which did strike me as weird but I didn't have the code loaded into an IDE and was just looking at it on the web making jumping between files a pain. Tonight I loaded a later version of XCode into a Catalina VM just to navigate it better and saw my dumb mistake.

So the bug is pretty simple. For a while what was at http://valid.apple.com/g3/v42 would have been bad. New High Sierra installs would be the only Macs that were on 42 as that was the seed (or snapshot) as it's called in the code.

I still have a copy of that from before so looking at the difference between the v42->v146 vs the v42->v147 update will show what's going on.

What's your "seed database" version? Open a Terminal and run this:

sqlite3 /System/Library/Security/Certificates.bundle/Contents/Resources/valid.sqlite3  "select * from admin"

What's your current DB version? Run those two commands:

ls -l /Library/Keychains/crls/valid.sqlite3
sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin"

Thank you for the help! I have been able to fix it by going into safe mode like BDAqua suggested. The download worked and I am now able to launch any apps. Just a quick question, is it safe to update to the latest version of High Sierra now? I've been staying away until now.

Thanks again for the help!

I did another clean install of High Sierra and all of my App Store apps worked fine again. Then I installed Security Update 2020-005 10.13.6 and they stopped working, so the security update may be the culprit. Though that update was released September 24, so it may be an interaction between the update and changes to Apple’s codesigning process.

Progress made. But may have broken something else, still investigating.

The same issue was reported on 1Password community site, and one suggestion that has been posted actually got MS Remote Desktop app to "magically" start working. I say magically because the cause of things starting to work again is still to be narrowed down.

I installed the "What's Your Sign" app from https://objective-see.com/, so to see if the issue would also get resolved the same way, and it did for me.

1. Install it, follow the prompts.
2. Right click on your app, click "Signing Info"
   1. 1st time, it will show the same "Unavailable" signer info.
   2. Do it a second time, it will now show all 3 CAs!

Once the above has been done, you may be able to launch your app.

WARNING: I don't know what this 3rd party app does, and once I tried to log in to this Apple Discussion site, Safari would not connect anymore to the Apple auth server. I don't know if this is because HS did not have the latest Security Patch (hence maybe supporting stronger TLS than what this site offers), or if this is related to this 3rd party app breaking something else.

I also did notice a new entry in KeyChain being created at around the same time I installed that 3rd party app. The entry name showed "<unknown>", and looking at the details of the entry it was related to com.apple.LaunchServices.TrustedSignature.

I'll have to see if rebuilding the Launch Service Database could be a safer workaround that using this 3rd party app, for which the source code is available. Next in line: source code review...

We're probably in autopsy territory now, but I wanted to add a data point.

In addition to launch failure, my problem system would not download any iWork apps via the App Store. It would offer me the older compatible versions but then threw up an error message before the actual download started. So I ran What's Your Sign on the App Store app itself (This was in Safe Boot mode, but that probably doesn't matter). Now the downloads are working fine.

Not too fast ! Sadly, it only works the time the system is rebuilding the file.

If you run the command

sudo rm -f /Library/Keychains/crls/valid.sqlite3

Without killing trustd, you get en empty file.

-rw-r--r--  1 root  wheel  0 26 nov 16:01 /Library/Keychains/crls/valid.sqlite3

That's why it let apps launch

When you reboot, just wait a few minutes and let the database being filled.

Then no app will launch.

The good command to clear the cache is from https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution/resolving_common_notarization_issues :

sudo killall -9 trustd; sudo rm /Library/Keychains/crls/valid.sqlite3

If you launch one app, trustd restarts and begins to fill the database.

Size of the file when database is full :

-rw-r--r--  1 root  wheel  17293312 26 nov 16:08 /Library/Keychains/crls/valid.sqlite3

When it is full, no app will launch.

So, I am still waiting for a better solution.

The thing is, complaining won't help. Getting to Apple Support and saying "I can't launch an app" won't get past Level 1 support. However, I hope that talking about code signing of apps and revocation of Apple's signing certifcate will hopefully get you past Level 1 as they may have no clue what we're talking about here.

Something is wrong with trustd

• It owns this valid.sqlite3 file
• It incorrectly marks the app signing cerificate as revoked
• It is in charge of verifying signatures
• Some apps are failing to install, could be related to signature
• I can't sign in to this discussion using Safari on the High Sierra mac I am having an issue because Safari complains about a TLS error

Hopefully your case with Apple will get transferred at a Level 2 / Level 3 support engineer. If so, point them to our discussion.

Would this be a credible summary of the problem: Unlike previously existing installs of High Sierra, High Sierra full installers (but not updaters) since November 12 have a problem with trustd.

 We have several workarounds but they only address a symptom of the problem, the database in /Library/Keychains/crls/. There are other problems involving downloading, installing, and status ("install" vs "open") of apps in the App Store, and possible problems logging into Apple discussions in Safari.

An alternative description might involve the recent issues with other versions of OS X, which may point to the problem being much larger than just a High Sierra installer—something more widespread with Apple's validation system, which is way over my pay grade.

So I had my "conversation" with the mothership yesterday. Got to talk to a "specialist" who at least seemed to have a good idea how things function (he seemed to know what I was asking about. The bad news is that there is NO "know" action being discussed or taken regarding this issue. All I got was the fresh install nonsense, we pretty much KNOW that is not going to work. Yes, mention was made about old, vintage operating systems and even older hardware, I tried to explain they DID do an update and THAT was what broke things. I stressed to him he NEEDS to read through this thread, AND escalate the issue through his chain of command. The only option he really gave me was to file a "complaint." At least he did that for me, so I now I have a registered complaint case number.

He had me do a safe boot BUT was not able to answer the question about startup items loading (by this point, I doubt he was tier 3, AND he seemed to have no intention of escalating me there) on safe boot, just that it takes it's time because it runs first aid before loading. My mistake was not first deleting the edited db file before running this process, BUT my expectation was it wouldn't have made any difference.

I tried to remind him the issue was that can NOT use the app store to d/l any new software until they address the issue. Like I said before, I am kinda lucky in that MOST everything I use is not affected. I CAN live without a menubar temp or being able to run BlackMagic. While I visit the store a fair amount on my phone and tablet, I almost never do on my desktop. AND it has zero affect on my running win10 better than most pcs do.

Looks like we have 2 options here... using the automator script to edit the db or installing an older version of it. Frankly I am not so sure the "older version" is as long lasting as some say, there may be events that cause it to be replaced in some fashion. THAT also might be the cause for the "edit the file" approach would fail. Frankly, I am sticking to the "edit the file" approach. It's been 2 days and several boots and it is still "working." We MAY get a Xmas present and have the mothership FIX the issue THEY created, but I have zero hope they will ever do it... they are hyper focused on acquiring new customers and getting old ones to buy new shizz, old customers using old OSes on old machines can be left to rot, won't change their bottom line and THAT is all they care about.

The URL path is from https://opensource.apple.com/source/Security/Security-59306.140.5/trust/trustd/SecRevocationNetworking.m.auto.html

It's harder to see in the html version as it mangles the printf format string:

"https://%@/g3/v%ld", host, version

I downloaded the full tarballs of the source as it was easier to work with local files.

It will be worth splitting up v145 into it's plist components to take a look. The original source linked explains the binary format but all you really need to know if that if split by the magic of 'bplist' then the 4 bytes prior say how long that plist is, and just extract that much. The 4 bytes prior to the first bplist and its size marker (e.g 8 bytes away form the first) is how many plists to expect.

My money is still on a software bug and if it was described generically I'd call it a graph node parenting issue. Serial numbers are not unique at all but only unique within the scope of the issuer, so they are in effect meaningless without a link to the issuer. So the DB structure is

dates <- group -> issuers

|

V

serials

I think the terms are little mixed up and really the better way to see it in your head would be to rename things as:

dates <- issuing authority -> issuer CA certs

|

V

serials

That's the only way that would make sense if serials are non unique across issuing authorities (or groups they are called in the schema).

The updates coming down from valid.apple.com appear to prefer using serials + issuing authority to identify the certs rather than a SHA. So I think if we delve into a db copy of v42 we'll find the old WWDC there in a group. Then when it goes to v145 something in v145 may be saying that old issuing CA cert is done and dusted but it's being interpreted incorrectly as to kill off anything from that authority, which would include everything signed off with the newer CA cert.

I'm guessing that older installations will have already done the clearing out of stuff signed with the old cert on a prior update, so there is nothing to do on that score when they get v145 and therefore don't make a mess.

Riverside_Guy wrote:

"Wait until you see version 146• (sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin")

i.e. how do I see this?

Open a Terminal, and run

sqlite3 /Library/Keychains/crls/valid.sqlite3 "select *  from admin"

...

I AM operating with a db file that was "edited" by your automator

With everything found so far, I would not recommend my script anymore. Not that it breaks something, but more that it does not address the realy issue. The automator script I created deletes the serial number of one certificate, so that since trustd does not find it in its disk cache, it somewhat assumes it isi valid, or does some other checks.

But I created that script while we were all staring at version 145 which was marking the issuer of that certificate with Flag 0.

Now in version 146, the issuer Flag is 16, which makes more sense and trustd properly launches apps which could not be launched with a default version 145.

That leads me to think that Apple is slowly pushing some updates, but they have not fixed everything.

And there is also the problem of an "updated version 146" which seems to be around 17MB, and a "fresh from scratch version 146" which is 5.8MB.

Try disconnecting from the Internet & see if they open.

My current build is 17G14042... my bet is you have yet to do the 006 security update?