App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

While I admire the tenacity of all of you, spending a crapton of time digging into the internals to solve some issue that 100% was caused by some obviously sloppy work on Cupertino's part, it is still on them to really fix the issue. The fact that so far it has been ignored doesn't bode very well for the future... do they really think I am NOT going to hesitate for a while when contemplating ANY further purchases? That being said, I said way earlier that I tried directly contacting customer support, hoping to get escalated to some tier 3 support, which my guess is may escalate SOMETHING inside the mothership (plus I find it hard to believe they aren't already fully aware of this). So far, I get the typical delaying kind of responses, as in the "tell me this" I do they then says tell me that. I think it was better than a week that I didn't hear anything so I called and asked for a call back. Few days pass, I kinda forget that I was expecting a call. THEN I get an email saying they scheduled a call back for today. BUT the very next email says that it was off because I canceled it! BUT there was a link to "re-open" it so I did (with a few choice words and not swearing or bad language). Today I get an email from support (a different person) AND a new call schedule that was NOT canceled right away. So we shall see what happens.

Oh, the framing of this was "I can not use the app store to buy any software" rather than some talking about code signing errors or revocation of certs. While at least one old app has the issue, I AM thankful Mail continues to work, I'd be almost on a plane to Cupertino to find Cook to give him a piece of my mind!

Riverside_Guy wrote:

My route was updating a Sierra installation... which by definition seems to be from what we used to call a full installer. EXCEPT even what they were making available a little less than a week go was NOT a combo... because I HAD to separately install security 005 AND 006!

By "full installer", I simply meant a High Sierra installer recently downloaded via the App Store. I've given up any other kind of downloads directly from Apple (I.e. not via the App Store). I used to carry around a hard drive with about 10 different OS combo installers. Once they started disabling those because of expiration dates, I gave up on that. The whole idea of a combo installer doesn't really matter anymore.

The recent security updates 005 and 006 do not seem to introduce the problem by themselves.

softmusic wrote:

I would very happy to know if other people can reproduce the problem on a High Sierra installed before Novembre 11th.

To reproduce the problem quit every apps, launch terminal and type :

sudo killall -9 trustd; sudo rm /Library/Keychains/crls/valid.sqlite3

Then reboot and wait a few minutes before trying to launch apps from the Mac App Store.

My High Sierra was installed in July, and I can't recall if I got it from Apple downloads or the App Store. It's been working fine. Before doing your experiment I made a copy of /Library/Keychains/crls/valid.sqlite3, whose size is 6.4mb. Before doing the reboot, I simply opened that folder in the Finder and watched. It took two minutes to re-create the file and the size was 15.5mb. I did not try opening any applications.

Then I deleted the file again using the terminal command (retaining my backup copy of course), rebooted, and watched the finder window. There was an interesting sequence of events:

I did not get screen shots of every one, but over the course of a couple

of minutes that journal file would disappear and then reappear with

various sizes, some kb as seen, some up to 15mb. Finally both of the temp documents disappeared leaving a

rebuilt database of 17.3mb:

Now my July High Sierra is broken. While I can successfully launch a couple of App Store apps — a Devonthink utility, a couple of other utilities — most fail with a codesigning error, notably Pages, Numbers, and the usual suspects.

Next step is to restore the backup database and see if things are back to normal.

I restored my original 6.4mb database and rebooted. Worryingly, that "update-current" text file reappeared but only for a second and, to my relief, the database remains at 6.4mb and applications are launching. 

This leads me to think that replacing the larger databases with a pre-existing smaller one is the best workaround. After all, my July High Sierra install has maintained that 6.4mb database all along and it's working. Heaven knows what the larger database has in it that breaks things, or what Apple changed to make the database rebuild itself so much larger and broken. Apparently it's not something just in recent High Sierra installers since my July HS install created a broken database to replace a deleted one.

It's really worrying that Apple has not even acknowledged this problem, much less provided a fix.

Incidentally, here is my exact HS version as reported in the System Report:

macOS 10.13.6 (17G14033)

Small correction. 16 is 1<<4, not <<5.

And <<4 as per the source, is kSecValidInfoAllowlist, which is described as "set if this entry describes valid certs (i.e. is allowed)"

So it does make sense to update the value to 16.

This allows me to launch the apps correctly, but after Trashing "Display Menu", the Apple Store still thinks it is installed and won't allow me to reinstall it.

FrancoisQC wrote:

And it looks like someone found a bit more, documented at https://eclecticlight.co/2020/11/25/macos-has-checked-app-signatures-online-for-over-2-years/. Not only have Jhonno found the same revoked certificate, but he also found another one:

CN: Mac App Store and iTunes Store Receipt Signing

Serial: 0x0EEB5787E79E098D.

...

I need to run some tests from fresh, see if that would be the best workaround. But I expect this change to be reverted back after a while, like we have seen happening so far.

The comments in that linked blog post are very interesting. Johnno indeed reports that the change is only temporary.

I learned one interesting thing which you guys probably already knew: CRL stands for certificate revocation list. Makes sense.

FrancoisQC wrote:

This is what I have been using:

sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 "update groups set flags=16 where groupid = (select groupid from serials where hex(serial) = \"04CA81F77D5E33F7\")"

The change did survive a reboot, but for how long? And it did allow me to reinstall (and launch!!) Display Menu.

So I would recommend this update over my previous suggestion of deleting the row.

Kudos go to @softmusic !

With further testing, my solution n°1 updating the flag does not seem to work as good as solution n°2 deleting all the serials from this group.

In a very particular case :

I have acquired BBEdit on Oct. 2019 with my account A and try to install BBEdit on account B who is using connected to the App Store with his Apple ID on High Sierra but never acquired BBEdit.

B can not install BBEdit from BBEdit App Store because it needs macOS 10.14.

B can install BBEDit though A's purchased items because A & B are in the same family.

With solution n°1, I got a "Download Error" very often. I have to try many times to succeed.

With solution n°2, it downloads directly.

I don't know what happens, App Store seems to be very unreliable and it's very painful to test.

On this screen capture I have hidden other apps information and kept only useful information.

It tells me to use "Purchases" page and retry when I am on "Purchases" page !

I have to close and reopen App Store to retry.

Well, looking at the App Store, BBEdit shows it can now only be installed on macOS 10.14.2. So say goodbye to new installs of BBEdit on High Sierra, unless maybe they already show up in "Purchased" and HS allows you to download the same older version still compatible with HS.

Note: I had missed the updates from softmusic@ that indeed the CRL database was storing certificates as revoked. This was something I'd looked for independently before finding this thread and had convinced myself, with some help in another forum that although there was an erroneous certificate revocation status, it was not being stored in those particular databases.

I'd also looked at this http://developer.apple.com/certificationauthority/wwdrca.crl (plus the others) as the expected place to find genuine revocations as that would be where I'd expect to find them as that's the authority above the certs being seen as revoked.

softmusic@ appears to be much further ahead with a solution than I am, and it's always best to follow just one person.

Often applying all suggested fixes in scenarios like this can create more mess or confusion.

It stands to reason that getting databases etc into the same/similar state as installations prior to Nov 11 would be a better fix than what' I'm doing which is constantly forcing an online check of status to update them (or caches of them) with correct info, as that creates one thing fighting with another in the system and chaos is never really a good solution to a problem.

From scanning the thread it's hard to tell if somebody has found why certs are being marked locally as revoked. I think the main protocols involved (fetching the big CRL or using OCSP on a cert by cert basis) are all doing things correctly as if they were not (or I missed where somebody here has found this data being received from the net), they'd be knocking out a lot more systems and also the temporary fix of validating signing (which results in an outbound OCSP from trustd when that data is not cached) wouldn't work either.

My suspicions are that all this in general might have something to do with certificate pinning, perhaps to older versions of certificates. Perhaps existing installations re-pinned themselves somehow but new ones get certificates pinned back as if its still 2016 or something really daft like that. When databases start to get new rows in them from out of the blue, then out of the blue will either be online or another source locally. I guess leaving a machine or VM running offline for a long while might help prove things either way. My suspicion is that the smoking gun will be some kind of post install patch, downloaded only once and that the erroneous data used to rebuild the CRL dbs will be sitting on people's machines and getting re-used every now and again.

Looking at where everyone is here with debugging, there is clearly enough info for an engineer at Apple to work with and I suspect they may provide a patch. If not a simple fixing tool could be written an posted on github by the community.

I'm 2 weeks late to the party, and have in effect been having my own fixing party all by myself for a while, so apologies if I've added to confusion here.

The revocation source here isn't normal CRLs or OCSP I think it's a set of files on CMS (Content Management System) servers on the Akami network with the name of valid.apple.com. It looks like a hand rolled attempt of certificate revocation.

It looks like you'll need to extract out a client certificate to be able to talk to those machines but on there somewhere I bet we'll find the rouge data set. So the fix from Apple may just be to update some files on their CMS.

Of course we don't know how close to the version of trustd on High Sierra this source is, so it's quite easy to jump to conclusions here. I'm very good at doing that.

Note - 0 is no flags at all not kSecValidInfoComplete which is 0x1 so that could be the problem.

Note - 0 is no flags at all not kSecValidInfoComplete which is 0x1 so that could be the problem.

You're right. My bit shifting is definitely rusty...

So the flag for the CA goes from 16 (kSecValidInfoAllowlist) to 0, which is either nothing, or undefined, or all good, or unset, or a defect.

I had noticed that "valid.apple.com" in the admin table. There seems to be a replying server at that FQDN. But I agree, HS is pulling that file "from somewhere" and this "somewhere" has incorrect data. Or trustd alters the data incorrectly.

I left a VM non upgraded stuck at 17G66 without any patches and it still went wrong so my gut says this bug has been there for a while rather than in those patches. The exact trigger and code error may never be known, nor for sure if there really is a bug with code or data I'd think.

Tried every way I thought of but I can not seem to do this:

• "Wait until you see version 146 (sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin")

i.e. how do I see this?

My db is now running 17.7MB, BUT the few apps that didn't work, now work. Been several days, few reboots each day still holding. Even thought to have it run your automator app at boot time, but so far, so good.

Thought to go check out the app store, maybe get some freebie to see what was up. BUT as opposed to the iOS version, my desktop one is got only a few high profile paid apps, that is it (was trying to browse through it). Yeah, there is stuff there, you just can NOT browse for ANY of it. ***. So I looked at MacTracker, something I know works now and does show on my "purchased" page. Button said Install (when it should know I already acquired it)... so I did that, got all the indications of downloading, then button changed to Open. THAT button did nothing. HOWEVER, double clicking the app in the Finder did launch it. BUT thing is I AM operating with a db file that was "edited" by your automator!

FordhamAl wrote:

i had the exact same problem, i looked around and found this advice on another post:

cmd shift + G

/library/keychains/crls

delete "Valid.sqlite3"

worked for me, idk if it is fix for everyone but give it a try

Unless Apple has changed something, that fix is only temporary at best.