App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

Reposting, as my typos have confused some people:

1- Do a clean, normal reboot.

2- Get and install What's Your Sign from https://objective-see.com/

1. 1. Full disclosure, I have no idea why this is required. However the thing it does when checking the signature info seems to have an impact on the next steps

3- In terminal, run

spctl --assess -v /Applications/YourApp.app

4- From a Finder window, under Applications, right-click (Ctrl-Click) on your app, and select "Signer Info"

Sure, re-signing the app with codesign may apply a new signature using some self-signed certificate. But I would not recommend this as a viable secure solution.

you're on the safe side, keeping a backup. I was troubleshooting on a Mac that I didn't mind breaking. If someone wants to come up with THE clean solution, here is what you need to do:

• use sqlite3 to edit this small SQLite DB
• Find and remove the serial number of the certificate "wrongfully" marked as revoked.

I don't have time for that, and I'm sure this CRL DB will get recreated in a clean way by trustd anyway...

arg! @Softmusic is right, the CRL file gets rebuilt and still marks the certificate as revoked.

So, Apple is pushing this CRL file to new High Sierra installs. But @cavenewt gave me an idea, which is now giving me some progress and some real strane results..

By default, with the file valid.sqlite3 in place, from the App Store using High Sierra, I can only see and download version 8.0 of MS Remote Desktop, and of course it fails to launch because of the revoked certificate.

Now if I delete valid.sqlite3, search again on the App Store for MS Remote Desktop, I can now see and download version 10.4.1. The install acted strange (had to try 2-3 times), but it eventually did. But the app is in its own "Microsoft Remote Desktop" folder in Applications.

and strangely, this new version of Remote Desktop does not embed certificates. Using --extract-certificates extracts no files.

So, assuming Apple has revoked the signing cert because of a legitimate reason, I now see two things happening:

1. 1. They have broken older apps and we can't run them anymore. Bad for users, good for security.
   2. They are not revoking new apps. Good for users, bad for security.

I am really at lost.

For now, I'm running with the newer app, which is only available if you delete valid.sqlite3.

So far so good, even after a reboot and waiting 30 minutes, the certificate serial number does not come back, hence the certificate is not marked as revoked.

So to recap.

1. Open a Terminal window, and run

sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'delete from serials where hex(serial) = "04CA81F77D5E33F7"'

What is left to figure out is why would Apple mark the "Apple Mac OS Application Signing" as revoked in a manually maintained CRL?

The thing is, complaining won't help. Getting to Apple Support and saying "I can't launch an app" won't get past Level 1 support. However, I hope that talking about code signing of apps and revocation of Apple's signing certifcate will hopefully get you past Level 1 as they may have no clue what we're talking about here.

Something is wrong with trustd

• It owns this valid.sqlite3 file
• It incorrectly marks the app signing cerificate as revoked
• It is in charge of verifying signatures
• Some apps are failing to install, could be related to signature
• I can't sign in to this discussion using Safari on the High Sierra mac I am having an issue because Safari complains about a TLS error

Hopefully your case with Apple will get transferred at a Level 2 / Level 3 support engineer. If so, point them to our discussion.

@softmusic: my bet is that when you hit Restart, trustd does not rebuild the DB, but rather write to disk what it has in memory. SQLite3 is a powerful DB that you can use in-memory and on-disk, and common sense is to write on disk what's in memory when you reboot.

And I have to stand corrected (again). What I thought was an incomplete DB yesterday (previous post), well this morning the DB shows version 146, and size was 1.6MB prior to reboot, and 5.8MB after reboot.

Has everyone noticed that when this issue started, the DB version was 145 and the issuer flag was 0. Now it is version 146.

Also, still playing around with Display Menu as my test app, Trashing it and trying to reinstall would always fail and ask me to use the Purchased page. Right-clicking on the app, and choosing "Hide purchase" forced the Apple Store to show me the "Get" button, which when clicked asked me for my iTunes credentials, and the install went fine.

So my "cleanest" workaround was

• Boot in Recovery of from USB to allow renaming /System/Library/Security/Certificates.bundle/Contents/Resources/valid.sqlite3 to /System/Library/Security/Certificates.bundle/Contents/Resources/valid.sqlite3.ori
• Reboot
• Let the system build you a clean /Library/Keychains/crls/valid.sqlite3
• Wait until you see version 146 (sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin")
• Reboot
• Open App Store to "Hide all purchases"
• Install your missing apps

I could also probably share the clean 5.8MB valid.sqlite3 file... let me see what I can do...

Tried every way I thought of but I can not seem to do this:

• "Wait until you see version 146 (sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin")

i.e. how do I see this?

My db is now running 17.7MB, BUT the few apps that didn't work, now work. Been several days, few reboots each day still holding. Even thought to have it run your automator app at boot time, but so far, so good.

Thought to go check out the app store, maybe get some freebie to see what was up. BUT as opposed to the iOS version, my desktop one is got only a few high profile paid apps, that is it (was trying to browse through it). Yeah, there is stuff there, you just can NOT browse for ANY of it. ***. So I looked at MacTracker, something I know works now and does show on my "purchased" page. Button said Install (when it should know I already acquired it)... so I did that, got all the indications of downloading, then button changed to Open. THAT button did nothing. HOWEVER, double clicking the app in the Finder did launch it. BUT thing is I AM operating with a db file that was "edited" by your automator!

@Johnno_uk made me remember one thing: Time Machine. ALL Mac users should always have that configured btw!

Let me share what seems to be my most "stable" fix for now.

1. Map your TM sparsebundle so you can browse it with Finder. Extract and copy those two files, for a backup done somewhere in October

/System/Library/Security/Certificates.bundle
/Library/Keychains/crls/valid.sqlite3

Certificate.bundle will be seen by Finder as a file, but it is in fact a folder. Anyway. Copy those on your desktop.

Boot from USB / Recovery. Use Terminal to replace those on your "Machintosh HD" volume.

Reboot.

In my case, valid.sqlite3 was up at version 137.

After reboot, the file did get upgraded to version 142, but without any corruption (no certs ended up being marked as revoked). I could launch installed apps, I was able to download and install Pages and Numbers (had to Unhide past purchases though and try 3-4 times before the install succeeded).

In the end, things seems to be "okay". Next step: see if I can run version 137 and turn off updates using the plist method.

So my assumpion is

• Update from version 42 to 146 gets the DB corrupted.
• Update from version 137 to 146 is "somewhat" ok.

And here is a bit more.

• The SN of "Apple Mac OS Application Signing" is present in version 42
• The SN of "Apple Mac OS Application Signing" is not present in version 137
• The SN of "Apple Mac OS Application Signing" is present in version 145 and 146 upgraded from version 42, with respective issuer flag of 0 and 16
• The SN of "Apple Mac OS Application Signing" is not present in in version 146 upgraded from version 137

So the assumpion of DB upgrade issues seems to be the most probable issue.

It would be nice if Apple could just push a complete copy of a "clean" DB rather than pushing upgrade scripts to it...

Thanks so much to everyone who has worked so hard on this and helped find solutions -- you are amazing!

I posted yesterday before I realized how extensive the thread was, and that there were some solutions (even if temporary) floating around. I was able to get my App Store apps working by replacing the valid.sqlite3 file with the older one, which was very exciting! As of today (day 2) the apps are still launching.

A few questions I'm wondering about:

1. If I've replaced the valid.sqlite3 file with the older version (as posted by softmusic on p.13 of this thread), is this just temporary -- will it go back to the way it was at some point? Also, will it cause any other problems on the computer?
2. Does it matter if the hard drive is formatted with HFS+ or APFS? I'm running it on an internal SSD, but I first installed High Sierra on it using a SATA/USB adapter, so it ended up with the HFS+ format. Hope this is okay.
3. So far I haven't installed the Security Updates from the App Store. With this solution, does it matter if I install those or not?

Many thanks again!

I can confirm that killing trustd (to prevent cache write to disk), deleting the DB, and rebooting does the trick as of today.

So, the steps

• Utilities --> Terminal, and run those commands

sudo killall -9 trustd
sudo rm -f /Library/Keychains/crls/valid.sqlite3

• Reboot
• Wait 5-10 minutes for the new version to be installed.

To confirm, after reboot, run this:

sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin"

db_version|5|
db_format|3|
db_source|0|valid.apple.com
version|147|
check_again|0|o??????A

version show show 147.

Hopefully this is the end of this saga.

Yes that's exactly what they are up to. They change all the updates including those from the past.

I took a full set of what they had when the end version was 146 and also tried to get a few indexed on the Wayback machine to prove this without it having to be my say so.

The difference this time is the digests are different. I haven't taken a full set yet but I think all the digests will be the same across the updates. I also need to see if the version number is included in the digest hash or not, the code shows this but I haven't looked at it yet.

What Apple may be trying to do is to reset every Mac to it's seed database and are cycling through digests that match the seeds of the different databases that come with different macOS version installations. We might see a new empty update with a different digest let's say every 2 weeks (to give time for machines to see them), cycling through different seed db digests.

Perhaps they were working their way back through the OS versions but have figured that going forward from High Sierra is a better approach given how High Sierra is susceptible to this problem here. I don't think the seed dbs change between point versions of the OS but I haven't looked as I don't have any old install images other than the final ones of each non current major version.

The problem with this theory is they could perhaps do this by sending a deliberately bad update which might be easier.

I don't have the full protocol including digest checks, database reseeding etc mapped out in my head yet as I've only really done a half-baked reading of the source code. Though it's all in there and possible to figure it out.

My other theory from yesterday was that non incremental updates have always been broken and the update generation code was applying these in a broken way to a database server side used only for this purpose (so not breaking the host machine in any way), and then sending out digests for the broken database. They may have fixed full updates at that end and so now the digests will be that of a correct database rather than a broken one. Though if all the digests are the same across increments that would be weird unless increments did nothing. This is why my replaying seed db digests every couple of weeks came to me as an idea of what might be going on this morning whilst in the shower.

Whatever they are up to I'm convinced they are trying to clean up a mess that is bigger than the problem here, but doing so without patching any code but rather being a little liberal with their own protocol to get to the end point they want.

Note that another way to reset that I spotted from the source code.

1. sudo touch /Library/Keychains/crls/.valid_replace
2. reboot

This will make trustd reseed the database for you.

It does look though that for the issue here in this thread that Apple have done what they needed to do. So further investigation will just be to get hints of what the bigger picture might have been and also just to understand the mechanism out of interest. Given what it is, this mechanism is quite important to the security of many computing devices in the world and really there should be people outside of Apple that understand it IMHO.

Okay, so I reinstalled the OS one more time and this is how I managed to get the apps to load up. When I got to the setup screen I skipped the internet connection setup, skipped the sing in to my Apple ID, entered the customize settings and disabled locations, disabled Siri and disabled the share analytics whit apple. All of the settings on there are disabled. Once I got to the Mac desktop, then I connected to the wifi, opend the App Store, loved in to my Apple ID, downloaded the apps and once downloaded, they opened up no problem.

Here I have Pages as proof. If I install the updates that are available on the App Store as it can be seen on the icon in the dock, as soon as the Mac has rebooted after the install, the apps stop working and give the now already known codesing error.

I really hope that Apple addresses this soon, because even if manageable to get the apps to work, it's not a stable way to even start work whit the apps because who knows when the apps will stop working again.