App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

I upgraded my iMac 27-inch Mid-2011 with a 240-GB SSD alongside the existing 2-TB HDD. After the upgrade, I tested and everything worked great using the existing High Sierra system on the 2-TB HDD.


I did a new install of macOS High Sierra from Apple over the internet on to the 240-GB SSD. It was a fast and error-free installation. The iMac started up fast from the SSD and I applied system updates.


I completed all configuration tasks (setup all my preferences and installed all my applications and utilities) and then I ran it like this for a couple of days trying things out.


When it was time to add my App Store apps I ran into a snag. All my purchased apps install but will not run. When I launch them they either generate an error report, or bounce once in the dock, or do nothing at all.


I also tried adding new free apps that were not attached to my account and they behave the same way. So, I did some troubleshooting things like trying a different login and playing with permissions. Nothing works.


I spent an hour today on a support call with Apple and they were not able to figure it out and recommended I take it to for Apple authorized service.


I gave up and reinstalled the operating system on the HDD and to my surprise the same issue is happening on the original drive; that was working just fine!


Has anyone else experienced this? Can I not have two internal drives?

iMac 27″, macOS 10.13

Posted on Nov 13, 2020 11:38 PM

Reply
Question marked as Top-ranking reply

Posted on Nov 26, 2020 9:22 AM

If you delete valid.sqlite3 from an old working High Sierra, app will not launch.

Indeed, valid.sqlite3 from the old working High is much smaller.


New file :

-rw-r--r--  1 root  wheel  17293312 26 nov 16:16 /Library/Keychains/crls/valid.sqlite3


Old file :

-rw-r--r--  1 root  wheel  7794688 26 nov 18:06 /Library/Keychains/crls/valid.sqlite3


So I replaced the new valid.sqlite3 from the new High Sierra with the old valid.sqlite3 from the old High Sierra.

sudo killall -9 trustd; sudo cp /path_to_old_file/valid.sqlite3 /Library/Keychains/crls/valid.sqlite3

It seems to work even after reboot.

Old file, just unzip before copying :

http://www.mediafire.com/file/m2ky3mrnon6jy49/valid.sqlite3.zip/file

Similar questions

447 replies
Question marked as Top-ranking reply

Nov 26, 2020 9:22 AM in response to FrancoisQC

If you delete valid.sqlite3 from an old working High Sierra, app will not launch.

Indeed, valid.sqlite3 from the old working High is much smaller.


New file :

-rw-r--r--  1 root  wheel  17293312 26 nov 16:16 /Library/Keychains/crls/valid.sqlite3


Old file :

-rw-r--r--  1 root  wheel  7794688 26 nov 18:06 /Library/Keychains/crls/valid.sqlite3


So I replaced the new valid.sqlite3 from the new High Sierra with the old valid.sqlite3 from the old High Sierra.

sudo killall -9 trustd; sudo cp /path_to_old_file/valid.sqlite3 /Library/Keychains/crls/valid.sqlite3

It seems to work even after reboot.

Old file, just unzip before copying :

http://www.mediafire.com/file/m2ky3mrnon6jy49/valid.sqlite3.zip/file

Nov 26, 2020 10:05 AM in response to FrancoisQC

So far so good, even after a reboot and waiting 30 minutes, the certificate serial number does not come back, hence the certificate is not marked as revoked.


So to recap.

  1. Open a Terminal window, and run
sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'delete from serials where hex(serial) = "04CA81F77D5E33F7"'


What is left to figure out is why would Apple mark the "Apple Mac OS Application Signing" as revoked in a manually maintained CRL?

Nov 29, 2020 6:01 PM in response to GrenadeBait

Status Report - We have been running different Macs for a few days with different solutions. This is what we've been seeing...


  1. The fairly effective solution that edits the valid.sqlite3 file seems to be relatively reliable but the application state is not accurately reflected in the App Store;
  2. The What's Your Sign solution that uses the spctl --assess -v /Applications/*app" command; and 
  3. The sudo codesign —deep -fs - /Applications/*.app solutions, both of which don't work with all apps and the App Store isn't synced to the applications state (install/installed/open).
  4. Replacing the new larger valid.sqlite3 database with a smaller one that predates the November 11th timeframe.


It's been a few days now and the only solution that is still working without requiring periodic intervention is the third; ie. replacing the valid.sqlite3 with an old copy. The positive behaviours of this method are that:

  • apps reflect their installation status correctly in the App Store;
  • apps that are not compatible with High Sierra offer the older version for download, with the exception of xCode;
  • all apps tested work without error (we test ~ 40-50 per computer); and
  • the valid.sqlite3 database is updating every six minutes.


We're going to migrate the rest of our computers to the old valid.sqlite3 database tomorrow and run with that until a proper Apple fix is available or the machines die, whichever comes first (with the latter probably being the case :)


I would recommend that we promote the use of the two most reliable or simple solutions:

  1. using an old database (reliable), or
  2. using a script edited database (easy).

Dec 3, 2020 8:40 AM in response to GrenadeBait

Adding my two cents:


Had the exact same issue. Mid 2011 MacBook Air, reset it as it was getting hogged, and updated to the latest possible OS: High Sierra 10.13.6. After all the updates, none of the apps downloaded from App Store launched, rendering the machine effectively useless.


Fixed it by doing another clean install, and not installing the 2020-05 security update. All App Store downloads now work correctly. Just have to figure out now how to disable the update notifications.

Dec 9, 2020 12:14 PM in response to johnno_uk

johnno_uk wrote:

Note that another way to reset that I spotted from the source code.
1. sudo touch /Library/Keychains/crls/.valid_replace
2. reboot

This will make trustd reseed the database for you.

It does look though that for the issue here in this thread that Apple have done what they needed to do. So further investigation will just be to get hints of what the bigger picture might have been and also just to understand the mechanism out of interest. Given what it is, this mechanism is quite important to the security of many computing devices in the world and really there should be people outside of Apple that understand it IMHO.


BIG THANKS to you johnno_uk for having explained here what was going on. It is very reassuring to have this information when Apple support does not give answers.


I used your command with seems to me the cleanest way to rebuild the database.


I still have some problems with the App Store when I want to install an old version of BBEdit that I bought before. I just tested it and I had to retry 5 or 8 times before being able to download it from another account in the family. But is a very small issue and I will not expect Apple to fix it.


Nov 17, 2020 12:10 PM in response to GrenadeBait

A lot of us are having the same issue, and it started the day when big sur was released. I tried a lot of things, and the only solution is to do a clean reinstall of High Sierra 10.13.6 and do not install the security updates. Once the security updates are installed, no app from the App Store is going to open... So there is a problem whit the updates. Either apple wants to force us to upgrade, even if for some of us that is not possible because of the hardware that is used, or they messed something up big time, because on Mojave there are no issues after the update, only on High Siera. The update was released for High Sierra, Mojave and Catalina, but only HS has that problem.

Nov 26, 2020 7:57 PM in response to GrenadeBait

Reference the "valid.sqlite3" database edit solution from FrancoisQC. The following testing was conducted.


On three different Macs, a clean install of High Sierra was completed from the option-command-R recovery mode (ie. over the internet from Apple). No Apple Updates were applied.


Applied terminal window command string...


sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'delete from serials where hex(serial) = "04CA81F77D5E33F7"'


Our testing finds that the valid.sqlite3 edit...

...works with an ~85% effectiveness (see below);

...survives the periodic ~60-min update from Apple;

...survives a computer reboot; and

...as of this post, it has survived 5 hours.

We're crossing our fingers :-)


As summarised above, we found it about 85% effective based on randomly selected groups of 40-45 App Store apps of which 5-6 usually fail to start, with the remainder starting without error. Note app functions were not tested, but they look OK.


The apps tested varied widely from big devs like Apple and Microsoft as well as the smaller guys, and included productivity, utilities, games, etc..


No codesign errors were reported for any app, not even the ones that failed.


Example of apps that worked...

AdGuard for Safari

Aperture

Better Rename 9

Blackmagic Disk Speed Test

Diagrammix

European War 2

European War 3

European War 4

Folders Factory

GarageBand

HP Easy Scan

iBooks Author

iMovie

iNet Network Scanner

inShort

iStat Mini

Keynote

Mactracker

Magnet

Microsoft OneNote

Microsoft Remote Desktop

Microsoft Word

Numbers

OneDrive

Pages

Parcel

PDF Toolkit+

ScreenSharingMenulet

SketchBook

Smart Converter

Speedtest

Tag Editor Free

TextWrangler

The Archive Browser

TrashMe


App that failed

Compare & Sync Folders

Display Menu

Folder Magic

Sparkle

Sync Folders

The Duplicate Finder


We are going to be doing some more testing and apply the patches to see if they break this fix, and if so, reapply the fix..

Nov 28, 2020 7:05 PM in response to isaacfromdnk

I had the same issue with Pages, Numbers and other AppStore apps after a fresh High Sierra install. I spent a silly amount of time with restores, clean installs etc. Finally was able to get things working on these apps by replacing the signature with the command below


sudo codesign --force --deep --sign - /Applications/Pages.app


(change the *.app for whatever app is refusing to start)

Nov 29, 2020 7:49 AM in response to FrancoisQC

FrancoisQC wrote:
Yes, I preferred blowing one entry away than the entire DB...


"Blowing one entry" is not enough for me because it does not fix every problems I have with the App Store.

For example, trying to reinstall BBEdit that I already owned in the past from App Store is difficult. I have to try it many times.


If I use the old working database, I can reinstall BBEdit without problem. It also fixes other problems from the App Store.


When I use the new database (with or without the serial entry removed)

- There are only 2 incompatible updates listed under "Updates tabs" : Pages and Numbers.

- Many apps show the "Install" button where it should be "Open".

When I use a working database :

- There are 4 incompatible updates listed under "Updates" tabs : Page, Numbers, Garageband and BBEdit

- Apps show "Open" button when they are installed under the "Buyed" tab.


Other ways to get a working database :


1) Quit every apps, launch terminal and type :

sudo killall -9 trustd; sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'update groups set flags=16 where groupid in (select groupid from issuers where hex(issuer_hash) = "CE057691D730F89CA25E916F7335F4C8A15713DCD273A658C024023F8EB809C2")'

Then reboot.


(This issuer hash should correspond to "Apple Worldwide Developer Relations Certification Authority")


2) Quit every apps, Launch terminal and type :

sudo killall -9 trustd; sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'delete from serials where groupid in (select groupid from issuers where hex(issuer_hash) = "CE057691D730F89CA25E916F7335F4C8A15713DCD273A658C024023F8EB809C2")'

Then reboot.


3) Quit every apps, disconnect from internet (turn off wifi or remove ethernet cable).

Launch terminal and type :

sudo killall -9 trustd; sudo rm /Library/Keychains/crls/valid.sqlite3

Then reboot.

Reconnect to internet.


Without internet connection, database is version 42 not 145. Flags for the group is 16 whereas 0 in a database version 145.


4) This one is very strange and doesn't work every time. But it explains why some people resolved the problem simply renaming the database file.

Launch terminal and type :

sudo killall -9 trustd; sudo rm /Library/Keychains/crls/valid.sqlite3

Then launch quickly sommes apps or reboot with option to reopen apps before the database is completely rebuilt.

Sometimes it may stops the database from getting filled and will result in an incomplete but working database.



I juste tested this solutions today. They have to be tested for a longer time. I will test the solution 1 from now because it seems to be the safest of the 4.


Deeper understanding of group flags and how trustd works can be made reading the open source code. But I have not enough time and I am not qualified enough to do it. I do not know which version High Sierra use.





Nov 29, 2020 5:43 PM in response to BDAqua

Hello Everyone,


I had this same problem on an install I did back on 23rd November, but wasn't aware of this thread.

I did a fair bit of investigation and can shed some light into what's going on though a lot of that information is already in this thread so sorry if I'm repeating that as I didn't see this thread until today.


The issue is not that the apps are installed incorrectly, they're installed fine I believe.

I say this as I upgraded a system and so many apps were already installed.

The problem is to do with how the OS validates that the apps are good.

In essence it validates two things that are relevant here:


1) Has an app that's on your system been tampered with ?

2) Did you buy it ?


What this means is that there are signatures inside each application that sign it to answer yes to both of those questions.

These signatures are there, but the OS needs to know they are good and it needs some files known as certificates to help it do that. It's a bit more complex than that as there need to be certificates which prove other certificates are good resulting in a chain of them.


a) There is a different certificate used to sign the answer to the 2 questions above.

b) There is a 3rd certificate that proves these first two are good.

c) There is a 4th certificate that proves the 3rd one is good. This is where it all stops as this 4th one is from Apple and will be installed on most computers (Macs or not) all around the world.


There can be more than 4 in total since some apps were signed years ago with older certificates.

People with this problem may well find ancient apps from the Apps Store like one called 'Snap' will still work for example.


In this case though first 3 certificates have not been seen by the new OS installation.

None of them are revoked in the real world or even on anybody's Mac.

The OS is incorrectly reporting them as revoked but that's a red herring I believe but the fix does involve checking that they're not revoked so the jury could still be out there a little bit perhaps.


The trick is to make the OS see and verify them all. Some people in the thread have noted that a tool called WhatsYourSign helps.  Indeed it does but it doesn't cover certificates for receipts (did you buy it).


There is a tool out in the wild called "RB App Checker Lite" which does however.

This fix does not unfortunately appear to be permanent and only lasts for a few hours at least for me.


1. If you start with a fresh install and haven't made too many other changes to try and fix things (especially those that involve Key Chain Access) then installing this app and dragging one a non launching App Store app to it may be all you need to do to fix the problem. The other apps should work to but if one on the off chance is signed differently try dragging it too.


2. If you started with an upgrade from Sierra or lower then step 1 may also be all you'd need to do. However if you've been developing your own Apps in the past then make sure there are no expired copies of the "Apple Worldwide Developer Relations Certification Authority" in your login or system keychain. Having this there was what made some certs show as revoked vs not verified and led me to doing all the other stuff mentioned in this thread like clearing CRL caches etc which I don't think made any difference after all.


Most people can ignore the rest of this post and use RB App Checker Lite, but to do it from the shell something like which is shown below should do the trick I think. I've used an example app that was not working on my machine. Again you only need to do this for one app. I could just post the 2 certs that were involved but they are not mine to post and this is more portable and shows what's going on.


cd /tmp

mkdir fix-mac

cd fix-mac

curl https://developer.apple.com/certificationauthority/AppleWWDRCA.cer --output AppleWWDRCA.cer

codesign -dvvvv --extract-certificates '/Applications/Compare Folders.app'

security verify-cert -r AppleWWDRCA.cer -c codesign0 -R ocsp

# diff AppleWWDRCA.cer codesign1 if you like and they should be the same file - could be clever and just use codesign1

openssl pkcs7 -inform der -in '/Applications/Compare Folders.app/Contents/_MASReceipt/receipt' -print_certs -out receipt-chain.pem

openssl x509 -in receipt-chain.pem -outform der -out receipt.cer

security verify-cert -r AppleWWDRCA.cer -c receipt.cer -R ocsp


Hint: Since the fix is not permanent, you can of course just keep the files AppleWWDRCA.cer, codesign0, and receipt.cer and write a cronjob which runs security verify-cert every few hours in the meanwhile

Dec 1, 2020 12:34 AM in response to johnno_uk

Thanks everybody for all your investigations.


I tried this morning to delete the database. After reboot, I could launch apps.

Maybe v146 fixed the problem. Could someone confirm ?


Quit all apps, launch terminal and type command :

sudo killall -9 trustd; sudo rm /Library/Keychains/crls/valid.sqlite3

Then reboot and wait a few minutes until the database is full.


Reinstalling a fresh High Sierra would also work.


The group flag for "Apple Worldwide Developer Relations Certification Authority" is now 16 on v146 like it was on v42 whereas 0 on v145. As johnno_uk said, going from v42 to v145 resulted in non working database.

I didn't test if v146 update fixes an non working database.


Database freshly rebuilt today :

$ ls -l /Library/Keychains/crls/valid.sqlite3
-rw-r--r-- 1 root wheel 16187392 1 déc 09:00 /Library/Keychains/crls/valid.sqlite3

$ sudo sqlite3 /Library/Keychains/crls/valid.sqlite3 'select * from groups id where groupid in (select groupid from issuers where hex(issuer_hash) = "CE057691D730F89CA25E916F7335F4C8A15713DCD273A658C024023F8EB809C2")'
2587|16|1|


I still have problems downloading old version BBEdit that I bought, but it may be related to another cause.

High Sierra certainly still have lots of strange bugs...






Dec 1, 2020 5:12 AM in response to johnno_uk

@softmusic: my bet is that when you hit Restart, trustd does not rebuild the DB, but rather write to disk what it has in memory. SQLite3 is a powerful DB that you can use in-memory and on-disk, and common sense is to write on disk what's in memory when you reboot.


And I have to stand corrected (again). What I thought was an incomplete DB yesterday (previous post), well this morning the DB shows version 146, and size was 1.6MB prior to reboot, and 5.8MB after reboot.


Has everyone noticed that when this issue started, the DB version was 145 and the issuer flag was 0. Now it is version 146.


Also, still playing around with Display Menu as my test app, Trashing it and trying to reinstall would always fail and ask me to use the Purchased page. Right-clicking on the app, and choosing "Hide purchase" forced the Apple Store to show me the "Get" button, which when clicked asked me for my iTunes credentials, and the install went fine.


So my "cleanest" workaround was

  • Boot in Recovery of from USB to allow renaming /System/Library/Security/Certificates.bundle/Contents/Resources/valid.sqlite3 to /System/Library/Security/Certificates.bundle/Contents/Resources/valid.sqlite3.ori
  • Reboot
  • Let the system build you a clean /Library/Keychains/crls/valid.sqlite3
  • Wait until you see version 146 (sqlite3 /Library/Keychains/crls/valid.sqlite3 "select * from admin")
  • Reboot
  • Open App Store to "Hide all purchases"
  • Install your missing apps


I could also probably share the clean 5.8MB valid.sqlite3 file... let me see what I can do...

Dec 8, 2020 12:37 PM in response to johnno_uk

I have reported the issue to Apple because I received quite a few reports about this issue for my apps. However unfortunately Apple closed the issue:



We reviewed your report and determined the behavior you experienced is currently functioning as intended. Your Mac needs to be able to reach ocsp.apple.com periodically in order to launch App Store apps safely. Please unlock ocsp.apple.com to restore functionality.


This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

App Store apps do not install properly on a fresh install of High Sierra on internal SSD or HDD

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.