Lost mail in OS Catalina upgrade

Several people have reported lost or blank e-mail messages after updating macOS to Catalina. I lost 50,000 messages. Here's my solution, which requires a back-up of messages made before the Catalina update.

1. Restore the folder V6 from User/Library/Mail to the desktop.
2. Open Mail if it is not already open.
3. Create a new mailbox where you will put recovered messages so that you can review them before storing them with other messages.
4. Select the command "File" - "Import Mailboxes  … ", then select "Apple Mail" as the format.
5. Look in the folder V6 on your desktop for a mailbox that you want to restore, select it, then click "Continue". The import will start. A message will tell you when the import is done.
6. After the messages have been imported, the messages will be in a new mailbox called "Import". Within that folder, they will be either in the simple folder structure that you would expect or in a multi-level folder hierarchy. If they are in a simple folder structure, click on a message in the message list, press command-A to select all the messages, and drag them to the folder that you created in step 3. If they are in a multi-level hierarchy, option-click on the triangle next to the folder called "Import" to expand the hierarchy. Command-click on all folders named "Messages". Click on one of the messages in the message list. Press command-A to select all messages in the message list. Click and drag the messages to the new mailbox that you created in step 3.
7. Progress in moving the messages will be shown at the bottom of the folder list. After all the messages have been moved, delete the "Import" folder by right-clicking on it and selecting "Delete Mailbox  … "
8. Repeat steps 3 to 7 for other mailboxes.

Methods that did not work for me:

• Re-building mailboxes.
• Deleting envelope files in folder V7 and forcing Mail to re-index V7 messages.
• Temporarily removing folder V7 and forcing Mail to re-convert messages from folder V6.
• Restoring a Time Machine back-up from within Mail. Mail crashes.

Oh, it was posted by me as Tripteron. I don’t know why different user names are showing up . Here is my method. Since sharing it, I have had three discussions with Apple, and their engineers have been investigating. I am waiting for their next report.

Mail recovery method

Several people have reported lost or blank e-mail messages after updating macOS to Catalina. I lost 50,000 messages. Here's my solution, which requires a back-up of messages made before the Catalina update.

1. Restore the folder V6 from User/Library/Mail to the desktop.
2. Open Mail if it is not already open.
3. Create a new mailbox where you will put recovered messages so that you can review them before storing them with other messages.
4. Select the command "File" - "Import Mailboxes … ", then select "Apple Mail" as the format.
5. Look in the folder V6 on your desktop for a mailbox that you want to restore, select it, then click "Continue". The import will start. A message will tell you when the import is done.
6. After the messages have been imported, the messages will be in a new mailbox called "Import". Within that folder, they will be either in the simple folder structure that you would expect or in a multi-level folder hierarchy. If they are in a simple folder structure, click on a message in the message list, press command-A to select all the messages, and drag them to the folder that you created in step 3. If they are in a multi-level hierarchy, option-click on the triangle next to the folder called "Import" to expand the hierarchy. Command-click on all folders named "Messages". Click on one of the messages in the message list. Press command-A to select all messages in the message list. Click and drag the messages to the new mailbox that you created in step 3.
7. Progress in moving the messages will be shown at the bottom of the folder list. After all the messages have been moved, delete the "Import" folder by right-clicking on it and selecting "Delete Mailbox … "
8. Repeat steps 3 to 7 for other mailboxes.

Methods that did not work for me:

• Re-building mailboxes.
• Deleting envelope files in folder V7 and forcing Mail to re-index V7 messages.
• Temporarily removing folder V7 and forcing Mail to re-convert messages from folder V6.
• Restoring a Time Machine back-up from within Mail. Mail crashes.

I had the same problem. I found all my lost mails in a hidden folder on my MacBook. Try the following: Show all hidden files. (See hidden files on Mac via Finder: 1.) In Finder, open up your Macintosh HD folder. 2.) Press Command+Shift+Dot.

Your hidden files will become visible. Repeat step 2 to hide them again!)

Then go to: Macintosh HD/Users/your-username/Library/Mail/

Here I found all accounts and every single mail. (.emlx files)

I did a copy of all the Data but didn't try to reimport it to my Mail application. I am glad to have the files again. Maybe this is helpful for you too.

techy-layman,

Did you tried to recover, using Time Machine from the Finder with the window Library/Mail open) a copy of the state of this folder at Dec 2019?

If not, try but do not overwrite your actual folder (or make a copy first just in case) and put this recovered folder on your desktop and try to import the mailboxes. Perhaps…

coxorange: yes, those are the folders (and mailboxes) that I mean.

You would not need a new account for each folder, just one IMAP account, which you probably already have. You must also have enough storage in your IMAP account for all your messages.

In your place, I might do these things:

1. Back-up everything.
2. Check message counts in every folder and write down the counts. Add up the counts.
3. Ensure that I had a good back-up of the V6 folder. Put it on my desktop. Lock it.
4. Ensure that I had a good back-up of the V7 folder. Put it on my desktop. Lock it.
5. Copy all the folders, mailboxes, and messages that are in my "On My Mac" account into an active IMAP account.
6. Upgrade to Catalina. Open Mail. Let the automatic import process run. Note how many messages it says it's importing. Compare it with the number that I came up with in Step 2 above.
7. Check message counts in every folder and write down the counts.
8. If all the messages are in my "On My Mac" account, check that they are being displayed properly with bodies and correct dates. If everything is good, delete the duplicate folders and mailboxes in my IMAP account. I'm done.
9. If not all the messages are in my "On My Mac" account or if they are not being displayed properly with bodies and correct dates, see if all the messages are in my IMAP account, displayed properly with bodies and correct dates. If everything is good in the IMAP account, delete the duplicate mailboxes and folders in my "On My Mac" account. Copy all the folders, mailboxes, and messages that used to be in my "On My Mac" account from my IMAP account back into my "On My Mac" account. Check message counts in every folder, display of bodies, and dates. If everything looks good, delete those folders, mailboxes, and messages from the IMAP account. I'm done.
10. If not everything looks good, I have to decide if I want to revert to a previous version of macOS or stay with Catalina and restore messages from the V6 folder using the import command from within Mail (see my earlier instructions for that). My choice is to stay with Catalina but use a different mail program. You may have a different preference.

G'day,

I used the procedure described by triperton to successfully recover 40,000 odd e-mails and attachments. The e-mails were recovered in the original multi-level mailbox structure into which I had filed them. Random sampling has failed to find any damaged e-mails (eg header only, no text).

Even if you have given up and accepted the loss of your e-mails you can still apply the procedure and get them back provided you have a backup of your e-mail system from before the upgrade. This is true even if you wiped your disk to do a clean Catalina install, providing that back up wasn't on the disk you wiped. (A time machine backup wouldn't be).

If you don't have such a backup, and you haven't wiped the disk since the first upgrade to do a clean install, you may still be able to recover your e-mails. For me the Catalina upgrade left the e-mails in the /Users/<your user id>/Library/Mail/V7 folder in folders with names like <hex number>...<hex number>. You may be able to apply triperton's procedure to them. (I haven't tried this but can't see why it wouldn't work.) You would want to apply the procedure to folders whose names are of the form <Hex number>...<Hex number> and whose date is the day of your upgrade. Don't apply it to folders of that form of name whose dates are after the date of the upgrade because these are for your current mail accounts.

I first came across triperton's procedure at https://discussions.apple.com/thread/250970504?page=1 and have added some comments there to amplify the procedure which some may find helpful.

Good luck.

You can restore the mail from your last backup prior to upgrading to Catalina. All of the mail files will be in your User/Home/Library/Mail/V7 folder.

You can restore that one folder from your backup or the entire Mail folder.

(1) Yes, Time Machine. Yes, Mojave.

You should restore the V# folder with the highest number (apparently V4). When new versions of macOS and/or Mail come out, Apple increases the number of the folder. Sierra must have used V4. High Sierra must have used V5. Mojave used V6. Catalina uses V7.

(2) This method should work for any messages, I think, because all messages must get backed up.

(3) My attachments were restored.

(4) The messages are restored in new folders named the same as the folders that they were in when they were backed up. The new folders are sub-folders of a folder called "Import". Everything is done by folder. You do not have to re-file messages.

(5) I believe that messages get restored with with their attributes (read/unread, flagged, junk), but I'm not sure. I cannot tell because I did not have attributes set on my messages.

Good luck.

In my case the problem can at least partly be explained by a corrupted Mail system on my High Sierra system when I used Migration Assistant to update my new MBP running Catalina. Most (perhaps all?) of my problems were in nested “On My Mac” folders that didn’t transfer over correctly. In trying to recover lost emails, when I exported from my HS system, the inner folders *acted* like they transferred, but appeared empty on Catalina. I had to individually, export (from HS) and import each folder; it would not correctly transfer nested folders. I could read the mail in those nested folders just fine on my HS system, but the export function failed to capture them properly. Not sure that Catalina was a fault for me. Instead, looks like a broken export or migration issue that only exposed itself on Catalina.

So, an update.

I think I've addressed this disaster, the disappearance of more than 20 years of e-mails deleted by Apple Mail.

So far, it seems to be working:

From Apple Mail full of e-mails that only had their headers (blank e-mails ), I did a mailbox import. These mailboxes, I looked for them first with Time Machine (and with the Time Machine tool, outside the Mail application) in users/xxxx/Library/Preferences/Mail. This reconstructed Mail folder (and which I have hosted on my desktop) therefore contains the V7 folder which itself contains folders with unlikely names (7DCC37D6-22A9… ect.) which contain mailboxes that I have imported into Mail with Mail together (e.g. Deleted Messages.mbox + Drafts.mbox + INBOX.mbox etc.), folder after folder.

This is long. Very long. And annoying. Very annoying.

But in the end, in Apple Mail, we have our imports, and we still have to guess which accounts these mailboxes correspond to and move their e-mails (by hand, en bloc) to the right folder: no need to remove the blank e-mails. By being a little patient sometimes, mutilated e-mails find back their content.

I just thought I had the same problem.

BUT: When I turnrd off Catalina‘s Filter function for each account nearly all my mails were back. A few very old ones >20 years had lost their dates And appeared as new.

That filter function might be improved by giving some better visual feedback that there are filtered messages in a mailbox

As simple as it seems, i just went into Mail > Preferences > Accounts and unchecked "Enable this account," restarted Mail, rechecked "Enable…" and it all came back. Maybe I was lucky, but it's a quick option to try.

Sorry if anyone else posted about this, didn't have time to read all responses…

My method recovered those messages and I have access to them. But Apple Mail does not display them.

Apple Mail has a problem with the automatic conversion of messages from version 6 to version 7 (Catalina) and several problems with displaying messages properly.

So the complete solution for me includes exporting messages, importing them into a more reliable mail application than Apple Mail, and using that program instead of Apple Mail. I'm using Postbox, which displays all my messages, including the ones that Apple Mail does not.

Hello tech-layman,

As far as I can say, yes, pop e-mails are kept forever on the machine as long as they're not erased.

The problem, as far as I saw that, is a new Apple Mail importation of its e-mails (with Catalina) erases (sometimes) e-mails in its own database (or V6, V7 etc.). We have a name for that: a bug.

By "erase" I mean "erase": they are no longer on your machine (because Apple Mail decided to wipe them up). So if you don't have a backup of them on a disk somewhere else, no way to recover them. Sadly.

Otherwise, as I did with a Time Machine backup, I repopulated my Apple Mail with the missing e-mails by importing them again with the function File->Import Mailboxes.

What follows is too much for most people, but here it is in case it helps someone.

When I updated macOS to Catalina, I lost almost 50,000 locally-stored messages and Apple Mail failed to display many others properly.

After recovering my messages from a pre-Catalina back-up (see my method in this thread: https://discussions.apple.com/thread/250716550), I switched to Postbox (www.postbox-inc.com). Since then, I have become frustrated with some aspects of Postbox. I just switched again, this time to eM Client (www.emclient.com).

eM Client does not have the problems of Apple Mail or some of the annoyances of Postbox. It has a nicer user interface than Postbox. It handles Exchange, iCloud, and IMAP accounts well. It seems to get regular maintenance. The vendor's blog says that an update is due in April.

Transferring locally stored mail from Apple Mail to eM Client may not be as easy as you want it to be because eM Client can import only one MBOX folder at a time. There is a neat way around that, but it requires some knowledge. I got Thunderbird and moved all my MBOX folders into the appropriate OS-level folder in Thunderbird. Then I used the Thunderbird-specific import function in eM Client to transfer many folders at once. You can get Thunderbird at https://www.thunderbird.net/en-US/.

Or you could just move all your locally stored message folders to an IMAP account, have eM Client sync with that account, then move the folders to local storage. If you have tens of thousands of messages as I do (120,000 now), then that process will take a long while.

The free version of eM Client allows two e-mail accounts. The pro version, which allows more accounts, is expensive relative to what people are used to paying for good software these days, $49. One can buy a lifetime license for $50.

To use an iCloud account in eM Client, you will need to get an app-specific password from Apple at www.appleid.apple.com and use that as the password.

If you decide to switch back to Apple Mail later, transferring messages in that direction is more difficult. One can use the IMAP method mentioned above. If you don't want to wait days for tens of thousands of messages to transfer, it seems that you'll have to buy another program to convert the eml files exported from eM Client into Apple's format. I tried the demo version of Emailchemy for this step and it worked well (https://weirdkid.com/emailchemy/).

Yup, too much hassle for e-mail functions that should be completely reliable in 2020.