Happy to say that I have, quite accidentally, found a solution to this problem.
Background: My problem with this bug centred on my iMac installation of macOS, as I described in my initial post in this thread. Messages simply said 'Downloading messages from iCloud...' indefinitely on the iMac, though never actually did any downloading (confirmed by network activity monitors). I spent a few hours with tech support like many others, though also found no solution with any of them.
Then, for entirely unrelated reasons (connected to beta testing some software in development), I had to re-format the hard drive in my iMac last week, re-create it, and freshly install the system including a new installation of macOS. Sadly, no change in the issue. But, in an odd moment of ironically-helpful coincidence, I also dropped my iPhone X down the stairs and shattered the display, which meant taking it in to have it repaired. As I have beta iOS software installed, this meant — and this is the relevant and useful part — that they had to wipe the phone entirely and reinstall iOS anew. Not a problem, as *everything was backed up in iCloud, including all my iMessages (important).
What happened: When I got back my phone which was thoroughly wiped, I went in and did a restore from iCloud backup. This repopulated my iMessages app on the phone with my entire messaging history, which had been stored in the cloud backup — and as soon as this was restored on to the phone, with Messages in iCloud activated, it immediately began to sync / upload that whole history into the cloud anew. At precisely the same time (and I could watch it happen in sync on the desktop), the messages began to download to the messages app in macOS, until the entire content of all my messages history was fully synced on both devices. Since that time, they are perfectly in sync, including new / incoming messages, deleting old ones on any of my devices, etc. So from this stage, iCloud syncing of messages now works properly across my iOS installation on iPhone X, and macOS on both my MacBook and my iMac.
What this means: (If you're just looking for how to use this information to fix the bug on your system, scroll down a bit.) This seems to suggest that the nature of the bug is in downloading message history, stored in iCloud, on the messages app in macOS (and perhaps for some in iOS as well, though I have not tested that). Namely, when you turn on iCloud syncing in Messages on macOS, it immediately connects to message via iCloud, meaning that new messages are synced (and if you are suffering this constant 'downloading from iCloud...' bug, you've probably noticed that even while this hang remains, new messages do come through properly, and are synced across devices such that if you delete one on your phone, it will also be deleted on the iMac, etc.); however, synchronisation across devices of old messages hangs because the iCloud sync somehow doesn't trigger acknowledgement of their existence on additional devices that you add to the sync. So if your iPhone has been your main iMessage interface in the past (which for most of us it has), when you add macOS Messages and set it to sync via iCloud, it doesn't seem to register the existence of those previously-existing messages (Even though they are, in fact, stored in the cloud) and only syncs things that arrive after you initiate that setting.
What happened in my case was that having to re-install iOS on my iPhone, and then restore from an iCloud backup, was that the backup manually populated the Messages iPhone app with my whole history from the backup, which were then uploaded to iCloud as the 'new' contents of my Messages app — and these 'new' messages then were synced across the other devices, and the downloading actually happened in the macOS app.
HOW TO "FIX" THE BUG:
I put 'fix' in inverted commas because this isn't really a fix — the bug still exists (and is a big one, given the emphasis on syncing messages via iCloud in the new iOS and macOS releases), but this is a way to work around it so that you can actually get your iMessages in sync on all your devices, in spite of the bug.
The first 'method' is what I actually did, not by design but by necessity given my situation, and how you can do the same to (hopefully) the same end results. It is clunky and laborious and not at all fun, but certainly worked in my case. The second method I will describe is something that I think might work, based on my speculations above as to what is really going on. If someone tries it out, perhaps let everyone else know whether or not it actually works.
As should be understood, but I'll state clearly: I don't take responsibility for problems you might encounter along the way; and this method involves formatting systems and restoring, so don't try it if you're afraid of data loss, etc. Attempt at your own risk.
METHOD 1: Laborious, time consuming and awful, but tried and works.
- Ensure that your iPhone is set up to use iCloud, that you have Messages enabled in your iCloud settings (so that it is syncing messages with iCloud), and also that you have iCloud backups on in settings. Trigger a manual backup to iCloud to ensure it has your latest messaging content.
- In your phone, go to Settings > General > Reset > Erase All Content and Settings. Click through the various confirmations and verify your password, then confirm and have the iPhone fully reset itself, deleting all data, apps, settings, etc.
- When it finishes, you will go through the normal setup screens as if it were a new iPhone out of the box. Go through this process as normal; and when the time comes, select to restore from a previous iCloud backup. Select your most recent backup.
- Give the phone ample time to download everything anew (it takes a while). All your previous settings should be restored, including the iCloud settings for Messages, meaning that when it gets that far, it will populate the Messages app with your message history from the backup, which will then be uploaded to / identified by icloud as 'new' (by which I don't mean unread; they properly differentiate between read and unread messages so you won't have to go through everything to read it; but 'new' as in newly identified as existing in the iCloud store).
- Assuming you have iCloud sync turned on in your macOS Messages installation, you should then begin to see messages actually downloading. You may need to quite the app and restart it for this actually to trigger.
METHOD 2: Might be considerably easier, but I haven't tried it.
Given that it appears that the issue stems from existing messages not triggering the system as being 'new' and therefore syncing, it might be possible to achieve the same effect as method 1 by a much simpler means. However, as I now have everything in sync I can't test it myself, so I have to leave it to others to see if this idea works.
- On your iPhone, go to Settings > iCloud (i.e. your name at the top of the settings screen > iCloud > and scroll down to the Messages setting. It should already be slid right to 'on', indicating that you are syncing with iCloud. Switch this to off.
- A warning screen will pop up indicating that messages have not been fully downloaded to this iPhone, and asking whether you want to cancel, or 'Disable and Download Messages'. Select 'Disable and Download Messages.' This should cause the app to download the whole message history to local storage on the phone and break the iCloud sync connection.
- Once this is done, Restart your iPhone; then go back to the same settings screen, and switch iCloud sync for Messages back on. This should cause your phone to upload / newly sync all its messages from local storage into iCloud, which should (if this theory is correct) then cause them all to appear as 'new' to your macOS installation, which should then download them — and you haven't had to reset the entire phone. (Again, you might need to quit and restart the macOS Messages app to trigger the download.)
If Method 2 actually works, it will be much preferred and much faster; but again, as I currently have everything synced up properly, I can't really test it (it would be impossible to see if it had an effect).
Let me know if either of these works for you. Again, Method 1 I did by necessity, and it worked for me; but you can see that these methods involve backups, restores, etc., all of which run the risk of losing your data or things otherwise going awry, so try only at your own risk until Apple gets the bug more properly fixed.
