Very slow boot on Fusion and Big Sur

Here the result of a new test I did today. I copied to my iMac some big files from the network (about 8 GB). Then I installed Photoshop. After the installation, I restarted the iMac and the booting time jumped from the previous 19 seconds to about 1 minute. I started Photoshop and it took some minutes (including the verification). I closed Photoshop and I left the system idle for about 20 minutes. Then I restarted. That time the booting time came down to 30 seconds. I left the system idle for 45 minutes. Then I restarted. The booting time now was fully recovered to 19 seconds, as before the installation of Photoshop. Photoshop takes about 40 seconds to start. Word and Excel take about 13 seconds.

This test confirms that the Fusion Drive is now working as expected: when something is written, the SSD is always used. But when the system is idle, it moves back to the SSD all the blocks that are frequently used.

Dear Fabio,

Thank you, very informative!

If ever you install another large app like this or copy a number of large files, and then let the system be "idle" for 30 minutes or more, could you please first open Activity Monitor and check under the "CPU" tab which process(es) are using the CPU most (I suppose you know how to do that, to arrange the processes according to the percentage they are using, from most to nothing?). This could reveal what the OS is doing during the 30 or 40 minutes of "idle" time. This process or processes should be the one(s) that manage how apps and data and system files are arranged on the Fusion Drive.

This can be another interesting thing to do the next time I'll move big files or I'll install big programs. By the way, Activity Monitor by itself uses about 5% - 7% CPU and I'm not sure the Fusion Drive housekeeping processes can start when Activity Monitor is running.

When looking in Activity Monitor, the only processes that seems to generate a lot of disk activity are kernel_task, launchd and mds_stores (related to Spotlight). I also now notice that Memory pressure is way low: no apps or processes go higher than 250MB RAM. Before reinstalling Big Sur (when my system was extremely slow) I noticed processes easily going over 1GB. Unfortunately, I can't remember which ones.

While playing with my now fast iMac, I notice that every app I previously opened and quit, now opens within seconds, including big apps like Outlook. It did not do that before. I have 24GB of RAM, plenty of storage to keep apps actively in memory, but that was clearly also something that was broken before.

I was also wondering: are you all using the new APFS file system in combination with the Fusion Drive? It creates two volumes, a (non-writable) system volume and data volume. I would assume the system volume is on the SSD, but maybe that's not a given?

No, indeed, we cannot be sure if that process (or processes) start when Activity Monitor is running, but there is a good chance they do if you leave the system idle after starting up Activity Monitor. Several other processes do run constantly even if you leave the system "idle" (i.e. if you don't start any app), so the Fusion Drive "housekeeping" process apparently ignores them when it's doing its work.

Just to remind that starting from Catalina, the Mac OS disk must be APFS formatted. Then I assume that all of us are using APFS ...

Concerning your last assumption, it's probably wrong. It seems the OS moves to SSD the blocks that are more frequently read, independently from the System or Data volume. Maybe the new 11.3 update changed some rules but, based on the results of my test (that show an increased boot time when large file are written on the Fusion Drive), my guess is that the system disk is not managed in a different way.

Big Sur demands APFS on all internal drives, so yes to the first part of your question.

On which part of a Fusion Drive (fast but small SSD or slower but large HDD) resides what and when, is totally invisible to the user and is totally dependent on the logic (software) that manages the Fusion Drive. In the past there were Terminal commands one could use to see which part of the FD was being written to when e.g. saving larger files, but I am not sure that Big Sur still allows these commands to be executed or gives the feedback they ask. The software that manages how the OS, the apps, and the data are spread out over the two parts of the Fusion Drive (not even as complete files but on block level!) probably works quite like the software that manages RAID-systems, but with an extra layer of intelligence that normally gives preference to writing and keeping the apps and data that are most often used, onto the SSD blade (because it's faster than the HDD to read from and write to). That means that bits and bytes are being swapped almost all the time from HDD to SSD and vice versa whilst we use our iMacs. The problem with Big Sur seems to be that, until now at least (I haven't had the opportunity yet to test out 11.3 as you have had), during startup (boot) the OS was constantly reading and/or writing a number of very large and essential parts of itself from and to the HDD, and not (or only partially) from and to the SSD part of the FD. Catalina never had that problem and considering that Big Sur itself is much and much larger than the Catalina OS, it seemed that Big Sur until now could not or would not use the SSD for those essential parts to reside on during the boot procedure. It had to use the slower HDD for that. 11.3 seems to have found a solution for that, but there are other users upgrading or updating to 11.3 and reporting absolutely no difference with boot times still up to 3 minutes...

You're right about the APFS part, I forgot about that and confused it with APFS for Time Machine support that only came more recently...

I've read that the SSD part of Fusion Drives always has 4GB free as a designated landing place for active read/writes, and only later files are moved to the HDD part in the background, thus guaranteeing fast speeds. The OS itself should also remain on the SSD. So your experience that boot times temporarily went up after installing Photoshop is a weird one.

I have now reinstalled iWorks and Xcode, in addition to MS Office, and rebooted in between every additional installation, and at no moment were boot times affected. I can only conclude that a faulty Big Sur installation either puts crucial parts of the OS on the HDD drive and/or the 4GB free space on the SSD is no longer intact and every read/write operation occurs on the HDD part instead.

If you run "diskutil list" from the terminal prompt, you can see the configuration of your Fusion Drive. As you can see I have the 24GB SSD version. I do notice that the Macintosh HD⁩ volume (which contains the OS) is 15.3GB, so (while it's hard to conclude this from the output as it's unclear what is stored on which drive), there's at least theoretically room left on the SSD for the 4GB landing zone and some additional frequently used files.

But the fact remains, we will never know what is stored where, and when the OS decides to move files based on what criteria.

/dev/disk0 (internal, physical): (SSD part)

   #:                  TYPE NAME                    SIZE       IDENTIFIER

   0:      GUID_partition_scheme    *24.0 GB    disk0

   1:                 EFI ⁨EFI⁩                     314.6 MB   disk0s1

   2:                 Apple_APFS ⁨Container disk2⁩         23.7 GB    disk0s2

/dev/disk1 (internal, physical): (HDD part)

   #:                       TYPE NAME                    SIZE       IDENTIFIER

   0:      GUID_partition_scheme                        *1.0 TB     disk1

   1:                        EFI ⁨EFI⁩                     209.7 MB   disk1s1

   2:                 Apple_APFS ⁨Container disk2⁩         1000.0 GB  disk1s2

/dev/disk2 (synthesized): (full Fusion Drive)

   #:                       TYPE NAME                    SIZE       IDENTIFIER

   0:      APFS Container Scheme -                      +1.0 TB     disk2

                               Physical Stores disk0s2, disk1s2

   1:                APFS Volume ⁨Macintosh HD - Data⁩     186.1 GB   disk2s1

   2:                APFS Volume ⁨Preboot⁩                 285.0 MB   disk2s2

   3:                APFS Volume ⁨Recovery⁩                622.1 MB   disk2s3

   4:                APFS Volume ⁨VM⁩                      1.1 MB  disk2s4

   5:                APFS Volume ⁨Macintosh HD⁩            15.3 GB  disk2s5

   6:              APFS Snapshot ⁨com.apple.os.update-...⁩ 15.3 GB    disk2s5s1

My SSD disk is disk0 and the HD is disk1. If you want to have more control in the creation of the Fusion Drive, after separating it with the command:

diskutil apfs deleteContainer <identifier>

and after formatting the two distinct disks, you can run the command:

diskutil apfs createContainer -main <ssd identifier> -secondary <hard drive identifier>

diskutil apfs addVolume <container identifier> APFS "Volume Name"

As ssd and hard drive identifiers you should use the identifiers of the free partitions (disk0s2 and disk1s2), not the one of the disks (disk0 and disk1).

A wrong identification of the SSD can be the cause of the low speed of your system.

BTW: there is a simple way to determine if the low speed is determined by the HW configuration or by your profile: after resetting the fusion drive and reinstalling the OS, you can try to boot the clean installed system (3 times and letting the time for the optimization).

Before I do anything more, I should know if what you mean by "low speed of (my) system" is the same as what I call "3 to 4 minute boot time". After boot and after waiting about one more minute (i.e. a total of 4 minutes from Apple boot logo to responsive apps), my system runs flawlessly and very fast. I never had a problem with Big Sur slowing down my system *after* boot. I only had and now have again boot times that are unacceptably long. So, are we talking about the same thing?

When FileVault is enabled, the login prompt is shown at an earlier phase of the boot process, because the password is required to unlock the disk. Then when booting with FileVault enabled the login screen appears few seconds after you pushed the power button.

Back in 2017, The 2TD Fusion drive with 128GB of SSD was a substantial upgrade from the standard 1TB Drive. It offered a balance of capacity, performance and affordability. So now that Big Sur doesn't perform well with the Fusion drive, it's our fault that we didn't spend an extra $$$ for a 2TB SSD drive back in 2017.

Well, it's just seemingly strange that a proprietary OS can't support its own proprietary hardware fully, isn't it? And why market it in the first place before careful consideration for long term support? I didn't know about the existence of fusion drive until the current iMac purchasing online ordering steps. If just given the choice of HDD vs pure SSD at the point of purchase, I would have taken the SSD option since all my other machines were already on SSD.

https://en.wikipedia.org/wiki/Fusion_Drive

I understand your frustrations, and I had them for 6 months. I was already planning to replace my late 2015 iMac and its measly 24GB SSD Fusion Drive with a new Apple Silicon machine later this year.

But now that I have done a clean install of macOS 11.3 I’m truly impressed with the performance of my machine. At this point I have the impression it’s even running better than Catalina. I fully realise that doing a clean install can be a hassle but so is buying a new machine out of frustration.

I also realise that doing a clean install may not be the solution for everyone and that my experience is no guarantee whatsoever it might work for you, but I do think that 11.3 has introduced some fixes. Our issue can’t be that widespread otherwise it would imply Apple has given up on machines with Fusion Drives, and that can’t be the case.

I have now performed a clean install of Big Sur 11.3, first wiping my Fusion Drive, resetting it, installing 11.3 from a bootable thumb drive, and finally creating a new user account whilst leaving it empty. I rebooted the system a few times, each time leaving it idle for about 30 minutes. Boot times were totally normal and well under 30 seconds, from Apple logo to responsive desktop and apps.

Then I used Migration Assistant to copy over everything from my old account. This increased the boot time again, but instead of more than 3 minutes on previous installations of Big Sur (11.1 and 11.2), my 1TB Fusion Drive initially started up from Apple logo to responsive desktop and apps in around 2 minutes. An improvement, but not nearly good enough to call it OK.

After the first boot after the migration of my old user account, I allowed the system to run idle, for around 40 minutes. Then I rebooted and the boot time was noticeably better again (around 1m30), but still nowhere near to 30 seconds or less. I have let the system run idle for several hours yesterday evening, hoping that future boot times will improve even more.

Once the system is booted up, every single app starts very fast and runs very smoothly. For now, I do not intend to downgrade back (once again...) to Catalina. Neither do I have the time right now to install and fine-tune every app I have and need for my daily bread as a photographer, from scratch. I'm pretty sure now that doing a clean install of both Big Sur 11.3 and all the apps would give me totally normal boot times. Maybe one day I will have the time to do that. For now, I will let everything be and see if my system doesn't return to normal boot times by itself, after giving it some days or a week of normal use to figure out what part of the OS to put where (either SSD blade or HDD part of the Fusion Drive) to achieve normal boot times. I will report back when I have more news. For now, I thank all of you who have had the patience and the intelligence to approach this problem with logic, a lot of testing, and some very precise observations of what happens at what time and in what circumstances. The problem of the very slow boot times, however, is not solved, as they still exist when simply upgrading from a system running Mojave or Catalina to Big Sur (even in its current version of 11.3). We have to do a totally clean install of both the OS and all the apps we have, to be sure our Fusion Drive iMacs are able to start up and be responsive in less than 30 seconds. It's not perfect, but nothing and nobody is, and Apple engineers can be helped helping us by giving them feedback that is as precise as possible. Not here, but on the Apple feedback page.