Issues with sparsebundle after 11.3

I've been experiencing this too, roughly once a day. I've been using the mechanism described https://coderwall.com/p/mgi8ja/case-sensitive-git-in-mac-os-x-like-a-pro and https://stackoverflow.com/questions/8904327/case-sensitivity-in-git/12085233#12085233 for years without problem. The sparsebundles live in my ~/Documents folder. They are unencrypted but the volume that ~/Documents lives on is encrypted. This worked flawlessly since 2016 until 11.3. Now the volume will suddenly go 'read-only' and the contents seem to disappear though the volume stays mounted. I work around it so far by 1) using `lsof /Volumes/CM` (where my CM.sparsebundle gets mounted) to find what processes are still accessing the volume, stopping/killing them. 2) `sudo lsof /Volumes/CM` will always show at least one `mds` process still active on the volume so; 3) I just end up force ejecting the volume. system.log shows a flurry of mds processes being automatically killed at about the same time the volume goes 'read only'. My current suspicion is that SpotLight indexing is causing a problem; so I excluded that by adding /Volumes/CM to my work MBPs SpotLight Privacy exclusions. Will see if that helps at all. On my personal Air this has occurred only once so far but that gets exercised less than the work MBP. And yes, I have run Disk Utility on the sparsebundle volume as well as the underlying physical disk (from Recovery Mode) - no problems found by fsck in those.

As a followup. I may have found a workaround for this.

I noticed that the bundle that doesn't spontaneously unmount lives in my home directory. ~, /Users/myusername.

The bundle that shows the erratic behaviour lives in ~/Desktop (so I can mount it by clicking).

I've moved that bundle from ~/Desktop to ~, and it no longer automatically unmounts!

Could still be a coincidence, I just did "mv ~/Desktop/name_of_partition.sparsebundle ~/" so the files did not really move on the disk, but it might still be a coincidence.

The downside is that you can no longer click the icon on your desktop to mount the volume, you either have to open a finder window to your home directory, or type "open ~/name_of_volume.sparebundle" in Terminal.

Can someone confirm that this solves their issue?

I have now tested a bit further, and I have found no way to get sparseimage to work, when it is to be mounted from a NAS SMB share. But...

If you create the exact same encrypted sparseimage with exact the same content as a MacOS (journaled) drive (HPFS), the problem does not exist. I haven't tested sparsebundle, since I'm not using that, but hope this can be of help/inspiration to sparsebundle users affected by sudden "APFS instability" .

The problem is quite clearly located in Big Sur sparseimage/sparsebundle handling, where APFS is used (I have a suspicion, what could be the cause, alas... it's not my task to do the work Apple hasn't done).

I'm enrolling my windows 10 gear for future use on that front, since Bitlocked disk images have worked and mounted trouble free without any hickups for years and years and years and... reliably! Fiddling around with the still somewhat flakey APFS file system based storage is to insecure for my taste. Who knows, the HPFS support may also vanish without warning at some time. Whether intended or caused by a bug.

Regards

P.S. An APFS formatted sparseimage, that mounts without any problems from a locally connected Thunderbolt SSD, exhibits the following behaviour (if password is located in keychain, otherwise password must be entered immediately, when mounted from an SMB NAS share):

• The "image" is checked for a loooong time:

• When completed, the following error message is shown:

Stating that: No archives can be activated.

I will stop using APFS for disk images in general, and - of course - any Apple formats on all external disks (especially when encrypted). Who'knows, what new surprises the APFS format "work (still) in progress" may turn up in form of spanners for existing and currently working functionality ;-)

@jpcornet Indeed, you seem to have found a workaround! After testing for 3+ weeks, I can confirm that I had not any sparsebundle unmount itself when the image resides in the user's home root folder "~/". Interesting, isn't it. I think this again points to some bug with permissions as has been discussed in this thread before. I could sometimes witness "spontaneous unmounting" when some app asked for file access permissions in ~/Documents or an external drive.

I can also confirm that the sparsebundle issue otherwise still happens in Big Sur 11.5.2. But now I have moved all sparsebundles to the home directory. Bug report submitted to Apple, and nothing will happen, as usual.

There are many other serious issues in Big Sur, to be discussed in other threads.

I have been struggling with this same issue for months! Currently on 11.4. My sparsebundles were originally HFS+ (Mac OS Extended), and those would just unmount themselves at random times (generally when working with files within the bundle volume). So I figured, maybe Apple has been ignoring HFS+ testing and I'll recreate them as new AFPS bundles. Did that, cloned the data over using CCC (and of course kept the old bundles just in case things go wrong with APFS.) Now the behavior has changed -- With APFS, the volumes no longer unmount themselves, but many files or entire folders simply disappear at random times. Again, seemingly correlated with rapid file activity (e.g. saving a project file, building code, etc.) Whatever app I am using starts to wig out and show errors about the files missing. Ejecting the volume and then remounting solves it, but this is not a real workaround.

To earlier advice about decrypting bundles before upgrades, or other host disk issues: No. This has nothing to do what that. Both the host volume and the bundles checks out fine using Disk Utility. And as mentioned I even tried creating a new APFS disk image from scratch.

Apple clearly has a widespread bug in their Disk Image support on Big Sur. The process "diskimages-helper" is likely the where the bug exists.

The issue is intermittent, but frequent enough to drive you insane. Depending on usage, it may take a few hours, or a day or two. Generally when not actively using the volume it behaves -- You can check in and see the files are still visible. But soon after you start reading/writing files, it glitches out. Haven't been able to pin down the exact cause, but might try writing some stress tests.

I do not believe this is user account specific. I've got a personal Air and a work MBP. The user account management is completely different - enterprise vs. out of the box macos. The work MBP has all kinds of enterprise tools monitoring access to files (virus scanners, etc.); the personal machine does not. I use the same APFS case sensitive sparsebundle technique on both that primarily contain collections of git repos I'm doing software development on. Both exhibit this spontaneous unmount of the sparsebundle behavior. Apps that are accessing / modifying the content of the sparsebundle include a wide variety of command line and developer tools; in addition to the enterprise tools on the work MBP.

I've gotten frustrated enough by this that I've tried various techniques to try to insure I lose no more work:

1) My first, simple approach of having a script append a character to a file every minute to make the volume appear to be in use (and provide me a clue when the filesystem switched to read-only) did not provide a solution.

2) My next approach, creating git hooks so that whenever I made a git commit that would get immediately pushed to an upstream repository did not prevent me from losing modifications that had not yet been committed.

3) My current 'solution' involved creating a Hammerspoon-based filesystem watcher, so that whenever any file gets changed on the volume a script is run to rsync the changes of selected contents to Linux-based mirrors. So far that has saved me from losing any additional work, but doesn't speak well for the impact of Big Sur updates. (This began for me with 11.3; I encountered no problem from 2015 up through 11.2.

FWIW it seems as though the unmounts occur for me when the volume is relatively idle, but as noted above, periodically writing to the volume was insufficient. As I think others have noted, I don't believe excluding the volume from Spotlight indexing had any positive effect either.

Since the latest updates by others to this discussion mention moving the sparse disk image to their home folder from either the Desktop or Documents folders. Perhaps it's a bug with the privacy protections?

Also, no one mentioned it but has anyone tried mounting the sparsebundle via the Terminal's "hdiutil attach" command instead of mounting via Finder?

This is an Apple Customer Forum. There are Apple employees on this forum but only as moderators and curators. You need to get this information to the right people at Apple.

You should file feedback and a bug report to Apple. Run through your testing, take detailed notes and document it all. Include as much detail as possible. Be sure to mention the specific NAS you have, the disk format of the NAS and version information, etc. They need to be able to reproduce the problem before they can fix it. I would suggest using smaller sparse image files for testing.

Developer Feedback Assistant

In Safari type in applefeedback:// on the location / search bar

or

https://feedbackassistant.apple.com

or if you have a (free or paid ) Developer Account

https://developer.apple.com/bug-reporting/

then it is entirely possible your data storage is having trouble staying healthy.

http://www.binaryfruit.com/ DriveDX

can verify your drive's health.

Have you tested it in another user account to see if the problem is system wide or limited to your account?

Another question to all: Which format do your failing sparsebundles have? AFPS?

If everybody who is having problems deals with AFPS-formatted bundles, maybe we can avert the bug by using good ole "Mac OS extended". Might be worth a try.

Try this simplistic fix: boot into Safe Mode according to Start up your Mac in safe mode - Apple Support and test to see if the problem persists. Reboot normally and test again.

NOTE: Safe Mode boot can take up to nearly twice as long as a normal boot as it's doing the following; 

• Verifies your startup disk and attempts to repair directory issues, if needed

• Loads only required kernel extensions (prevents 3rd party kernel/extensions from loading)

• Prevents Startup Items and Login Items from opening automatically

• Disables user-installed fonts 

• Deletes font caches, kernel cache, and other system cache files

Thanks for the tip @jpcornet. I moved my .sparsebundle file my home directory and it has not unmounted on me now for nearly a week.

WRT to being able to double-click on a desktop item, have you tried creating an alias on your desktop to the image file? I would expect that work.

I can confirm that, my sparse bundle would start having problems every few hours, sometimes after ten minutes, files would disappear, then I wasn't able to eject it anymore. I moved the sparse bundle in my home directory, and boom, working like before. Huge thanks to @jpcornet and the community. hope apple fixes this "soonish"...

Have you ran (Disk Utility's) First Aid on the volume inside the disk image?