Music hangs on Saving Music Library

Here's an update on the slow progress I've been making.

As an experiment, I exported the library to an XML file and I made a clone of the original Media Library folder on an external SSD. I then edited the Library.xml file so the path pointed to the external volume*; created a new blank music library on Ventura, and imported that XML into it. It kind of worked, but indicated it got some errors.

*Before I imported it, I opened it with Bbedit to edit the path. Bbedit gave me an alert about malformed UTF-8. Which I just ignored, but turns out it was significant.

I used 'xmllint' to look for issues. Sure enough, there were a couple files that had invalid UTF-8 in their metadata. xmllint tells you the line number of the offending data, I then scrolled to that line in Bbedit and scrolled back a few lines to determine the track / artist with the issue. I then went into Music.app and edited the Metadata (with Song -> Info) and either removed the offending field (2 were in Genre, the other in the track filename).

I exported the library again, and this time xmllint showed the file was valid.

Great! Although it's still taking forever to load into the new instance, so I don't know if it truly works. But the bad unicode may be what was crashing it when it goes to save the library. I'll keep you posted.

Correct. this is happening when Music on Ventura attempts to update a Music library originally created in Catalina

lsof is a command line tool to show all open files, by process ID. It's useful for debugging and to see what files a program is changing/updating. (It's basically the same thing as showing open files from Activity Monitor when you get more info on a process).

I tried the option-start-Music as well, same results.

If I open the library, and don't do anything, if will close and NOT hang.

But if you play a file (which seems to rename it to the '01 Title' format from just 'Title' based on changes to the file system I see), it will hang on closing again.

I also confirmed that everything has read/write access.

So when I open it in safe mode, and click on a file, it will sometime play. Othertimes, another song than I click plays, and if I click the original again, it will play. BUT when I look at the file system to see what has changed, I see the file I clicked that didn't play at first and then played the 2nd time has been renamed to the "01 Name" format. And on top of that, in safe mode, it closes quickly. (Safe Mode still takes about 5 or so minutes to load the library).

What all does safe mode do that might change this behavior? I shouldn't have any plugins (it's a fresh install).

My ned step is doing the "Uncheck Keep Files Organized" routing and then re-checking it, which will make it re-org the whole library IN SAFE MODE to see if that fixes things. It takes about 90 minutes to do this, so I'll update later today.

Here's some comments I got from jwz (.org), who had done some open source tools working with the iTunes library file:

"HFS+ would normalize Unicode characters in pathnames using the NFKD form, but APFS does not normalize at all (arbitrary bytes are allowed) and I believe that iTunes/Music.app normalizes using the NFKC form prior to writing to APFS. So depending on when and where that file was first "kept organized", it might be encoded differently, and you can end up with two different files that are equivalent Unicode strings once normalized."

This might explain issues moving to to a NAS as well.

I thought I solved this, and it seemed to work, until I quit it and restarted; then the files got lost again.

Here's what I did that I thought finally fixed things:

1. Made a clone of the original media library on a blank APFS case-insensitive hard drive. (Same as the original on Catalina).

2. Copies the (Catalina) Music Library database (with a new name) to that filesystem as well.

3. Plug that disk into a Monterey machine, opened with Command to choose a new library, and selected the library. This takes a couple minutes to load.

3a. Make sure "Keep Library Organized" is turned off.

4. Music -> Preferences -> Files and change the Media Folder Location to a new folder on a second, empty 4TB SSD. DO NOT CHECK "Keep Media Folder Organized"

5. Spot check your library by playing a few tracks. I did Cmd-I Get Info, under the File tab check the location to confirm that they were on the disk I started with.

6. File -> Library -> Organize Library. Check Consolidate Files. Press OK, cross your fingers and wait a very long time. (At least it will give you a dialog listing all the files it is copying to the new disk). This took about 12 hours for 3TB and 275k tracks.

7. Come back many hours later. Spot check some files via Command-I Get Info -> File Tab and confirm the location in the new disk.

8. Quit Music.app and cross your fingers.

But I'm not sure why when I re-opened things the 2nd or 3rd time that the library database lost track of files again. It's like something was getting cached somewhere, and the cache was outdated.

Today, I have tracked down another Catalina machine to experiment with. I'll report back hopefully tomorrow what I learn. ¯\_(ツ)_/¯

One more thing to add. This library goes back a long way. When iTunes 8 came out and removed the option of not including the track number in the file name, there was a way do disable it this way:

defaults write com.apple.iTunes create-filenames-with-tracknumber -bool FALSE

That seems to be carried over into Catalina/Music with 'addTrackNumber'

defaults read com.apple.music   addTrackNumber
0

Apparently now, versions post Catalina no longer respected this setting.

The external drive is APFS SSD. But I've also had the same issue when copying the library to the internal drive on the Ventura machine as well... but I just did a test which I copied the library into a new location on the internal SSD and opened that via the Command-Open option to select the library. Made sure Keep Library Organized was unchecked, played a few songs to get their last played time set, and it seemed to work. Quitting was fast, opening was slow like normal but after that worked ok.

So now I'm setting back on the Keep Music Folder organized, it took over an hour to do that. I took a look at the file system to confirm the files have been renamed with the leading track number. And they play properly, etc. So now comes the final test: quitting. And so far, after 25 minutes, it's still saving.

This is a fresh, new machine with minimal software installed and definitely no third-party security software. No .tmp files. There is a 'sentinel' file that gets created when Music.app is launched.

I've tried running "sudo fs_usage -f pathname Music" from cmd line to see what files it's opening/reading/closing, as well as doing. I've run find to dump all files in the library to a textfile, and then 30 minutes later do it again to see if anything changed. Which it hasn't. (I also used this procedure to confirm that when experimenting with checking and unchecking Keep Files Organized that I can see what it was doing... and that's how I know it was actually renaming all the files to have the track number in front of them.

But something is keeping the library database from being written to.

I have even tried creating a new account, and making a fresh copy of the music library, and loading it in, and it still hangs on exit, infinitely showing "Saving Music Library". OK maybe not infinite, I'll let it go for a much longer while and let you know. But it's already been going 30 minutes.

Edit: While it's saving, I'm running 'sudo lsof | grep XXX' where XXX is the PID of the Music app. I'm looking for changes, and nothing is changing. And looking at the mod times for all those open files from the Music app, none of the mod times have changed since I initially selected quit on the Music.app. (It's been an hour...) Even the timestamp on the sentinel file is old, but not as old as the timestamp on Library.musicdb

This part "the automatic file renaming and organising" seems to be the key.

I spent many hours yesterday working on this, and still haven't found a reliable solution; but I think the root cause of the problems were due to having 2 similar files that only had a difference I the case of their names, on a normal, case-insensitive file system. Keep Folder Organized would overwrite files in an album name folder if the case was the only thing different.

And because the filenames are Unicode, that makes doing checks for case a bit harder.

Your NAS probably uses a case-sensitive file system. APFS and HFS+ default to non-case-sensitive.

But this definitely seems like a big in Music.app that's causing this hang.

Here's an interesting datapoint. I just started with a blank library on the Ventura machine, and manually dragged all the files into a new, blank library. 270k files. Took a while but eventually loaded. Spot checked them, the seemed to all be there.

Then I quit music.

The "Saving Library" dialog came up for about 15 minutes and then the app exited normally.

Except the library file wasn't updated.

And when reopening it, nothing was there.

It failed silently. I'm trying again, bringing about 1/4the of the library at a time, and will report back on how that goes. I'm not sure if it's something in the metadata of one of the tracks, or the size of the library.

I did experiment with bringing in just a few thousand tracks and that saved properly and was there when reopening.