Disk utility stuck on checking catalog? Does it make sense to wait long?

I answer to my own question:

It makes sense to wait overnight to let First Aid finish its work!

There was a growing of the filesystem necessary to make the volume usable for a larger amount of data. This was not properly communicated by the macOS and maybe alredy in progress in the background.

Disclaimer: This report is actually describing the situation on a legacy archive system riunning under Catalina on a MacBookPro 15 2014 with no connection to the internet running as a low energy server with integrated USV in a HengeDock. The same may apply to current macOS and hardware or maybe has changed already (to the better or worse). Hopefully someone can find and use this documentation in a similar situation.

The First Aid finished overnight with success and the report was continued after some other actions ending with:

... omitted
... at the beginning ...
Performing fsck_cs -n -x --lv --uuid 54933A9D-*********************-62E2A7A02B15
... most omitted
Checking volume information.
The volume TimeMachine_accrStudioServer06 appears to be OK.
File system check exit code is 0.
Restoring the original state found as mounted.
Growing Logical Volume
Resizing Core Storage Logical Volume structures
Resized Core Storage Logical Volume to 5.000.164.016.128 bytes
Growing file system

... Success (similar to that, for me it was in german)

The interesting part is:

Growing Logical Volume
Resizing Core Storage Logical Volume structures
Resized Core Storage Logical Volume to 5.000.164.016.128 bytes
Growing file system

Due to the fact that we store a lot of old Archive stuff on that legacy server, this ends up with a lot of small files. Then making backups over a long period to keep track of the history, it seems that the APFS Core Storage Logical Volume structures were not sufficient and the reorganisation of the disk may have been been started already by Apple, but without notification while a backup was in progress. This leads to conflicting operations and disk trashing head movement.

Fortunately TimeMachine can usually take on on disk operations after breaks better than usually reported when using APFS.

Conclusion

The First Aid managed to resize Core Storage Logical Volume structures and grow the file system.

Can you rely on this disk after repair?

1. We use a special device driver to run smartctl on external USB drives
2. The lifecycle was reported with a bit more than 11000 hours equaling around 400 days and a bit more of a year of running time. The disk is stored in a safe place in rotation with 2 other media (2 attached, one on a different location)
3. Due to the fact that the issue was due to software crash, we expect the hardware of the drive as OK.
4. The repair had an obvious fact causing the issue.
5. The repair was successful
6. smartctl extended report did not show any recorded errors at any time on that drive
7. We have 3 media in rotation
8. Our decision is just to add one more drive soon and keep an eye on that until the next drive has enough generations.
9. We keep that drive and maybe it can save our life in a situation the others fail.

Addendum

For those who are interested to run extended smartctl operations on external drives

(do not do this at home ;-):

We use:

1. Homebrew and installed `brew install smartmontools`
2. We installed the free External USB / FireWire drive diagnostics support Driver suggested from e.g. https://binaryfruit.com/drivedx/usb-drive-support on our own risk!

If you want to perform external drive diagnostics on macOS – currently there is only one option – you can install 3rd party kernel extension – SAT SMART Driver. SAT SMART Driver is free open source project (published under Apple Public Source License) by Jarkko Sonninen. It is hosted on GitHub.