panic: thread_call group 'high' reached max thread cap (500): active: 1, blocked: 499, idle: 0 @thread_call.c:409

I am not doing anything unusual. I wasn't using the machine when it karked. Just had a few browser tabs open and a few VScode instances open and a few Terminal.app tabs open plus Emacs. The VScode instances had some attached rust analysis processes, and I imagine various Emacs buffers also had rust analysis processes attached.

Which `sysctl` parameters in particular do you suggest? https://opensource.apple.com/source/xnu/xnu-7195.81.3/osfmk/kern/thread_call.c has a panic at line 409. The code below is an excerpt. In general, manifest constants cannot be adjusted by `sysctl`, so unless there is a newer version of this file which just happens to have the same panic at line 409, but has been changed so as to allow `sysctl` to change `THREAD_CALL_GROUP_MAX_THREADS`, .... I also checked all the `sysctl -a`

and cannot find anything remotely applicable.

#define THREAD_CALL_GROUP_MAX_THREADS   500

...

static bool
thread_call_group_should_add_thread(thread_call_group_t group)
{
    if ((group->active_count + group->blocked_count + group->idle_count) >= THREAD_CALL_GROUP_MAX_THREADS) {
        panic("thread_call group '%s' reached max thread cap (%d): active: %d, blocked: %d, idle: %d",
            group->tcg_name, THREAD_CALL_GROUP_MAX_THREADS,
            group->active_count, group->blocked_count, group->idle_count);
    }

I'd be interested in being able to determine what all 499 blocked threads are blocked on. Clearly, the code expects 500 threads to be sufficient to handle the "high" queue, and rather than just returning `false` and adding the work item, it panic()s.

I'm not at all familiar with this code, so any comments I make regarding it are likely to be rubbish.

I did not use migration assistant. I did (several) full timeMachine backups of old machine onto external SSDs, then setup the new machine, added my user, and copied $HOME from backup into ~/FromLust (`lust` was the old machine) on the new machine. I'll eventually pull things from ~/FromLust as I need them.

At this point, I can't see a strong indication that this has anything to do with kernel extensions. Turns out the `Keybase` kext does have an /arm/ variant:

bash-3.2# pwd
/Library/Filesystems/kbfuse.fs/Contents/Extensions/11/kbfuse.kext/Contents/MacOS
bash-3.2# file *
kbfuse: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit kext bundle x86_64
- Mach-O 64-bit kext bundle x86_64] [arm64e:Mach-O 64-bit kext bundle arm64e
- Mach-O 64-bit kext bundle arm64e]
kbfuse (for architecture x86_64):	Mach-O 64-bit kext bundle x86_64
kbfuse (for architecture arm64e):	Mach-O 64-bit kext bundle arm64e

Nothing really indicates memory failing. I can't figure out if there is ECC support or not in the studio. I have been buying only ECC-based machines for decades now, when possible. My TrashCanPro had 64GB ECC memory, but I never saw any "single bit corrected" messages nor any "uncorrectable error" messages, but I'm sure they would be lost in the torrent of system messages anyway (except the "uncorrectable" one, natch).

I believe Keybase.app has their own MacFuse version. Not sure why.

If I ever reinstall pCloud Drive, it uses MacFuse.

If I ever reinstall CryptoMator, it uses MacFuse.

When I eventually reinstall Borg, I might again want MacFuse, for `borg mount`.

There are probably all sorts of other "drive" apps I might or might not reinstall which would want to use it.

I never want to have much of anything to do with NTFS formatted drives.

Currently, I have turned back on full security and am not loading 3rd party kernel extensions. I use Keybase mainly for the `git` integration with `pass`, so don't care about that.

None of this diversion regarding MacFuse has anything to do with the panic from above.

As I indicated, it would be helpful to display all the "blocked" work items. At panic() time, printing them before the panic() call would make sense. It may be time for me to figure out (again) how to attach a kernel debugger to this box.

If I do ever get a repeat of this panic, I will enable kernel debugging, wait for another panic, and see if I can dump the relevant data structures and, if possible, perform per-thread stack traces for all the dedicated service threads that are blocked. It may be that it is reasonable that they are blocked. Some users of the `thread_call` are for delay timers. Not clear if they are "high" group, (the default group when the call struct is created), but certainly possible to have more than 500 ACK timers, for example, or more than 500 mach port timers, currently active. I see no reason to panic in that case, other than to defer the writing of the code to handle things.

It does seem expensive to me to have one kernel thread dedicated per active timer.

Again: not at all familiar with this code, so anything I say is bound to be rubbish.