Persistant autofs crash - 10.5.6

Given both panics are due to a dereference of the memory location 0x38, it's pretty clear that somewhere a pointer is being zeroed that of course should not have been which definitively points the finger at the autofs kext.

However, the question of what and why is a bigger issue I can't answer.

You had mentioned adding printfs, so if you want to pursue this further you can download the source code for autofs and appropriate kernel debug kit and follow the procedure in Apple Technical Note 2063: Understanding and Debugging Kernel Panics if you want to pursue this further.

I'll try to do so as well when I have a chance but I can't promise when that might be.

OK, it bugged me, so I dug into it.

This really isn't going to do you any good, but the panic is due to a NULL dereference here (using the PowerPC code in this case):

0x0082bfa8 <auto_lookup+768>: lwz r2,60(r31)
0x0082bfac <auto_lookup+772>: lwz r3,56(r2)

Note that 56 decimal is 0x38 hex, and your panics indicate a dereference of memory location 0x38:

Unresolved kernel trap(cpu 3): 0x300 - Data access DAR=0x0000000000000038 PC=0x000000000082BFAC

which in this case means r2 was loaded with the value 0.

This translates into line 619 of autofs-109.1/kext/auto_vnops.c:

vnodeisvroot(fntovn(dfnp->fnparent))) {

fntovn is #defined as:

#define fntovn(fnp) (((fnp)->fn_vnode)) in autofs-109.1/kext/autofs_kern.h, so that means dfnp->fn_parent is 0.

How or why is unknown; fn_parent should only be NULL (0) when the autofs node has been freed (which corresponds nicely to your not having any NFS nodes mounted at the time you panic), so something is causing Mac OS X to call auto_lookup() for a freed vnode.

I don't know where the problem lies - in Apple specific code or some type of lock issue - but if you look at the header for the file you'll see Apple got it from OpenSolaris, which does not suffer from a similar issue, and contains essentially the same code construct at line 526:

526 ((fntovn(dfnp->fnparent))->vflag & VROOT)) {

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/auto fs/auto_vnops.c#526

So, bottom line and mea culpa, it does in fact appear to be a Mac OS X kernel bug, either in the kext or elsewhere in their filesystem code.

Feel free to quote this message in your next bug report to Apple. 🙂

(Yes, all this was analysis was made possible by your posted panic.log files.)

Wow. That's a very impressive piece of debugging. I think multi-machine kernel debugging is probably a bit more involved than I want to get into. I will submit a bug to Apple and generously quote your analysis. The good news is that you have confirmed that just disabling autofs is probably a reasonable workaround until Apple fixes this (if they ever do).

Thanks for all of your help!

--tom

BTW, no multi-machine debugging was needed for this; the analysis was all done just from the data in your posted panic.log.

Ladies and Gentlemen,

I just got a bug report from a java application on an Intel machine. So it seems not only to related to PPCs.

panic(cpu 1 caller 0x001A9C68): Kernel trap at 0x8dc4da17, type 14=page fault, registers:
CR0: 0x8001003b, CR2: 0x00000038, CR3: 0x00ce9000, CR4: 0x00000660
EAX: 0x00000000, EBX: 0x00000002, ECX: 0x21000001, EDX: 0x24859f80
CR2: 0x00000038, EBP: 0x8f4ab998, ESI: 0x16708d04, EDI: 0x00000000
EFL: 0x00010246, EIP: 0x8dc4da17, CS: 0x00000008, DS: 0x194e0010
Error code: 0x00000000
Backtrace (CPU 1), Frame : Return Address (4 potential args on stack)
0x8f4ab748 : 0x12b4f3 (0x45b13c 0x8f4ab77c 0x1335e4 0x0)
0x8f4ab798 : 0x1a9c68 (0x464700 0x8dc4da17 0xe 0x463eb0)
0x8f4ab878 : 0x1a038d (0x8f4ab890 0x2bc3f2e8 0x8f4ab998 0x8dc4da17)
0x8f4ab888 : 0x8dc4da17 (0xe 0x48 0x8f4a0010 0x16110010)
0x8f4ab998 : 0x1f2c7d (0x8f4ab9b8 0x2 0x8f4ab9e8 0x451c8a)
0x8f4ab9e8 : 0x1d464c (0x24859f80 0x8f4abdf4 0x8f4abf08 0x18f30844)
0x8f4aba78 : 0x1d53d1 (0x8f4abddc 0x15d25412 0x1 0x400)
0x8f4abb38 : 0x1e4060 (0x8f4abddc 0x8f4aba34 0x6 0x9675461f)
0x8f4abd88 : 0x1e4484 (0xb14276e4 0x0 0x0 0x0)
0x8f4abf48 : 0x1e4567 (0xb14276e4 0x0 0x0 0x0)
0x8f4abf78 : 0x3df460 (0x194e5130 0x18f30740 0x18f30784 0xffffffff)
0x8f4abfc8 : 0x1a0887 (0x163ec880 0x0 0x1a30b5 0x163ec880)
No mapping exists for frame pointer
Backtrace terminated-invalid frame pointer 0xb1427758
Kernel loadable modules in backtrace (with dependencies):
com.apple.filesystems.autofs(2.0.1)@0x8dc4b000->0x8dc55fff
BSD process name corresponding to current thread: java
Mac OS version:
9G55
Kernel version:
Darwin Kernel Version 9.6.0: Mon Nov 24 17:37:00 PST 2008; root:xnu-1228.9.59~1/RELEASE_I386
System model name: MacPro3,1 (Mac-F42C88C8)