Diagnosing cause of kernel panic on a Mac Pro

See Pierre Igot's previous posts on this thread. He cites a supplemental update for 10.8.2 that may help, and also a suggestion to use an older version of a particular driver is mentioned on his blog. Read everything before you do anything.

cryptomagic:

I just went through the System/Library/Extensions/ folder and identified the kext files that might or might not have to do with our current predicament. My focus remains on these files:

IOGraphicsFamily.kext

IONDRVSupport.kext

IOPCIFamily.kext

since they were the ones that were changed in OS X 10.8.2, when we started getting these KPs. But it could very well be that the problem is a complex one involving interactions between several different files, so who knows?

Like I've said before, this bug is completely unpredictable and sometimes does not occur for days, then occurs several times in a single day, with no rhyme or reason. But it's really shaking my faith in the reliability of Apple's systems. It's unacceptable to have such a serious bug that remains unaddressed several months after it was first uncovered and after it's been a "known issue" for quite a while. I don't want to have to wait until 10.8.3 is out. I want a special fix for this particular problem NOW. This is a **crashing** bug that affects a number of people using their expensive machine perfectly normally, with no weird third-party interactions or incompatiblities.

Apple is really dropping the ball here.

Answers to your other questions:

I don't think the CUDA updates do anything (good or bad). CUDA is only used by some specific applications. I had the KPs before I had even heard of CUDA, let alone tried to install it. (I did, just in case. Didn't make any difference.) The KPs don't require the use of CUDA. I don't think it's involved at all.

As NVIDIA, I submitted a bug report directly to them back on Sept. 20 via http://nvidia-submit.custhelp.com. Never heard a peep from them. Haven't tried the forums.

This seems not to be limited to the Mac Pro, because I have been experiencing similar kernel-panics on my iMac 24 inch (early 2009) which also has the NVIDIA Geoforce GT 130 graphics.

All of my panic reports include reference to the "NVDAResman(8.0)" and the three files you mentioned above:

IOGraphicsFamily.kext

IONDRVSupport.kext

IOPCIFamily.kext

I didn't notice the panics until sometime after the 10.8.2 update, and I'm not sure how many days after; I don't remember when I installed 10.8.2. I believe the panics started after the iTunes 11 update, but not sure if that could be related.

In the beginning, the panics only occurred with an automatic restart during cold startup in the morning, and (not watching the startup) I first noticed it when the alert to report to Apple appeared. On perhaps two other occasions, the panic occurred during other operations, once during Safari usage, and another while unattended. Both of these others also included an automatic restart.

My troubleshooting is quite slow, because I can only reproduce the KP on demand in the morning after being shut down all night. I did reboot one morning from a Lion partition with no problem. Also, startup in Single User mode seems to work OK. I have two backup drives with 10.8.2, so I'm considering reinstalling 10.8 and updating only to 10.8.1, which never had this probem.

So, I'm monitoring this discussion with interest. Appreciate your efforts.

Reinstalled Mac OS X 10.8. Updated to OS X 10.8.1. No kernel panics since.

For what its worth, my Mac Pro Mountan LIon freeze/kernel-panic nightmare is much worse when trying to run either VMWare or Parallels Windows VMs. I've got 3 GT120s so tag this under the Mountain Lion cant deal with Mac Pros running multiple video cards files.

My kernel panics ceased after I cleaned out four years of dust and reseated RAM.

Pierre, I agree with your CUDA assessment. Could you link me directly to your posting on the NVidia forum so I can add my 2 cents. Perhaps a few more comments on that thread might wake someone up over there.

FWIW, my panics are pretty spread out now. Here is my complete history

09.29.12 - KP every ~6 hours until I rolled back to 2.3.4 in /System/Library/Extensions/IOGraphicsFamily.kext

10.11.12 - KP

10.15.12 - KP

10.29.12 - KP

11.06.12 - KP so I upgraded NVidia drivers and installed CUDA package

11.26.12 - KP so I updated the CUDA package again and I reverted to 2.3.5 of IOGraphicsFamily.kext

01.02.13 - KP

FWIW, I'm going to replace all my RAM in the next couple days as part of a system upgrade. Maybe I'll get lucky like Dick?

cryptomagic:

I didn't post anything on the forum, I only submitted a bug report at http://nvidia-submit.custhelp.com. I had to register first and the bug report is tied to my account, so I don't believe I can give you access to it. It has a reference no of "120920-000178" and is dated Sept. 20, 2012. Still no response from NVIDIA, of course.

You are lucky you're not getting more KPs. :-)

I haven't had a single one since switching to connecting the two monitors to the same GeForce GT120 with a DVI-to-MiniDisplayPort adapter for the second one. Of course, it means I have half the VRAM I normally should have (there is a noticeable degradation in graphic performance) and I also feel that the GeForce card that is driving both monitors is now hotter and noisier :-(.

Pierre

I post a second bug report #130118-000210

That's a bummer having to put both screens on one card.

I'm not totally out of the woods. I seem to get a KP ever two weeks. Today I gave my computer a tune up

1. I popped out all 3 of my video cards, dusted them off and re-seated them.
2. I upgraded from 8G to 16G of RAM.
3. I cleaned all the fan housings to be free of dust

Maybe it will be happier after some love.

I could probably live with one every two weeks. But two or more a day is not an option.

I find it rather doubtful that clearing the dust would make any difference, as I haven't done any cleaning myself and yet with both screens on the same card (EITHER card), I get zero KPs.

It's a software bug, and it won't disappear until Apple fixes its software. Given how long it's already been with us, the signs are not encouraging.

Thanks for the updates.

I agree. I've given up tinkering. This is a software bug and no permutations of fiddling with settings will fix it. I give apple until 10.8.3 to fix it. Otherwise I sadly have to abandon OSX and go to Windows since I need multimonitor functionality. Pretty depressing.

Hey, no fair dispelling my self-delusions. After all Nvidia just sent me this nice email:

Nvidia Submit Customer Care <nvidia_submit@mailca.custhelp.com>

Jan 18 (1 day ago)

to chris

Thank you for your submittal to the NVIDIA Product Feedback Page.

We will carefully review your submission and may contact you for further information if needed as we attempt to reproduce your issue.

Sincerely,

NVIDIA Customer Care

I just want them to tell me if the C++ file that throws the exception is their's or Apple's

I have this KP with my Mac Pro and 3 monitors only when he wake up. Not all wake up but 2 or 3 times per day.

Interval Since Last Panic Report: 47605 sec

Panics Since Last Report: 3

Anonymous UUID: B21FBBDE-4735-885F-8E1C-8148B2131709

Fri Jan 25 03:53:55 2013

panic(cpu 1 caller 0xffffff8017eb7bd5): Kernel trap at 0xffffff7f995c7032, type 14=page fault, registers:

CR0: 0x000000008001003b, CR2: 0x0000000000000000, CR3: 0x000000001a7d1000, CR4: 0x0000000000000660

RAX: 0xffffff7f995c7026, RBX: 0x0000000000000000, RCX: 0x0000000000000000, RDX: 0x0000000000000000

RSP: 0xffffff80d01c3eb0, RBP: 0xffffff80d01c3ec0, RSI: 0x0000000000000000, RDI: 0xffffff802a014000

R8: 0x0000000000000001, R9: 0xffffff80c37fb9b0, R10: 0x00000000023ee803, R11: 0x0000000000000206

R12: 0x0000000000161644, R13: 0x0000000000000000, R14: 0x0000000000000000, R15: 0x0000000000000001

RFL: 0x0000000000010286, RIP: 0xffffff7f995c7032, CS: 0x0000000000000008, SS: 0x0000000000000010

Fault CR2: 0x0000000000000000, Error code: 0x0000000000000000, Fault CPU: 0x1

Backtrace (CPU 1), Frame : Return Address

0xffffff80d01c3b50 : 0xffffff8017e1d626

0xffffff80d01c3bc0 : 0xffffff8017eb7bd5

0xffffff80d01c3d90 : 0xffffff8017ece4ed

0xffffff80d01c3db0 : 0xffffff7f995c7032

0xffffff80d01c3ec0 : 0xffffff7f99c12cea

0xffffff80d01c3ef0 : 0xffffff80182472a8

0xffffff80d01c3f30 : 0xffffff8018245daa

0xffffff80d01c3f80 : 0xffffff8018245ed9

0xffffff80d01c3fb0 : 0xffffff8017eb26b7

Kernel Extensions in backtrace:

com.apple.AMDRadeonAccelerator(1.0)[D89E4100-9C89-3C99-B34B-F7790E1A0B8B]@0xfff fff7f99c0c000->0xffffff7f99fd9fff

dependency: com.apple.iokit.IOAcceleratorFamily(19.0.26)[7DC411F1-A646-3A4A-B160-A1492F5EC0 ED]@0xffffff7f99bba000

dependency: com.apple.iokit.IOPCIFamily(2.7.2)[B1B77B26-7984-302F-BA8E-544DD3D75E73]@0xffff ff7f984d4000

dependency: com.apple.iokit.IONDRVSupport(2.3.5)[86DDB71C-A73A-3EBE-AC44-0BC9A38B9A44]@0xff ffff7f98a88000

dependency: com.apple.iokit.IOGraphicsFamily(2.3.5)[803496D0-ADAD-3ADB-B071-8A0A197DA53D]@0 xffffff7f98a45000

com.apple.kext.AMDFramebuffer(8.0)[4CD59A3C-4442-36CB-8026-2095FE6B032A]@0xffff ff7f995bf000->0xffffff7f995d1fff

dependency: com.apple.iokit.IOACPIFamily(1.4)[A35915E8-C1B0-3C0F-81DF-5515BC9002FC]@0xfffff f7f989c9000

dependency: com.apple.iokit.IOPCIFamily(2.7.2)[B1B77B26-7984-302F-BA8E-544DD3D75E73]@0xffff ff7f984d4000

dependency: com.apple.kext.AMDSupport(8.0.0)[7B2EC6D1-101A-3928-919E-337D6724752A]@0xffffff 7f993d5000

dependency: com.apple.iokit.IOGraphicsFamily(2.3.5)[803496D0-ADAD-3ADB-B071-8A0A197DA53D]@0 xffffff7f98a45000

BSD process name corresponding to current thread: kernel_task

Mac OS version:

12C60

Kernel version:

Darwin Kernel Version 12.2.0: Sat Aug 25 00:48:52 PDT 2012; root:xnu-2050.18.24~1/RELEASE_X86_64

Kernel UUID: 69A5853F-375A-3EF4-9247-478FD0247333

Kernel slide: 0x0000000017c00000

Kernel text base: 0xffffff8017e00000

System model name: MacPro5,1 (Mac-F221BEC8)

System uptime in nanoseconds: 12256052900275

last loaded kext at 7359859637769: com.apple.driver.AppleUSBCDC 4.1.22 (addr 0xffffff7f9a21d000, size 16384)

last unloaded kext at 7466051955362: com.apple.driver.AppleUSBCDC 4.1.22 (addr 0xffffff7f9a21d000, size 12288)

loaded kexts:

com.parallels.kext.prl_vnic 7.0 15107.796624

com.parallels.kext.prl_netbridge 7.0 15107.796624

com.parallels.kext.prl_hid_hook 7.0 15107.796624

com.parallels.kext.prl_hypervisor 7.0 15107.796624

com.parallels.kext.prl_usb_connect 7.0 15107.796624

com.Cycling74.driver.Soundflower 1.5.2

com.apple.filesystems.afpfs 10.0

com.apple.nke.asp_tcp 7.0.0

com.apple.filesystems.smbfs 1.8

com.apple.driver.AppleHWSensor 1.9.5d0

com.apple.driver.AppleBluetoothMultitouch 75.15

com.apple.filesystems.ntfs 3.10

com.apple.driver.AppleTyMCEDriver 1.0.2d2

com.apple.driver.AGPM 100.12.69

com.apple.driver.AppleMikeyHIDDriver 122

com.apple.driver.AppleHDA 2.3.1f2

com.apple.filesystems.autofs 3.0

com.apple.driver.AudioAUUC 1.60

com.apple.iokit.IOUserEthernet 1.0.0d1

com.apple.driver.AppleMikeyDriver 2.3.1f2

com.apple.driver.AppleUpstreamUserClient 3.5.10

com.apple.driver.AppleMCCSControl 1.0.33

com.apple.iokit.IOBluetoothSerialManager 4.0.9f33

com.apple.Dont_Steal_Mac_OS_X 7.0.0

com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport 4.0.9f33

com.apple.driver.ACPI_SMC_PlatformPlugin 1.0.0

com.apple.driver.AppleLPC 1.6.0

com.apple.kext.AMDFramebuffer 8.0.0

com.apple.driver.ApplePolicyControl 3.2.11

com.apple.AMDRadeonAccelerator 1.0.0

com.apple.driver.XsanFilter 404

com.apple.iokit.IOAHCIBlockStorage 2.2.2

com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1

com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1

com.apple.BootCache 34

com.apple.iokit.SCSITaskUserClient 3.5.1

com.apple.driver.AppleUSBHub 5.2.5

com.apple.driver.AppleFWOHCI 4.9.6

com.apple.driver.Intel82574L 2.3.0b4

com.apple.driver.AirPort.Brcm4331 602.15.22

com.apple.driver.AppleAHCIPort 2.4.1

com.apple.driver.AppleUSBEHCI 5.4.0

com.apple.driver.AppleUSBUHCI 5.2.5

com.apple.driver.AppleEFINVRAM 1.6.1

com.apple.driver.AppleRTC 1.5

com.apple.driver.AppleHPET 1.7

com.apple.driver.AppleACPIButtons 1.6

com.apple.driver.AppleSMBIOS 1.9

com.apple.driver.AppleACPIEC 1.6

com.apple.driver.AppleAPIC 1.6

com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0

com.apple.nke.applicationfirewall 4.0.39

com.apple.security.quarantine 2

com.apple.driver.AppleIntelCPUPowerManagement 196.0.0

com.apple.security.SecureRemotePassword 1.0

com.apple.driver.IOBluetoothHIDDriver 4.0.9f33

com.apple.driver.AppleMultitouchDriver 235.28

com.apple.driver.DspFuncLib 2.3.1f2

com.apple.kext.triggers 1.0

com.apple.iokit.IOAudioFamily 1.8.9fc10

com.apple.kext.OSvKernDSPLib 1.6

com.apple.iokit.IOSurface 86.0.3

com.apple.driver.AppleSMBusController 1.0.10d0

com.apple.iokit.IOFireWireIP 2.2.5

com.apple.iokit.IOSerialFamily 10.0.6

com.apple.iokit.IOBluetoothFamily 4.0.9f33

com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.0.9f33

com.apple.driver.AppleSMBusPCI 1.0.10d0

com.apple.driver.IOPlatformPluginLegacy 1.0.0

com.apple.driver.IOPlatformPluginFamily 5.2.0d16

com.apple.driver.AppleHDAController 2.3.1f2

com.apple.iokit.IOHDAFamily 2.3.1f2

com.apple.driver.AppleGraphicsControl 3.2.11

com.apple.driver.AppleSMC 3.1.4d2

com.apple.iokit.IOAcceleratorFamily 19.0.26

com.apple.iokit.IONDRVSupport 2.3.5

com.apple.kext.AMD5000Controller 8.0.0

com.apple.kext.AMDSupport 8.0.0

com.apple.iokit.IOGraphicsFamily 2.3.5

com.apple.driver.AppleUSBHIDKeyboard 165.5

com.apple.driver.AppleHIDKeyboard 165.5

com.apple.iokit.IOUSBHIDDriver 5.2.5

com.apple.driver.AppleUSBMergeNub 5.2.5

com.apple.driver.AppleUSBComposite 5.2.5

com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.5.1

com.apple.iokit.IOBDStorageFamily 1.7

com.apple.iokit.IODVDStorageFamily 1.7.1

com.apple.iokit.IOCDStorageFamily 1.7.1

com.apple.iokit.IOAHCISerialATAPI 2.5.0

com.apple.iokit.IOSCSIArchitectureModelFamily 3.5.1

com.apple.iokit.IOFireWireFamily 4.5.5

com.apple.iokit.IOUSBUserClient 5.2.5

com.apple.iokit.IO80211Family 500.15

com.apple.iokit.IONetworkingFamily 3.0

com.apple.iokit.IOAHCIFamily 2.2.1

com.apple.iokit.IOUSBFamily 5.4.0

com.apple.driver.AppleEFIRuntime 1.6.1

com.apple.iokit.IOHIDFamily 1.8.0

com.apple.iokit.IOSMBusFamily 1.1

com.apple.security.sandbox 220

com.apple.kext.AppleMatch 1.0.0d1

com.apple.security.TMSafetyNet 7

com.apple.driver.DiskImages 344

com.apple.iokit.IOStorageFamily 1.8

com.apple.driver.AppleKeyStore 28.21

com.apple.driver.AppleACPIPlatform 1.6

com.apple.iokit.IOPCIFamily 2.7.2

com.apple.iokit.IOACPIFamily 1.4

com.apple.kec.corecrypto 1.0

Model: MacPro5,1, BootROM MP51.007F.B03, 4 processors, Quad-Core Intel Xeon, 3.2 GHz, 6 GB, SMC 1.39f11

Graphics: ATI Radeon HD 5770, ATI Radeon HD 5770, PCIe, 1024 MB

Memory Module: DIMM 1, 2 GB, DDR3 ECC, 1066 MHz, 0x802C, 0x394A53463235363732415A2D314734443120

Memory Module: DIMM 2, 2 GB, DDR3 ECC, 1066 MHz, 0x802C, 0x394A53463235363732415A2D314734443120

Memory Module: DIMM 3, 2 GB, DDR3 ECC, 1066 MHz, 0x802C, 0x394A53463235363732415A2D314734443120

AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x8E), Broadcom BCM43xx 1.0 (5.106.98.81.22)

Bluetooth: Version 4.0.9f33 10885, 2 service, 18 devices, 1 incoming serial ports

Network Service: Wi-Fi, AirPort, en2

PCI Card: ATI Radeon HD 5770, sppci_displaycontroller, Slot-1

Serial ATA Device: HL-DT-ST DVD-RW GH80N

Serial ATA Device: Hitachi HUA722010CLA330, 1 TB

Serial ATA Device: ST3802110AS, 80,03 GB

Serial ATA Device: MAXTOR STM3160815AS, 160,04 GB

USB Device: iPad, apple_vendor_id, 0x12a4, 0xfd100000 / 2

USB Device: Keyboard Hub, apple_vendor_id, 0x1006, 0xfa200000 / 2

USB Device: Apple Keyboard, apple_vendor_id, 0x0250, 0xfa220000 / 3

USB Device: BRCM2046 Hub, 0x0a5c (Broadcom Corp.), 0x4500, 0x5a100000 / 2

USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8215, 0x5a110000 / 3

FireWire Device: built-in_hub, 800mbit_speed

arkange-

That is a Page Fault (often a reference to memory that does not exist). It occurred in code called by the Display Drivers.

The Mac OS X kernel alone does not tend to panic in this way. But you have added a number of kernel extensions which may be making the problem worse, or may even be the cause of the problem.

com.parallels.kext.prl_vnic 7.0 15107.796624

com.parallels.kext.prl_netbridge 7.0 15107.796624

com.parallels.kext.prl_hid_hook 7.0 15107.796624

com.parallels.kext.prl_hypervisor 7.0 15107.796624

com.parallels.kext.prl_usb_connect 7.0 15107.796624

com.Cycling74.driver.Soundflower 1.5.2

At a minimum, you need to be absolutely certain that all these extensions are completely appropriate for the version of Mac OS X you are running, and that they are the latest versions.

Apple will not commit substantial resources to your problem unless/until these problems occur with no third-party extensions present.

Why is your iPad showing as connected when you say your Mac was asleep?