Mac Pro 2023 will not boot with PCIe SSD installed.

We have seen about 10 instances. All have the following in the top of the panic trace:

dart-apciec3 (0xfffffe1b39d40800): DART(DART) error:

This is the same error as MacOS/M1 had up until 13.4 was released, which fixed this problem.

In case this is a MacOS issue, can you post the MacOS build number of your System? I am especially interested in build numbers for MacOS versions that are NOT crashing.

So far, I have seen this with RAID 0, RAID 4, RAID 5. All with 4 drives. This panic does not happen with RAID 0 2-drive volumes.

Apple RAID does not seem to be affected.

HFS and APFS will both panic a system, so it is not file system related.

One trick that MIGHT work, I would like any affected users to try this, is with the drives disconnected, launch SoftRAID and "uninstall SoftRAID" from utiliities menu. restart, and connect the drives. (or put them back in the Mac Pro)

Does this prevent the panic?

Your volume will still mount and be usable.

Last update from me. Not sure why I didn't do it until after 30 hours, but I used Raid Assistant in MacOs and both my 4M2 8TB drives are working in raid 0 now with no issues.

100% the issue is connected to using SoftRaid, or an external application.

16TB's SSD are now fully working and no issues booting. My HDD via the Pegasus never had any issues, but are not setup in a Raid format.

Hey OP, I'm the product manager for internal storage at OWC. We are monitoring this thread and did discover an OS-level issue that causes M2 Mac Studios and M2 Mac Pros to restart whenever four or more SoftRAID NVMe blades are attached. We published an official update yesterday that outlines the problem and a fix (full post: tinyurl.com/2xotar5b).

TL;DR:

- There is a bug in macOS 13.4 unique to the 2023 M2 Mac Studio and Mac Pro that causes the computers to restart whenever four or more SoftRAID NVMe blades are attached

- We have published a temporary fix that works immediately

- We are confident that Apple engineers will have a new version of macOS which fixes this problem in the very near future

I'm happy to answer any follow-up questions, but rest assured our team here is hunkered down and doing everything we can to help our customers and minimize impact to their workflows.

Here is where we are with this bug.

We have discovered a bug with the new M2 Mac Studio and Mac Pro, when running macOS 13.4, which makes most NVMe blades unusable with SoftRAID.  We have reported this problem to Apple and have been actively investigating possible solutions.  Until Apple delivers a fix in future release of macOS, we suggest that users of these new Macs use Apple's Disk Utility to configure their NVMe blades as RAID 0 volumes.

This problem only affects users of the 2023 Mac Studio and 2023 Mac Pro.

The other thing I can add is this primarily affects NVMe blades with 4K sector size, which is why some enclosures work and some do not.

If I were to guess, this may be fixed in 13.5, when released.

I have a similar set up as the original post and the exact same problems except using a 2023 Mac Studio M2 Ultra with a Sonnetech PCI chassis hosting the cards. WHenever the 4m2 was plugged in to the chassis the computer would kernal panic and shut down on boot.

I reformatted the 4m2 using the apple raid format and it works fine for now with no kernel panics. After I got it working, I tried, for experimentation, to use softraid to convert the drive to softraid format and suffered an immediate kernal panic and shut down. Luckily, I was another to use another computer with softraid to convert the drive back to Apple Raid with no loss of data. Thus, until a fix is found, I will probably leave the drive as is.

<<HEY ALL>> Try something here. Look at your SoftRaid formatting on the drives that don't work and check your block size. A friend of mine just sent me this. I'm going to do an experiment and report back asap.

Is the card formatted to use 64kb block size? The issue looks a lot like a problem with the I/O system that others experienced a year ago on the first M2 laptops and from what I’ve seen there’s anecdotal evidence it’s caused by formatting the drive with a suboptimal format.

Basically, the I/O on the M2 is so fast that it can cause issues if the card isn’t formatted for the massive I/O that’s happening. The hardware can deadlock or the massive I/O rate can expose bugs in the firmware that won’t happen on older machines.

Lukiffer

If you run "uninstall SoftRAID", do your cards work?

(no need to delete the app just "Uninstall SoftRAID - all components)

What is your MacOS build number.

I am collecting data on this issue.

My assumption (based on the “workaround” posted earlier of zeroing the disks and recreating the RAID set) would be that indicates the issue has something to do with what’s on the disk rather than the disk hardware itself. My further assumption based on the kernel panic stack trace is that it has something to do with the kext’s (or the system API’s) interpretation of the RAID config. This is of course conjecture without actually having source code visibility, but find it interesting that no one has reported having this issue with Disk Utility (native) RAID sets.

After wasting 30+ hours testing this with the same setups you all have, I’m done. I found that just keeping each SSD formatted by Apple in recovery and not using OWC makes it bootable. The moment I do anything with the SoftRaid program, it’s not rebootable.

Just going to use each ssd separately until a resolution is made on here.

Ventura OS 13.5 (22G74) released a few hours ago.. Updated my M2Ultra tower and Initialized my new 4M2 with Softraid 7.5. So far, all is working.. Please do post back if anyone can confirm their rig is working (or not!) with this new OS update. thanks all.

Well..it didn't work on my end for the older 1tb/4M2. But, something to think about regarding Optimizing the OWC card via Soft Raid (according to a tech friend of mine):

With unified memory the new chips go from making thousands of 16kb reads per second to make tens of millions, and that could overload the firmware of the cards. So using larger block sizes will cut the number of read operations down considerably. Also, there’s also some anecdotes that switching to RAID 1+0 fixes a very similar looking kernel panic. It likely works because it cuts the I/O throughput in half for the striping operation.

This is starting to get over MY head.. but I'm planning on trying more experiments (and hopefully there might have just been an issue with the chipset in the newer OWC card...But setting the block size to 64 makes sense (since I'm doing mostly ProTools and Logic with my machine).

==========================

From the SoftRaid help file

Optimized for Workstation:

Rebuilding mirrors and validating volumes will proceed at full speed unless any SoftRAID volume is reading or writing files. If a SoftRAID volume is reading or writing files, the rebuild or validate will only proceed during 50% of the time (0.5 sec rebuild, 0.5 sec rebuild paused, 0.5 sec rebuild, etc). The default stripe unit size when creating a new stripe volume is set to the best value for general use. 

Optimized for Server:

Rebuilding mirrors and validating volumes will proceed at full speed unless any SoftRAID volume is reading or writing files. If a SoftRAID volume is reading or writing files, the rebuild or validate will only proceed during 75% of the time (0.75 sec rebuild, 0.25 sec rebuild paused, 0.75 sec rebuild, etc). The default stripe unit size when creating a new stripe volume is set to the best value for the small i/o sizes commonly found on database, web and mail servers. 

Optimized for Digital Video:

Rebuilding mirrors and validating volumes will proceed at full speed unless any SoftRAID volume is reading or writing files. If a SoftRAID volume is reading or writing files, the rebuild or validate will pause until 15 seconds after the last file read or write operation. The default stripe unit size when creating a new stripe volume is set to the largest size possible for the disks used by the volume. This will offer the highest throughput for large files and uncompressed video. 

Optimized for Digital Audio:

Rebuilding mirrors and validating volumes will proceed at full speed unless any SoftRAID volume is reading or writing files. If a SoftRAID volume is reading or writing files, the rebuild or validate will pause until 15 seconds after the last file read or write operation. The default stripe unit size when creating a new stripe volume is set to the smallest size which SoftRAID supports. This ensures that disk i/o does not starve the buffers on some digital audio PCI cards. 

Optimized for Digital Photography:

Rebuilding mirrors and validating volumes will proceed at full speed unless any SoftRAID volume is reading or writing files. If a SoftRAID volume is reading or writing files, the rebuild or validate will pause until 5 seconds after the last file read or write operation. The default stripe unit size when creating a new stripe volume is set to 128 KB. This will offer high throughput for large files and still provide good performance with PhotoShop which reads and writes files in smaller blocks.