RAID array offline, disk IO error

Question

RAID array offline, disk IO error

Fun.

A drive died in one of the raids, which I replaced. Then the entire array went offline. Instead of doing the normal 'rebuilding array'. Then I got some IO errors in my logs and Xsan misbehaving.

So, is this offline array complaining of a bad replacement drive? or is it a controller? hard to tell. Xraid Admin says all is good, except for the "raid offline" which is not good.

Hmm...

Posted on Oct 12, 2007 10:41 PM

Reply

Answer 1

Oct 13, 2007 1:07 AM in response to Mathieu Mauser

Restarted the Xserve RAID which had the offline array. It came back up saying the just replaced drive had failed, so we put in yet another new one. Still said drive failed. But now the array was "degraded" instead of offline. The Xsan volume mounted on two of three clients we tried. Hmm.. Starting to think it could be a controller or some bad hardware. No bad drives. (That would be a lot of bad drives).

Reply

Answer 2

Oct 13, 2007 2:16 AM in response to Mathieu Mauser

Plugging in a "real" Apple Xserve RAID module seemed to bring the "degraded" array back to life. Now it's rebuilding the array, as it should. It didn't like our cheap ATA drives we bought. No, sir.

There's a lesson in there somewhere. Hmm.. Don't buy cheap drives? Always backup. Backup often. Backup everything and you'll worry less? Something like that.

Reply

Answer 3

Oct 15, 2007 2:00 AM in response to Mathieu Mauser

Was you new disk bigger than the old one?
you can not plugin a 3rd party disk with the same size.

In some threads this is explained as follows:

apple puts its own firmware on the disk. This firmware gives some of the blocks that are 'spare' for the upcoming bad blocks to the normal blocks.
An apple disk is slightly bigger than the equivalent IBM or Hitachi disk.

Off course I can not check this. I do know from experience that disks of the same size do not work ;-(

Hope this helps
Donald

Reply

Answer 4

Oct 15, 2007 9:51 AM in response to Donald Kok

Wow. That's the first I ever heard of that. Has to be bigger, eh? Gosh darn. I've was very hesitant to try a 3rd party drive in the Xserve RAID, but the price difference ($500) and an the Apple retailer who sold it to me claimed there'd be no problem. The drive I inserted was the same size. So a bigger one would work better... Crazy. Might have to test that out. One of these days.

Reply

Answer 5

Oct 17, 2007 10:39 AM in response to Mathieu Mauser

Because of the way the array is set up (parity, striping, etc.) Raid-5 requires that all disks be "the same size". I put that in quotes, because while the RAID controller doesn't require that all the disks be the same, it bases the total array size on the smallest disk present in the array. If you put six 400GB and one 250GB disk into a seven-disk Raid-5, the total raw volume size will be 250x7 GB, and not 400x6 + 250.

So, when you replace a drive in a Raid-5, you need to replace it with a drive that is >= the smallest drive in the array.

The problem is, as was indicated previously, the Apple drive firmware reports a slightly larger drive size than a standard, off-the-shelf one. Your off-the-shelf 400 GB drive may report a capacity of 400 GB, but the Apple drive will report something slightly larger--400.1 or something.

So, you can use off-the-shelf drives in an Xserve RAID (but there's no Apple support, functionality not guaranteed, etc.), but if you have an array built from Apple drives, and try to replace an Apple-firmware drive with a non-Apple drive of the same size, the Raid controller will see the non-Apple drive as being slightly too small, and will refuse to use it. You can replace it with a larger off-the-shelf drive (500 GB instead of 400 GB, say) and it will (should!) work. But you're wasting 100 GB of disk space.

Check the Xserve RAID forum for discussions on non-Apple firmware drives--should be informative.

Hope this helps!

-Josh

Reply