Rebuilding 1 TB RAID 1 -- 23 More Days?!?

Question

Rebuilding 1 TB RAID 1 -- 23 More Days?!?

I have two 1 TB drives connected to my iMac (10.6.5) via USB and combined in a 1 TB RAID1. A while back I had to physically move a few things and it was too late when I realized one drive had come disconnected. I tried to use Disk Utility to re-attach it to the RAID, but it had been marked as no good and I could not connect it. I finally had to go through several steps to remove it from the RAID, reformat it, and re-attach it to the RAID.

Then Disk Utility started rebuilding my RAID. That was before Christmas (note the date is now after New Year's). Disk Utility will crash anywhere from 1-4 times a day and when I restart it, I always get a different length of time for how much longer a rebuild will take.

Currently it's not quite 1/2 done and tells me it'll take 23 days and 13 hours to finish rebuilding the RAID.

I seriously was thinking of just wiping the disks, deleting the RAID, and copying everything back onto it, but that means that for the better part of a day I get a bottleneck on my LAN while all the files are copied back to the RAID. That either messes up whatever I'm working on when I need to use the LAN, or it effects Netflix and Instant Watch because of the LAN traffic.

Early in the process, I'd get messages telling me it'd take 35 or 40 or more days to rebuild the RAID.

So why does it take so long, when copying the files, over a 10/100 LAN would take less than 24 hours? Is there a better way to do this?

Thanks for any insight into this weirdness.

iMac, Mac OS X (10.5.6)

Posted on Jan 2, 2011 6:58 PM

Reply

Answer 1

Jeff Donald

Level 6

11,732 points

Jan 3, 2011 2:10 AM in response to HalNineThousand

Sounds like a LAN issue. What speed is the data coming over the LAN. Are the files on the LAN in use by another Application?

Reply

Answer 2

ajduguid

Level 3

650 points

Jan 3, 2011 2:29 AM in response to HalNineThousand

If you owned a NAS device where all the drives are connected via SATA it would still take literally hours to rebuild a RAID - I don't know exactly why it should take so long but if it takes hours on a dedicated box using SATA you can imagine the impact using USB will have where there is a huge difference between burst mode and what it can cope with in terms of continuous transmission.

I don't know what to say I'm afraid except you might be best off with your idea copying the files and wiping the RAID.

Reply

Answer 3

Jan 3, 2011 8:04 AM in response to Jeff Donald

Jeff, I think you misunderstand. The RAID is attached to my iMac via USB, so LAN congestion would not be an issue now.

The reason I brought up the LAN is to point out that wiping it and copying data from scratch would mean heavy LAN congestion for about 16 hours. With just the rebuild, no LAN is involved, just a connection from my iMac to the RAID drives through USB.

Reply

Answer 4

Jan 3, 2011 8:11 AM in response to ajduguid

I've wondered if USB is an issue. That's possible. I don't understand, though, how I could copy 400 - 600 GB of data in less than 18 hours, most of that over a LAN, through my iMac, then over USB, to the RAID and still have it take at least a week to rebuild the drive. I'm also taking into account that copying files to a RAID1 means writing to BOTH drives, while rebuilding means copying from one and writing to the other.

Does anyone know why it takes so long to rebuild a RAID compared to copying the data over to it with rsync?

Another interesting point: the first few days progress was VERY slow, with very little change, but after about a week, the amount finished each day or night is much more than it was at first. I'm measuring progress by the length of the progress bar on the display and, at this point, I'm getting curious to see if the speed of the rebuild will increase even more as it gets closer to completion.

Reply

Answer 5

ajduguid

Level 3

650 points

Jan 3, 2011 8:37 AM in response to HalNineThousand

Sadly I think rebuilding in terms of RAID must be an extremely dumb process right now as I've had it myself where the length of time to rebuild is far out of proportion to what data actually existed on the RAID. I can only assuming it is fastidiously checksumming empty sectors and either nobody thought about optimising rebuilding or I don't understand some really intensive part of the process.

Reply

Answer 6

Jan 3, 2011 8:44 AM in response to ajduguid

At least it's not just me.

I was beginning to wonder if it read one bit at a time, triple-checked the read, then wrote it, then triple-checked the new bit with the old one.

Sorry for the sarcasm, but, yeah, I don't see why or how it can be THIS slow.

Thanks!

Reply

Answer 7

The hatter

Level 9

60,980 points

Jan 3, 2011 8:56 AM in response to HalNineThousand

RAID on USB, as can be attested by many, is a disaster waiting to happen, at least until USB3 is a reality.

Doesn't have the bandwidth, USB has to borrow cpu overhead.

Also, SoftRAID.com has a product that is a more robust and reliable product than Apple RAID.

If you can't use FW800 or eSATA, or even NAS switch, and even an Atom-based NAS server can be slow, I think the best you can do is something like Quad interface, and see what OWC offers.

http://www.macsales.com/firewire as they do have bridge RAID with 0,1,5, with 2-4 drives.

Reply

Answer 8

Jan 3, 2011 9:17 AM in response to The hatter

I'm not thrilled with the USB RAID, but other than this case, it's been working for a backup.

I'll be changing to something else eventually - I hope not more than 3-6 months off. I've heard good things about the D-LINK DNS-321 for NAS.

I have a relative that lives just over a block from me, I have this fantasy of getting wifi routers and cantennas with enough range to have an offsite backup at her place.

Reply

Answer 9

Jan 6, 2011 9:58 AM in response to HalNineThousand

It's about halfway done and I'm going to let it rebuild, even with the long delay for a couple reasons: 1) It's easier than having to re-rsync all the data from every computer on my LAN and deal with congestion, and 2) I'm curious to see just how long it'll take.

I wish I had logged when it started, but I know it was right around Christmas.

What I find interesting is that Disk Utilities freezes a lot of the time, or it crashes sometimes. There are days when I see a LOT of progress and think, "If it kept going at that rate, it could have done the whole thing in 2-3 days." Then there are times when I watch and the status bar doesn't change, making me wonder if the process is stuck.

For example, last night, I checked where the status bar was and went to bed. Over 8 hours later I check and it hasn't moved at all. So I quit Disk Utilities and restart and in under 90 minutes there's a measurable change in the status bar.

Reply

Answer 10

Mar 24, 2011 1:14 PM in response to HalNineThousand

This was never really answered, but in the long run something went wrong (forgot if it was power outage or something else) and I wasn't able to let it finish rebuilding. So I ended up wiping out what was on the RAID and reformatting the drives, then using rsync to rebuild the contents.

With rsync, it took less than a day to transfer about 500 GB of data from several computers onto the USB RAID.

In the long run, I think it's worth noting that keeping two drives in two different USB enclosures working as a RAID is problematical. If one gets disconnected, there can be issues and that sometimes requires a rebuild. I decided I was better off getting an NAS and using that instead.

Reply

Answer 11

Feb 4, 2012 10:52 PM in response to HalNineThousand

Almost a year later... I have had the same experience. Two 1TB USB external drives, directly attached to a mac mini, running 10.7.2 Lion Server. After a reboot, the RAID set became Degraded and Disk Utility just keeps prolonging the estimated rebuild-time. Time scale is days.

Perhaps a solution is to use three 1TB disks and reconstruct the RAID set from two wiped/reformatted disks any time this happens. I would be interested to hear other's thoughts on this "solution".

I have downloaded the free trial of SoftRAID and will try that too.

This sure seems dysfunctional... a Ferrari sans wheels.

Reply

Answer 12

Feb 11, 2012 3:55 AM in response to UserNameX

I decided to give up on this.

RAID, by any flavor, appears to be orthogonally exclusive with external USB disks.

Reply

Answer 13

boudris

Level 1

4 points

May 25, 2015 1:28 AM in response to HalNineThousand

I have the same problem with two mirrored raid 1 TByte usb 2.0 disks. They are half full and takes three days to be rebuilt. I use OS X Lion

It is a price you have to pay due to USB 2.0 (MacBook air 2011), slow portable disks and big capacity. However during rebuilding you can write new files. The rebuilt can be stop and continue any time.

The problem with usb raid disks is that is easy one of them to be physically disconnected. So a raid rebuilt is needed every time. A small usb hub just for the raid disks is a solution. Windows 7 Pro don't support mirrored usb disks.

Reply