Time Machine backup and main drive corrupted. Help! (REWARD OFFERED)

Question

Level 1

5 points

Time Machine backup and main drive corrupted. Help! (REWARD OFFERED)

Here's the deal:

I have a Macbook Pro and a Mac Mini both runnign Snow Leopard. I use the Mac Mini as a kind of media center / server, it has a few external drives connected to it. On of these drives (1GB) is dedicated to Time Machine, the Mac Mini (80 GB hard drive) backs up to it directly and the Macbook Pro (500 GB hard drive) does it over the network (Time Machine created a sparsebundle). This has worked well for years now. Occasionally I got the error that Time Machine needed to start a new backup because the old one was corrupt. That happened about 2-3 times a year (did the same thing when I backued up via USB). Now about 2 weeks ago, that error came up and I just let the Macbook Pro on overnight and connected the ethernet cable for faster transfer.

When I woke up, the Macbook Pro didn't respont at all, spinning beachball, no response at all beside mouse movement. I let it do it's thing for another 10 hours (while I was at work) and just held down the power button to power off and restart it. But all I got was the gray-on-gray flashing folder with the question mark in it, that's what you get when the Mac can't find bootable partitions. So I popped in the OSX Snow Leopard install disk, ran disk utility. It saw the hard drive, but no partition (i.e. Machintosh HD) on it. I checked the Time Machine backup and the sparsebundle was 300 GB (the Macbook Pro had 400 GB used, the remaining 100 GB were free). There is no way to restore from an unfinished Time Machine backup...

First thing I did was clone the internal (Macbook Pro) hard drive to a DMG disk image using DiskDrill (the only program I found that could recognize the drive at all, not even DiskWarrior could). I also bought the exact same hard drive model and partitioned it like the cloned the corrupted hard drive to the new one using ddrescue (a command line tool that doesn't quit upon i/o errors but proceeds and tries to recover as much as it can). It copied everything except 65 kilobytes, the corrupted drive seemed to be physically damaged in a bunch of sectors relatively at the beginning of the disk. Since I had now an exact copy on a fresh, healthy drive, I went crazy trying out Disk Warrior (didn't recognize the drive at all), data rescue, testdisc, p a Windows isk, etc. Only R-Studio (on windows) showed the EFI and Macintosh HD partitions on there, they started and ended on the same sectors on the corrupted drive and its clone. After some research, I figured that the partition table was corrupt so I reformated the clone disk using the OSX Snow Leopard install disk (1 HFS Journaled Partition with GUID Partition table). R-Studio showed the EFI and Macintosh HD on that reformated drive, again, same sectors as before. So I figured I could just copy just the bytes where the Macintosh HD starts from the corrupted drive to the clone (using ddrescue). That worked, after almost 24 hours, I had the clone drive with a "disk1" partition on it that even disk utility could see.

Now I was able to run Disk Warrior on it, but all it could do was recover a few Application folders (Resource-Folders and lproj-stuff), about 100 MB in total. It couldn't repair more of catalog file apparently. Luckily, Time Machine backed up quite a bit (300 GB out of 400 GB of data) and I was able to manually copy all the Dokuments, Desktop, user Library, Applications, Music, Download and Movies. Unfortunatley, only a little bit of the Pictures folder was copied. iPhoto library (80 to 100 GB) was nowhere to be found, backup must have failed right then. I can salvage the drives (time machine drive, original hard drive with a few broken sectors, DMG-image of that drive, 1-1 copy of that drive with partition table repaired) but that only gives me files with numeric names and today's date on teh JPEGs (instead of the date the picture was taken).

Is there any way I can recover that iPhoto library? It appears the catalog file got corrupted because the hard drive (only 8 months old...) failed on a few sectors. If I understand it correctly, the catalog file on HFS+ file systems is where the folder structure and file names are stored in a B-Tree. I can't imagine that some i/o error during backup can totally annihilate that file when it was working perfectly before. Here's a few things I want to try out but haven't figured out how so far:

- Time Machine had to start a new backup. There's plenty of free space on that drive so there's a good chance there's old data left on it. Is there a way to restore files (including file names) and fodlers from deleted time machine backups?

- Is there any way to re-build that catalog file from what is there left on the original hard drive? I can't imagine 65 kilobytes destroys it all.

- Are there other ways to recover my iPhoto Library? The raw JPEG (and AVI) files with correct file names or metadata would suffice.

Thanks in advance for any help, I'll actually reward the person with a working solution, 5 years of photo memories are somewhat important. It really ***** that a failure during a backup destroys that...

Posted on Dec 19, 2011 3:20 AM

Reply

Answer 1

Best reply

Pondini

Level 8

38,863 points

Dec 19, 2011 8:12 PM in response to MrCrowbar

MrCrowbar wrote:

. . .

Is there a way to restore files (including file names) and fodlers from deleted time machine backups?

It's theoretically possible to recover some of it, but won't be cheap or easy, and will certainly be "hit and miss." The best you can hope for is a partial folder structure with some files in it. Most likely, you may find a number of "loose" files, perhaps some with names, etc.

It's a little complicated by them being inside a sparse bundle, unfortunately. That sort of scatters the pieces of files over a number of 8 MB "bands." Since the sparse bundle was deleted, it was already suspect, and I'd guess it's directory is probably gone; I'd be afraid to guess whether it's possible to reassemble the pieces without that.

There have been some reports here of folks who have gotten data out of sparse bundles on failed drives, but I don't recall any where the sparse bundle had been deleted.

I don't know if any of the data recovery apps can deal with it, but you might find a service that can (probably a few hundred dollars, from what I hear, so it might be worth the gamble to try one of the apps first). See Data Recovery for more.

Good luck, and let us know if you have any success.

Reply

Answer 2

MrCrowbar Author

Level 1

5 points

Dec 20, 2011 1:37 AM in response to Pondini

Thank you for that insight. Sounds like I'm doomed then regarding the time machine thing. I already did a full scan for deleted files on that drive and go a lot of pictures, a lot of them are duplicates and quite a lot are damaged (weird colors, mosaics with other pictures, only a few lines visible, etc.). I guess Time Machine just wrote over the good stuff with the new backup.

After almost 2 weeks, I have almost given up on restoring the original Library. I'm currently usind DiskDrill to recover all the Photos from the failed hard drive (well, the clone actually, ddrescue was able to read out the original except 4 kilobytes and copy it all). I can then filter by file size and resolution (I never really edit photos, I just dump them into iPhoto) and re-sort the good ones as I go.

I haven't given up on restoring the original hard drive (clone) though. With only 4 kylobytes of unreadable data, it can't be that broken. I know where the partitions are supposed to start and end and I have a copy of the failing drive on another drive (the exact same model). Maybe restoring partitions actually solves the problem, haven't found a tool that does it though (testdisk, Disk Warrior, etc.)

Reply

Answer 3

MrCrowbar Author

Level 1

5 points

Dec 24, 2011 5:16 AM in response to MrCrowbar

Final Update:

The catalog file on the original hard drive could not be fixed. Seems like Mac OSX tried to repair the catalog file while the sectors this file resides on failed. To make things worse the partition table was also broken beyond repair, even overwriting the sectors with a new correct partition table didn't help. DiskWarrior found less then 100 MB worth of stuff, mainly Applications folders.

I recovered pretty much everything from the incomplete Time Machine backup by right-clicking the sparsebundle and browsing through the folders with the long alphanumeric names, looking for the version of the folder with the most files in there. All I was missing was part of the ~/Pictures folder, i.e. photobooth pictures and the whole iPhoto Library. My best option was to recover these files using data recovery tools.

DiskDrill proved to be the absolute best, fast, responsive, efficient, and the only one able to mount the DMG-file with no valid file system on it. As there were many i/o errors and broken sectors on the original hard drive, I made a copy of it using a free command line tool called ddrescue (the standard dd tool just aborted when it encountered the i/o error). ddrescue copied the whole drive to a DMG image, I had 56 kilobytes with errors on the first pass, but it managed to shrink that down to just 4 kilobytes (wow!) after the second pass where it tries to re-read the broken secors. It took about 24 hours for a 512 GB 2.5" drive (5400 rpm) but well worth it. Be advised that ddrescue is really persistent and tries everything to recover those last errorneous bytes. At the very end of the process, the read/write head of the hard drive just goes wild trying to catch the data on the sectors with different momentum. This works but I assume this is pretty damaging for the original drive. I also copied it all to a new hard drive (again using ddrescue) and tried partition and catalog repair tools on that (DiskWarrion, testdiks, pdisk, etc.). Still no hint of a good result.

I made a deep scan on the clone hard drive with DiskDrill. At the end (after about 8 hours over USB) it found 13 partition (I assume that's the Macintosh HD, EFI and some DMG files lying around) and hundreds of thousands of pictures. I restored some JPG files just to check the quality, some were damaged, some were good with all the EXIF data intact. I just made it copy all JPG files into a folder. I know the pictures taken from my camera produce JPGs larger than 1 MB and smaller than 5 MB, so I sorted them by size and moved the smaller and larger files into seperate folders. I took the remaining folder (100 GB) and just dragged it into iPhoto. It imported them overnight. Auto-Split by events and I got my library back, alas with different file names, originals and edited versions side by side, lots of duplicates, some damaged, some not. But hey, all the pictures in chronological order. Okay there was also one large event with all the JPGs without valid EXIF data landed inside, iPhoto just takes the file creation date (i.e. the date where the recovered file was copied). As far as I can tell, these are all just data corpses, halfway overwritten copies, random pictures from the internet, desktop pictures, etc.

I started to work my way back through the events, deleting the duplicates and renaming the events. There's an app called "Duplicate Annihilator" which apparently can find duplicate pictures in iPhoto and mark them for you. The free version only does 500 pictures but if it works, I'll get the full version. It can mark th eduplicate photos by adding something to the picture comment in iPhoto so you can manually review it all. Good stuff!

Reply

Answer 4

Pondini

Level 8

38,863 points

Dec 24, 2011 7:36 AM in response to MrCrowbar

Wow! You got back far more than I would have expected! 🙂

Quite a lengthy, tedious, and scary ordeal, of course. 😟

I think I'll bookmard this thread, for folks who think a single set of backups makes them safe.

Thanks for the report, and congratulations!

Reply

Answer 5

MrCrowbar Author

Level 1

5 points

Dec 24, 2011 8:30 AM in response to Pondini

Scary indeed, and stressful. On the other hand it lets you appreciate life's little moment more. I'm rather paranoid about back backing up my "important" files, i.e. the stuff that would hurt me a bit financially. But what is it all worth if you lose precious memories but have 3 backups of your business stuff?

My backup plan from now on is this:

- Time Machine backups over WIFI (these just automatically happen as long as I'm in the vicinity of the house)

- Automatic weekly backup of the entire Time Machine backup drive using SuperDuper. This way if the Time Machine drive gets messed up like it did in my case, I can just use the weekly backup.

- A manual monthly backup (iCal reminder) onto a USB hard drive that I store somewhere safe, I'm thinking fireproof box in the basement. This way, a house fire or a power surge (annihilating everything somehow connected to a power outlet) can't destroy that backup. I think the basement is actually the best option even against solar flares and EMPs.

It's ironic that's I'm known as a backup nazi with little compassion for people losing their important stuff due to hard drive failure (or stolen/lost laptop). The lesson to learn is to ask yourself what your personal files are worth to you and how much it would bother you to come home one to a completely burnt down house. A 2 TB USB hard drive costs about $100...

Maybe this thread can help someone in distress and thanks for the compassion.

Reply

Answer 6

Pondini

Level 8

38,863 points

Dec 24, 2011 8:54 AM in response to MrCrowbar

MrCrowbar wrote:

. . .

- A manual monthly backup (iCal reminder) onto a USB hard drive that I store somewhere safe, I'm thinking fireproof box in the basement.

Be sure that's a backup of each Mac, not the TM backup drive. Backing-up a backup isn't as safe, since a problem on the primary backups will be copied to the secondaries. And if you use a different app, such as SuperDuper or CarbonCopyCloner, you're also protected against a problem with either Time Machine itself or the cloning app -- no app is perfect.

Some folks use a pair of portable externals for that; keep one in their safe deposit box, workplace, relative's house or other secure off-site location, and swap with the other one, so one is always off-site.

Reply

Answer 7

Dec 24, 2011 8:59 AM in response to Pondini

Maybe I am not as silly as they say doing two backup using different methods for each.

Thirty years in IT support does that to you.

Lose of data is an absolute the only question is when.

Allan

Reply

Answer 8

Pondini

Level 8

38,863 points

Dec 24, 2011 9:19 AM in response to Allan Eckert

Absolutely! Lessons many of us learned, usually the hard way. 😉

Reply

Answer 9

MrCrowbar Author

Level 1

5 points

Dec 24, 2011 9:22 AM in response to MrCrowbar

The cool thing about backing up the backup is that it can happen automatically while I'm out and about with the laptop. I have a Mac Mini with a bunch of USB hard drives connected to it, one of which is the Time Machine backup drive. The Mac Mini backs its system drive and another media drive up to it, the Macbook Pro backs up to the same drive via WIFI. All I really notice is the little icon in the menu bar spinning.

The Mac Mini (over 4 years old now) takes very little power when it's not doing anything and the hard drives (mostly Western Digital MyBook) go to sleep after a few minutes of inactivity. The whole rig uses under 15 Watt when idling and is also my media center running on the TV. It's all tucked away neatly in a large drawer (IKEA Malm) with a bit of soundproofing and holes in the back so the cool outside wall helps cooling it down.

I wholeheartedly agree with not trusting one thing (hardware or software) fully. In my case, multiple things failed at the same time: hard drive failed in the important sectors, OSX tried to repair the catalog file and actually destroyed more than it helped, Time Machine insisted on deleting the backup and start a new one. Lesson learned. :-)

Reply

Answer 10

MrCrowbar Author

Level 1

5 points

Jan 2, 2012 4:15 PM in response to MrCrowbar

I got the replacement drive running LION for a while now, but I get loads of i/o errors, system freezes, etc. The replacement drive failed Apple Hardware Test, but even using an older 80 GB drive (which didn't fail in AHT), I still get serious errors. I suspect my SATA controller or interface is bust, but there's no way to tell apparently...

Anyway, since this has nothing to do with this thread, I'll close this and link to the new one: sata failing

Reply