How can I easily delete duplicate files in old Time Machine archives?

Question

How can I easily delete duplicate files in old Time Machine archives?

I have multiple older Time Machine archives from different Macs that I have been going through manually and picking files out and copying them to another drive file by file. I'm running into massive quantities of duplicate files and the biggest problem is muliple iphoto pictures including the same picture in different sizes.

I wanted to give up and just throw the whole Time Machine archive onto another disk so I could format the one it was on and use it for something else. But I got a message reading, "The backup can’t be copied because the backup volume doesn’t have ownership enabled." So I went to manually copy each "Macintosh HD" folder for each date into it's their own respective folders. It worked but required my password for each folder and each of the 19 folders were all ~75GB. Which is impossible because all of that data was on a 500GB drive.

What should I do? This has become a two month long nightmare.

Posted on Apr 7, 2012 1:21 AM

Reply

Answer 1

Best reply

ds store

Level 7

30,715 points

Apr 7, 2012 8:12 AM in response to Blueprint Records

I really don't know exactly, but since no one else has a answer I'll give it a shot.

I don't know much about TimeMachine at all, however I did assist someone that their TM was corrupted and they needed to salvage what they could off the drive, so I'll start with that.

There is software called Data Rescue and it's designed to read the 1's and 0's of any data on the drive, deleted or not, regardless of the file structure, and output the results to another drive.

TimeMachine tries to save space, so it uses what appears to be duplicate files, but contain no data as they are placemarks for files in later states that went unchanged from the previous states.

If Data Rescue has the option of recovering files independently of the file/folder structure this would result in all your files from the recoved TM drive being on all one level on the second drive.

From that second drive one could run software called DeCloner, which would flag the actual duplicate files of content, not just by name.

The next stage perhaps would be to eliminate files with no data, the placeholders, using the Finder "size" as a sorting option.

Saving all this remaining content to another drive to work on further, perhaps all the remaining image files can be seperated, sorted according to largest size, the largest ones (usually the ones you want to keep as they contain more quality) and opened into iPhoto or Aperture for further refinement as you add a small batches of smaller image sizes in to compare with the larger versions for duplicates/unwanted sizes.

Reply

Answer 2

R C-R

Level 6

17,782 points

Apr 7, 2012 9:05 AM in response to Blueprint Records

What you see in any Finder view of a hard drive's file system are basically just links (pointers) to the locations where the file & folder data is stored in the drive's file system. This distinction isn't important as long as there is only one link to that file's data, which is usually the case.

However, this isn't true for the items in the time stamped Time Machine backup folders. To save space, Time Machine actually saves just one copy of the data of each unique file, no matter how many of the time stamped backup folders links to it appear in.

So what you are actually doing when you use the Finder to manually copy items from each of those time stamped folders to some other location is making a new copy of its data for every appearance of its link! That's why when you added up the sizes of all 19 backup folders the total was larger than the capacity of the TM drive: the data sizes were counted up to 19 times, depending on how many backup folders links to it appeared in.

Reply