Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Does the 10.6.x Finder do a bit-for-bit verification of copied files?

When using the Finder in Snow Leopard to copy files from one drive to another, does the Finder perform a bit-for-bit comparison of the original file to the just-created copy, verifying that an absolutely identical file has not only been written to but, more importantly, can be read from the second drive?


I have heard multiple and opposite answers to this question from many respected leaders on the topic of Mac OS X data integrity. For example, I have been told that the Finder does NOT do any post-copying verification, that only third-party products (one such recommendation was ChronoSync) can do this. I look forward to new input.

[1] Mac Pro 2x3GHz QC/10.6.6/32GB/8800GT: Aperture 3.1.1, FC 7.0.3, [2] MBP 2.66GHz i7 17"/10.6.6/8GB/GT 330M/SSD: Aperture 3.1.1, F

Posted on Jun 15, 2011 10:14 AM

Reply
Question marked as Top-ranking reply

Posted on Jun 15, 2011 10:17 AM

AFAIK, no.

10 replies
Sort By: 

Jun 15, 2011 11:56 AM in response to baltwo

Thank you to baltwo and etresoft, both of you well-respected members of this community. This is where I find that the discussion gets interesting, if not outright confusing. Another respected community member, Kappy, previously stated exactly the opposite of both of you, that the Finder does perform verification (although he did not specify the type or completeness of verification). Here is the link to that February of 2009 discussion:


https://discussions.apple.com/thread/1895741


As I've tried to get a definitive answer on this, here's more of what I have found:


Peter Krogh, author of "The DAM Book" and arguably an expert on data integrity, states in his book that a post-copy verification process is vital and indicates that a Finder copy does not provide this. (See pages 225-226 of the current edition.) He recommends using a backup utility, not the Finder, to make drive-to-drive copies as, "This ensures everything that I intended to copy has copied and it checks to make sure that each copy is exactly like the original." He goes on to recommend ChronoSync or SuperDuper! However, note that following the publication of book, in his own forum, Krogh corrected himself when it was discovered that SuperDuper! does not do verification, thus leaving ChronoSync by default as his sole Mac recommendation. Forum link: http://thedambook.com/smf/index.php?topic=4478.0


And SuperDuper!'s lack of verification was confirmed to me by developer Dave Nanian of Shirt Pocket Software. In a response to my 2010 inquiry regarding whether his SuperDuper! application performed a bit-for-bit verification, he wrote:

"No, we don't verify. Modern disk controllers check both read and write operations, so verification is generally redundant."

Honestly, the phrase "generally redundant" leaves me deeply concerned as that sounds to me like exceptions to verification, meaning unseen and/or unannounced errors, can actually happen. Otherwise, please omit the word "generally" and say something else. No, I want bullet-proof verification.


Now, I'm no shill for ChronoSync; frankly, if all I want is a verified copy of a folder full of files (I'm a photographer, copying thousands of images across multiple drives on a regular basis), ChronoSync is gigantic overkill and, honestly, quite confusing and cumbersome for this simple task. What I want is a simple GUI way to copy my files across drives and KNOW that they have arrived intact. The Finder would have been/could have been/should have been that method, but as of this discussion, apparently does not seem to have that absolute capability.


By the way, etresoft, thank you for pointing out the possibility of doing a manual "cksum" and using a script to automate the process, but, honestly that is WAY over my head, as it would be for most consumers.

Reply

Jun 15, 2011 12:15 PM in response to bosskev.

Bombich, Carbon Copy Cloner developer says:


CCC doesn't currently offer verification of data that already exists on your backup volume. CCC will calculate a checksum of the data that it copies, but it doesn't currently use this to determine if there are differences between files on the source and destination. This functionality will be present in the next version of CCC.

Reply

Jun 15, 2011 2:03 PM in response to bosskev.

bosskev. wrote:


Another respected community member, Kappy, previously stated exactly the opposite of both of you, that the Finder does perform verification (although he did not specify the type or completeness of verification). Here is the link to that February of 2009 discussion:


https://discussions.apple.com/thread/1895741

In Kappy's defense, that question was specifically asking about differences in verification between Windows and Mac. There is verification in the sense that an error will stop the copy. Both Windows and Mac behave the same in this regard. That is one of the complaints and annoyances of the Finder. Sometimes, you just want it to copy everything and ignore the errors. It won't do that.


Peter Krogh, author of "The DAM Book" and arguably an expert on data integrity, states in his book that a post-copy verification process is vital and indicates that a Finder copy does not provide this.


Vital? I don't know about that. The general consensus is that the people would scream bloody murder if their file copies suddenly started taking twice as long. That is what a bit-for-bit comparison would require. You can read the source and calculate the checksum at the same time. But to compare it, you must them re-read the target and compare the checksum.


No, I want bullet-proof verification.


I don't think you really need to worry about that. I've still got files from my old Mac Plus that are intact. There is a lot of low-level verification going on with modern hardware. It isn't bullet-proof, but shooting are very, very rare. If you were crossing a media boundary by putting files on tape, then checksums might be something to think about.


By the way, etresoft, thank you for pointing out the possibility of doing a manual "cksum" and using a script to automate the process, but, honestly that is WAY over my head, as it would be for most consumers.


It is an awful lot of trouble too. Sometimes I think about doing exactly what you propose. But when it comes down to it, I just don't have the time. It is a lot of effort, a lot of time, for a very low likelihood of failure.

Reply

Jun 15, 2011 4:52 PM in response to etresoft

etresoft wrote:


The general consensus is that the people would scream bloody murder if their file copies suddenly started taking twice as long. That is what a bit-for-bit comparison would require.


Understood. And that is exactly what I want, at least when moving thousands of irreplaceable photos, despite the twice as long aspect.


We know that the Finder is capable of doing this automatically, we see it happen anytime we copy from drive-to-optical disc: the first pass is writing, the second pass is verifying. Now why can't this same two-pass Finder copying/verifying behavior be toggled on/off, as desired, on a drive-to-drive copy? Which leads to the following pair of questions:


1) Is there is an accessible command to toggle on/off a Finder copy verification pass and, if so, what is that command? (If necessary, I'd bite the bullet and learn this one and only on/off command.)


2) Does anyone know if there is a GUI that already does this—like a TinkerTool or an Onyx or similar application—via a consumer-friendly GUI, granting access to that very command line instruction? (I've been looking and have not found any as yet.)

Reply

Jun 15, 2011 4:54 PM in response to bosskev.

That is not a standard file copy. That is copying to different media.


Try looking here: http://osx.iusethis.com/search?q=md5


MD5 is the standard hash algorithm and if you search for that, it should turn up the kinds of applications you are looking for. There were 19 hits on the above link, most of them freeware. There are six matches in the Mac App Store too.

Reply

Mar 24, 2012 10:39 AM in response to bosskev.

Hi Bosskev et all:


Thanks for all the information and work. I was wondering exactly that.

You may know that there is some free software (from Linux open source) that make just verified copies. It´s Grsync, it´s ported to mac, and it´s a Graphical User Interface (GUI) for the rsync.

I tried it only once, but it looks nice. (last version is 2009).


links:

http://grsync-mac.tuxfamily.org/?lang=en

http://en.wikipedia.org/wiki/Grsync

http://en.wikipedia.org/wiki/Rsync


Some first instructions:

- External disks are in: File System / volumes

- In advanced options: check Always checksum, uncheck Protect Remote Args.


Also, I think a copy and verify action is feasible in automator, but beyond my knowledge.


Marcelo

from Buenos Aires

Reply

Mar 24, 2012 8:37 PM in response to bosskev.

bosskev. wrote:


does the Finder perform a bit-for-bit comparison of the original file to the just-created copy

I think there is some considerable confusion here. "Bit-for-bit comparison" is not necessarily the same thing as verification. For modern media and computers, in many cases, it doesn't even make sense; I think this notion comes from the fact that many users have no notion of how data is actually stored on magnetic or optical media.


I suggest consulting two Wikipedia entries


<http://en.wikipedia.org/wiki/File_verification>

<http://en.wikipedia.org/wiki/Checksum>


I haven't read the book referred to, but I've taken a peek at


Data Validation Overview

<http://www.dpbestflow.org/node/259>


by the same author, and, IMHO, he either doesn't understand or doesn't explain clearly this issue. For instance, he writes,


"[…] The only way to be absolutely sure that everything was transferred properly is to perform a bit-for-bit comparison between the original and the copy."


That is, in fact, completely wrong. As etresoft already said, "bit-for-bit comparison" between different media is non-sense. Moreover, the example Krogh gave in fig 1 would have been caught by even the most basic checksum verification. It almost seems as if Krogh uses "data validation" and "bit-for-bit comparison" as synonymous, when, in fact, they're not. In fact, for many in the IT business, "data validation" has quite a different meaning than file verification.


As to ChronoSync, a google search on the developer's site for "bit-for-bit" returns no hits. Nor does its manual mention this, so I'd like a specific link to this issue WRT ChronoSync.

Reply

Does the 10.6.x Finder do a bit-for-bit verification of copied files?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.