rsync: copies all files, not just the updated ones!

Question

Level 1

8 points

rsync: copies all files, not just the updated ones!

Hi,

I have an external hard drive that I formatted using my iMac (runs on Leopard). After that, I took a backup of a folder which resides on the hard disk of a linux machine of mine on the external hard drive. Now, when I try to update this folder using rsync, I see that many of the older files, which have remained unchanged since the time I first took the backup, also get copied instead of just the modified and new files that are inside the folder I am copying/syncing. Has anyone seen such a problem? If so, is there a fix to it? (e.g., can I run rsync with suitable switches/flags that take care of this issue?) . . . Or does this have to do with the discrepancy between Apple's and other linux/unix OS's file handles? Let me know. Thanks!

P.S. I access the folder on my linux desktop remotely.

iMac and iBook, Mac OS X (10.5.6), Latest version of Tiger on my iBook

Posted on Jan 13, 2009 11:20 AM

Reply

Answer 1

Kappy

Level 10

362,007 points

Jan 13, 2009 12:03 PM in response to pradeep_bhat

Yes, rsync has a lot of switches, and doing a man rsync in the Terminal will tell you what they are.

OS X filenames are not all the same as in Linux. OS X uses special characters not allowed in Linux. If you use rsync to do a simple copy all, then that's what it does, and that's the default, I believe.

Reply

Answer 2

Jan 13, 2009 1:26 PM in response to pradeep_bhat

pradeep_bhat,

As Kappy has stated, rsync has many, many options. In fact, it has at least two "modes" of operation, and you are probably using the first, basic one.

Its more powerful mode is that of a file "syncing" tool, and it seems clear that it is this mode you wish to use. In rsync terms, this is the "archive" mode. There is one downside to this mode, however: It can only be run as "root," accessed through the use of "sudo." This mode's assumption is that source directories may contain files from diverse users, with diverse ownership and permissions. It is for this reason that the "archive" mode assumes the need to run as root, and preemptively requires privilege elevation via sudo.

The benefit is that the archive mode recursively scans a source directory, and only copies new or changed files to the destination. If the proper flags/options are used, it preserves ownership, permissions, and extended attributes (only supported on HFS+ volumes). An example of this in use would be:

<pre style="overflow:auto; font-family: 'Monaco'; font-size: 10px">sudo rsync -aE /Source /Destination</pre>

In the above example, the "-a" option calls the archive mode, and the "-E" option specifies that extended attributes and resource forks will be preserved.

Scott

Reply

Answer 3

pradeep_bhat Author

Level 1

8 points

Jan 14, 2009 8:31 AM in response to Scott Radloff

Dear Scott,

thanks for the detailed reply. I forgot to mention in my earlier email that I do run rsync in the archive mode, i.e., I use the -a switch. (In fact, I run rsync as "rsync -a --progress source/ destination/". I also tried the -E switch but that didn't alter the result.) However, I have not done the syncing, i.e., I didn't run rsync, by logging in as the root. This I will try and see what happens. The funny thing is, I don't see this problem with rsync when I am synchronizing on linux or unix platforms, and, also, between my iBook that runs on Tiger and a linux desktop! I see this only with Leopard.

- Pradeep

Reply

Answer 4

pradeep_bhat Author

Level 1

8 points

Jan 14, 2009 8:34 AM in response to Kappy

Dear Kappy,

see my reply below for Scott. I use rsync with the following options/switches: -a and --progress. In other platforms like linux, unix, and Tiger for that matter, I get the expected result (rsync updating only the modified files and preserving all file attributes as I am running on the archive mode), but (curiously) not in Leopard!

- Pradeep

Reply