different rsync issues - "file has vanished"

Question

Level 1

0 points

different rsync issues - "file has vanished"

I'm glad I'm not the only one having rsync issues. I just was reading this thread where there seems to be some thinking that rsync isn't copying some files:

http://discussions.apple.com/thread.jspa?threadID=484729

I'm having some different problems, I'm not super adept at the command line, more of a GUI fellow traditionally but I've been trying to do some scheduled server backups with CRON and scripts. I've been using ASR to back up the boot volume which is working great and I've been using rsync for the data partition because of the huge number of files and because I'm concerned as to what might happen if the system went down while ASR was underway. The partial approach seems much safer...

So on my test server, if I do:

rsync -aE -delete /Volumes/XserveData/ /Volumes/DataBack/

I get a bunch of "file has vanished" errors:

file has vanished: "/Volumes/XserveData/.Trashes/._501"
file has vanished: "/Volumes/XserveData/._.Trashes"
file has vanished: "/Volumes/XserveData/._Desktop DB"
file has vanished: "/Volumes/XserveData/._Desktop DF"
file has vanished: "/Volumes/XserveData/sharepoint/. .DSStore"
file has vanished: "/Volumes/XserveData/sharepoint/._enigmo2.dmg"
file has vanished: "/Volumes/XserveData/testa/. .DSStore"
file has vanished: "/Volumes/XserveData/testa/. Command_Line_v10.4_2ndEd.pdf"
file has vanished: "/Volumes/XserveData/testa/._The NATural.rtf"
rsync warning: some files vanished before they could be transferred (code 24) at /SourceCache/rsync/rsync-24/rsync/main.c(717)

What's up with this? It seems to be related to extended attributes because if I leave the -E flag off I don't get the errors. I do want to copy extended attributes though, especially since I am using ACL's. I also noticed that every file that has vanished is preceded by ._ which I don't understand. Does anyone know what that is? the ._ ???

I wonder, should I even be messing around with rsync? What about ditto or psync or anything else that can check two directories, copy the new stuff and delete the old stuff? What do you folks use?

Help!

The weirdest thing though is it seems to work. Files SEEM to get properly copied but I'm just not comfortable enough to rely on this as a solution for a production server unless someone can assure me these errors are trivial. I've tried doing it with the -v flag but it gives me just the same error and not much else that I can tell.

I'm in 10.4.6, these are HFS+ Journaled not case-sensitive volumes btw.

Dual 2.7 G5, Xserve, G4 Powerbook, many others Mac OS X (10.4.6)

Posted on May 24, 2006 10:25 PM

Reply

Answer 1

HarryMcQ Author

Level 1

0 points

May 25, 2006 5:49 PM in response to HarryMcQ

I find myself replying to myself quite a bit in my limited time here, but people seem to chime in days, weeks even months later so I guess it's worth my time....

Alright, so I've done some digging....

This thread on afp548.com (great site):
http://www.afp548.com/forum/viewtopic.php?forum=35&showtopic=12035

answers some of my questions even though his problems are different. The ._ is for a split resource fork so evidently rsync is having trouble finding the split resource forks on the source volume... my question is why is it looking for them in the first place?

Therein is posted a link for a patch to rsync which looks promising:

http://www.lartmaker.nl/rsync/

though I'm not sure if it applies to the version of rsync that I'm using. I've also been reading that these Security Updates are modifying rsync (for the better?)
I read it somewhere on the web but how do you find out what version of rsync you're using, also if there are multiple versions on your system?

Reading about ditto seems like it will just over-write everything with everything else which is more time consuming but it does (as far as my limited understanding goes) do it bit by bit, therefore reducing the chance of complete disaster I forsee with ASR if the system goes down.

Reply

Answer 2

May 26, 2006 4:38 PM in response to HarryMcQ

The only time that I've seen those ._ files was when I was running XSAN with the metadata mixed with the data. I read that they were mac double files. My backup software couldn't handle them, so I ended up dropping XSAN.

Roger

Reply

Answer 3

Jun 1, 2006 7:55 PM in response to HarryMcQ

Hi Harry,
As Roger points out, those files are the resources forks of files on an HFS+ filesystem. Of course on an HFS+ filesystem, they should be combined with the data forks. Why they have been split off is a mystery. UNIX utilities that have been rewritten by Apple to handle resource forks create these "._" files to hold the resource forks when you use them to copy files from an HFS+ filesystem to another filesystem that doesn't handle resource forks. That's the mystery. I would expect that a non-HFS+ filesytem to have been involved in the creation of your "._" files but you've not mentioned such.

There is a better version of a patched rsync than the one you found at lartmaker, patched by Quinton Dolan. You can obtain it and read about it at Fixing rsync on MacOS X 10.4 (Tiger).
--
Gary
~~~~
You can have peace. Or you can have freedom.
Don't ever count on having both at once.
-- Lazarus Long

Reply

Answer 4

HarryMcQ Author

Level 1

0 points

Jun 15, 2006 12:47 AM in response to Gary Kerbaugh

Thanks! The patched version works great. I also ended up turning off ACL's which seemed to be causing lots of hassles so that might be part of it. The patched rsync you posted has been working real well!

-Harry

Reply