Currently Being ModeratedFeb 18, 2013 12:10 PM (in response to BordeauxQuill)
It seems that the rsync command is completing (results in its own log), but the shell session is still throwing the error and causing the script to abort. I tried adding an 'exit' command after rsync, but the problem is still occurring intermittently. Any ideas would be appreciated. Thank you.
Currently Being ModeratedFeb 18, 2013 1:16 PM (in response to BordeauxQuill)
I finally managed to replicate (what I assume to be the same) error from within Automator (so not using the Application, which threw the error more often than not). In the Automator Log I saw 'Run Shell Script failed - 1 error'; on the next line was the error icon (white cross against red circle) and '(24).
Have been unable to understand what the '(24)' means. Again, this is not happening on every execution. The script seems to be performing the rsync command. For this last run that command was followed by an 'exit' instruction.
Again, any ideas would be appreciated. Thank you.
Currently Being ModeratedFeb 18, 2013 1:18 PM (in response to BordeauxQuill)
rsync error 24 is "Partial transfer due to vanished source files". did your source files move before rsync was finsished?
Currently Being ModeratedFeb 18, 2013 1:37 PM (in response to twtwtw)
Thank you, twtwtw (especially for highlighting that this was an rsync error and not shell - I should have been able to find it, sorry).
I am guessing you are on to something here: I copy the Library folder in my user account and sometimes Preferences files (as I recall) vanish between the building of the file list and the copy. When I run in Terminal I sometimes see a warning that some files have 'vanished' but this is never recorded in the rsync log-file, which seems to include what it copied rather than what it thought about copying! This also, of couse, would explain why the abort occurs only intermittently.
I now need to find out how to trap this one specific error.
Again, thank you.
Currently Being ModeratedFeb 18, 2013 1:53 PM (in response to BordeauxQuill)
Easiest might be to write the error stream to a file. Automator Run Shell Script will always stop when it gets an error message back from a script, but if you can catch the error before it returns it may continue processing.
Currently Being ModeratedFeb 18, 2013 2:19 PM (in response to twtwtw)
Thanks. I did a little Googling re redirecting STDERR and tried adding the following (bold) to my command line:
rsync -av --exclude-from="/Volumes/X/rsync Library/rsync Exclusions/user-exc.txt" --log-file="/Volumes/X/rsync Library/rsync Logs/rsync-user.txt" "/Volumes/Mac HD/Users/user/" "/Volumes/X/rsync-user" 2>"/Volumes/X/rsync Library/rsync Logs/Errors/rsync-user errors.txt"
It is still ending with an error 24 occasionally (and this file is empty, but created). My guess is that your idea is correct and my implementation wrong. Shall continue to play with redirecting the output and let you know how I get on. Thank you for your supprt.
Currently Being ModeratedFeb 18, 2013 2:26 PM (in response to BordeauxQuill)
Well, rsync is not exactly a typical utility; it has it's own style, and it runs multiple sub processes, so maybe normal redirection doesn't work as expected with it. Unfortunately I'm only middling good with unix. you'll probably need to wait until one of the unix pros weighs in.
Currently Being ModeratedFeb 20, 2013 2:49 PM (in response to twtwtw)
Thanks again, twtwtw. Just to update you, I managed to redirect output and error stream to a file by adding the following to the rsync command:
With the above, I could see the error (24) codes in the file, e.g.
rsync warning: some files vanished before they could be transferred (code 24) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
However, unfortunately, this redirection did not prevent the shell script from terminating. Still, I'd like to thank you for your support here.
Currently Being ModeratedFeb 20, 2013 4:33 PM (in response to BordeauxQuill)
If I were you I'd just exclude the preferences folder, or maybe make a static copy to a local directory and archive that. Preference files are modified frequently (I see changes in that folder as fast as the Finder updates its views, so at least every ten seconds), and the user defaults system seems to use a delete-then-rewrite strategy for plist updates, rather than updating existing files. That means you'll always have files disappearing and reappearing in that folder, and rsync will never be happy.
Currently Being ModeratedFeb 20, 2013 6:24 PM (in response to BordeauxQuill)
Have you tried "exit 0" as the last command in shell script? I ask this because you only wrote you tried adding "exit" after rsync. When you use exit without number, the exit status is that of the last command executed, which is 24 from rsync in this case.
E.g., try something like this if you have not:
#!/bin/bash rsync OPTIONS SRC DEST > /path/to/errlog.txt 2>&1 exit 0
Currently Being ModeratedFeb 23, 2013 11:27 AM (in response to twtwtw)
Thank you, twtwtw. Yes, I am adding 'busy' folders to the exclusions list - I only really need to copy user content, and have less interest in configuration settings, etc. Still, there is content in Library - fonts, mail folders, chat messages, etc.
Currently Being ModeratedFeb 23, 2013 11:40 AM (in response to Hiroto)
Hiroto, Thank you. This solved the problem: adding 'exit 0' allows the scipt to continue.
For others' reference, Automator application for copying files with rysnc now looks like this:
rsync -a --stats --exclude-from="/path/exclude.txt" --log-file="/path/log.txt" "/path/src/" "/path/dest"
For me, there is no need for the -v switch, as the log contains a verbose listing. After Run Shell Script, I have New Text File, which shows the statistics (--stats switch) and any errors. If we remove --stats then this file is empty if there are no errors. I end with Ask For Confirmation so that I know the script has ended.
Improvements might be:
a) exit 0 only if error = 24
b) Show a Notification Centre badge in Mountain Lion
I suppose that (a) can be done with Apple Script, and (b) with terminal-notifier (a solution I know nothing about, but which apparently is robust). Still, I have not wanted to complicate things - the script does what I need, if perhaps not entirely what I desire!
Thank you to everyone who helped to solve this.
Currently Being ModeratedFeb 23, 2013 1:04 PM (in response to BordeauxQuill)
My pleasure. Glad to hear it helped.
It is quite easy to implement a) exit 0 only if error = 24. Like this:
rsync -a --stats --exclude-from="/path/exclude.txt" --log-file="/path/log.txt" "/path/src/" "/path/dest" ret=$? if [[ $ret -eq 24 ]]; then exit 0; fi exit $ret
As for b), I have no idea because I don't know 10.8.
Message was edited by: Hiroto (fixed the code)
Currently Being ModeratedFeb 23, 2013 3:02 PM (in response to Hiroto)
Thank you, Hiroto. So, I shall type this into the Run Shell Script into Automator - great. I see how the (now updated) syntax works, but I am unsure why I need to use double square brackets as opposed to single. Have googled but cannot find a satisfying answer that seems to apply. I am using /bin/hash (this seems relevant). May I abuse your kindness and ask for the rationale, please? Again, thank you.