rsync output via launchd redirection has troubles with Umlauts

Question

rsync output via launchd redirection has troubles with Umlauts

Hello all,

I use rsync to backup a Mac OS X 10.6.x machine to a remote QNAP NAS through a ssh-tunnel. The command:

/opt/local/bin/rsync -avz --modify-window=1 --iconv=UTF8-MAC,UTF8 --partial --delete --ignore-errors -e "ssh -a -p $qnapPort" $sourceFolder admin@$qnapURL:$destinationFolder 2>&1 | while read line

do

date "+%Y-%m-%d %H:%M:%S [rsync_backup.daily] ${line}"

done

works perfectly fine. However, there is one little problem occuring. The script is called daily by a launchd deamon. All output is redirected via launchd's StandardOutPath and StandardErrorPath to myrsync.log. The problem is that Umlaut's (äöü) are not shown correctly. They show correctly on the QNAP NAS, so there the iconv command works.

Any ideas where the problem lies?

HFS+ uses UTF-16 while the myrsync.log is UTF-8?
My locale is UTF-8 but should be UTF-8MAC?
etc.

Help is appreciated.

Example:

Filename in Finder: ProdÜbersicht_V12en.doc

Filename in logfile: ProdU#314#210bersicht_V12en.doc

Posted on Mar 2, 2013 12:11 AM

Reply

Answer 1

Tom Gewecke

Level 10

122,258 points

Mar 2, 2013 7:39 AM in response to Alexander Harm

In case it helps, #314#210 is the octal representation of a UTF-8 encoded combining umlaut (U+0308, utf-8 hex bytes CC 88). Unfortunately I don't know how one gets a logfile to show U#314#210 as the equivalent Ü.

Reply

Answer 2

Vombatus

Level 1

0 points

Jan 11, 2015 6:11 AM in response to Tom Gewecke

I do have exactly the same problem with rsync.

If I issue the command in terminal directly -- Output (filename to be synchronized by rsync) is (in terminal) like this: .... Passwörter_2013 ...

If I do the same command in applescript "do shell" the output in the variable is: ... Passwo\#314\#210rter_2013 ...

I tried several things:

" as class <<class utf8>>" in the do shell command --> No effect

BTW: All files are copied correctly. So no urgent problem there, but the output in my log (text) file looks bad (o\#314\#210)

Any new ideas?

Reply

Answer 3

Tom Gewecke

Level 10

122,258 points

Jan 11, 2015 8:11 AM in response to Vombatus

What you have is the utf-8 bytes being represented in the form or Octal (i.e. base 8) escape codes. It's what you could expect to get if you set Terminal > Preferences > Advanced to "escape non ascii output". Perhaps applescript has that kind of setting on by default and there is some function to turn it off?

Reply

Answer 4

Vombatus

Level 1

0 points

Jan 11, 2015 10:54 AM in response to Tom Gewecke

Dear Tom,

thanks for your quick answer. You brought me on the right track.

If I use "do shell "locale" I can see the following output:

LANG=

LC_COLLATE="C"

LC_CTYPE="C"

LC_MESSAGES="C"

LC_MONETARY="C"

LC_NUMERIC="C"

LC_TIME="C"

LC_ALL=

So LANG is not defined.

I added now in my do shell command:

do shell "export LANG=\"de_DE.UTF-8\"; opt/local/bin/rsync -vzrtn .... "

AND this will solve the problem!!!

Output WITHOUT the export LANG=\"de_DE.UTF-8\" command: ... swo\#314\#210rt ...

Output WITH the export LANG=\"de_DE.UTF-8\" command: ... swört ...

And thats how I wanted it!

It looks like the important part is the UTF-8 and not the de_DE part of the LANG environment variable. But I have not checket this in detail. However, with de_DE.UTF-8 it works fine.

Again thank you for your help in this matter!! Have a nice day!

Best Regards

Stephan

Reply