How to prevent Automator's "run shell script" to create fully decomposed forms of my strings ?

It's not an encoding issue, but normalization, e.g. NFC (precomposed) vs NFD (decomposed). Unfortunately I don't know enough about the stuff you are working with to suggest a way to get it the way you want...

Jean-Christophe Helary wrote:

how to make sure that what I input and what it outputs are the same form (ie which command to add inside the service to ensure that this is the case).

You probably need a command which converts any string to Unicode Normalization form C. One app which has this as an individual service is UnicodeChecker.

ok, I had some time to test this out. if you download and install the satimage osax, then you can use statements like the following to perform unicode-sensitive regexp directly in applescript:

set txt to "été"

set txt2 to find text "^(é.)" in txt using "\\1" with regexp and string result

set the clipboard to the result

if you feed non-decomposed characters into this, you will get non-decomposed characters out of it (though I don't think it's smart enough to mix the two.)

Stumbled upon the same problem while trying to pass Japanese characters to a shell script as arguments. It works perfectly well on the Terminal; When passed as an argument via an Automator service, it all breaks down.

I discovered it was decomposing characters eg: turning ど (U+3069) into ど (U+3099), which looks about the same but was breaking the rest of my script.

Here's what worked for me, getting Automator to run an applescript which calls the shell script and passes the text to it as an argument.

Automator only decomposes the characters when calling a Shell script directly not when calling an Apple script. So, you don't need to re-write your shell-script. ^^~

Hope this helps the next guy with the same problem ^^~

Ran into a similar problem recently and resolved the issue by setting the following shell variable in my Run Shell Script action:

LC_CTYPE=UTF-8