Corrupted text in Mail app (iOS), Chinese characters in Spanish

Question

Level 1

4 points

Corrupted text in Mail app (iOS), Chinese characters in Spanish

I'm programmatically sending e-mail which contains Spanish text. When I receive this e-mail on iPhone (native Mail app), I can see that some letters of Spanish alphabet are replaced by Chinese symbols. All those characters replaced are letters with acute, such as "á", "é", "í", "ó" or "ú", or special symbols, such as "¡" or "¿".

When character is replaced, it actually takes two letters, so for example:

"¿N" is replaced by "燒"
"ác" is replaced by "塶"
"án" is replaced by "嫕"
"át" is replaced by "嫢"
"ér" is replaced by "廨"
"ít" is replaced by "癃"
"ón" is replaced by "鏮"

This issue is however not limited just to Chinese symbols, it also does following replacement:

"¡G" replaced by "：" which is ： / U+FF1A
"úm" replaced by "🕙" which is &#128345 / U+1F559

When I compare value of original two characters and resulting symbol, I don't really see any relation - offset for each combination is totally different - it doesn't look like there's any connection.

I have identified several causes of this happening:

e-mail contains encoded and non-encoded characters together: once e-mail body contains "á", but another time HTML encoded character is used "á"
when I remove just "¡" ("¡"), e-mail is rendered correctly despite not all characters are properly HTML-encoded
when I remove ~half of CSS styles defined in e-mail it is then rendered correctly (and I don't have to change anything else, like encoding of non-encoded characters)

CSS does not contain anything special, for repro purposes I even left empty classes in my message. HTML itself is cleaned out so there are no fancy inline styles, declared CSS is not even used at this stage. (Keep in mind that original e-mail is much bigger and realistic, my repro case is using minimum content to eliminate false traces)

I have submitted question on StackOverflow here: https://stackoverflow.com/questions/67672624 - it can provide more details about this issue.

Only Apple's iOS Mail application suffers from this, other e-mail clients, such as Outlook or GMail, are rendering very same e-mail correctly. What is causing this issue?

Posted on May 24, 2021 1:11 PM

Reply

Answer 1

Tom Gewecke

Level 10

118,282 points

May 25, 2021 5:50 AM in response to wdolek

My guess would be that iOS Mail is misreading the encoding of the text for some reason. I would start by comparing the byte sequences involved in those character strings under different encodings with the byte sequences of the Chinese characters under various encodings, to see if there is any logical fit. It's a tedious process and may not totally clarify the underlying bug. For an example of something similar on a different app/platform from long ago see this page.

You might want to report it to Apple via

http://www.apple.com/feedback

or their bug reporting system if you are developer with them.

My impression is that best practice in html is to always use the actual utf-8 characters rather than the corresponding ascii numerical character reference escapes.

Reply