SMTP error (cannot send) when sending rich text format (RTF) messages

Question

Level 1

0 points

SMTP error (cannot send) when sending rich text format (RTF) messages

OS X 10.4.11 and Mail 2.1.1 AND OS X 10.5.1 and Mail 3.1

All of these, I believe, were released on 11/15/07, which is the same date we started having the problems described below. At 2 a.m. that same date -- yes, before I knew about the OS X updates -- I began moving our mail server to a new machine and because it was a newer server OS I had to upgrade the mail server software, too. How is that for an unfortunate coincidence?

Here are the problems:

WITH Preferences, Composing SET to Message Format: Rich Text and/or Use the same message format as the original message CHECKED

THEN doing ANY of the following:

Replying to a message in RTF format that has a URL link,
Forwarding a message in RTF format that has an attachment or a URL link,
Creating a new message in RTF format with an attachment or URL link

WHEN sending Mail connects to the SMTP server and after receiving the 354 command from the server it times out.

During the connection Activity Viewer shows "Delivering message Sending Data (100% done)". I believe it is referring to the DATA command it sends to the mail server. After the connection times out Mail gives an error dialog that shows "Cannot send message using the server mail.domain.com The server response was: An error occurred while delivering this message via the SMTP server "(null)" " etc.

The mail server is Mdaemon 8.1.3 running on Windows Server 2003. The transcript from the Mdaemon session log for one of these unsuccessful SMTP sessions is:

Mon 2007-11-26 15:04:10: ----------
Mon 2007-11-26 15:04:21: Session 9044; child 6; thread 1040
Mon 2007-11-26 14:54:22: Accepting SMTP connection from [70.62.213.174 : 60457]
Mon 2007-11-26 14:54:22: --> 220-domain.com ESMTP MDaemon 8.1.3; Mon, 26 Nov 2007 14:54:22 -0500
Mon 2007-11-26 14:54:22: --> 220 Unauthorized relay prohibited. All connections are logged.
Mon 2007-11-26 14:54:22: <-- EHLO [192.168.1.108]
Mon 2007-11-26 14:54:22: --> 250-domain.com Hello [192.168.1.108], pleased to meet you
Mon 2007-11-26 14:54:22: --> 250-ETRN
Mon 2007-11-26 14:54:22: --> 250-AUTH=LOGIN
Mon 2007-11-26 14:54:22: --> 250-AUTH LOGIN CRAM-MD5
Mon 2007-11-26 14:54:22: --> 250-8BITMIME
Mon 2007-11-26 14:54:22: --> 250 SIZE 0
Mon 2007-11-26 14:54:23: <-- MAIL FROM:<user1@domain.com>
Mon 2007-11-26 14:54:23: --> 250 <user1@domain.com>, Sender ok
Mon 2007-11-26 14:54:23: <-- RCPT TO:<user2@domain,com>
Mon 2007-11-26 14:54:23: --> 250 <user2@domain.com>, Recipient ok
Mon 2007-11-26 14:54:23: <-- DATA
Mon 2007-11-26 14:54:23: Creating temp file (SMTP): e:\mdaemon\queues\temp\md50000022800.tmp
Mon 2007-11-26 14:54:23: --> 354 Enter mail, end with <CRLF>.<CRLF>
Mon 2007-11-26 15:04:21: Connection timed out!
Mon 2007-11-26 15:04:21: SMTP session terminated (Bytes in/out: 100/410)
Mon 2007-11-26 15:04:21: ----------

(KEY: --> = Mdaemon server, <-- = Mail client)

This example is from one local account to another local account but it doesn't matter, it also happens when sending to a non-local account. Note that the Mail client reports that the server responded with "an error occurred" but the server's session transcript does not show sending an error response.

This ONLY happens with the above OS and Mail versions. It does not happen with OS X 10.4.10 and Mail 2.1 except for a notable exception, which is if a 10.4.10/2.1 user is replying to or forwarding one of the three types of emails described above that originated from a 10.4.11/2.1.1 user (and, I presume, a 10.5.1/3.1 user although I have not tested that).

It does NOT happen when sending those types of emails using Entourage on OS X 10.4.11, Thunderbird on OS X 10.4.11, Outlook on Windows XP Pro or Outlook Express on Win XP Pro from the same LAN accessing the same mail server.

It also DOES happen with OS X 10.4.11 - Mail 2.1.1 users and OS X 10.5.1 - Mail 3.1 users accessing the same mail server from other locations, i.e. at their home, wireless hot spots, etc.

The WORKAROUND is to change the Preferences to always send messages using plain text. Also, opening any of the messages that can't be sent from the outbox, changing them to plain text and sending them again will get them sent successfully.

But we need a permanent solution. We need to be able to send rich text messages, especially with URL links, and attachments.

Help please. Thanks in advance. Jeff.

Mac Pro, Mac OS X (10.4.11)

Posted on Nov 27, 2007 12:56 PM

Reply

Answer 1

Best reply

Desertmac

Level 1

20 points

Nov 27, 2007 5:43 PM in response to Jeff Rx

Thanks very much for your post. I don't have a fix, but can add some data. I have no problem sending any kind of emails from my Verizon FIOS network using Mail. But when I am using my AT&T AirCard, I get exactly the behavior that your are describing, and the fix you recommend (converting outgoing mail to Plain Text) works as well.

Reply

Answer 2

Jeff Rx Author

Level 1

0 points

Nov 28, 2007 4:55 AM in response to Desertmac

And thanks much for your post Desert. Very helpful because I have two internet connections here.

Primary connection is TWC (RoadRunner Biz Class) 3 Mbs X 2 Mbs Cable used for our general internet traffic. Secondary connection is AT&T/Local ISP T1 used for failover and VPN to our co-located servers at that ISP, including the mail server.

I did some more testing this morning after pulling the TWC cable wire so all traffic went out and in the T1.

I was able to forward RTF messages with attachments that previously would not send.
I still could not send an RTF message with a link UNTIL I stopped them, selected them in the Outbox, selected Send Again in the Message menu and hit send. I did not change them to Plain Text.
I still could not send a new RTF message with an Office doc attached even trying the above resend procedure.

All messages that could not be sent gave the same results in the Activity Viewer and final error message.

Reply

Answer 3

Nov 28, 2007 7:19 AM in response to Jeff Rx

Thanks Jeff. That's helpful info. More info on my side: I get the same problem behavior when I am using T-Mobile at Starbucks or the airport.

Reply

Answer 4

Nov 29, 2007 9:34 AM in response to Jeff Rx

I don’t know what’s going on here, but since changing the way you’re connected to Internet appears to make a difference, I’ll suggest some basic network troubleshooting that I do know Mail is susceptible to.

Go to Apple Menu > System Preferences > Network, choose Network Port Configurations from the Show popup menu, and make sure that the configuration used to connect to Internet appears at the top of the list. Leave checked (enabled) only the port configuration needed to connect to Internet and Built-in Ethernet (in that order if not the same), uncheck (disable) the rest of network port configurations and see whether that helps — if it doesn’t, turn ON again the ones you want enabled.

Reply

Answer 5

Nov 30, 2007 4:32 AM in response to David Gimeno Gost

Thanks for the suggestion. It was a good thought. Unfortunately, didn't solve the problem. Only the service that I am using to connect to the internet is activated under all of the problem situations I've encountered.

Reply

Answer 6

Nov 30, 2007 5:37 AM in response to Jeff Rx

Quick question: does the problem persist if you either

1. set Mail to use your ISP's outbound SMTP server instead of the local Mdaemon server, or
2. use mail server software other than Mdaemon?

If yes to either, then great catch on your part, I think you've found a bug.

Thanks,
Jonathan

Reply

Answer 7

Jeff Rx Author

Level 1

0 points

Nov 30, 2007 9:23 AM in response to jonathandl92

Very good suggestions. I did try using TWC's outbound SMTP server and it worked. I was able to forward an RTF message with an attachment and a new RTF messages each with a link and an attachment.

However, I still think this may be an issue with these versions of mail because, the routing is completely different:

When using our outbound SMTP server over our RoadRunner connection routes through Level3's network (one of TWC's tier 1 backbone provider).

When using the TWC outbound SMTP it the routing appears to go another way. I say appear because after 7 hops the routers stop responding to ICMP requests. Regardless, it's obviously going a different route.

Is is possible Apple changed the way messages are encoded and it is getting thrown out on some networks, depending on their router and firewall settings, and not others? That is my best theory right now.

I can still test another SMTP server but I don't think I have to since the above test worked, wouldn't you say?

Reply

Answer 8

Tom Gewecke

Level 10

115,800 points

Nov 30, 2007 9:34 AM in response to Jeff Rx

Is is possible Apple changed the way messages are encoded and it is getting thrown out on some networks, depending on their router and firewall settings, and not others? That is my best theory right now.

It seems hard to believe that encoding has changed or could be an issue. But if you have access to the various kinds of messages, you could compare the exact encodings (both charset and content-transfer) by doing View > Message > Raw Source on them, either from a Sent folder or the inbox, and see if there are any differences. Note that when you use the menu or preference settings called "Rich Text" in Tiger Mail, you are actually creating email in html and not in "Rich Text Format" (= .rtf).

Reply

Answer 9

Dec 1, 2007 10:34 PM in response to Jeff Rx

Interesting observations. Maybe it is an incompatibility between Mail and Mdaemon?

However, here is a question that maybe somebody more technically oriented than myself might be able to answer: does Mail, at the end of each line of message DATA, send <CR> only, or <LF> only, or <CR><LF>? I believe that the <CR> character is mandatory.

Reply

Answer 10

Tom Gewecke

Level 10

115,800 points

Dec 2, 2007 5:30 AM in response to jonathandl92

I believe that the <CR> character is mandatory.

Any reference for that? Aren't all Unix systems just LF?

Reply

Answer 11

Jeff Rx Author

Level 1

0 points

Dec 10, 2007 8:59 AM in response to Tom Gewecke

Sorry for the delay. Perhaps it is not the encoding but when Mac OS X Mail versions as described above sends Rich Text (HTML as you correctly observed -- sorry, just used to the RTF designation) it is sending mail differently.

I ran some tests while running Wireshark to capture packets. The significant difference between sending Rich Text and Plain Text appears to be in the protocols used.

Here is what I found:

Protocols used when Mail sends a Rich Text message: [Protocols in frame: nettl:eth:ip:tcp]

Protocols used when Mail sends a Plain Text format message: [Protocols in frame: eth:ip:tcp]

And for comparison,
when Entourage (on the same Mac) sends a Rich Text format message it uses: [Protocols in frame: eth:ip:tcp]

and when Outlook Express under Win XP sends a Rich Text format message: [Protocols in frame: eth:ip:tcp]

The only thing I know about nettl is it's a type of unix protocol (HP-UX) used for capturing packets and I know that only from googling it.

I do think it is curious, however, this protocol is only used in the mails that will not send as described in my opening post.

Anyone have any ideas about this?

Thanks. Jeff.

Reply

Answer 12

Jeff Rx Author

Level 1

0 points

Dec 14, 2007 9:44 AM in response to Jeff Rx

I solved the problem by changing all 22 of our OS X users to use port 587 and SMTP auth.

Please do not reply to this by saying that my ISPs were blocking port 25. If that were the case then Mail would not send plain text messages either, nor could I send rich text using Thunderbird or Entourage. I could do all of those things.

As to why rich text can be sent through some networks and not others, here is my theory:

Those networks with routers (or firewalls, too, I suppose) set to do stateful inspection of SMTP packets. Because Mail v2.1.1 (Tiger) and v3.1 (Leopard) are sending the nettl protocol in the frame when sending mail using rich text formatting (including attachments and links), these routers are dropping the packets. Since these routers are only inspecting traffic for port 25, they let traffic for port 587 go through.

To implement this solution, your mail server must accept SMTP connections on port 587 in addition to port 25. (Most mail servers are capable of this but it may have to be turned on. Also, SMTP authentication may or may not be required, or optional, using port 587 depending on your mail server.) Go to your Mail preferences, Account settings. Change the SMTP server settings server port to 587. If your mail server requires authentication on port 587 enter your account name and password. The first time you send a message you will be asked for your password. Enter it and check the box to save it in the Keychain (unless you want to be asked every time you send).

Whether Apple says this is a bug or not I am betting the next versions of Mail do not use the nettl protocol when sending messages using rich text (html) formatting.

That's all I got. Jeff.

Reply

Answer 13

Jeff Rx Author

Level 1

0 points

Dec 14, 2007 9:49 AM in response to Jeff Rx

See my last post. Thanks to all who tried. Special thanks to desertmac for the informative posts and jonathandl92 for the suggestions.

Reply

Answer 14

Dec 15, 2007 9:20 AM in response to Jeff Rx

Thanks for starting this thread Jeff and for seeing it through. I hope your prediction for future Mail versions is accurate.

Reply

Answer 15

Jeff Rx Author

Level 1

0 points

Dec 17, 2007 11:37 AM in response to Desertmac

And thanks for your posts Desert... they are what got me thinking about the network piece. Are you able to use port 587 on your SMTP server and, if so, does that make a difference on the AirCard network and on T-mobil at those hotspots? Yeah, I hope I'm right about the next versions, too.

Reply