10.10.4 Mail SMTP problem

Question

10.10.4 Mail SMTP problem

Hi,

I know, that there are other discussions regarding Apple Mail not sending mails via SMTP. Most of these SMTP-servers do not use a strong DH key (logjam). But my problem is different.

I'm using a self operated mail server with dovecot (and dovecot SALS) and postfix. The server already uses strong DH keys and strong encryption. TLSv1.0 is available, but not v1.1 or higher.

Actually I had no problems before 10.10.4. The problems started after I upgraded to 10.10.4.

I use a payed Google Apps account in combination with my own SMTP server for sending mails. So Google Apps IMAP for incoming, my own SMTP server for sending mails.

I tried changing the configuration, but it simply does not work. Apple Mail connects, but sends no password.

The servers mail.log says simply:

Jul 13 10:09:21 aldur postfix/smtpd[28176]: warning: unknown[x.x.x.x]: SASL LOGIN authentication aborted

The connection log says (garion is my MacBook):

Jul 13 10:09:47 garion Mail[1346] <Debug>: Connected: <MFSMTPConnection: 0x60000057a580> (Connected) account: A{SMTP - 534CDE8D-59E7-4698-8A0E-ABF14A273AB5}

hostname: hostname.domain.de, port: 465, security layer: kCFStreamSocketSecurityLevelTLSv1_0

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> EHLO (19 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-hostname.domain.de

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-PIPELINING

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-SIZE 110000000

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-VRF

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-ETRN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-AUTH PLAIN LOGIN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-AUTH=PLAIN LOGIN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-ENHANCEDSTATUSCODES

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-8BITMIME

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250 DSN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> AUTH (5 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 334 (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 334 (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> * (0 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 501 5.7.0 (22 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> QUIT (0 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 220 (28 additional bytes)

Interessting are the bold lines. Apple Mail successfully connects to my SMTP server via TLSv1.0. It sends the EHLO and starts AUTH (first bold line). Thre server responds "give my your login name" (second bold line). Mail responds "here it is" (third bold line, 12 Byte BASE64 encoded login name). Then servers asks "ok, now give me your password" (fourth bold line). And Apple Mail respons with 0 bytes, so an empty password.

But the password is correctly set. It's stored in keychain, too.

My first solution was to use Airmail 2 and wait until a solution may be discovered (I upgraded to 10.10.4 when it was released). But up to now, there is no fix...

The problem is clearly Apple Mail, because other clients do not have any problems. Even the local postfix on my MacBook works. I configured the local postfix on my MacBook to forward all e-mails to my SMTP server, using PLAIN authentication. This works without problems.

Currently I configured Apple Mail to send mails through the local postfix (which forwards them to my SMTP server...).

But this can't be the final solution, as Apple Mail should be able to do it, too.

Currently I'm out of ideas... Maybe someone else can help.

Best regards,

Eike Hoffmann

MacBook Pro (Retina, 13-inch, Late 2013), OS X Yosemite (10.10.4), null

Posted on Jul 13, 2015 1:32 AM

Reply

Answer 1

ASnoeren

Level 1

0 points

Jul 14, 2015 2:09 PM in response to eike.hoffmann

FWIW, I am experiencing this *exact* same problem, but not with a home-brew SMTP server--with a major university's SMTP server. Everything used to work fine, but now Mail.App refuses to send a password (my logs look the same as the above). I have tried removing the entires from the Keychain, as well as removing and re-adding the SMTP server to Mail.App, all with no success. Any suggestions would be much appreciated.

Reply

Answer 2

Jul 14, 2015 2:44 PM in response to ASnoeren

Just to clarify some things: My "home brew" server is nothing experimental. I usually know what I do 🙂 (I think). This SMTP server is used by about 80 clients with very different mail clients (Thunderbird, Outlook, Lotus Notes, Apple Mail on OS X < 10.10.3) without any problems. And until the upgrade to 10.10.4 I had no problems, too. The problems started with the upgrade to 10.10.4.

And it is not a problem of weak encryption or a weak Diffie Hellman key. I corrected these issues shortly after logjam vulnerability was published. I even experimented with different settings for the SMTP server to get it working again, but it did not work. As the logs show, Apple Mail successfully connects via TLS to my SMTP server.

I tried removing passwords from the keychain, too. Did not work. I tried removing and adding the SMTP server, did not work. I tried removing and re-adding the account, did not work.

Mail does not send the password.

@ASnoeren: You could configure the postfix MTA of your OS X installation to forward your mails to the server of your university. Then you configure Apple Mail to sent mails trough your local postfix installation (just a SMTP server at localhost, without authentication). This is what I did to switch back from Airmail 2 to Mail. Using the local postfix installation works with SSL/TLS (when forwarding to my SMTP server) and authentication without any problems for me.

Here is a short tutorial how to do it: http://benjaminrojas.net/configuring-postfix-to-send-mail-from-mac-os-x-mountain -lion/

It is for sending mails with PHP, but that does not matter. SMTP is SMTP 🙂

Reply

Answer 3

Jul 15, 2015 12:11 PM in response to eike.hoffmann

Have you tried editing the Accounts.plist file, as suggested here: Mail SMTP (outgoing) does not work after Yosemite upgrade

And if so, does that seem to be a stable fix, or just a fix until the issue crops up again?

Reply

Answer 4

Jul 15, 2015 12:45 PM in response to Lionchild

Yes, I tried that. But it did not work -- no difference. The option mentioned in this thread is also available in the SMTP server settings, so you do not need to edit the plist file. But I tried that, too. Made no difference, eighter with vi or using the option in the settings.

Reply

Answer 5

Jul 15, 2015 12:57 PM in response to eike.hoffmann

Yeah, I noticed it's available in the actual settings, but....interestingly enough, I can have the in-app settings one way, while the plist file is the other.

The first time I went into the file, my app had it checked, but the plist indicated it was false. Only after you toggle it in the application does it sync up.

...

Actually, after a little testing, ...sometimes they sync up, but mostly they don't. I can set the app one way, close it and change the plist the other. When I open the app it's still the one way while the plist is the other, even if I close the app again.

Reply

Answer 6

Jul 15, 2015 1:19 PM in response to Lionchild

Yes, I know. The setting in the plist file is always false, even if it's set to true in-app. That's why I tried with vi. The setting was on true in the plist file (and stayed true) after manually editing it, but it did not change the behavior of Mail.app. No mails send...

Maybe it is something with the AUTH MECHS. My SMTP server only provides PLAIN and LOGIN. I can't use MD5 challenge/response, because the user passwords are encrypted with a different (and more secure hash). Both PLAIN and LOGIN are not secure when used without connection encryption, but as the connection is secured by TLS it should work. And even GMail does provide only these AUTH MECHS.

I'm out of ideas currently...

Reply

Answer 7

ASnoeren

Level 1

0 points

Jul 15, 2015 1:33 PM in response to eike.hoffmann

Yes, I am pretty sure it has something to do with the fact the server only supports LOGIN. I have no trouble using gmail's SMTP server, which uses XOAUTH2. (Manually editing the plist didn't work for me either.)

Reply

Answer 8

Jul 15, 2015 1:50 PM in response to eike.hoffmann

@ASnoeren Google uses XOAuth2, but I'm pretty sure Mail.app does not use this. XOAuth is very new and the list in Mail.app says -- see screenshot (sorry, it's german) ... there is no XOAuth or something similar. So it uses Password, which is PLAIN or LOGIN.

Reply

Answer 9

ASnoeren

Level 1

0 points

Jul 15, 2015 1:51 PM in response to eike.hoffmann

Not according to my logs:

WROTE Jul 07 20:04:58.519 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:smtp.googlemail.com -- port:25 -- socket:0x6080004a5ca0 -- thread:0x610000a63dc0

AUTH XOAUTH2 dXN....

Reply

Answer 10

Jul 15, 2015 2:00 PM in response to ASnoeren

Yes, that's using XOAuth2. ****.

But it starts auth:

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> AUTH (5 additional bytes)

It should not start auth, if no usable mech is available. It think it's time for tcpdump...

Reply

Answer 11

Jul 15, 2015 3:39 PM in response to eike.hoffmann

This is the detailed log. Everything after // is a comment. ehoffman is my username. xxx.de is not the real server name.

-------------------------------

// Connection is kCFStreamSocketSecurityLevelTLSv1_0

READ Jul 15 23:57:04.584 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

220 xxx.de ESMTP Postfix

WROTE Jul 15 23:57:04.590 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

EHLO garion.xxx.local

READ Jul 15 23:57:04.637 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

250-xxx.de

250-PIPELINING

250-SIZE 110000000

250-VRFY

250-ETRN

250-AUTH PLAIN LOGIN

250-AUTH=PLAIN LOGIN // So we have PLAIN and LOGIN

250-ENHANCEDSTATUSCODES

250-8BITMIME

250 DSN

WROTE Jul 15 23:57:04.640 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

AUTH LOGIN // Mail.app chooses LOGIN

READ Jul 15 23:57:04.688 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

334 VXNlcm5hbWU6 // Username:

WROTE Jul 15 23:57:04.688 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

ZWhvZmZtYW4= // Yeah, that's me: ehoffman

READ Jul 15 23:57:04.736 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

334 UGFzc3dvcmQ6 // Password:

WROTE Jul 15 23:57:04.818 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

* // Mail.app responds with an asterisk. No, this is not my password and I did not hide it. The asterisk is exactly what Mail.app send to the SMTP server

READ Jul 15 23:57:04.865 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

501 5.7.0 Authentication aborted // Sure as password is junk.

WROTE Jul 15 23:57:04.866 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:xxx.de -- port:465 -- socket:0x6080004b5c00 -- thread:0x60000166ec00

QUIT // And bye.

--------------------------------------------------------------------------

That's crazy. Why an * ?

Reply

Answer 12

iW00

Level 6

8,488 points

Jul 15, 2015 3:50 PM in response to eike.hoffmann

Did you see this documentation: Configuring Mail in OS X Yosemite v10.10.4 to connect to unsecure mail servers - Apple Support

Is it any good for you?

Reply

Answer 13

Jul 15, 2015 4:15 PM in response to iW00

No, I did not know this documentation. Thank you for the link.

But this is not the problem. As I said before: The server is not unsecure. The connection is established with TLSv1.0. The DH key is 2048bit. It is a new, self generated DH key.

The following screenshot shows the output of Console.app when sending an email (as described in the linked documentation).

The failed SASL authentication step (line 3 and 4 in the screenshot) is sending the password (see my logs in the other posts), I guess.

I'm sure this is a really stupid, simple problem. Maybe not even related to Mail.app. Could be Keychain or something else.

"Unable to find a callback: 32775" sounds like "I did not get the password when I asked for it."

Reply

Answer 14

Jul 16, 2015 8:15 AM in response to eike.hoffmann

Eike -

This seems like an issue specific to the Mail App. Have you tried any other mail applications, such as Thunderbird or Airplane to see if the issue follows to the new application, and therefore might be related to the Keychain? Or, in the least, rules out the Keychain.

Reply

Answer 15

Jul 16, 2015 8:21 AM in response to Lionchild

@Lionchild

Yes, indeed. I tried Airmal 2 (and worked with it for some days) and Outlook. Both have no problems. So this issue is indeed specific to Mail.app. :/

Reply