Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

10.10.4 Mail SMTP problem

Hi,

I know, that there are other discussions regarding Apple Mail not sending mails via SMTP. Most of these SMTP-servers do not use a strong DH key (logjam). But my problem is different.

I'm using a self operated mail server with dovecot (and dovecot SALS) and postfix. The server already uses strong DH keys and strong encryption. TLSv1.0 is available, but not v1.1 or higher.

Actually I had no problems before 10.10.4. The problems started after I upgraded to 10.10.4.

I use a payed Google Apps account in combination with my own SMTP server for sending mails. So Google Apps IMAP for incoming, my own SMTP server for sending mails.

I tried changing the configuration, but it simply does not work. Apple Mail connects, but sends no password.

The servers mail.log says simply:

Jul 13 10:09:21 aldur postfix/smtpd[28176]: warning: unknown[x.x.x.x]: SASL LOGIN authentication aborted

The connection log says (garion is my MacBook):

Jul 13 10:09:47 garion Mail[1346] <Debug>: Connected: <MFSMTPConnection: 0x60000057a580> (Connected) account: A{SMTP - 534CDE8D-59E7-4698-8A0E-ABF14A273AB5}

hostname: hostname.domain.de, port: 465, security layer: kCFStreamSocketSecurityLevelTLSv1_0

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> EHLO (19 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-hostname.domain.de

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-PIPELINING

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-SIZE 110000000

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-VRF

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-ETRN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-AUTH PLAIN LOGIN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-AUTH=PLAIN LOGIN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-ENHANCEDSTATUSCODES

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250-8BITMIME

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 250 DSN

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> AUTH (5 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 334 (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 334 (12 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> * (0 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 501 5.7.0 (22 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] >> QUIT (0 additional bytes)

Jul 13 10:09:47 garion Mail[1346] <Debug>: [0x60000057a580] << 220 (28 additional bytes)

Interessting are the bold lines. Apple Mail successfully connects to my SMTP server via TLSv1.0. It sends the EHLO and starts AUTH (first bold line). Thre server responds "give my your login name" (second bold line). Mail responds "here it is" (third bold line, 12 Byte BASE64 encoded login name). Then servers asks "ok, now give me your password" (fourth bold line). And Apple Mail respons with 0 bytes, so an empty password.

But the password is correctly set. It's stored in keychain, too.

My first solution was to use Airmail 2 and wait until a solution may be discovered (I upgraded to 10.10.4 when it was released). But up to now, there is no fix...

The problem is clearly Apple Mail, because other clients do not have any problems. Even the local postfix on my MacBook works. I configured the local postfix on my MacBook to forward all e-mails to my SMTP server, using PLAIN authentication. This works without problems.

Currently I configured Apple Mail to send mails through the local postfix (which forwards them to my SMTP server...).

But this can't be the final solution, as Apple Mail should be able to do it, too.

Currently I'm out of ideas... Maybe someone else can help.

Best regards,

Eike Hoffmann

MacBook Pro (Retina, 13-inch, Late 2013), OS X Yosemite (10.10.4), null

Posted on Jul 13, 2015 1:32 AM

70 replies

Jul 16, 2015 8:23 AM in response to eike.hoffmann

It's good to have confirmation.

I'll be curious to hear if anyone using the 10.10.5 beta would confirm that this issue has been addressed.

Jul 16, 2015 11:56 AM in response to eike.hoffmann

Hey eike.hoffmann,

I've got the exact same problem as you do. A number of different SMTP servers I use with Mail are always being sent a * for the password. Both AUTH PLAIN and AUTH LOGIN don't work. But, I've also seen Google's XOAUTH2 work successfully with Mail. I did break out Wireshark to see if the actual protocol matched Mail's debug logs. Using unencrypted SMTP, I did see the single raw * sent as a password.

Obviously all my testing has been done with OS X 10.10.4 which is when my problems started too. I also saw Apple released the 10.10.5 Beta two days ago. No specific notes on what changed other than "bug fixes". Anyone willing to be a guinea pig and install it to see if the problem goes away? 😉

Chris.

Jul 16, 2015 12:00 PM in response to chrissa51

Chris -

Have you managed to see any rhyme or reason why it might work sometimes, but not work other times? It would be really nice if we might be able to pin down why it stops working, so we can know what to avoid doing. Does Googles XOAUTH2 work consistently, or is it also a hit-and-miss event?

I'm dubious we'll get to hear much from the 10.10.5 Beta until it goes live, unfortunately. But, it would be okay for me to be wrong.

ASnoeren

Level 1

0 points

Jul 16, 2015 12:02 PM in response to Lionchild

FWIW Thunderbird works fine for me too, so it does not seem a Keychain problem. Gmail (w/XOAUTH2) works for me 100% of the time, and the server with regular AUTH LOGIN 0% of the time.

Jul 16, 2015 12:25 PM in response to eike.hoffmann

Hi Chris and ASnoeren,

so it is 100% a problem of Mail.app and AUTH PLAIN/LOGIN. I think we can not solve the problem without help from Apple.

I could test 10.10.5 beta2 (I'm a registered developer), but my wife would kill me if I install beta software on "my MacBook for testing and development" (for that I bought it), because now it's here MacBook... 😉 She threw away here Windows laptop...

I think I open a bug report tomorrow. And I have some open support cases, maybe they have more information.

Good, that I'm not the only one with this problem.

Best regards,

Eike

P.S. Out of curiosity, where are you from. You start posting, when my day is almost finished 🙂

CTP_9

Level 1

0 points

Jul 16, 2015 2:37 PM in response to eike.hoffmann

I'm also experiencing this problem connecting to my company's SMTP server using Mail. I am able to connect to and send through Gmail with SMTP and the Mail app consistently.

I am also able to successfully send mail through the same account using Thunderbird. Additionally, I was able to connect and authenticate from OpenSSL on the command line.

Jul 16, 2015 3:10 PM in response to Lionchild

I've yet to see SMTP with PLAIN or LOGIN work on 10.10.4 with any of my testing. I've only done a short bit of testing with Google's XOAUTH2, and so far so good. In the Keychain, passwords for XOAUTH2 are stored very differently, and the password is not actually passed back and forth during a regular login. Everything is token based. Google's SMTP server also does support PLAIN and LOGIN, but my laptop doesn't use it since setting up a brand new account through Mail.

I'm not sure how Mail chooses which AUTH method to use. Those who claimed that they couldn't send initially with Google, then deleted their accounts from Mail, recreated them, and then had their mail start sending again, I suspect Mail was using an old authentication method like PLAIN for the old account. When they recreated the account, Mail got smart and started using XOAUTH2. That checkbox "Automatically detect and maintain account settings" that everyone suggests to turn on is probably what triggered Mail to switch to XOAUTH2 on a new account.

I've also seen Mail attempt AUTH CRAM-MD5 and DIGEST-MD5. Neither of those worked. It keeps sending an * when it gets time to do something with the password.

Eiki, I'm located in western Canada. I sometimes don't get to replying to emails till a little later in the day. I also hear you about applying the Beta. I make my living with my laptop, and I'm not willing to apply the Beta either. I've installed Postbox for the short term to use as my Mail client for sending. Not terribly impressed with its performance, but it sends which is what counts right now.

Chris.

jfdoby

Level 1

0 points

Jul 16, 2015 5:59 PM in response to eike.hoffmann

I hate to rain on the collected replies, but I don't think it is a Mail application problem. We upgraded 1 MacBook to 10.10.2 on Tuesday/Wednesday [July 14/15] on a network of three MacBooks. All four use a SMTP server of a private service, but only the upgraded MacBook uses Mail. The others use Thunderbird. Starting Thursday morning, July 16, around 0900 PDT, all of our Macs could not communicate with the SMTP server. But all of them could receive and communicate with the POP server for inbound eMail. Both servers require a password, but we are using older 6 and 7 character passwords. We got voice mail from our private service that our IP address, presumably the associated SMTP, had been blocked. They unblocked the IP address, but want us to change to the more secure passwords. While I have not yet gotten the full technical information about which IP address and how it could be blocked, I am totally perplexed, based on your collected comments, on how this can happen. During the outage, I tried pinging the private services POP servers and made contact but could not get a ping response from the SMTP servers. I have yet to dig into logs, so I can not give any more details at this point. I suspect the issue is with 10.10.x trying to solve LOGJAM problem.

CTP_9

Level 1

0 points

Jul 16, 2015 7:57 PM in response to jfdoby

I tried strengthening my password and attempting to connect again, but the problem remains. The password does not appear to be sent during the authentication process.

The Logjam issue is related to weak DH keys for TLS connections on the target server and I do believe the Apple updates were related to improving their security in this area. I manage the SMTP server that I am trying to connect to. We made sure that we had a 2048-bit DH group deployed and that we excluded weak DH ciphers.

I am able to successfully send mail through the server using Thunderbird from my Mac. I know many other people that are able to send mail through this SMTP server from PCs, mobile devices and other mail clients on a Mac device. This all suggests that the problem is not with the SMTP server or even the client device, but specifically with the Mail app itself.

Jul 16, 2015 11:30 PM in response to CTP_9

A small conclusion:

The problem is not related to weak DH group keys: my SMTP server uses strong, self generated 2048bit DH)
The problem is not related to a weak encryption cipher: my server uses TLSv1.0 with DHE-RSA-AES256-SHA.
openssl command line client connects successfully.
The problem is not related to IP blocking or filtering: my server does not block my IP and I tried different networks (work network, private network with dialup connection, mobile 4G network, ...).
The problem is not related to weak passwors, as I use a password with 10 characters, randomly generated from lower- and uppercase characters, with digits and special characters.
The problem is not related to other mail clients: Thunderbird, Outlook (Mac Office 2016), Airmail 2, ... and a lot of other clients work without any problems. And even with the openssl command line client I can authenticate successfully using the SMTP protocol.
It is not a configuration problem, as I tried almost (if not all) all combinations of settings in Mail.app.

The problem seems to be related to:

It is a specific problem of Mail.app, ...
if the SMTP server only supports plain text authentication methods, as AUTH MECH=PLAIN or LOGIN (or both).

In this case Mail.app sends a "*" as the base64 encoded password. When it should send the base64 encoded password.

There is definitely something wrong with Mail.app. To not support PLAIN and/or LOGIN authentication (over a secure connection) would be a problem for a lot users.

jfdoby

Level 1

0 points

Jul 16, 2015 11:49 PM in response to CTP_9

Thanks for the reply and after reading Eike's logs and talking to the user of the upgraded MacBook, I have to agree that this looks like a Mail application problem. That MacBook sat idle for 1 1/2 days after the upgrade, while the Thunderbird Macs were in continual use. Only after the upgraded MacBook sent its first request for SMTP connection did things come unglued. All the Macs point to the same SMTP server with Port 587 open. The Thunderbird set ups use None for Connection Security, a Password transmitted insecure, and requires the user to enter the password manually for the first outbound transmission. For the Mail set up, the SMTP server is the same as Thunderbird but without password authentication; presume that is buried in Keychain. Sorry that I am not that knowledgeable about the Mail app, I personally don't use it.

What I do not understand is how Mail can send or not send a message to a third party SMTP server that would cause that server to block an IP address, while Thunderbird does not cause the same problem. Also, I am bothered by Chris's comment "I've yet to see SMTP with PLAIN or LOGIN work on 10.10.4 with any of my testing.". Does this mean I have to insure that Mail is always using XOAUTH2 and that my third party server is configured this way? Will know more on Friday after upgrading the password for the Mac using Mail. Hopefully, this will stop locking out our IP address.

Doby

Jul 17, 2015 12:29 AM in response to ASnoeren

yes, I see exact the same results!

Jul 17, 2015 1:09 AM in response to eike.hoffmann

Now it's getting strange, really strange!

I found a workaround.

Try this: Open the smtp server list and create a working SMTP server entry, e.g. use Google or something. This SMTP server must be working!

Now follow these steps.

Make sure the working SMTP server is selected as the sending server for your account.
Make sure the checkbox "Use only this server" (in german "Nur diesen Server verwenden", I don't know whats written there in the englisch version of Mail).
Check that everything is working: Window > Connection Doctor. Everything needs to be green.
Quit Apple Mail
Start Apple Mail again and open the SMTP server list settings dialog. Make sure that the working SMTP server is selected for the account when opening the SMTP server list!! I think this is the important part.
Open Window > Connection doctor. It should now check the connections and everything needs to be green.
Create a new SMTP server entry for the SMTP server which did not work before. Use the same settings as before. For me this is Port 465, Use SSL=YES, Auth=Password, Loginname + Password.
Close the SMTP server list settings.
Now Connection doctor should NOT check the new SMTP server connection.
Close the settings dialog completly.
Click on "Check again" (or something similar, in german it is "Erneut prüfen") in Connection doctor. Now connection doctor should check the SMTP server entry we created in step 7.
And voila it should be green.
Now open settings dialog and use the old SMTP server as the sending server for your account.
... and everything should work again without problems.

I tried these steps 5 times. Everytime it works. The SMTP server works with the same settings as before using this procedure to set it up.

That's strange.

Looks like Mail has a serious bug in settings dialog.

EDIT: After sending one mail, the server went offline again. Sh**. Connection doctor now says authentication failed, again. But after creating the account with the above steps it said it was working.

There is something going on with auto configuration....

EDIT2: If you disable "Automatically detect and maintain account settings" on the "Advanced"-Tab of the mail account AND in the SMTP server settings, the SMTP server stays online and can send mails. But then, at least for me, Mail can't connect to the incoming IMAP server (it is Google).

I guess Mail the setting "Automatically detect and maintain account settings" influences which authentication method is used. For the incoming mail server (Google IMAP) it is XOAuth2. Then Mail.app thinks it should be XOAuth2 for the SMTP server, too. Seems that the configuration for the incoming and the outgoing mailserver is mixed up if booth are set to use "Automatically detect and maintain account settings" and "Password" as authentication method.

This is a bug.

ASnoeren

Level 1

0 points

Jul 17, 2015 1:19 AM in response to eike.hoffmann

I cannot get your work-around to work either unless I do as you suggest in EDIT2 (disable the "auto detect" for incoming as well); perhaps because I am not quick enough in sending a message! It does indeed show it working in the connection doctor immediately after (re-)creating the server, but then stops working.

Luckily for me, I do seem to still be able to pull down incoming mail with the "auto detect" feature off, so at least I think I'm back in business for now.

Thanks, Eike for all of your detective work here! Hopefully there's enough for Apple to fix the **** bug.

Jul 17, 2015 1:27 AM in response to ASnoeren

EDIT3: Some more things I found out. When creating the SMTP server using the steps above the password for the SMTP account is correctly saved. When I close the SMTP settings dialog and reopen it without setting the newly created SMTP server entry as the active SMTP server I can see that the password is saved (the dots...).

Now, when I set the newly created SMTP server entry as the active SMTP server and reopen the SMTP server settings the password field is empty. So Mail lost the password ... but ony if "Automatically detect and maintain account settings" is enabled in both places.

@ASnoeren: Yes, you are right. Sending only works, if you disable this auto detect feature. I accidently used the other server for sending 😉. All seems to be related to EDIT3. Mail app loses the password if auto detect is enabled and the server is set as the active server for sending.

10.10.4 Mail SMTP problem