Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

El Capitan Mail.app causes massive load on Cyrus IMAPd server

Hello,


Updating three of our 40 users to El Capitan has caused a service-affecting emergency for the entire company. There is some sort of interop problem here. I admit it could be a misconfiguration of our Cyrus IMAPd server, but we use it successfully with a great number of IMAP client applications, Apple Mail.app included, until that all changed on October 1.


Background:

Cyrus IMAPd maintains three files for each IMAP mailbox:


1. cyrus.header

2. cyrus.index

3. cyrus.cache

These files can grow to be very large (especially the cyrus.cache file)



For some reason, this latest version of Apple Mail seems to get itself into a tight loop talking to our Cyrus IMAPd, causing the cyrus.cache to get repeatedly re-written (or at least written to heavily). With only three employees' IMAPd clients checking their email, the disk sub-system of the IMAP server is almost completely consumed, resulting in high IOWAIT times, terrible performance for all other clients, and even rejection of new incoming connections (because of the extreme load).



Steps to Reproduce:


1. Configure Apple Mail to connect to my email account (which contains a lot of messages, and a lot of folders)

2. Wait until mail check is completed.

3. Open Mail.app's connection doctor, tell it to show detail, and an intense stream of "EXPUNGE, OK Completed" scrolls past.



Expected Results:


We expect to see an idled IMAPd connection once the initial mail check is complete - it should not checking again for 15 minutes as we have configured it do.



Actual Results:


Disk I/O on our IMAPd server is thrashed, and we see the following type of messages repeating over and over in Mail's logs:



Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16156.18 OK

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16157.18 EXPUNGE

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16157.18 OK

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16158.18 IDLE

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16158.18 OK

Oct 5 22:06:07 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16159.18 EXPUNGE

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16159.18 OK

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16160.18 IDLE

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16160.18 OK

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16161.18 EXPUNGE

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16161.18 OK

Oct 5 22:06:08 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16162.18 IDLE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16162.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16163.18 EXPUNGE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16163.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16164.18 IDLE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16164.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16165.18 EXPUNGE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16165.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16166.18 IDLE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16166.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16167.18 EXPUNGE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16167.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16168.18 IDLE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16168.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16169.18 EXPUNGE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16169.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16170.18 IDLE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16170.18 OK

Oct 5 22:06:09 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16171.18 EXPUNGE

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16171.18 OK

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16172.18 IDLE

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16172.18 OK

Oct 5 22:06:10 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16173.18 EXPUNGE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16173.18 OK

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16174.18 IDLE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16174.18 OK

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16175.18 EXPUNGE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16175.18 OK

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16176.18 IDLE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: +

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Wrote: DONE

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Idle mailbox=Trash - darren@dazza]> Read: 16176.18 OK

Oct 5 22:06:11 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16177.18 EXPUNGE

Oct 5 22:06:12 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * EXISTS 459

Oct 5 22:06:12 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: * RECENT 9

Oct 5 22:06:12 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Read: 16177.18 OK

Oct 5 22:06:12 Mail[17422] <Debug>: <0x7f8f890c8a30:[Selected mailbox=Trash - darren@dazza]> Wrote: 16178.18 IDLE


There's many thousands of these lines in Mail's syslog.


I'm a bit of a newcomer to Apple's forums, please let me know if there's other information I can provide.


-Darren

Mac Pro (Late 2013), OS X El Capitan (10.11)

Posted on Oct 6, 2015 7:19 PM

Reply
30 replies

Oct 12, 2015 11:28 AM in response to dazza321

Hello,


I have the same Problem. I'm no the server admin but a client. The problem is with my University's IMAP.


Other IMAP Servers do not show this problem.


I see the exact same behavior in Mail.apps connection log. This problem is killing my internet connection and my battery.



Is someone out there who knows a solution?


-Carsten

Oct 18, 2015 7:29 PM in response to dazza321

Same here--our server admins are going batty trying to figure out how to mitigate it.


IMAP connections to iCloud and GMail don't seem to suffer from this problem so it must be something about Cyrus IMAPd it doesn't like.

Running El Cap's Mail.app turns into a denial of service tool!


Apple sometimes likes to revise some code where it more strictly adheres to a given protocol standard.

Since it's just Cyrus, I'm wondering if it's a bug or loosely implemented standard on their end?


Anyone know what the scuttlebutt is on the cyrus channels?

Oct 18, 2015 7:32 PM in response to dazza321

I have an open bug report with Apple (via Apple Bug Reporter). The bug ID is 22996765, but the case is still very preliminary ... no indication that it's getting any attention other than a request for more details, which I have provided. If anyone knows how to get this escalated to Apple's attention, please point them here.

Oct 21, 2015 10:18 PM in response to Mactotum

Sadly, this problem still occurs with 10.11.1. Nothing has changed.


It's always the same. I can't use this mail account.

Mail.app is even ddosing the mail server when I disable 'Store deleted Messages on the server'.


Im not a mail Expert but:


All other mail servers send as a response to EXPUNGE - 'OK EXPUNGE COMPLETED'.


This Server only sends OK COMPLETED. The IMAP specs says that OK EXPUNGE COMPLETED is the correct answer. Maybe Apple built their IMAP lib strict. It's the last idea that came to my mind.


Here are a few lines from the connection log:

WROTE Oct 22 07:07:50.496 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7edb8300

3434.426 EXPUNGE



READ Oct 22 07:07:50.547 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7edb8300

* 2 EXISTS

* 0 RECENT

3434.426 OK Completed



WROTE Oct 22 07:07:50.550 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7edb8300

3435.426 IDLE



READ Oct 22 07:07:50.578 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7edb8300

+ idling



WROTE Oct 22 07:07:50.579 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

DONE



READ Oct 22 07:07:50.609 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

3435.426 OK Completed



WROTE Oct 22 07:07:50.609 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

3436.426 EXPUNGE



READ Oct 22 07:07:50.641 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

* 2 EXISTS

* 0 RECENT

3436.426 OK Completed



WROTE Oct 22 07:07:50.644 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ed92650

3437.426 IDLE



READ Oct 22 07:07:50.672 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ed92650

+ idling



WROTE Oct 22 07:07:50.673 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

DONE



READ Oct 22 07:07:50.703 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

3437.426 OK Completed



WROTE Oct 22 07:07:50.703 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

3438.426 EXPUNGE



READ Oct 22 07:07:50.741 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa7ba0ebe0

* 2 EXISTS

* 0 RECENT

3438.426 OK Completed



WROTE Oct 22 07:07:50.744 [kCFStreamSocketSecurityLevelTLSv1_0] -- host:mailbox.uni-due.de -- port:143 -- socket:0x7ffa7e3150c0 -- thread:0x7ffa795b9700

3439.426 IDLE

Oct 22, 2015 3:40 AM in response to IanEiloart

IanEiloart wrote:


Don't forget that you have to close the Preferences dialog box, or click on another account, to save the preference changes.


Why Apple didn't put an "apply changes" button there, I don't know. It's terribly confusing when experimenting with different changes in order to get things working!

Thanks but I already knew this. And the problem still occurs.

Oct 22, 2015 3:15 PM in response to dazza321

I've been working with our Cyrus mail admin. He found that the root of the behavior is some augmenation in the expunge dialog between Mail.app and the server. Essentially, Apple expanded its expectations beyond the normal dialog to want more information. The result is some confusion betwixt the client and the server where the client is expecting something that isn't coming.


To mitigate the issue, we found that disabling the mailbox behavior "Move deleted messages to the Trash" and then removing the "Compact mailboxes automatically" option under the advanced tab (don't forget to do the apply changes dance first or it's greyed out) stops this looping madness.


It's still a problem with the default settings, but if you get the client users to change these two settings, it appears to halt the behavior.


Give it a crack and see if your results match up.

Oct 22, 2015 4:03 PM in response to jStephens8

Not really for this one. PRAM zaps are mysterious black magic but this seems to be a problem with the actual code that's requesting an expunge command. It's expecting the response to be qualified with extra arguments that are outside the spec.


Disabling the "Compact Mailboxes Automatically" inhibits the commands from executing as it crawls through the mailbox tree.

El Capitan Mail.app causes massive load on Cyrus IMAPd server

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.