Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

User profile for user: ferrerod
ferrerod Author
User level: Level 1
110 points

iOS 18 Mail Sync Issue with IMAP and Dovecot Server

Why did Apple discussion moderators suddenly lock the thread "Error in downloading mail on IMAP after ios 18 upgrade" when it was actively being posted and with thousands of users upvoting it?


I was just about to reply with this:


dovecot imap server is also having a sync problem. They are also seeing a problem with iOS 18. I'm honestly not convinced this PR is the best solution...

IETF IDLE RTC 2177 says:


The IDLE command is terminated by the receipt of a "DONE"

continuation from the client; such response satisfies the server's

continuation request. At that point, the server MAY send any

remaining queued untagged responses and then MUST immediately send

the tagged response to the IDLE command and prepare to process other

commands. As in the base specification, the processing of any new

command may cause the sending of unsolicited untagged responses,

subject to the ambiguity limitations.  The client MUST NOT send a

command while the server is waiting for the DONE, since the server

will not be able to distinguish a command from a continuation.


IETF IMAP4 rev1 RFC 3501 says:

...

68) Clarify that commands which break command pipelining must wait

for a completion result response.

...


I searched IETF IMAP docs for more information about "command pipelining" but turned up empty. The way I'm interpreting the specs for IDLE, the client MUST not send another command while server is waiting for "DONE" ...AND "commands which break command pipelining" must wait for the (server) to send the completion result response (before sending new commands)....

Although Stalwart Dev mdecimus believes his PR is the way to fix iOS 18, I dare say I think this is premature...


IMO this is Apple's bug to own and fix! 3 months in...come on.


[Re-Titled by Moderator]

Posted on Jan 4, 2025 7:39 PM

Reply
Question marked as Top-ranking reply
User profile for user: nxnw
nxnw
User level: Level 2
233 points

Posted on Jan 17, 2025 12:39 PM

ferrerod wrote:

FWIW, I uncommented the imap_capabilities flag in my conf.d/20-imap.conf file and added everything to that line except IDLE.

imap_capability =IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=GSSAPI

On a macOS server dovecot installation, I edited the the imap_capability key in /Library/Server/Mail/Config/dovecot/conf.d/20-imap.conf to read only:


imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE


I omitted "STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=LOGIN" because, when I included those items in imap_capability, they were doubled up in the response to  "telnet localhost 143". It appears that they are not supplied by imap_capability, because "telnet localhost 143" gets the desired response of the prior string, but leaving out IDLE: 


CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=LOGIN


This change was made two weeks ago and iOS mail has since been functional. Mail on desktops and notebooks has been a bit less functional – it is delayed 3-5 minutes since it presumably is no longer using IMAP idle. There are no settings in mac mail for idle, push or fetch frequency, so I don't now what mechanism is involved, but I am curious to know.

View in context
7 replies
Sort By: 
Question marked as Top-ranking reply
User profile for user: nxnw
nxnw
User level: Level 2
233 points

Jan 17, 2025 12:39 PM in response to ferrerod

ferrerod wrote:

FWIW, I uncommented the imap_capabilities flag in my conf.d/20-imap.conf file and added everything to that line except IDLE.

imap_capability =IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=GSSAPI

On a macOS server dovecot installation, I edited the the imap_capability key in /Library/Server/Mail/Config/dovecot/conf.d/20-imap.conf to read only:


imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE


I omitted "STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=LOGIN" because, when I included those items in imap_capability, they were doubled up in the response to  "telnet localhost 143". It appears that they are not supplied by imap_capability, because "telnet localhost 143" gets the desired response of the prior string, but leaving out IDLE: 


CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=LOGIN


This change was made two weeks ago and iOS mail has since been functional. Mail on desktops and notebooks has been a bit less functional – it is delayed 3-5 minutes since it presumably is no longer using IMAP idle. There are no settings in mac mail for idle, push or fetch frequency, so I don't now what mechanism is involved, but I am curious to know.

Reply
User profile for user: stephen boyle
stephen boyle
User level: Level 1
50 points

Jan 20, 2025 12:42 PM in response to nxnw

good catch - i see them doubled up as well so will modify my config.


FYI i was able to talk to Apple Business Support and worked with someone who did some digging and got back to me. He for sure had engineers check my feedback assistant submission outlining the issue and all the other information i provided regarding the IDLE functionality and patches.


At this point they confirmed that NO ONE ELSE has submitted this as an ACTIVE ISSUE with Business Support. The org this affects on my end is fairly small and doesnt move the needle at all as far as bug priority at Apple. I am curious if anyone else who is having this issue has been able to report the issue/bug properly and/or is a part of a larger org that they may care more about. I dont want this to sounds negative, as Apple has been very clear with test early, submit feedback early and provide number of affected users.


If anyone reading this cares and thinks they can move the needle, i have this submitted under Feedback Assistant (FB15701211) and Apple Support (102513275820).


At this point i dont really have a choice but to migrate to another locally hosted email service for this one org. Considering Stalwart as they were the ones that identified this issue and patched their own software quickly. Not sure iRedMail will really be able to fix issues if they are just bundling dovecot etc.

Reply
User profile for user: stephen boyle
stephen boyle
User level: Level 1
50 points

Jan 5, 2025 7:13 AM in response to ferrerod

yes quite frustrating that apple locked the other thread? no idea why?? (https://discussions.apple.com/thread/255760058?r_s_legacy=true&sortBy=newest_first&page=1)


i removed IDLE from one of our dovecot servers the other day and while it is the weekend now, reports have been positive. In our line of business no news usually means good news.... people like to only complain and not give positive feedback.


i have not scrolled all the way back in the folders on iOS but i do expect it to take some time to cache all the messages.


Overall removing IDLE is not a "fix" but a temp workaround with Apple being unresponsive to bug reports and feedback assistant tickets.


from the mail server side, i am seeing ~ 40% increase in imap connections and about 70% increase in outbound traffic on average which makes sense since FETCH is less efficient than PUSH.

Reply
User profile for user: nxnw
nxnw
User level: Level 2
233 points

Jan 17, 2025 12:02 PM in response to stephen boyle

stephen boyle wrote:

yes quite frustrating that apple locked the other thread? no idea why?? (https://discussions.apple.com/thread/255760058?r_s_legacy=true&sortBy=newest_first&page=1)

Well, I wouldn't want to speculate, but a lot of people were subscribed to the other thread, and therefore received posts before they were "moderated".

Reply
User profile for user: ferrerod
ferrerod Author
User level: Level 1
110 points

Jan 5, 2025 1:10 AM in response to ferrerod

FWIW, I uncommented the imap_capabilities flag in my conf.d/20-imap.conf file and added everything to that line except IDLE.


imap_capability =IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS XAPPLEPUSHSERVICE AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=GSSAPI


After removing IDLE from the capabilities flag and reloading the config files, mail started to sync to iOS 18 Mail. This is strange...but glad to have email flowing to my iPhone over several months!


I also noticed what appears to be a very HEAVY use of dovecot by the mail client as it tries to use a new 'categorization' feature of Mail in iOS 18...it seems to read every single mail and try to categorize it --- even when I have chosen NOT to use the new Categorization feature in iOS 18 Mail.


Lastly I was looking at how far back I could scroll within the iOS 18 Mail, and there are many many emails missing from iOS....older mail....but I see them on macOS Sequio 15.2 Mail. I don't think this is a dovecot issue. I think this is a Mail app parsing issue. I don't know if it breaks with the new Categorization feature OR if there's another parsing bug that's causing the iOS client to not show older emails. I will check my iPad running iOS 17 and see if it can see the older emails and report back.

Reply
User profile for user: ingo-t-h
ingo-t-h
User level: Level 1
8 points

Jan 22, 2025 7:15 AM in response to ferrerod

Some observations from my side. MacOS holds the connection to the mail server on the IMAP port if the Mail app is open. IPadOS does the same; here you can close the app, but it looks like it runs in the background further. Here in both cases, the connection is refreshed every 5 minutes. On iOS 18, I see the connection is immediately closed if I close the Mail app. If I leave it open, the connection to the server is closed after around 3 minutes and no reestablishing of it. If the connection is closed, no IDLE is possible. Looks like there is something wrong with the network, maybe energy settings, which leads to a closing of the connection and no reestablishing to the IMAP server. I see this if I look in the connection list of my firewall.

Reply

iOS 18 Mail Sync Issue with IMAP and Dovecot Server

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.
Learn more Sign up