Mail app constantly downloads messages, slowing performance

Mail constantly "Downloading Messages"

I know this issue is old and there are several threads related to the same problem, but I have not found an answer yet!

Long story short - Mail keeps this progress bar indicating that it is downloading thousands of messages (this number also varies, from a few messages to thousands...). While this is only mostly annoying, it also make Mail rather slow - which is a problem as I deal with many emails and different accounts.

Now:

• I have the latest update
• I've closed and opened Mail
• I've tried deleting all accounts and adding them again (temporary fix only)
• I've tried to rebuild the different mailboxes

Any usable ideas?

Thank you in advance