Mail takes a long time to load

I have about five accounts, some of them IMAP others Exchange. In the past month Mail became extremely slow to load and start. It opens the app right away but I don't see any email and all the account inboxes have the spinning gray lines next to their names. Sometimes it takes five to ten minutes and all of a sudden mails start to appear. The rest of the day it works fine, but I can't wait ten minutes everytime I open the app.

I tried a couple of weeks ago to delete the com.apple.mail and it seemed to work fine for a few days, then it went back to extremely slow. Is there any solution beyond loosing preferences or reinstalling stuff?

thanks