Mail keeps popping up

Whatever I do, the Mail App keeps popping up and interrupting me. I noticed this happens whenever there is an issue with one of the e-mail addresses. For example, the servers for my Yahoo! e-mail addresses keep failing and whenever that happens mail keeps popping up, probably to let me know. I understand why this might be vital. You don't want to miss an important mail because the servers or the app is failing.

Is there a way to stop this behaviour, though? Maybe by playing around in Terminal? I would like the Mail App to never pop up by itself again, there must be something to restrict its ability to do that.