Cannot Send Message With Selected Server - Error When Sending Multiple E-mails

When I try to send more than maybe 5 e-mails in a short span of time I get the following error:

Cannot send the message using the selected server.

Select a different outgoing mail server from the list below or click Try Later to leave the message in your Outbox until it sends.

If I try to use the selected server again, the e-mail duplicates (ie instead of one copy in my outbox there are now two of the same message) and the same error pops up. If I try later they just sit in my outbox and may or may not send on their own later or if I force try to send them again.

This happens daily, it's like Mac Mail is limiting how many e-mails I can send in a period of time.