Mail doesn't always get my emails

For a while now, I seem to be having a problem with Mail collecting my emails.

Everything seems okay, emails come in and I can send, but then I'll get a call from someone asking if I have seen an email that they sent a few hours previously. I check "Mail" and although sunsequent emails have hit my box, older ones haven't.

I then go into "Mailbox" take all accounts offline, wait a few seconds and then take all account back online.

Low and behold I then quite often get several emails from earlier in the day that for some reason didnt hit my in box.

Any ideas of why this might be and what I can do to maybe stop this problem?