Google IMAP All Mail folder not syncing with Inbox folder

When accessing GMail through Mail.app, I sometimes don't get email in my inbox that appears in my IMAP "All Mail" folder. Sometimes it eventually syncs, others I have to restart the app to get it syncing again. I setup the account according to Google's instructions, but am still running into this issue.
It's really an annoyance to have to check 2 places for new email, considering my other IMAP and POP accounts all filter into the inbox without issue.

Any ideas?