There has been a lot of threads about this issue. Seems like gmail and apple calendar are not the two most compatible services.
I had a similar problem, where I would get email invites rather than notifications, although I could reply to the invitation without problems.
Anyway, I have found a solution for myself. If that doesn't work for you, I suggest you browse the forums here and at macrumors for more help.
My problem was a new and rather "hidden" setting in gmail. I say new because I have always used gmail, but the problem only started a year or two ago. And I say hidden because I had to look though gmail settings several times before I found it, even though after the fact it seems quite straightforward.
1. Log in to you Google account ON A DESKTOP COMPUTER (I tried on mobile and it didn't work for me).
2. Access your google calendar. One way is through the "9 squares" icon on the top right of your browser.
3. On the middle-left portion of your screen, you will see your various calendars listed (e.g. Personal calendar, contacts birthdays, holidays, etc.). The top one should be your primary calendar, and immediately to the left of it is a downward pointing arrow. Click on it to reveal an options menu.
4. In that options menu, choose "Settings".
5. In the settings menu, at the top, there is a navigation menu. Click on "Notifications".
6. Inside the notification menu, there are a number of options for Email and SMS alerts. DISABLE ALL NOTIFICATIONS.
7. Don't forget to save the changes.
For me that did the trick. I assume that when a new invite is being processed through iCloud, it first checks with the email address if there are notifications settings enabled, and if so let's it handle the invite. Which could actually be the desired behaviour, although it should be more obvious. Either that or Google Calendar somehow hijacks the process.
Anyway it seems that when there as no notification setup with Google, the invite god through to the Calendar Inbox as usual.
This problem was a thorn for me, so I hope that fixes it for you as well!
Info: Using iPhone 5S, with iOS7, but previously had the same probl with iPhone 3GS using iOS6. Have been using the same Gmail account since the beginning and always had my Apple ID ash gmail address.
Appreciate the detailed description. This did NOT work for me, infact, Google Calendar is not updating on my iphone (5s 7.03) quickly at all.
After I turned off notifications, as you instructed, I obviously did not get any emails or invites, but they also did not show up on my calendar...weird.
Basically, my inbox on ios cal is always blank even know there are invites in my gmail inbox.
Fairly frustrating...since with my iPhone 4 (even though it was slow as heII) and ios 6, and exchange/active sync set up, I got the invite badge in the inbox button on the bottom of calendar in ios 6. Now my inbox always is blank, even though the replied section shows that I have replied to invites.