Found this on Google forums.
2. Select the calenars your want tot sync. Click "Save".
3. Go to Calendar.app, hit Command-R to refresh and sync calendars.
This seemed to solve the issue for me after trying everything else (deletion, cache clearing, etc). Not sure how long, but it's working for now. I'll update if it fails again.