Im also getting the same problem.
Ive tried deleting passwords through keychain, force stopping Caldav processes in Activities, and deleting and reinstalling Calendar accounts.
Im on 10.8 and using delegates (8).
Here's what I get:
8/13/12 5:01:17.479 PM CalendarAgent[59836]: [com.apple.calendar.store.log.caldav.queue] [Account refresh failed with error: Error Domain=CoreDAVHTTPStatusErrorDomain Code=401 "The operation couldn’t be completed. (CoreDAVHTTPStatusErrorDomain error 401.)" UserInfo=0x7fe89606d290 {AccountName=Google, CalDAVErrFromRefresh=YES, CoreDAVHTTPHeaders=<CFBasicHash 0x7fe8938b9e40 [0x7fff78720190]>{type = immutable dict, count = 10,
entries =>
0 : Case Insensitive Key: X-Content-Type-Options = <CFString 0x7fe893895e80 [0x7fff78720190]>{contents = "nosniff"}
1 : Case Insensitive Key: Content-Type = <CFString 0x7fe893895c80 [0x7fff78720190]>{contents = "text/html; charset=UTF-8"}
2 : Case Insensitive Key: Server = <CFString 0x7fe8938d3310 [0x7fff78720190]>{contents = "GSE"}
3 : Case Insensitive Key: Transfer-Encoding = <CFString 0x7fff78b9c6b8 [0x7fff78720190]>{contents = "Identity"}
6 : Case Insensitive Key: Date = <CFString 0x7fe8938e4080 [0x7fff78720190]>{contents = "Mon, 13 Aug 2012 16:01:17 GMT"}
7 : Case Insensitive Key: X-Frame-Options = <CFString 0x7fe8938c17c0 [0x7fff78720190]>{contents = "SAMEORIGIN"}
8 : Case Insensitive Key: X-XSS-Protection = <CFString 0x7fe896016b70 [0x7fff78720190]>{contents = "1; mode=block"}
9 : Case Insensitive Key: Www-Authenticate = <CFString 0x7fe8938e0630 [0x7fff78720190]>{contents = "BASIC realm="Google CalDAV""}
11 : Case Insensitive Key: Cache-Control = <CFString 0x7fe8938f1c60 [0x7fff78720190]>{contents = "private, max-age=0"}
12 : Case Insensitive Key: Expires = <CFString 0x7fe896002560 [0x7fff78720190]>{contents = "Mon, 13 Aug 2012 16:01:17 GMT"}
}
}]