Mavericks Server Delegates Drop off

I have this exact same problem. I have no clue why or exactly when all delegates on all users calendars dissapears. It happened 3 imes over 2 months. I have about 10 users/clients. All latest software including the server on a mac mini.

Any suggestions appreciated..

Hi Guys,

I was wondering if you have aver found a solution?

Same issue here. Only have seen it happen on one account a couple of times now. I did get a 403 error in iCal both times. I was being connected over VPN. Don't know if that might be related to the issue though.

No. I still have the same problem. Happens like once monthly. Annoying as h.

It sounds like an OpenDirectory problem. The actual delegate assignments are stored in a sqlite database /Library/Server/Calendar and Contacts/Data/proxies.sqlite . You can view them via:

# sqlite3 proxies.sqlite

sqlite> .dump

[output such as:]

INSERT INTO "GROUPS" VALUES('0E2A527C-5BE8-48AF-9502-8362BE7A48DB#calendar-proxy-write','6D6241DC-59 81-4D87-9B71-672203E81ACB');

I am guessing that proxies.sqlite has the assignments still in there? Or is that sqlite DB getting emptied out? If OD says that a record is not there, then any delegate assignments for that record will be suppressed by calendar server to the client, but the assignments should still be in the proxies.sqlite for when the OD record reappears.

On (seemingly) random occasions iCal would come up with a 403 or 404 error when trying to connect to our iCal Server. Using a networking tool we noticed that the reverse DNS was not properly set for the servers (WAN) IP address: iCal was connecting with the reverse domain which our ISP uses by default.

After asking our ISP to fix the reverse DNS the issue of dropping delegates seems to be gone.

You might want to check your server's reverse DNS (PTR Record) asing Terminal command: dig -x 0.0.0.0

Where 0.0.0.0 is your server's WAN IP address. You'd want the PTR answer to read your server's DNS name.

PS: We also found our router had gone haywardwith it's UPnP feature (which we don't even need) which caused networking problems. Most likely not related to the iCal issue but we also deactivated UPnP on the router.

I have a customer that has this problem every day. Reverse DNS is perfect both inside the private network and outside on the public network.

Tarny, next time it happs can you examine the contents of the delegates database (proxies.sqlite) as described above?

The delegate entries were missing from the proxies.sqlite database. I examined it with a dump of the database and also later an SQL SELECT on the GROUPS table in that database. I haven't re-examined that database for some time as I worked out the following work around.

I managed to hack together a script using dscl and some looping which would allow me to create a sql import file using dscl and another file with a list of the account names like this:

PRAGMA foreign_keys=OFF;

BEGIN TRANSACTION;

INSERT OR IGNORE INTO "GROUPS" VALUES('1B2C18A8-A883-4A8A-9469-36480B512FFD#calendar-proxy-read','1F6D271F-43F 7-4849-B010-1175AE968966');

INSERT OR IGNORE INTO "GROUPS" VALUES('1B2C18A8-A883-4A8A-9469-36480B512FFD#calendar-proxy-read','268EFA56-90E D-4C97-92C8-FD27E4BFF8A7');

…many deleted lines here…

INSERT OR IGNORE INTO "GROUPS" VALUES('EEE68FB9-656B-4D91-B01D-3119B8E98CAA#calendar-proxy-read','D95EE638-5EF 2-45AD-A8F1-CB14AE60E9C1');

INSERT OR IGNORE INTO "GROUPS" VALUES('EEE68FB9-656B-4D91-B01D-3119B8E98CAA#calendar-proxy-read','E2BFD237-84F D-481A-8206-E114F111855A');

COMMIT;

As a workaround, I could then use the .read command in sqlite3 with this file above, to repopulate the Delegates into the calendar service. But, the users still had to go into their Calendar app:Preferences:Accounts:Delegation and check to show (re-show actually) all the calendars.

I've created a model test server by copying many configuration and data files to try to get the test server to fail in the same manner as the customers, but it hasn't yet and I can't take the production server offline for long enough to make an image copy (to make sure that I get the exact same configuration.)

At times the Calendar service is not responding to any client requests, even though it is apparently operating from the server's point of view. Then I have to stop/re-start the service. I've no idea if that is related to the Delegates problem.

/var/log/caldav/error.log has many entries for many users as follows:

2014-06-11 09:48:41-0400 [-] [caldav-4] [PooledMemCacheProtocol,client] [calendarserver.platform.darwin.od.opendirectory#error] Digest auth error: Error Domain=com.apple.OpenDirectory Code=5103 "Authentication server failed to complete the requested operation." UserInfo=0x7fafff000c70 {NSLocalizedDescription=Authentication server failed to complete the requested operation., NSLocalizedFailureReason=Authentication server failed to complete the requested operation.}

I'm not really sure if these are significant because these error messages appear from time to time when the Calendar service is working well.

Then we have many of these Python/Twisted error messages:

2014-06-11 09:46:50-0400 [-] [caldav-3] [PooledMemCacheProtocol,client] [twext.web2.server#error] Exception rendering error page

2014-06-11 09:46:50-0400 [-] [caldav-3] Traceback (most recent call last):

2014-06-11 09:46:50-0400 [-] [caldav-3] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twi sted/internet/defer.py", line 551, in _runCallbacks

2014-06-11 09:46:50-0400 [-] [caldav-3] current.result = callback(current.result, *args, **kw)

2014-06-11 09:46:50-0400 [-] [caldav-3] File "/Applications/Server.app/Contents/ServerRoot/usr/share/caldavd/lib/python/twex t/web2/server.py", line 679, in _cbFinishRender

2014-06-11 09:46:50-0400 [-] [caldav-3] d.callback(response)

2014-06-11 09:46:50-0400 [-] [caldav-3] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twi sted/internet/defer.py", line 368, in callback

2014-06-11 09:46:50-0400 [-] [caldav-3] self._startRunCallbacks(result)

2014-06-11 09:46:50-0400 [-] [caldav-3] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twi sted/internet/defer.py", line 464, in _startRunCallbacks

2014-06-11 09:46:50-0400 [-] [caldav-3] self._runCallbacks()

2014-06-11 09:46:50-0400 [-] [caldav-3] --- <exception caught here> ---

2014-06-11 09:46:50-0400 [-] [caldav-3] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twi sted/internet/defer.py", line 551, in _runCallbacks

2014-06-11 09:46:50-0400 [-] [caldav-3] current.result = callback(current.result, *args, **kw)

The most disturbing error message is:

2014-06-11 09:46:50-0400 [-] [caldav-0] [PooledMemCacheProtocol,client] [twext.web2.server#error] An error occurred. We tried to report that error. Reporting that error caused an error. In the process of reporting the error-reporting error to the client, there was *yet another* error. Here it is. I give up.

On my test server the error messages as pasted in my post directly above, occur occasionally. But apparently aren't related to the problems with the Delegates because the delegates are still there.

And the Delegates dissappeared again today.

Tarny, what version of Server.app are you running?

You say this happens every day? A specific time of day? Let's narrow down the possible sources of this. It could be:

1) A calendar client deleting them. This is unlikely if literally all of the assignments are disappearing. Can you grep through /var/log/caldavd/access.log for PROPPATCH entries? That's the command a client sends to either add or remove a delegation assignment. They'll look something like:

173.8.150.161 - morgen [13/Jun/2014:12:47:42 -0700] "PROPPATCH /principals/__uids__/6D6241DF-5983-4D87-9B71-672203E81ACE/calendar-proxy-read/ HTTP/1.1" 207 391 "-" "Mac_OS_X/10.9.3 (13D65) CalendarAgent/176.2" i=0 or=1 t=47.0

2) Calendar server is deleting them. There used to be an old version of the server that would notice delegates had gone missing from the directory service and then remove the relevant assignments, but that code has not been used in a few versions AFAIK, which is why I was asking what version you're running. Do you see any /var/log/caldavd/error.log entries with "Delegate is missing from directory" or "Removing missing proxy principal" ?

3) Something on the system is deleting them. What is the backup/restore system in place on the effected server? Is it possible the proxies.sqlite3 file is getting replaced nightly or something like that? The timestamp on the proxies.sqlite3 file should normally always match the most recent PROPPATCH access.log entry containing calendar-proxy-read or calendar-proxy-write. Does that timestamp line up?

System Version: OS X 10.9.3

Server app: 3.1.2

I think I have ruled out a rogue client deleting the delegates, it is everyone that is loosing the delegations. The PROPPATCH messages revealed that the last one before this mornings problem was at [12/Jun/2014:21:07:24 -0400]. The time stamp on the proxies.sqlite database was Jun 12 12:53 (in all the Time Machine backups) till this morning when I ran through my restart and repopulated the database routine because the problem had occured.

I checked the /var/log/caldavd/error.log very carefully, especially around the [12/Jun/2014:21:07:24 -0400] time. It was unhelpful. There were only two error message between the last PROPPATCH and the report of the problem:

• error while performing digest authentication
• Connection aborted - took too long to close

And I don't think those were significant because I see them all the time while the service is apparently operating normally. I think that there is something with in the Calendar servcie that is dropping the delegates, but it's at lower database level and isn't getting logged in that log file.

A SHA1 hash on the backup versions of the proxies.sqlite database revealed that it had not been changed from 2014-06-12-165704 to 2014-06-13-080343. My own changes to the database occurred about an hour after that last backup. And unfortunatly, I didn't have the presence of mind to make a backup copy of the bad database file first.

I've started a process to log a SHA1 hash of the database every minute so I can see when it changes next. So far it hasn't today, except when I purposfully changed a delegate to see the hash change.