Q: OCSP Service using up quite a bit of bandwidth

Feel like I'm beating on a dead horse here, but wanted to throw this out there, as I found it very strange.

On my iMac, OCSPD keeps calling to DEVIMAGES.APPLE.COM

On my Mac Mini, the calls are to www.apple.com

I've turned off checking in the Key prefs, and even created a firewall rule on my router to prevent connectivity to devimages.apple.com.  Thought it was strange that one went to a 'dev' IP and the other to a www.

I did…

$ sudo rm -rf /var/root/Library/Caches/ocspd

$ sudo ls -lah /var/root/Library/Caches/ocspd

-->> ls: /var/root/Library/Caches/ocspd: No such file or directory

Rebooted

sudo ls /var/root/Library/Caches/ocspd

-->> ls: /var/root/Library/Caches/ocspd: No such file or directory

I left this for an hour or so (with Mail & App Store open)

The oscpd process appears but the folder remains non-existant, 0 bytes data is shown in Activity Monitor, I toggled the CRL & OSCP settings in Keychain Access. It had no effect.

I recreated the folder & rebooted, the files were created, new fsCacheData appeared. Downloads have started again. It's been running for 30 minutes & the incoming data is sitting at 1.2 MB.

The files in /var/root/Library/Caches/ocspd are now

4.0K  2 Dec 02:08 Cache.db

32K  2 Dec 02:20 Cache.db-shm

443K  2 Dec 02:20 Cache.db-wal

306B  2 Dec 02:20 fsCachedData

My system.log only has my commands that were run via sudo, All messages shows nothing.

I do have the following in an old system.log, possibly after the install completed. It repeats twice.

Oct 29 07:04:35 localhost mDNSResponder[50]:  12: Could not write data to clientPID[83](ocspd)  because of error - aborting connection

Oct 29 07:04:35 localhost mDNSResponder[50]:  12: DNSServiceGetAddrInfo      v4v6 crl.apple.com. PID[83](ocspd)

Linc, do you know how to log the data a process is sending? Watching Activity Monitor for a process that disapears is frightfully dull .

I'd be happy dumping numbers into a text file for later calculation if it would help. I'd guess the akamai server names that bdiamond18 is blocking are probably going to change in the future (isn't it a CDN?). Who know what other Apple services that will break too?

I can run wireshark/ tshark if that is the only way to count the data. It's a test setup on a spare disk so it can be erased & reimaged.

I guess it may be a good time to pick one person to erase one of those DB files to see if we can find a specific one to address the problem?

Permissions looked fine on the files before the initial deletion (-rw-r--r--  1 root  wheel), so perhaps the db was in a bad state after install?

It's been running for 30 minutes & the incoming data is sitting at 1.2 MB.

Is that better than before?

Watching Activity Monitor for a process that disapears is frightfully dull

You don't have to. It shows a cumulative total for each process. Mine is about 6 MB over a few days.

Yes it's better, it was a several MB downloaded first time around with about 4.4 MB Cache.db-shm

My ocspd deamon disappears & returns, no log messages suggest a crash or sudden termination. I can only assume it has stopped due to no more requests from Apps. It's an on-demand daemon, so it should spawn at will right?

Any idea on what tasks could keep it running? It returns with 0 bytes in Activity Monitor.

I'll leave it running overnight & compare to a reimaged OS tomorrow if possible.

It runs on demand by registering a Mach service, with WebKit processes as the clients.

@drew

I deleted the contents of /var/root/Libary/Caches/ocspd and rebooted (I didn't delete the actual directory as there didn't see any point).  The three files were recreated but NOT the fsCachedData directory.

-rw-r--r--   1 root  wheel    4096  2 Dec 08:40 Cache.db

-rw-r--r--   1 root  wheel   32768  2 Dec 08:40 Cache.db-shm

-rw-r--r--   1 root  wheel  613912  2 Dec 08:42 Cache.db-wal

I renabled the OCSP and CRL options in Keychain Access preferences.... and waited....

Slightly worrying log entry at around the same time:

Dec  2 08:41:14 hostname.deleted.com usernoted[213]: Connection does not have the proper entitlement (com.apple.private.notificationcenter-system) to connect to the system notification center. All communication will be denied. center com.apple.storeagent

The log entry that I always associated with a OCSP download event:

Dec  2 08:41:40 hostname.deleted.com storeagent[353]: multibyte ASN1 identifiers are  not supported.

Has not occured since 08:41 when the machine booted.

Two hours in and the files have not been written to since 08:42 (ie. when they were created).

Machine has downloaded a total of 4.49Mb in the two hours (ie. nothing of consequence).

Not sure what my next steps are going to be, but I think I have to look into the log entry above and check that things are working as expected.

Would anyone advise re-creating the fsCacheData directory, or should I just wait and see if the OS does it for itself when it's needed?

Thanks to everyone whose been involved in this discussion, it does feel like we are making some progress...

Update:

Started the Appstore which initated a small certificate download from Akamai.

New file sizes:

-rw-r--r--   1 root  wheel    4096  2 Dec 08:40 Cache.db

-rw-r--r--   1 root  wheel   32768  2 Dec 10:57 Cache.db-shm

-rw-r--r--   1 root  wheel  659232  2 Dec 10:57 Cache.db-wal

And 'lsof | grep ocspd' produced:

ocspd     691            root  cwd       DIR                1,2      1224       2 /

ocspd     691            root  txt       REG                1,2    132752   12217 /usr/sbin/ocspd

ocspd     691            root  txt       REG                1,2     50744 2099199 /private/var/db/mds/system/mdsDirectory.db

ocspd     691            root  txt       REG                1,2     32768 2099202 /private/var/db/mds/messages/se_SecurityMessages

ocspd     691            root  txt       REG                1,2     32768 2099255 /private/var/root/Library/Caches/ocspd/Cache.db-shm

ocspd     691            root  txt       REG                1,2  23548880   48540 /usr/share/icu/icudt51l.dat

ocspd     691            root  txt       REG                1,2     71616 2100812 /private/var/db/crls/ocspcache.db

ocspd     691            root  txt       REG                1,2    600832   11337 /usr/lib/dyld

ocspd     691            root  txt       REG                1,2 343060674  986325 /private/var/db/dyld/dyld_shared_cache_x86_64

ocspd     691            root    0u      CHR                3,2       0t0     306 /dev/null

ocspd     691            root    1u      CHR                3,2       0t0     306 /dev/null

ocspd     691            root    2u      CHR                3,2       0t0     306 /dev/null

ocspd     691            root    3u   KQUEUE                                      count=2, state=0x2

ocspd     691            root    4u      REG                1,2      4096 2099239 /private/var/root/Library/Caches/ocspd/Cache.db

ocspd     691            root    5u      REG                1,2      4096 2099239 /private/var/root/Library/Caches/ocspd/Cache.db

ocspd     691            root    6u      REG                1,2    659232 2099254 /private/var/root/Library/Caches/ocspd/Cache.db-wal

ocspd     691            root    7u      REG                1,2     32768 2099255 /private/var/root/Library/Caches/ocspd/Cache.db-shm

ocspd     691            root    8u      REG                1,2    659232 2099254 /private/var/root/Library/Caches/ocspd/Cache.db-wal

ocspd     691            root   11u     unix 0x71451fa84179d6e3       0t0         ->0x71451fa840cb25b3

ocspd     691            root   13u     unix 0x71451fa840cb25b3       0t0         ->0x71451fa84179d6e3

ocspd     691            root   85u     unix 0x71451fa83cabf1cb       0t0         ->0x71451fa83d5815bb

And now back to waiting....

Please excuse the tiny graph, but it may help somehow.  From RubberNet, x axis is time, y download (top) upload (bottom).  Time span is a day and a half.  You can see very frequent downloads for the first 6-8 hours, then some user activity (me logging users out in the evening).  Sparse downloads follow for the night, bit of a blip in the early morning (middle of graph) as I checked what was happening, block of user activity about midday (2/3 of the way along).  The user left themselves logged in from then on, so this user doesn't seem responsible for increased download frequency seen at the start of the graph.  In fact, they were working solidly for a couple of hours towards the end where there's a notable gap in the traffic.

I am logging them all out bar an unlinked sys admin overnight, will see what it looks like then...

@emlynuk, could you tell if the multiple certificates returned were part of a (or a whole) CRL?

Linc Davis wrote:

 

It runs on demand by registering a Mach service, with WebKit processes as the clients.

Sorry Linc, that means almost nothing to me.

I took a look at Apple's Developer notes on the kernel & Mach services but that didn't made much sense either.

My oscpd process has changed PID, from 98 to 465 overnight. Is that normal for a Mach service?

It simply means I can't track network usage via the process.

Would anyone advise re-creating the fsCacheData directory, or should I just wait and see if the OS does it for itself when it's needed?

You don't need to recreate it. That happens automatically (or it doesn't.)

Linc Davis wrote:

 

Would anyone advise re-creating the fsCacheData directory, or should I just wait and see if the OS does it for itself when it's needed?

 

You don't need to recreate it. That happens automatically (or it doesn't.)

Thanks Linc, that was my conclusion too, but good to hear someone else say it

Coming up on 6 hours post deletion process and no unwanted downloads so far...

@Elrainia

My fscachedata reappeared after the second reboot. I expected oscpd to recreate it's own folder but it just didn't happen first time around.

It recreated 2nd time with the following permissions…

drwxr-xr-x  9 root  wheel   306B  2 Dec 02:20 fsCachedData

I'm afraid I need to figure out tcpdump to get an accurate idea of data usage, I was hoping ps or another command would show network usage per process but that doesn't seem possible to me. I don't think I need Little Snitch & really want to avoid buying it if possible.

@emlynuk,

Do you have a list of urls that seem to be only from oscpd?