Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

woodmeister50 Author

Level 6

19,723 points

Caching Server service still broke for Mac App Store apps

Well, I thought I had found a fix to my issue of downloads from the Mac App Store not caching.

WRONG!

I explicitly set the flags, via terminal commands, for AllowPersonalCaching and AllowSharedCaching

as the two flags were not present in config.plist for the service. I then downloaded an app from the

Mac App Store on one of my Macs and it did actually get cached and served from the Caching Service.

Well, thought all was good, until today!

Saw that I had an update for 1Password app. So downloaded it from the MacApp Store onto my iMac.

Then I thought, better check my MiniServer to see if the Caching Service was still working for MacApp

Store downloads. Guess what? NOT!!! Did the update on two other Macs and the Mini Server and none

of them made the Caching Service budge!

However, all the Macs do see the caching server and a look in the log shows that the Caching Service is working

for local iCloud caching as when each Mac was powered up for the update I was attempting, there were multiple

PUTs and GETs for cloudd/xxx.xx.

Now, on top of that, the formerly working iTunes Store app update caching when downloading on the Macs is also

not working any more!!! It did however, serve an app directly to an iDevice that I tested.

So, has the Caching Service just basically a useless consumption of memory now in terms of working with Mac?

Has it turned into an iDevice only service? I have tried re-starting the service, shutting down the Server app, rebooting

the server all to no avail.

All the functionality of the service was fine under Yosemite. Somewhere along the line though, either El Capitan updates

or Server App updates have some how borked this service.

So, any clues or do people simply not use this service anymore and no one cares?

OS X El Capitan (10.11.4), Home Server

Posted on Apr 5, 2016 8:43 AM

45 replies

Leopardus

Level 4

1,303 points

Apr 29, 2016 5:05 AM in response to woodmeister50

You can set the LogClientIdentity to True to log the IP and port numbers. I have not tried it on the client. That value is normally set to 0 (which means any port) as far as I know.

Leo

Leopardus

Level 4

1,303 points

Apr 29, 2016 7:22 AM in response to woodmeister50

woodmeister50 wrote:

Also, on the client side, is there anywhere to find what a particular Mac is using for the

port for the service or does it simply try looking for a port that is active for this service?

It should look for any port afa I know. It is the value which is set to 0.

woodmeister50 Author

Level 6

19,723 points

Apr 29, 2016 8:44 AM in response to Leopardus

Okay, changed the port to 50190 and the following happened:

1. Client #1, opened and signed into MacApp Store. The following updates were available:

Software Update -> Camera Raw

iMovie

2. Ran the Camera Raw update. Server got the request cached and served.

3. Thought, okay maybe it is working now and updated iMove. No joy in Mudville!

Caching server never got any request. Checked in debug log which is set to verbose,

Showed the Camera Raw request and that was it.

4. Started up client #2.

Client #2 had the following:

Software Update -> Camera Raw

iMovie

One Cloud

1Password

5. Caching server delivered the Camera Raw update. Saw no request for any of the the app updates.

It did, however, see requests for iCloud caching part of service.

I saw a similar scenario as well on 2 other clients.

Being in the U.S., these other apps should have been cached and served but no request was even recognized.

As I don't have any other Software Updates (via MacApp Store) for Apple system stuff, I don't know if this was a

one shot deal (which happened once before) or just Software Update works.

Again, in neither the normal log viewed with the Server GUI or in the debug log set to verbose was

there any indication of a request.

Most times not caching isn't an issue as many apps are small but with some updates like Xcode which are typically GBs

it makes quite a difference.

woodmeister50 Author

Level 6

19,723 points

Apr 29, 2016 9:12 AM in response to Leopardus

Some additional info. It seems that the Caching service is working again for iDevice apps

updated via iTunes on a Mac client, which wasn't working.

So, it seems at this point, apps from the MacApp Store are not caching nor are requests recognized.

And until another Apple system update of some sort comes along, I'm not sure that it will work again

for that either.

Leopardus

Level 4

1,303 points

Apr 29, 2016 9:49 AM in response to woodmeister50

woodmeister50 wrote:

Okay, changed the port to 50190 and the following happened:

1. Client #1, opened and signed into MacApp Store......

I am sure I said 50910 or was this a typo?

Getting late here and still need to get home. When I wake up tomorrow morning at home, I will type out the steps for you to look into what is in the database itself and what is happening there, which you might not notice in the log files. Your stuff might just be there.

Leo

woodmeister50 Author

Level 6

19,723 points

May 1, 2016 4:09 AM in response to Leopardus

The 190 is a typo.

Yesterday, I booted up a Yosemite install and there were a few "Software Updates"

available via the MacApp Store (Security and Safari and a few others). It seems

the security update and the Safari update were recognized by the Caching Service

and dutifully cached. As this install did not have any MacApp Store updates available,

I was unable to verify if it worked on it.

At this point, it is not conclusive that the Caching Service is working for Software Update

until some El Capitan Software Update comes along. The caching always worked for

Yosemite.

Perhaps the issue may not be Server Caching but client requesting in El Capitan for

MacApp Store Apps? In many cases, having or not having the service may not be a

big deal as many cases updates are only a few megs. However,apps like Xcode and

some others like iMovie can be on the order of GBs, which is a big deal.

On the client side, is there a log that can be checked to see if all update requests

are being issued or even being refused?

Leopardus

Level 4

1,303 points

May 2, 2016 7:24 AM in response to woodmeister50

woodmeister50 wrote:

Yesterday, I booted up a Yosemite install and there were a few "Software Updates"

available via the MacApp Store (Security and Safari and a few others). It seems

the security update and the Safari update were recognized by the Caching Service

and dutifully cached. As this install did not have any MacApp Store updates available,

I was unable to verify if it worked on it.

At this point, it is not conclusive that the Caching Service is working for Software Update

until some El Capitan Software Update comes along. The caching always worked for

Yosemite.

I learned the following from Reid Bondonis - El Capitan Foundation Services (his ebooks are really worth every cent), and no I don't get anything nor do I wish any:

'Inside the Data folder (/Library/Server/Caching/Data), there is an SQLite database (AssetInfo.db) that keeps track of all the cached files. It can be viewed to discover what is on the server'

Use the sqlite3 command to access the database.

sudo sqlite3 /Library/Server/Caching/Data/AssetInfo.db

and authorise with admin password

This will put you in the command line interface. If you now enter .tables it will list the tables in the database. It will return this list:

The next step is to search for everything in the ZASSET table by typing after the prompt sqlite>

.mode column

.headers on

select * from ZASSET

It will return the list of all the assets which are stored by the Caching Server. The output will look like this (an extract)

from which you can see a lot of relevant information, like a serie Z_PK (in this instance ranging from 155 to 159), in the first row, and three columns later the size of the update, and in the second row the date and time for the last modification.

The final value is the package name of what was cached to the server(XProtectPlistConf). Thus, the method is there to verify what is contained in your caching server's data store. It is even possible to extract it if you really want to.

On the client side, is there a log that can be checked to see if all update requests

are being issued or even being refused?

I am not aware of any as yet, sorry! I am sure Reid Bondonis, John Lockwood, Pterobyte and MrHoffman would know better.

Remember, that the Caching Service will purge data which to stay within the limits that you have set. Thus some older or not recently accessed stuff could be gone. It is recommended that for legacy stuff the SUS is used.

HTH

Leo

Leopardus

Level 4

1,303 points

May 4, 2016 5:04 AM in response to woodmeister50

Hi WM50,

Have you checked whether the new downloads are in your asset database? That is the litmus test.

Leo

woodmeister50 Author

Level 6

19,723 points

May 5, 2016 7:13 AM in response to Leopardus

Sorry for not getting back sooner (busy with other things). I did as you asked and after

entering the command .tables, I get absolutely nothing. It just returns to the command

prompt. It was a few days since I did any downloads. Could the database been purged.

Then, there was an update for Xcode available. I then downloaded the update and simultaneously

watched the Network activity in Activity Monitor on the server. There was absolutely no change in

the network activity except for a few kB every now and then which I guess was related to Screen

Sharing or just other minor services. There was no indication of a 200MB+ download. I then ran

the command line test again and it yielded exactly the same results as before the download. Also,

there was nothing shown in the Debug.log file or the log available in the Server GUI that any request

was made. The only thing in Debug.log was the periodic cache usage status. There was no error

or indication of an aborted attempt. It was almost as if no request was even made.

I then saw that there were several iDevice app updates in the iTune Store. I proceeded to download

those updates to the Mac and those that were cached were served from cache and those that weren't

were downloaded and cached. Debug.log and the server GUI log were flooded with the info for those

update requests. Network and disk activity verified the above as well.

Now here is something interesting, I did the command line sql thing again after this, and the results were

identical to the previous attempts. Nothing displayed and a return to the command prompt after entering

the .tables command. Odd!?!??

woodmeister50 Author

Level 6

19,723 points

May 5, 2016 7:28 AM in response to Leopardus

Now this looks like a problem, don't know the fix but I did the following:

sudo ls -al /Library/Server/Caching/Data

and shows the following for AssetInfo.db:

-rw-r--r-- 1 root _assetcache 0 May 5 09:38 AssetInfo.db

size is 0 bytes. Not sure if those would be the correct permissions.

If this is hosed, how is the caching for iTunes app downloads working

or iCloud stuff or anything?

Leopardus

Level 4

1,303 points

May 5, 2016 11:30 AM in response to woodmeister50

You do seem to have a problem with your setup somewhere.

Make a backup. You have the option of resetting the Caching Server, but I think that you would have done it given your experience.

You could do the following to inspect the last couple of lines from the Debug log to see some or more (just replace the 12 with another number to see less):

tail -n 12 /Library/Server/Caching/Logs/Debug.log

In terms of completeness for;

sudo ls -al /Library/Server/Caching/Data

this is what the output (taken from my test setup), should more or less look like:

total 5712

drwxr-x---@ 43 _assetcache _assetcache 1462 May 4 00:54 .

drwxrwxr-x 5 _assetcache _assetcache 170 Feb 4 13:49 ..

drwxr-x--- 3 _assetcache _assetcache 102 Apr 5 21:54 03CED5A2-9F00-44AA-A7A0-510D1DAB3FDC

drwxr-x--- 3 _assetcache _assetcache 102 Apr 19 19:14 08F26B13-9720-44AB-94DE-9A3C06025DFD

drwxr-x--- 3 _assetcache _assetcache 102 Apr 21 19:33 13CF0180-6645-48DE-99C1-D241375918D0

drwxr-x--- 3 _assetcache _assetcache 102 Apr 7 09:58 2500BF48-63B6-4827-BD4B-15F315C132C8

drwxr-x--- 3 _assetcache _assetcache 102 Mar 30 05:47 2BE755AA-5CE1-487B-9EAD-7DC680815CAF

drwxr-x--- 3 _assetcache _assetcache 102 May 3 20:54 2E5145CF-F7A3-4755-ABFC-45CBB5A75129

drwxr-x--- 3 _assetcache _assetcache 102 Mar 29 05:47 36AE9A4F-A1EA-4A55-9CC6-B7614BA745B3

drwxr-x--- 3 _assetcache _assetcache 102 Mar 29 20:16 3CF3C620-1259-4D15-BE4A-8C663658A690

drwxr-x--- 3 _assetcache _assetcache 102 May 3 20:54 3DC81070-7294-46C6-8897-14158DC0AD5B

drwxr-x--- 3 _assetcache _assetcache 102 Apr 20 19:15 54780044-7C00-4A46-A091-F790AA385508

drwxr-x--- 3 _assetcache _assetcache 102 Apr 28 07:21 56CCB1C4-702A-4E14-8E64-28589A3ACDAC

drwxr-x--- 3 _assetcache _assetcache 102 Apr 21 21:13 5D602438-BC6C-4546-9165-566BEB8103AD

drwxr-x--- 3 _assetcache _assetcache 102 Mar 31 21:09 68858974-D5CA-4552-B795-23858F73B07D

drwxr-x--- 3 _assetcache _assetcache 102 Apr 10 09:00 694FA6D7-01D3-4EAE-BB6C-507249D35013

drwxr-x--- 3 _assetcache _assetcache 102 Apr 8 20:26 6A4B071E-DEE0-4ADB-8414-BF245D0E3A4D

drwxr-x--- 3 _assetcache _assetcache 102 Apr 28 19:13 761A003C-23AE-43FF-BA9A-38DCEE843B2A

drwxr-x--- 3 _assetcache _assetcache 102 Apr 8 15:31 7CB38E5B-024E-4CBB-8EEF-EA884ABF6636

drwxr-x--- 3 _assetcache _assetcache 102 Apr 29 02:35 7D0A2092-78D2-4FC4-BAEF-9C5B16E38AAB

drwxr-x--- 3 _assetcache _assetcache 102 Mar 31 06:12 80F62BC4-7677-4BD7-B4B5-39AE4DE37E6D

drwxr-x--- 3 _assetcache _assetcache 102 Apr 5 21:54 832A2617-0550-4177-A7DD-EE758339327D

drwxr-x--- 3 _assetcache _assetcache 102 Mar 29 20:16 86972E74-D6B0-4083-9A55-AFABBDFDEFF7

drwxr-x--- 3 _assetcache _assetcache 102 Apr 24 04:27 91E6FB8D-0E00-43E7-9BAA-9A12B0CD176B

drwxr-x--- 3 _assetcache _assetcache 102 Apr 29 02:35 94E449AB-24A9-4D5D-A026-6BBE496E0BC1

drwxr-x--- 3 _assetcache _assetcache 102 May 4 00:54 9C874587-8618-4922-A4D5-B9274FA958BE

drwxr-x--- 3 _assetcache _assetcache 102 Apr 20 22:25 A96A70C5-0502-4609-B4E5-1A3F3D8CEDD0

-rw-r----- 1 _assetcache _assetcache 28672 Feb 4 13:49 AssetInfo.db

-rw-r--r-- 1 _assetcache _assetcache 32768 May 3 21:15 AssetInfo.db-shm

-rw-r----- 1 _assetcache _assetcache 2859312 May 4 05:35 AssetInfo.db-wal

drwxr-x--- 3 _assetcache _assetcache 102 Apr 7 09:58 B1F8A775-0F0D-43E0-BD54-F1E479BF0A66

drwxr-x--- 3 _assetcache _assetcache 102 Apr 27 21:50 B4DE9526-2027-4D6B-B545-D8DD13C40550

drwxr-x--- 3 _assetcache _assetcache 102 Apr 15 00:41 C345A505-4F12-4DD2-8105-BE083AE7A4C2

drwxr-x--- 3 _assetcache _assetcache 102 Mar 29 20:17 C3A8423D-59F1-4830-8F77-EB9BDEFE2732

drwxr-x--- 3 _assetcache _assetcache 102 Apr 20 19:15 D510969A-0543-439C-B38A-9ACF360247E3

drwxr-x--- 3 _assetcache _assetcache 102 Apr 27 10:04 DF1B167A-2D63-4116-8763-45B1777B0130

drwxr-x--- 3 _assetcache _assetcache 102 Mar 29 05:47 E1A4D4C7-271B-4FBF-B4FD-010EFE2C91CF

drwxr-x--- 3 _assetcache _assetcache 102 Apr 27 21:50 E4447B0C-25FF-43D7-A140-761A7DE93AA6

drwxr-x--- 3 _assetcache _assetcache 102 Mar 31 21:09 E7032931-189A-4097-A24F-559353EF2E82

drwxr-x--- 3 _assetcache _assetcache 102 Apr 19 19:14 EAA013FF-4670-40BD-98D7-70C500272ED8

drwxr-x--- 3 _assetcache _assetcache 102 Apr 2 19:45 F778D1D7-3037-4A18-BDF3-8B9955DC047B

drwxr-x--- 3 _assetcache _assetcache 102 Apr 10 09:00 F8C07D12-819F-4E41-B474-2EA92C4C3BBE

drwxr-x--- 2 _assetcache _assetcache 68 Mar 25 12:16 tmp

Leo

Leopardus

Level 4

1,303 points

May 5, 2016 12:00 PM in response to woodmeister50

Further to the above, if you look at:

https://www.yesdevnull.net/2013/10/os-x-mavericks-server-setting-up-caching-ser ver/ and

https://www.yesdevnull.net/2013/10/caching-server-2-and-ios-7-confusion/

there are some great tools to help you with your diagnostics and setup.

HTH

Leo

woodmeister50 Author

Level 6

19,723 points

May 6, 2016 4:10 AM in response to Leopardus

Thanks for the info I will check out those links. Also, doing ls in that directory not only yielded

the 0 byte database file, but the only other files present were the "." and ".." files. So, how or

where were the iTunes app files cached??? I will dig a little deeper into that but at least with

Activity monitor, they did come from a disk and went over the network.

Just to be clear, the iTunes app updates were requested from iTunes on Macs and not iDevices

and iDevices were not attached when doing the updates. So at least those are being cached and

served from somewhere.

Leopardus

Level 4

1,303 points

May 6, 2016 4:53 AM in response to Leopardus

WM50,

I discovered that I made a syntax error, so I would kindly ask you to revisit Re: Re: Caching Server service still broke for Mac App Store apps and the faulty line should actually read: (These are separate input commands after you have started sqlite.)

sudo sqlite3 /Library/Server/Caching/Data/AssetInfo.db

.tables

.mode column

.headers on

select * from ZASSET;

to get the output from the database. Could we try this again?

I humbly apologise for my error.

Leo

Leopardus

Level 4

1,303 points

May 6, 2016 5:18 AM in response to woodmeister50

In my post referred to we will use the line for the FirmwareUpdate.pkg

155 2 2 0 130080155 483479425.435412 483479425.435418 3881014aa6179e987c5a06ea95c701c0 E4447B0C-25FF-43D7-A140-761A7DE93AA6 Tue, 26 Apr 2016 18:43:15 GMT /content/downloads/55/34/031-61255/90v8ccbj3vx900ahiiyxu2dij567d29p6m/FirmwareU pdate.pk

The ninth column with the heading ZGUID has a value of E4447B0C-25FF-43D7-A140-761A7DE93AA6 which represents (matches) a location on the HD. The installation package can be found in /Library/Server/Caching/Data/E4447B0C-25FF-43D7-A140-761A7DE93AA6

and the archive is called 0 (Zero) and is a .xar archive. It can be looked at with an extra utility or you can simply add pkg to the name. This will allow the file to be run as an installer. Much more available in the books by Reid Bondonis with some nifty tricks and advice too.

Leo

Caching Server service still broke for Mac App Store apps