Q: Caching Server service still broke for Mac App Store apps
-
May 6, 2016 5:18 AMby Leopardus,In my post referred to we will use the line for the FirmwareUpdate.pkg
155 2 2 0 130080155 483479425.435412 483479425.435418 3881014aa6179e987c5a06ea95c701c0 E4447B0C-25FF-43D7-A140-761A7DE93AA6 Tue, 26 Apr 2016 18:43:15 GMT /content/downloads/55/34/031-61255/90v8ccbj3vx900ahiiyxu2dij567d29p6m/FirmwareU pdate.pk
The ninth column with the heading ZGUID has a value of E4447B0C-25FF-43D7-A140-761A7DE93AA6 which represents (matches) a location on the HD. The installation package can be found in /Library/Server/Caching/Data/E4447B0C-25FF-43D7-A140-761A7DE93AA6
and the archive is called 0 (Zero) and is a .xar archive. It can be looked at with an extra utility or you can simply add pkg to the name. This will allow the file to be run as an installer. Much more available in the books by Reid Bondonis with some nifty tricks and advice too.
Leo
-
May 6, 2016 6:40 AMby mercurialuser,I confirm word by word the problem that woodmeister50 is reporting: osx application updates don't go via Caching Server if you have 10.11.4 installed...
Please note that if you update from 10.10 you are actually downloading and installing the full application. Recently a new "incremental update" feature has been implemented and it may have broken caching...
On my 10.11.4, App Store application reports a 22Mb application update but the download counter stops before reaching that number...
So, if you have devices with 10.11.4, please ask the lan people to sniff the lan traffic to 17.253.0.0/16 network and you will see that some devices are downloading updates directly from Apple. You won't and can't see these downloads in other ways (except if you do them from your workstations and have a look at Debug.log).
It seems that iOs updates are ok, but I couldn't fully test them yet.
-
May 7, 2016 4:01 AMby woodmeister50,I found the problem with the empty database file. Unlike your configuration,
my Caching Service is using an external drive and all that data is located there.
I haven't had time to re-investigate but I am lead to believe that Mercurialuser may have
some substance though, the issue has been around before 10.11.4. When
MacApp Store apps are updated, there is no log activity what so ever on the server
and also monitoring the server during the act of updating an app on another
Mac shows no additional disk or network activity than what one would see when
just idle.
As I said though, I will look at the database in detail but I suspect based on other
data that I will still find MacApp Store apps updates not present.
So, the current state is:
iCloud caching -> working from all devices (at least the log is showing activity).
iDevice direct app updates -> downloads cached or served from cache
iDevice app updates via iTunes on a Mac -> downloads cached or served from cache
MacApp Store OS X component updates -> cached and served from cache for a single update (Camera Raw in this case).
Won't know if it is still the case until another update to an OS X component
comes along. NOTE: This worked after a change to port the config file.
MacApp Store app updates -> Not cached or served from cache or even recognized as being requested. The most recent
of which was Xcode which was sizable enough to monitor.
-
May 18, 2016 3:36 AMby woodmeister50,Well, the recent updates to Server and OS X have not changed a single thing.
Even did a cache reset after the update.
It is looking more and more that it is a client side issue since no log anywhere
on the server shows any hint that a Mac has made an update request from the
MacApp Store.
-
Jun 2, 2016 11:27 AMby woodmeister50,I'm back again.
Had some time and thought I would try one last ditch effort to see what is going on with
MacApp Store updates not working. On two Macs I had 3 updates available. One one Mac,
updated an app and checked the Caching Server log in the GUI as well as debug.log. Nothing
served, nor was anything even requested, i.e. no mention of even an attempt at a request.
Next, I figured I would run Wireshark while doing an update to an app and look for traffic filtered
for my server IP.
I then attempted an update with the next app in line. Well, guess what, the app update request
was received and served via web and cached in the server. There was all manner of traffic in Wireshark
when this took place. Did something fix itself? NOT!!!!!
When requesting the 3rd app update, no request seen by the Caching Service and looking at Wireshark,
the was nary a peep from the Mac to the server.
On the other Mac, this time started with Wireshark running. Of the three apps that were updated, only one
was serviced by the Caching Server, but not the same app that was served on the first Mac but a different one.
Again, when the apps were not served, there was not network traffic from the Mac to the Server IP, but multitudes
of data when the app was served.
So, a request to update an app on one client worked with the Caching Server, the same app didn't on
another for the same app. And, only 1 out of the three updates were serviced on either machine.
It is almost looking as if the MacApp Store app just randomly decides on its own when it wants to make
a request from the server or just simply ignore the fact that the Caching Server is even there!!!!!
And to re-iterate, iCloud Caching, iDevice app request Caching, and iTunes Store on Mac app caching is
working. Why is the MacApp Store hosed????
-
Jun 2, 2016 6:30 PMby chattphotos,Hi there, I've been tracking your updates, that is really strange the Mac app updates aren't working.
Shot in the dark question, is your Caching server running DHCP/DNS for your network? (I can't remember)
If it's not running those, try and see if you can encourage your Mac clients to send all domain lookups through the DNS on the caching server
-
Jun 3, 2016 12:18 AMby Leopardus,Have you checked in the db what has happened recently wrt to the OS X updates, if you had any? Think there were also some iOS 9 stuff to be updated as well?
There is no client configuration to be done at all. Everything happens with server and its unique GUID code which it receives and is registered with Apple's Servers, which 'appears' as a ZGUID code for the specific software, referencing a unique location on your HD. You would thus receive a verbatim copy of the software, exactly as if it comes from Apple.
What is essential, however is that DNS must work correctly for your network. A tiny typo can have a massive negative influence in DNS.
Leo
-
Jun 3, 2016 1:34 AMby mercurialuser,I did a test today, after days in which I have good caching results.
I have 2 Caching servers, mini1 and mini2, in peer. On both I install a lot of "free" apps, almost everything in the "Great Free Apps & Games" panel. So, to be clear, my "user" and my "server" are on the same phisical machine. I don't own another mac to do other tests, at the moment.
Today I had 2 updates on both servers: iTunes 12.4.1 and Simplenote.
I started with iTunes. I was very surprised that both retrieved the software from cache ! From their own cache. Each one had a copy of the software.
I then updated Simplenote on mini1 and I was a lot more surprised ! mini1 asked mini2 if it had the update, and mini2 answered it didn't. mini1 downloaded from apple. When I updated mini2, it asked mini1 if it had the update and of course it had it ! So the software was transferred from mini1 to mini2 and the installed. It's the first time in my tests that a software update on mac osx fully uses the Caching servers - my first tests were in January 2016.
About iTunes update. I wrote that mini1 and mini2 both had the software in their cache. I browsed the logs and saw that this morning one client asked for the update to mini1 (that downloaded from Apple) then another client asked to mini2 which received the update from mini1 and stored it in the local cache. It seems to me that this in a undocumented caching policy: data is be duplicated on peering servers. It's not necessarily bad, it's a policy with pros (imagine a broken server, you may have duplicated ready-to-serve contents) and cons (you have less available storage space).
I'm sure that when I will update other apps I won't get them all from the cache. It's Apple that tells the clients from where to update, and I think it also depends on how the update is packaged, the version of osx you have installed and a lot of other variables.
-
Jun 3, 2016 2:25 AMby mercurialuser,@woodmeister50
I've been having good caching results in the last few days. I don't know exactly why but the stats are a lot better.
I asked the wifi people to extend the lease time of dhcp to minimum 7 days... my dream is to assign every user a "static" ip address... in this way they present themselves to the server with the same IP. I don't know if this helped or if it is just a casualty, but it's what we did.
-
Jun 3, 2016 3:21 AMby mercurialuser,Just to continue on "something is strange about mac osx updates" thread, one mac propose an update to Twitter 4.1.2, published on Jun 2, the other mac has 4.1.1 and no update available.
I update Twitter to 4.1.2 and Slack to 2.0.3 and both DID NOT ask the Caching Server.
-
Jun 3, 2016 3:29 AMby woodmeister50,My DHCP/DNS setup has not changed for the last several years and is managed
by my Airport Extreme Base Station (basically since I first purchased it).
If you follow through the thread, you will see that all worked just fine until somwhere
in OSX 10.11.x and Server 5.x update chain and was flawless with 10.10.x and Server
5.x or 4.x.
And, it is only the MacApp Store that is a problem!!!
iOS app updates, whether via iDevice or Mac/iTunes work flawlessly. iCloud caching is
responding for all clients. All of which are verified in the logs yesterday by doing iOS updates
via iDevice or iTunes on Mac. iCloud caching is responding as I made changes in iCloud Drive
or apps that utilize it and see the events in the log.
As of yesterday, as stated, the MacApp Store app caching is intermittent at best and when not
cached, the client issued no request to the server, verified by no packets sent from client to server at all.
A bit more detail in the scenarios yesterday, in one case it was the "middle" update that worked and in the other
scenario it was the first update that worked.
-
Jun 3, 2016 10:55 PMby Leopardus,mercurialuser wrote:
@woodmeister50
I've been having good caching results in the last few days. I don't know exactly why but the stats are a lot better.
... my dream is to assign every user a "static" ip address... in this way they present themselves to the server with the same IP. I don't know if this helped or if it is just a casualty, but it's what we did.
You should be able to 'reserve' an IP for the MAC address and/or the 'Bonjour' name, which would in essence give you a 'static' address per device.
HTH
Leo
-
Jun 5, 2016 3:43 AMby woodmeister50,mercurialuser wrote:
Just to continue on "something is strange about mac osx updates" thread, one mac propose an update to Twitter 4.1.2, published on Jun 2, the other mac has 4.1.1 and no update available.
I update Twitter to 4.1.2 and Slack to 2.0.3 and both DID NOT ask the Caching Server.
Welcome to the club! I have come to the conclusion that somewhere in the upgrade paths
from OS X 10.10 or 10.11.0 and Server 5.0 to current there have been changes that have
created random breaking of the MacApp Store Caching service. This service worked
perfectly previous to the path I have mentioned. The randomness of the working and not working,
especially with my recent tests says the whole concept has become borked!!!!
