afp server issue - very hign cpu load

So far, killing the AFS process appears to be the simplest and quickest workround. The AFS will automatically restart without the client notice.

The whole issue is cache related. That is why a reboot can bring a lower CPU back, and disabling the RAID caching may do the trick if RAID is used, and moving out the users' library files may be helpful. The security patch may not fix the problem, but the reboot after the applying the patch does. Client machine crashes may also cause the trouble. With that said, removing crashed .ds_store file or effects of stopping the .ds_store files may also be helpful. When the CPU issue occurs, network traffic may be high or may not, and it may not be directly related to the issue at all. When maxing out, the AFS does not take much memory, and so it does not matter whether the server has 2 GB or 14GB of it. Hard drive space may be an issue only if the server does not have enough space to hold files and caches such as in some cases where usable HD space was lower than 8%. And apparently, even if the AFS design has been trying to follow Apple's philosophy of not keeping your investment idle, the AFP CPU problem is nothing more than a side-effect.

Well, has the above paragraph summarized the whole thing?

Message was edited by: marbles1

I can have very little traffic after my users login and the CPU still maxes out long after they have logged out and left. I do not see the security patch as having any real effect on this situation. I have lots of free disk space and about 4GB of RAM per server and killing the AFP process is the only workaround that brings things into line.

I manage a network of about 20 users. We are running 10.5.6 Server, with 10.5.6 Clients.

The server was installed over a weekend and we ran fine until Friday, when everything fell apart. AFP peaking maxing out the CPU. We're getting the same issues as everyone else here.

While I don't doubt there are AFP issues Apple needs to resolve... I reviewed out system logs and found MDS errors around the same time we experienced slowdowns.

I have turned spotlight indexing OFF on our RAID, however the MDS errors seem relate to share points on our RAID. Is it possible that MDS is trying to index these sharepoints even though the RAID indexing has been disabled?

I wish I could just remove Spotlight altogether from our Server!!

Well its been 2 weeks since I did the security update and posted the image with our server load problems.

We haven't experienced any crashes related to the cpu load problem. Our server is now typically running at between 10-20% with small spikes here and there when I do things like VNC into the server to check it remotely or when our back up tape drive is running.

Other than that smooth sailing! Knock on wood(crest).

Another thing which was causing huge spikes before was transferring data from a CD/DVD from a client dvd drive directly onto the server. Seemed to cause much larger spikes than if the data was first transferred onto the client computer and then onto the server in two steps. At the suggestion of comments in this thread we've also been making sure to close all files when not being used and trying to shut down programs at the end of the day specifically microsoft office.

Again, we only have 10 clients so I believe it may be easier to handle client bad habits which may easily be aggravating the cpu load problem. A lot of you guys have hundreds of clients and its probably next to impossible to make sure they are all using whatever "best practices" you come up with.

So now that it looks like we've become stable again, I'm not even going to think about updating anything on that server until Apple has 100% addressed and solved this issue with some kind of certainty.

The security patch worked for some, but not me. I had great success after (logged in as root) moving System/Library/KerberosPlugIns/KerberosAuthData/odpac.bundle to somewhere else (root/Desktop in my case- just get it out of that folder).

Also, in the root of each share: touch .metadata neverindex

I am VERY happy now! No CPU spikes. AFP is under control.

The security patch doesn't solves the issue for me. moving the odpac.bundle helps ne since 4 days too.

Intel Xserve (no RAID) was clean upgraded from 10.4.11 to 10.5.6 many weeks ago. Last client upgrades from 10.4.11 to 10.5.5 were completed on 3/5. The AFS service problem started shortly after 8am on 3/11 and continued periodicaly throughout the day. The server was very slow to respond to Finder actions even when not pegged.

After the workday on 3/11, the Finder was still very slow to respond. No client machines were running, but there were still several afp connections showing in Server Monitor. Activity Monitor showed the AFS process was not among the top ten when sorted by CPU.

I rebooted the server, then installed the SecurityUpdateServer 2009-001-1.0. The Finder was responding normally. No other changes were made.

The system then ran OK, with occasional Finder sluggishness on the server, but not as severe, until yesterday evening (4pm on 3/19, or about 8 days), when the spiking started again.

So, the Security Update does not solve the afs spiking problem.

This is a severe problem. Management is losing data and losing patience.

Apple: Help!

Steve

Developer releases of Leopard 10.5.7 apparently have a number of AFP fixes. Perhaps the real solution to this problem is on it's way.

Hoping, but not holding my breath....

I highly doubt OS X S 10.5.7 is going to fix this problem or AFP overall. The AFP service has always been an issue in OS X S for me anyway.

I'm really no file systems guru.. But is there some movement underway at Apple that we're not aware of? Should we all be migrating to something other than AFP? Should we be moving to NFS? Does NFS perform better than AFP?

Just thought I should ask..

thanks!
ev

As with so many of you, I also have a 10.5.6 server 2 Dual Core Xeon at 2Ghz with 4GB RAM. Clients are mostly 10.4.11 and 10.5.6 managed clients with server home directories. I have seen my server spike as well at given points during the day. That does not bother me too much as it does not always seem to cause a problem and goes away after a few minutes. It may cause some beach balling, though. WHat bothers me the most is that my AFP process also shoots up over 110 and just stays there. I will still see the typical spikes that I might see, they are just on top of this "plateau" and cause even more issues than they would have normally. I cannot drop this "plateau" down to normal levels without reboot of server. What I typically notice is that the CPU shows a baseline of about 25% (in Server Admin) even with nobody logged in in the middle of the night. It NEVER drops below this baseline. AFP will sho it is running over 110% in activity Monitor and often times I see a single core pinned for a while, then it switches to another core and that is pinned. I had similar issues in the fall and it was directly related to Spotlight (which I disabled to fix it.) Now, it is back and I don't know if it is still Spotlight possibly fighting with an AFP share that it can't resolve or something because it still operates at over 110% even when nobody is connected (though AFP still shows some users connected for some reason). I installed that patch and it did not work. I opened a case with Apple and they thought the patch fixed it. I sent them a data collection and it was going to be escalated (probably to some of the same people some of you have dealt with.) Interestingly, when I look for open Spotlight files on the server, I still find a bunch. Can anyone else confirm they still shop open Spotlight files after disabling Spotlight on all AFP shares? Maybe it is not completely off and still causing the issue?

hi ev,

we did not know, what else we could do. The AFP problem was so bad, that our company are experiences damages from down time of our userfolder-server.

so we decided to move to NFS. Since 10.5.6 Update, NFS is working fine with serverbased/managed users. We have no problems so far, except issues with font collections in Linotype Fontexplorer. Moving to NFS looses the Resource Fork from font files. we deleted the Font Explorer Databases, font files and reinstalled the fonts. All other files, user settings are working fine.

The performance is dramatically boosted and we have no problems so far (except the font issues). CPU is at 5&-40% and 150 serverbased users.

i would recommend, testing around with NFS and when it's working fine, move your userfolders to NFS, deactivate the buggy AFP protocol.

Philipp

servers 10.5.6, clients 10.4.11 and 10.5.6

we've also had this problem. AFS will bounce to 100 or 200 or 300 or 400% and stay that way, even when all users have logged out.

what has, so far—the last 2 to 3 weeks, eliminated this is redirecting the fonts and caches of our network-based users to the local hard drives at login.

so far so good.

Sorry, I forgot to mention that it doesn't work with resource forks. You might try smb, it should support resource forks and might not be as buggy as afp. Just a thought. I have the people who resource forks would matter to (my instructors, TAs, and staff) on afp (about 10 max at a time) and my student homes on NFS.