afp server issue - very hign cpu load

Actually redirecting the whole Library Folder helps massively, it just causes a lot of other problems.

Since updating to 10.5.5 the server has been more stable. Half of my users now have a non-redirected Library folder but with most of the folders internal to the Library (such as Fonts, Cache etc) redirected. Additionally I have folders within the Library/Preferences/ folder redirected (things like Microsoft, Adobe etc).

So far these users are working well though the server does CPU spike quite a lot when a lot of them log in at once and performance suffers. Though the performance gets better once they've all successfully logged in.

My other users still have a full library folder redirect, these users hardly even tickle the server performance.

I am using the Managed Client.app manifest method already but thanks muchly for the link to that document!

CPU usage in OS X is typically misunderstood, and often a poor metric.
OS X is designed to use the available CPU: that is what it is there for.

You have I/O issues concurrent with seeing CPU usage spike, in all likelihood the CPU usage is not a direct cause of your problems.

One of the better explanations I've seen about CPU usage specifically in relation to OS X should be read here - this from Josh Wisenbaker of afp548.com :
http://lists.psu.edu/cgi-bin/wa?A2=ind0605&L=macenterprise&T=0&O=D&P=75812

"The goal of the OS is to execute tasks as quickly as possible. That means
that it wants to utilize the system at 100% whenever possible... ...The goal of a server isn't to have it as idle as possible, but as fully used as possible."

Watch load averages, take a baseline using iostat, and proceed from there.
It's worth checking the AFP FileService logs, not so much the error log but the file access log
and look for anything noteworthy.

Have you tried 10.5.5 they fixed some AFP issues

I do understand that it's better for the CPU to be utilised rather than not, it's just that the original problem here was that AFP would go nuts under 10.5.4 and would hog the CPU, it would go up to 100% and not come down.

This would be accompanied by a crippling slow down on clients, where the beachball would just spin for ages and made the machines practically unusable.

Under 10.5.4 the only thing that stopped this happening was to redirect the Library folder. The Cache folder has always been redirected in my setup.

I've upgraded to 10.5.5 on both server and clients and the server seems more stable. I've more selectively redirected some of my users Library folders, working on the contents rather than just the whole thing.

Other users still currently have the whole folder redirected.

The only reason I'm looking at the CPU is that even now under 10.5.5, the users without the full redirection of the Library folders cause a lot more CPU activity when logging in en masse (though not during normal use) and this is accompanied by a fair amount of beachballing on clients.

It may be normal though and it may be now an I/O issue, my OD server is presently only using one NIC on a gigabit ethernet network.

care to explain this
"it's better for the CPU to be utilised rather than not" ??????

Yes, I mean I'd rather see the 8 cores that I invested in working hard rather than sitting idle. Just not hitting 100% and staying there for no reason I can fathom.

same here.

xserve with a fresh installed 10.5.5 (OD-Replica, 4 EthernetPorts to one virtual interface (channel bonding). setting the afp threads to 400 worked with 10.5.4, but after a fresh install, using 10.5.1 + combo update 10.5.5, the cpu load is again at 98% (all 4 cpu's). managed users (150) can't work anymore. we've redirected the Library/Preferences folder and setting the afp threads to 400, both did not work.

what is going on with afp services??? it is not possible to work with managed users and this buggy afp service. does anyone have a solution ??

thanks in advice

Philipp

You may be hitting some stated limits for AFP and home folders.

From the Apple 10.5 guide on User Management:
"If you use network home folders, they require one dedicated home folder server for
every 150 concurrent connections. If you use mobile accounts with portable home
directories, you need one dedicated home folder server for every 300 concurrent
connections."

http://www.apple.com/server/macosx/resources/
http://images.apple.com/server/macosx/docs/UserManagementv10.5.mnl.pdf

Now, the following article is for 10.4 and I've not tried any of it with 10.5, but have a look at
http://www.afp548.com/article.php?story=20060329213629494

Have you tried looking at what files are being accessed by the AFP service?

sudo fs_usage AppleFileServer.

Scan the output for the same filename appearing repeatedly over an extended period of time (like a few minutes). If you find one, can you trace which connection it relates to? If you can, does disconnecting that user help the load significantly?

Idle thoughts...

Hi Guys, Specifically Fridgemagnet or whoever else has a mixture of AD/OD with network home folders.

So I have a AD/OD mixture and my users login with AD but their network home folders are on my server. I used the MCX instructions that you guys gave to redirect the Library/cache folder and it works great for my OD users but won't work with AD users. Any thoughts? I have the MCX setup by computer groups and for the life of me it won't redirect AD users (Augmented Records with Home Directories)

Thanks.

we're seeing this with anything over about 50 users... an Intel Quad Xeon just pegs, and everyone gets BBoD (Spinning Beachballs of Death), and the system slows to a crawl.

It's further exacerbated by people thinking they're smart, and shutting off the machine, then relogging in, but that just puts more load on the system.

At this point, I have nothing. I don't see anything AFP is trying to do, I don't see swapping (we have 4 gig of ram, and have ordered another 4 gig), I don't see any abnormal fs_usage info (except maybe Flash Player?!), etc.

We have caches (and microsoft folders) redirected with NHR (the built in MCX one does the same thing). We've set wan_threshholds, and otherwise done as much tweaking as I've found.

any ideas?

Its been noted on this site that when using PHD with AFP that you should use between 30-40 users and then use another server. I cant speak to the truth about that or the actual link but it has been said. Have you called Apple?

Look at your xraid cache flushing setting. "Allow host Cache Flushing" will cause high cpu load as the raid will force AFP to wait tell the flushing operation is completed. Uncheck this option on the raid to get better performance. It is found under the Performance tab of the Settings of your Xraid.

S

Thanks for that, turning off "Allow host Cache Flushing" seems to have had a noticeable impact on my CPU load. Who knew that setting was even there!?

I've been having this high CPU load problem since upgrading our servers to Leopard. I've tried all the tricks with redirecting Caches, removing DS_Store files, etc. but nothing seemed to stick. I disabled "Allow host cache flushing" last night and for the first time, the CPU usage on my file servers isn't maxing out.

I'll be monitoring this for another week or so but if this did the trick, I owe someone a beer 🙂

FYI, this tech article mentions the fix Shawn mentioned: http://support.apple.com/kb/HT1765

"Likewise, if the Xserve RAID is used for file storage for File Services such as AFP, you may wish to deselect the option Allow Host Cache Flushing."