afp server issue - very hign cpu load

OK...I'm not using 10.5.x, but I am seeing a similar issue with AFP under Server 10.4.11. The next time that this happens, can any of you issue the following command in the terminal on the server that is exhibiting the issue:

fs_usage -f filesys AppleFileServer

Let me know if you get a BUNCH of these:

14:52:32 getdirentriesat 0.001715 AppleFileSer
14:52:32 getdirentriesat 0.001088 AppleFileSer
14:52:32 getdirentriesat 0.002078 AppleFileSer
14:52:32 getdirentriesat 0.001953 AppleFileSer
14:52:32 getdirentriesat 0.000982 AppleFileSer

We've noticed that when our AFP server goes out of control, we see several hundred of these come up.

I'm working with Apple on this, and they can't seem to figure anything out. I'm wondering if the issue with Leopard server and Tiger server AFP are the same.

Thank you in advance.

I am also on 10.4.11 and when I do:

fs_usage -f filesys AppleFileServer | grep getdirentriesat

I see a lot of results, maybe not hundreds in a few seconds but the system is running 'normal' right now. What I do see as well is this:

reallocating bitmap for threadid 74249856, fd = 349, setsize = 256

Is that normal? Color me clueless - I've never used fs_usage before.
TIA
George

I think that it is normal to see some of these "getdirentriesat". It's when we see a BUNCH of these (they outnumber any other entries) that we have the problems.

On a side note, don't do the GREP. This will only give you the getdirentriesat items. Listing this entire output of the commands shows how many of these bad entries there are in relation to other entries.

Any other Leo server users that are seeing this?

here is something else to watch for....

if you are using network home accounts, afp, od......

simultaneous logins will gobble-up cpu cycles on your server.

even 6 or 9 simultaneous logins by the same user will suck-up 300-400% of a 4-core 6GB RAM server.

After about 3 weeks of good behavior, I am back to high CPU cycles (high 300-400). I tries the getdirentriesat command and saw a bunch, but they were not overly abundant compared to everything else I was seeing come up. Going to kill AFP again to try to reset it.

Unfortunately, killing AFP did nothing this time. I spiked right back up to where I was before. I will have to try to reboot the server as soon as I get a chance.

Strangely enough, after upgrading our server to 10.5.6, our AFP issues are GONE.

*I must state that we were using Mac OS X Server 10.4.11.*

In anticipation of problems, we did move all users' cache directories to the local hard drive.

Are people still having the problems discussed here or have things been fixed? I have 9 xserves with the same issues outlined above.

Still fubar'd - maybe 10.5.7 will fix it... hopefully.

Meanwhile a launchd job to restart servers at 3am every night is keeping the situation under control for us.

The problem isn't fixed yet. We switched all of our user folders from AFP to NFS. We had no problems since switching to nfs, and users asked us, what we've done? the reported a significant performance boost with nfs.

Philipp

Jason,
As far as I'm concerned (one OS X server 15 clients) the situation is mostly but not completely solved since 10.5.6. That said I had to kill applefileserver two days ago. Under the very soft load conditions that I have afs goes awol about once or twice per month.

Still having the issue here. I just restart AFP when I notice that it is acting up again. At this point, I see it maybe a few times per month, but it is definitely still here. Had to restart AFP a few days ago.

Philipp Reinheimer wrote:
The problem isn't fixed yet. We switched all of our user folders from AFP to NFS. We had no problems since switching to nfs, and users asked us, what we've done? the reported a significant performance boost with nfs.

Philipp

Did you have many problems switching to NFS? Were there any authentication (kerberos?) or resource fork issues?

Would be useful to know any good sources on this.

we are an typical advertising agency, using MS Office, iWork, Adobe CS3 (Photoshsop, Indesign), Cinema 4D etc. and serverbased userfolders.

We had only problems with Linotype Fontexplorer after switching to NFS. we had to setup the font-library in Linotype again, because Fontexplorer looses links to font-files. All other apps and files are working very well. but you need MacOS X 10.5.6 on your clients. there was a bug in older leopard version using serverbased userfolders with nfs. users can't customize their dock. after relogin, the dock was reseted to default OS X dock. this was fixed with 10.5.6 update.

Kerveros authentiation is working with nfs.

as i've written before, NFS is much faster than afp. our server is running at normal conditions at 10-30% CPU (8-core Intel XServe). no performance problems, no abnormal afp service.

hope this helps.

Philipp

Same here. 2 or 4 days with no issue. Then the 30/60/90 plateauing. I generally kill afp when it gets above 50% since it doesn't hurt anything (or hasn't so far). I forgot to check last week before leaving early one day - 100% and problems with all kinds of things (login, preferences lost, etc.) all afternoon and the next morning until I noticed. Behaving nicely today. Sigh. Also waiting for 10.5.7 or a miracle.

Please tell me this was fixed in 10.5.7 as it was better but definitely not fixed in 10.5.6. Would also like an explanation of what was going on under the hood. Thx