Posts:
2
From:
Springfield, MO
Registered:
Nov 7, 2007
|
|
|
|
SSHD Processes Using 100% CPU
Posted:
Dec 5, 2007 8:26 AM
|
|
|
Two Processes, both SSHD make up over 95% of my cpu utilization making my Leopard server run at 100% cpu utilization all the time. I kill them and they come back in 1/2 hour or so. Any ideas?
Xserver g5 x2
Mac OS X (10.5)
|
|
Posts:
73
Registered:
Apr 3, 2005
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 5, 2007 2:45 PM
in response to: Worsham
|
|
|
No solution (sorry), but you are not alone. Can be observed here as well sometimes.
Mac OS X (10.5.1)
|
|
Posts:
22
Registered:
Jun 6, 2006
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 5, 2007 8:24 PM
in response to: Worsham
|
|
|
Just wanted to ditto this issue. Very peculiar. If I disable remote login, of course the sshd processes (a bunch of them) go away, but as soon as I re-enable it they all come back, chewing up all my Mini's CPU.
Mac Mini 1.66 GHz
Mac OS X (10.5.1)
|
|
Posts:
932
Registered:
May 7, 2005
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 8, 2007 4:19 PM
in response to: Worsham
|
|
|
Just a thought, if these are Internet facing machines, are you being attacked?
Roger
XServer
Mac OS X (10.4.7)
XRAID, Xasperated
|
|
Posts:
1
From:
Canada
Registered:
Dec 10, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 10, 2007 12:37 PM
in response to: Roger Smith7
|
|
|
I'd like to note this as well.
I noticed my computer was quite sluggish over the past couple of days so though I would actually look into it. I noticed SSHD was at 99% cpu usage as well.
I did a tcpdump on port 22 and nothing was coming through.
I did a general tcpdump and there wasn't a lot of traffic. Just general arps and switch traffic.
Any more thoughts on this?
Pete
PowerMac G5
Mac OS X (10.5.1)
|
|
Posts:
1
From:
Mountain View, CA
Registered:
Dec 14, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 14, 2007 9:19 AM
in response to: Worsham
|
|
|
My Leopard system has this problem. My observation is that it happens if I try to ssh into the machine and enter the wrong password - my ssh hangs, and the sshd on the server side sits and uses 100% CPU until I kill it. It's very annoying, and easily reproducible. Can anyone on this thread verify if they see the same behavior?
I don't see it happening if I try to log in as a random account that doesn't exist, but I do see the problem if I try to log in as root, even though root login is disabled by default. If your ssh port is exposed to the internet, this could explain cpu-eating sshd processes appearing randomly - some would-be attacker may be trying to log in to your machine as root.
24" iMac
Mac OS X (10.5.1)
|
|
Posts:
21
Registered:
Feb 25, 2003
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Dec 21, 2007 10:47 AM
in response to: Worsham
|
|
|
I am running OS X Server 10.5.1 on a new X Serve and have this problem. Since this is a server on a public network, it is under constant SSH attack. I can't really kill SSH on a production server since that is how we admin it. It is causing the machine to run slow and hot due to the processor utilization. Everytime I kill that process, it obviously comes back quickly since there are tons of bots out there trying to SSH into it all the time with the wrong password.
Apple: Please fix this problem!
MacBook Pro
Mac OS X (10.5.1)
|
|
Posts:
7
From:
New York, NY
Registered:
May 28, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 3, 2008 10:03 AM
in response to: Etep15
|
|
|
Chiming in - I have the exact same issue on only ONE of my 4 XServe (3 are 10.5.1, 1 is 10.4.10). They are all internet facing and have contiguous IP addresses (ie, when there are BF attacks, they are all attacked at once). My machine is not being brute forced - I can verify this with a peek in the secure.log and netstat. However, whenever I try to SSH in MYSELF, after a few days of uptime on the server, the SSHD process skyrockets to 100% and I am unable to SSH in until I reboot the server (ARD still works though). If I ARD in and kill the process, then the machine drops back down towards idle and no new SSHD is spawned until I try again myself. Cycle repeats.
Anyone else seeing something similar? I find it incredibly strange that I'm only seeing this on one of my 10.5.1 XServe (though it is a different hardware configuration - 3ghz instead of 2.66 and it has a SAS drive, I sincerely hope that this is not due to the hardware difference)
XServe (Intel; 3 x 2.66ghz quad, 1 x 3ghz quad) ; Mac Pro (50 x 3ghz octo)
Mac OS X (10.5.1)
|
|
Posts:
1,720
Registered:
Oct 10, 2000
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 3, 2008 7:42 PM
in response to: Jon Bell
|
|
|
Take proper steps to secure your ssh.
I manage a number of OS X Servers (10.4.x) - either public-facing or still reachable via ssh, and have not had this problem.
Set ssh to use a non-standard port (this is NOT extra security, it cuts away the bulk of the dumb/bot scans/attacks), secure ssh at the firewall (allow only the IP ranges you truly need to let in - takes some work but specific ISPs use specific blocks of IPs), and only allow connections via ssh shared keys.
For changing the port, see
http://www.macosxhints.com/article.php?story=20050707140439980
Choose carefully for the alt. port of course.
For ssh keypair auth, see
http://www.afp548.com/article.php?story=20040816224717742
|
|
Posts:
7
From:
New York, NY
Registered:
May 28, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 3, 2008 8:00 PM
in response to: davidh
|
|
|
Thanks for the tips David -
I've already been running a brute force detection script that I put together (scans the secure.log file for > 10 failed logins in 10 minutes, then adds to the ipfw block list & distributes the change to all of my machines). Due to the nature of my users it would be highly inconvenient to run a whitelist only operation.
My problem is that I did not have this problem running the same machine under 10.4 server - it only arose after the upgrade (a clean install). My bruteforce detection operation is running fine too - it's nabbing attacks as they come in, but I get stalled SSHD even when there are no attacks coming in. On a somewhat related note, I also run half of a dozen CentOS4 (RHEL clone) servers with fractions of the horsepower as the XServes, with the same (modified) BFD protection measures for 3-4 years now and haven't ever seen anything like this.
Until the recent issues with the 4-core 3ghz/4gb ram XServe, the worst I've ever seen openssh balk under a brute force is a massive slowdown and huge latency on a 600mhz celeron/256MB ram, while experiencing multiple attacks at once. I've seen no attacks like this on my XServe, yet sshd is hanging completely. So what gives?
Is anyone else still seeing SSH issues even after locking it down?
XServe (Intel; 3 x 2.66ghz quad, 1 x 3ghz quad) ; Mac Pro (50 x 3ghz octo)
Mac OS X (10.5.1)
|
|
Posts:
1
Registered:
Jan 4, 2008
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 4, 2008 3:06 PM
in response to: Worsham
|
|
|
I see similar CPU spinning occasionally trying to launch Terminal.app. I also see the problem on occasion with sshd.
In both cases I think the following kind of message in system.log appears:
Jan 4 17:51:28 calamari kernel[0]: devfs: ttys000: name slot allocation failed (Errno=17)
Since both Terminal.app and sshd are in the business of dealing with virtual terminals, it seems plausible that what is happening in both cases is a tight loop wrapping a system call which is not expected to fail persistently (but which is in this case).
Or something. 
|
|
Posts:
21
Registered:
Feb 25, 2003
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 4, 2008 3:20 PM
in response to: Jon Bell
|
|
|
I have locked down SSH (by blocking the port in the firewall), rebooted the server and then launched terminal.app. It hangs and I have to force quit it.
I then open a hole in the firewall to SSH via only 1 secure IP. Reboot server. Try to SSH in. Hangs and see 100% sshd utilization in activity monitor. Simply quitting or force quitting sshd (or terminal.app) does not fix the problem or allow a log-in.
To answer the possible hardware specific problem, this is a dual 2 Ghz machine, not the 3 Ghz machine listed by another poster.
I seem to remember the problem with 10.5 as well but I am also certain I could SSH in after force quitting sshd with version 10.5 and we can't do that with 10.5.1. Perhaps it was not 10.5.1 but a security update that finally broke it for good since we installed them both at the same time.
MacBook Pro
Mac OS X (10.5.1)
|
|
Posts:
7
From:
New York, NY
Registered:
May 28, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 5, 2008 8:29 PM
in response to: davidh
|
|
|
FWITW, I also have recently put together a port of sshdfilter into an easy OS X install package. The script scans incoming ssh connections and detects brute force attacks, then blocks them. It's helpful if you are in a position where due to your users, you must allow for arbitrary IP's to access on a standard port. The port/installer is still in beta testing, but seems pretty stable and would appreciate any comments if anyone is looking for such a script: http://projects.seas.columbia.edu/sshdfilter/.
XServe (Intel; 3 x 2.66ghz quad, 1 x 3ghz quad) ; Mac Pro (50 x 3ghz octo)
Mac OS X (10.5.1)
|
|
Posts:
2
From:
Canada
Registered:
Jul 29, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 6, 2008 7:07 PM
in response to: Worsham
|
|
|
No good solutions from me, but I am another sufferer on a mac mini. First it was with sshd, then Terminal.app went south.
It pains me to say it, but the "windows treatment" aka reboot cleared it up, for now. Not a useful solution for those running servers though.
Hope apple fixes this one soon.
mac mini 2.2 Ghz
Mac OS X (10.5.1)
wireless only
|
|
Posts:
7
From:
New York, NY
Registered:
May 28, 2007
|
|
|
|
Re: SSHD Processes Using 100% CPU
Posted:
Jan 8, 2008 8:18 AM
in response to: Worsham
|
|
|
New symptom:
Left an ssh session open last night when I went to sleep. Woke up, used it just fine. With it open, attempted to open another session to the same machine: sshd goes up to 99%, but the original session continues to function fine. There are 0 lines in my secure.log in betweeen when I opened the session last night and when I tried logging in this morning.
Jan 8 11:07:45 projects com.apple.SecurityServer35: checkpw() succeeded, creating credential for user jsb2125
Jan 8 11:07:45 projects com.apple.SecurityServer35: checkpw() succeeded, creating shared credential for user jsb2125
But that's it - it never finishes opening my session.
XServe (Intel; 3 x 2.66ghz quad, 1 x 3ghz quad) ; Mac Pro (50 x 3ghz octo)
Mac OS X (10.5.1)
|
|
|
