Firewall Issues - ssh_dispatch_run_fatal errors during SSH

For those of you bypassing by disabling the firewall -- that's not a practical solution for enterprise users who either should not or cannot disable firewall. macOS developers, I hope you're tuned in to these discussions -- it's a huge blocker for those of us who rely on Mac as a developer's tool.

Guys, this is a very old and erratic bug in the Network Extension (I believe). Here are some references:

• Since Sonoma update, multiple times I get… - Apple Community
• https://forums.developer.apple.com/forums/thread/729348

The problem with this issue is that it comes and goes. It becomes more severe if you have a docking station with ethernet, VPN or a network filter (little snitch or lulu). Apple developers have been ignoring this issue deliberately for years. YEARS. The internet is FLOODED with people complaining about this (networking timeouts), but because this issue is not easily reproducible, and even worse, because this issue can at times only affect the speed of the connection (which can be blamed on other factors), no one hammered Apple to take it seriously. I know many people who abandoned Mac because of this.

If you want to know whether this issue is happening, while the internet/network is slow, just run a ping on your router. You'll get something like this:

$ ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 164 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=2306.502 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1300.413 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=295.091 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=1.242 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=1830.929 ms
64 bytes from 192.168.1.1: icmp_seq=5 ttl=64 time=2687.323 ms
64 bytes from 192.168.1.1: icmp_seq=6 ttl=64 time=1685.573 ms
64 bytes from 192.168.1.1: icmp_seq=7 ttl=64 time=680.508 ms
64 bytes from 192.168.1.1: icmp_seq=8 ttl=64 time=0.857 ms
64 bytes from 192.168.1.1: icmp_seq=9 ttl=64 time=0.929 ms
64 bytes from 192.168.1.1: icmp_seq=10 ttl=64 time=197.700 ms
64 bytes from 192.168.1.1: icmp_seq=11 ttl=64 time=2764.140 ms
64 bytes from 192.168.1.1: icmp_seq=12 ttl=64 time=1762.121 ms
64 bytes from 192.168.1.1: icmp_seq=13 ttl=64 time=756.605 ms
64 bytes from 192.168.1.1: icmp_seq=14 ttl=64 time=0.983 ms
64 bytes from 192.168.1.1: icmp_seq=15 ttl=64 time=2152.170 ms
64 bytes from 192.168.1.1: icmp_seq=16 ttl=64 time=1150.504 ms
64 bytes from 192.168.1.1: icmp_seq=17 ttl=64 time=1859.144 ms
64 bytes from 192.168.1.1: icmp_seq=18 ttl=64 time=854.277 ms
64 bytes from 192.168.1.1: icmp_seq=19 ttl=64 time=0.943 ms
64 bytes from 192.168.1.1: icmp_seq=20 ttl=64 time=0.790 ms
64 bytes from 192.168.1.1: icmp_seq=21 ttl=64 time=0.545 ms
64 bytes from 192.168.1.1: icmp_seq=22 ttl=64 time=1.522 ms
Request timeout for icmp_seq 2564 bytes from 192.168.1.1: icmp_seq=23 ttl=64 time=3073.333 ms
64 bytes from 192.168.1.1: icmp_seq=24 ttl=64 time=3373.110 ms

Meaning, you'll get timeouts and unstable networking randomly. Notice how the time value goes up and down like crazy. This is why your internet is slow!

A restart might solve the problem temporarily. Another reason why many people just shrug it off.

I would've built the networking kernel module and debugged it myself, but this seems to be an issue with the Network Extension (since it's related to the OS firewall, filters, etc)... which is not open source! If it were open source, I swear I would happily build it in debug mode and fix it myself... this is how bad this is!

I think the only question that matters here is: How can we push Apple to do something about this? I'm happy to run a debug session with a developer to fix this... but they don't seem to want to take this seriously! Wtf do we do?!

[Edited by Moderator]

15.0.1 caused the issue to skyrocket in severity. Since 15.0 SSH connections aborted but Remote Desktop was fine (connecting to an RD farm, not just open RDP .. dont freak out).

Installed 15.0.1 today and now my RDP connections drop every 30 seconds or so. After disabling the firewall (wow, really?) the disconnects are gone entirely. I also had trouble with page loads in the browser.

These have to get fixed, I can't consider macOS a viable platform anymore if these basic issues are not resolved.

I have contacted apple support seems like there is no fix at the moment which really sucks and is a complete and total blocker for any of use that need remote access to remote machines. This completely blocked me and made my mac a giant paper weight. I hope they fix this bug soon. Cant believe such a basic thing was not tested...

So I am on an M3 .. and I am seeing

• SSH Connections die just generally
• SSH PortForwarding connections dying VERY consistently
• Webex having major problems with maintaining connectivity
• AWS Console sessions dying

I also correlated that it's happening across connections at the same time; not each connection.

Hi everyone, I'm having the same problem too.

I'm using LAN connection (Home) and WiFI (Office) and the Issue having with both.

Today the SSH connection was constantly interrupted and after a few seconds or even minutes and I couldn't work.

I disabled the "Limit IP Address tracking" option and so far (barring surprises, and I'll keep you updated it seems to have disappeared) I haven't disabled firewalls and other programs.

I've been ssh'd for about an hour now and am having no problems. Let's hope so.

Seeing the same behavior here on Sequoia with Microsoft Defender and Palo Alto Cortex XDR installed. Cannot turn off either of them, as it's a company-managed device where I literally don't have the needed privileges to disable or reconfigure the software.

Oddly, a coworker with the same set-up isn't able to reproduce the issue. Only obvious difference is he's on m3, I'm on m2.

To reproduce the issue, I'm ssh'ing into several servers at once and running this loop on all of them -- they all stop at the same time when the issue occurs, and it only takes a few minutes:

while [ true ]; do echo "$(hostname) $(date)"; sleep 1; done

Even more strange, I opened an AWS ec2 instance using SSM connection manager, which is a browser-based terminal interface that communicates with an ec2 instance via an open https connection to AWS, so there's no ssh between my workstation and the ec2 instance, and the connection manager session also loses connectivity between my workstation and AWS at the same time that the other connections drop. I can just refresh the web page containing the session, and it's back again, but this indicates that it's not just ssh connections being impacted -- it's potentially any open tcp connection.

Thank you for making this post! I spent 12 hours today troubleshooting my network trying to isolate this issue. I cannot confirm that this is the solution just yet, although the symptoms and timing is spot on. But I'm working with IT at work to disable firewall for testing.

For anybody else searching for a solution on this, here is a full writeup on the issues I am seeing https://community.ui.com/questions/Entire-network-halts-crashes-every-minute/0c0625b0-5658-4cb2-b95c-ddb17c6920a2 . The weird thing is that I am not seeing these issues when I am connected directly to my router, only when I add a switch between myself and whatever I am trying to communicate with.

Tagging this post with: Sequoia, ERR_SSL_PROTOCOL_ERROR, Bad packet length, ssh_dispatch_run_fatal

I came on to post a different question, but thought this post was potentially relevant.

I factory reset to Sonoma and upgraded to Sequoia yesterday. I before started adding anything, I was updating the settings, layouts, folders etc to make sure things are how I like it to be.

For some reason a few folders and items have been added to my shared folder - which I've never seen before. I don't think I've had anything in my shared folder before. (it might've always been there and that I wasn't observant/forgot)

So the new folder/items that have been put there are:

'/Users/Shared/Relocated Items/Configuration/private/etc/ssh.system_default'

Inside the ssh.system_default folder there is:

• crypto.conf symlink
• crypto/
  • apple.conf
  • fips.conf
• ssh_config.d/
  • 100-macos.conf
• sshd_config.d
  • 100-macos.conf
• moduli
• ssh_config
• sshd_config

Again, this could be absolutely fine/normal, but I was coming on here to ask if it was and saw this thread about ssh issues, so decided to message here first to see if maybe this is what's causing an issue for you?

I spent 2 days debugging my Debian server until I realised this might be a problem on the client side. then I looked over the logs and

2024-09-22T07:53:26.484700+03:00 TheBrain sshd[285596]: Received disconnect from 192.168.0.180 port 56574:11: disconnected by user
2024-09-22T07:53:26.484948+03:00 TheBrain sshd[285596]: Disconnected from user user 192.168.0.180 port 56574
2024-09-22T07:53:26.485790+03:00 TheBrain sshd[285590]: pam_unix(sshd:session): session closed for user user
2024-09-22T07:53:26.491402+03:00 TheBrain systemd-logind[468]: Session 309 logged out. Waiting for processes to exit.
2024-09-22T07:53:26.493102+03:00 TheBrain systemd-logind[468]: Removed session 309.
2024-09-22T07:53:59.923525+03:00 TheBrain sshd[285605]: Connection closed by 192.168.0.180 port 56575
2024-09-22T07:53:59.923785+03:00 TheBrain sshd[285605]: Close session: user user from 192.168.0.180 port 56575 id 0 

so it's on the client side. And I solved this by disabling LuLu.

The default firewall configuration allows any built-in app, and any 3rd party signed app, to accept incoming connections. It doesn't block anything. In some versions, if you manually change settings to block software, it will just change it back to allow. But assuming you have a version that works, then it is only going to block local connections, like from your printer or other computers in your house.

No argument that the default config allows signed apps (built-in or otherwise) to accept incoming connections. This configuration allows general users the ability to install and use tools w/o having to know much about the network; and IMHO even if there was an alert .. that same level of user would click right through it same as their user agreement :)

Could Apple do better, sure no argument there either. However for users that want more security can use third-party tools that do better ( snitch,etc ) .. tho I would agree that it would be nice if the inherent tool was more applicable.

For the vast majority of users, no outside connection is going to make it past their modem in the first place.

This comment assumes only a home or behind-a-physical firewall use-case; I am considering this and all the places one takes one's lappy: coffee, airplane, school, conference, etc. To your point those same services are open if installed and my be an exploitable vector; however.. it's a layer that may need to be worked around for those being malicious .. and again doesn't hurt (except in this case where it's borked).

This is one area where I think Microsoft had a better approach by having profiles based on network location that apply different levels of security profile; and other tools could benefit from the approach.

This is a user-to-user support forum for consumer users. Companies typically have all kinds of specialized configurations and software that we don't have and can't support.

Per Community etiquette guidelines - Apple Support This is a general support site; not limited by use-case.

SamWantsYouToChill wrote:

Here are some references:

• Since Sonoma update, multiple times I get… - Apple Community

• https://forums.developer.apple.com/forums/thread/729348

Well, that first link is you. The second link is good because it has lots of great and easy solutions.

It becomes more severe if you have a docking station with ethernet, VPN or a network filter (little snitch or lulu).

Those are all radically different. If you are having a problem with a docking station, it would be best to start your own question about that problem.

There is no such thing as "VPN". There are dozens of 3rd party products that advertise VPN services. They range from legitimate networking tools, to software piracy and file sharing tools, to scams, and even straight-up malware. If you are having a problem with a particular product, state the product so that other people can confirm, deny, or suggest alternatives.

Beyond that, you certainly aren't wrong saying that various 3rd party "privacy" and "security" apps have been a problem for many years. This tech support forum would be a ghost town without such apps.

I know many people who abandoned Mac because of this.

Apple has plenty of users, more than they need or want, probably.

If you want to know whether this issue is happening, while the internet/network is slow, just run a ping on your router. You'll get something like this:

$ ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1): 56 data bytes

Request timeout for icmp_seq 0

Request timeout for icmp_seq 164 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=2306.502 ms

Looks fine here:

$ ping 192.168.2.1

PING 192.168.2.1 (192.168.2.1): 56 data bytes

64 bytes from 192.168.2.1: icmp_seq=0 ttl=64 time=4.008 ms

64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=7.438 ms

64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=4.012 ms

64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=8.262 ms

64 bytes from 192.168.2.1: icmp_seq=4 ttl=64 time=4.858 ms

64 bytes from 192.168.2.1: icmp_seq=5 ttl=64 time=6.671 ms

I know you qualified that with "while the internet/network is slow" but what can I do? It's never slow.

I would've built the networking kernel module and debugged it myself, but this seems to be an issue with the Network Extension (since it's related to the OS firewall, filters, etc)... which is not open source! If it were open source, I swear I would happily build it in debug mode and fix it myself... this is how bad this is!

What's "the Network Extension"?

Wtf do we do?!

Don't run any of those firewalls or filters, either from Apple or 3rd party developers. You don't need them.

This is happening pretty consistently when using streaming tools across forwarded ports on SSH. I hit it when using MS Remote Desktop.