NFS broken in Sequoia 15.6

elisatems wrote:

Trying SMB is a nonstarter I'm afraid - mount -t smbfs on the Linux box returns "unknown filesystem type", despite the mount(8) man page listing -t smbfs as supported. There is no mount_smbfs command either.

It's Linux. You'll have to install it first.

On my relatively recent Ubuntu system, I had to do "apt install gifs-utils smbclient". Then, to connect, it's something like this:

sudo mount -t cifs //<IP address>/<share> /mountpoint -o username=<you>

Edit: it went stale in <5 minutes. So there is apparently no solution. Sigh.

I'm afraid the solution is most definitely SMB.

I tried again with my NFS mounts. This time, seemingly doing the same thing that I did yesterday, I did get an external volume to share over NFS. Two problems:

1. APFS fake paths are funny. /tmp/share didn't work, but /private/tmp/share did. Yet both /Volumes/Archives and /System/Volumes/Data/Volumes/Archives both worked.
2. I needed to manually give nfsd full disk access in System Settings.

"nfsd checkexports" was helpful in getting the external drive to share.

But once I let it set for 5 minutes, both mount points were dead, just like you experienced. As it turns out, this has been a problem for a few years now:

NFS mount disconnecting after a few minut… - Apple Community

NFS exports seem to die after five minutes - Apple Community

That's what I mean about my confusion at your statement, "NFS broken in Sequoia 15.6". If someone says NFS has been broken since Big Sur, that makes a lot of sense. But Sequoia 15.6? That's very odd.

After restarting the Linux machine, I tried again with SMB. It's been running fine for at least half an hour now.

I did have to manually add the external to the list of Shared Folders in System Settings.

PS: Perhaps I could have added "-f" to my umount commands on Linux to avoid the restart. I don't do this very often.

elisatems wrote:

You're making a lot of assumptions about what I'm running there.

I'm doing exactly the opposite - expressing my ignorance of how your network is configured.

Why don't you ask questions instead of assuming what I'm doing or not doing?

As I said, I did, but then I deleted them as I was afraid it would be overwhelming. Usually, if I ask someone 3 questions, they'll pick one at random to answer. That's always awkward. Can you imagine how it is when I ask 20?

First, just to clarify: my NFS server settings are the defaults on MacOS, so it's possible that something did change there as I've never made any customizations on the NFS config files, other than my /etc/exports.

As I said, I don't think it's an NFS problem. You are able to connect initially but then the connection goes stale. That strongly indicates that some portion of the link has changed and the other side didn't know about that. There are multiple places in Sequoia's low-level networking settings that will do precisely that.

It's possible that Apple's SMB implementation will handle this situation better. You can test that in about 2 minutes.

It's also possible that some low-level networking default (not NFS) has changed and you can simply change it back. You can test that in about 2 days.

Also: there is no 3rd party security or networking software running on my network, other than on the Arris modem itself. I use the modem's firewall with all external ports blocked. This is all determined by my ISP - I do not have a lot of choice as to networking hardware.

You see? This is what I mean about the twenty questions. You were very specific about 3rd party security or networking software. OK. That's good. But what about built-in software? One of the things that changed in Sequoia was the built-in, useless application firewall. Before Sequoia, it never did anything anyway, and was riddled with bugs, but at least it was fairly benign. But there are new bugs in Sequoia that push the application firewall from merely useless, to a harmful detriment.

I can go on for pages and pages with diagnostic suggestions. But there's nothing I can say that would be more useful than you poking around with the actual devices for a few minutes. I'm just saying that I doubt the problem is anywhere with NFS. I think the problem is a low-level networking change that you can simply undo. If not that, I think it would probably work fine with SMB making no changes. Either way, it would take me longer to type all that than for you to just check and try a few things. And if I'm wrong and the problem is intimately related to Apple's NFS implementation, then it's a total lost cause. SMB is your only option in that case.

elisatems wrote:

WiFi has ALWAYS been a thorn in my side with this, since the /etc/fstab entry on Linux is based on a local fixed IP address, and if WiFi is running, the Linux box often does not see the Mac's ethernet interface but only the WiFi.

To be honest, one thing to try would be WiFi instead of ethernet. Modern WiFi is going to be faster than all but the most hard-core ethernet. In theory, ethernet is going to be more stable and reliable than a radio signal, but not a Mac. Ethernet just doesn't get the Apple love.

Yes, as I said above, this is a way that things have broken before, but it is not the problem this time - at least not the multiple interfaces. But yes, the Mac is connected via a USB ethernet dongle, in fact through a USB hub because the USB ports on this 2021 MBP are so few. That is frequently a problem when switching devices, since I have to disconnect the dongle to do that (I also have my home directory on an external SSD drive). But it never causes the ethernet connection to outright fail once established. This setup also predates even Sequoia, never mind 15.6.

Debugging these things can be tricky. It's not always straightforward. It doesn't matter that something stopped working when you applied the 15.6 update. It could have always been wrong and you were just lucky before.

At a bare minimum, even if you do nothing else at all, you absolutely must connect the ethernet dongle directly. This is not negotiable. It might not help, but you must try. This is an absolute non-starter. Nowhere in the constitution does it say you have a right to a connection through a USB hub. Disconnect the hub - completely.

And remember, what little love Apple gives to ethernet is exclusive to those Apple devices that include built-in ethernet. With a 3rd party dongle going through a 3rd party hub, you have three different companies that will be more than happy to blame the others. None of them will make the slightest effort of any kind.

And I'm not done. You must also disconnect every single other external hardware device. There are widespread reports of interference and cross-talk between radio waves, bluetooth, and even hard-wired USB. These things always start with OS updates. Your description of changing nothing but the OS version is an absolutely textbook example. I realize it's frustrating. While you do have options, they aren't the options that you think you have. You have the option to make no changes and continue guessing about what the cause might be, or you can do a lot of work to try to find out. That work will require you to disassemble a substantial portion of your network.

Remember, you don't have to do that. You can always take the "guess" option. Maybe try SMB/WiFi and hope that something about that is going to be more resilient. That's why I keep suggesting it. You can test that, in a couple of minutes, without changing anything. Maybe it works. If not, then you have to keep looking, but if there's a 2 minute chance involving nothing but the keyboard, why not try it?

By "local export" do you mean exporting to the Mac and nfs-mounting the exported volume locally, on the Mac? I haven't tried that, in fact have never had occasion to try it even on Linux.

I just meant exporting a share from the startup drive instead of an external. At first, that's what I thought the problem was. I can setup an NFS export from /private/tmp, but not from an external. But I actually didn't even test keeping it running all day. Maybe my local connection would have gone stale too.

To be honest, as Mac user, I don't even use any file servers regularly. Network connections simply aren't reliable on modern macOS and haven't been for years. I realize sometimes people apply some random update and get upset that it breaks. But I've been in situations where I had top-of-the-line corporate hardware, with a full team of well-paid, full-time, truly competent network admin people, with literally unlimited funding, and we couldn't make it work on a Mac. This was 2017. I'm sure it's not as reliable today as it was then.

My goal on these forums is to help people have the best experience. Sometimes the best option is to just recommend people avoid techniques that probably aren't going to result in a good experience. It simply doesn't matter how well those techniques work on Windows or Linux. This is where we are. None of us can change Apple. We all know the trade-offs involved when switching, so we probably don't want to do that either. Yes, absolutely all these companies have the resource to fix these problem, but that simply isn't going to happen. We're on our own here.

AFAICT, NFS is all but dead at Apple.

SMB would be the most expedient approach and the next thing to test here.

You can struggle with the current setup, or can try using what Apple spends more time with, SMB.

As for network limitations, if your ISP modem supports bridged mode, and various providers do, then you can use your own network gear. That path requires more effort, more costs, more knowledge, and more control. And likely unnecessary, if the Apple SMB server works, where the NFS server (may be) failing.

But if neither NFS nor SMB works here, I’d then be looking at the Linux box, and at the local network.

elisatems wrote:

Do you have any links to documentation, or even suggestions of which settings to check?

The first place to look is System Settings > Network. I don't know how your system is configured, so some things might not apply. One new setting in Sequoia was Private WiFi Address. As soon as I saw that I knew it would be trouble. I changed the new default to "fixed" so it would behave as it did before. This is the most likely candidate.

There is also a "low power mode" for WiFi. Both Ethernet and WiFi also have a setting for Limit IP Address Tracking. I don't know if these settings are related to the problem you're experiencing.

I don't know much about Ethernet since it's been many years since I've had a Mac with built-in ethernet. If you're using a USB ethernet dongle, all bets are off. You're on your own there. That could break things for multiple reasons, including for the simple fact that you (might?) have multiple interfaces.

If you're referring to the built-in firewall in MacOS that's configurable via System Settings -> Network, it's disabled.

That's good! All it ever did was protect you from that modem and Linux server on your local network.

Understood. Unfortunately since I've never played with the low-level network settings before, I would have no idea what has changed. Probably best if I just try SMB without making any changes to the low-level settings and see if the shared filesystem will stay reachable.

Also try a local export and see if that behaves the same way. What's most confusing to me is the fact that your setup is working at all. It's mainly the crazy Mac permissions that cause problems. Somehow you seem to have avoided that and are dealing with some link-layer problem.

It's possible that SMB will somehow avoid that link-layer problem. That's something you could test really easily. But it would still be interesting to know what precisely is causing your problem. It's all a question of how much time you want to put into debugging it. Or just switch to SMB and maybe get lucky.

It would be better to use SMB. NFS hasn't been supported on the Mac in many years.

The fundamental problem is permissions. NFS is designed to run as the root user. But on modern macOS, root is one of the least powerful user accounts.

elisatems wrote:

I have no interest in proving anyone wrong

I'm somewhat sincere about that. I would really like to know if this is fixable. I just don't think it's a worthwhile exercise.

I'm just not that familiar with SMB - I only remember it as a protocol for sharing volumes between Windows-based and Unix or Linux systems via Samba. I guess I'll have to do some research to learn how to do SMB between Linux and MacOS.

SMB's not difficult. It supports modern file system permissions, which is the big issue. To this day, Apple still officially supports Time Machine over SMB. So if Apple's SMB can handle something that intense, it should be pretty reliable. (Full disclosure: I haven't tested Time Machine over SMB in a long time. And the last time I did a couple of years ago, an Apple bug disabled it for the better part of a year. That's the level of support you get for Apple's "full support" - it works, and if it doesn't work, it might work again in a few months.)

But I *am* surprised that you couldn't get it to work at all.

As I said, I didn't try very hard.

The volume I've been sharing is an external volume and it's worked fine until 15.6. It still works, initially, but becomes unreachable within less than an hour typically.

Perhaps it's not an NFS problem at all. It could be a networking problem. Most of those breaking network changes were in the initial Sequoia release, but you can always double-check things like MAC randomization, energy-efficient ethernet, etc. If you're someone who uses NFS, you might be using other funky network settings. High risk of random breakage with those.

elisatems wrote:

I don't have any unusual options in my NFS server settings on the Mac - no added switches, just the defaults. And my /etc/exports is exactly as posted in the OP. My local network is controlled by an Arris cable modem - but the settings you suggested to check are not options the modem's admin web page even displays. I did check the basic network setup and noticed that the starting address for Wi-Fi had defaulted to a value that conflicted with both the Linux box and the Mac (both connect via ethernet), so I fixed that - but none of my devices/appliances that use Wi-Fi that I'm aware of had been assigned conflicting addresses, and anyway an IP address conflict would affect more than just NFS.

I'll look further into SMB - for now, I've removed the backup share from /etc/fstab and disabled the cron job. I gather from a quick web search that SMB is supported in the MacOS kernel, and the mount command on the Linux box does support -t smbfs, so hopefully the switch will be painless. Thank you for your input.

I don't know if SMB is going to help.

You need to re-evaluate how your network is configured. I don't know what else to tell you. I could write a whole page worth of questions. I need, I just deleted a whole page worth of questions.

My suggestion is to look at the low-level network settings on the NFS server. It sounds like it's doing some kind of MAC rotation or randomization. That is the default. And Apple applied that new default when Sequoia was released. I don't know why it would suddenly start affecting you a year later, but that's sure what it sounds like.

And then there's a whole world of things that I just assume you aren't doing, mostly involving 3rd party "security" or "network" software. People who usually swear they aren't using any of that typically are running the worst of the worst.

elisatems wrote:

Trying SMB is a nonstarter I'm afraid - mount -t smbfs on the Linux box returns "unknown filesystem type", despite the mount(8) man page listing -t smbfs as supported. There is no mount_smbfs command either.

Here is an article for installing "cifs utilities" on various Linux distributions:

https://linuxize.com/post/how-to-mount-cifs-windows-share-on-linux/

I had a similar problem. Locally on the Mac, I had a volume at /Volumes/Archive. But I couldn't connect to it with:

sudo mount -t cifs -v //macbookpro/Volumes/Archive /mnt/archives -o username=jdaniel

I had to manually add it to the list of shared volumes in System Settings. Once I did that, it showed up there as just "Archive". So then, I tried with this:

sudo mount -t cifs -v //macbookpro/Archive /mnt/archives -o username=jdaniel

And that worked.

elisatems wrote:

But what changed in the latest OS update? Why does this not work anymore?

No clue. Last time I used NFS on any system was 2014. I'm not surprised it doesn't work. I'm only surprised that it stopped working for you recently. I would have expected a post like this in 2018 or 2019 maybe.

I fired it up to test and I couldn't get an external volume to share via NFS. An internal volume worked fine. I didn't try very hard. There are dozens of things that could potentially be breaking it. And it might simply not be possible at all.

You can give up now, switch to SMB, and have it working before lunch. Or keep trying for a few more days and give up then. If you want to prove me wrong, please make sure to post a detailed explanation of how you did it.