Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

nfsd refusing mounts after upgrading to 11.3 Big Sur?

Hello All,


I have a MacMini with a connected Drobo 6D that I use as my central home server. This includes a few VMWare ESXi servers & Linux VMs.


After upgrading to the Big Sur, I am no longer able to use NFS on the ESXi & Linux clients because all clients are seeing permission denied when attempting to mount.


This all worked before upgrading.


On the Macmini:

bash-3.2# showmount -e localhost

Exports list on localhost:

/Volumes/Drobo/Media 192.168.100.55

/Volumes/Drobo/Backups/_Isos 192.168.100.50 192.168.100.60

bash-3.2# cat /etc/exports

/Volumes/Drobo/Media -maproot=daemon 192.168.100.55

/Volumes/Drobo/Backups/_Isos -maproot=daemon 192.168.100.50 192.168.100.60


On a Linux client (the 192.168.100.55):

ubuntu@ubuntu18-Plex:~$ showmount -e macmini

Export list for macmini:

/Volumes/Drobo/Media 192.168.100.55

/Volumes/Drobo/Backups/_Isos 192.168.100.50,192.168.100.60

ubuntu@ubuntu18-Plex:~$ grep mymedia /etc/fstab

macmini:/Volumes/Drobo/Media /mymedia nfs auto,ro,vers=3,bg,intr 0 0

ubuntu@ubuntu18-Plex:~$ sudo mount /mymedia

[sudo] password for ubuntu:

mount.nfs: access denied by server while mounting macmini:/Volumes/Drobo/Media

ubuntu@ubuntu18-Plex:~$ sudo mount -v /mymedia

mount.nfs: trying text-based options 'vers=3,bg,intr,addr=192.168.100.30'

mount.nfs: prog 100003, trying vers=3, prot=6

mount.nfs: trying 192.168.100.30 prog 100003 vers 3 prot TCP port 2049

mount.nfs: prog 100005, trying vers=3, prot=17

mount.nfs: trying 192.168.100.30 prog 100005 vers 3 prot UDP port 902

mount.nfs: mount(2): Permission denied

mount.nfs: access denied by server while mounting macmini:/Volumes/Drobo/Media


According to the showmounts, I've exported /Volumes/Drobo/Media on the macmini and again, this was working before Big Sur changed something ad broke it.


I've tried grepping for nfs & mountd in /var/log/* without seeing anything.


Where does MacOS hide the mountd error messages? What did Big Sur change that has broken my NFS mounts?

Mac mini, macOS 11.3

Posted on May 2, 2021 1:51 PM

Reply
Question marked as Best reply

Posted on May 4, 2021 11:36 AM

phmparis wrote:

I've tried grepping for nfs & mountd in /var/log/* without seeing anything.

Where does MacOS hide the mountd error messages? What did Big Sur change that has broken my NFS mounts?

That would be in the Console app. You can also try the "log" command-line tool for more low-level access. Note that using Console can be overwhelming. Expect several hundred messages per second. Using "log" is more manageable, but you'll have to learn the predicate language. On multiple occasions, I have seen people literally lose their minds looking at console messages. I am not exaggerating.


This sounds like the same problem documented here: https://github.com/hashicorp/vagrant/issues/12330

It seems to be related to a security issues accessing external volumes.


Unfortunately, this kind of problem is getting more and more common. Apple really doesn't have great networking support at all anymore. You might have better luck using SMB, which is likely to be better tested. Nobody cares about NFS.


But going beyond that, there is literally an army of "internet security researchers" looking for any Apple bugs to either 1) see their name up in lights, or 2) collect a "bug bounty". Just like SMB/NFS, marketing and popular media are a big factor here. Nobody cares about hacking Linux or Windows. Apple is the golden ticket to security fame and fortune. Plus, it is a little-known fact that some of these "security researchers" are full-time employees of Apple's biggest competitors who are paid to find these bugs.


Therefore, whenever someone finds a bug, that starts a 90-day countdown. Apple has to patch that bug at all costs or the exploits will be publicized. That's both bad PR and a (newly) public security hole. It is essentially legal extortion. Apple doesn't have a lot of time to develop a fix and test it. There are likely to be side effects. The further away from mainstream consumer Mac usage that you are, the more likely you are to experience a side effect. Running an NFS server on a Mac? Yeah, that's going to be hard in this environment. Try the workarounds mentioned in that link. File a bug report with Apple. There is little to no chance that a bug like this will ever be fixed in macOS 11 with macOS 12 so close. Chances are, you are now committed to test the fix in the macOS 12 betas, undoubtedly with a whole bunch of other bugs.

Similar questions

15 replies
Question marked as Best reply

May 4, 2021 11:36 AM in response to phmparis

phmparis wrote:

I've tried grepping for nfs & mountd in /var/log/* without seeing anything.

Where does MacOS hide the mountd error messages? What did Big Sur change that has broken my NFS mounts?

That would be in the Console app. You can also try the "log" command-line tool for more low-level access. Note that using Console can be overwhelming. Expect several hundred messages per second. Using "log" is more manageable, but you'll have to learn the predicate language. On multiple occasions, I have seen people literally lose their minds looking at console messages. I am not exaggerating.


This sounds like the same problem documented here: https://github.com/hashicorp/vagrant/issues/12330

It seems to be related to a security issues accessing external volumes.


Unfortunately, this kind of problem is getting more and more common. Apple really doesn't have great networking support at all anymore. You might have better luck using SMB, which is likely to be better tested. Nobody cares about NFS.


But going beyond that, there is literally an army of "internet security researchers" looking for any Apple bugs to either 1) see their name up in lights, or 2) collect a "bug bounty". Just like SMB/NFS, marketing and popular media are a big factor here. Nobody cares about hacking Linux or Windows. Apple is the golden ticket to security fame and fortune. Plus, it is a little-known fact that some of these "security researchers" are full-time employees of Apple's biggest competitors who are paid to find these bugs.


Therefore, whenever someone finds a bug, that starts a 90-day countdown. Apple has to patch that bug at all costs or the exploits will be publicized. That's both bad PR and a (newly) public security hole. It is essentially legal extortion. Apple doesn't have a lot of time to develop a fix and test it. There are likely to be side effects. The further away from mainstream consumer Mac usage that you are, the more likely you are to experience a side effect. Running an NFS server on a Mac? Yeah, that's going to be hard in this environment. Try the workarounds mentioned in that link. File a bug report with Apple. There is little to no chance that a bug like this will ever be fixed in macOS 11 with macOS 12 so close. Chances are, you are now committed to test the fix in the macOS 12 betas, undoubtedly with a whole bunch of other bugs.

May 4, 2021 11:53 AM in response to etresoft

I tried Console but as you said, its very verbose and I was unable to pick out any kernels from the chaff of all it's many messages.


I posted here as a first step before contacting Apple Support.


I appreciate that NFS is a corner case and likely little tested but as Anton has pointed out, I'm not the only one encountering this issue so hopefully it will be identified and fixed. My use is for a home Plex server so it's annoying but not life threatening & I have started looking at moving to SMB but that has it's own issue(*).


I'm downloading 11.13.1 because you never know, I might just get lucky and Apple will have fixed the issue.


Thank you and Anton for commenting. I'll look into the GitHub link after updating.


  • With SMB the problem I'm seeing is that I've created a media user & shared the folder but Ubuntu want's a workgroup in addition to a username+password, "." isn't accepted and I haven"t been able to figure out what is needed yet.

May 9, 2021 12:19 PM in response to phmparis

these are my shares (similar to yours)




so i have to mount /volumes/backup and /volume/film first? i dont want to remove the shares i already have as im afraid kodi will delete my whole library. anyway, ill never update macos again. only final releases (11.5, 12.5, 13.5 etc). everytime theres a problem! tired of being a betatester!

May 2, 2021 2:48 PM in response to Old Toad

There is no issue with the volume hosted by the Drobo on the MacMini. It's a 8D (not a 6D as I mistakenly said earlier) so it uses Drobo Dashboard to mount the volumes on the Drobo & I see the files on the Macmini.


The problem is that Big Sur seems to have changed something and now all mount requests are getting permission denied and I haven't been able to find any error messages from nfsd explaining why.


Normally/previously, adding the following line to /etc/exports and performing a "nfsd update" would authorize 192.168.100.55 to mount /Volumes/Drobo/Media.

/Volumes/Drobo/Media 192.168.100.55


As I stated in my first post, the client 192.168.100.55 sees that /Volumes/Drobo/Media is exported (using showmount).

May 4, 2021 11:15 AM in response to phmparis

I’m seeing this as well; exports which worked under 11.2 fail under 11.3. nfs logging doesn’t seem to show anything.


Apple hasn’t posted 11.3 source code yet, so can’t see what changed there.


getfh() appears to be failing:


root# dtrace -n 'getfh:return { printf("%d %d", arg0, arg1) }'

...

CPU     ID                    FUNCTION:NAME

  8    484                     getfh:return -1 -1

  0    484                     getfh:return -1 -1


I haven't been able to see errno or figure out why no error is being logged (anywhere I've found) for this, though.

May 5, 2021 4:13 AM in response to phmparis

Hello All,


Upgrading to 11.13.1 didn't solve anything (except the Safari vulnerabilities it corrected).


However, Etresoft's reference to the GitHub.com post did.


The Github article mentions that Apple corrected a local privilege escalation in APFS with "improved state management". This "improved state management " seems to have borked nfsd's ability to export subdirectories of all external volumes (Like my Drobo) directly - even though Drobo doesn't use APFS. The workaround is to export the volume root (/Volumes/Drobo in my case) and mount that somewhere on your NFS clients first, after which Apple's nfsd may allow you to mount subdirectories.


In my case, even though I was only exporting folders inside /Volumes/Drobo & not /Volumes/Drobo itself, everything was already mounted RO everywhere and I don't mind having the Drobo's other contents available to clients. So, I just simplified /etc/exports to just export /Volumes/Drobo, added -ro to the export options, mounted that to /Drobo on the clients and replaced /mymedia with a symlink to /Drobo/Media on the Plex VM.


People using Automount and those who only export parts of external volumes for security reasons won't be so lucky. Hopefully Anton and others seeing this will be able to work around the bug too.


Thanks for your help Anton & Etresoft!

May 8, 2021 2:40 PM in response to Damian83

Hi Damian,


Hopefully the workaround of exporting & using the volume root (/Volumes/Drobo fo me) is sufficient for your situation too. If not, and you check if you can mount folders inside the volume after mounting the volume root (First mount /Volumes/Drobo and then see if /Volumes/Drobo/Media) works I'm sure that confirming that it kludges around the problem would help others.

May 9, 2021 11:03 AM in response to Damian83

Hi Damian,


You never detailed exactly what your "shares" and your volumes are so I can't really answer. In addition, the subject is NFS so it's exports & not shares. According to the Github discussion, the mount of the volume root needs to be done before you can mount a subfolder of an external volume. According to tests I've performed, the NFS mount will only work on the external volume's root (until Apple fixes the bug).


In my case, my external volume is /Volumes/Drobo. Before this bug, I was exporting /Volumes/Drobo/Media to my Plex server and /Volumes/Drobo/Backups/_Isos to my ESXis. By only exporting specific folders & not the volume root I was limiting the files on my Drobo that are made available to NFS clients. In my case, as I said earlier, I don't really need to hide the rest of /Volumes/Drobo from my Plex server or the ESXis so I just changed my export to /Volumes/Drobo & it works on both Plex & ESXi.


There was one question I had; Once you export and mount /Volumes/Drobo, can you then mount /Volumes/Drobo/Media? The answer is no.


I just tested mounting /Volumes/Drobo (OK) and then attempted to mount /Volumes/Drobo/Media (KO) on my Plex VM. I assume that /Volumes/Drobo/Media was refused because it's not explicitly exported from the Mac. However given that /Volumes/Drobo/Media is a subset of /Volumes/Drobo the export fails with:

bash-3.2# nfsd checkexports

exports:2: /Volumes/Drobo/Media conflicts with existing export /Volumes/Drobo


You'll need to detail your exports & mounts explicitly if you want us to help you kludge around the bug.

May 12, 2021 6:31 AM in response to Damian83

Your volume roots are /Volumes/Backup and /Volumes/Film as you suspected.


If you change the exports from /Volumes/Backup/Film to /Volumes/Backup and then attempt to mount those on your NFS clients somewhere, you can then use a symlink to point to Film inside wherever you mounted it and make Kodi happy again.


In my case I was exporting/mounting /Volumes/Drobo/Media to /mymedia on my Plex. After exporting /Volumes/Drobo on the Mac I was able to mount it to /Drobo on the Plex. On the Plex NFS client I removed the old /mymedia mount point and created a symlink from /Drobo/Media to /mymedia so that Plex would see the files and directories in the expected /mymedia/* location . To make the symlink I used "ln -s /Drobo/Media /mymedia". You should be able to do something very similar for your two volumes.


On Plex you have to Empty trash explicitly from a volume to make missing files disappear from the index. I assume that Kodi is like Plex in that it doesn't remove temporarily absent files automatically from its databases either so unless you do so explicitly, your Kodi isn't going to delete your library.


Yeah this bug is a pain and one could wish that Apple had caught all the bugs but using obsolete and vulnerable MacOS versions is it's own larger danger. This is a pretty exceptional time for Apple updates because with Big Sur they are weeding out all the 32 bit code (which is A BIG DEAL). Future updates are extremely unlikely to see changes as big. I work in network security and am thus conscious of the severe risks you run in in not upgrading so now that you know the workaround I'd counsel against avoiding upgrades. Wait a month to let others debug, sure but don't let yourself get hacked because you're still using vulnerable MacOS versions.




[Edited by Moderator]

May 12, 2021 6:41 AM in response to phmparis

yes, i shared root volumes and touched nothing on kodi. however i have to scrap my whole library again as something messed during all this tweaking. i also had to change permissions to subfolders so only those which movies inside are readable from kodi. anyway, these issues with nfs its a recurring nightmare that started from catalina. think that nfs is an unix native protocol... wondering why macos uses smb as default sharing protocol, while nfs can be accessed only by terminal, by experienced users or, like in my case, with dedicated apps that arent even on app store...

May 12, 2021 10:35 AM in response to Damian83

Ah, sorry to hear you had issues with Kodi. I can unfortunately relate, having had to rebuild my Plex library last year. It did get me to rename all my media files so that Plex would recognise them all automatically instead of having to manually re-match many files. I'd advise learning what action you took on Kodi that made the temporarily missing files a problem, to avoid future issues.


NFS is a UNIX origin protocol. MacOS uses SMB after deprecating AFS because almost everyone prefers SMB but as I've been administering Unix servers for almost 30 years & it's the best way to share files with VMWare ESXi servers, using it for my Plex was just easier.

May 12, 2021 11:27 AM in response to Damian83

Damian83 wrote:

nfs is an unix native protocol... wondering why macos uses smb as default sharing protocol, while nfs can be accessed only by terminal, by experienced users or, like in my case, with dedicated apps that arent even on app store...

NFS is old-school UNIX, from before people worried too much about security. When Solaris went away, it pretty much took NFS with it. These days, enterprise networks mean Active Directory, even with UNIX, and that means SMB. macOS still supports all of that, but it is definitely "supports" with air quotes.

nfsd refusing mounts after upgrading to 11.3 Big Sur?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.