You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Big Sur: too many open files

I am working as a software developer and I use maven for building a heavy Java project. I am trying to take advantage of maven's abilities to build modules in parallel but am stumped by a problem cause by macOS. Every time I try running parallel builds I run into an error message like this:


java.io.FileNotFoundException: ... (Too many open files)


I have tried setting bigger limits by following the guide here: https://www.macobserver.com/tips/deep-dive/evade-macos-many-open-files-error-pushing-limits/ but it doesn't seem to make a difference at all. Even setting the value to 10 Million causes the same issue, and I am not opening 10 Million files here. My guess is that macOS simply ignores the setting.



MacBook Pro (2020 and later)

Posted on Jul 29, 2021 10:04 AM

Reply

Similar questions

7 replies

Jul 29, 2021 11:20 AM in response to dprodrigueza7

Open Terminal and issue the following command(s):


ulimit -a


What is the "Maximum number of open file descriptors"? (256 is the default)


Try


ulimit -n 10240 (others have determined this to be some sort of maximum with macOS)


Then test maven from this same command prompt as the value will reset to the default if you close out this shell session.

If this works, you can set the ulimit in your .zshrc / .bashrc etc.





Jul 29, 2021 11:29 AM in response to dprodrigueza7

Another possibility to set the soft and hard limits is to create a LaunchDaemon. Apparently, this is the way to do it since High Sierra.


Create a file at /Library/LaunchDaemons/limit.maxfiles.plist


<?xml version="1.0" encoding="UTF-8"?>  
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"  
        "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">  
  <dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
      <string>launchctl</string>
      <string>limit</string>
      <string>maxfiles</string>
      <string>64000</string>
      <string>524288</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
  </dict>
</plist> 


Make sure the file is owned by root:wheel


Then


sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist


Test


launchctl limit maxfiles


Jul 30, 2021 10:59 AM in response to dprodrigueza7

How many files and processes are you actually opening with your maven jobs?


lsof -p <pid of jvm> > lsof.log



Here's another launchDaemon for max processes


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxproc</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxproc</string>
          <string>2048</string>
          <string>2048</string>
        </array>
      <key>RunAtLoad</key>
        <true />
      <key>ServiceIPC</key>
        <false />
    </dict>
  </plist>


FYI,


I see several bug fixes on Apache Issues for Maven in regards to this issue. Are you running an updated Maven version?

Perhaps you are generating a ridiculously large number of open files / processes.



Big Sur: too many open files

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.