How to increase the limit of “maximum open files” in C?

The default limit for the max open files on Mac OS X is 256 (ulimit -n) and my application needs about 400 file handlers.

I tried to change the limit with setrlimit() but even if the function executes correctly, i'm still limited to 256.

Here is the test program I use (compiled with gcc):


#include <stdio.h>
#include <sys/resource.h>
main()
{
struct rlimit rlp;
FILE *fp[10000];
int i;
getrlimit(RLIMIT_NOFILE, &rlp);
printf("before %d %d ", rlp.rlim_cur, rlp.rlim_max);
rlp.rlim_cur = 10000;
setrlimit(RLIMIT_NOFILE, &rlp);
getrlimit(RLIMIT_NOFILE, &rlp);
printf("after %d %d ", rlp.rlim_cur, rlp.rlim_max);
for(i=0;i<10000;i++) {
fp = fopen("a.out", "r");
if(fp
==0) { printf("failed after %d ", i); break; }
}
}


and the output is:


before 256 -1
after 10000 -1
failed after 253


I cannot ask the people who use my application to poke inside a /etc file or something. I need the application to do it by itself.

MacBook Pro 15", Mac OS X (10.6.4)

Posted on Jul 2, 2010 8:28 AM

Reply
28 replies

Jul 5, 2010 10:36 PM in response to xnav

etresoft: did you read the man???

"Limits on the consumption of system resources by the *current process * and each process it creates may be obtained with the getrlimit() call, and set with the setrlimit() call."

setrlimit sets the limit of the CURRENT PROCESS and it's child. So the test program should work as it.

xnav: it's interesting to see that on XCode, the default current limit is 9472 but that you cannot change it neither.

Jul 6, 2010 6:38 AM in response to acemtp

acemtp wrote:
etresoft: did you read the man???


Yes

"Limits on the consumption of system resources by the *current process * and each process it creates may be obtained with the getrlimit() call, and set with the setrlimit() call."


You missed the most interesting bit on that man page, by far. Read the very last line:

4th Berkeley Distribution June 4, 1993 4th Berkeley Distribution


I wouldn't rely too much on that man page.

setrlimit sets the limit of the CURRENT PROCESS and it's child. So the test program should work as it.


Clearly, it only sets the limit on child processes.

After further testing, there do appear to be some bugs in this function. When I ran tests last night, setting "rlp.rlim_max" was a requirement. I couldn't get it to work otherwise. I also had to run as root. This morning, neither requirement holds any longer. The only part that is consistent is that any changes apply only to child processes.

The moral of the story? Don't use this function!

Jul 7, 2010 8:48 AM in response to acemtp


7/7/10 5:42:42 AM, Jul 7 [0x0-0x2e02e].com.yourcompany.ForumObjC[618] fopen(): Too many open files
7/7/10 5:42:42 AM, Jul 7 [0x0-0x2e02e].com.yourcompany.ForumObjC[618] i == 9993
7/7/10 5:42:42 AM, Jul 7 [0x0-0x2e02e].com.yourcompany.ForumObjC[618] before 256 -1
7/7/10 5:42:42 AM, Jul 7 [0x0-0x2e02e].com.yourcompany.ForumObjC[618] after 10000 -1
7/7/10 5:42:42 AM, Jul 7 com.apple.launchd.peruser.501[134] ([0x0-0x2e02e].com.yourcompany.ForumObjC[618]) Exited with exit code: 1

It appears to work as expected.

Jul 8, 2010 12:13 PM in response to acemtp

acemtp wrote:
Ok, so it'll work with an .app and will not with the terminal.

I continue to wonder why setrlimit doesn't work like "man" says


You know, sometimes people here on AppleDiscussions call me an "Apple fanboy", claiming that I always try to defend Apple and blame the user. I don't consider that an accurate assessment. I just feel that Apple has thousands of engineers who are experts in these areas and, based on overwhelming evidence, they are really good at what they do. Of course, some bugs inevitably fall through the cracks, but 99 times out of 100, it is user error.

Upon closer reading of the man page and continued testing, I have concluded that the man page is correct and setrlimit works properly. The bugs that I thought I had found were just user error on my part. The same applies for the bugs you thought you found.

The whole problem here is your printf() function. When you call printf(), you are initializing internal data structures to a certain size. Then, you call setrlimit() to try to adjust those sizes. That function fails because you have already been using those internal structures with your printf(). If you use two rlimit structures (one for before and one for after), and don't print them until after calling setrlimit, you will find that you can change the limits of the current process even in a command line program. The maximum value is 10240.

If you are going to open 400 files at once, please be smart about it and use select() if you need to read them. I looked at the /usr/include/sys/_structs.h file and the default maximum number of file descriptors that select() accepts is 1024. It appears that you can change this by defining your own FD_SETSIZE. I suspect this would only work when calling select() directly.

Jul 8, 2010 2:03 PM in response to xnav

xnav wrote:
Good sleuthing fanboy! 🙂


I just wish I could delete all my other replies in this thread 🙂

One of the unique aspects of being a programmer is having to fail at something 63 times in a row before it works. Then, to reward yourself, you move on to another program and fail 63 times in a row. Usually, however, that process takes place in private and all anyone ever sees is the end result. In this thread, I have documented the workings of my mind for all to see - and it isn't pretty.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

How to increase the limit of “maximum open files” in C?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.