Is there a launch daemon size limit?

etresoft wrote:

BDAqua wrote:

Sorry folks, Apple can do better than this. 😉

No true. I have spent many years studying Panglossian engineering and I can assure you, that in any system you encounter, the people who built it were always doing their best work.

Hi etresoft! the key word in your advocacy for the "Panglossian engineering" (altruism) is "the people who built it". It's the people who exploit it that we are concerned with. In the real world, the people who use the brain child of genius are concerned with one principle and one principle alone. "what's in it for me". This is NOT a condemnation, this is a fact of life. Most of us lessor beings have to consider what we can do to insure that we have an income in order to survive. We, therefore are obliged to build on the base of the geniuses that provide us with the tools to make things work. It's a hard world out there friend.

....goldie

Goldenbill wrote:

Hi etresoft! the key word in your advocacy for the "Panglossian engineering" (altruism) is "the people who built it". It's the people who exploit it that we are concerned with. In the real world, the people who use the brain child of genius are concerned with one principle and one principle alone. "what's in it for me". This is NOT a condemnation, this is a fact of life. Most of us lessor beings have to consider what we can do to insure that we have an income in order to survive. We, therefore are obliged to build on the base of the geniuses that provide us with the tools to make things work. It's a hard world out there friend.

....goldie

Oh, heavens, let's not delve into melodrama. The problem - as always - is that people expect things to work the way they personally want things to work, without bothering to consider what that implies for others. Hindsight is 20/20, and it's so much easier to be a critic than a producer. Anybody can sit back and complain that a juggler ought to add a chainsaw to the seven balls he's already got in the air, but few of those who'd voice that complaint knows what it takes to get seven balls going in the first place.

Yes, in an ideal world apple could do better. No, this is not an ideal world. Feel free to complain - I do all the time; complaining (done properly) is good for the soul - but don't make more out of it than it is. It's not by design, it's because everyone has their own little head-trip going on. Including us...

I am pretty sure the key word is my reference to Dr. Pangloss. I am a big fan of hacking. But hackers, more so than "professionals", need to have a good understanding of the technology involved. They aren't being paid by the hour or in stock options. If you wanted to test the limits of launchd, then you might want to experiment with edge cases, globbing for the win, or funky shells. If you actually want to get something done, then you have to assume a healthy level of incompetence and stay away from those edge cases.

Well, back on topic... You might look at /usr/include/sys/syslimits.h., in particular MAX_INPUT. This seems to be an issue with the shell not launchd.

Neville Hillyer wrote:

Sorry folks but I would prefer not to have my thread hijacked any further.

There have been several Terms of Use violations including:

2 Submissions

1. Stay on topic - -
2. Be polite - -
3. Post constructive comments - -

The ultimate arbiter is always the behavior shown by the current, running operating system. Everything else is secondary to that behavior. In short:

• If it hurts, don't do it.
• If you believe you have found a software bug or a documentation omission, please report it to Apple.
• Don't expect any observed (mis)behavior to change until after a patch arrives, and don't expect any particular patch to be back-ported to any earlier versions. Put another way and barring influence with the vendor that is not in evidence here, you're going to have to use a work-around pending a fix. This assuming that this (mis)behavior will even be considered a bug, and not an undocumented limit, of course.

In general, when dealing with programming and with application interfaces:

• It's best to stay well within the expectations and assumptions of the folks that wrote the code, and within the available documentation.
• Don't expect code that's dependent on undocumented behaviors to keep working, if it works at all.
• formal answers come from formal people, and not (usually) from forum postings here.
• If there's no specific documentation of limits, stay within what the vendor uses for their own applications and environments where that's feasible.
• whether the current behavior of any API is considered the "correct" behavior is a decision that only Apple can make.

Going forward? Acquire and use developer access to test on newer and actively-developed software versions, and discuss problems with those via the bug reporter and via the developer forums.

As for expectations? If you want to know the details of the current OS X implementation and its innate limits, please don't expect me (or most other folks here) to read and trace the source code for you. This is your bug and your application, and not one of mine, and it's thus on you to find and report and work-around the bug, and not on me (or folks other than Apple) to look at or possibly alter the code, or document the code.

Having done operating system coding for many years for a commercial vendor and been on the receiving end of more than a few of these sorts of forum discussions and bug reports, I can understand your frustration with the lack of documentation evident here, but I can also tell you that particular discussions here are unlikely to produce a quick solution to your current predicament, nor any likely acquire guidance more specific than "log a bug" and "if it hurts, don't do that" and to then work to stay within the constraints of the available interfaces, with whatever those constraints might be observed.

As for alternatives and available options here, you're also obviously free to switch to a different operating system that better meets your needs and expectations of course, or to create your own OS if you're so inclined — building and maintaining an operating system is very interesting and very challenging — or to hire somebody to dig through the OS X source code to find the current limits if you're not already familiar with the programming languages used and not in a position to learn more about those implementation languages yourself. (Again, digging through the source code can take several hours and variously longer — this for finding and then reading the applicable library and operating system code.)

Good luck with resolving this limit and this problem of course, but do expect to use a workaround for at least a while.

Hello

As far as I can tell after browsing the source codes, 4096 limit is hard-coded in CF2launch_data() function defined in launchctl.c.

cf.

http://www.opensource.apple.com/tarballs/launchd/launchd-258.25.tar.gz

launchd-258.25/launchd/src/launchctl.c

CF2launch_data()

char buf[4096]

And 1024 limit when used with EnableGlobbing key is due to the implementation of glob(3) which is used when EnableGlobbing key is specified as stated in man page of launchd.plist(5).

cf.

launchd-258.25/launchd/src/launchd_core_logic.c

job_start_child()

http://www.opensource.apple.com/tarballs/Libc/Libc-498.1.7.tar.gz

Libc-498.1.7/gen/FreeBSD/glob.c

glob()

char patbuf[MAXPATHLEN]

/Developer/SDKs/MacOSX10.5.sdk/usr/include/sys/param.h

#define MAXPATHLEN PATH_MAX

/usr/include/sys/syslimits.h:

#define PATH_MAX 1024 /* max bytes in pathname */

Regards,

H

Edit: fixed launchd.plist(3) to launchd.plist(5).

Thanks for the information.

I would like to know if and where Apple documented these limits other than within source code.

An insight into why these low limits were applied would be interesting.

Perhaps this type of information was available on the Developer forum about the time that OS X 10.5 was released.

It's not documented. (We'd not be three pages into this discussion, if it was documented.)

If you want to see this documented, then log the bug report that was requested days ago, or log a formal incident with the developer folks.

It was purely an arbitrary decision on the part of the coder. Here is the same code from 10.9

// Big enough. Don't feel like jumping through CF's hoops.char *buff = malloc(4096);

The string is for program arguments, not programs. If you feel this is a problem do as everyone else here has said, file a bug report.

Frank Caggiano wrote:

It was purely an arbitrary decision on the part of the coder. Here is the same code from 10.9

// Big enough. Don't feel like jumping through CF's hoops.char *buff = malloc(4096);

The string is for program arguments, not programs. If you feel this is a problem do as everyone else here has said, file a bug report.

I'm tempted to suggest that Neville download the launchd source, edit in a larger buffer size (or maybe even recode it for arbitrary length code), recompile it, and swap it out for the version that ships with the system. It is open source, after all... However, I suspect that would merely extend this discussion for another 12 pages as he tried to iron out the host of system errors that would likely be the result of the first 10 (or 50) revisions.

But then again, maybe that would give him a taste for the difficulties involved...

Neville Hillyer wrote:

I would like to know if and where Apple documented these limits other than within source code.

An insight into why these low limits were applied would be interesting.

Perhaps this type of information was available on the Developer forum about the time that OS X 10.5 was released.

You just don't get it, eh? That is the documentation. There is nothing else. This is as good as it gets.

Hello

I don't know whether the 4K size limit of string value in launchd.plist is documented anywhere other than the source code. (Good source code is self-documenting and indeed may be considered the primary document per se, though.) I myself think the limit is rather implementation details at programmers' discretion whilst it wouldn't hurt for them to state it briefly in launchd.plist(5) manual page. Size limit of pattern in glob(3) is briefly mentioned in glob(3)'s manual page [1].

The limit could have been 32K or 64K or even 200K because execve(2) can accept argv of size 256K [2]. Instead it has been set to 4K, which would tell that string value in launchd.plist is not supposed to hold program itself but the parameters for program and they think 4K is enough for each parameter.

[1] glob(3)

BUGS

Patterns longer than MAXPATHLEN may cause unchecked errors.

[2] /usr/include/sys/syslimits.h: #define ARG_MAX (256 * 1024) /* max bytes for an exec function */

---

I would save long script as external file and invoke it instead of putting it directly in string value in launchd.plist. But if you really inclined to put every thing in launchd.plist, you could try perl or ruby to exec the shell script, for these interpreters have -e option which can be used multiple times. As 4K limit is applied to each string value between <string> and </string>, you may divide the script into multiple parts less than 4K and supply each using inidividual -e option of these interpreters.

E.g.,

/usr/bin/perl -e "exec('"  -e 'cd ~/desktop' -e 'pwd' -e "')"

/usr/bin/ruby -e "exec('"  -e 'cd ~/desktop' -e 'pwd' -e "')"

Note that the final combined script passed to exec function in perl or ruby must be properly quoted in each language. In this example, single quote is used as quoting character whilst shell script does NOT contain single quote. If shell script contains single quote, you must either quote the single quote properly or use different quoting character such as -

/usr/bin/perl -e "exec('"  -e "echo \"Quote (\') in quoted string is tricky.\"" -e "')"
/usr/bin/perl -e "exec(q@"  -e "echo \"Quote (') in quoted string is tricky.\"" -e "@)"

/usr/bin/ruby -e "exec('"  -e "echo \"Quote (\') in quoted string is tricky.\"" -e "')"
/usr/bin/ruby -e "exec(%q@"  -e "echo \"Quote (') in quoted string is tricky.\"" -e "@)"

* Snippets are tested to work as they are written but NOT tested in launchd.plist.

Regards,

H

Edit: fixed typos.

Hi Neville,

Best document I could find:

Refer:https://developer.apple.com/library/mac/documentation/macosx/conceptual/bpsystem startup/Chapters/CreatingLaunchdJobs.htm…

A snippet from the document: " Creating a launched Property List File "

"For sample configuration property lists, look at the files in

/System/Library/LaunchDaemons/

If you look at the plist files in this directory, they all have short ProgramArguements definitions and they don't embed a script into the plist.

I found the technique to embed the script within the launchd plist interesting, and the discussion has found in source where the limitations are applied.

Cheers