Is there a launch daemon size limit?

I suggest you don't do that. launchd is designed to be 'light'; it's so central to the system that they don't want it getting bogged down, so they've geared it to being simple.. If you have a long bash script to run, write it as a separate script and call it from the plist file by writing its address into the Program or ProgramArguments key.

I understand the desire to keep things together in one place, but if you try for a workaround to achieve that relatively minor goal in this particular case you may find it has unfortunate unintended consequences on the entire OS.

I am unaware of any such limit. What do you mean by "gets confused"?

You could try looking at the man pages for launched.plist and launchctl, specifically the HardResourceLimits key in the former and the limit subcommand in the latter. launched will use those to set resource limits on particular jobs, though I can only guess at which limits you might be exceeding. That may work if the issue you're having is a per-job limitation; it won't help at all if it's a system-wide limitation.

And don't get me wrong: putting the bash script in a separate file is the sensible thing to do so I'm obliged to point it out, but I know that many computer-oriented people have an aversion to being sensible. Coders have an artistic temperament that sometimes brooks no opposition. You're good enough at this stuff so that if you torch your system - a distinct possibility if you muck around with launched the wrong way - you can reinstall it without too much trouble. Just so long as you know it's a possibility.

Neville Hillyer wrote:

I have discovered that with EnableGlobbing set to true the maximum size of the bash script which will work is 1024 bytes. As the size is increased further the name of tmp.txt starts getting truncated and eventually random errors are generated such as - unexpected EOF while looking for matching `"' - I have had errors claiming a missing quote or bracket even when they do not exist in my script.

With EnableGlobbing set to false the maximum size of the bash script which will work is 4096 bytes. As the size is increased further output is totally inhibited without file truncation or any error reports - random or otherwise.

I have noted that any bash comments count towards the above maxima.

I would be grateful if anybody can shed further light on this.

I will investigate alternatives to EnableGlobbing - I have not used it before and only need it this time to expand '~' in one location.

This points to what I was trying to talk about earlier; launchd is a stripped-down process. Maybe a quick explanation of how launchd works will help.

Launchd has three elements:

• The launchd daemon: this is process 0, the root process that starts up the system and directly or indirectly launches and governs all other processes. There are actually separate launchd processes for each user which are launched by the systemwide launchd, but never mind that. The launchd daemon is designed to be efficient to an extreme because it is the core process. any inefficiency in launchd affects the whole system.
• launchd jobs: a job is a set of conditions, usually loaded from a plist file. If you imagine that launchd has a refrigerator with a whiteboard attached to it, a job is a note on that whiteboard (e.g., 'buy carrots', 'go to dentist at 3:00', 'when Mary arrives, start roast'). The whiteboard itself doesn't accomplish any of these, it merely lays out what needs to be done under what conditions.
• launched processes: any process that is launched because one of the conditions in a launchd job has been met. the launchd daemon monitors these processes lightly, taking care of killing off subprocesses when the process quits or restarting the process if it quits unexpectedly (if the job specifies it should do that)

The problem you're having (to continue the analogy) is that you're running out of whiteboard space because you're trying to write the entire recipe for the roast straight onto the whiteboard, rather than making an indirection like "start roast (recipe book pg. 3)". That's what's happening with the globbing: by enabling globbing you're asking launchd to add code for a more complicated file system search (one potentially involving wildcards or shell special characters) and the added code is taking away from the internal space reserved for the job, resulting in truncation of the process code and subsequent errors. Again, launchd is stripped-down for efficiency: there is no meaningful error checking.

The question you want answered is how to convince launchd to allocate more internal whiteboard space for a specific job listing or for all jobs on its list. For all I know the job allocation size is hard-coded into launchd itself (that would be the efficient way to do it, at any rate), but if it's written into a preference file I have no idea where, and if none of the launchctl or launchd.plist keys affects it (which I have doubts about - they seem geared to the resource limits of the resultant process, not of the job), then I don't know where to turn. The launchd daemon itself has no configurable options.

And note: Even if you figure out how to manipulate the job allocation size I recommend you approach it gingerly. launchd is what makes your system run.

Here's the source code for launched on 10.5.8 launchd-258.25

It's open source, see how Program Arguments is being allocated.

To find the particular limit, you'll want to read the source code behind the execvp system call which is what launchd uses to get the processes going, and also have a look at the source code to launchd to see if there are storage limits there. That latter code is located here:

http://www.opensource.apple.com/source/launchd/launchd-258.22/launchd/src/

I've spent a half-hour or so digging around in the launchctl source code and don't see anything immediately obvious, but don't immediately see the limit, and lack the time to walk the entire code path from the database entry through to the process launch. I've only made a cursory look at the execvp and execve calls.

Alternatively, log a bug report with Apple and request that the limit be better documented.

In the interim and pending any response from Apple, you already know that this approach doesn't work with these long command strings — digging out the details of the limits that you've already identified really doesn't really help after all, where using shorter commands will allow the environment to operate.

Alternatively (and probably more work than it's worth), create and use your own bash script launcher, and use that.

Ah, I took a tour of that source code myself, since you pointed it out, and I may have figured it out. in the section where (I'm guessing) it creates a new launchd job (job_new(), in core.c) it keeps filtering the char* that (I'm assuming) contains the string from the Program and/or ProgramArguments keys through the snprintf utility, and snprintf uses a pattern string that allows no more than 1000 characters. From what I'm reading on the snprintf man page the code was probably designed that way to prevent buffer overflow attacks, but the nitty-gritty of programming on this level are (frankly) over my head. But for what it's worth...

Actually I'm surprised your example runs at all, the reason being you've added the redirect symbol and the file to redirect to as part of the third argument to execvp

As a test try breaking out the redirect and the file as two separate arguments. That is add

<string>></string>

<string>/Users/neville/Desktop/tmp.txt</string>

(note note sure how you add the redirect symbol to the file do whatever has been working)

As for limits to the length of an argument to a program that is set by the OS, I'd be surprised if launched (or any program that execs other programs) would limit it. And the limit (even back in 10.5) was greater then what you are seeing.

See my last post, which suggests that the limit is hardcoded.

Personally I'd rather not have my OS (and the OS of everyone who owns a mac, anywhere) to slow down because you want Apple to code in error messages on things they could not reasonably expect you'd be trying to do. If you want to work outside the system for your own personal reasons that's fine, but accept the consequences of that independence, and don't expect the rest of us to pay for your peculiarities.

So there! 😝 👿 😎