launchd ignores certain plist keys

<key>label</key>

<key>Label</key>

Linc Davis wrote:
I was able to reproduce your problem with a user launch agent that ran at 60-second intervals.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
 <dict>
 <key>Label</key>
 <string>com.etresoft.ticker</string>
 <key>ProgramArguments</key>
 <array>
 <string>/Users/jdaniel/bin/launchd_test.sh</string>
 </array>
 <key>RunAtLoad</key>
 <true/>
 <key>StartInterval</key>
 <integer>60</integer>
 <key>StandardErrorPath</key>
 <string>/tmp/launchd_test.err</string>
 <key>StandardOutPath</key>
 <string>/tmp/launchd_test.out</string>
 </dict>
</plist>

Thanks for the solution. I run the devdaily.com website, and just ran into the same problem on my Mac OS X 10.6.7 system. I probably tested the code shown above on a Mac OS X 10.5.x system, but I really don't remember. Either way, removing OnDemand fixes the problem.

(According to the launchd.plist man page, I don't think this should have been a problem, but it obviously is. I also see it has been replaced by the KeepAlive option.)

FWIW, setting ThrottleInterval to the same time as StartInterval actually "fixed" my problem when I still had the OnDemand statement in the file, prior to finding this thread.

Thanks again for the solution.

OnDemand=false (in 10.5/10.6) most likely translates to KeepAlive=true, which means that your job was trying to keep the script running constantly. since the script is just a one-liner, this means that the job is constantly launching the script (which does its thing and dies), throttling itself for the ThrottleInterval period, and then relaunching the script. Since the job was constantly trying to launch the script, it would naturally ignore the StartInterval key (it's senseless to try to start an already-running or throttled job).

You should only use KeepAlive/OnDemand for daemons/agents/processes that are expected to run continuously in the background - it's mostly a failsafe to ensure that a process doesn't die.

Even on Mountain Lion, StartInterval appears to be simply broken. My process takes >30 sec to run. I tried with and without both OnDemand=true and KeepAlive=false, and it would be (almost) immediately restarted each time it exited. Here's the plist file, BTW:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>UserName</key> <string>_news</string>

<key>Label</key> <string>mac.texpire</string>

<key>StartInterval</key> <integer>86400</integer>

<key>WorkingDirectory</key> <string>/opt/local/var/spool/news</string>

<key>ProgramArguments</key>

<array>

<string>/opt/local/sbin/texpire</string>

</array>

<!-- Don't run unless the news spool image is mounted -->

<key>QueueDirectories</key>

<array>

<string>/opt/local/var/spool/news/leaf.node</string>

</array>

</dict>

</plist>

The only thing I can find that keeps it from re-running the job continuously is to use ThrottleInterval instead of StartInterval

I sincerely doubt StartInterval is broken; I use it regularly. Make absolutely sure you haven't made mistakes in your code before assuming it's a system bug.

With that in mind...

This is not the way QueueDirectories works. QD fires a process when something is added to a directory, and disables itself until the directory is completely empty, that's all. You can't use it to check whether an image is mounted.

Also, unless you've changed permissions, /opt/local/sbin is owned by root/wheel - standard users might not have execute privileges in that directory, meaning you'd have to run the launchd job from /Libray/LaunchAgents or /Library/LaunchDaemons.

Do not use OnDemand unless you're running 10.4 Do not use KeepAlive unless you have an process that needs to run continuously. For tasks that only need to run periodically, leave launchd to decide when to start and stop them.

My advice to you is to use this plist, and put it in /Library/LaunchDaemons so it runs as root.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>UserName</key>

<string>_news</string>

<key>Label</key>

<string>mac.texpire</string>

<key>StartInterval</key>

<integer>86400</integer>

<key>Program</key>

<string>/Users/yourname/some_script</string>

</dict>

</plist>

write some_script to check whether the correct volume is mounted and run texpire; you'll get more control that way than trying to encapsulate it all in the plist file.

twtwtw wrote:

I sincerely doubt StartInterval is broken; I use it regularly. Make absolutely sure you haven't made mistakes in your code before assuming it's a system bug.

Well, naturally I did try to make absolutely sure of that, and my experimentation was extensive. To prove I wasn't insane I created this launchd testbench. It revealed that StartInterval does indeed work, though if the job is started later (e.g. by StartOnMount) it may appear to run again too soon, the first time around. It's easy to be misled into thinking StartInterval repeats too quickly if you don't wait through at least two complete runs.

twtwtw wrote:

This is not the way QueueDirectories works. QD fires a process when something is added to a directory, and disables itself until the directory is completely empty, that's all. You can't use it to check whether an image is mounted.

I can if I know the directory isn't present until the image is mounted. My testbench proves that much (see for yourself).

twtwtw wrote:

Also, unless you've changed permissions, /opt/local/sbin is owned by root/wheel - standard users might not have execute privileges in that directory, meaning you'd have to run the launchd job from /Libray/LaunchAgents or /Library/LaunchDaemons.

Considering I'm setting UserName=_news, I have to run it from there anyway. I'm well aware of the permissions issues, thanks. 😉

twtwtw wrote:

Do not use OnDemand unless you're running 10.4 Do not use KeepAlive unless you have an process that needs to run continuously.

The only reasons I dabbled with those was to prove to myself that they weren't relevant to my case. That seemed necessary, given the number of places I've read posts that say they will fix issues like the ones I was having. Note also that I was using KeepAlive=false, which is presumably the default, so it shouldn't affect anything.

twtwtw wrote:

For tasks that only need to run periodically, leave launchd to decide when to start and stop them.

It's not entirely clear what you mean by that. Every plist file contains some kind of specification (even if only implicit) about when jobs should be started or stopped.

twtwtw wrote:

My advice to you is to use this plist, and put it in /Library/LaunchDaemons so it runs as root.

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>UserName</key>

<string>_news</string>

<key>Label</key>

<string>mac.texpire</string>

<key>StartInterval</key>

<integer>86400</integer>

<key>Program</key>

<string>/Users/yourname/some_script</string>

</dict>

</plist>

write some_script to check whether the correct volume is mounted and run texpire; you'll get more control that way than trying to encapsulate it all in the plist file.

It's probably overly cute, but I am trying to prevent launchd from even launching a process for my job if it shouldn't run. QueueDirectories seems to do that.

bewst wrote:

twtwtw wrote:

You can't use [QD] to check whether an image is mounted.

I can if I know the directory isn't present until the image is mounted. My testbench proves that much (see for yourself).

In my testing (which I did back in 10.6, so things may have changed - I'll test again), if you try to load a QueueDirectories job that points to a non-existent directory path, the job disables itself and needs to be restarted explicitly. It's possible that if you load the job with an existing directory, then remove the directory, the job will continue to work when the directory becomes available again, but to my mind that falls into the 'questionable hack' category of programming. If I were going to do a launchd QD on a folder attached to a removable file system, I'd make it a two-stage process: have one job that watches for the file system to mount (using the StartOnMount key) and loads the second job that watches the folder. That way you're not depending on side effects to get main effects.

Also, as an aside, QDs are annoyingly touchy. It took me forever to realize that QDs will not reactivate even if there are invisible files present. So, common fubar scenario: a QD watched folder gets opened in the Finder, a .DS_Store file is created, the QD script does not dispose of it: QD no longer fires. Argh.

bewst wrote:

Considering I'm setting UserName=_news, I have to run it from there anyway. I'm well aware of the permissions issues, thanks. 😉

The UserName key is only applicable when you are running the job as root, meaning it must be run as a LaunchDaemon, not a LaunchAgent. Which, of course, I assume is what you're doing. 😝

bewst wrote:

twtwtw wrote:

For tasks that only need to run periodically, leave launchd to decide when to start and stop them.

It's not entirely clear what you mean by that. Every plist file contains some kind of specification (even if only implicit) about when jobs should be started or stopped.

Yes, but my point was that you should leave launchd to determine when those conditions are met. I've never had much success trying to co-opt a launchd trigger to do something it wasn't specifically designed to do, the way you're trying to co-opt the QD key. As best I can figure, launchd is simple-minded (designed to be robust and quick rather than smart and flexible) so it doesn't deal particularly well with novel approaches.

bewst wrote:

It's probably overly cute, but I am trying to prevent launchd from even launching a process for my job if it shouldn't run. QueueDirectories seems to do that.

Yeah, I figured; I frequently have that same kind of urge myself, and constantly have to fight against it. Meditate on the fact that you're wasting of a heck of a lot of human-scale time in an effort to save yourself a tidbit of computer-scale resources that you simply will not ever miss. Or at least, if your machine is running so tight that launching a momentary process actually creates a noticeable disruption in anything, you need a new machine. I'm a great fan of elegant code, yes, but I make sure it stops well short of OCD.

In my testing (which I did back in 10.6, so things may have changed - I’ll test again), if you try to load a QueueDirectories job that points to a non-existent directory path, the job disables itself and needs to be restarted explicitly.

Again, try my testbench script for yourself. That’s not what it does.

If I were going to do a launchd QD on a folder attached to a removable file system, I’d make it a two-stage process: have one job that watches for the file system to mount (using the StartOnMount key) and loads the second job that watches the folder. That way you’re not depending on side effects to get main effects.

That is an excellent idea. I’ll try it.

Also, as an aside, QDs are annoyingly touchy. It took me forever to realize that QDs will not reactivate even if there are invisible files present. So, common fubar scenario: a QD watched folder gets opened in the Finder, a .DS_Store file is created, the QD script does not dispose of it: QD no longer fires. Argh.

That’s not what I’m seeing. As I wrote in this report,

If you set QueueDirectories and launchd sees that the directories are not empty after the job runs, it considers that to be a failure and runs the job again after the ThrottleInterval expires. Therefore, if you’re trying to set up a repeating job that runs only when a given directory exists, you can’t use QueueDirectories for that purpose.

bewst wrote:

Considering I’m setting UserName=_news, I have to run it from there anyway. I’m well aware of the permissions issues, thanks. 😉

The UserName key is only applicable when you are running the job as root, meaning it must be run as a LaunchDaemon, not a LaunchAgent. Which, of course, I assume is what you’re doing. 😝

That is what I’m doing now, but of course I had to discover that the hard way. I find Apple’s documentation on launchd woefully inadequate. If it ever explicitly says that jobs in /Libraries/LaunchAgents are not run as root, I never saw it (no, saying that the job runs “on behalf of the user” does not necessarily imply it will be run as that user).

bewst wrote:

twtwtw wrote:For tasks that only need to run periodically, leave launchd to decide when to start and stop them.

It’s not entirely clear what you mean by that. Every plist file contains +some+ kind of specification (even if only implicit) about when jobs should be started or stopped.

Yes, but my point was that you should leave launchd to determine when those conditions are met.

I still don’t understand the distinction you’re pointing at. I never try to tell launchd that the conditions for start/stop are or aren’t met, and I wouldn’t know how to do so. As far as I can tell, it’s okay to use the mechanisms that Apple describes in “man launchd.plist,” and that’s all I’m trying to do.

I’ve never had much success trying to co-opt a launchd trigger to do something it wasn’t specifically designed to do, the way you’re trying to co-opt the QD key.

Well, the problem is that the documentation of these things and how they interoperate is so vague that it’s not at all clear exactly what they were specifically designed to do. The best I’ve been able to do is black-box testing to discover what to expect.

As best I can figure, launchd is simple-minded (designed to be robust and quick rather than smart and flexible) so it doesn’t deal particularly well with novel approaches.

How to know if what I’m doing is novel? Just try, I guess, and see if it works :-(

bewst wrote:

It’s probably overly cute, but I am trying to prevent launchd from even launching a process for my job if it shouldn’t run. QueueDirectories seems to do that.

Yeah, I figured; I frequently have that same kind of urge myself, and constantly have to fight against it. Meditate on the fact that you’re wasting of a heck of a lot of +human-scale+ time in an effort to save yourself a tidbit of +computer-scale+ resources that you simply will not ever miss. Or at least, if your machine is running so tight that launching a momentary process actually creates a noticeable disruption in +anything+, you need a new machine. I’m a great fan of elegant code, yes, but I make sure it stops +well+ short of OCD.

Yeah, I’ll cop to being a little bit obsessive about battery life.

The problem isn't so much that the documentation is vague or incomplete, but that the people who wrote the docs are Apple software engineers who have a skewed understanding of what constitutes common knowledge and what needs to be explained. For instance, at the very bottom of man launchd.plist you'll find this:

~/Library/LaunchAgents Per-user agents provided by the user.

/Library/LaunchAgents Per-user agents provided by the adminis-

trator.

/Library/LaunchDaemons System-wide daemons provided by the admin-

istrator.

/System/Library/LaunchAgents Per-user agents provided by Mac OS X.

/System/Library/LaunchDaemons System-wide daemons provided by Mac OS X.

Which tells us that LaunchAgents run as users and LaunchDaemons run as root. Whoever wrote the man page apparently assumed that this was so obvious it only needed a footnote - go figure - but it is there. Have you seen the technote on agents and daemons and the daemons and services programming guide? A bit more low-level than you need, but possibly helpful for background info.

With respect to launchd and the distinction I'm making about novel approaches, my understanding of launchd is that it is narrowly and strictly designed to launch denoted processes when pre-defined conditions are met. Period. The plist defines conditions and constraints and names the process to be launched, and once the plist is loaded launchd checks periodically to see if any of the conditions it sets out are satisfied. If so, it starts the process. There are a number of conditions that can be used as triggers: startup, date, time interval, change to a file path, adding to an empty directory, file system mount, and for the adventurous certain system conditions (laid out in the KeepAlive key) and unix sockets. Note, as an example, that KeepAlive/true doesn't actually keep a process alive, rather it launches the process on the particular condition that the process is quit.

Keeping that in mind, my point was that you were trying to use the QueueDirectories trigger to capture an entirely different triggering condition (whether "the news spool image is mounted"). Your reasoning is sound - yes, it seems logical that the QD trigger would only work when the image was mounted - but unlike more elaborate programs I don't think launchd was designed to compensate for the vagaries of human reason. QD is designed to watch for conditions inside a directory, not for the presence of a directory or file system, so using it the way you did relies on a side effect of QD's (unknown) internal logic.

Put another way, you're effectively asking launchd to determine when a process should *not* be launched, using the absence of a file system as an implicit negative conditional. But launchd doesn't do that: it only determines when a process *should* be launched. To the human mind 'launch' and 'do not launch' are interchangeable inverses; the assumption that launchd thinks that way as well is questionable.

It seems as though launchd had some internal tweaking done in the 10.6 to 10.7 transition, maybe in the 10.8 as well. I'll have to look at your benchtest more closely. But I hope you see my point.

twtwtw wrote:

The problem isn't so much that the documentation is vague or incomplete, but that the people who wrote the docs are Apple software engineers who have a skewed understanding of what constitutes common knowledge and what needs to be explained.

No, that's the cause of the problem. The result is that the docs are vague and incomplete.

For instance, at the very bottom of man launchd.plist you'll find this:

~/Library/LaunchAgents Per-user agents provided by the user.

/Library/LaunchAgents Per-user agents provided by the adminis-

trator.

/Library/LaunchDaemons System-wide daemons provided by the admin-

istrator.

/System/Library/LaunchAgents Per-user agents provided by Mac OS X.

/System/Library/LaunchDaemons System-wide daemons provided by Mac OS X.

Which tells us that LaunchAgents run as users and LaunchDaemons run as root.

Disagreed. Neither who the agents are "provided by" nor whether they are "per-user" or "System-wide" definitively implies anything at all about the UID that actually runs the jobs.

Whoever wrote the man page apparently assumed that this was so obvious it only needed a footnote - go figure - but it is there. Have you seen the technote on agents and daemons and the daemons and services programming guide? A bit more low-level than you need, but possibly helpful for background info.

Actually, no, I hadn't seen that first one. At this point I'd appreciate Apple erring on the side of "more low-level than I need"

It seems as though launchd had some internal tweaking done in the 10.6 to 10.7 transition, maybe in the 10.8 as well. I'll have to look at your benchtest more closely. But I hope you see my point.

Mostly. It's easy to say, with 20/20 hindsight, that I was misusing the facilities of launchd. I still maintain that (modulo that technote that I haven't read yet) Apple makes it pretty darned hard to figure out what misuse is.