1 2 3 Previous Next 34 Replies Latest reply: May 7, 2012 6:36 PM by Jonathan Pool Go to original post
  • 15. Re: launchd disobeying plist
    etresoft Level 7 Level 7 (24,270 points)

    Both launchd and rsync are really meant for software developers. Documentation is always poor in this context. It is expected that developers will take a systematic approach to learning how the software works and what the documentation means. Furthermore, rsync is much more complex than launchd. You are using both. That's just painful.

     

    I suggest looking for a 3rd party, commercial solution. In addition to all of the above, rsync is "free software". Neither support nor documentation is provided. You are expected to exercise your freedom and hack up rsync as you wish. Because of this, no software developer has any incentive to make rsync easier to use like the developer of Lingon has make launchd easier to use.

  • 16. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Thanks for the comments.

     

    The script is:

     

    /usr/local/bin/rsync --recursive --update --executability --times --omit-dir-times --delete --delete-excluded --compress-level=9 --exclude=.* --8-bit-output --progress /Topics/panlex/panlex-dics/panlex-dics-data/used ec2-user@panlex.net:/var/www/html/panlex/sources

     

    /usr/local/bin/rsync --recursive --update --executability --times --omit-dir-times --delete --delete-excluded --compress-level=9 --exclude=.* --8-bit-output --progress /Topics/panlex/panlex-dics/panlex-dics-data/panlex-dics-data-todo/queued ec2-user@panlex.net:/var/www/html/panlex/sources

     

    As for your statement that "what you quoted about QueueDirectories says explicitly that the job will only launch if the directory is empty", I once again reply that this tells me I cannot understand the launchd documentation. When I read "the job will only be started if the path is a directory and the directory is not empty", I don't understand "the job will only launch if the directory is empty". Likewise, nothing in the QD documentation makes me understand that "if something is added to the directory, it must be removed before the trigger will fire again". Instead, what I understand is that if anything in the directory is added or subtracted so that the directory's modification date is updated then the job will run. If I need to study something as a prerequisite to correcting my understanding, I would appreciate being advised as to what I need to study.

     

    I'm sorry that you misinterpret my motive as spreading misinformation. My motive is to understand whether I (and persons with similar capabilities and needs to mine) can apply launchd to problems by reading and obeying the instructions for its use.

  • 17. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Thanks for the cautions about how the real world works.

     

    As you sugggest, I'm used to starting with the man pages and experimenting until things work right. That's what happened with rsync and with cron. When launchd brought me to a higher level of confusion that experimenting didn't resolve, and I couldn't find solutions in public discussions, I decided to ask here.

     

    As for commercial products, it looks as if iCal can run a script every day at a particular time. I'll look at that option, too. Thanks for the suggestion.

  • 18. Re: launchd disobeying plist
    twtwtw Level 5 Level 5 (4,690 points)

    I'll have to check the rsync man page and parse through your commands (I know they work, but I want to know precisely what they are doing).  I'll get back on that a bit later.  I assume you're using the default rsync (v. 2.6.9) and that your script starts with a bash shebang and has its executable bit set.

     

    as to the rest, launchd is no more (or less) difficult to use than cron.  it's just a different paradigm.  you seem to know how to use cron, so you are certainly capable of using launchd, and it's just a question of getting your head out of the old box and into the new.  it might help to understand the structure.  launchd has three basic elements: plist files (xml-style configuration files), jobs (things entered into launchd, usually from plist files), and utilities (utilities are scripts launched by jobs when the specific criteria laid out in plist files are met).  launchd takes care of starting and killing utilities (and their subprocesses) according to its job list.

     

    so a QueueDirectory job needs three criteria to be met in order to launch its associated utility: that the directory exist, that it be empty, and that a change occurs.  the only way those criteria can all be satisfied is by adding files to an empty directory, so that is the only time a QD job will fire.  It's assumed one can make the logical deduction that a non-empty directory must be emptied of any contents before the job will be able to fire the utility again.

  • 19. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Thanks for the explanations.

     

    I was using the built-in rsync, but one rsync guru urged users to upgrade, so I'm running 3.0.9 in this script.

     

    Assuming your description of QD is correct, then reading the man page I had no idea this is what was intended. If you are correct, then to understand this I would want the man page to be modified, such as:

     

    FROM:

     

    WatchPaths <array of strings>

         This optional key causes the job to be started if any one of the listed paths are modified.

     

    QueueDirectories <array of strings>

         Much like the WatchPaths option, this key will watch the paths for modifications. The difference being

         that the job will only be started if the path is a directory and the directory is not empty.

     

    TO:

     

    WatchPaths <array of strings>

         This optional key causes the job to be started if any one of the listed paths are modified.

     

    QueueDirectories <array of strings>

         Much like the WatchPaths option, this key will watch the paths for modifications. The difference is

         that the job will be started only if the path is a directory and the directory changes from being empty to

         being non-empty.

  • 20. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    I see that iCal runs only AppleScripts. To run a shell script, I seem to need iCal + Automator.

  • 21. Re: launchd disobeying plist
    Mark Jalbert Level 5 Level 5 (4,475 points)

    This is a simple example that I think will help you out. It writes the date and time to a log file every time a file is added to 2 directories or when a filesystem is mounted.

     

    <?xml version="1.0" encoding="UTF-8"?>
    
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    
    <plist version="1.0">
    
    <dict>
    
         <key>Label</key>
         <string>com.example.WPtest</string>
    
         <key>ProgramArguments</key>
         <array>
         <string>/usr/local/bin/wp.ksh</string>
         </array>
    
         <key>StartOnMount</key>
         <true/>
    
         <key>WatchPaths</key>
          <array>
         <string>/tmp/wptest</string>
         <string>/tmp/wptest2</string>
         </array>
    
    </dict>
    
    </plist>
    

    and the shell script:

     

    #!/usr/bin/env ksh
    
    date -j >> /tmp/logfile
    
  • 22. Re: launchd disobeying plist
    etresoft Level 7 Level 7 (24,270 points)

    A noble effort, but the original poster wants to run rsync in the script.

     

    Can you update the script so it has the ability to modify the directories it watches

  • 23. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Thank you for this example. This could be a basis on which to build.

     

    One issue is that the directories in my cases contain thousands of subdirectories, subsubdirectories, etc., and I would want this to run whenever any of the directories at any level below the head is modified. Unless launchd were to offer a recursive WatchPaths key, I would need to run the script periodically anyway. So I could do that, assuming that AbandonProcessGroup lets rsync complete execution and prevents launchd from respawning the job.

     

    I'll make a note of your and others' advice for future applications of launchd.

  • 24. Re: launchd disobeying plist
    Mark Jalbert Level 5 Level 5 (4,475 points)

    I'm not doing all the work..:-)

  • 25. Re: launchd disobeying plist
    Mark Jalbert Level 5 Level 5 (4,475 points)

    Stick with cron. It's portable and is working for you.

  • 26. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Well, if launchd's WatchPaths routine looks at the path directory's modification date and the script can read the modification date and then reset it to the original value at the end of its execution, wouldn't that work?

  • 27. Re: launchd disobeying plist
    twtwtw Level 5 Level 5 (4,690 points)

    My 2¢, I think you need to give up on the idea of continuous real-time synchronization.  no sync or backup solution I know of does that; ey all work on demand or on a periodic basis. 

  • 28. Re: launchd disobeying plist
    etresoft Level 7 Level 7 (24,270 points)

    Mark Jalbert wrote:

     

    Stick with cron. It's portable and is working for you.

    I think I'm going to have to agree. Cron is a crude, overused tool, but that doesn't mean it isn't useful.

     

    The problem here is one of recursion. The script would have to immediately load its launchd configuration, run rsync, and then reload the launchd configuration. That is a mess and bound to be a disaster. There are a number of tools that perform folder syncing without so much effort.

  • 29. Re: launchd disobeying plist
    Jonathan Pool Level 1 Level 1 (40 points)

    Thanks to etresoft, twtwtw, and Mark Jalbert for the additional helpful comments.