launchd fails to execute a perl command from a shell script

Question

launchd fails to execute a perl command from a shell script

Hello all,

i have a little script that runs rsync. I use perl to add a timestamp to each line of output. Basically it goes like this:

#!/bin/bash

rsync -avz $sourceFolder $destinationFolder 2>&1 | perl -e "while(<>){s/^/$(date "+%Y-%m-%d %H:%M:%S") [rsync_backup.daily] /g; print;}"

Invoking this script from terminal works perfectly! Invoking it via launchd however fails. There is no error thrown, it just seems to not proceed.

Any help is appreciated.

Cheers.

Posted on Feb 26, 2013 4:57 AM

Reply

Answer 1

Feb 26, 2013 6:54 AM in response to Alexander Harm

Add the full path to both the rsync and the perl command. The environment when run in launchd is severely limited. The PATH variable will not be set or if it is it will be limited.

How are sourcefolder and destinationfolder being set?

Reply

Answer 2

Feb 26, 2013 7:17 AM in response to Frank Caggiano

I did as suggested and changed perl to /usr/bin/perl. However, it makes no difference. Run via shell/terminal the timestamp is added to all rsync output, run via launchd, no timestamp is added.

#!/bin/bash

rsync -avz $sourceFolder $destinationFolder 2>&1 | /usr/bin/perl -e "while(<>){s/^/$(date "+%Y-%m-%d %H:%M:%S") [rsync_backup.daily] /g; print;}"

Reply

Answer 3

twtwtw

Level 5

4,936 points

Feb 26, 2013 7:29 AM in response to Alexander Harm

First, where did you put the launchd plist, in the user LaunchAgents folder or the local LaunchDaemons folder? if the former, you might be running into eprmissions problems; jobs from that folder will only run with user privileges. Second, does the perl script rely on being run from a particular directory? If so, you may need to set the WorkingDirectory key of the in the plist file. Again, launchd jobs are run in a shell that's been set up differently than the one you get when you run in Terminal, so you can't make the same assumptions that you do when running from terminal.

Reply

Answer 4

BobHarris

Level 9

53,339 points

Feb 26, 2013 7:46 AM in response to Alexander Harm

Change your shell script to do the following, so that you know what your environment is:

#!/bin/bash
( pwd
  id -a
  echo $0
  echo $#                         # number of command line arguments
  for a; do echo $a; done  # list of command line arguments
  /usr/bin/printenv
  type rsync
  type perl
) >/tmp/my.launchd.output.txt 2>&1

Then verify that the environment contains all the things you need to run your script.

Compare this against what the same commands generate during your interactive session.

Reply

Answer 5

Feb 26, 2013 8:16 AM in response to Alexander Harm

seeing your plist file might also offer some clues.

Reply

Answer 6

Feb 26, 2013 9:19 AM in response to BobHarris

This is the output from the shell:

/Users/sadmin/Library/rsync_backup

uid=501(sadmin) gid=20(staff) groups=20(staff),501(com.apple.local.ard_admin),256(com.apple.access_vpn),403(c om.apple.sharepoint.group.2),204(_developer),100(_lpoperator),98(_lpadmin),81(_a ppserveradm),80(admin),79(_appserverusr),61(localaccounts),12(everyone),406(com. apple.sharepoint.group.5),502(com.apple.local.ard_reports),253(com.apple.access_ chat),252(com.apple.access_addressbook),401(com.apple.access_screensharing),504( com.apple.local.ard_manage),503(com.apple.local.ard_interact),250(com.apple.acce ss_afp),404(com.apple.sharepoint.group.3),500(com.apple.access_ssh),257(com.appl e.access_backup),254(com.apple.access_calendar),402(com.apple.sharepoint.group.1 ),255(com.apple.access_mail),251(com.apple.access_smb),405(com.apple.sharepoint. group.4),407(com.apple.sharepoint.group.6)

de.open-service.rsync_backup.daily.sh

0

TERM_PROGRAM=Apple_Terminal

SHELL=/bin/bash

TERM=xterm-color

TMPDIR=/var/folders/A5/A5M1umP6EsKqyAig3q3Bak+++TI/-Tmp-/

Apple_PubSub_Socket_Render=/tmp/launch-VgtdY6/Render

TERM_PROGRAM_VERSION=273.1

USER=sadmin

COMMAND_MODE=unix2003

SSH_AUTH_SOCK=/tmp/launch-4Jjpml/Listeners

__CF_USER_TEXT_ENCODING=0x1F5:0:3

PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin :/usr/X11/bin

PWD=/Users/sadmin/Library/rsync_backup

LANG=de_DE.UTF-8

HOME=/Users/sadmin

SHLVL=2

LOGNAME=sadmin

DISPLAY=/tmp/launch-219jsQ/org.x:0

_=/usr/bin/printenv

rsync is /opt/local/bin/rsync

perl is /usr/bin/perl

This is the output via launchd:

/

uid=501(sadmin) gid=20(staff) groups=20(staff),501(com.apple.local.ard_admin),256(com.apple.access_vpn),403(c om.apple.sharepoint.group.2),204(_developer),100(_lpoperator),98(_lpadmin),81(_a ppserveradm),80(admin),79(_appserverusr),61(localaccounts),12(everyone),406(com. apple.sharepoint.group.5),502(com.apple.local.ard_reports),253(com.apple.access_ chat),252(com.apple.access_addressbook),401(com.apple.access_screensharing),504( com.apple.local.ard_manage),503(com.apple.local.ard_interact),250(com.apple.acce ss_afp),404(com.apple.sharepoint.group.3),500(com.apple.access_ssh),257(com.appl e.access_backup),254(com.apple.access_calendar),402(com.apple.sharepoint.group.1 ),255(com.apple.access_mail),251(com.apple.access_smb),405(com.apple.sharepoint. group.4),407(com.apple.sharepoint.group.6)

/Users/sadmin/Library/rsync_backup/de.open-service.rsync_backup.daily.sh

0

SHELL=/bin/bash

TMPDIR=/var/folders/A5/A5M1umP6EsKqyAig3q3Bak+++TI/-Tmp-/

USER=sadmin

PATH=/usr/bin:/bin:/usr/sbin:/sbin

PWD=/

SHLVL=1

HOME=/Users/sadmin

LOGNAME=sadmin

_=/usr/bin/printenv

rsync is /usr/bin/rsync

perl is /usr/bin/perl

Reply

Answer 7

Feb 26, 2013 9:21 AM in response to Frank Caggiano

The script is located here:

/Library/LaunchDaemons/de.open-service.rsync_backup.daily.plist

It looks like this:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>Label</key>

<string>de.open-service.rsync_backup.daily</string>

<key>UserName</key>

<string>sadmin</string>

<key>ProgramArguments</key>

<array>

<string>/bin/sh</string>

<string>/Users/sadmin/Library/rsync_backup/de.open-service. rsync_backup.daily.sh</string>

</array>

<key>StandardOutPath</key>

<string>/Users/sadmin/Library/Logs/de.open-service.rsync_backup. log</string>

<key>StandardErrorPath</key>

<string>/Users/sadmin/Library/Logs/de.open-service.rsync_backup. log</string>

<key>StartCalendarInterval</key>

<dict>

<key>Hour</key>

<integer>18</integer>

<key>Minute</key>

<integer>50</integer>

</dict>

<key>AbandonProcessGroup</key>

<true/>

</dict>

</plist>

Reply

Answer 8

Feb 26, 2013 9:27 AM in response to Alexander Harm

Don't know if it is significant but your using two differnt rsync's. The launchd one i the systems in /usr/bin

When you run from the command line you're using one from /opt/local/bin

Where the one in opt come from?

Reply

Answer 9

BobHarris

Level 9

53,339 points

Feb 26, 2013 10:21 AM in response to Alexander Harm

rsync -avz $sourceFolder $destinationFolder

Stupid question 1: I'm assuming that your script is actually longer and defines $sourceFolder and $destinationFolder, or that you removed the actual paths from your posting and just substituted the $variables?

Stupid question 2: Does the rsync work?

Stupid question 2.1 Does the rsync work without the perl command?

Chances are you are just going to say "Of Course", but since the obvious is not telling us anything, I figured we should make sure we are not ASSUMING anything.

Reply

Answer 10

Feb 26, 2013 11:24 AM in response to BobHarris

Correct. The script is indeed longer but doesn't do much more than defining the variables and printing some information to the log file.

Actually I do call rsync from its full path /opt/local/bin/rsync and it does work (I checked the versions and all and it uses rsync 3.0.9 from MacPorts). It also works from launchd, however it seems that the perl command is not executed and it just prints to my log file (via STDOUT defined in the plist) without the timestamp.

Seems strange to me that it just ignores the perl command, not even throwing an error.

Reply

Answer 11

BobHarris

Level 9

53,339 points

Feb 26, 2013 11:49 AM in response to Alexander Harm

rsync -avz $sourceFolder $destinationFolder 2>&1 | /usr/bin/perl -e "while(<>){s/^/$(date "+%Y-%m-%d %H:%M:%S") [rsync_backup.daily] /g; print;}"

Correct me if I'm wrong, but the $(date "+%Y-%m-%d %H:%M:%S") expression is being processed by the shell, so that perl sees

while(<>){s/^/2013-02-26 14:42:39 [rsync_backup.daily] /g; print;}

I do not know if this means anything or not, but I'm trying to break down the code so I understand all the components.

I have another idea, which is not solving your problem, but is changing the approach away from perl

rsync -avz $sourceFolder $destinationFolder 2>&1 | while read line
do
    date "+%Y-%m-%d %H:%M:%S [rsync_backup.daily] ${line}"
done

Reply

Answer 12

Feb 26, 2013 11:59 AM in response to BobHarris

Fantastic! That gets the job done. Thanks a lot.

Reply

Answer 13

Feb 26, 2013 12:04 PM in response to Alexander Harm

Try a test run where you change the perl command to something simple like

/usr/bin/perl -e "print 'hello'" or some such.

Just something simple to see if the perl is running and getting messed up by the script or if it is not running at all.

Reply

Answer 14

Feb 26, 2013 1:16 PM in response to Frank Caggiano

Works in both cases:

perl -e "print 'hello'"

/usr/bin/perl -e "print 'hello'"

I guess the problem lies in passing the "date" part that is actually processed by the shell itself. I will test this tomorrow and come back.

Reply

Answer 15

Mar 1, 2013 11:54 PM in response to Frank Caggiano

Hello Frank,

played around with it a little bit, but it simply doesn't work. However, BobHarris' solution works fine for me. Thanks a lot for your help.

Cheers.

Reply