Bash script running on my MBP 10.7.4 but not on other Macs (all 10.7.4)

Question

Level 2

371 points

Bash script running on my MBP 10.7.4 but not on other Macs (all 10.7.4)

A friend of mine wrote me this short script to run in terminal that downloads a particular file every n seconds from a password protectected FTP site. The script worked on his linux machine and on my Mac but not on my collegues Macs. I'm wondering if anybody knows reasons why it may be failing on other Macs.

We changing the first line from: #!/bin/sh to #!/bin/bash

and exceuting with

user_prompt_$ bash ./download_stats.sh

Another idea is to script cyberduck but it doesn't support applescript though may have a comand line interface (looking into that at present).

There error output from one of the accounts it fails on follows the script listing.

#!/bin/sh

#

# =================================================================

# Download XML_stats Bash Script

#

# Filename: download_stats.sh

# Arguments: input_filename - file to be downloaded from remote server. Optional.

# poll_interval - seconds delay between downloads. Optional (input_filename must be present)

# Use: Invoke from shell, e.g. ./download_stats.sh

# Author: Luke Neeson

# Date Created: 2012-05-31

# Comments: This script will download a given file, input_filename, at the given interval, poll_interval.

# Notes:

# - .netrc is required in the current directory and must contain the logon details for the fox server. This could be installed in the user home directory, if so, delete the line marked below.

# =================================================================

# If the .netrc is in the directory of this script, the following must be enabled. If the user's home .netrc is to be used, comment out the following line:

export HOME="`pwd`"

export input_filename="the_file_I_want_as_default.xml"

export poll_interval=10

if [ ! -z "$1" ] ; then

export input_filename="$1"

if [ ! -z "$2" ] ; then

export poll_interval="$2"

fi

while [ true ]

do

ftp access.foxsports.com.au << EOF 2>&1 | sed 's/.*AUTH GSSAPI.*//g' | sed 's/.*KERBEROS.*//g'

get "$input_filename"

quit

EOF

sleep $poll_interval

done

ERRORS RETURNED:

a-mbp:~ macca$

a-mbp:~ macca$ cd ~/stats/download_rugby_stats/

a-mbp:download_stats a$ sh download_rugby_stats.sh

: command not founds.sh: line 16:

: command not founds.sh: line 19:

: command not founds.sh: line 22:

download_stats.sh: line 39: syntax error: unexpected end of file

a-mbp:download_stats a$

Those are the blank lines and 39 is the last line of script.

MacBook Pro, Mac OS X (10.7.3), 10.7.4 actually Wacom 6x11

Posted on Jun 7, 2012 9:39 PM

Reply

Answer 1

Jun 8, 2012 3:08 AM in response to wideEyedPupil

If I have your script formatted correctly, there are 37 lines in the script. If there are any blank lines at the end of the script then remove them.

Reply

Answer 2

X423424X

Level 6

14,265 points

Jun 8, 2012 3:37 AM in response to Mark Jalbert

A blank line at the end of the script shouldn't matter nor should the other blank lines causing errors either. So I am assuming those blanks aren't really blanks. What editor did you use to create this file? Also when I cut/pasted the script into BBedit there does appear to be a blank at the beginning of those blank lines. I assume that the original character got lost in the posting and extracting from the post into my editor window.

So try to remove the character at the start of each of those "blank" lines.

You might also want to use TextWrangler as your text editor (BBEdit's little brother).

Reply

Answer 3

Jun 8, 2012 5:53 AM in response to X423424X

A blank line at the end of the script shouldn't matter nor should the other blank lines causing errors either.

Well, they should not matter. The idea was to have the OP look at the file in a text editor then operate on the file, saving it with unix line endings (hopefully).

What editor did you use to create this file?

I use vi (vim) and the file is double spaced when you copy it from the forum. I removed the double spacing. The "syntax error: unexpected end of file" error looks like it is coming from the while loop. There are parts of his script that aren't well formed such as:

if [ ! -z "$1" ] ; then
  export input_filename="$1"
  if [ ! -z "$2" ] ; then
    export poll_interval="$2"
  fi
fi

Reply

Answer 4

BobHarris

Level 9

53,003 points

Jun 8, 2012 6:20 AM in response to wideEyedPupil

I did a copy and paste into a text editor window. And when I ran the text file it did not get an error, and ftp prompted me for a password.

As others have suggested, I think you have a problem with the way the file is formatted.

I would suggest a) download TextWrangler (free) which is a very good text editor:

<http://www.macupdate.com/info.php/id/11009/textwrangler>

Copy and paste the script into TextWranger, then save the file.

Now try:

bash saved.file

Or change its permissions "chmod +x saved.file" and run it directly.

Reply

Answer 5

wideEyedPupil Author

Level 2

371 points

Jun 8, 2012 7:08 AM in response to wideEyedPupil

The original file should have had unix line endings since it was authored on a linux machine. When I editted the file to change the first line I editted it in TextWrangler. When I posted the script I openned it in XCode b/c it's contextual colouring copy/pastes into browsers.

Note the script runs on my Mac no problems with spaces and blank lines but I'm willing to try removing blank lines and saving with unix LFs.

The author thought maybe the errors indicated it wasn't finding the bash command or something… not sure what he meant and I thought maybe it had something to do with subtle differences b/w the UNIX install he was using and OS X flavored unix? Thoughts?

Oh we changed the permissions too to allow execution with:

chmod u+x download_stats.sh

Reply

Answer 6

wideEyedPupil Author

Level 2

371 points

Jun 8, 2012 7:09 AM in response to wideEyedPupil

How do I edit the OP, I want to edit a line I shouldn't have left as per original, got all the rest.

Reply

Answer 7

BobHarris

Level 9

53,003 points

Jun 8, 2012 7:35 AM in response to wideEyedPupil

If the file is good, then check the user's PATH and make sure all the commands used in the script can be found in the PATH

which ftp

which pwd

which true

which sed

which sleep

You can check to see that line endings are good using

cat -vte download_rugby_stats.sh

You might also want to check if the user has a $HOME/.netrc (or that you do) and if that is affecting the behavior of ftp (long shot and very unlikely).

Finally, make sure EOF is in column 1. No leading spaces or tabs in front of it.

Reply

Answer 8

BobHarris

Level 9

53,003 points

Jun 8, 2012 7:37 AM in response to wideEyedPupil

How do I edit the OP, I want to edit a line I shouldn't have left as per original, got all the rest.

You get about 15 minutes to edit a post, and then it is lock and the only way to make changes to it after that is through a moderator, and only tend to edit offensive material (if they don't just delete the post altogether 🙂).

Reply

Answer 9

wideEyedPupil Author

Level 2

371 points

Jun 8, 2012 7:44 AM in response to BobHarris

Thanks Bob, Luke mentioned which but didn't go into it.

cat -vte download_rugby_stats.sh ——> Does this look good to you?

# ================================================================= $

# Download Rugby Stats Bash Script$

#$

# Filename: download_rugby_stats.sh$

# Arguments: input_filename - file to be downloaded from remote server. Optional.$

# poll_interval - seconds delay between downloads. Optional (input_filename must be present)$

# Use: Invoke from shell, e.g. ./download_rugby_stats.sh$

# Author: Luke Neeson$

# Date Created: 2012-05-31$

# Comments: This script will download a given file, input_filename, at the given interval, poll_interval. $

# Notes: $

# - .netrc is required in the current directory and must contain the logon details for the fox server. This could be installed in the user home directory, if so, delete the line marked below.$

# ================================================================= $

$

# If the .netrc is in the directory of this script, the following must be enabled. If the user's home .netrc is to be used, comment out the following line:$

export HOME="`pwd`"$

$

export input_filename="Rugby_IRB20120101_Client.xml"$

export poll_interval=10$

$

if [ ! -z "$1" ] ; then$

export input_filename="$1"$

if [ ! -z "$2" ] ; then$

export poll_interval="$2"$

fi$

$

while [ true ]$

do $

ftp access.foxsports.com.au << EOF 2>&1 | sed 's/.*AUTH GSSAPI.*//g' | sed 's/.*KERBEROS.*//g'$

get "$input_filename"$

quit$

EOF$

sleep $poll_interval$

done$

$

Reply

Answer 10

wideEyedPupil Author

Level 2

371 points

Jun 8, 2012 7:49 AM in response to Mark Jalbert

I'm not sure how these conditionals get parsed but what exactly is poor form about them? Best-practice would be…

Reply

Answer 11

BobHarris

Level 9

53,003 points

Jun 8, 2012 8:07 AM in response to wideEyedPupil

cat -vte looks OK. The $ at the end of each line indicates <LF> terminated. If this had been <CR> terminated, it would have ended with ^M, and if it was DOS <CR><LF> terminated it would have been ^M$

I ONLY see $ indicating proper Unix line terminations. This is good.

You can easily look at your PATH using:

(IFS=:  ; for j in $PATH;do echo $j;done)
/usr/bin
/bin
/usr/sbin
/sbin
/usr/local/bin
/usr/X11/bin

The above PATH is a typical vanilla PATH. If the user has modified their PATH, then it can have a lot more in it.

The 'which' commands should tell you what versions of the programs were used. 'which' on my system found:

/usr/bin/ftp
/bin/pwd
/usr/bin/true
/usr/bin/sed
/bin/sleep

And again (long shot) check for a $HOME/.netrc on either their or your system that might change the ftp behavior

Reply

Answer 12

BobHarris

Level 9

53,003 points

Jun 8, 2012 8:12 AM in response to wideEyedPupil

I'm not sure how these conditionals get parsed but what exactly is poor form about them? Best-practice would be…

For what its worth, they look fine to me. I personally use slightly different style, but not so much different from what is being shown. And I have been writing shell scripts since '85 (lots of shell scripts).

Reply

Answer 13

wideEyedPupil Author

Level 2

371 points

Jun 8, 2012 11:48 PM in response to BobHarris

BobHarris wrote:

And again (long shot) check for a $HOME/.netrc on either their or your system that might change the ftp behavior

Is that just a matter of doing:

…user$ cd ~/

~ user$ ls -a

And looking for .netrc?

If that's what you do, I don't have it in ~/ directory I'l ask others to check.

My PATH

/opt/local/bin

/opt/local/sbin

/usr/bin

/bin

/usr/sbin

/sbin

/usr/local/bin

/usr/X11/bin

Reply

Answer 14

BobHarris

Level 9

53,003 points

Jun 9, 2012 5:58 AM in response to wideEyedPupil

/opt/local/bin and /opt/local/sbin are not normally in a PATH, which means this user has a custom PATH.

I again suggest you use 'which' to find out what commands are being run

which ftp
which pwd
which true
which sed
which sleep

Here is what I get, and would be standard for Mac OS X

/usr/bin/ftp
/bin/pwd
/usr/bin/true
/usr/bin/sed
/bin/sleep

Having additional directories in PATH is not bad, but when having scripting problems, it is something to look at.

Reply

Answer 15

wideEyedPupil Author

Level 2

371 points

Jun 9, 2012 7:03 PM in response to BobHarris

The added paths were me not him. We did the which commands earlier and all std responces.

Client found a solution he is using FTP Scheduler which downloads every few seconds, and the Pro (paid) Version can be scheduled to do DL/UL at required set time(s). It's quite cusatomisable infact for wildcard file lists ASCII/binary and so on.

Thanks for the help and explainanations, Bob!

Reply