Apache Enable Apache shebang python path

Question

Gnarlodious Author

Level 4

3,313 points

Apache Enable Apache shebang python path

I am trying to update all my Python scripts to use the canonical shebang line for compatibility with Linux:

#!/usr/bin/env python

The problem is it doesn't work in Apache. The old shebang line works in Apache, which is

#!/usr/local/bin/python

The scripts run normally in Terminal, finding the Python executable specified by /usr/bin/env. But in Apache, the error I get is:

[cgid:error]~ End of script output before headers:

Why would Apache not find Python? Is there a diagnostic for this? I searched for hours looking for an explanation but found none. I am running OSX 10.13.3, Apache/2.4.29 and Python 3.6.4. Thank you for the help.

MacBook Pro with Retina display, macOS High Sierra (10.13.2), Thunderbolt display, iPhone 6S

Posted on Feb 6, 2018 3:27 PM

Reply

Answer 1

Top-ranking reply

Gnarlodious Author

Level 4

3,313 points

Feb 7, 2018 9:37 AM in response to Gnarlodious

The problem is solved after spending many hours trying every solution I could find on the internet. Strangely, no page mentioned this solution, I had to figure it out on my own. I added this to my CGI script to get the actual PATH inside Apache:

print("\n %20s %s" % ("PATH",os.environ['PATH']))

It told me that the command search PATH is:

/usr/bin:/bin:/usr/sbin:/sbin

I then linked the python3 executable to inside /usr/bin:

sudo link /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 /usr/bin/python3

Now the shebang line works on both my OSX and Linux machine:

#!/usr/bin/env python3

There may be another solution that doesn't require a system specific hardlink. For starters, I did not know PATH is the canonical search path in Apache2, it may be that PYTHONPATH is obsolete. Also, the “sitecustomize.py” file has no effect on Python inside Apache2, maybe it is also deprecated.

I am sure there is a bug somewhere in here. Probably the failure of the Python installer to insert a hardlink into /usr/bin. I hope this helps someone else to save time.

Reply

Answer 2

VikingOSX

Level 10

123,296 points

Feb 6, 2018 7:41 PM in response to Gnarlodious

For Python 3.6.n and #!/usr/bin/env python in scripts:

export PYTHONPATH=/Library/Frameworks/Python.framework/Versions/3.6/lib

For macOS Python 2.7.10 and #/usr/bin/env python:

unset PYTHONPATH

Reply

Answer 3

VikingOSX

Level 10

123,296 points

Feb 6, 2018 4:13 PM in response to Gnarlodious

There is a PYTHONPATH environment variable that is unset for the default operating system installed python. When you install another version of python via a package installer, or build your own from sources, then is when you need to set PYTHONPATH to the specific library path used by the /usr/local/bin/python version. Once you do that, the #!/usr/bin/env python syntax will discover the correct python libraries and function as it should — unless Apache is still messing things up.

Reply

Answer 4

etresoft

Level 9

56,338 points

Feb 6, 2018 7:34 PM in response to Gnarlodious

Apache runs in a system context and doesn't know about your custom Python. There are any number of ways to hack it up, maybe via launchd or maybe in Apache. Isn't there a mod_python you could use?

Reply

Answer 5

Gnarlodious Author

Level 4

3,313 points

Feb 6, 2018 6:39 PM in response to VikingOSX

That doesn't work. I made a file at:

/Library/Frameworks/Python.framework/Versions/3.6/site-packages/sitecustomize.py

with text:

import sys

sys.path.append('/Library/Frameworks/Python.framework/Versions/3.6/bin')

I restarted Apache:

sudo launchctl unload /Library/LaunchDaemons/org.apache.httpd.plist

sudo launchctl load -w /Library/LaunchDaemons/org.apache.httpd.plist

but it still doesn't work.

However if I make the top line of my script:

#!/Library/Frameworks/Python.framework/Versions/3.6/bin/python3

suddenly the script runs!

I also tried some other things but none works.

Reply

Answer 6

Gnarlodious Author

Level 4

3,313 points

Feb 7, 2018 7:21 AM in response to VikingOSX

When I run my test script I get this in error.log:

env: python3: No such file or directory

This after adding

export PYTHONPATH=/Library/Frameworks/Python.framework/Versions/3.6/bin

to file

/Library/Frameworks/Python.framework/Versions/3.6/site-packages/sitecustomize.py

and restarting Apache.

Apparently Apache is looking for a variable named ‘python3’ and not searching folder

/Library/Frameworks/Python.framework/Versions/3.6/bin

The strange thing is I have searched Google up and down and nowhere tells specifically how to add an Apache variable. In fact, I can say:

/usr/bin/env

in Terminal and I get all the variables, but ‘python’ is not listed. Despite that, if I say

/usr/bin/env python

I get the python3 interactive session. So I don't understand what is going on.

Reply

Answer 7

VikingOSX

Level 10

123,296 points

Feb 7, 2018 9:25 AM in response to Gnarlodious

I installed Python 3.6.2 using Homebrew awhile ago. It does not install "python" anywhere — instead it installs python3, or other python3.* in:

/usr/local/bin

/Library/Frameworks/Python.framework/Versions/3.6/bin

A #!/usr/bin/env python script invocation will only find the System python (2.7.10) binary, regardless of the PYTHONPATH environment setting pointed at the /lib location of the second entry above.

Reply

Apache Enable Apache shebang python path

Similar questions