No Webservices starting on OSX Server 5.0.15

Question

Level 1

9 points

No Webservices starting on OSX Server 5.0.15

Hi @all,

i have a problem with my OSX Server 5.0.15. Today i've insalled the latest security patches for OSX 10.10 and i have to reboot the server. After the reboot no webservices on the server comes up. Not the normal Webserver and also not the Calender and the Contactsserver. I don't know how i can troubleshoot this issue. In lsof -i -n -P i see that nothing listens on port 80 or 443 but i see other ports listening by httpd:

sudo lsof -i -n -P |grep httpd

httpd 520 root 4u IPv4 0x447c87e2169cc417 0t0 TCP 127.0.0.1:35443 (LISTEN)

httpd 520 root 5u IPv4 0x447c87e2169cbb47 0t0 TCP 127.0.0.1:35480 (LISTEN)

httpd 531 _www 4u IPv4 0x447c87e2169cc417 0t0 TCP 127.0.0.1:35443 (LISTEN)

httpd 531 _www 5u IPv4 0x447c87e2169cbb47 0t0 TCP 127.0.0.1:35480 (LISTEN)

httpd 4355 root 4u IPv4 0x447c87e21904f9a7 0t0 TCP 127.0.0.1:35243 (LISTEN)

httpd 4355 root 5u IPv4 0x447c87e2150f0277 0t0 TCP 127.0.0.1:35280 (LISTEN)

httpd 4363 _www 4u IPv4 0x447c87e21904f9a7 0t0 TCP 127.0.0.1:35243 (LISTEN)

httpd 4363 _www 5u IPv4 0x447c87e2150f0277 0t0 TCP 127.0.0.1:35280 (LISTEN)

httpd 4367 root 4u IPv4 0x447c87e218e71417 0t0 TCP 127.0.0.1:34580 (LISTEN)

httpd 4367 root 5u IPv4 0x447c87e218e68417 0t0 TCP 127.0.0.1:34543 (LISTEN)

httpd 4380 _www 4u IPv4 0x447c87e218e71417 0t0 TCP 127.0.0.1:34580 (LISTEN)

httpd 4380 _www 5u IPv4 0x447c87e218e68417 0t0 TCP 127.0.0.1:34543 (LISTEN)

I've also resetted the hole webserver with the command serveradmin command web:command=restoreFactorySettings and deleted the /Library/Server/Config/apache2 directory but this also did not helped.

Could anyone help me troubleshoot this issue?

Thanks a lot

Regards

Manuel

Mac mini, OS X Server

Posted on Feb 9, 2016 11:00 AM

Reply

Answer 1

pterobyte

Level 6

11,098 points

Feb 12, 2016 1:08 AM in response to menzbua

Your best bet is to look at the Apache Error and Access logs. Those should contain clues. system.log might as well.

Next have a look at the output of:

sudo server-apachectl configtest

OS X Server 5 changes management of websites substantially under the hood. Sites are proxied through ports other than 80 (of course you should still see a listener on 80), so it is possible you have a problem with configuration files.

Reply

Answer 2

Feb 12, 2016 2:49 AM in response to menzbua

Close server App, go to \library\server and either remove, or rename the "web server" folder. Open server app and it will rebuild the web server side of things.

Sometimes perms can get messed up causing this (have you tried disk repair?)

I moved my service data and had as similar problem which was fixed by removing the folder.

Reply

Answer 3

menzbua Author

Level 1

9 points

Feb 12, 2016 6:40 AM in response to pterobyte

In the config seems to be a problem:

sudo server-apachectl configtest

[Fri Feb 12 15:37:29.485511 2016] [core:warn] [pid 35538] AH00111: Config variable ${SERVER_INSTALL_PATH_PREFIX} is not defined

httpd: Syntax error on line 190 of /Library/Server/Web/Config/apache2/httpd_server_app.conf: Cannot load ${SERVER_INSTALL_PATH_PREFIX}/usr/libexec/apache2/mod_auth_digest_apple.so into server: dlopen(/usr/${SERVER_INSTALL_PATH_PREFIX}/usr/libexec/apache2/mod_auth_digest_a pple.so, 10): image not found

I've checked the /System/Library/LaunchDaemons/org.apache.httpd.plist file and there the SERVER_INSTALL_PATH_PREFIX variable is set

<key>SERVER_INSTALL_PATH_PREFIX</key>

<string>/Applications/Server.app/Contents/ServerRoot</string>

I've also tried to delte the /Library/Server/Web directory. But it also seems not to work.

I've setup a fresh server in VM and copied over the Web config dir. But this also not helps.

sudo lsof -i -n -P |grep 443

java 185 root 52u IPv4 0x447c87e21368af37 0t0 TCP 127.0.0.1:44331 (LISTEN)

httpd 520 root 4u IPv4 0x447c87e2169cc417 0t0 TCP 127.0.0.1:35443 (LISTEN)

httpd 531 _www 4u IPv4 0x447c87e2169cc417 0t0 TCP 127.0.0.1:35443 (LISTEN)

Thanks a lot

Regards

Manuel

Reply

Answer 4

pterobyte

Level 6

11,098 points

Feb 12, 2016 6:50 AM in response to menzbua

That error can be ignored.

What do the logs say?

Reply

Answer 5

menzbua Author

Level 1

9 points

Feb 12, 2016 7:04 AM in response to pterobyte

A grep for apache in /var/log/system.log show the message

AH00526: Syntax error on line 14 of /Library/Server/Web/Config/apache2/sites/0000_127.0.0.1_34543_.conf:

When i look into the conf file there is the Certificte File configured

SSLCertificateFile "/etc/cert...."

The file shown in the config exists.

Reply

Answer 6

pterobyte

Level 6

11,098 points

Feb 12, 2016 7:09 AM in response to menzbua

What do the Apache logs say?

They are in

/private/var/log/apache2

In particular look at

access_log
error_log
service_proxy_access.log
service_proxy_access.log

If you have configured your sites to use different logs, check those too.

You may also want to restart web services and look at the logs while services start.

Reply

Answer 7

menzbua Author

Level 1

9 points

Feb 12, 2016 7:25 AM in response to pterobyte

I've rested the Webservices many times today. So there should be logs i think.

in the access_log, service_proxy_access.log and service_proxy_error.log are no actual log entries

tail error_log

[Tue Feb 09 19:59:10.970369 2016] [mpm_prefork:notice] [pid 4676] AH00163: Apache/2.4.16 (Unix) OpenSSL/0.9.8zg configured -- resuming normal operations

[Tue Feb 09 19:59:10.970715 2016] [core:notice] [pid 4676] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND -f /Library/Server/Web/Config/apache2/httpd_server_app.conf -E /var/log/apache2/error_log -D WEBSERVICE_ON'

[Fri Feb 12 15:38:01.395608 2016] [mpm_prefork:notice] [pid 4676] AH00169: caught SIGTERM, shutting down

Syntax OK

[Fri Feb 12 15:43:09.312081 2016] [mpm_prefork:notice] [pid 35866] AH00163: Apache/2.4.16 (Unix) OpenSSL/0.9.8zg PHP/5.5.29 configured -- resuming normal operations

[Fri Feb 12 15:43:09.312191 2016] [core:notice] [pid 35866] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND -f /Library/Server/Web/Config/apache2/httpd_server_app.conf -E /var/log/apache2/error_log -D WEBSERVICE_ON'

[Fri Feb 12 15:43:25.407328 2016] [mpm_prefork:notice] [pid 35866] AH00169: caught SIGTERM, shutting down

[Fri Feb 12 15:43:30.864391 2016] [mpm_prefork:notice] [pid 35909] AH00163: Apache/2.4.16 (Unix) OpenSSL/0.9.8zg PHP/5.5.29 configured -- resuming normal operations

[Fri Feb 12 15:43:30.864511 2016] [core:notice] [pid 35909] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND -f /Library/Server/Web/Config/apache2/httpd_server_app.conf -E /var/log/apache2/error_log'

[Fri Feb 12 15:43:31.362518 2016] [mpm_prefork:notice] [pid 35909] AH00169: caught SIGTERM, shutting down

Reply

Answer 8

menzbua Author

Level 1

9 points

Feb 15, 2016 7:28 AM in response to menzbua

I've investigated the problem further. What i see now is when i have enabled the Webserver in the Server App i don't have the service proxies running. But the proxies are the daemons that are serving port 80 and 443. When i take a look on my fresh installed OSX Server vm i see that there is running the service proxys and the webservice httpd.

I tried to disable the Webserver in the Server App and start the service proxy manually. After starting it with sudo launchctl load /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.a pple.serviceproxy.plist i see in ps auxw that six proxy instances are running. But there is no webserver at the moment who serving the data. Now i want to start the Webserver in the Server App but after that the proxies are died and the listeners on 80 and 443 are gone too.

I think the problem must be somewhere between the proxy and the webserver config.

Reply

Answer 9

pterobyte

Level 6

11,098 points

Feb 16, 2016 2:31 AM in response to menzbua

El Capitan and Yosemite, both come with a standard Apache installation which works without proxying anything. As soon as you install Server.app and activate Web Services, the Apache instance in Server.app is used over the standard one. This is probably why you see the port discrepancies.

Unless you have lots of services configured, I'd probably delete Server.app and all server related directories and start from scratch.

Reply

Answer 10

Feb 16, 2016 4:32 AM in response to menzbua

Is this related to anything with the reverse proxy used in El Cap? Something to do with SSL 443 now being port 60443 and 80 being 6080 ?

Reply

Answer 11

Feb 16, 2016 7:38 AM in response to menzbua

menzbua wrote:

dlopen(/usr/${SERVER_INSTALL_PATH_PREFIX}/usr/libexec/apache2/mod_auth_digest_a pple.so, 10): image not found

There's whitespace in that path, if that's what you're actually getting, and not a cut-and-paste error here.

Reply

Answer 12

pterobyte

Level 6

11,098 points

Feb 16, 2016 7:44 AM in response to MrHoffman

I'd say that is just a cut and paste error. A fully functioning web server default installation under OS X Server 5 will through that error (no spaces)

Reply

Answer 13

menzbua Author

Level 1

9 points

Feb 16, 2016 8:01 AM in response to pterobyte

yes, it was a copy & paste error. There was a break in the Terminal window.

Deleting all my server settings would not be fine. There is running a Mailserver that i don't want to re-setup. It is easier for me to move the Weberver to another server but that couldn't be the solution, hopefully.

Reply

Answer 14

Feb 16, 2016 9:22 AM in response to menzbua

Disable HTTPS temporarily, and see if rebooting and restarting works. I've had that block Apache startup; when a certificate is incorrect or invalid or corrupted.

Also see what the Apache configtest tool self-test reports for the local files. It's rather more convoluted on recent OS X Server versions, unfortunately:

$ export SERVER_INSTALL_PATH_PREFIX=/Applications/Server.app/Contents/ServerRoot

$ apachectl -t -f /library/server/web/config/apache2/httpd_server_app.conf -D WEBSERVICE_ON

Syntax OK

$

Reply

Answer 15

menzbua Author

Level 1

9 points

Feb 16, 2016 10:27 AM in response to MrHoffman

frodo:~ manuelmenz$ export SERVER_INSTALL_PATH_PREFIX=/Applications/Server.app/Contents/ServerRoot

frodo:~ manuelmenz$ apachectl -t -f /library/server/web/config/apache2/httpd_server_app.conf -D WEBSERVICE_ON

Syntax OK

That looks good.

How can i disable the HTTPS Server?

Reply