Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

MacOS Server 5.6.3 certificate auto renewal cause collaboration services disruption

Hello, on Friday September 4th 2020, in the heat of the office work, we started to get authentication errors for the email accounts, calendars, the wikis and the web services became unaccessible on http and https, internally and externally.


It seems that on that day, the Server application v5.6.3 auto renewed it's self created certificate at 3:52PM on a MacPro 2013 running MacOS X High Sierra 10.13.6.


With Recovery I was able to reinstalled the same day's morning Time Machine backup and after initial settings procedures the MacOS Server was back functioning normally with its previous self created certificate expiring on Nov 30th 2020.


For my great misery, the Server app redid a certificate update and the collaboration services disrupted again. After a three levels escalation with Apple phone support, I've been kindly informed that this version of the software isn't supported anymore and Apple couldn't provide anymore help for this issue, except to rebuild the server, upgrade MacOs and the reference to the 2018 “MacOS Server

Service Migration Guide v1.2”.


The tech representative wasn't aware of anyone else with a similar issue and suggested to get a third party certificate to possibly solve this issue!


The disruption of the services is a major blow to the activities of our small company. My short term goals are :


1 - Secure the users' data from the wikis, imap mailboxes, calendars and contacts.

2 - Having a Web service back on, mostly for http, https is used only for the wikis.

3 - Having a Mail service back on, most probably PostFix.

4 - Having a Calendar service back on, ideally being able to import the .ics or .icbu data from the previous one.

5- I hopefully wish that the datas for the wikis can be put back to users' accesses.


The short term actions, I am planning:


1- Try to get a third party certificate, probably from LesEncrypt, the problem I am seeing is; to have CertBot to create a certificate, port 80 should be functional and its actually not functioning because of the disruption to the web service. I wonder what would be the workaround for this?


2 - Try to figure out if the actual MacOS X High Sierra 10.13.6 and Server 5.6.3 can be put back in a functional state as it's doing just fine normally.


3 - Maybe downgrading Server to a previous 5.3.1 version and maybe also MacOS to Sierra 10.12?


4 - Migrating all collaboration services following the guidelines of the “Service Migration Guide”.


-------


This disruption is happening at a very bad moment, any help will be welcome as I am in a state of emergency to solve this. Professional help is welcome!



André

Many thanks in advance.


==============


Terminal reports:


apachectl configtest


[Sun Sep 06 08:28:30.877240 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_ADDR} is not defined


[Sun Sep 06 08:28:30.877586 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_PORT} is not defined


[Sun Sep 06 08:28:30.877596 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_ADDR} is not defined


[Sun Sep 06 08:28:30.877600 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_PORT} is not defined


[Sun Sep 06 08:28:30.885353 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_NAME} is not defined


[Sun Sep 06 08:28:30.885371 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_PORT} is not defined


[Sun Sep 06 08:28:30.887101 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_ACCESS_GROUP} is not defined


[Sun Sep 06 08:28:30.887119 2020] [core:warn] [pid 26222] AH00111: Config variable ${WFS_SERVER_NAME} is not defined


AH00526: Syntax error on line 23 of /private/etc/apache2/other/httpd_webdavsharing.conf:


Port must be specified


============


sudo postfix check


-> returns <empty>



[Personal Information Edited by Moderator]

Mac Pro

Posted on Sep 6, 2020 7:34 AM

Reply
Question marked as Best reply

Posted on Sep 7, 2020 2:55 PM

Well there won't be another thread! For those reading down this far!


The issue of the Certificate was solved with the guidance for LetsEncrypt, mentioned above and I am looking forward to install auto renewal.


As for the Wikis issue, after checking as much as I can the A & B above and also if the FQDN of domain was correctly inscribed in the configuration file, I was down to C, for a full reinstall of the Server App.


Luckily, I have several backups of different types, Time Machine, Disk Images and Full Volumes copies. I used the Time Machine to restore the dedicated volume for the Server App specific and users datas because it was the freshest one, the others were a few days older. After manually comparing the content of different backups I notice that the Time Machine restoration was incomplete.


I wasn't aware that Time Machine would not back up all the files, as I never restored a full non booting volume with TM before. I was under the expectation that all files, even system files were to be copied. I decided then to experiment a feature of Carbon Copy Cloner to fusion the content of two different volumes with the “preserve newer files” option. What I had to loose, better loosing a few days of office work than many years!


It worked perfectly, better that I could hope for! On putting the Wikis back on, It loaded in a snap! All the calendars that are linked to the Wikis also reappeared on the spot! What a relief before reopening the office tomorrow morning!


I'll try to perform a system upgrade on the server as soon as possible and definitely do a migration in a short future. My main concern is to find the best solution to preserve the Wikis datas and functionnality, is anyone with experience would have a suggestion?


Thanks to all that helped on this issue and to all those that were considering helping!


Especially, thanks to Bombich Software! A life saver!

Similar questions

12 replies
Question marked as Best reply

Sep 7, 2020 2:55 PM in response to Photox

Well there won't be another thread! For those reading down this far!


The issue of the Certificate was solved with the guidance for LetsEncrypt, mentioned above and I am looking forward to install auto renewal.


As for the Wikis issue, after checking as much as I can the A & B above and also if the FQDN of domain was correctly inscribed in the configuration file, I was down to C, for a full reinstall of the Server App.


Luckily, I have several backups of different types, Time Machine, Disk Images and Full Volumes copies. I used the Time Machine to restore the dedicated volume for the Server App specific and users datas because it was the freshest one, the others were a few days older. After manually comparing the content of different backups I notice that the Time Machine restoration was incomplete.


I wasn't aware that Time Machine would not back up all the files, as I never restored a full non booting volume with TM before. I was under the expectation that all files, even system files were to be copied. I decided then to experiment a feature of Carbon Copy Cloner to fusion the content of two different volumes with the “preserve newer files” option. What I had to loose, better loosing a few days of office work than many years!


It worked perfectly, better that I could hope for! On putting the Wikis back on, It loaded in a snap! All the calendars that are linked to the Wikis also reappeared on the spot! What a relief before reopening the office tomorrow morning!


I'll try to perform a system upgrade on the server as soon as possible and definitely do a migration in a short future. My main concern is to find the best solution to preserve the Wikis datas and functionnality, is anyone with experience would have a suggestion?


Thanks to all that helped on this issue and to all those that were considering helping!


Especially, thanks to Bombich Software! A life saver!

Sep 7, 2020 12:30 PM in response to Photox

Server as we knew it is gone. Ponder your migration paths.

Photox wrote:

Thank you for the clarifications. I do understand the necessity of auto-renewal and I suppose that this is an internal setting of the Server App to renew its own certificate 60 days in advance without an alert.


Used to be that Server didn’t auto-renew self-signed (or maybe it’d fail to renew), and you had these same failures and an expired self-signed certificate.


I will definitely look forward for auto-renewal with LetsEncrypt! Thanks for the additional informations.


LetsEncrypt expires every ninety days. Other commercial certificate providers can be yearly.


There’s no security difference between self-signed and commercial certificates, beyond the availability of the public key for commercial certificate providers present in most clients. The commercial providers have a trusted path to distribute the public key. Private CA chains must provide that trusted distribution themselves, and must maintain their root CA private key security.


The real trouble is that the Server 5.6.3 App generates a new self signed certificate that is incompatible with itself, which in all practicality means the auto dysfunctioning of the collaboration services. Hoping it's not auto destructions. That's the waning I try to express to other potential users!


Self-signed certs do that. Always have. That’s how self-signed certs get renewed.


The alternative is to set up your own self-signed certificate chain, which would have been a nice enhancement for Server, in the era prior to LetsEncrypt and their ACME support, or similar.


But again, Server is gone. High Sierra is likely falling off all patches this fall with the arrival of Big Sur, too. Time to figure out what you’re using and what to migrate and to what and where.


--------

Practically, the main issue of this thread is solved, as the new certificate seems to perform normally, minus ONE huge issue with the Wikis.

During the two days of the intense stress that this issues caused, not knowing wha's exactly what was going on, my first actions were to salvage as much datas as possible, save all settings and make multiple backup volumes of the startup partition with the previous old certificate.


Can’t help, there. Roll in complete backups, maybe?


Maybe should I start a new thread for this?


Yes. This (now) isn’t a cert-related error.

Very few references to that error, one of which references a database issue.


Sep 7, 2020 10:21 AM in response to BDAqua

Thanks for links Yikes!


There will be very helpful, especially if I decide to continue to use LetsEncrypt certificates. I'm new to it and it seems to do a good job!


I may report the occurence of a major bug with Server 5.6.3 running MacOS X 10.13.6 localized for French-Canada. When the renewal of a self signed certificate is initiated by the application itself, it will produce a new certificate that is disrupting the collaboration service of the Server. This new certificate is also considered invalid by the client devices.


In the office over here, this trouble started on the afternoon of Friday September 4th 2020, when the Server application initiated automatically the renewal of its self signed certificate that was due to expire on December 6th 2020. Within minutes, this renewal gradually causing a major havoc in the office.


As a first time user of LetsEncrypt, I followed the Getting Started instructions created a startup partition from a backup previous to the certificate renewal to have the Web service running and to have a web site running on http port 80. This was complicated by the fact that during the time it took to install





Sep 7, 2020 10:44 AM in response to BDAqua

Thanks for links Yikes!


There will be very helpful, especially if I decide to continue to use LetsEncrypt certificates. I'm new to it and this organization seems to do a good job!


I may report the occurence of a major bug with Server 5.6.3 running MacOS X 10.13.6 localized for French-Canada. When the renewal of a self signed certificate is initiated by the application itself, it will produce a new certificate that is disrupting the collaboration services of the Server. This new certificate is also considered invalid by the client devices.


In the office over here, this trouble started on the afternoon of Friday September 4th 2020, when the Server application initiated automatically the renewal of its self signed certificate that was due to expire on December 6th 2020. Within minutes, this renewal gradually causing a major havoc in the office.


As a first time user of LetsEncrypt, I followed the Getting Started instructions and I created a new startup partition from a backup previous to the certificate renewal to have the Web service running and have a web site active on http port 80. This was complicated by the fact that during the time it took to install HomeBrew and CertBot, the certificate was renewing itself and I was loosing the Web service.


On the CertBot certificate installation:


sudo certbot certonly --apache


I was getting all the times the error:


Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.”


Which I taught was caused by the disruption of the Web service.


The workaround I found, was the follow the instructions of Seth Schoen on his July 19 2019 answer to Cannot issue cert for Mac Server where he propose two options to get the certificate, the alternate being to use this terminal command instead:


certbot certonly --webroot


And manually install the certificate with the Server interface.


I did this a few hours ago and it went well for the new certificate's installation, there were some issues restarting the Mail service, but the Mail service seems to be back on for now. My main problem at this moment is with the Wikis that I can reach now on https, but it hangs on the window's opening with Safari and ultimately returns the 502 Proxy Error.

Sep 7, 2020 10:52 AM in response to BDAqua

BDAqua wrote:

A Bug report will go in the Round Filing drawer.:(


Most likely, especially with the current status of this version the MacOS X Server application.


But it may warns other users of this dangerous potential issue as it may kill a business as it is causing great harms and stresses to mine at this moment.


I am not so sure that the auto certificate renewal has anything to do with the App Store, it's most probably an internal setting inside the application or in a configuration file that I wasn't able to locate.


Many thanks

Sep 7, 2020 11:23 AM in response to Photox

This mess is pretty much normal.


Private self-generated certs that don’t get auto-renewed will (also) cause piles of connection failures.


This was going to happen.


What to do?


You can set up your own root certificate chain and learn how to regenerate and renew those certificates from the private key without disrupting the existing self-generated certificates (I’ve done this, and it involves learning about SSL/TLS and certificates and DNS and related tooling), or a site can choose to periodically push out the new self-generated public certificate using profiles or using manual certificate loads or such, or you can purchase and can use commercial certificates.


All commercial certificates now expire within 398 days, and LetsEncrypt certificates and some others will expire more frequently. Which means cert updates yearly. The folks at LetsEncrypt have an auto-renewal mechanism ACME around (certbot, et al), but I don’t know off-hand if that was ported to macOS Server <5.7. A tool such as certbot or another certificate-renewal tool may may have been ported or macOS Server <5.7, I simply haven’t looked recently.


If you have questions, lemme know and I’ll try to answer them here. But you’re going to end up learning about certificates and DNS regardless, as that’s inherent with running a server.

Sep 7, 2020 12:05 PM in response to MrHoffman

MrHoffman wrote:

This mess is pretty much normal.
Private self-generated certs that don’t get auto-renewed will (also) cause piles of connection failures.


Thank you for the clarifications. I do understand the necessity of auto-renewal and I suppose that this is an internal setting of the Server App to renew its own certificate 60 days in advance without an alert.


I will definitely look forward for auto-renewal with LetsEncrypt! Thanks for the additional informations.


The real trouble is that the Server 5.6.3 App generates a new self signed certificate that is incompatible with itself, which in all practicality means the auto dysfunctioning of the collaboration services. Hoping it's not auto destructions. That's the waning I try to express to other potential users!

---------


Practically, the main issue of this thread is solved, as the new certificate seems to perform normally, minus ONE huge issue with the Wikis.


During the two days of the intense stress that this issues caused, not knowing wha's exactly what was going on, my first actions were to salvage as much datas as possible, save all settings and make multiple backup volumes of the startup partition with the previous old certificate.


With our office settings with Server 5.6.3 all the application and users datas are located on a dedicated volume. Yesterday the volume was erased with Disk Utility with the MacOS extended format. The “Library” folder was copied back from a fresh Time Machine backup while being in root user.


Now it seems that all services can be put back on except for the Wikis that return in Safari:


1) 502 Proxy error

The proxy server received an invalid response from an upstream server.

The proxy server could not handle the request GET /wiki.

Reason: Error reading from remote server


or


2) 503 Service Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Additionally, a 503 Service Unavailable error was encountered while trying to use an ErrorDocument to handle the request.

Apache Server at wiki.photographex.com Port 35343


My questions how to solve this major problem:


A) Is it a folders and files permissions issue? If yes, how to solve that?


B) Is it a configuration issue in the config files? I have none experience with these particular one located in: </Library/Server/Wiki/Config>


C) Should I just uninstall Server by putting it the the garbage, reboot the computer and put the Server App back in the application folder to force a reinstallation?


Any hint will be welcome!


Maybe should I start a new thread for this?


Many thanks for your help!


Regards


Sep 7, 2020 12:45 PM in response to MrHoffman

Mr Hoffman,


Well! I know too well that Server is gone, I am just late, if I survive all this I will definitely get out of this as soon as possible.


Thanks for all the informations clarifying the correct procedure for certificate uses. This is much appreciated.


I am still trying to figure out the issue with Wikis, I am very afraid I've lost a lot essential datas. I am not sure but it looks like Time Machine back up isn't complete of the “Library” folder!


I'll start a new thread for this!


Many thanks and my best regards!

MacOS Server 5.6.3 certificate auto renewal cause collaboration services disruption

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.