Set up VPN with only IP

Ah, okay. macOS Server has Open Directory and not Active Directory. The latter is Microsoft.

This is pretty close to the "fun" of a fresh installation, having been through this on several occasions.

DNS is still not involved with the VPN services, unless the domain that's been used for the internal services and the internal OD was what was lost. If that's been lost—and I'm guessing that's what has happened here—then you're headed toward an OD migration and rebuild. That's not a small project.

Hopefully, the changeip command will fix this. That's the command-line command that can rename and re-address a server.

The man page for changeip has some command syntax details:

changeip [-v] [-d path] old-ip [new-ip] [old-hostname new-hostname]

If the changeip tips over for some reason, then this'll involve an export of all directory data using slapcat, edit the resulting LDIF for the new domain, and then reload the directory data. Any Base64-encoded data in the directory will have to be decoded and checked for the domain name, and I'd look for line-wrapped names.

There may well be a script to do this with LDIF data, though I've not looked for one of those.

This'll also involve reconfiguring internal DNS to match the new domain and new host names.

And re-creating the server certificates, or requesting and loading new commercial certificates.

For external access? Get a new domain registered, and reconfigure or re-provision the DNS client to use that. Use a subdomain of that or use a second and separate registered domain for the internal network.

What'll be a smaller effort... Move the VPN server out to the firewall, using a firewall that includes an embedded VPN server. You're probably going to have to do that eventually, given the deprecation of VPN services and the deprecation of most of the rest of the macOS Server.app features, and given that AirPort hardware has also been retired. The firewall-embedded VPN server can also avoid a lot of the above as a workaround, if you're willing to reconfigure some clients to use firewall-based authentication and not check with the server.

Given most of Server.app has been deprecated, I'd also look at where you're headed next, given what you're going through here. If changeip doesn't work, you're well along on performing a migration, pragmatically.

In no particular order...

The VPN is clearly working.

Your VPN clients will want static DNS references to the target shares. mDNS (.local domains) don't get broadcast over most VPN connections, so the remote clients don't have access to mDNS, which means the shares best work with DNS translations from the local DNS server and the VPN configured to pass the internal IP address of the local DNS server to the VPN clients.

My preference is a VPN server embedded in the firewall, as that removes the complexity of implementing VPN pass-through on the firewall NAT. VPNs and NAT work at cross purposes. The VPN tries to identify the ends of the VPN connection, and NAT tries to hide the end-points of the connections.

Do you have any client references to any DNS servers located off of your network, other than references from the DNS server on your network? It's very common for folks to assume they can use ISP DNS services, or to include DNS references to both the local DNS server and to the ISP servers, and that doesn't work.

More generally, I'd suggest getting some IT help, or spending some time learning how IP and IP routing and subnet routing works, how DNS and DNS server resolution works, how VPNs work, and how Wi-Fi works.

Trying to describe the assumptions and addressing and the rest of building a network or of troubleshooting a network in a forum posting reply in a text input box, not so much.

I'd like to provide suggestions on books for this topic, but I don't know of any. Maybe this or this?

Somebody may have a pointer to some details on setting up a server on a NAT'd network, too.

Biggest requirement here is setting up local DNS services, and referencing only the local DNS server.

Again, VPNs can connect to an IP address, assuming it's a static (fixed) IP address.

If your connection is not working, it's because there's a deeper configuration problem with the server.

Fill in the VPN target as an IP address.

And getting a domain name registered is ~$10 or $15 per year, and variously less, and it avoids going through hassles when IP addresses change.

It can be more than enough "fun" to troubleshoot a DNS or OD problem given local access. Trying to troubleshoot this remotely is more of a hassle.

Get somebody to configure a firewall-embedded VPN server for you, and use that as your firewall, and use that as what the VPN clients connect to. And the VPN clients can connect by a domain name that you've registered, or by IP address if you really want to create what is usually a bigger mess later.

Can you explain how your VPN is connected to your Microsoft Active Directory?

A VPN in isolation isn't tied to a DNS translation, which implies your VPN server and its client authentication is somehow tied to your AD. That's going to be specific to the VPN server involved here.

And if your domain is tied to your own Microsoft AD and your domain has changed and if that domain is used throughout your AD, then you're in for some effort. Given the gnashing from the folks managing AD configurations, changing domains in an AD is gnarly.

If this is your own network and if it's NAT'd, set up a VPN server on the firewall. That's preferable to running VPN connections through NAT.

In short, we're going to need a little more detail about your VPN server and your configuration.

The external static IP is what the VPN connects to, after it has translated the domain name tyypically used. The VPN then provides an IP from the client to the network of the VPN server, and the VPN server typically passes along the internal DNS server IP address to the remote client as part of the same DHCP-like “welcome” that any VPN client receives.

The remote client has an IP address and a DNS server address on the target network, just as would a client local to the private network.

That DNS address allows the VPN client to translate host names to host addresses on the target IP network, using the DNS server on the target network.

Probably more familiar, a DHCP server passes out the IP address for the client, as well as the IP gateway address, and the DNS server address. These are the atteibutes that an IP (IPv4) host needs to connect to and operate on an IP network.

Your network DHCP server would want to be passing out the local DNS server address, too.

A VPN is an IP connection, where the VPN software on the client presents a network interface with a “direct” connection on the target network. If you think of a VPN client as being (mostly) local to the target network, and the VPN providing DHCP, you’re pretty close to how this works.

And macOS server (prior to 5.7.1) sets up a local DNS server, as Server.app expects and needs DNS.

Where folks get in trouble is trying to mix ISP DNS and local DNS. Once you have a server and have local DNS, that’s what your clients and your DHCP server should all be referencing.

Server networking is different from what clients have to deal with, as the clients depend on the servers to provide the network configuration and settings. And DNS, as most servers require DNS. And ISP DNS cannot provide translations for private IP addresses.

OK, I figured it out. I had to put in the local IP address, not the static IP when connecting to the shares over VPN. Now it works. Thanks for your help.

Which means your local DNS is incorrect, or is not being used.

Embedding static IP addresses is just not a good path.

Please indicate what has happened with the domain name; what was meant by "stop using the domain name".

Do you have a replacement registered domain name available?

Please also indicate if this has previously worked correctly; that the domain name issue was the precipitating change.

And so far, this isn't the VPN server, as that's clearly connecting. That's using the public DNS name to resolve the static IP of the firewall, if the VPN client configuration hasn't been switched over to a different DNS name or to the public static IP address of the firewall.

Launch Terminal.app on the server, and issue the (harmless) diagnostic command:

sudo changeip -checkhostname

This command will show some output, indicating whether the local IP configuration is valid (command output will be: dirserv:success = "success"), or some other and different output indicating an issue.

You'll need to have internal private DNS going and probably on your server if you don't have another local private DNS server available, and your server will need a host name on the internal network, and it's... auspicious... if the domain name being used for the server matches the domain name used in the Open Directory database. And the domain name used here is best one that your organization has registered.

Your VPN server will have to provide the IP address(es) of the private DNS server(s) operating on your internal network, and that'll have to be configured correctly. That's how the remote clients of the VPN can resolve the IP addresses on the internal network; the VPN connection passes the clients the DNS server addresses, and the VPN clients then use that to resolve the host names.