Block access to Default/ website when using IP

You'll need to set up a default site in Server.app for the incoming connection. It seems that whatever the public DNS name of the path into your server, or the public IP address that you're using, doesn't have a web site, so it's getting the default site. If you've got that working locally but not externally, then there's a difference in the host name that the remote client is connecting to as compared with the host name that the local client is connecting to. It's this client-specified host name that's passed over the HTTP or HTTPS connection and it's this string passed from the client that's used to select the web site that Apache httpd server shows to the client.

Given that you're here specifying the public IP address of the server from within the remote client, make that a site or a synonym/alias of the site you want to show. Otherwise, you'll get the default site.

You'll also then decide whether you want to invest some time and effort in sculpting some added rewrite rules into the Apache httpd configuration to mask the specific URLs you don't want external folks to see.

This sort of "leakage" is one of the reasons why folks use an isolated "DMZ" network configuration and separate servers for internal usage and for externally-accessible activities. Also because various web servers have been breached using configuration errors or weak passwords or software vulnerabilities, and attackers have then sometimes gained further access into the underlying server and the rest of its data.

And more generally, the web server and other parts of macOS Server are being deprecated in an upcoming release, so you're also headed for a migration to a different web server on macOS, or to a different server platform.