"Why won't it connect when I enter the domain name, but it will with the 192.168.1.x IP?"
Two reasons. You haven't configured external DNS properly (your domain registrar has not been given instructions by you to do so or you don't know how to do it using your domain's control panel?) and/or you haven't opened up the relevant port in whatever you're using at your network's edge for your firewall. All running services on your server (which is behind your firewall and therefore in your local LAN) will have a port associated with it. You need to open up the relevant service port on your firewall and forward external requests on your non-LAN IP address (your WAN address) to your server's LAN IP address. How you do this will depend on whatever hardware you're using at your network's edge for your firewall.
"The host name doesn't work on the LAN either. Just the local IP."
Same as above. You need to tell your LAN clients which DNS server they must use to resolve hostnames to IP addresses. This would typically be your mac server as it would definitely need DNS just for itself. Configure whatever you're using as a DHCP server to 'deliver' your server's IP address to your client macs. Your server will do the rest assuming you've configured DNS correctly?
I notice you've asked a few questions of similar nature before and most of them will be down to firewall control and DNS.
For some reason, I can't manage my server over the internet. I can only manage it when I'm on my LAN, or connected to a vpn (hosted by the router, not the server).
Why won't it connect when I enter the domain name, but it will with the 192.168.1.x ip? The host name doesn't work on the LAN either. Just the local IP.
In general, you really don't want to be managing your server remotely, except via VPN. That's a security exposure, and I don't trust a management port to be entirely secure against remote access attacks. (It probably is, but then several major server vendors have also shipped out wide-open IPMI management connections in recent years, too.) Requiring remote management via VPN means the management port (TCP port 311) can't be probed by remote users.
If this were a discussion of HTTP (TCP Port 80) or HTTPS (TCP port 443), which are services that are (usually) open to the Internet, then the reported behavior would most likely be either an issue with the public DNS services translation — this is entirely different from your local DNS activities and local DNS services — or with the port-forwarding configuration of whatever device you're using as a firewall-gateway-router, or possibly a firewall block at the ISP as is common with residential-class tier of service with many ISPs. Either your DNS translation isn't going to the proper external IP address of your firewall-gateway-router box, or your firewall box isn't forwarding that port, or your ISP isn't allowing the port through NAT and along to the server. It's also possible that the service is blocked against remote access, though that's less common on systems after 10.6. (Blocking external IP addresses was trivial with OS X Server 10.6, but was removed from the GUI in 10.7 and later.)
If you're not getting the host name via the VPN, make sure the VPN client is set to use your LAN-local DNS when it's connected.
If the above doesn't cover your case, please consider providing a few more details about the configuration and what's working and not working, and about your particular network configuration.
This document lists all the well-known ports used by Apple applications, including OS X Server:
To my knowledge, even though the name of some of the services has changed between Lion and Mountain Lion Server, the ports have remained the same. If that's the case, this support article could also be useful:
Remember to only open and forward the ports you need.