How (do I relay)??
My ISP has an SMTP server I can relay to. How do I configure ML Server so that outbound Alert e-mails get sent to that SMTP server? I checked off the option to Relay Outgoing Mail through ISP on the Mail config screen (and entered SMTP server address and credentials). But Alert messages aren't getting sent.
You have to configure your mail server to relay to your ISP.
The notifications and various other services all expect to go to the local mail server.
I don't have a Mountain Lion 10.8 server box handy to check the exact syntax, but it should be via a "Relay outbound mail via ISP" setting in the Mail section of Server.app. (It's also possible to set up the relay via Postfix commands, but I'm guessing you don't want to go that route.)
Which is unfortunately exactly what you have said you didn't want to do here. Rock, meet Hard Place.
You're correct: there is a "Relay outbound mail through ISP" setting in the Mail section of ML Server. I've enabled that option, and entered my ISP's SMTP URL, plus my access credentials. But the ML Server is not sending out notifications.
You're also correct that I don't want to set up the relay via Postfix commands. I have no clue how to go about doing that. But I do understand Terminal and rudimentary unix commands, so I'm willing to give it a try if I had a recipe to do so. I guess the fundamental question is whether the ML Server mail service can be configured to only do outbound mail (for notications, and nothing else) but not incoming mail (no POP, no IMAP, no MX record, no rDNS entry).
If the mail server is started and the relay is not working, then there's some sort of configuration issue with the specification fo the relay (did you specify the target port required by your ISP? Are the ISP mail server login credentials correct?) or there's an error with the operation mail server itself.
Check the Server.app logs and Console.app for any problems related to the mail server startup, too. Various errors can be logged there.
As for another potential trigger, it's very common to skip DNS set-up with OS X Server systems, and that can cause weird errors with services that require network authentication, and that's most services. To confirm DNS is correct, launch Terminal.app and issue the command
sudo changeip -checkhostname
That'll report no changes are required, or potentially diagnostics related to any network configuration errors detected. (Why DNS? The mail server can get confused if it can't determine its host name, for instance.)
To secure the local mail server against remote access (which seems to be at the core of your plan), you could choose to block the in-bound mail server ports at your gateway-firewall-NAT device at the edge of your network, which would prevent all but local users from accessing the mail server. That'll block remote access, but should not disrupt local mail server activity. If in-bound ports are blocked at the edge of your network, your server will only be able to send outbound mail, so only your internal systems should be able to send mail.)
Thanks for the pointers and suggestions.
The config for my ISP's SMTP server is correct. I specified the port (:465), and double-checked userid/password. It's the same as used by all our e-mail clients (a half-dozen or so on various Macs, iPhones, iPads, Linux boxes), all of which work fine from my LAN.
As for DNS, it seems OK. (I am indebted to you for the excellent series of articles on servers and networking at http://labs.hoffmanlabs.com/node/1705 )
The changeip command reports:
The names match. There is nothing to change.
dirserv:success = "success"
However, I should point out that I do not have the DNS service enabled on ML Server. I already have a local DNS running on my gateway router. (it's an instance of the dnsmasq process that is part of Tomato firmware). It has local names defined for all the servers and hosts on my LAN, and provides rDNS for them as well.
However, what I haven't done is enabled the Mail service. You've (slightly) misunderstood the core of my plan. My intent is not simply to secure the local mail server against remote access. I don't want either remote or local users from accessing the mail server. I don't want it listening for POP or IMAP connections from anywhere (on my LAN or remotely). I don't want it receiving inbound mail from anywhere. I don't want it relaying outbound mail for any client on my LAN. All I want the ML Server to be able to do is to send me alerts (by e-mail) when it's in need of attention. Other appliances on my LAN (eg. a NetgearReadyNAS) can send me e-mails when they are unhappy, and I hope I can coerce the ML Server to do the same.
I know I don't have the expertise to operate and maintain my own mail server, and I'm quite happy with the mail service provided by my existing network service provider. Plus, I'm behind a cable modem with a dynamic IP address, so I'm unable to establish my own rDNS record (which I understand is required to handle inbound mail).
It looks like the next step is to turn on the Mail service. But I'm sure it's going to be unhappy since there's no MX record set-up. I was hoping there was a way that ML Server could simply send out its own outbound alerts (using my ISP's SMTP relay) without having the incoming part of the mail service active. But perhaps Apple didn't anticipate that sort of configuration.
Somebody else might be able to better assist with this question and your concerns. (This given I'm increasingly confused around what's happening here, and the requirements, and my responses clearly aren't moving you forward.)
As for DNS, OS X Server doesn't care where the DNS translations are acquired, so long as there are translations available for the local (usually NAT'd network) addresses and domain names, and assumg the local network DNS is not in .LOCAL or similar domain you don't have registered. Your Tomato router DNS server would be the source for the MX record for the mail server.