how to disable OSX server 3 SMTP authentication

Something ike the following is how you can allow local clients to bypass the restrictions.

$ postconf -c /Library/Server/Mail/Config/postfix smtpd_client_restrictions

smtpd_client_restrictions = permit_mynetworks permit_sasl_authenticated permit

$ postconf -c /Library/Server/Mail/Config/postfix mynetworks

mynetworks = 127.0.0.0/8, [::1]/128

$

Basically, mynetworks is allowed onto the mail server without credentials. In your case, add the printer IP address into mynetworks, and your problematic printer should be able to connect to the printer. I'd be cautious about allowing unnecessary additional unauthenticated access, as — though your network is closed — there is malware around that looks for "open" SMTP servers and then starts generating spam via those servers.

Commands to modify these two settings — you may only need the second — would be akin to the following (replacing the 10.20.30.40 shown below with the address of your printer):

$ postconf -c /Library/Server/Mail/Config/postfix -e "smtpd_client_restrictions = permit_mynetworks permit_sasl_authenticated permit"

postconf -c /Library/Server/Mail/Config/postfix mynetworks -e "mynetworks = 127.0.0.0/8, [::1]/128, 10.20.30.40"

$

You'll then need to reload the Postfix server (once) to pick up the new settings:

sudo postfix reload

The firmware in your printer is about as primitive as I've encountered for something claiming SMTP support. That's unfortunately only barely capable as an SMTP client, and probably a device that's only suitable for residential or small office use. (Larger organizations and those that are looking to keep a lid on SMTP mail activities will usually have these Postfix settings or similar settings on other mail servers enabled and locked down by default. Those organizations that are more security-conscious can be concerned about rogue firmware being loaded into these printers, too; there've been various printer exploits in recent years. But I digress.)

As with the earlier settings, you can bypass the HELO settings for specific hosts:

smtpd_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname

As for your postscript, yes; that postconf for the mynetworks setting was a cut-and-paste error on my part.

Why not edit the configuration files directly? That works, certainly. I generally prefer to avoid directly editing various (particularly complex) configuration files. Mistakes can happen when the file is open for editing, and those mistakes can sometimes be more difficult to diagnose and resolve. With the command-line postconf-based approach, I can screw up just one line of the configuration file, and specifically only the line that I'm working on. Or the command line tool variously catches and prevents my error. If I have the file open in an editor session, well, we've probably all had the experience of accidentally adding or deleting a character, or adding a newline somewhere else. (e.g. that previous cut-and-paste mea culpa.)