Yep, it's BIND 9. I think it's 9.9.7 on 10.12, dunno for 10.13.
Your problem here is that /etc/hosts is a way of brute-forcing DNS for the local computer only; nothing in that file gets used by Server.app. So when Safari on the server needs a domain, it gets it first from /etc/hosts, and then goes to Server.app's DNS server (assuming the server is pointed to itself for DNS service). Other Macs on the network will look at their *own* /etc/hosts files, then at Server.app, but never at the /etc/hosts on your server.
So if you *really* want to block this in DNS, here's what you do: the rest of the Macs on your network can *only* get their DNS from your server. If they have multiple DNS servers in System Preferences, that's not a rank order, that's a round robin: sometimes they use your server, sometimes they'll go to Google DNS or whomever.
Your server, meanwhile, *does* point to other servers for any domain for which it's not authoritative. So your iMac might ask your server, "hey, where's google.com?" And your Mac Pro checks its own tables and says, "I don't know that, so I'm going to look it up elsewhere."
The secret sauce: you can make your server *think* it knows any given domain, just put it into your DNS tables. That will instantly blackhole those domains; put google.com into your own DNS, and your server stops looking for the real Google and just hands out whatever IP addresses you've manually entered. This (deliberately) breaks every computer looking at the server for DNS—those domains effectively no longer exist—but it doesn't break anyone else on the Internet, because no one else is talking to your DNS server.
But here's the thing: why do all this? You can much more easily set up ad blockers in all of your browsers, which have the added benefit that someone else does the job of constantly updating the block tables. Plus, browser plug-ins fail gracefully when they don't retrieve an ad. Your method is going to leave big honking whitespaces on every page with flashing question marks, or whatever now gets shown when servers are missing. If the page is written such that the Javascript waits for the ads to load before it shows any actual page content (not uncommon), you'll never see anything on the page as it'll be waiting forever.
