You said that you know that Target Display Mode won't work. ("Target Display Mode was dropped in 2014."). I think your question was in regards to workarounds such as Luna Display and DuetDisplay.
I don't have experience with either, but here's what I've learned about Luna Display. (You can find a lot of this if you look at blogs and support articles on the Astropad site.)
----------
As far as I know, Luna Display is the only one of the two that supports 5K resolution. Blogs on the Astropad site say that Luna Display is "not recommended for high motion content" – apparently just moving a window quickly might cause the screen to "temporarily become pixelated before it renders in at full resolution."
----------
Note that even if you set up Luna Display in "headless mode," you will need to attach a real monitor to the Mac mini in order to do the initial setup. If you have that monitor, you may just want to keep it attached to the Mac full-time.
----------
The USB-C Luna Display device would probably tie up at least one Thunderbolt port. Astropad says that "Using Luna with any adapters, hubs, or docks would be at your own discretion, as we cannot guarantee compatibility since they are not officially supported.". You'd want a USB or Thunderbolt cable connection between the Macs, and if that was a Thunderbolt connection, it could tie up another Thunderbolt port (unless you had a dock).
Then you'd need a place to plug in the real monitor (the HDMI port?).
---------
