It won't be easy to use UTM for macOS. macOS installers checks the system for key elements in order to allow installation which UTM (aka QEMU utilized by UTM won't have as they are emulating basic & generic PC hardware). While there are instructions for running macOS on other systems, most of them involve utilizing customized installers made specifically for a VM or Hakintosh community...some may include manual instructions, but most are not entirely clear. I tried years ago and was not entirely successful as I never wanted to utilize an installer from an unknown source (probably ok, but not for me).
If you want to use Mavericks or some older version of macOS, you will have a much easier time utilizing an older Mac as has already been suggested. FYI, does your special app work with macOS 10.10 or 10.11? You will have a much easier time installing those versions of macOS on older hardware since Apple provides access to those installers unlike macOS 10.9 which currently has no download links from Apple unless you can perhaps access a "Previous Purchase" from the App Store (no idea whether you can create a bootable USB installer from it like you can with 10.10 or 10.11....still requires a compatible Mac to make the USB installer...so you are back to needing an older Mac anyway).
