Using my iMac as display for other devices

If your iMac is 27" late 2009 or mid 2010, it can be done, but not with a simple Mini DisplayPort to HDMI adapter. Those adapters are for connecting HDMI displays to Mini DisplayPort computers. They do not work in reverse.

You need a Kanex XD converter

https://www.amazon.com/Kanex-HDMI-Mini-Displayport-Converter/dp/B003LGOWNQ

and a Mini DisplayPort cable.

Target display only works with other macs using the few macs which support the feature as display read Use your iMac as a display with Target Display Mode - Apple Support

so it will not work with some random security system

maybe a external usb tvtuner which has inputs and support osx will be an option

I have a 27" 2010 iMac and have used Target Display Mode with a MacBook Air connected directly with the Mini Display video cable. But when I tried using the Kanex XD HDMI adapter I could never get it to work with two different PC laptops, screen goes black for a minute and then returns to Mac screen. Not sure if the particular Kanex XD unit I had was a dud, or some other problem.

For the security system you have, do they have some kind of WiFi connection available as an alternative to the hardware video ports? Could something like a SlingBox grab the hardware video signal and broadcast it over WiFi/internet?