How to get rid of an "accept incoming network connections" popup

Ever since upgrading to El Capitan, I'm receiving a popup message asking "Do you want the "???.app" to accept incoming network connections? with a yellow triangle containing an exclamation point.

This is popping up with multiple applications. I've searched through all of the system settings and cannot find any way of turning this off. Anwering either Deny or Allow still results in it popping up again and again when the application is restarted. Any idea how to disable this?