The application can't be opened even if right click to open
I'm on Catalina 10.15.6 and I'm trying to test my own Unity apps. I used to be able to right click and select "open" to bypass the security and launch these apps. But recently, this option has stopped working. I just get this empty dialog that says "The application [application name] can't be opened" with no additional text. And if I click on the question mark for more info, it takes me to an empty help page.
I know there's nothing wrong with the app because if I manually navigate into it through terminal and chmod -x the app, it works. Everything I read from Apple indicates I should be able to right click and bypass security for the app ... any idea why it's not working?
MacBook Air 13″, macOS 10.15