homekit annoying automation authorization promt

I have a home kit enabled garage door and set it up for location based opening. But homekit forces me to respond to a prompt before opening the door. This defeats the entire purpose of making this automatic. It is neither convenient or timely to pull your phone out of your pocket and respond to an authorization promt to perform the automation. It is then not an automation but pressing a button to perform a task. I want to turn this off. If there is no way to do so then please put it in a software update in the future to allow this. It again defeats the entire purpose of automation. I may as well just press the button on my cars visor and forget the HomeKit enabled garage door.