Apple Watch Ultra Action Button Turning Off Action Button

My Apple Watch Ultra (2)'s Action Button continues to get activated after I deactivate in the app. I think it happens about each time the watch goes through an OS update, which is frequent enough that it's annoying. Is there a way to prevent this moving forward so I don't have to keep going in and manually adjusting?