My driving mode keeps going on and off all day when I'm not driving.
I've changed the setting to automatic and to only when connected to car's bluetooth and the same thing happens.
iPhone 12 Pro, iOS 15
Are you sitting close to your car? i.e. within Bluetooth range?
In part, Driving Mode is based on whether your phone is connected to your car, regardless of whether you're actually driving or not.
If you're sitting close to the car (maybe it's just the other side of the wall, or outside the window), then it's possible the phone is connecting to the car and engaging driving mode.