Voice control triggered by headphones

A persistent and annoying issue I have found within the last few months is that, when listening to music, the hold button on headphones seems to switch on voice control instead of switching on music.

For example, if I am listing to music but need to pause for a few minutes, when I press the headphone button to restart, it switches on voice control instead of restarting the music. If I restart music after pausing for only a few seconds it seems to be ok, but anything much longer than that seems to default the button back to voice control, rather than music control. This never used to happen and I could pause and restart music after long periods of inactivity.

This means that you then have to fish the phone out of your pocket to manually switch off voice control and restart music. Sometimes, when I could not be bothered to do this and just leave it with voice control on, I then have voice control suddenly telling me it is randomly calling one of my contacts resulting in me then having to frantically get my phone out to try and end the call!

I have switched off voice control for the home button but this doesn't deactivate it from the headphone button. Very irritating and surprised I haven't seen this reported elsewhere. I really wish I could just deactivate voice control completely - I don't want it and have no intention of ever using it!