Music volume changes when switching song iOS

A few months now I'm having issues with the volume output of my iPhone. When a new song comes on then every now and then the volume is much louder, seems really random.

The funny thing is, the volume slider didn't change. When switching to another song this fixes itself most of the time and the volume output is 'normal' again.

I wear an Apple Watch 3 and it might be relevant because timing wise it seems to correspond.

• It only occurs after switching to a new song, not while playing a song.
• The volume slider doesn't change (both iPhone and Watch), but turning down the volume still works only it's just louder.
• Pausing doesn't fix it.
• I use Apple Music with mostly downloaded songs.
• It isn't song specific, it can happen to all songs and sometimes changing to another song and then back to the song that had a higher output can suddenly fix it.
• Tried different headphones, didn't matter.
• Had a few iOS updates in the meantime, didn't help.
• Yes, I tried turning it off and on again.

This is so weird, and I wonder if anyone has an idea of what seems to be happening here.