I bought iPhone 12 mini two days ago and I have this same issue. I am actually considering returning the phone, since listening to music is such a huge part of the way I use my phone, and it is impossible with the volume levels currently used in iPhones.
I have the newest updates in the phone, I have the headphones safety on and it helped a fraction, but not enough. I also downloaded a volume control app, which allows to take the volume down to 1% and even that is too loud! This is absolutely ridiculous. I'm using JBL headphones. Does anyone have experience of non-apple headphones that would allow for more volume control? Or an app to fix this?
By adjusting balance from the sound settings you can make either left or right earpiece silent enough, but only left or right, not both. So currently I only use the right earpiece of my headphones, which is ridiculous.