lowest volume is too loud
lowest volume on airpods is still too loud
There isn't anything that can be done about that. I also noticed that on phone calls, the lowest volume is too loud. It's something that will have to be worked out maybe in a future update.
Sorry, but that's the best that we can offer you.