Airpods Pro keeps connecting when I charge the case.

My Airpods Pro were recently replaced due to defect (just the buds, not the case) and ever since then they connect to my phone when I plug the case in to charge. They connect fine the normal way. I'm not sure if there is any harm in them doing this or if it's a sign of an issue but they didn't do that before the replacement. Any ideas how to make it stop connecting while charging?