Shared iCloud account to separate Apple IDs and Family Share

After years of 4 of us sharing one iCloud account, we tested moving me off (next our kids, both not living with us anymore). I created my own Apple ID and my wife invited me, through Family Sharing, to take part in the iCloud storage we pay for. It all seemingly went very well. I moved my phone, macbook, watch and iPad to my new account. Nothing got lost. But I did encounter one problem: ALL the photos in our iCloud, 22,000, backed up onto my phone. How do I get rid of them? Originally, I had about 6,000 photos on my phone. I just want those 6,000 photos only. Many thanks for your help!