Family plan
right Now 3 family members have their own Apple Music subscription, if we switch to the family plan will we lose the music we already downloaded?
No. You should not lose anything.
Don't manually cancel each member's personal subscription. Simply invite them to the Family Group, with the Family Subscription, and the transfer should be seamless; automatically cancelling their subscriptions while maintaining their library.
See here: Get an Apple Music Family subscription - Apple Support