My family can unlock my iPhone Face ID

Read: About Face ID advanced technology - Apple Support - https://support.apple.com/HT208108

"The probability that a random person in the population could look at your iPhone X and unlock it using Face ID is approximately 1 in 1,000,000 (versus 1 in 50,000 for Touch ID). As an additional protection, Face ID allows only five unsuccessful match attempts before a passcode is required. The statistical probability is different for twins and siblings that look like you and among children under the age of 13, because their distinct facial features may not have fully developed. If you're concerned about this, we recommend using a passcode to authenticate."

Also consider that if you two look alike, and this person knows your passcode.. every time they enter your passcode the iPhone will start 'learning' like it's an alternate appearance of you.

So you could try setting up Face ID again, changing your passcode, make sure no one else is in the cameras view, don't share your passcode.