Face ID is unlocking with multiple faces on iPhone

• Does your brother know your Passcode, or ever used the Passcode in the past?
• Have you deleted all Face data and the Alternate and set it up again?
• Have you changed your Passcode?
• Do not give anyone else your Passcode.

When a face is not detected and the correct passcode is entered, the Face data is updated to reflect those changes, so if you allow other people to access your device with your Passcode, then it will "learn" their face details. Some important points about Face ID can be found here:

• The probability that a random person in the population could look at your iPhone or iPad Pro and unlock it using Face ID is less than 1 in 1,000,000 with a single enrolled appearance whether or not you're wearing a mask.
• The statistical probability is higher — and further increased if using Face ID with a mask — for twins and siblings that look like you, and among children under the age of 13, because their distinct facial features might not have fully developed. If you're concerned about this, we recommend using a passcode to authenticate.
•  If there is a more significant change in your appearance, like shaving a full beard, Face ID confirms your identity by using your passcode before it updates your face data. 

About Face ID advanced technology - Apple Support