Maybe someone else will have a better answer, but here is what I found in this link:
https://appletoolbox.com/multiple-face-id-users-ios-12/#Can_I_Delete_or_Change_An_Alternate_Appearance_for_Face_ID
“Once you enter an alternate appearance, there’s no option to delete it! Instead, you need to Reset Face ID to change your alternate appearance. Unfortunately, that also means you must re-enter all Face IDs! When you tap Reset Face ID, your device removes and deletes all your Face ID data–including your original Face ID and your alternate appearance.”
My question is how are your friends managing to add their faces to your ID? It cannot be done without your passcode. You may want to change that.
Set a passcode on iPhone - Apple Support