The Selfie album is a smart album. It essentially is holding all photos that have been taken with a front camera of your iPhone, iPad, iPod Touch. My Photos library is showings all the photos I took of the ceilings of a church with the iPhone lying flat on the ground as "Selfie" just because they are front camera photos, rather silly.
To remove a photo from the Selfie album you have to remove the EXIF lens tag. That would require brute force. Export the photo from Photos, then edit in an external editor that removes the EXIF data and reimport it. I noticed on my iPhone that editing a photo in Pixelmator removed it from the Selfie album.