Does Face ID take into account different lighting conditions (dim light, bright sunlight, night)?
Apple does not have these details on their website. I read an article - *** which states that Face ID does take into account different lighting conditions and also, varies a threshold based on the type of user activity (like doing a payment, unlocking a phone, etc.).
Is this really true about Face ID?
iPhone X, iOS 12.1
Read here about Face ID from Apple: About Face ID advanced technology - Apple Support
Face ID security
Face ID is designed to confirm user attention, provide robust authentication with a low false match rate, and mitigate both digital and physical spoofing.
The TrueDepth camera automatically looks for your face when you wake Apple devices that feature Face ID by raising it or tapping the screen, as well as when those devices attempt to authenticate you in order to display an incoming notification or when a supported app requests Face ID authentication. When a face is detected, Face ID confirms attention and intent to unlock by detecting that your eyes are open and your attention is directed at your device; for accessibility, this is disabled when VoiceOver is activated and, if required, can be disabled separately.
After it confirms the presence of an attentive face, the TrueDepth camera projects and reads over 30,000 infrared dots to form a depth map of the face, along with a 2D infrared image. This data is used to create a sequence of 2D images and depth maps, which are digitally signed and sent to the Secure Enclave. To counter both digital and physical spoofs, the TrueDepth camera randomizes the sequence of 2D images and depth map captures, and projects a device-specific random pattern. A portion of the A11 and newer SoCs neural engine—protected within the Secure Enclave—transforms this data into a mathematical representation and compares that representation to the enrolled facial data. This enrolled facial data is itself a mathematical representation of your face captured across a variety of poses.
Facial matching is performed within the Secure Enclave using neural networks trained specifically for that purpose. We developed the facial matching neural networks using over a billion images, including IR and depth images collected in studies conducted with the participants’ informed consent. Apple worked with participants from around the world to include a representative group of people accounting for gender, age, ethnicity, and other factors. The studies were augmented as needed to provide a high degree of accuracy for a diverse range of users. Face ID is designed to work with hats, scarves, glasses, contact lenses, and many sunglasses. Furthermore, it’s designed to work indoors, outdoors, and even in total darkness. An additional neural network that’s trained to spot and resist spoofing defends against attempts to unlock your iPhone X with photos or masks.
Face ID data, including mathematical representations of your face, is encrypted and available only to the Secure Enclave. This data never leaves the device. It isn’t sent to Apple, nor is it included in device backups. The following Face ID data is saved, encrypted only for use by the Secure Enclave, during normal operation:
• The mathematical representations of your face calculated during enrollment.
• The mathematical representations of your face calculated during some unlock attempts if Face ID deems them useful to augment future matching.
Face images captured during normal operation aren’t saved, but are instead immediately discarded after the mathematical representation is calculated for either enrollment or comparison to the enrolled Face ID data.
For more info, read the iOS Security Guide for iOS 12 by Apple: https://www.apple.com/business/site/docs/iOS_Security_Guide.pdf
ckuan,
Again, this information does not highlight detailed functionality of Face ID working in different light conditions.
There are hints from online reviews written by people that Face ID considers that the verification image and the template image stored in the phone may have been captured under different lighting conditions. Face ID takes that into account by setting a different threshold (score between 0 and 1) for different lighting scenarios.
Can you comment whether this the actual thing happening in Face ID?
Understood ckuan. Thanks for your help.
I actually asked the author as well. However, he said it was based on what he read online. This made me curious to confirm whether - what is available online is actually true or are just some hoax stories.
It seems that the online available information about Face ID is an exaggerated version made up by some people.
You're welcome.
Thanks ckuan for your help.
However, I have already gone through this document. It has general information on Face ID working in different light conditions like: Furthermore, it's designed to work indoors, outdoors, and even in total darkness.
I need detailed technical information on how Face ID works in different light conditions to understand the functionality.
For example - I read in an article How does facial recognition on iPhone X actually work? | GearBest Blog
When you want to unlock your phone, the phone will capture the image you are offering and compare it to the template of your face that has been set earlier. The computing then relies on the score from 0 to 1 to make the final verdict. If the score approaches 1, the phone considers it as the same face and unlocks the phone. If it's close to 0, the verification process won't be successful.
Naturally, the verification image and the template image stored in the phone may have been captured under different lighting conditions. The phone takes that into account and there is a different "threshold" set for different scenarios. It is lower when you are just unlocking the phone (for instance, you can pass with a similarity score of 0,7) and higher when you are making a pricey purchase.
I did not get similar Face ID working detail in Apple's website or support community blog, so was curious whether this information is true about Face ID's functionality or just a hypothesis?
There is no template image, and there's no different lightning conditions stored. It is a mathematical representation of your face and is not an image. What algorithms /methods /hardware used are Apple proprietary.
BTW, why don't you ask the author who wrote what you read?
Face ID - Does it work in different lighting conditions?
