You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

đź’ˇ Did you know?

⏺ If you can't accept iCloud Terms and Conditions... Learn more >

⏺ If you don't see your iCloud notes in the Notes app... Learn more >

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Visual lookup technology

How does visual lookup (also known as image recognition) work. I’m guessing that there is a table of image types (plants, trees, rocks, statues, etc.) in iOS and that once you opt to recognize the specific image in your photo that the iOS connects to a database for that type of image and lists what it is or the choices. Am I correct on this? Thanks.

Posted on Feb 18, 2023 7:13 AM

Reply
1 reply

Feb 18, 2023 7:47 AM in response to ChuckDVM

At its simplest, a whole lot of statistics calculations; of certain parts of images probably being certain things.


Apple has substantial machine learning research online:

https://machinelearning.apple.com/research


AFAIK, Apple hasn’t documented the technical details of Visual Look Up, though general info and related is available, as are the available application programming interfaces (APIs):

On-device APIs - Machine Learning - Apple Developer


Here is a list of what can purportedly be classified; a list of results the statistics might (correctly or otherwise) infer:

httphttps://pysource.com/2020/10/20/create-a-simple-image-classifier/


Apple has documented more about the statistical recognition of people.


Here are papers from Apple research describing details of how people are recognized:

https://machinelearning.apple.com/research/face-detection (2017)

https://machinelearning.apple.com/research/recognizing-people-photos (2021)


The statistics sometimes don’t always work as intended, either:

https://www.theguardian.com/technology/2021/mar/08/typographic-attack-pen-paper-fool-ai-thinking-apple-ipod-clip


As happened with Microsoft Tay, “Adversarial training” is a thing, too.


There are other image classifiers around from other sources that are documented:

https://pysource.com/2020/10/20/create-a-simple-image-classifier/


Yes, the term “machine learning” within computer programming is arguably another name for some forms of what mathematicians call statistics. With hardware, and side-effects.


Visual lookup technology

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.