![](\"https:\/\/media.wired.com\/photos\/5ccccce48cb4955f51aeb0ea\/master\/pass\/Ai-Image-Detection_1.gif\"\/)
<\/p>\n<\/p>\n
Credit to Author: Louise Matsakis| Date: Wed, 08 May 2019 14:39:59 +0000<\/strong><\/p>\n Thanks to advances <\/span>in machine learning<\/a>, computers have gotten really good at identifying what\u2019s in photographs. They started beating humans<\/a> at the task years ago, and can now even generate fake images<\/a> that look eerily real. While the technology has come a long way, it\u2019s still not entirely foolproof. In particular, researchers have found that image detection algorithms remain susceptible to a class of problems called adversarial examples<\/a>.<\/p>\n Adversarial examples are like optical (or audio<\/a>) illusions for AI. By altering a handful of pixels, a computer scientist can fool a machine learning classifier into thinking, say, a picture of a rifle is actually one of a helicopter<\/a>. But to you or me, the image still would look like a gun\u2014it almost seems like the algorithm is hallucinating<\/a>. As image recognition technology is used in more places, adversarial examples may present a troubling security risk. Experts have shown they can be used to do things like cause a self-driving car to ignore a stop sign<\/a>, or make a facial recognition system falsely identify<\/a> someone.<\/p>\n Organizations like Google<\/a> and the US Army<\/a> have studied adversarial examples, but what exactly causes them is still largely a mystery. Part of the problem is that the visual world is incredibly complex, and photos can contain millions of pixels. Another issue is deciphering whether adversarial examples are a product of the original photographs, or how an AI is trained to look at them. Some researchers have hypothesized they are a high-dimensional statistical phenomenon<\/a>, or caused when the AI isn\u2019t trained on enough data<\/a>.<\/p>\n Now, a leading group of researchers from MIT have found a different answer, in a paper<\/a> that was presented earlier this week: adversarial examples only look like hallucinations to people<\/em>. In reality, the AI is picking up on tiny details that are imperceptible to the human eye. While you might look at an animal\u2019s ears to differentiate a dog from a cat, AI detects minuscule patterns in the photo\u2019s pixels and uses those to classify it. \u201cThe only thing that makes these features special is that we as humans are not sensitive to them,\u201d says Andrew Ilyas, a PhD student at MIT and one of the lead authors of the work, which has yet to be peer-reviewed.<\/p>\n The explanation makes intuitive sense, but is difficult to document because it\u2019s hard to untangle which features an AI uses to classify an image. To conduct their study, the researchers used a novel method to separate \u201crobust\u201d characteristics of images, which humans can often perceive, from the \u201cnon-robust\u201d ones that only an AI can detect. Then in one experiment, they trained a classifier using an intentionally mismatched dataset of images. According to the robust features\u2014i.e., what the pictures looked like to the human eye\u2014the photos were of dogs. But according to the non-robust features, invisible to us, the photos were in fact of cats, and that\u2019s how the classifier was trained\u2014to think the photos were of kitties.<\/p>\n The researchers then tested showing the classifier new, normal pictures of cats it hadn\u2019t seen before. It was able to identify the kitties correctly, indicating the AI was relying on the hidden, non-robust features embedded in the training set. That suggests these invisible characteristics represent real patterns in the visual world, just ones that humans can\u2019t see. And adversarial examples are instances where these patterns don\u2019t line up with how we view the world.<\/p>\n When algorithms fall for an adversarial example, they\u2019re not hallucinating\u2014they\u2019re seeing something that people don\u2019t. \u201cIt\u2019s not something that the model is doing weird, it\u2019s just that you don\u2019t see these things that are really predictive,\u201d says Shibani Santurkar, a PhD student at MIT and another lead author on the paper. \u201cIt\u2019s about humans not being able to see these things in the data.\u201d<\/p>\n The study calls into question whether computer scientists can really explain<\/a> how their algorithms make decisions. \u201cIf we know that our models are relying on these microscopic patterns that we don\u2019t see, then we can\u2019t pretend that they are interpretable in a human fashion,\u201d says Santurkar. That may be problematic, say, if someone needs to prove in court that a facial recognition algorithm identified them incorrectly<\/a>. There might not be a way to account for why the algorithm thought they were a person they\u2019re not.<\/p>\n Engineers may ultimately need to make a choice between building automated systems that are the most accurate, versus ones that are the most similar to humans. If you force an algorithm to rely solely on robust features, there\u2019s a chance it might make more mistakes than if it also used hidden, non-robust ones. But if the AI also leans on those invisible characteristics, it may be more susceptible to attacks like adversarial examples. As image recognition tech is increasingly used for tasks like identifying hate speech<\/a> and scanning luggage<\/a> at the airport, deciding how to navigate these kinds of trade offs will only become more important.<\/p>\n