![](\"https:\/\/media.wired.com\/photos\/5a39c0a4dd6e67710f5871a4\/master\/pass\/AI-MIT-144230438.jpg\"\/)
<\/p>\n<\/p>\n
Credit to Author: Louise Matsakis| Date: Wed, 20 Dec 2017 17:07:16 +0000<\/strong><\/p>\n Tech giants love <\/span>to tout how good their computers are at identifying what\u2019s depicted in a photograph. In 2015, deep learning algorithms designed by Google, Microsoft<\/a>, and China\u2019s Baidu<\/a> superseded humans at the task, at least initially<\/a>. This week, Facebook announced<\/a> that its facial-recognition technology is now smart enough to identify a photo of you, even if you\u2019re not tagged in it.<\/p>\n But algorithms, unlike humans, are susceptible to a specific type of problem called an \u201cadversarial example<\/a>.\u201d These are specially designed optical illusions that fool computers into doing things like mistake<\/a> a picture of a panda for one of a gibbon. They can be images, sounds, or paragraphs of text. Think of them as hallucinations for algorithms.<\/p>\n While a panda-gibbon mix-up may seem low stakes, an adversarial example could thwart the AI system that controls a self-driving car, for instance, causing it to mistake<\/a> a stop sign for a speed limit one. They\u2019ve already been used to beat other kinds of algorithms, like spam filters.<\/p>\n Those adversarial examples are also much easier to create than was previously understood, according to research released Wednesday from MIT\u2019s Computer Science and Artificial Intelligence Laboratory. And not just under controlled conditions; the team reliably fooled Google\u2019s Cloud Vision API<\/a>, a machine learning algorithm used in the real word today.<\/p>\n An adversarial example could thwart the AI system that controls a self-driving car, causing it to mistake a stop sign for a speed limit one.<\/p>\n Previous adversarial examples have largely been designed<\/a> in \u201cwhite box\u201d settings, where computer scientists have access to the underlying mechanics that power an algorithm. In these scenarios, researchers learn how the computer system was trained, information that helps them figure out how to trick it. These kinds of adversarial examples are considered less threatening, because they don\u2019t closely resemble the real world, where an attacker wouldn\u2019t have access to a proprietary algorithm.<\/p>\n For example, in November another team at MIT (with many of the same researchers) published a study<\/a> demonstrating how Google\u2019s InceptionV3<\/a> image classifier could be duped into thinking that a 3-D-printed turtle was a rifle. In fact, researchers could manipulate the AI into thinking the turtle was any object they wanted. While the study demonstrated that adversarial examples can be 3-D objects, it was conducted under white-box conditions. The researchers had access to how the image classifier worked.<\/p>\n But in this latest study, the MIT researchers did their work under \u201cblack box\u201d conditions, without that level of insight into the target algorithm. They designed a way to quickly generate black-box adversarial examples that are capable of fooling different algorithms, including Google\u2019s Cloud Vision API. In Google\u2019s case, the MIT researchers targeted the part of the system of that assigns names to objects, like labeling a photo of a kitten \u201ccat.\u201d<\/p>\n What it looks like when MIT’s system attacks Google’s algorithm.<\/p>\n Despite the strict black box conditions, the researchers successfully tricked Google\u2019s algorithm. For example, they fooled it into believing a photo of a row of machine guns was instead a picture of a helicopter, merely by slightly tweaking the gradient and light in the photo. To the human eye, the two images look identical. The indiscernible difference only fools the machine.<\/p>\n The researchers didn\u2019t just tweak the photos randomly. They targeted the AI system using a standard method. Each time they tried to fool the AI, they analyzed their results, and then intelligently inched toward an image that could trick a computer into thinking a gun (or any other object) is something it isn\u2019t.<\/p>\n The researchers randomly generated their labels; in the rifle example, the classifier \u201chelicopter\u201d could just as easily have been \u201cantelope.\u201d They wanted to prove that their system worked, no matter what labels were chosen. \u201cWe can do this given anything. There\u2019s no bias, we didn\u2019t choose what was easy,\u201d says Anil Athalye, a PhD student at MIT and one of the lead authors of the paper. Google declined to comment in time for publication.<\/p>\n What Google’s algorithm originally "saw."<\/p>\n What the algorithm "saw" after MIT’s researchers turned the image into an adversarial example.<\/p>\n MIT\u2019s latest work demonstrates that attackers could potentially create adversarial examples that can attack commercial AI systems. Google is generally considered to have one of the best security teams in the world, but one of its most futuristic products is subject to hallucinations. These kinds of attacks could one day be used to, say, dupe a luggage-scanning algorithm into thinking an explosive is a teddy bear, or a facial-recognition system into thinking the wrong person<\/a> committed a crime.<\/p>\n It\u2019s at least, though, a concern Google is working on; the company has published research<\/a> on the issue, and even held an adversarial example competition. Last year, researchers from Google, Pennsylvania State University, and the US Army documented<\/a> the first functional black box attack on a deep learning system, but this fresh research from MIT uses a faster, new method for creating adversarial examples.<\/p>\n 'We can do this given anything. There\u2019s no bias, we didn\u2019t choose what was easy.'<\/p>\n Anil Athalye, MIT CSAIL<\/p>\n These algorithms are being entrusted to tasks like filtering out hateful content on social platforms, steering driverless cars, and maybe one day scanning luggage<\/a> for weapons and explosives. That\u2019s a tremendous responsibility, given that don\u2019t yet fully understand why adversarial examples cause deep learning algorithms to go haywire.<\/p>\n There are some hypotheses, but nothing conclusive, Athalye told me. Researchers have essentially created artificially intelligent systems that \u201cthink\u201d in different ways than humans do, and no one is quite sure how they work. \u201cI can show you two images that look exactly the same to you,\u201d Athalye says. \u201cAnd yet the classifier thinks one is a cat and one is a guacamole with 99.99 percent probability.\u201d<\/p>\n