{"id":24378,"date":"2024-04-24T03:30:08","date_gmt":"2024-04-24T11:30:08","guid":{"rendered":"http:\/\/www.palada.net\/index.php\/2024\/04\/24\/news-18108\/"},"modified":"2024-04-24T03:30:08","modified_gmt":"2024-04-24T11:30:08","slug":"news-18108","status":"publish","type":"post","link":"https:\/\/www.palada.net\/index.php\/2024\/04\/24\/news-18108\/","title":{"rendered":"How to read encrypted messages from ChatGPT and other AI chatbots | Kaspersky official blog"},"content":{"rendered":"

Credit to Author: Alanna Titterington| Date: Wed, 24 Apr 2024 11:27:49 +0000<\/strong><\/p>\n

Israeli researchers from Offensive AI Lab have published a paper<\/a> describing a method for restoring the text of intercepted AI chatbot messages. Today we take a look at how this attack works, and how dangerous it is in reality.<\/p>\n

What information can be extracted from intercepted AI chatbot messages?<\/h2>\n

Naturally, chatbots send messages in encrypted form. All the same, the implementation of large language models<\/a> (LLMs) and the chatbots built on them harbors a number of features that seriously weaken the encryption. Combined, these features make it possible to carry out a side-channel attack<\/a> when the content of a message is restored from fragments of leaked information.<\/p>\n

To understand what happens during this attack, we need to dive a little into the details of LLM and chatbot mechanics. The first<\/strong> thing to know is that LLMs operate not on individual characters or words as such, but on tokens, which can be described as semantic units of text. The Tokenizer<\/a> page on the OpenAI website offers a glimpse into the inner workings.<\/p>\n

\"Example<\/a><\/p>\n

This example demonstrates how message tokenization works with the GPT-3.5 and GPT-4 models. Source<\/a><\/p>\n<\/div>\n

The second<\/strong> feature that facilitates this attack you’ll already know about if you’ve interacted with AI chatbots yourself: they don’t send responses in large chunks but gradually\u00a0\u2014 almost as if a person were typing them. But unlike a person, LLMs write in tokens \u2014 not individual characters. As such, chatbots send generated tokens in real time, one after another; or, rather, most chatbots do: the exception is Google Gemini, which makes it invulnerable to this attack.<\/p>\n

The third<\/strong> peculiarity is the following: at the time of publication of the paper, the majority of chatbots didn’t use compression, encoding or padding<\/a> (appending garbage data to meaningful text to reduce predictability and increase cryptographic strength) before encrypting a message.<\/p>\n

Side-channel attacks exploit all three of these peculiarities. Although intercepted chatbot messages can’t be decrypted<\/em>, attackers can extract useful data from them\u00a0\u2014 specifically, the length of each token sent by the chatbot. The result is similar to a Wheel of Fortune puzzle: you can’t see what exactly is encrypted, but the length of the individual words<\/span> tokens is revealed.<\/p>\n

\"Attackers<\/a><\/p>\n

While it’s impossible to decrypt the message, the attackers can extract the length of the tokens sent by the chatbot; the resulting sequence is similar to a hidden phrase in the Wheel of Fortune show. Source<\/a><\/p>\n<\/div>\n

Using extracted information to restore message text<\/h2>\n

All that remains is to guess what words are hiding behind the tokens. And you’ll never believe who’s good at guessing games: that’s right \u2014 LLMs<\/a>. In fact, this is their primary purpose in life: to guess the right words in the given context. So, to restore the text of the original message from the resulting sequence of token lengths, the researchers turned to an LLM\u2026<\/p>\n

Two LLMs, to be precise, since the researchers observed that the opening exchanges in conversations with chatbots are almost always formulaic, and thus readily guessable by a model specially trained on an array of introductory messages generated by popular language models. Thus, the first model is used to restore the introductory messages and pass them to the second model, which handles the rest of the conversation.<\/p>\n

\"Overview<\/a><\/p>\n

General scheme of the attack. Source<\/a><\/p>\n<\/div>\n

This produces a text in which the token lengths correspond to those in the original message. But specific words are brute-forced with varying degrees of success. Note that a perfect match between the restored message and the original is rare\u00a0\u2014 it usually happens that a part of the text is guessed wrong. Sometimes the result is satisfactory:<\/p>\n

\"Example<\/a><\/p>\n

In this example, the text was restored quite close to the original. Source<\/a><\/p>\n<\/div>\n

But in an unsuccessful case, the reconstructed text may have little, or even nothing, in common with the original. For example, the result might be this:<\/p>\n

\"Example<\/a><\/p>\n

Here the guesswork leaves much to be desired. Source<\/a><\/p>\n<\/div>\n

Or even this:<\/p>\n

\"Example<\/a><\/p>\n

As Alice once said, “those are not the right words.” Source<\/a><\/p>\n<\/div>\n

In total, the researchers examined over a dozen AI chatbots, and found most of them vulnerable to this attack\u00a0\u2014 the exceptions being Google Gemini (n\u00e9e Bard) and GitHub Copilot (not to be confused with Microsoft Copilot).<\/p>\n

\"List<\/a><\/p>\n

At the time of publication of the paper, many chatbots were vulnerable to the attack. Source<\/a><\/p>\n<\/div>\n

Should I be worried?<\/h2>\n

It should be noted that this attack is retrospective. Suppose someone took the trouble to intercept and save your conversations with ChatGPT (not that easy, but possible), in which you revealed some awful secrets. In this case, using the above-described method, that someone would theoretically<\/em> be able to read the messages.<\/p>\n

Thankfully, the interceptor’s chances are not too high: as the researchers note, even the general topic of the conversation was determined only 55% of the time. As for successful reconstruction, the figure was a mere 29%. It’s worth mentioning that the researchers’ criteria for a fully successful reconstruction were satisfied, for example, by the following:<\/p>\n

\"Example<\/a><\/p>\n

Example of a text reconstruction that the researchers considered fully successful. Source<\/a><\/p>\n<\/div>\n

How important such semantic nuances are \u2014 decide for yourself. Note, however, that this method will most likely not extract any actual specifics (names, numerical values, dates, addresses, contact details, other vital<\/em> information) with any degree of reliability.<\/p>\n

And the attack has one other limitation that the researchers fail to mention: the success of text restoration depends greatly on the language the intercepted messages are written in: the success of tokenization varies greatly from language to language. This paper was focused on English, which is characterized by very long tokens that are generally equivalent to an entire word. Hence, tokenized English text shows distinct patterns that make reconstruction relatively straightforward.<\/p>\n

No other language comes close. Even for those languages in the Germanic and Romance groups, which are the most akin to English, the average token length is 1.5\u20132 times shorter; and for Russian, 2.5 times: a typical Russian token is only a couple of characters long, which will likely reduce the effectiveness of this attack down to zero.<\/p>\n

\n
\n
\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t\t\"Examples \t\t\t\t\t\t\t\t\t<\/a> \t\t\t\t\t\t\t\t<\/dt>\n
\t\t\t\t\t\t\t\t\tTexts in different languages are tokenized differently. An English sample \t\t\t\t\t\t\t\t<\/dd>\n<\/dl>\n
\n
\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t\t\"Examples \t\t\t\t\t\t\t\t\t<\/a> \t\t\t\t\t\t\t\t<\/dt>\n
\t\t\t\t\t\t\t\t\tTexts in different languages are tokenized differently. A German sample \t\t\t\t\t\t\t\t<\/dd>\n<\/dl>\n
\n
\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t\t\"Examples \t\t\t\t\t\t\t\t\t<\/a> \t\t\t\t\t\t\t\t<\/dt>\n
\t\t\t\t\t\t\t\t\tTexts in different languages are tokenized differently. A Russian sample \t\t\t\t\t\t\t\t<\/dd>\n<\/dl>\n
\n
\t\t\t\t\t\t\t\t\t \t\t\t\t\t\t\t\t\t\t\"Examples \t\t\t\t\t\t\t\t\t<\/a> \t\t\t\t\t\t\t\t<\/dt>\n
\t\t\t\t\t\t\t\t\tTexts in different languages are tokenized differently. A Hebrew sample \t\t\t\t\t\t\t\t<\/dd>\n<\/dl>\n<\/div>\n

At least two AI chatbot developers\u00a0\u2014 Cloudflare and OpenAI\u00a0\u2014 have already reacted to the paper by adding the padding method mentioned above, which was designed specifically with this type of threat in mind. Other AI chatbot developers are set to follow suit, and future communication with chatbots will, fingers crossed, be safeguarded against this attack.<\/p>\n


https:\/\/blog.kaspersky.com\/feed\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/p>\n

Credit to Author: Alanna Titterington| Date: Wed, 24 Apr 2024 11:27:49 +0000<\/strong><\/p>\n

Researchers have developed a method for reading messages intercepted from OpenAI ChatGPT, Microsoft Copilot, and other AI chatbots. We explain how it works.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[10425,10378],"tags":[10245,5313,13431,28405,31305,11047,10439,12038,10516,30663,714,1331,10438],"class_list":["post-24378","post","type-post","status-publish","format-standard","hentry","category-kaspersky","category-security","tag-ai","tag-attacks","tag-chatbots","tag-chatgpt","tag-copilot","tag-cryptography","tag-encryption","tag-machine-learning","tag-microsoft","tag-openai","tag-security","tag-technology","tag-threats"],"_links":{"self":[{"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/24378","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/comments?post=24378"}],"version-history":[{"count":0,"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/24378\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/media?parent=24378"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/categories?post=24378"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/tags?post=24378"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}