{"id":20960,"date":"2023-01-10T07:21:04","date_gmt":"2023-01-10T15:21:04","guid":{"rendered":"https:\/\/www.palada.net\/index.php\/2023\/01\/10\/news-14693\/"},"modified":"2023-01-10T07:21:04","modified_gmt":"2023-01-10T15:21:04","slug":"news-14693","status":"publish","type":"post","link":"http:\/\/www.palada.net\/index.php\/2023\/01\/10\/news-14693\/","title":{"rendered":"Artificial intelligence now a match for natural ignorance"},"content":{"rendered":"

Credit to Author: Chester Wisniewski| Date: Tue, 10 Jan 2023 14:00:07 +0000<\/strong><\/p>\n

\n

End-user security training has traditionally been an important aspect of cybersecurity efforts, as it aims to educate and empower employees to identify and prevent potential threats. However, with the advancement of artificial intelligence (AI), end-user security training is becoming less effective in protecting organizations from cyber attacks. This is because AI-powered threats are becoming more sophisticated and harder to detect, making it difficult for even well-trained employees to spot them. As a result, organizations are finding that traditional end-user security training is becoming less effective in protecting against these types of threats.<\/p>\n

So what does this mean? I think the days of relying on our end users to play a role in defending our organizations against inbound threats is over. For years we have lectured our staff on how to spot phishy links<\/a>, look for padlocks in their browsers<\/a>, and steer clear of scary Wi-Fi<\/a>. Somehow, despite all this, ransomware remains at an all-time high and data breach notifications are more common than AOL CDs in the 90s.<\/p>\n

One of the last remaining effective elements of training programs instructed users to watch for grammar and spelling errors or text that didn\u2019t \u201csound right.\u201d This might be especially effective against business email compromise (BEC) attacks, where the recipient may know the compromised requestor quite well.<\/p>\n

Enter ChatGPT<\/a>, an artificial intelligence from OpenAI with which you can interact and ask to do things. The current iteration is using a training model called GPT-3.5 and it is creepily good at spewing out believable \u2013and sometimes even accurate\u2014responses. When asked for lengthy responses, it tends to go a little off the rails, but for shorter-form questions and written text in English, it is excellent.<\/p>\n

Will tools like ChatGPT remove the last detectable element of many of the scams, spams, and phishes we already struggle with? I think it might. Let\u2019s look at a few examples.<\/p>\n

Here is a lure from a BEC scammer trying to redirect someone\u2019s paycheck to the attacker\u2019s account. On the top is the original handwritten lure from the attacker; on the bottom is one I asked ChatGPT to write.<\/p>\n

\"\"<\/a><\/p>\n

\"\"<\/a><\/p>\n

Not too bad. Sounds like an email I might write. Good punctuation, spelling, grammar…<\/p>\n

Next let\u2019s take a look at a gift card scam. Again, the top message is from a real scammer, the bottom from ChatGPT.<\/p>\n

\"\"<\/a><\/p>\n

\"\"<\/a><\/p>\n

Are these perfect? No. Are they good enough? I would have to say yes, as the scammers are already making millions with their shoddily crafted lures. Imagine if you were chatting with this bot over WhatsApp or Microsoft Teams. Would you know?<\/p>\n

In an attempt to understand this better, I reached out to Konstantin Berlin, Head of AI for Sophos X-Ops. When I asked if AI has developed capabilities that are no longer easily detectable to humans he replied bluntly, \u201cThey\u2019ll probably need help.\u201d<\/p>\n

All sorts of applications of AI are already to a point where they can fool a human being nearly 100% of the time. The \u201cconversation\u201d you can have with ChatGPT is remarkable and the ability to generate fake human faces that are nearly indiscernible (by humans) from real photos is already upon us. Need a fake company to perpetrate a scam? No problem. Generate 25 faces and use ChatGPT to write their biographies. Open a few fake LinkedIn accounts and you\u2019re good to go.<\/p>\n

While there has been a lot of press and fussing about deepfake audio and video, that at least isn\u2019t quite trivial to do yet, though it is only a matter of time. Konstantin implied that it is around the corner and probably it should be on our radar to mitigate soon.<\/p>\n

What do we do about all of this? According to Konstantin, \u201cyou\u2019re just going to have to be armed.\u201d Like my last post<\/a> on the proliferation of domain names being used by SaaS vendors and how it\u2019s moved beyond practical to educate users on good and bad, we need to turn to technology to give us a fighting chance.<\/p>\n

We will all need to don our Iron Man suits when braving the increasingly dangerous waters of the internet. Increasingly this is looking like we will need machines to identify when other machines are trying to fool us. An interesting proof of concept has been developed by Hugging Face that can detect text generated using GPT-2<\/a>*, which suggests similar techniques could be used to detect GPT-3 output.<\/p>\n

Yes, I\u2019m saying it: AI has put the final nail in the end-user security awareness coffin. Am I suggesting we stop doing it entirely? No, but we need to do a hard reset on our expectations. Just as security through obscurity isn\u2019t a strategy to rely on, it still doesn\u2019t hurt to try.<\/p>\n

We need to teach users to be suspicious and to verify communications that involve access to information or have monetary elements. Ask questions, ask for help, and take the extra few moments necessary to confirm things are truly as they seem. We\u2019re not being paranoid; they really are after us.<\/p>\n

P.S. The first paragraph was written by ChatGPT3 using the following queries: \u201cWrite an introductory paragraph explaining why end-user security training will is [sic] becoming unhelpful because of artificial intelligence,\u201d followed up with \u201cremove the part about ai powered defenses.\u201d The illustration at the top of the article was likewise generated by AI. In that case, DALL-E was given the phrase \u201cA photograph of a robot standing by the open grave of its human master, holding metal flowers.\u201d The resulting image was amended with generated extensions to fit the required horizontal format.<\/p>\n

 <\/p>\n

* Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., … & Wang, J. (2019). Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203.<\/p>\n<\/p><\/div>\n

http:\/\/feeds.feedburner.com\/sophos\/dgdY<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/p>\n

Credit to Author: Chester Wisniewski| Date: Tue, 10 Jan 2023 14:00:07 +0000<\/strong><\/p>\n

Language-generation tools such as ChatGPT have changed security training forever, but how will your end users know what\u2019s no longer working?<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[10378,10377],"tags":[24552],"class_list":["post-20960","post","type-post","status-publish","format-standard","hentry","category-security","category-sophos","tag-security-operations"],"_links":{"self":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/20960","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/comments?post=20960"}],"version-history":[{"count":0,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/20960\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/media?parent=20960"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/categories?post=20960"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/tags?post=20960"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}