![](\"https:\/\/images.idgesg.net\/images\/article\/2021\/04\/spot_analytics_04_cso_bottle_of_poison_by_arek_socha_aka_qimono_cc0-like_via_pixabay_binary_spiral_by_gordon_johnson_aka_gdj_cc0-like_via_pixabay_3x2_2400x1600_hero-100884335-small.jpg\"\/)
<\/p>\n
Credit to Author: eschuman@thecontentfirm.com| Date: Mon, 12 Feb 2024 03:00:00 -0800<\/strong><\/p>\n The IT community of late has been freaking out about AI data poisoning<\/a>. For some, it\u2019s a sneaky mechanism that could act as a backdoor into enterprise systems by \u00a0surreptitiously infecting the data large language models<\/a> (LLMs) train on and then getting \u00a0pulled into enterprise systems. For others, it\u2019s a way to combat LLMs that try to do an end run around trademark and copyright protections.<\/span><\/p>\n Put simply, these two fears amount to data poisoning being either 1) an attack tool for cyberthieves and cyberterrorists or <\/i>2) \u00a0a defense tool by artists and enterprises trying to protect their intellectual property.\u00a0<\/span><\/p>\n In reality, AI data poisoning is not much of a threat in either scenario \u2014 but IT folk do very much love to freak out.<\/span><\/p>\n It\u2019s the defense tactic that is <\/span>getting a lot of attention these days<\/span><\/a>, with people downloading a pair of freeware apps from the University of Chicago called <\/span>Nightshade<\/span><\/a> and <\/span>Glaze<\/span><\/a>.\u00a0<\/span><\/p>\n These kinds of defensive data poisoning apps work by manipulating the targeted file to trick the LLM training function. With Nightshade, it typically manipulates the code around an image. The image might be a desert scene with cactuses (or cacti, if you want to get all Latin on me), but the labeling is changed to say that it is an ocean with waves. The idea is that someone asks the LLM for ocean images, the amended image will show up. But because it is clearly a desert scene, it will be rejected.\u00a0<\/span><\/p>\n Glaze works more directly on the image, in essence cloudying it to make it less desirable. Either way, the goal is to make it less likely that the protected image is used via LLM.<\/span><\/p>\n This technique, although imaginative, is unlikely to work for long. It will not be long before LLMs will be taught how to see through these defensive techniques.\u00a0<\/span><\/p>\n \u201cTo protect your works, you have to degrade your work,\u201d said George Chedzhemov, the cybersecurity strategist at data firm BigID. \u201cI am going to place a bet that companies with billions of dollars systems and workloads, that they are more likely to prevail in this cat-and-mouse game. In the long run, I simply don\u2019t think this is going to be effective.\u201d<\/span><\/p>\n The offensive technique is potentially the more worrisome, but it is also highly unlikely to be effective, even in the short term.\u00a0<\/span><\/p>\n The offensive technique works in one of two ways. One, it tries to target a specific company by making educated guesses about the kind of sites and material they would want to train their LLMs with. The attackers then target, not that specific company, but the many places where it is likely to go for training. If the target is, let\u2019s say Nike or Adidas, the attackers might try and poison the databases at various university sports departments with high-profile sports teams. If the target were Citi or Chase, the bad guys might target databases at key Federal Reserve sites.<\/span><\/p>\n The problem is that both ends of that attack plan could easily be thwarted. The university sites might detect and block the manipulation efforts. To make the attack work, the inserted data would likely have to include malware executables, which are relatively easy to detect.<\/span><\/p>\n Even if the bad actors\u2019 goal was to simply feed incorrect data into the target systems \u2014 which would, in theory, make their analysis flawed \u2014 most LLM training absorbs such a massively large number of datasets that the attack is unlikely to work well.<\/span><\/p>\n \u201cThe planted code would end up being extremely diluted. Only a tiny amount of the malicious code would likely survive,\u201d Chedzhemov said.\u00a0<\/span><\/p>\n The other malicious AI data poisoning tactic amounts to a spray-and-pray mechanism. Instead of targeting a specific company, the bad actors would try and contaminate a massive number of sites and hope the malware somehow ends up at a company with attractive data to steal.<\/span><\/p>\n \u201cThey would need to contaminate tens of thousands of sites all over the place,\u201d Chedzhemov said. \u201cAnd then they need to hope that LLM model somehow hones in on one of them.\u201d<\/span><\/p>\n Chedzhemov argued that the only viable approach would be to \u201cpick an extremely esoteric area for which there is not a lot of stuff out there, something very niche.\u201d<\/span><\/p>\n The tech industry is quite familiar with these counter-measures and they rarely work for long, if ever. Consider antivirus programs that published definitions and then the bad guys changed the technique. Then the AV players looked for patterns instead of specific definitions, and so on. Or think of search engine spiders and their battles with robot.txt scripts that told them to go away. Or Youtube versus ad blockers.<\/span><\/p>\n LLM data poisoning is something that IT needs to be aware of and to guard against. But in this contest, I think IT has almost all of the advantages. How refreshingly rare.<\/span><\/p>\n http:\/\/www.computerworld.com\/category\/security\/index.rss<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Credit to Author: eschuman@thecontentfirm.com| Date: Mon, 12 Feb 2024 03:00:00 -0800<\/strong><\/p>\n The IT community of late has been freaking out about AI data poisoning<\/a>. For some, it\u2019s a sneaky mechanism that could act as a backdoor into enterprise systems by \u00a0surreptitiously infecting the data large language models<\/a> (LLMs) train on and then getting \u00a0pulled into enterprise systems. For others, it\u2019s a way to combat LLMs that try to do an end run around trademark and copyright protections.<\/span><\/p>\n<\/p>\n