It’s becoming really hard to tell what was written by an actual person, and what was written by tools like ChatGPT, and has led to students being falsely accused of using ChatGPT<\/a>. However, students that are<\/em> using those tools shouldn’t be receiving grades that they don’t deserve.<\/p>\n Worse than that could be an influx of so-called scientific articles that either add nothing new or bring \u201challucination<\/a>\u201d to the table\u2014where LLMs make up \u201cfacts\u201d that are untrue.<\/p>\n Several programs that can filter out artificial intelligence (AI) texts have been created and tests are ongoing, but the success rate of these, mostly AI-based tools, hasn’t been great.<\/p>\n Many have found the existing detection tools not very effective, especially for professional academic writing. These tools have a bias against non-native speakers. Seven common web-based AI detection tools all identified non-native English writers\u2019 works as AI-generated text more frequently than native English speakers\u2019 writing.<\/p>\n But now it seems as if chemistry scientists have found an important building block in creating more effective detection tools. In a paper titled \u201cAccurately detecting AI text when ChatGPT is told to write like a chemist<\/a>\u201d they describe how they developed and tested an accurate AI text detector for scientific journals.<\/p>\n Using machine learning (ML)<\/a>, the detector examines 20 features of writing style, including variation in sentence lengths, the frequency of certain words, and the use of punctuation marks, to determine whether an academic scientist or ChatGPT wrote the examined text.<\/p>\n To test the accuracy of the detector, the scientists tested it against 200 introductions in American Chemical Society (ACS) journal style. For 100 of these, the tool was provided with the papers\u2019 titles, and for the other 100, it was given their abstracts.<\/p>\n It showed astonishing results. It outperformed the online tools provided by ZeroGPT and OpenAI by identifying ChatGPT-3.5 and ChatGPT-4 written sections based on titles with 100% accuracy. For the ChatGPT-generated introductions based on abstracts, the accuracy was slightly lower, at 98%.<\/p>\n Image courtesy of<\/em> ScienceDirect<\/em><\/p>\n The graph shows the accuracy of three detectors against texts written by humans (to determine the number of false positives), ChatGPT-3.5, and ChatGPT-4. P1 is the texts based on titles and P2 the ones based on abstracts.<\/p>\n What\u2019s important about this research is that it shows that with specialized tools one can achieve a much better detection rate. That could mean that efforts to develop AI detectors could receive a significant boost by tailoring software to specific types of writing.<\/p>\n Once we learn how to quickly and easily build such a specialized tool, we can soon expand the number of areas for which we have specialized detectors. According to one of the researchers, the findings show that \u201cyou could use a small set of features to get a high level of accuracy.\u201d<\/p>\n To put this into perspective, the development time to generate the detector was a part-time project, done in approximately one month by a few people. The scientists designed the detector prior to the release of ChatGPT-4, but it works just as effectively on GPT-3.5, so it’s unlikely that future versions would create text in a way that would significantly change the accuracy of this detector.<\/p>\n We don\u2019t just report on threats\u2014we remove them<\/strong><\/p>\n Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today<\/a>.<\/p>\n![\"chart](\"https:\/\/www.malwarebytes.com\/wp-content\/uploads\/sites\/2\/2023\/11\/comparison.jpg\")
<\/figure>\n
\n