{"id":6680,"date":"2017-02-17T13:10:30","date_gmt":"2017-02-17T21:10:30","guid":{"rendered":"http:\/\/www.palada.net\/index.php\/2017\/02\/17\/news-499\/"},"modified":"2017-02-17T13:10:30","modified_gmt":"2017-02-17T21:10:30","slug":"news-499","status":"publish","type":"post","link":"http:\/\/www.palada.net\/index.php\/2017\/02\/17\/news-499\/","title":{"rendered":"Explained: Bayesian spam filtering"},"content":{"rendered":"

Credit to Author: Pieter Arntz| Date: Fri, 17 Feb 2017 16:30:10 +0000<\/strong><\/p>\n

Bayesian spam filtering is based on Bayes rule, a statistical theorem that gives you the probability of an event. In Bayesian filtering it is used to give you the probability that a certain email is spam.<\/p>\n

The name<\/strong><\/p>\n

Named after the statistician Rev. Thomas Bayes<\/a> who provided an equation that basically allows new information to update the outcome of a probability calculation. The rule is also called the Bayes-Price rule after the mathematician Richard Price<\/a>, as he recognized the importance of the theorem, made some corrections to Bayes\u2019 work and put the rule to use.<\/p>\n

Spam<\/strong><\/p>\n

When dealing with spam the theorem is used to calculate a probability whether a certain message is spam based on words in the title and message, learning from messages that were identified as spam and messages that were identified as not being spam (sometimes called ham).<\/p>\n

False positives<\/strong><\/p>\n

The objective of the learning ability is to reduce the number of false positives. As annoying it might be to receive a spam message, it is worse to not receive a message from a customer just because he used a word that triggered the filter.<\/p>\n

Scoring<\/strong><\/p>\n

Other methods often use simple scoring filters. If a message contains specific words a few points are added to that messages\u2019 score and when it exceeds a\u00a0 certain score, the message is regarded as spam. Not only is this a very arbitrary method, it\u2019s also a given that this will result in spammers changing their wording. Take for example \u201cViagra\u201d which is a word that will surely give you a high score. As soon as spammers found that out they switched to variations like \u201cV!agra\u201d and so on. A cat and mouse game that will keep you busy creating new rules.<\/p>\n

Learning <\/strong><\/p>\n

If the filtering is allowed for individual input the precision can be enhanced on a per-user base. Different users may attract specific forms of spam based on their online activities. Or what is spam to one person is a \u201cmust-read\u201d newsletter to the next. Every time the user confirms or denies that a message is spam, the filtering process can calculate a more refined probability for the next occasion.<\/p>\n

Poisoning<\/b><\/p>\n

A downside of Bayesian filtering in cases of more or less targeted spam is that spammers will start using words or whole pieces of text that will lower the score. During prolonged use, these words might get associated with spam, which is called poisoning.<\/p>\n

Bypasses<\/strong><\/p>\n

A few methods to bypass \u201cbad word\u201d filtering.<\/p>\n