{"id":13918,"date":"2018-11-27T15:10:04","date_gmt":"2018-11-27T23:10:04","guid":{"rendered":"https:\/\/www.palada.net\/index.php\/2018\/11\/27\/news-7685\/"},"modified":"2018-11-27T15:10:04","modified_gmt":"2018-11-27T23:10:04","slug":"news-7685","status":"publish","type":"post","link":"http:\/\/www.palada.net\/index.php\/2018\/11\/27\/news-7685\/","title":{"rendered":"Why Malwarebytes decided to participate in AV testing"},"content":{"rendered":"<p><strong>Credit to Author: Malwarebytes Labs| Date: Tue, 27 Nov 2018 22:44:00 +0000<\/strong><\/p>\n<p>Starting this month, Malwarebytes began participating in the antivirus software for Windows <a href=\"https:\/\/www.av-test.org\/en\/antivirus\/home-windows\/windows-10\/october-2018\/malwarebytes-premium-3.5--3.6-184012\/\" target=\"_blank\" rel=\"noopener\">comparison test<\/a> performed by AV-test.org. This is uncharted territory for us, as we have refrained from participating in these types of tests since our inception. Although recent testing results show Malwarebytes protecting against more than 97 percent of web vector threats and detecting and removing 99.5 percent of malware during a scan on any machine, we still maintain reservations about the entire testing process.<\/p>\n<h3>Why participate now?<\/h3>\n<p>In the past, we\u2019ve avoided AV comparison tests because we felt their methods did not allow us to demonstrate how our product works in a real environment. By testing only a small portion of our product\u2019s technologies, AV comparison tests are often unable to replicate Malwarebytes\u2019 overall effectiveness. However, we understand the importance of independent reviews for those considering a Malwarebytes purchase, so we decided to participate.<\/p>\n<p>Malwarebytes is <a href=\"https:\/\/blog.malwarebytes.com\/101\/2015\/09\/whats-the-difference-between-antivirus-and-anti-malware\/\" target=\"_blank\" rel=\"noopener\">not a traditional antivirus<\/a>, and detecting files based on signatures\u2014which is what the testing companies review\u2014is only one of the methods we use to protect our customers from threats. We probably never will be the best performer in this category; it simply isn\u2019t our focus. We mostly rely on other methods, such as hardening, application behavior, and vector blocking defenses that disrupt malware earlier in the attack chain.<\/p>\n<h3>What did the test miss?<\/h3>\n<p>Some of our best technologies block malware before it has the chance to execute. Our application behavior and web protection modules, for example, stop threats earlier in the attack\u2014at the point of delivery instead of the point of execution. However, the URLs tested only represent the final stage of an attack (i.e. the URL pointing to the final payload EXE).<\/p>\n<p>In addition, testers often do not replicate the original infection vector used by malware campaigns, such as malspam, exploits, or redirects. Instead, they download the malware directly, bypassing typical delivery methods. By doing this, they\u00b4re controlling the environment, but also missing out on the trigger for many of our detections.<\/p>\n<h3>What exactly is checked in these monthly AV-Test.org tests?<\/h3>\n<ul>\n<li>Detections\u00a0<u><a href=\"https:\/\/www.av-test.org\/en\/about-the-institute\/test-procedures\/test-modules-under-windows-protection\/\">(specifications)<\/a><\/u>\n<ul>\n<li>Detection of URLs pointing directly to malware EXEs (i.e. \u201cweb and email threats\u201d test)<\/li>\n<li>On-demand scan of a directory full of malware EXEs (i.e. \u201cwidespread and prevalent malware\u201d test)<\/li>\n<\/ul>\n<\/li>\n<li>Performance impact, such as browsing slowdown, application load slowdown, slowdown of file copy operations, etc.<\/li>\n<li><a href=\"https:\/\/www.av-test.org\/en\/about-the-institute\/test-procedures\/test-modules-under-windows-usability\/\">Usability test<\/a>, with focus on false positives<\/li>\n<\/ul>\n<p>More information about the test procedures can be found at <a href=\"https:\/\/www.av-test.org\/en\/about-the-institute\/test-procedures\/\" target=\"_blank\" rel=\"noopener\">AV-Test.org<\/a>.<\/p>\n<h3>Unsolicited tests<\/h3>\n<p>A number of times in the past, Malwarebytes has been <a href=\"https:\/\/www.pcmag.com\/article2\/0,2817,2455577,00.asp\" target=\"_blank\" rel=\"noopener\">included in tests<\/a> that we were not aware of or in which we didn\u2019t choose to participate. Some even compared our <a href=\"https:\/\/www.mrg-effitas.com\/wp-content\/uploads\/2018\/05\/MRG-Effitas-2018Q1-360-Assessment.pdf\" target=\"_blank\" rel=\"noopener\">free, limited scanner<\/a> against fully functional AVs. No surprises there: while the other vendors may have scored higher in their detections, our free scanner still outperformed them in remediation and removal.<\/p>\n<h3>Change the tests<\/h3>\n<p>If the tests miss out on our best protection modules, you would expect us to try and change the testing methods altogether, right? We did look into this, and it\u2019s not entirely off the table. We feel sure that using live malware or duplicating real-life attacks would show our excellence, but these conditions are hard to replicate for a controlled and equal testing environment.<\/p>\n<p>What we would like to see is a test for zero-day effectiveness, and not a test based on relatively old samples and infection vectors. But again, we also understand that this is hard to achieve for a testing organization that likes to have some control over the environment and in order to create a level playing field.<\/p>\n<h3>When and where can we expect to see your test results?<\/h3>\n<p>As of November 27, 2018, AV-Test.org will include results for our flagship consumer product, Malwarebytes for Windows versions 3.5 and 3.6. AV-Test.org publishes their results publicly every two months. The November 2018 results are the summary of tests performed during September and October. Our participation is only in the <a href=\"https:\/\/www.av-test.org\/en\/antivirus\/home-windows\/\" target=\"_blank\" rel=\"noopener\">\u201cWindows Antivirus\u201d test for home users<\/a>.<\/p>\n<p>We still do not believe in the \u201cpay-to-play\u201d model, and especially the \u201cpay-to-see-what-you-missed\u201d model that some organizations use. (AV companies, for an additional fee, can see the samples they did not catch in the test and develop fixes in the product for future tests\/use.) Nonetheless, we want to give our customers some idea of what we are capable off, even when the playing field is skewed.<\/p>\n<p>We would just like you to keep in mind that, when reviewing our scores, these tests only show part of the whole picture. Many of our best protection modules have been left out of the test entirely\u2014which basically misses what Malwarebytes is truly capable of.<\/p>\n<p>So what would you rather have: a product that does well on AV tests, or a product that detects, blocks, and cleans up threats in the real world?<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/blog.malwarebytes.com\/malwarebytes-news\/2018\/11\/malwarebytes-decided-participate-av-testing\/\">Why Malwarebytes decided to participate in AV testing<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/blog.malwarebytes.com\">Malwarebytes Labs<\/a>.<\/p>\n<p><a href=\"https:\/\/blog.malwarebytes.com\/malwarebytes-news\/2018\/11\/malwarebytes-decided-participate-av-testing\/\" target=\"bwo\" >https:\/\/blog.malwarebytes.com\/feed\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p><strong>Credit to Author: Malwarebytes Labs| Date: Tue, 27 Nov 2018 22:44:00 +0000<\/strong><\/p>\n<table cellpadding='10'>\n<tr>\n<td valign='top' align='center'><a href='https:\/\/blog.malwarebytes.com\/malwarebytes-news\/2018\/11\/malwarebytes-decided-participate-av-testing\/' title='Why Malwarebytes decided to participate in AV testing'><img src='https:\/\/blog.malwarebytes.com\/wp-content\/uploads\/2018\/11\/shutterstock_469156640.jpg' border='0'  width='300px'  \/><\/a><\/td>\n<\/tr>\n<tr>\n<td valign='top' align='left'>Malwarebytes recently participated in the antivirus software for Windows comparison test performed by AV-test.org. This is uncharted territory for us, as we have had reservations about the testing process&#8217; inability to capture our best protection technologies. See how we scored, and learn why we maintain the results don&#8217;t reflect our true capabilities.<\/p>\n<p>Categories: <\/p>\n<ul class=\"post-categories\">\n<li><a href=\"https:\/\/blog.malwarebytes.com\/category\/malwarebytes-news\/\" rel=\"category tag\">Malwarebytes news<\/a><\/li>\n<\/ul>\n<p>Tags: <a href=\"https:\/\/blog.malwarebytes.com\/tag\/av-comparison-tests\/\" rel=\"tag\">AV comparison tests<\/a><a href=\"https:\/\/blog.malwarebytes.com\/tag\/av-tests\/\" rel=\"tag\">AV tests<\/a><a href=\"https:\/\/blog.malwarebytes.com\/tag\/detections\/\" rel=\"tag\">detections<\/a><a href=\"https:\/\/blog.malwarebytes.com\/tag\/infection-vector\/\" rel=\"tag\">infection vector<\/a><a href=\"https:\/\/blog.malwarebytes.com\/tag\/malware-delivery\/\" rel=\"tag\">malware delivery<\/a><a href=\"https:\/\/blog.malwarebytes.com\/tag\/test-results\/\" rel=\"tag\">test results<\/a><\/p>\n<table width='100%'>\n<tr>\n<td align=right>\n<p><b>(<a href='https:\/\/blog.malwarebytes.com\/malwarebytes-news\/2018\/11\/malwarebytes-decided-participate-av-testing\/' title='Why Malwarebytes decided to participate in AV testing'>Read more&#8230;<\/a>)<\/b><\/p>\n<\/td>\n<\/tr>\n<\/table>\n<\/td>\n<\/tr>\n<\/table>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/blog.malwarebytes.com\/malwarebytes-news\/2018\/11\/malwarebytes-decided-participate-av-testing\/\">Why Malwarebytes decided to participate in AV testing<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/blog.malwarebytes.com\">Malwarebytes Labs<\/a>.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[10488,10378],"tags":[20252,20253,20254,20255,20256,10546,20257],"class_list":["post-13918","post","type-post","status-publish","format-standard","hentry","category-malwarebytes","category-security","tag-av-comparison-tests","tag-av-tests","tag-detections","tag-infection-vector","tag-malware-delivery","tag-malwarebytes-news","tag-test-results"],"_links":{"self":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/13918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/comments?post=13918"}],"version-history":[{"count":0,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/13918\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/media?parent=13918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/categories?post=13918"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/tags?post=13918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}