VB100 Comparative Review – April 2017

Martijn Grooten

Virus Bulletin

Copyright © 2017 Virus Bulletin


 

Introduction

There are many common misconceptions about anti-virus (or anti-malware) software. For example, there is the belief that it protects your computer against all threats and, at the other end of the scale, the belief that, in practice, it rarely protects your machine. Both are wrong, and both can have harmful consequences if you take them to be the truth.

Anti-malware should always be used as part of a layered approach against malware threats, in which it is both the first and last line of defence. As the first line of defence, it can block a lot of threats simply by scanning files that are downloaded or copied onto a device, and as the last line of defence, if everything else has failed, its runtime protection will often stop the threat.

Virus Bulletin's anti-malware tests have always focused on the first-line aspect: how well does a solution protect users against malware that it is asked to scan, or that is copied onto a device under its control? The VB100 award is earned by products that block all files from the WildList (malware reported as having been seen in the wild by security professionals), while generating no false positives when scanning Virus Bulletin's large and regularly updated collection of clean files.

VB100 is a baseline award: the standards it requires are the minimum one should expect of a well-performing product. It is also an award that a good product should achieve regularly – thus it is also important to check a product's VB100 history. A product that achieves a VB100 award all or at least most of the time can be assumed to be a decent anti‑virus product.

In this test, we put 30 solutions from 27 different vendors on our test bench; 21 of them achieved a VB100 award, thus not only showing that there are plenty of good anti-malware solutions on the market, but also justifying vendors' efforts to distinguish their products in other ways.

Changes to the VB100 testing set-up

As the threat landscape continues to change, we have made some changes to our VB100 test set-up.

Tests are still run every two months, but rather than changing the platform on which the test is run each time, we now test every product every time on the two most popular desktop operating systems – currently, these are Windows 7 and Windows 10. A product earns VB100 certification if, on both platforms, it blocks all files from the WildList, and doesn't generate any false positives when scanning the full clean set.

As on-demand scanning for malware has become increasingly rarely used, the test now focuses solely on on‑access scanning of malware. In practice, this means that both malware samples and clean files are copied into a directory on a machine onto which the anti-malware product has been installed, with its latest updates downloaded and, generally, with a connection to the cloud.

Testing on two operating systems rather than one means that we have had to focus on the automation of the tests, as well as on reducing the number of indicators we report; we believe this makes the reports easier to read and to digest, which is important given the large number of available solutions.

This is thus the first of the new-style bi-monthly reports. It is also the first test report written by a new author, which wouldn't be complete without a thank you to my predecessor, John Hawes, for the 64 VB100 reports he put together, during the course of which he tested almost 2,000 products. It will not be easy to fill John's shoes, but thanks to the work of a great team, I expect both the test results and the reports that go with them to be just as interesting.

Results

In the following results, the RAP images display an average of the RAP scores across the two platforms.

 

adaware antivirus free

  Windows 7 Windows 10
Main version 12.0.649.11190 12.0.649.11190
Update versions 7.70283, 7.71133, 7.71177, 7.71067 7.70283, 7.71083, 7.71108, 7.71044
ItW catch rate 100.00% 100.00%
False positives 0 0
adaware-free-Apr17.gif vb100-04-17.jpg

 

 

adaware antivirus pro

  Windows 7 Windows 10
Main version 12.0.649.11190 12.0.649.11190
Update versions 7.70284, 7.71150, 7.71089 7.70284, 7.71091, 7.71118, 7.71051
ItW catch rate 100.00% 100.00%
False positives 0 0
adaware-pro-Apr17.gif vb100-04-17.jpg

 

 

Arcabit Antivirus

  Windows 7 Windows 10
Main version 2017.03.20 2017.03.20
Update versions 2017.04.28, 2017.04.27, 2017.04.06 2017.04.26, 2017.04.27, 2017.04.06
ItW catch rate 100.00% 100.00%
False positives 0 0
Arcabit-Apr17.gif vb100-04-17.jpg

 

 

Avast Free Antivirus

  Windows 7 Windows 10
Main version 17.2.2288 17.2.2288
Update versions 17032000, 17032200, 17.4.2294/17050500, 17.3.2291/17040602 17032000, 17032105, 17.3.2290/17032801, 17.3.2291/17040602
ItW catch rate 100.00% 100.00%
False positives 0 0
Avast-Apr17.gif vb100-04-17.jpg

 

 

AVG Internet Security

  Windows 7 Windows 10
Main version 17.2.3008 17.2.3008
Update versions 17032000, 17.3.3011/17042700, 17.3.3011/17042402, 17040602/17040602 17032000, 17.3.3011/17050400, 17042600, 17040602
ItW catch rate 100.00% 100.00%
False positives 0 0
AVG-Apr17.gif vb100-04-17.jpg

 

 

Bitdefender GravityZone Security for Endpoints

  Windows 7 Windows 10
Main version 6.2.18.884 6.2.18.884
Update versions 7.70282, 7.70325, 7.70474, 6.2.19.894/7.70667 7.70282, 7.70323, 6.2.19.899/7.71098, 6.2.19.894/7.70671
ItW catch rate 100.00% 100.00%
False positives 0 0
Bitdefender-Apr17.gif vb100-04-17.jpg

 

 

BullGuard Antivirus

  Windows 7 Windows 10
Main version 17.0.330.2 17.0.330.2
Update versions 16.0.0.87 16.0.0.88, 17.0.330.2, 17.0.330.2, 17.0.331.2
ItW catch rate 100.00% 100.00%
False positives 0 0
BullGuard-Apr17.gif vb100-04-17.jpg

 

 

CompuClever Antivirus Plus

  Windows 7 Windows 10
Main version 19.6.0.326 19.6.0.326
Update versions 7.70284/7877508, 7.70327/7875994, 7.70480/7970995, 7.71041/8688235 7.70284/7877508, 7.70320/7876235, 7.71092/8739627, 7.70707/8354620
ItW catch rate 100.00% 100.00%
False positive rate 0 0
Compuclever-Apr17.gif vb100-04-17.jpg

 

 

Cyren Command Anti-Malware

  Windows 7 Windows 10
Main version 5.1.38 5.1.38
Update versions 5.4.25/201703201028, 5.4.25/201703221959, 5.4.25/201703291153, 5.4.25/201704071514 5.4.25
ItW catch rate 100.00% 100.00%
False positives 35 35
Cyren-Apr17.gif VB100-fail.jpg

 

 

Defenx Security Suite

   Windows 7 Windows 10
Main version 15.1.0103   15.1.0103
Update versions 10.6.22769, 15.1.0104/10.10.23222, 10.7.22847, 15.1.0104/10.8.22943 10.6.22769, 15.1.0104/10.9.23160, 15.1.0103/10.7.22847, 15.1.0104/10.8.22943
ItW catch rate 100.00% 100.00%
False positives  2  2
Defenx-Apr17.gif VB100-fail.jpg

 

Emsisoft Anti-Malware

  Windows 7 Windows 10
Main version 7.70282 7.70282
Update versions 7879563, 7.71078/8732080, 7.71057/8708161, 7.70769/8408438 7.70315, 7.70451, 7.70669
ItW catch rate 99.76% 99.76%
False positive rate 0 0
Emsisoft-Apr17.gif VB100-fail.jpg

 

 

eScan Internet Security Suite

  Windows 7 Windows 10
Main version 14.0.1400.1979 14.0.1400.1979
Update versions N/A N/A
ItW catch rate 100.00% 100.00%
False positives 0 0
eScan-Apr17.gif vb100-04-17.jpg

 

 

ESET Internet Security

  Windows 7 Windows 10
Main version 10.0.390.0 10.0.386.0
Update versions 15117, 15126, 15160, 15220 15117, 15126, 15194, 15216
ItW catch rate 100.00% 100.00%
False positive rate 0 0
ESET-Apr17.gif vb100-04-17.jpg

 

 

Essentware PCKeeper Antivirus PRO

  Windows 7 Windows 10
Main version 8.3.44.10 8.3.44.10
Update versions 8.12.160.44, 8.12.156.166, 8.12.160.148, 8.3.44.18/8.12.161.88 8.12.160.44, 8.3.44.18/8.12.161.2, 8.12.161.88, 8.3.44.32/8.12.162.158
ItW catch rate 100.00% 100.00%
False positives 0 0
Essentware-Apr17.gif vb100-04-17.jpg

 

 

ESTsecurity ALYac

  Windows 7 Windows 10
Main version 3.0.1.3 3.0.1.3
Update versions 16.7.12.1, 16.7.12.1, 16.7.12.1, 8.3.44.18/8.12.161.88 16.7.12.1/3.0.1.3.30307/634381.2017032021/7.70278/7885509.20170320, 7.70304/7871350.20170321/634555.2017032215, 635702.2017032819/7.70455/7953332.20170328, 637514.2017040723/7.70669/8404894.20170407
ItW catch rate 100.00% 100.00%
False positives 0 0
ESTsoft-Apr17.gif vb100-04-17.jpg

 

 

Fortinet FortiClient

  Windows 7 Windows 10
Main version 5.4.1.0840 5.4.1.0840
Update versions 5.00233/45.00497, 45.00527, 45.00671, 45.00931 5.00233/45.00497, 46.00482, 45.00820, 45.00906
ItW catch rate 100.00% 100.00%
False positives 0 0
Fortinet-Apr17.gif vb100-04-17.jpg

 

 

G DATA Antivirus

  Windows 7 Windows 10
Main version 25.3.0.1   25.3.0.1
Update versions AVA 25.11295/GD 25.9129, AVA 25.12171/GD 25.9423, AVA 25.10892/GD 25.8963, AVA 25.11706/GD 25.9280  AVA 25.11296/GD 25.9129, AVA 25.11341/GD 25.9146, AVA 25.11697/GD 25.9277
ItW catch rate  100.00%  100.00%
False positives  0  0
GData-Apr17.gif vb100-04-17.jpg

 

 

IKARUS anti.virus

  Windows 7 Windows 10
Main version 2.13.19    2.13.19   
Update versions 99180, 99293, 99203, 99232 99180, 99187, 99203, 99233
ItW catch rate 100.00% 100.00%
False positives 12 12
Ikarus-Apr17.gif VB100-fail.jpg

 

 

K7 Total Security

  Windows 7 Windows 10
Main version 15.1.0304 15.1.0304
Update versions 10.6.22771, 10.10.23224, 10.7.22857, 10.9.23100 10.6.22771, 10.6.22799, 10.7.22861, 10.22969
ItW catch rate 100.00% 100.00%
False positives 2 2
K7-Apr17.gif VB100-fail.jpg

 

 

Kaspersky Internet Security

  Windows 7 Windows 10
Main version 10.0.0.611 17.0.0.611d
Update versions N/A 17.0.0.611e, 17.0.0.611d, 17.0.0.611d
ItW catch rate 100.00% 100.00%
False positives 0 0
(Product not included in RAP tests.) vb100-04-17.jpg

 

 

MSecure Endpoint ATP

  Windows 7 Windows 10
Main version 99181 99180
Update versions 99181, 99309, 99206, 99240 99316, 99310, 99233
ItW catch rate 100.00% 100.00%
False positives 70 70
MSecure-Apr17.gif VB100-fail.jpg

 

 

NANO Antivirus Pro

  Windows 7 Windows 10
Main version 1.0.70.81193 1.0.70.81193
Update versions 0.14.27.9233, 0.14.27.9125, 0.14.27.9270, 0.14.27.9325 0.14.27.9233,1.0.70.81508/0.14.27.9245, 0.14.27.9270/1.0.70.81508, 0.14.27.9341/1.0.72.81720
ItW catch rate 100.00% 100.00%
False positive rate 1 2
NANO-Apr17-v2.gif VB100-fail.jpg

 

 

Quick Heal Seqrite Endpoint Security

  Windows 7 Windows 10
Main version 17.00 (10.0.1.26) 64bit 17.00
Update versions N/A 10.2.3.1
ItW catch rate 100.00% 100.00%
False positives 0 0
QuickHeal-Seqrite-Apr17.gif vb100-04-17.jpg

 

 

Quick Heal Total Security

  Windows 7 Windows 10
Main version 17.00 (1.2.3.1) 64bit 17.00
Update versions N/A 10.0.1.26
ItW catch rate 100.00% 100.00%
False positives 0 0
QuickHeal-TS-Apr17.gif vb100-04-17.jpg

 

 

Tencent PC Manager

  Windows 7 Windows 10
Main version 12.1.26390.901 12.1.26390.901
Update versions 2.3.26413.901, 12.1.26375.901 12.3.26413.901, 12.3.26415.901, 12.3.26397.901
ItW catch rate 100.00% 100.00%
False positives 0 0
Tencent-PCManager-Apr17.gif vb100-04-17.jpg

 

 

ThreatTrack VIPRE Internet Security Pro 2016

  Windows 7 Windows 10
Main version 9.3.6.3 9.3.6.3
Update versions 56782,  56858, 56976, 57228 56782, 56856, 56980, 57288
ItW catch rate 99.21% 100.00%
False positives 0 0
ThreatTrack-Apr17.gif VB100-fail.jpg

 

 

Total Defense Internet Security

  Windows 7 Windows 10
Main version 9.0.0.645 9.0.0.645
Update versions 3.0.0.6767, 3.0.2.1015 3.0.2.1015
ItW catch rate 100.00% 100.00%
False positives 0 0
TotalDefense-IS-Apr17.gif vb100-04-17.jpg

 

 

Total Defense Premium Internet Security

  Windows 7 Windows 10
Main version 9.0.0.645 9.0.0.645
Update versions 3.0.2.1015 3.0.2.1015
ItW catch rate 100.00% 100.00%
False positives 0 0
TotalDefense-Premium-Apr17.gif vb100-04-17.jpg

 

 

TrustPort Antivirus Sphere

  Windows 7 Windows 10
Main version 17.0.0.6026  17.0.0.6026 
Update versions N/A  N/A
ItW catch rate 100.00% 100.00%
False positives
0 0
TrustPort-Apr17.gif vb100-04-17.jpg

 

 

Vir.IT eXplorer PRO

  Windows 7 Windows 10
Main version 8.3.87 8.3
Update versions 8.3.91, 8.4.17, 8.3.74 8.3.87, 8.3.91, 8.3.94, 8.4.3
ItW catch rate 100.00% 100.00%
False positives 0 0
VirIT-Apr17.gif vb100-04-17.jpg

 

 

Tables

Certification tests Windows 7 Windows 10 VB100
FPs FP rate WildList misses WildList catch rate FPs FP rate WildList misses WildList catch rate
adaware antivirus free 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
adaware antivirus pro 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Arcabit Antivirus 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Avast Free Antivirus 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
AVG Internet Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Bitdefender GravityZone Security for Endpoints 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
BullGuard Antivirus 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
CompuClever Antivirus Plus 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Cyren Command Anti‑Malware 35 0.01% 0 100.00% 35 0.01% 0 100.00% vb100 fail
Defenx Security Suite 2 0.00% 0 100.00% 2 0.00% 0 100.00% vb100 fail
Emsisoft Anti-Malware 0 0.00% 6 99.76% 0 0.00% 6 99.76% vb100 fail
eScan Internet Security Suite 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
ESET Internet Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Essentware PCKeeper Antivirus PRO 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
ESTsecurity ALYac 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Fortinet FortiClient 0 0.00% 0 100.00% 0 00.00% 0 100.00%  vb100
G DATA Antivirus 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
IKARUS anti.virus 12 0.00% 0 100.00% 12 0.00% 0 100.00% vb100 fail
K7 Total Security 2 0.00% 0 100.00% 2 0.00% 0 100.00% vb100 fail
Kaspersky Internet Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
MSecure Endpoint ATP 70 0.02% 0 100.00% 70 0.02% 0 100.00% vb100 fail
NANO Antivirus Pro 1 0.00% 0 100.00% 2 0.00% 0 100.00% vb100 fail
Quick Heal Seqrite Endpoint Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Quick Heal Total Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Tencent PC Manager 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
ThreatTrack VIPRE Internet Security Pro 2016 0 0.00% 20 99.21% 0 0.00% 0 100.00% vb100 fail
Total Defense Internet Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Total Defense Premium Internet Security 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
TrustPort Antivirus Sphere 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100
Vir.IT eXplorer PRO 0 0.00% 0 100.00% 0 0.00% 0 100.00% vb100

 

 

RAP (Reactive And Proactive) tests – Windows 7 Reactive Reactive average Proactive Proactive average RAP weighted average‡
Set -2* Set -1* Set +1† Set +2†
adaware antivirus free 95.9 97.0 96.5 91.8 90.6 91.2 94.7
adaware antivirus pro 94.9 97.1 96.0 92.1 90.6 91.4 94.4
Arcabit Antivirus 94.0 96.3 95.1 95.9 95.3 95.6 95.3
Avast Free Antivirus 93.5 94.0 93.7 93.5 89.9 91.7 93.1
AVG Internet Security 97.9 98.2 98.1 93.3 89.8 91.6 95.9
Bitdefender GravityZone Security for Endpoints 93.9 94.0 93.9 91.5 90.6 91.1 93.0
BullGuard Antivirus 94.1 93.5 93.8 92.1 90.6 91.3 93.0
CompuClever Antivirus Plus 95.0 94.9 94.9 91.8 90.6 91.2 93.7
Cyren Command Anti‑Malware 73.8 78.7 76.3 69.4 71.3 70.3 74.3
Defenx Security Suite 86.3 88.5 87.4 76.0 78.0 77.0 83.9
Emsisoft Anti-Malware 95.4 95.5 95.4 92.0 90.6 91.3 94.1
eScan Internet Security Suite 94.7 95.5 95.1 92.1 90.8 91.4 93.9
ESET Internet Security 90.2 92.6 91.4 90.0 89.5 89.7 90.9
Essentware PCKeeper Antivirus PRO 94.7 95.1 94.9 92.5 91.5 92.0 93.9
ESTsecurity ALYac 93.9 94.6 94.3 92.7 90.8 91.8 93.4
Fortinet FortiClient 97.1 97.7 97.4 89.6 91.2 90.4 95.1
G DATA Antivirus 96.5 97.9 97.2 98.3 96.5 97.4 97.2
IKARUS anti.virus 98.7 98.6 98.7 93.3 95.1 94.2 97.2
K7 Total Security 90.0 90.1 90.1 76.0 78.0 77.0 85.7
MSecure Endpoint ATP 91.3 90.1 90.7 85.9 85.7 85.8 89.1
NANO Antivirus Pro 88.4 91.5 89.9 81.7 84.2 82.9 87.6
Quick Heal Seqrite Endpoint Security 97.4 97.2 97.3 96.1 95.2 95.7 96.8
Quick Heal Total Security 96.9 97.0 96.9 96.1 95.2 95.7 96.5
Tencent PC Manager 83.1 88.1 85.6 96.3 95.2 95.8 89.0
ThreatTrack VIPRE Internet Security Pro 2016 93.3 86.9 90.1 87.3 90.1 88.7 89.7
Total Defense Internet Security 93.7 93.5 93.6 91.8 90.6 91.2 92.8
Total Defense Premium Internet Security 94.0 93.5 93.7 91.8 84.2 88.0 91.8
TrustPort Antivirus Sphere 98.5 98.5 98.5 96.8 96.7 96.7 97.9
Vir.IT eXplorer PRO 61.1 61.4 61.2 59.1 62.4 60.7 61.1

*Set -1 = Samples discovered 1 to 5 days before testing; Set -2 = Samples discovered 6 to 10 days before testing.
†Set +1 = Samples discovered 1 to 5 days after updates frozen; Set +2 = Samples discovered 6 to 10 days after updates frozen.
‡ Weighted average gives equal emphasis to the two reactive weeks and the whole proactive part.

 

 

RAP (Reactive And Proactive) tests – Windows 10 Reactive Reactive average Proactive Proactive average RAP weighted average‡
Set -2* Set -1* Set +1† Set +2†  
adaware antivirus free 96.6 96.6 96.6 91.8 90.6 91.2 94.8
adaware antivirus pro 96.6 97.0 96.8 92.1 90.6 91.4 95.0
Arcabit Antivirus 93.8 95.9 94.8 92.4 90.7 91.6 93.7
Avast Free Antivirus 97.4 98.0 97.7 93.5 87.6 90.6 95.3
AVG Internet Security 97.0 97.9 97.4 93.5 89.7 91.6 95.5
Bitdefender GravityZone Security for Endpoints 94.2 95.2 94.7 91.5 90.6 91.1 93.5
BullGuard Antivirus 94.7 93.4 94.1 92.1 90.6 91.4 93.2
CompuClever Antivirus Plus 94.2 95.2 94.7 91.8 90.6 91.2 93.5
Cyren Command Anti‑Malware 76.7 73.6 75.1 69.4 71.3 70.3 73.5
Defenx Security Suite 87.4 89.3 88.4 76.0 78.0 77.0 84.6
Emsisoft Anti-Malware 92.5 94.3 93.4 92.0 90.6 91.3 92.7
eScan Internet Security Suite 95.4 96.0 95.7 92.1 90.8 91.4 94.3
ESET Internet Security 90.8 93.1 92.0 90.0 89.5 89.7 91.2
Essentware PCKeeper Antivirus PRO 95.1 94.8 95.0 92.5 91.5 92.0 94.0
ESTsecurity ALYac 93.5 94.1 93.8 94.4 91.0 92.7 93.5
Fortinet FortiClient 97.3 97.6 97.4 89.7 91.2 90.5 95.1
G DATA Antivirus 97.2 97.3 97.3 98.3 96.5 97.4 97.3
IKARUS anti.virus 98.6 98.7 98.7 93.3 95.1 94.2 97.2
K7 Total Security 86.2 86.5 86.3 76.0 78.0 77.0 83.2
MSecure Endpoint ATP 87.4 91.6 89.5 85.9 85.7 85.8 88.3
NANO Antivirus Pro 89.3 90.3 89.8 81.6 84.2 82.9 87.5
Quick Heal Seqrite Endpoint Security 96.6   97.4 97.0 96.1   95.2 95.7  96.6
Quick Heal Total Security 96.4 97.4  96.9 96.1  95.2 95.7  96.5
Tencent PC Manager 95.2 95.8 95.5 96.3 95.2 95.8  95.6
ThreatTrack VIPRE Internet Security Pro 2016 93.9 78.9 86.4 80.2  90.3 85.3 86.0
Total Defense Internet Security 95.5 95.4 95.4 91.8  90.6 91.2  94.0
Total Defense Premium Internet Security 94.4 94.1 94.3  91.8  90.6 91.2 93.3 
TrustPort Antivirus Sphere 98.6 98.5 98.5  97.1  96.9 97.0 98.0
Vir.IT eXplorer PRO 60.9 63.2 62.1  59.1 62.4  60.7 61.6 

 *Set -1 = Samples discovered 1 to 5 days before testing; Set -2 = Samples discovered 6 to 10 days before testing.
†Set +1 = Samples discovered 1 to 5 days after updates frozen; Set +2 = Samples discovered 6 to 10 days after updates frozen.
‡ Weighted average gives equal emphasis to the two reactive weeks and the whole proactive part.

 

 

RAP-quadrant-Apr2017.jpg

 

Conclusion

We were pleased to find that all but two participating solutions detected the full WildList – after all, this is the minimum one should expect from an anti-malware product.

Of course, a good detection rate is only relevant if the product doesn't regularly cause disruption by blocking legitimate files. It was thus disappointing to see six products blocking one or more legitimate programs (noting that the clean set had been purged of uncommon programs and those that showed suspicious behaviour).

Overall though, we are excited about this next step in Virus Bulletin's anti-malware testing and we are looking forward to further developing the tests in line with the ever-changing threat landscape.

Appendix: the test set-up

The main test on each platform was run in three parts, over three consecutive weeks. Products were installed on clean installations of both Windows 7 and Windows 10. At the beginning of each part of the test we made sure the latest updates were downloaded, while throughout the test, products were connected to the Internet, thus allowing for real-time cloud look-ups.

For each part of the test, we used the most recent version of the WildList, together with one third of our constantly updated collection of widely used legitimate software. Using a shared drive, the files were copied onto the client machine and we recorded whether (and how) files were blocked by the anti-malware product.

If files weren't blocked, a custom-built tool was used to open the file, thus triggering AV detection by products that don't (always) scan files on being copied.

As mentioned in the introduction, a product passed the test if, and only if, on both platforms it blocked all files from the WildList, and didn't generate any false positives (i.e. incorrect detections) when scanning the full clean set.

The clean set consists of more than 400,000 files, all widely used programs, with any files that show suspicious behaviour being excluded from the set.

For the 'RAP' (reactive and proactive) test, the same set‑up was used, but for the proactive part of the test products were not connected to the Internet. This allowed us to measure their proactive detection abilities by having a 'frozen' version of each product scan two sets of malware files: those seen in the wild between one day and five days after the product 'freeze' date, and those seen in the wild between six and 10 days after this date.

Note: A slightly different approach when it comes to tidying up the set of malware, as well as a different approach to testing, means the individual RAP scores should not be compared with those seen in previous tests.

Download PDF

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VBSpam comparative review

The Q1 2024 VBSpam test measured the performance of nine full email security solutions, one custom configured solution and one open‑source solution.

VBSpam comparative review

The Q4 2023 VBSpam test measured the performance of eight full email security solutions, one custom configured solution, one open-source solution and one blocklist.

VBSpam comparative review

In the Q3 2023 VBSpam test we measured the performance of eight full email security solutions, one custom configured solution, one open-source solution and one blocklist.

VBSpam comparative review

In the Q2 2023 VBSpam test we measured the performance of nine full email security solutions, one custom configured solution, one open-source solution and one blocklist.

VBSpam comparative review

In the Q1 2023 VBSpam test we measured the performance of eight full email security solutions, one custom configured solution, one open-source solution and one blocklist.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.