VB100 certification report June 2018

Martijn Grooten

Virus Bulletin

Copyright © 2018 Virus Bulletin


 

Introduction

Though in the ever-changing threat landscape anti-virus products have long ceased to be the only thing needed to keep devices secure, they remain as important as ever, as both a first and a last line of defence: to scan potentially malicious files before they make it onto a device, and to block them from running if all other defences have failed.

For more than two decades, the VB100 certification scheme has provided an easy to recognize (but not always easy to obtain) award that affirms that a product satisfies the minimum requirements one should expect of an anti-virus product.

This test report details the performance of 31 anti-virus products from 29 different vendors tested during May and June 2018, each of which achieved a VB100 award.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList1 and a selection of clean files taken from Virus Bulletin's own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn't blocked is considered a miss.

A product achieves a VB100 certification if:

  • No more than 0.5% of WildList samples are missed

and

  • No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

 

Diversity Test

As per the methodology, another test was executed alongside the Certification Test: the Diversity Test, which is designed to measure the detection of a more diverse collection of malware. However, after validation of the 2,000 malware samples that were used in this test, and discarding those that, for various reasons, weren't perfectly suited to the test, we ended up with a set of fewer than 750 malware samples.

Since we do not believe this is a sufficient number of samples for the provision of relevant data, we have decided not to publish the results of the Diversity Test on this occasion. It should be noted that, as per the methodology, performance in the Diversity Test would not have counted towards VB100 certification.

We are currently re-evaluating how we source and select the samples to be used in this test. It should be noted, though, that it is an important feature of the Diversity Test that most validation doesn't happen until after the test has run.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

 

AhnLab V3 Internet Security 9.0

Windows 7 version  9.0.46.6 06-18.jpg
Windows 10 version  9.0.46.6
WildList detection  99.9%
False positive rate  0.000%

 

Arcabit AntiVirus

Windows 7 version 2018.05.10  06-18.jpg
Windows 10 version  2018.05.09
WildList detection  100.0%
False positive rate  0.000%

 

Avast Free Antivirus

Windows 7 version 18.3.2333 06-18.jpg   
Windows 10 version 18.4.2338
WildList detection 100.0%
False positive rate 0.000%

 

AVG Internet Security

Windows 7 version 18.3.3051  06-18.jpg
Windows 10 version 18.4.3056
WildList detection  100.0%
False positive rate  0.000%

 

Bitdefender Endpoint Security

Windows 7 version 6.2.36.1014 06-18.jpg
Windows 10 version 6.2.36.1014
WildList detection 100.0%
False positive rate  0.000%

 

BullGuard Antivirus

Windows 7 version 18.0.347.18  06-18.jpg
Windows 10 version 18.0.347.18 
WildList detection 100.0%
False positive rate 0.000%

 

Cyren Command Anti-Malware

Windows 7 version 5.1.38  06-18.jpg
Windows 10 version 5.1.38 
WildList detection 100.0%
False positive rate 0.002%

 

Defenx Security Suite

Windows 7 version 15.1.0108  06-18.jpg
Windows 10 version 15.1.0.0108 
WildList detection 100.0%
False positive rate 0.001%

 

Emsisoft Anti-Malware

Windows 7 version 2018.4.0.8631  06-18.jpg
Windows 10 version 2018.4.0.8631 
WildList detection 100.0%
False positive rate 0.000%

 

eScan Internet Security Suite for Windows

Windows 7 version 14.0.1400.2029  06-18.jpg
Windows 10 version 14.0.1400.2029 
WildList detection 100.0%
False positive rate 0.000%

 

ESET Internet Security

Windows 7 version 11.0.159.9  06-18.jpg
Windows 10 version 11.0.159.9 
WildList detection 100.0%
False positive rate 0.000%

 

ESTsecurity ALYac

Windows 7 version 3.0.1.3  06-18.jpg
Windows 10 version 3.0.1.3 
WildList detection 100.0%
False positive rate 0.000%

 

Faronics Anti-Virus

Windows 7 version 4.12.3102.401  06-18.jpg
Windows 10 version 4.12.3102.401 
WildList detection 100.0%
False positive rate 0.008%

 

FireEye Endpoint Security

Windows 7 version 26.21.8 06-18.jpg
Windows 10 version 26.21.8
WildList detection 100.0%
False positive rate 0.000%

 

Fortinet FortiClient

Windows 7 version  5.6.2.1117 06-18.jpg
Windows 10 version 5.6.2.1117 
WildList detection 100.0%
False positive rate 0.000%

 

G DATA Antivirus

Windows 7 version 25.4.0.3  06-18.jpg
Windows 10 version 25.4.0.3 
WildList detection 100.0%
False positive rate 0.000%

 

IKARUS anti.virus

Windows 7 version 2.16.20 06-18.jpg
Windows 10 version 2.16.20
WildList detection 100.0%
False positive rate 0.000%

 

K7 Total Security

Windows 7 version 15.1.0335  06-18.jpg
Windows 10 version 15.1.0335 
WildList detection 100.0% 
False positive rate 0.001%

 

Kaspersky Endpoint Security 10 for Windows

Windows 7 version 11.0.0.6499  06-18.jpg
Windows 10 version 11.0.0.6499 
WildList detection 100.0%
False positive rate 0.000%

 

NANO Antivirus

Windows 7 version 1.0.106.88154  06-18.jpg
Windows 10 version 1.0.106.88154 
WildList detection 100.0%
False positive rate 0.001%

 

Rising Security Cloud Client

Windows 7 version 3.0.0.78  06-18.jpg
Windows 10 version 3.0.0.78 
WildList detection 99.7%
False positive rate 0.002%

 

SafeCentral Security Suite

Windows 7 version 2.0.1430  06-18.jpg
Windows 10 version 2.0.1430 
WildList detection 100.0%
False positive rate 0.000%

 

TACHYON Endpoint Security

Windows 7 version 5.0.0.0  06-18.jpg
Windows 10 version 5.0.0.0 
WildList detection 100.0%
False positive rate 0.000%

 

TeamViewer ITbrain Anti-Malware

Windows 7 version 1.0.98986  06-18.jpg
Windows 10 version 1.0.98986 
WildList detection 100.0%
False positive rate 0.000%

 

Tencent PC Manager

Windows 7 version 12.3.26544.901  06-18.jpg
Windows 10 version 12.3.26544.901 
WildList detection 100.0%
False positive rate 0.000%

 

Total Defense Internet Security

Windows 7 version 9.0.0.747  06-18.jpg
Windows 10 version 9.0.0.747 
WildList detection 100.0%
False positive rate 0.000%

 

Total Defense Premium

Windows 7 version 9.0.0.747  06-18.jpg
Windows 10 version 9.0.0.747 
WildList detection 100.0%
False positive rate 0.000%

 

TotalAV

Windows 7 version 4.6.19  06-18.jpg
Windows 10 version 4.6.19 
WildList detection 100.0%
False positive rate 0.000%

 

VIPRE Advanced Security

Windows 7 version 10.1.4.33  06-18.jpg
Windows 10 version 10.1.4.33 
WildList detection 100.0%
False positive rate 0.000%

 

VirIT eXplorer PRO

Windows 7 version 8.6 06-18.jpg
Windows 10 version 8.6
WildList detection 100.0%
False positive rate 0.000%

 

Zemana EndPoint Security

Windows 7 version 6.2.34.998  06-18.jpg
Windows 10 version 6.2.34.998 
WildList detection 100.0%
False positive rate 0.000%

 

Appendix 1: excluded parts

Technical and other issues can render the data we collect insufficient or otherwise unsuitable for accurate reporting. In such cases, the methodology allows us to discard the affected test part and record the event below. Two healthy test parts out of three are required for the certification to be issued.

  • For BullGuard Antivirus, the first certification part on both test platforms was discarded due to a combination of a technical problem and a test configuration issue.
  • For TACHYON Endpoint Security, the third certification part on Windows 10 was discarded due to technical issues with the test framework.

 

Appendix 2: sample set sizes

The WildList contained 894 samples. The set of clean files used for the false positive test contained 100,000 files, of which 30,070 were portable executable (PE) files.

 

Footnotes

1 The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Download PDF

twitter.png
fb.png
linkedin.png
googleplus.png
reddit.png

 

Latest reviews:

VB100 Certification Report - August 2018

This test report details the performance of 30 anti-virus products from 29 different vendors tested during July and August 2018, all of which achieved a VB100 award.

VB100 Certification Report - June 2018

This test report details the performance of 31 anti-virus products from 29 different vendors tested during May and June 2018, each of which achieved a VB100 award.

VBWeb Comparative Review - Summer 2018

Because people make mistakes, and organizations find it hard always to patch software right away, web security products provide an important extra layer of defence - and the VBWeb report demonstrates that these products do a good job.

VBSpam Email Security Comparative Review - June 2018

In this test, 13 full email security solutions and eight blacklists of various kinds were assembled on the test bench to measure their performance against various streams of wanted, unwanted and malicious emails.

VB100 Comparative Review - April 2018

In the last of the current style VB100 tests, we put 35 anti-virus products from 32 vendors through their paces. 27 products achieved VB100 certification.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.