VB100 certification report - August 2018

Martijn Grooten

Virus Bulletin

Copyright © 2018 Virus Bulletin


 

Introduction

The endpoint security market is changing rapidly and marketing departments like to make a lot of noise about next-gen (or next-next-gen) technology, often using some kind of machine learning. These are important developments, but at the core of every product is an engine that, when given a file, determines whether it is malicious or not.

The VB100 certification reports check whether this engine, and thus the product, satisfies a minimum standard: when looking to purchase a new endpoint security product, we recommend looking for a product that consistently achieves VB100 certification.

This test report details the performance of 30 anti-virus products from 29 different vendors during July and August 2018, all of which achieved a VB100 award.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList1 and a selection of clean files taken from Virus Bulletin's own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn't blocked is considered a miss.

A product achieves a VB100 certification if:

• No more than 0.5% of WildList samples are missed

and

• No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

 

Diversity Test

The malware part of the VB100 certification uses the WildList, a regularly updated list of extremely well vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The 'Diversity Test', which debuts in this report, shows detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples and provide a measure of that detection.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

 

AhnLab V3 Internet Security 9.0

Windows 7 version 9.0.48.1 08.jpg
Windows 10 version 9.0.48.1
WildList detection 99.9%
False positive rate 0.000%
Diversity test rate 100.00%

 

Arcabit AntiVirus

Windows 7 version 2018.07.10  08.jpg
Windows 10 version 2018.07.11
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Avast Free Antivirus

Windows 7 version 18.5.2342  08.jpg
Windows 10 version 18.5.2342 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

AVG Internet Security

Windows 7 version 18.5.3059  08.jpg
Windows 10 version 18.5.3059 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Bitdefender Endpoint Security

Windows 7 version 6.6.2.49  08.jpg
Windows 10 version 6.6.2.49 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Cyren Command Anti-Malware

Windows 7 version 5.1.38  08.jpg
Windows 10 version 5.1.38 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Defenx Security Suite

Windows 7 version 15.1.0111  08.jpg
Windows 10 version 15.1.0111 
WildList detection 100.0%
False positive rate 0.001%
Diversity test rate 100.00%

 

Emsisoft Anti-Malware

Windows 7 version 2018.6.0.8750  08.jpg
Windows 10 version 2018.6.0.8750 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

eScan Internet Security Suite for Windows

Windows 7 version 14.0.1400.2029  08.jpg
Windows 10 version 14.0.1400.2029 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

ESET Internet Security

Windows 7 version 11.0.159.9  08.jpg
Windows 10 version 11.0.159.9 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

ESTsecurity ALYac

Windows 7 version 3.0.1.3  08.jpg
Windows 10 version 3.0.1.3 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Faronics Anti-Virus

Windows 7 version 4.12.3102.401  08.jpg
Windows 10 version 4.12.3102.401 
WildList detection 100.0%
False positive rate 0.005%
Diversity test rate 100.00%

 

FireEye Endpoint Security

Windows 7 version 26.21.8 08.jpg
Windows 10 version 26.21.8
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Fortinet FortiClient

Windows 7 version 5.6.2.1117  08.jpg
Windows 10 version 5.6.2.1117 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

G DATA Antivirus

Windows 7 version 25.4.0.3  08.jpg
Windows 10 version 25.4.0.3 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Hammock AssetView

Windows 7 version 10.2.0  08.jpg
Windows 10 version 10.2.0 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

IKARUS anti.virus

Windows 7 version 2.16.20  08.jpg
Windows 10 version 2.16.20 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

K7 Total Security

Windows 7 version 15.1.0339  08.jpg
Windows 10 version 15.1.0339 
WildList detection 100.0%
False positive rate 0.001%
Diversity test rate 100.00%

 

Kaspersky Endpoint Security 10 for Windows

Windows 7 version 11.0.0.6499  08.jpg
Windows 10 version 11.0.0.6499 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

NANO Antivirus

Windows 7 version 1.0.116.88902  08.jpg
Windows 10 version 1.0.116.88902 
WildList detection 100.0%
False positive rate 0.001%
Diversity test rate 100.00%

 

Rising Security Cloud Client

Windows 7 version 3.0.0.78  08.jpg
Windows 10 version 3.0.0.78 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

SafeCentral Security Suite

Windows 7 version 2.0.1430  08.jpg
Windows 10 version 2.0.1430 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

TACHYON Endpoint Security

Windows 7 version 5.0.0.57  08.jpg
Windows 10 version 5.0.0.57 
WildList detection 100.0%
False positive rate 0.007%
Diversity test rate 99.99%

 

TeamViewer ITbrain Anti-Malware

Windows 7 version 1.0.10118  08.jpg
Windows 10 version 1.0.10118 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Tencent PC Manager

Windows 7 version 12.3.26562.901  08.jpg
Windows 10 version 12.3.26562.901 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

TotalAV

Windows 7 version 4.6.19 08.jpg
Windows 10 version 4.6.19
WildList detection 99.8%
False positive rate 0.000%
Diversity test rate 44.63%

 

Total Defense Premium

Windows 7 version 9.0.0.747  08.jpg
Windows 10 version 9.0.0.747 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

VIPRE Advanced Security

Windows 7 version 10.3.4.2  08.jpg
Windows 10 version 10.3.4.2 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

VirIT eXplorer PRO

Windows 7 version 8.7 08.jpg
Windows 10 version 8.7
WildList detection 100.0%
False positive rate 0.001%
Diversity test rate 100.00%

 

Zemana EndPoint Security

Windows 7 version 6.6.1.36  08.jpg
Windows 10 version 6.6.1.36 
WildList detection 100.0%
False positive rate 0.000%
Diversity test rate 100.00%

 

Appendix 1: products not certified

All tested products were certified.

 

Appendix 2: excluded parts

Technical and other issues can render the data we collect insufficient or otherwise unsuitable for accurate reporting. In such cases, the methodology allows us to discard the affected test part and record the event below. Two healthy test parts out of three are required for the certification to be issued.

  • For TotalAV, the third certification part on Windows 7 was discarded due to technical issues.

 

Appendix 3: sample set sizes

The WildList contained 990 samples. The set of clean files used for the false positive test contained 100,000 files, of which 29,199 were portable executable (PE) files. The set used for the Diversity Test contained 997 samples.

 

Footnotes

1 The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Download PDF

twitter.png
fb.png
linkedin.png
googleplus.png
reddit.png

 

Latest reviews:

VBSpam Comparative Review - September 2018

In this comparative test of email security products 12 full email security solutions and eight blacklists of various kinds were assembled on the test bench to measure their performance against various streams of wanted, unwanted and malicious emails.

VB100 Certification Report - August 2018

This test report details the performance of 30 anti-virus products from 29 different vendors tested during July and August 2018, all of which achieved a VB100 award.

VB100 Certification Report - June 2018

This test report details the performance of 31 anti-virus products from 29 different vendors tested during May and June 2018, each of which achieved a VB100 award.

VBWeb Comparative Review - Summer 2018

Because people make mistakes, and organizations find it hard always to patch software right away, web security products provide an important extra layer of defence - and the VBWeb report demonstrates that these products do a good job.

VBSpam Email Security Comparative Review - June 2018

In this test, 13 full email security solutions and eight blacklists of various kinds were assembled on the test bench to measure their performance against various streams of wanted, unwanted and malicious emails.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.