VB100 certification report June 2018

Martijn Grooten

Virus Bulletin

Copyright © 2018 Virus Bulletin

Introduction

Though in the ever-changing threat landscape anti-virus products have long ceased to be the only thing needed to keep devices secure, they remain as important as ever, as both a first and a last line of defence: to scan potentially malicious files before they make it onto a device, and to block them from running if all other defences have failed.

For more than two decades, the VB100 certification scheme has provided an easy to recognize (but not always easy to obtain) award that affirms that a product satisfies the minimum requirements one should expect of an anti-virus product.

This test report details the performance of 31 anti-virus products from 29 different vendors tested during May and June 2018, each of which achieved a VB100 award.

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList¹ and a selection of clean files taken from Virus Bulletin's own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn't blocked is considered a miss.

A product achieves a VB100 certification if:

• No more than 0.5% of WildList samples are missed

and

• No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

Diversity Test

As per the methodology, another test was executed alongside the Certification Test: the Diversity Test, which is designed to measure the detection of a more diverse collection of malware. However, after validation of the 2,000 malware samples that were used in this test, and discarding those that, for various reasons, weren't perfectly suited to the test, we ended up with a set of fewer than 750 malware samples.

Since we do not believe this is a sufficient number of samples for the provision of relevant data, we have decided not to publish the results of the Diversity Test on this occasion. It should be noted that, as per the methodology, performance in the Diversity Test would not have counted towards VB100 certification.

We are currently re-evaluating how we source and select the samples to be used in this test. It should be noted, though, that it is an important feature of the Diversity Test that most validation doesn't happen until after the test has run.

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

AhnLab V3 Internet Security 9.0

Windows 7 version	9.0.46.6
Windows 10 version	9.0.46.6
WildList detection	99.9%
False positive rate	0.000%

Arcabit AntiVirus

Windows 7 version	2018.05.10
Windows 10 version	2018.05.09
WildList detection	100.0%
False positive rate	0.000%

Avast Free Antivirus

Windows 7 version	18.3.2333
Windows 10 version	18.4.2338
WildList detection	100.0%
False positive rate	0.000%

AVG Internet Security

Windows 7 version	18.3.3051
Windows 10 version	18.4.3056
WildList detection	100.0%
False positive rate	0.000%

Bitdefender Endpoint Security

Windows 7 version	6.2.36.1014
Windows 10 version	6.2.36.1014
WildList detection	100.0%
False positive rate	0.000%

BullGuard Antivirus

Windows 7 version	18.0.347.18
Windows 10 version	18.0.347.18
WildList detection	100.0%
False positive rate	0.000%

Cyren Command Anti-Malware

Windows 7 version	5.1.38
Windows 10 version	5.1.38
WildList detection	100.0%
False positive rate	0.002%

Defenx Security Suite

Windows 7 version	15.1.0108
Windows 10 version	15.1.0.0108
WildList detection	100.0%
False positive rate	0.001%

Emsisoft Anti-Malware

Windows 7 version	2018.4.0.8631
Windows 10 version	2018.4.0.8631
WildList detection	100.0%
False positive rate	0.000%

eScan Internet Security Suite for Windows

Windows 7 version	14.0.1400.2029
Windows 10 version	14.0.1400.2029
WildList detection	100.0%
False positive rate	0.000%

ESET Internet Security

Windows 7 version	11.0.159.9
Windows 10 version	11.0.159.9
WildList detection	100.0%
False positive rate	0.000%

ESTsecurity ALYac

Windows 7 version	3.0.1.3
Windows 10 version	3.0.1.3
WildList detection	100.0%
False positive rate	0.000%

Faronics Anti-Virus

Windows 7 version	4.12.3102.401
Windows 10 version	4.12.3102.401
WildList detection	100.0%
False positive rate	0.008%

FireEye Endpoint Security

Windows 7 version	26.21.8
Windows 10 version	26.21.8
WildList detection	100.0%
False positive rate	0.000%

Fortinet FortiClient

Windows 7 version	5.6.2.1117
Windows 10 version	5.6.2.1117
WildList detection	100.0%
False positive rate	0.000%

G DATA Antivirus

Windows 7 version	25.4.0.3
Windows 10 version	25.4.0.3
WildList detection	100.0%
False positive rate	0.000%

IKARUS anti.virus

Windows 7 version	2.16.20
Windows 10 version	2.16.20
WildList detection	100.0%
False positive rate	0.000%

K7 Total Security

Windows 7 version	15.1.0335
Windows 10 version	15.1.0335
WildList detection	100.0%
False positive rate	0.001%

Kaspersky Endpoint Security 10 for Windows

Windows 7 version	11.0.0.6499
Windows 10 version	11.0.0.6499
WildList detection	100.0%
False positive rate	0.000%

NANO Antivirus

Windows 7 version	1.0.106.88154
Windows 10 version	1.0.106.88154
WildList detection	100.0%
False positive rate	0.001%

Rising Security Cloud Client

Windows 7 version	3.0.0.78
Windows 10 version	3.0.0.78
WildList detection	99.7%
False positive rate	0.002%

SafeCentral Security Suite

Windows 7 version	2.0.1430
Windows 10 version	2.0.1430
WildList detection	100.0%
False positive rate	0.000%

TACHYON Endpoint Security

Windows 7 version	5.0.0.0
Windows 10 version	5.0.0.0
WildList detection	100.0%
False positive rate	0.000%

TeamViewer ITbrain Anti-Malware

Windows 7 version	1.0.98986
Windows 10 version	1.0.98986
WildList detection	100.0%
False positive rate	0.000%

Tencent PC Manager

Windows 7 version	12.3.26544.901
Windows 10 version	12.3.26544.901
WildList detection	100.0%
False positive rate	0.000%

Total Defense Internet Security

Windows 7 version	9.0.0.747
Windows 10 version	9.0.0.747
WildList detection	100.0%
False positive rate	0.000%

Total Defense Premium

Windows 7 version	9.0.0.747
Windows 10 version	9.0.0.747
WildList detection	100.0%
False positive rate	0.000%

TotalAV

Windows 7 version	4.6.19
Windows 10 version	4.6.19
WildList detection	100.0%
False positive rate	0.000%

VIPRE Advanced Security

Windows 7 version	10.1.4.33
Windows 10 version	10.1.4.33
WildList detection	100.0%
False positive rate	0.000%

VirIT eXplorer PRO

Windows 7 version	8.6
Windows 10 version	8.6
WildList detection	100.0%
False positive rate	0.000%

Zemana EndPoint Security

Windows 7 version	6.2.34.998
Windows 10 version	6.2.34.998
WildList detection	100.0%
False positive rate	0.000%

Appendix 1: excluded parts

Technical and other issues can render the data we collect insufficient or otherwise unsuitable for accurate reporting. In such cases, the methodology allows us to discard the affected test part and record the event below. Two healthy test parts out of three are required for the certification to be issued.

• For BullGuard Antivirus, the first certification part on both test platforms was discarded due to a combination of a technical problem and a test configuration issue.
• For TACHYON Endpoint Security, the third certification part on Windows 10 was discarded due to technical issues with the test framework.

Appendix 2: sample set sizes

The WildList contained 894 samples. The set of clean files used for the false positive test contained 100,000 files, of which 30,070 were portable executable (PE) files.

Footnotes

¹ The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Latest reviews: