VB100 certification report - February 2020

VB Test Team

Virus Bulletin

Copyright © 2020 Virus Bulletin


 

Introduction

The VB100 award remains as relevant now as it was at its inception 22 years ago – providing a stamp of quality and competence for anti-malware products that satisfy a minimum standard of detecting malicious executables that have recently been seen in the wild, while blocking few to no legitimate programs.

This report details the VB100 certification results of 42 such products from 36 different vendors during January and February 2020.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList1 and a selection of clean files taken from Virus Bulletin’s own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn’t blocked is considered a miss.

A product achieves a VB100 certification if:

  • No more than 0.5% of WildList samples are missed

and

  • No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://abs.new.virusbtn.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

 

Diversity Test

The malware part of the VB100 certification uses the WildList, a regularly updated list of extremely well-vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The ‘Diversity Test’ looks at products’ detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples, and provides a measure of that detection.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

Adaware Antivirus Free

Windows 7 version 12.7.1055.0 02-20.jpg
Windows 10 version 12.7.1055.0
WildList detection 100.0%
False positive rate 0.000%
Diversity Test rate 99.29%

 

Adaware Antivirus Pro

Windows 7 version  12.7.1055.0 02-20.jpg
Windows 10 version  12.7.1055.0
WildList detection  100.0%
False positive rate  0.000%
Diversity Test rate  99.29%

 

Ad-Spider

Windows 7 version  2020.01.13-7.83468 02-20.jpg
Windows 10 version  2020.01.10-7.83433
WildList detection  100.0%
False positive rate  0.000%
Diversity Test rate  99.29%

 

AhnLab V3 Internet Security 9.0

Windows 7 version 9.0.60.1  02-20.jpg
Windows 10 version 9.0.60.1 
WildList detection  100.0%
False positive rate  0.000%
Diversity Test rate  100.00%

 

Arcabit AntiVirus

Windows 7 version 2020.01.11  02-20.jpg
Windows 10 version 2020.01.11 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.29% 

 

Avast Free Antivirus

Windows 7 version 19.8.2393  02-20.jpg
Windows 10 version 19.8.2393 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00%

 

AVG Internet Security

Windows 7 version 19.8.3108  02-20.jpg
Windows 10 version 19.8.3108 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

BitSentry Elite

Windows 7 version 1.0.5.170  02-20.jpg
Windows 10 version 1.0.5.170 
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate 99.29%

 

Cynet 360

Windows 7 version 8.3.54.150  02-20.jpg
Windows 10 version 8.3.54.150 
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 98.17% 

 

Defenx Security Suite

Windows 7 version 1.5.0.19  02-20.jpg
Windows 10 version 1.5.0.19 
WildList detection 99.7% 
False positive rate 0.000% 
Diversity Test rate 98.27% 

 

Emsisoft Anti-Malware

Windows 7 version 2020.1.0.9926  02-20.jpg
Windows 10 version 2020.1.0.9926 
WildList detection 99.8% 
False positive rate 0.000% 
Diversity Test rate 100.00%

 

eScan Internet Security Suite for Windows

Windows 7 version 14.0.1400.2175  02-20.jpg
Windows 10 version 14.0.1400.2175 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

ESET Endpoint Security

Windows 7 version 7.1.2053.0  02-20.jpg
Windows 10 version 7.1.2053.0 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

ESET Internet Security

Windows 7 version 13.0.24.0  02-20.jpg
Windows 10 version 13.0.24.0 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

ESET Smart Security Premium

Windows 7 version 13.0.24.0  02-20.jpg
Windows 10 version 13.0.24.0 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

ESTsecurity ALYac

Windows 7 version 4.0.2.23116  02-20.jpg
Windows 10 version 4.0.2.23116 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.29% 

 

Exosphere Endpoint Protection

Windows 7 version 8.3.54.158  02-20.jpg
Windows 10 version 8.3.54.154 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

Faronics Anti-Virus

Windows 7 version 4.21.3102.484  02-20.jpg
Windows 10 version 4.21.3102.484 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

FireEye Endpoint Security

Windows 7 version 31.28.1  02-20.jpg
Windows 10 version 31.28.1 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.29% 

 

Fortinet FortiClient

Windows 7 version 6.2.3.0912  02-20.jpg
Windows 10 version 6.2.3.0912 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

G DATA Antivirus

Windows 7 version 25.5.5.43  02-20.jpg
Windows 10 version 25.5.5.43 
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Heimdal Thor Vigilance

Windows 7 version 2.5.223  02-20.jpg
Windows 10 version 2.5.223 
WildList detection 99.7% 
False positive rate 0.000% 
Diversity Test rate 98.06% 

 

IKARUS anti.virus

Windows 7 version 3.1.16  02-20.jpg
Windows 10 version 3.1.16 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Intego AV

Windows 7 version 1.0.0.13  02-20.jpg
Windows 10 version 1.0.0.13 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.59% 

 

ITL Total Security

Windows 7 version 1.0.0.13  02-20.jpg
Windows 10 version 1.0.0.13 
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 98.27% 

 

K7 Total Security

Windows 7 version 16.0.0128  02-20.jpg
Windows 10 version 16.0.0128 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Kaspersky Endpoint Security 11 for Windows

Windows 7 version 11.1.1.126  02-20.jpg
Windows 10 version 11.1.1.126 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

NANO Antivirus

Windows 7 version 1.0.134.90567  02-20.jpg
Windows 10 version 1.0.134.90567 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 83.15% 

 

PCProtect

Windows 7 version 4.14.31  02-20.jpg
Windows 10 version 4.14.31 
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 98.17% 

 

Rising Enterprise Security Management System

Windows 7 version 3.0.0.95  02-20.jpg
Windows 10 version 3.0.0.93 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 60.27% 

 

Scanguard

Windows 7 version 4.14.31  02-20.jpg
Windows 10 version 4.14.31 
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 98.17% 

 

Systweak Anti-virus

Windows 7 version 1.0.1000.9560  02-20.jpg
Windows 10 version 1.0.1000.9560 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.59% 

 

TACHYON Endpoint Security

Windows 7 version 5.0.0.57  02-20.jpg
Windows 10 version 5.0.0.57 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

TeamViewer Endpoint Protection

Windows 7 version 1.4.225047  02-20.jpg
Windows 10 version 1.4.225047 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.29% 

 

Tencent PC Manager

Windows 7 version 12.3.26607.901  02-20.jpg
Windows 10 version 12.3.26607.901 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

TotalAV

Windows 7 version 5.4.24  02-20.jpg
Windows 10 version 5.4.24 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.69% 

 

Total Defense Premium

Windows 7 version 12.0.0.243  02-20.jpg
Windows 10 version 12.0.0.243 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.29% 

 

VIPRE Endpoint Cloud Business

Windows 7 version 11.0.7633  02-20.jpg
Windows 10 version 11.0.7633 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.29% 

 

VirIT eXplorer PRO

Windows 7 version 9.0.95  02-20.jpg
Windows 10 version 9.0.94 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 63.13% 

 

Wontok SafeCentral Security Suite

Windows 7 version 2.0.1653  02-20.jpg
Windows 10 version 2.0.1653 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.59% 

 

Appendix 1: products not certified

There were two products that failed to achieve VB100 certification in this test: Cyren Command Anti-Malware and SecureAge Plus Pro.

 

Appendix 2: testing notes

BitSentry Elite had eight missed detections in the Diversity set. The product’s developers provided new product settings during the dispute period. Using the new settings the samples that were missed in the test would have been detected. However, the VB100 rules dictate that, since the data collection period had already ended, the results could not be changed in the final VB100 report. The amended settings will be used in future tests.

 

Appendix 3: sample set sizes

The WildList (certification) set contained 2,948 malicious samples. The set of clean samples used for the false positive test contained 100,000 files, of which 29,228 were portable executable (PE) files. The set used for the Diversity Test contained 981 malicious samples.

 

Footnotes

1 The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Download PDF

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VB100 Certification Report - August 2020

This report details the VB100 certification results of 43 anti-malware products from 36 different vendors during July and August 2020.

VBSpam Comparative Review - June 2020

In this test nine full email security solutions and four blacklists of various kinds were assembled on the test bench to measure their performance against various streams of wanted, unwanted and malicious emails.

VB100 Certification Report - June 2020

This report details the VB100 certification results of 44 anti-malware products from 38 different vendors during May and June 2020.

VB100 Certification Report - April 2020

This report details the VB100 certification results of 42 anti-malware products from 36 different vendors during March and April 2020.

VBWeb Comparative Review - Spring 2020

In the final VBWeb test we measure the performance of three web security products against a wide range of live web threats, including phishing threats. This report also includes an overview of the Spring 2020 threat landscape.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.