VB100 certification report June 2019

Martijn Grooten

Virus Bulletin

Copyright © 2019 Virus Bulletin


 

Introduction

For more than two decades, Virus Bulletin has been testing anti-malware products, reporting on their performance as both the threat landscape and the products themselves have evolved, and has provided the VB100 certification scheme as a mark of distinction for those products that live up to expectation. The VB100 logo is the 'stamp of approval' conferred on products that satisfy a minimum standard of detecting malicious executables that have recently been seen in the wild, while blocking few to no legitimate programs.

This report details the VB100 certification of 34 anti‑malware products from 31 different vendors tested during May and June 2019.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on two platforms: Windows 10 and Windows 7. On each platform, and at three different times in the test, the product is asked to scan both the latest version of the WildList1 and a selection of clean files taken from Virus Bulletin's own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn't blocked is considered a miss.

A product achieves a VB100 certification if:

  • No more than 0.5% of WildList samples are missed

and

  • No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-1/. This test used version 1.1 of the VB100 methodology.

 

Diversity Test

The malware part of the VB100 certification uses the WildList, a regularly updated list of extremely well-vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The 'Diversity Test' looks at products' detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples, and provides a measure of that detection.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follow refer to those at the start of the test.

 

Adaware Antivirus Free

Windows 7 version 12.6.1005.11662 06-19.jpg
Windows 10 version 12.6.1005.11662
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

Adaware Antivirus Pro

Windows 7 version 12.6.1005.11662  06-19.jpg
Windows 10 version 12.6.1005.11662 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.80% 

 

AhnLab V3 Internet Security 9.0

Windows 7 version 9.0.53.1  06-19.jpg
Windows 10 version 9.0.53.1 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00%

 

Arcabit AntiVirus

Windows 7 version 2019.05.08 06-19.jpg
Windows 10 version 2019.05.07
WildList detection 99.9%
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

Avast Free Antivirus

Windows 7 version 19.4.2374 06-19.jpg
Windows 10 version 19.4.2374 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

AVG Internet Security

Windows 7 version 19.4.3089  06-19.jpg
Windows 10 version 19.4.3089 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

Cynet 360

Windows 7 version 8.3.54.26  06-19.jpg
Windows 10 version 8.3.54.26 
WildList detection 100.0%
False positive rate 0.000%
Diversity Test rate 99.90%

 

Defenx Security Suite

Windows 7 version 15.1.0114  06-19.jpg
Windows 10 version 15.1.0112 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.80% 

 

Emsisoft Anti-Malware

Windows 7 version 2019.4.0.9412  06-19.jpg
Windows 10 version 2019.4.0.9412 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

eScan Internet Security Suite for Windows

Windows 7 version 14.0.1400.2029  06-19.jpg
Windows 10 version 14.0.1400.2029 
WildList detection 99.96% 
False positive rate 0.000%
Diversity Test rate 100.00%

 

ESET Internet Security

Windows 7 version 12.1.34.0  06-19.jpg
Windows 10 version 12.1.34.0 
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate 100.00%

 

ESTsecurity ALYac

Windows 7 version 4.0.2.23116  06-19.jpg
Windows 10 version 4.0.2.23116 
WildList detection 100.0%
False positive rate 0.000%
Diversity Test rate 99.90%

 

Exosphere Endpoint Protection

Windows 7 version 8.3.54.26  06-19.jpg
Windows 10 version 8.3.54.26 
WildList detection 99.9% 
False positive rate 0.000%
Diversity Test rate 100.00%

 

Faronics Anti-Virus

Windows 7 version 4.20.3102.471  06-19.jpg
Windows 10 version 4.20.3102.471 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00%

 

FireEye Endpoint Security

Windows 7 version 29.7.0  06-19.jpg
Windows 10 version 29.7.0 
WildList detection 99.9% 
False positive rate 0.000%
Diversity Test rate 99.90%

 

Fortinet FortiClient

Windows 7 version 5.6.2.1117  06-19.jpg
Windows 10 version 5.6.2.1117 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00%

 

G DATA Antivirus

Windows 7 version 25.5.3.4  06-19.jpg
Windows 10 version 25.5.0.4 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00%

 

Heimdal Thor Vigilance

Windows 7 version 2.5.173  06-19.jpg
Windows 10 version 2.5.173 
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate N/A

 

IKARUS anti.virus

Windows 7 version 2.19.14  06-19.jpg
Windows 10 version 2.19.14 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.90%

 

 K7 Total Security

Windows 7 version 15.1.0360  06-19.jpg
Windows 10 version 15.1.0360 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.90%

 

Kaspersky Endpoint Security 11 for Windows

Windows 7 version 11.1.0.15919  06-19.jpg
Windows 10 version 11.1.0.15919 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.90%

 

NANO Antivirus

Windows 7 version 1.0.134.90362  06-19.jpg
Windows 10 version 1.0.134.90324 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

PCProtect

Windows 7 version 4.13.75  06-19.jpg
Windows 10 version 4.13.75 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

Rising Security Cloud Client

Windows 7 version 3.0.0.85  06-19.jpg
Windows 10 version 3.0.0.85 
WildList detection 99.96% 
False positive rate 0.003% 
Diversity Test rate 86.57%

 

Scanguard

Windows 7 version 4.13.75  06-19.jpg
Windows 10 version 4.13.75 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

TACHYON Endpoint Security

Windows 7 version 5.0.0.57  06-19.jpg
Windows 10 version 5.0.0.57 
WildList detection 99.96% 
False positive rate 0.003% 
Diversity Test rate 99.60%

 

TeamViewer Endpoint Protection

Windows 7 version 1.1.175782  06-19.jpg
Windows 10 version 1.1.175782 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.90% 

 

Tencent PC Manager

Windows 7 version 12.3.26601.901  06-19.jpg
Windows 10 version 12.3.26601.901 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 96.36% 

 

Total Defense Premium

Windows 7 version 9.0.0.773  06-19.jpg
Windows 10 version 9.0.0.773 
WildList detection 100.0% 
False positive rate 0.000%
Diversity Test rate 99.90%

 

Total Defense Unlimited

Windows 7 version 11.0.0.775  06-19.jpg
Windows 10 version 11.0.0.775 
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

TotalAV

Windows 7 version 4.13.75  06-19.jpg
Windows 10 version 4.13.75 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

VIPRE Advanced Security

Windows 7 version 11.0.4.2 06-19.jpg
Windows 10 version 11.0.4.2
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

VirIT eXplorer PRO

Windows 7 version 8.9.41  06-19.jpg
Windows 10 version 8.9.23 
WildList detection 100.0% 
False positive rate 0.001%
Diversity Test rate 71.41% 

 

Wontok SafeCentral Security Suite

Windows 7 version 2.0.1548  06-19.jpg
Windows 10 version 2.0.1548 
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Appendix 1: products not certified

Cyren failed to achieve VB100 certification in this test because its number of false positives was just above the threshold for VB100 certification.

 

Appendix 2: excluded parts

  • For adaware AntiVirus Free and adaware AntiVirus Pro, the results of the Diversity Test on Windows 7 were discarded due to technical issues.
  • For Heimdal Thor Vigilance, the results of the Diversity Test were discarded due to technical issues.

 

Appendix 3: sample set sizes

The WildList contained 2,236 samples. The set of clean files used for the false positive test contained 99,999 files, of which 29,123 were portable executable (PE) files. The set used for the Diversity Test contained 990 samples.

 

Footnotes

1 The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Download PDF

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VBWeb Comparative Review - Autumn 2019

The VBWeb tests measure the performance of web security products against a wide range of live web threats, this time also including phishing threats.

VBSpam comparative review - September 2019

In the September 2019 VBSpam test 11 full email security solutions and five blacklists of various kinds are put through their paces, resulting in 11 VBSpam certifications and six VBSpam+ awards.

VB100 Certification Report

The August 2019 report details the VB100 certification of 37 anti‑malware products from 34 different vendors tested during July and August 2019.

VBWeb Comparative Review - Summer 2019

The VBWeb tests measure the performance of web security products against a wide range of live web threats, this time also including phishing threats.

VB100 certification report

The June 2019 VB100 report details the VB100 certification of 34 anti‑malware products from 31 different vendors tested during May and June 2019.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.