VB100 certification report - February 2021

VB Test Team

Virus Bulletin

Copyright © 2021 Virus Bulletin


Introduction

Anti-malware products don’t have an easy job. There are millions of malware samples (many of which are written with the explicit goal of bypassing anti-malware products) to be detected and blocked, while at the same time there are millions of legitimate programs (which occasionally use some of the same techniques as malware) that mustn’t be blocked. And there are dozens of competing products on the market.

Users should expect anti-malware products to satisfy a minimum standard of blocking malicious executables that have recently been seen in the wild, while blocking few to no legitimate programs.

For more than two decades, Virus Bulletin has set a minimum standard for anti-malware products, checking whether products live up to expectation and providing those that do with the VB100 ‘stamp of approval’.

This report details the VB100 certification results of 43 such products from 38 different vendors during January and February 2021.

 

The VB100 set-up

In the VB100 test, a copy of the product to be tested is installed on Windows 10. At three different times in the test, the product is asked to scan both the latest version of the WildList and a selection of clean files taken from Virus Bulletin’s own set of files belonging to widely used legitimate software.

A legitimate file that is blocked at least once is considered a false positive, while a WildList file that isn’t blocked is considered a miss.

A product achieves a VB100 certification if:

  • No more than 0.5% of WildList samples are missed

and

  • No more than 0.01% of legitimate files are blocked

For full details, we refer to the VB100 methodology on the Virus Bulletin website: https://www.virusbulletin.com/testing/vb100/vb100-methodology/vb100-methodology-ver1-2/. This test used version 1.2 of the VB100 methodology.

 

Diversity Test

The malware part of the VB100 certification uses the WildList1, a regularly updated list of extremely well-vetted malware samples, guaranteed to have been spotted in the wild multiple times. This makes them very suitable for a certification test like VB100.

The ‘Diversity Test’ looks at products’ detection of another set of recent malware samples, to acknowledge the fact that products detect malware samples beyond a standard set of samples, and provides a measure of that detection.

 

Products & results

Products were allowed to download updates during the course of the test. The version numbers listed in the results that follows refer to those at the start of the test.

 

Acronis Cyber Protect

Windows 10 version 12.5.23140 02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Acronis True Image 2021

Windows 10 version 2021 build 35860  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Adaware Antivirus Free

Windows 10 version 12.10.60.0  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.70% 

 

Adaware Antivirus Pro

Windows 10 version 12.10.60.0  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.70% 

 

Ad-Spider

Windows 10 version 2021.01.08  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.20% 

 

AhnLab V3 Endpoint Security

Windows 10 version 9.0.64.11  02-21.jpg
WildList detection 100.0%
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

Arcabit AntiVirus

Windows 10 version 2021.01.09  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

Avast Free Antivirus

Windows 10 version 20.10.2442  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000%
Diversity Test rate 100.00% 

 

AVG Internet Security

Windows 10 version 20.10.3157  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

CMC Malware Detection and Defense

Windows 10 version 1.8.2020 build 81  02-21.jpg
WildList detection 99.7% 
False positive rate 0.000% 
Diversity Test rate 59.10% 

 

CORE Antivirus

Windows 10 version 1.0.78.0  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Coronet SecureCloud

Windows 10 version 1.0.762.221  02-21.jpg
WildList detection 99.9%
False positive rate 0.000% 
Diversity Test rate 71.20% 

 

Cynet 360

Windows 10 version 8.3.62.140  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.20% 

 

CyRadar Endpoint Detection and Response

Windows 10 version 1.0.0.56  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 97.90% 

 

Defenx Security Suite

Windows 10 version 1.7.6.0  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.20%

 

Emsisoft Anti-Malware

Windows 10 version 2021.1.0.10621  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

eScan Internet Security Suite for Windows

Windows 10 version 14.0.1400.2228  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

ESTsecurity ALYac

Windows 10 version 4.0.2.23116  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.70% 

 

Exosphere Endpoint Protection

Windows 10 version 1.5.22.3  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Faronics Anti-Virus

Windows 10 version 4.21.3102.484  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

FireEye Endpoint Security

Windows 10 version 32.30.13  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.60% 

 

Fortinet FortiClient

Windows 10 version 6.2.7.0984  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

G DATA Antivirus

Windows 10 version 25.5.8.14  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

IKARUS anti.virus

Windows 10 version 3.2.4  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Intego AV

Windows 10 version 1.0.1.7  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80%

 

K7 Total Security

Windows 10 version 16.0.0665  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

PCProtect

Windows 10 version 5.5.83  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.20% 

 

Private Internet Antivirus

Windows 10 version 1.0.1.6  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Qi-ANXIN Tianqing Endpoint Security Management System

Windows 10 version 6.6.0.4088  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.80% 

 

Rising Enterprise Security Management System

Windows 10 version 3.0.0.99  02-21.jpg
WildList detection 100.0% 
False positive rate 0.002% 
Diversity Test rate 83.60% 

 

Scanguard

Windows 10 version 5.5.83  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.20% 

 

Shield Antivirus

Windows 10 version 4.7.2  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.20% 

 

Systweak Anti-virus

Windows 10 version 1.0.1000.11042  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

TACHYON Endpoint Security

Windows 10 version 5.0.1.33  02-21.jpg
WildList detection 99.9% 
False positive rate 0.010% 
Diversity Test rate 99.60% 

 

TeamViewer Endpoint Protection

Windows 10 version 20.9.1  02-21.jpg
WildList detection 99.9% 
False positive rate 0.000% 
Diversity Test rate 99.60% 

 

Tencent PC Manager

(see notes in Appendix 2)

Windows 10 version 12.3.26609.901  02-21.jpg
WildList detection 99.8% 
False positive rate 0.000%
Diversity Test rate 98.00% 

 

TotalAV

Windows 10 version 5.5.83  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

Total Defense Premium

Windows 10 version 12.0.0.306  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 99.90% 

 

TUXGUARD Endpoint Protection

Windows 10 version 1.0.1.96  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

United Endpoint Protector

Windows 10 version 1.3  02-21.jpg
WildList detection 100.0% 
False positive rate 0.006% 
Diversity Test rate 100.00% 

 

VIPRE Endpoint Cloud Business

Windows 10 version 12.0.7874  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 100.00% 

 

VirIT eXplorer PRO

Windows 10 version 9.3.44  02-21.jpg
WildList detection 100.0% 
False positive rate 0.000% 
Diversity Test rate 39.10% 

 

Appendix 1: products not certified

SecureAge SecureAPlus Pro didn’t achieve VB100 certification in this test.

 

Appendix 2: testing notes

  • Due to technical failure one test part was invalidated for Tencent PC Manager.

 

Appendix 3: sample set sizes

The Certification Set contained 1,384 malicious samples. The set of clean samples used for the false positive test contained 100,000 files, of which 27,102 were portable executable (PE) files. The set used for the Diversity Test contained 1,000 malicious samples.

 

Footnotes

1 The WildList is an extremely well-vetted set of malware recently observed in the wild by researchers: http://www.wildlist.org/.

Download PDF

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VBSpam Comparative Review - March 2021

The performance of 28 email security products was measured against various streams of wanted, unwanted and malicious emails. The results for the 10 products that elected to be tested publicly are included in this report.

VB100 Certification Report - February 2021

This report details the VB100 certification results of 43 anti-malware products from 38 different vendors during January and February 2021.

VB100 Certification Report - December 2020

This report details the VB100 certification results of 41 anti-malware products from 36 different vendors during November and December 2020.

VBSpam Comparative Review - December 2020

In this test six full email security solutions, one custom configured solution and four blacklists of various kinds elected to be publicly tested against various streams of wanted, unwanted and malicious emails.

VB100 Certification Report - October 2020

This report details the VB100 certification results of 41 anti-malware products from 35 different vendors during September and October 2020.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.