VB100 comparative review on Windows Server 2008 R2 SP1

2014-12-15

John Hawes

Virus Bulletin
Editor: Martijn Grooten

Abstract

The VB test team put 29 products through their paces on Windows Server 2008 and, for the first time ever on a Windows platform, all products achieved VB100-certified status - John Hawes has the details.


Introduction

After another rather epic desktop test last time, a server test came as something of a relief, with far fewer products submitted and considerably less likelihood of horrific instability – while home users may put up with wobbly software, server admins tend to demand better, and judging by our experience so far, their needs are respected by product and platform developers alike. With the last few tests overrunning, our schedule remained tight for this one, with much of the lab’s resources still devoted to completing the previous report as this one was getting under way.

The submission deadline was set for 27 August, and testing ran from mid-September to early November. On the deadline day, a manageable batch of 29 products were received, including most of our regulars with just a few less familiar names to keep us on our toes. Of the 29 products submitted, the ever-popular Bitdefender engine featured in nine, almost one in three.

The relatively low number of products contained in this report may be in part due to a change in the funding model of the VB100 certification programme – while up until the start of 2014, we allowed any vendor to submit a product into any test free of charge, free entry has now been limited to three tests per vendor per year and a fee now applies for any additional submissions. This has meant that some vendors have chosen to participate only in the desktop tests – we expect to see the usual wider range of products in our upcoming Windows 7 comparative.

Platform and test sets

This month’s platform was Windows Server 2008 R2 SP1, the ‘standard’ edition of a platform that supports only 64-bit hardware. The server sibling of Windows 7, the R2 version has been around for five years or so now but still has some life in it – the ‘extended support’ period even for the original Vista-like versions stretches on until 2020, and no end date has yet been set for R2.

Installation of the platform is a pretty straightforward task with which the lab team has ample experience, and there were no surprises or difficulties. The platform was deployed to our set of identical test systems as a fresh install from the standard MSDN installation media, with no later updates, and additional software was limited to a set of basic tools to aid in testing (mainly archive- and document-handling applications, plus an alternative browser). Snapshots of the systems were taken in preparation, and in the case of the products that could not be updated offline, additional snapshots were recorded after installation and updating of the products on the deadline day, for use in the retrospective parts of the RAP test.

The standard VB100 test sets were later deployed to all systems, with the certification set based on the v4.008 edition of the WildList released two weeks prior to the test deadline. We had another clear out of our clean sets to remove the usual effluvium of free software add-ons and other unsuitable items, and produced the RAP sets in the usual fashion – based on the most significant samples seen by us in the appropriate time periods (ten days prior to the test date in the case of the reactive sets and ten days after freezing products for the proactive part of the test). The sample sets used for the speed and performance measures remained unchanged, as did the processes and automation used for running these parts of the comparative.

Results

Agnitum Outpost Security Suite

Main version: 9.1

Update versions: 4653.701.1951 build 890, build 945, build 947, build 951

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 7 passed, 0 failed, 5 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Agnitum’s Outpost suite has put in a string of strong performances in our comparatives over the last year or so, with passes in all of its appearances.

The set-up process takes a little while, but there are various stages including updates and initial scans rolled in, and once available the interface is crisp, angular and uncluttered, with a decent range of configuration options that are simple to find and operate. Stability was flawless throughout testing, even under heavy pressure.

Scanning speeds were slow to start with, but very fast later on, and on-access overheads were likewise pretty high initially, and a little better once the product had settled in, but still not the lightest. Resource use was a little on the high side, and our set of activities saw a pretty significant slowdown.

Detection was decent, with a steady decline through the RAP weeks, and the WildList and clean sets were handled satisfactorily, earning Agnitum another VB100 award.

Avast File Server Security

Main version: 8.0.1603

Update versions: 140827-0, 141015-2, 141019-1, 141026-1

Last 6 tests: 3 passed, 2 failed, 1 no entry

Last 12 tests: 8 passed, 3 failed, 1 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Avast’s products have had some bad luck in our tests lately, having been denied awards on a couple of occasions thanks to fairly minor or transitory problems, but their long-term history shows a high rate of success and it seems likely that things will get back to normal.

Installation of the server edition has an ‘express’ option and runs through pretty quickly. The interface is smart, attractive and friendly, displaying a smiley face to indicate successful set-up. The layout is similar to the vendor’s home offerings, making operation easy for sysadmins familiar with the free solutions. Configuration options are provided in excellent depth, and are sensibly laid out and easy to find. No problems with stability were noted.

Scanning speeds were a little on the slow side, overheads light with the default settings (which do not analyse files on-read), and rather heavy with full protection against all file types enabled. RAM use was low, CPU use a little high, and our set of tasks was impacted noticeably.

Detection was strong, with only a slight dip noted into the proactive RAP sets, and the WildList sets were handled without problems. The clean sets were only marked by a fairly high number of warnings of over-compressed files, and thus Avast earns a VB100 award, hopefully putting it back on track.

AVG CloudCare Anti Virus 2014

Main version: 2014.0.4745

Update versions: 4007/8096, 2014.0.04765/4040/8398, 8433, 8468

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 11 passed, 1 failed, 0 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Our test history for AVG’s products is impressive, with long strings of consecutive passes and very few tests not entered. The set up for the ‘CloudCare’ version offers an express option, but still takes some time to complete, especially at the updating stage – one install needed almost 15 minutes to download its data.

The GUI has a dark background but the information and controls it displays are reasonably visible; depth of configuration is good and the layout is simple to figure out. Stability was knocked by a couple of incidents, including one where the machine halted under heavy load, but it remained in the ‘fair’ category.

Scanning speeds started pretty fast and sped up considerably in the warm runs; overheads were fairly low, and resource use was low too. Our set of activities ran through a little slower than the baselines, but not very much so.

Detection was very good, with the proactive sets especially well handled, and there were no issues in the WildList or clean sets, the only alert in the clean sets being a warning about a broken digital signature. AVG thus earns another VB100 award for its efforts.

Avira Server Security

Main version: 14.0.6.570

Update versions: 8.03.24.18/7.11.169.76, 8.03.24.38/ 7.11.178.234, 7.11.180.144, 8.03.24.040/7.11.181.152

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Avira’s history in our tests is excellent, with a high number of passes and no fails so far this decade – although we have seen a small number of missed tests lately. Installation of the server version is quick and simple with a brief appearance from the ‘Luke Filewalker’ scanner as part of the process.

The interface uses the MMC console and has something of a learning curve, but after some familiarization it becomes fairly simple to navigate and operate, with a thorough selection of configuration options provided. Stability was excellent, with no issues noted.

Scanning speeds were steady and consistent – not super fast, but not too bad – and overheads were very light thanks to minimal on-read scanning. Resource use was low, as was the impact on our set of activities.

Detection was excellent, with only the slightest dip in the very last part of the RAP sets, and the certification sets were handled impeccably, with just a single item in the clean sets flagged as a possible ‘logic bomb’. Another splendid performance from Avira comfortably earns the vendor another VB100 award.

Baidu Antivirus

Main version: 1.8.15.128

Update versions: N/A

Last 6 tests: 1 passed, 1 failed, 4 no entry

Last 12 tests: 1 passed, 1 failed, 10 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Baidu’s history in our tests is rather sporadic, with a single appearance from the international version a while ago, and this, the Chinese-only version aimed at the domestic market, also appearing only once and with rather less success.

Installation wasn’t too tricky despite the language barrier, and completed extremely rapidly. The interface looks fairly clean, simple and uncluttered, and with a little help from a guide document it provided all the options required for testing within fairly easy reach. Stability was hit by a couple of problems with scans crashing out and failing to record any results, as well as a single incident when the system locked up entirely and had to be power cycled to wake it up.

Scanning speeds were OK, with some signs of improvement in the warm runs, particularly over binaries. Overheads were very light, as on-read protection is not provided, and our activities test showed a fairly significant slowdown and rather high resource use.

Detection was decent, as one would expect from the Kaspersky engine included in this product: strong in the reactive sets and dipping just a little into the proactive weeks. The certification sets were handled well, with just a couple of ‘risktool’ warnings in the clean sets, earning Baidu’s Chinese edition its first VB100 award.

Bitdefender Endpoint Security

Main version: 5.3.8.408

Update versions: 7.56549, 7.57281, 7.57352, 7.57426

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 12 passed, 0 failed, 0 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Bitdefender’s history in our tests had been superb of late, with a clean sweep of the last 20 or so comparatives and a high pass rate going back much further. Installation of the ‘Endpoint’ version takes rather a while, and once up the interface is simple and pared down with only minimal controls provided to the end-user (fine-tuning is presumably offloaded to a central management system).

Stability was impeccable, with our entire suite of high-stress tests handled without problems.

Scanning speeds were OK initially, and lightning-fast in the warm runs; overheads were very light, resource use low, and impact on our set of activities pretty low.

Detection was excellent – the proactive weeks a little lower than the reactive ones, but still very impressive. The certification sets were handled just as well, and Bitdefender’s winning streak continues with another VB100 award.

BluePex AVware Internet Security

Main version: 1.5.0.18

Update versions: 4193, 5088, 5206

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 2 passed, 0 failed, 10 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

A relative newcomer to our tests, BluePex’s VB100 record includes just a single full appearance – a pass in the last comparative – although we’ve seen it on a number of occasions in the past, on each of which it proved too unstable to make it to the final report.

Installation takes a little while, partly thanks to the product’s reliance on the .NET framework, and the interface is simple with only minimal configuration options. Once again, stability was not great, with a number of problems including scans crashing out, logging failures and odd switches of GUI language.

Scanning speeds were pretty sluggish, and overheads a little high over binaries but not bad elsewhere. Resource use was average, but impact on our set of activities was a little high.

Detection was decent: strong in the first part of the RAP sets, then dipping a little but maintaining a good level. The certification sets were handled well, and BluePex earns its second VB100 award.

CYREN Command Anti-Malware

Main version: 5.1.31

Update versions: 5.4.11

Last 6 tests: 2 passed, 2 failed, 2 no entry

Last 12 tests: 2 passed, 7 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

CYREN’s history in our tests is complicated by multiple company mergers and rebrandings, and tainted by sporadic problems with false positives, but the Command name remains intact. Installation is very quick, as are updates, with the interface very simple and minimalist but offering all the basic controls required.

Stability was OK, with a few large scans crashing out and an instance of settings being ignored.

Scanning speeds were a little slow, on-read overheads distinctly high, and our set of activities was slow too, although resource use wasn’t too heavy.

Detection was splendid in the reactive sets, dropping sharply in the later parts of the RAP test, and the certification sets were properly handled, with no issues in the clean sets for once. Command therefore earns a VB100 award – its first under the CYREN name.

Defenx Security Suite 9.1

Main version: 9.1

Update versions: 4557.690.1951 build 890, build 945, build 947, build 951

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 5 passed, 0 failed, 7 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

There is something of a gap in Defenx’s VB100 test history, but its performance has been strong in the last year or so. Installation takes a little while to progress through its various steps, and the product interface – similar in layout to partner firm Agnitum’s – is angular, unfussy and sensible with a decent range of configuration options.

Stability was not a problem, with no issues observed throughout testing.

Scanning speeds were slow to start with but very fast indeed in the warm runs, and overheads also showed huge improvements from a fairly heavy start. Resource use was low, but our set of tasks did take quite some time to complete.

Detection was decent, with a gradual downward trend through the RAP sets, and the certification sets were well handled with no problems to report. Defenx thus earns another VB100 award, continuing to build a good run of passes.

Emsisoft Anti-Malware

Main version: 9.0.0.4324

Update versions: 9.0.0.4570

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 9 passed, 1 failed, 2 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Emsisoft’s performance in our tests has been pretty strong of late, with a good pass rate and only our Linux comparatives skipped in the last few years. Installation of the latest version took quite a long time, although later updates were very quick indeed.

The interface is pretty attractive and clear, with nice large icons indicating status and leading to a strong set of controls. Stability was great, with no issues noted.

Scanning speeds were on the slow side and overheads pretty light with the default settings, speeding up after initial exploration; turning the settings up to the max slowed things down somewhat, as one would expect. Our set of activities showed a noticeable slowdown, but resource use was very low.

Detection was strong, barely wavering through the RAP sets, and the certification sets were handled with no unwanted misclassifications. Emsisoft’s good work earns it a VB100 award.

eScan ISS Server Edition

Main version: 14.0.1400.1645 DB

Update versions: N/A

Last 6 tests: 5 passed, 1 failed, 0 no entry

Last 12 tests: 11 passed, 1 failed, 0 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

There’s a good ratio of passes in our test history for eScan, with the product having participated in all tests over the last few years. Installation of the current server version isn’t the speediest, and updates can take a little while too, but once available the interface is crisp and responsive, with the grey-on-grey styling brightening up each time we see it. Configuration options are provided in excellent depth, and are well ordered and simple to navigate.

Stability was decent, with just a single scan getting stuck and a few instances of rather odd desktop behaviour.

Scanning speeds were quick from the off and improved later; overheads were not bad initially and were negligible once settled in. Resource use was low, and our set of activities was completed in good time.

Detection was excellent, with just a small dip into the proactive sets, and with no problems in the certification sets, eScan easily earns another VB100 award.

ESET Endpoint Antivirus

Main version: 5.0.2229.1

Update versions: 10324, 10573, 10591, 10633

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 12 passed, 0 failed, 0 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

ESET’s history in our tests is remarkable in its consistency with a solid block of passes stretching back into the distant past. Installation of the vendor’s ‘Endpoint’ version is speedy, with a blink-and-you’ll-miss-it approach to updating. The interface has the usual pleasantly unflashy feel – simple and clear, with easy access to a complete range of configuration controls.

Stability was once again impeccable, with no problems anywhere in our test suite.

Scanning speeds weren’t too fast first time round, but the product blazed through the warm runs. There was a negligible impact on file access times, low resource use, and the impact on our set of activities was noticeable, but not too high.

Detection was very strong, as usual, with just the faintest hint of a downward slope through the RAP sets. Once again, the certification sets were dealt with perfectly, earning ESET yet another VB100 award.

ESTsoft ALYac 3

Main version: 3.0.0.4

Update versions: 13.3.21.1/536913.2014082718/7.56545/6806408.20140827, 13.3.21.1/540668.2014101800/7.57280/6012434.20141017, 13.3.21.1/541141.2014102320/7.57350/5652003.20141023, 13.3.21.1/541534.2014103018/7.57433/5688017.20141030

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 8 passed, 1 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

ESTsoft’s history in our tests is building up nicely, with a good run of passes of late. Installation of the current edition was quick, although updates could take a while and on one occasion seemed not to work at all.

The interface is, as ever, adorned with a cuddly cartoon character, looking friendly and accessible. There is a reasonable set of configuration options, although users may need a little practice to find their way around. Stability was decent, the update issues being the only real problem noted.

Scanning speeds were slow to start with but improved greatly into the warm runs; overheads were a touch high first time around but much better later on. Resource use was very low, but our set of tasks was slowed down somewhat.

Detection was very strong in the reactive sets, and still very good in the proactive parts of the RAP test. The certification sets proved no problem, and ESTsoft adds another VB100 award to its growing collection.

Faronics Anti-Virus

Main version: 3.40.2102.247

Update versions: 3.9.2595.2/32586, 33948, 34112, 34312

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 2 passed, 0 failed, 10 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Faronics first appeared on our test bench for the last VB100 comparative – on Windows 8.1 – notching up its first pass on its first attempt.

Installation was a little complex as we required a management console from which to deploy and configure the client, a modular system providing several useful solutions in a unified package. The client interface is simple and functional with minimal fuss and adornment, providing some nice clear status info and some basic local options; the console uses the MMC system and proved simple to navigate.

Stability was good, the only issues being some minor error messages popping up during deployment which could well have been down to unusual usage patterns.

Scanning speeds were not too fast initially but improved greatly, while overheads were mostly light, the first run over our set of executables showing the only noticeable impact. Our set of activities was a little affected, with fairly high RAM use, but CPU use was minimal.

Detection from the integrated VIPRE engine started very strong in the initial part of the RAP sets before plateauing at a good level, and the certification sets were handled nicely, earning Faronics its second VB100 award.

Fortinet FortiClient

Main version: 5.0.9.347

Update versions: 5.152/22.706, 5.158/23.003, 23.047, 23.076

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 9 passed, 1 failed, 2 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

It has been more than five years since Fortinet missed anything but a Linux test, and only in very few of its entries has it missed out on a VB100 award. Installation of FortiClient is reasonably fast with very quick updates, and the interface presented to end-users is simple in the extreme with very little control available but plenty of information.

Stability was again upset by a handful of sudden shutdowns, which intensive investigation seems to suggest is related to interaction with hard disk drivers and may be limited to a small set of hardware configurations. We also noted a single instance of wobbliness in the protection under extreme pressure.

Scanning speeds were OK and very dependable, overheads a little high with some improvement in later runs; resource use was average, and our set of activities ran through in reasonable time with just a little slowdown.

Detection was excellent once again, with a very slight downward slope through the RAP sets. The certification sets proved no problem at all, with flawless coverage, and Fortinet earns a VB100 award quite comfortably.

G Data Security Client/Administrator

Main version: 13.0.0.166

Update versions: AVA 24.3785/GD 25.3804, AVA 24.4513/GD 25.4048, AVA 24.4584/GD 25.4048, AVA 24.4659/GD 25.4090

Last 6 tests: 4 passed, 0 failed, 2 no entry

Last 12 tests: 8 passed, 0 failed, 4 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

G Data’s history in our tests is strong, with a good rate of passes and some superb performances on the detection front thanks to the combination of a strong in-house engine with that of Bitdefender. Installation required both the administration system, with associated dependencies, and a local client, so took a little while.

The admin interface is used for most configuration and control tasks and seemed simple to operate, although we do have some prior experience of using it, including in its native German. Stability was pretty good, the only issue observed being a very minor one: attempts to connect to the management system failed if performed too soon.

Scanning speeds were decent and steady, overheads fairly heavy first time off but minimal in normal use. Resource use wasn’t too bad, but our set of activities did take a while to complete.

Detection was excellent, near perfect in the reactive sets and solid in the proactive parts too. The certification sets presented no hurdle, and G Data earns a VB100 award in its usual style.

Ikarus anti.virus

Main version: 2.7.30

Update versions: 1.7.5/88691, 1.7.8/89418, 1.7.8/89449, 1.7.8/89478

Last 6 tests: 3 passed, 1 failed, 2 no entry

Last 12 tests: 6 passed, 3 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Our test history for Ikarus shows a steady improvement over the last few years, with problems with false positives that plagued it in the past far less frequent these days. The product still requires the .NET framework, adding a little to the install time, which remains pretty short nevertheless.

The GUI itself is very familiar to us, having changed little in the many years we’ve been testing it. It adopts a pretty simple and standard layout, which should make it simple to navigate even for first-time users; configuration depth is reasonable, with all the main areas covered.

Stability was decent, the only issue being at one point when it seemed impossible to deactivate the on-access protection properly without rebooting – most users, of course, would have no reason to do this anyway.

Scanning speeds were very slow at first but very fast indeed when rescanning; overheads were very heavy with a slight improvement later on. Our set of activities showed very little impact though, with fairly low resource use too.

Detection was very strong in the reactive sets and still good into the proactive parts of the RAP sets, and with no problems in the certification sets, Ikarus is well deserving of another VB100 award.

iSheriff Cloud Security

Main version: 5.1.1.0821

Update versions: 5.1.1/12.163, 5.1.1.1001, 5.1.2/ 12.163

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

The iSheriff cloud-focused solution has a rather complicated heritage, but its history in our tests now shows two full years of participation by the same Bitdefender-based product with passes on every attempt.

Installation involves operation of an online portal to access install packages, with Mac and Linux versions also available. The set-up process is reasonably speedy and not too arduous, although familiarity with the solution may have helped us considerably.

The interface is operated via a browser and provides a reasonable set of controls, but it suffered a little from lag and stickiness, and a number of jobs failed to complete properly or failed to log accurately, impacting our stability rating considerably.

Scanning speeds were decent with the default settings and overheads not too bad. There was some impact on our set of activities, but very little resource usage noticeable.

Detection was solid, with high scores in the reactive parts of the RAP sets and still strong coverage of the proactive sets. The certification sets presented no problems and iSheriff earns a VB100 award.

K7 Total Security

Main version: 14.2.0.242

Update versions: 9.183.13166, 14.2.0253/ 9.184.13719, 9.185.13753, 9.185.13851

Last 6 tests: 3 passed, 1 failed, 2 no entry

Last 12 tests: 5 passed, 2 failed, 5 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

K7’s history in our tests is steadily building up, with consistent entries in our desktop tests over the last few years and a decent number of passes. Installation is quick and easy, the interface offers the usual rugged, military styling with good clear status info and good access to a solid set of configuration options.

Stability was flawless, with no problems in dealing with the pressures of the test suite.

Scanning speeds were decent with good consistency, overheads were not bad initially and barely detectable once settled in. RAM use was a little higher than most, CPU use very low indeed, and our set of activities wasn’t badly impacted.

Detection was reasonable, with a minimal decline across the RAP sets, and the certification sets were dealt with well, earning K7 another VB100 award.

Kaspersky Anti-Virus 8 for Windows Servers Enterprise Edition

Main version: 8.0.1.923

Update versions: N/A

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 10 passed, 1 failed, 1 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

The first of three products from Kaspersky Lab this month, the plain ‘Anti-Virus’ version slotted as usual into our mainline test history for the firm – one of our most complete records, with very few gaps since the inauguration of the VB100 test and a strong pass rate in the last few years.

Installation of this server edition was very quick in itself, but updating took quite some time. The interface is another based on the MMC, with a fairly straightforward layout and the comprehensive set of controls one would expect from a serious business product.

Stability was rock-solid throughout, without the slightest wobble observed.

Scanning speeds were rather slow, overheads not bad with some further optimization in later runs, while RAM and CPU use was average and our set of tasks was not badly impacted.

Detection was pretty decent, with just the slightest dip into the very last part of the RAP sets. There were no problems in the certification sets, meaning that Kaspersky Lab’s first submission this month easily earns a VB100 award.

Kaspersky Endpoint Security 10 for Windows

Main version: 10.2.1.23

Update versions: N/A

Last 6 tests: 3 passed, 0 failed, 3 no entry

Last 12 tests: 4 passed, 0 failed, 8 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

The second offering from Kaspersky Lab this month fits into our test history for the vendor’s suite solutions, which shows a good pass rate. Installation is quick, but once again updates took rather a long time. The interface is similar in design to some of the product’s home-user siblings – very slick and glossy with the usual green colour scheme, and another complete set of controls readily available.

Stability was again flawless with nothing to report.

Scanning speeds were OK at first, with re-scans over in an instant. Overheads weren’t bad either, and again (perhaps more significantly), the re-runs proved barely affected by the protective layer. Our set of activities wasn’t slowed down much either, with low resource use.

Detection was a fraction higher than the Enterprise version, with good scores across the board, and again no issues in the certification sets with just a handful of ‘risktool’ warnings in our clean sets. Kaspersky Lab thus earns its second VB100 award this month.

Kaspersky Small Office Security 13.0.4.233

Main version: 13.0.4.233(b)

Update versions: N/A

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 3 passed, 0 failed, 9 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

This is the third product from the same family – we’ve seen this small-business version before, and it even has its own records in our test history with a few passes logged. Once again, the update process was slow after a fast installation. The interface will be familiar to those who’ve used the company’s home solutions, although the familiar green is replaced with a more sober grey, and once more the option selection is both detailed and well laid out.

Stability was not a problem for this version either, with every test taken comfortably in its stride.

Scanning speeds were decent, becoming super and overheads light, becoming lighter. There was a little impact on our set of activities, but resource use was not high at all.

Detection was again strong with just the faintest of declines through the RAP sets. The certification sets presented no surprises, so Kaspersky Lab manages a hat trick with a VB100 award for its SOS product too.

Microsoft System Center Endpoint Protection

Main version: 4.6.305.0

Update versions: 1.1.10904.0/ 1.183.428.0, 1.1.11005.0/ 1.185.3406.0, 1.1.11104.0/1.187.63.0, 1.1.11104.0/ 1.187.618.0

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 7 passed, 0 failed, 5 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Microsoft’s business offering has managed a pass in every VB100 test it has entered so far and has become a fixture in our comparatives over the last few years (although few will be surprised to know that it doesn’t appear in our annual Linux tests). Installation is quick and easy, the interface similarly simple and clear with a basic set of controls laid out nicely.

Stability was good, with the only incident of note a minor freeze of the GUI at one point.

Scanning speeds were not too bad, overheads a little high at first but better later on. Resource use was low, and impact on our set of activities reasonable.

Detection was reasonable too, with the proactive sets just a little lower than the reactive in the RAP test and no problems in the certification sets, thus earning Microsoft another VB100 award.

Norman Endpoint Protection Standalone

Main version: 11.00

Update versions: 7.04.04

Last 6 tests: 3 passed, 1 failed, 2 no entry

Last 12 tests: 6 passed, 5 failed, 1 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

There is a big change for Norman this month as its new version 11 adds the Bitdefender engine to its arsenal. Our test history for the vendor shows a reasonable pass rate, which could well improve from here on. Installation takes a few minutes but updates are very fast. The interface has had a bit of a redesign – once again relying on a browser, but looking nice with a good clear layout and a decent set of configuration options.

Stability was OK, with just a few of the script errors which seem so common with this type of GUI, and also a few instances of updates failing to complete first time.

Scanning speeds were not bad and overheads were OK too, with a reasonable impact on our set of tasks and very low CPU usage.

Detection was very strong, with no problems in the certification sets, and Norman’s new version earns a VB100 award without too much effort.

Panda Cloud Antivirus FREE

Main version: 6.81.11

Update versions: 7.00.00

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 8 passed, 2 failed, 2 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Now firmly established as a regular in our tests, with a full set of passes in the last year, Panda’s free cloud-based solution offers fairly speedy installation and a very minimalist interface dominated by clear and bold status updates. Configuration options are available, but only for the most basic requirements.

Stability was OK, with a few scans crashing out and a few minor error messages.

Scanning speeds were rather slow, overheads barely detectable with the default settings (which only scan certain file types on-read), and resource use was very low too, with not much of an effect on the runtime of our set of activities.

Detection was pretty decent in the reactive sets, with no score for the proactive sets thanks to the product’s reliance on the cloud; the certification sets were properly handled and another VB100 award goes to Panda.

Qihoo 360 Internet Security 5.0

Main version: 5.0.0.5061(x64)

Update versions: N/A

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 1 failed, 2 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Qihoo’s history in our tests is just hitting the six-year mark, and the last couple of years at least have seen a solid set of good performances with very few slips. Installation of the current version is very speedy indeed, with updates not too sluggish, and the product interface is bright and clear with nice big buttons.

Stability was mostly very good, with just a few minor errors, including an oddity with the reported update data.

Scanning speeds were very slow with no sign of getting better, and overheads look very light thanks to the delayed nature of the ‘real-time’ protection. Our set of activities showed only a minor slowdown though, and resource use was very low.

Detection from the combined in-house and Bitdefender engines was very strong in the reactive sets, decent in the proactive parts of the RAP test, and there were no problems in the certification sets, thus earning Qihoo a VB100 award.

Roboscan Enterprise Solution

Main version: 2.5.0.23

Update versions: 13.3.21.1/535818.2014082715/7.56549/6767724.20140827, 13.3.21.1/540668.2014101800/7.57280/6012434.20141017, 13.3.21.1/541141.2014102320/7.57350/5652003.20141023, 13.3.21.1/541534.2014103018/7.57433/5688017.20141030

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 8 passed, 1 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Alongside sibling product ESTsoft, Roboscan’s history in our tests is building steadily, with a good run of passes in the last couple of years. Installation this month was speedy, but initial updating took quite some time. The interface resembles that of ESTsoft, even down to the cute 'egg' character and the layout requiring a bit of thought to navigate but providing a reasonable set of controls.

Stability was a little shaky, with several scans crashing out and some problems with updating smoothly.

Scanning speeds were slow first time around but very quick on re-runs. Overheads started out high but became fairly light, while our set of activities was slowed down somewhat and resource use was very low.

Detection was very strong with just a slight dip into the proactive sets, and the certification sets were properly dealt with, earning a VB100 award for Roboscan too.

Tencent PC Manager

Main version: 8.10.25277.501

Update versions: N/A

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Tencent has put in a pretty solid couple of years in our comparatives, with passes in most tests since the first appearance of PC Manager. Installation is speedy and initial updates don’t add much to the set-up time. The interface looks crisp and clear with a professional feel – although with only Chinese available, we weren’t able to explore much beyond the guidance provided by the developers.

Stability was good with no problems across the full set of tests.

Scanning speeds were decidedly slow over archives and binaries, but fast elsewhere, with overheads reflecting the absence of on-read protection. Our set of tasks ran through in good time though, with decent resource use.

Detection from the combined in-house and Avira engines proved very solid indeed, and with similarly accurate handling of the certification sets, Tencent picks up another VB100 award.

Tencent PC Manager (TAV Version)

Main version: 8.9.25004.501

Update versions: N/A

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 2 passed, 0 failed, 10 no entry

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Tencent’s ‘TAV’ version is essentially the vendor’s standard product with the Avira component absent, relying only on the in-house engine – it managed a pass in its previous comparative entry. Installation was even faster this time, with updates over in seconds, and the interface is unchanged, again only in Chinese and looking professional.

Once again, there were no problems with stability, and our speed measures closely matched those recorded for the main product across the board.

Detection rates were considerably lower – there were some rather mediocre numbers in the reactive part of the RAP set with no proactive data thanks to heavy reliance on the cloud. The certification sets were well handled though, and a second VB100 award goes to Tencent for its TAV version.

Results tables

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the table)

On-demand throughput graph 1.

On-demand throughput graph 2.

(Click for a larger version of the table)

On-access lag graph 1.

On-access lag graph 2.

(Click for a larger version of the table)

Performance graph 1.

Performance graph 2.

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the chart)

(Click for a larger version of the chart)

Conclusions

After a fairly high pass rate in the last comparative, this month we have an unprecedented clean sweep, with all products submitted for testing achieving the required performance level for VB100 certification – the first time this has happened in a Windows comparative.

Along the way we’ve seen a number of wobbles and glitches, but stability has generally been good too, with no product rated lower than ‘Fair’. This may in part be the result of a happy conjunction of circumstances, with a nice, well-settled and stable platform that developers have had plenty of time to hone their products to play nicely with, and the more demanding server space in general – while home users might tolerate the occasional snafu, sysadmins won’t put up with shaky products unsettling their servers or producing unreliable data.

We saw a wide range of products though, several of which are clearly marketed more at the consumer than the business space, and most did well. As we are finalizing this report, we are already nearing the end of testing for the next comparative, on Windows 7, where there is a rather larger field of products, and where we expect much more variation in quality. We welcome the prospect of having our assumptions challenged though, and hope to see more of the generally high standard we’ve seen this month.

Technical details

Test environment. All tests were run on identical systems with AMD A6-3670K Quad Core 2.7GHz processors, 4GB DUAL DDR3 1600MHz RAM, dual 500GB and 1TB SATA hard drives and gigabit networking, running Microsoft Windows Server 2008 R2 SP1.

Any developers interested in submitting products for VB's comparative reviews, or anyone with any comments or suggestions on the test methodology, should contact john.hawes@virusbtn.com. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VB100 Certification Report

The August 2019 report details the VB100 certification of 37 anti‑malware products from 34 different vendors tested during July and August 2019.

VBWeb Comparative Review - Summer 2019

The VBWeb tests measure the performance of web security products against a wide range of live web threats, this time also including phishing threats.

VB100 certification report

The June 2019 VB100 report details the VB100 certification of 34 anti‑malware products from 31 different vendors tested during May and June 2019.

VBSpam comparative review - June 2019

In the June 2019 VBSpam test 12 full email security solutions and three blacklists of various kinds are put through their paces, resulting in 12 VBSpam certifications and six VBSpam+ awards.

VBWeb Comparative Review - Spring 2019

Most organizations use web security products to minimize the risk of malware making it onto the network - the VBWeb tests measure the performance of such web security products against a wide range of live web threats.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.