VB100 comparative review on Windows Server 2012

2014-09-15

John Hawes

Virus Bulletin
Editor: Martijn Grooten

Abstract

The VB test team put 29 products through their paces on Windows Server 2012. John Hawes has the details.


Introduction

Our standard pattern of alternating between desktop and server platforms brings us round once again to Microsoft’s Windows Server 2012 – essentially the server version of Windows 8. With our reports still running rather behind schedule, it was something of a relief to find that participant numbers were not too high – dipping below 30 for the first time in a while, on a Windows platform at least. Many more entrants are expected for the next test, which will be on this platform’s desktop sibling.

Platform and test sets

The set-up process for Windows Server 2012 is slick and glossy, as one would expect. Given previous headaches with the touch-oriented desktop system, we were tempted to go for the GUI-free ‘server core’ option during install, but operation of products required the full desktop experience. As usual, we installed a few basic tools such as archive and document-handling utilities, and adjusted the test systems to hook in with our automated re-imaging set-up, before moving on to the sample sets.

The test sets were frozen on the test deadline, 23 April, with only RAP sets continuing to accrue after this date. Our false positive sets were given the usual tidy up, and a selection of new items were added, maintaining the set size at just under a million samples. The WildList set was synchronized with v4.004 of the WildList, released on 16 April, and other detection sets were compiled from samples first seen in the days running up to each round of testing.

No changes were made to the sample sets used for speed and performance measurements, but some minor tweaks were made to the automation processes to improve the granularity of the data recorded (which we hope to start making more use of soon).

In total, 29 products made the final cut, ready to be put through their paces in our suite of tests.

Results

Agnitum Outpost Security Suite Pro 9.1

Main version: 9.1

Update versions: 4646.690.1951

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 5 passed, 1 failed, 6 no entry

Agnitum seems to have settled nicely back into our tests, with a good run of success lately. Installation takes a good few minutes, but includes updating and a quick scan of critical areas. The angular, uncluttered interface fits in well with the standard desktop styling and looks good, providing decent configuration options with minimal confusion.

Operation was smooth for the most part, with just some minor wobbles in the update process.

Scanning speeds were reasonable and showed some good optimization in later runs, while overheads were a little heavy in some areas, again speeding up considerably in the warm measures. Resource use was rather high, and our activities test took quite some time to get through.

Detection was a little mediocre in the RAP sets, particularly in the proactive parts, but the WildList sets were fully covered, and with no problems in the clean sets Agnitum earns another VB100 award, maintaining a strong record in the past year.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Arcabit Internet Security

Main version: 2014.04.23

Update versions: 140423020725, 2014.05.13/140513065602, 2014.05.20/140520080118, 2014.05.28/140528080747

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 2 passed, 0 failed, 10 no entry

After a lengthy absence from our tests, Arcabit returned in the last comparative with a new look and a new engine under the covers, making a good impression all round. This month, installation was pretty simple and speedy, the interface pared down and angular to fit in with the latest fashion for Windows software.

Things mostly ran smoothly, but we did note a couple of runtime error messages when tweaking the settings.

Scanning speeds were OK, and pretty steady, with file access overheads pretty low too, only executables showing any great increase in time. Our set of tasks was a little slowed down, but resource use was fairly low.

Detection was very impressive throughout, setting a good standard for the usual bevvy of products based on the same Bitdefender engine to aim for, and with no issues in the certification tests a VB100 award is well deserved, setting Arcabit on track for a good run of success.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

avast! File Server Security

Main version: 8.0.1603

Update versions: 140423-1, 140512-0, 140519-0, 140527-0

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 9 passed, 2 failed, 1 no entry

There was a slight upset last time for Avast, but the vendor’s long history shows a heavy majority of passes over fails, with only three red marks since 2008. The current product continues to impress with its glossy good looks, the colours a little muted in this server edition, and it installed rapidly and reliably too. Options are provided in depth, and everything ran very solidly with no stability issues noted.

Scanning was pretty fast, and lag times not too high on simple file access, although our set of tasks did take a while to complete, and resource use was perhaps a little on the high side.

Detection was decent though, tailing off just a little into the later parts of the sets, and with nothing to report in the WildList or clean sets, a VB100 award is comfortably earned, returning Avast to the right path.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

AVG Cloud Care Antivirus 2014

Main version: 2014.0.4355

Update versions: 3882/7376, 2014.0.4570/3931/7480, 3950/7518, 7570

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 11 passed, 1 failed, 0 no entry

AVG hasn’t missed a comparative since 2010, and has only recorded a handful of fails in the past decade. This month we see something a little new, with the ‘Cloud Care’ in the title hinting at some changes. After a fairly lengthy installation and some rather zippier updates, the product presents an interface that looks pretty similar to other offerings from AVG over the last few years, once again mirroring the standard boxy styling of Windows 8 but favouring a dark and brooding colour scheme. The layout is fairly sensible though, with a good set of options within easy reach.

Stability was mostly decent, but a few update errors were noted, and during one of our more intensive jobs the system restarted unexpectedly, heavily denting AVG’s stability rating.

Scanning was very fast indeed, even in the initial runs, and overheads were pretty light, barely detectable once files had been checked for the first time. Our set of activities ran through in good time, and resource use was low.

Detection was very strong indeed with excellent scores throughout our sets, and with no issues to report in the WildList or clean sets, a VB100 award is well deserved, adding another good result to that strong score.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Avira Server Security

Main version: 14.0.3.350

Update versions: 8.03.18.06/ 7.11.144.172, 8.03.18.18/ 7.11.149.40, 8.03.18.22/ 7.11.150.94, 8.03.18.32/ 7.11.15.182

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

Another regular with a strong history, Avira’s current run of passes dates back to 2009, with only three tests skipped in that time. The server solution installs and updates very rapidly, and presents an interface based on MMC – which makes it a little fiddly in places, but with a little exploration it soon becomes fairly simple to locate any of the wealth of options provided.

Stability was mostly good, but on one occasion when an update interrupted a scan, the whole program locked up and had to be restarted forcibly.

Scanning speeds were decent, and very steady across multiple runs; lag times were fairly low with some good use of fingerprinting, and our set of tasks got through in good time with very low resource use.

Detection was very good indeed, only dropping away in the very latest part of the RAP sets, and with proper handling of the WildList and clean sets, Avira earns another VB100 award to add to its impressive tally.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Bitdefender Endpoint Security

Main version: 5.3.8.408

Update versions: 7.54309, 7.54681, 7.54813, 7.54977

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 12 passed, 0 failed, 0 no entry

Bitdefender’s record over the last few years has been stellar, with every one of the last 22 tests entered and passed. The ‘Endpoint’ product doesn’t take long to set up, and presents a slick and appealing interface with only a very limited set of controls available to the local user. There were no signs of any stability issues throughout testing.

Scanning speeds were pretty impressive from the off and sped up even further in the warm runs, while overheads were very light and, for a change, our set of activities got through in good time too, with pretty low resource usage.

Detection was as excellent as ever, with no problems in the certification sets and a VB100 award easily earned, maintaining that long streak of passes.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

BullGuard Antivirus

Main version: 14.0.279.6

Update versions: 7.54310, 7.54682, 14.0.279.9/ 7.54813, 14.0.279.9/ 7.54977

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 10 passed, 0 failed, 2 no entry

Bundling the Bitdefender engine alongside its own technology, BullGuard has also maintained an excellent record for over three years, only skipping our annual Linux comparatives. The current offering installs swiftly and smoothly, and has a stylish interface which requires a little familiarization to get the hang of its layout but provides a decent set of configuration options. Again, stability was impeccable throughout testing.

Scanning speed was pretty decent too, with some good optimization, and overheads were very low. Our set of tasks didn’t take long to complete, and use of CPU and memory was low.

Detection was as good as expected, with excellent scores everywhere, and the certification sets properly dealt with, earning BullGuard another VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Cyren Command Anti-Malware

Main version: 5.1.23

Update versions: 5.4.2/201404240859, 201405140725, 201405210904, 201405291239

Last 6 tests: 1 passed, 3 failed, 2 no entry

Last 12 tests: 2 passed, 7 failed, 3 no entry

Cyren, formed after the amalgamation of Commtouch and Frisk Software, hasn’t had the best of luck in our tests for a while, and has skipped the last few, but returns this time hoping to turn things around. The product is little changed, the installer very compact, as usual, and gets everything done very rapidly, with speedy updates too. The interface is fairly basic, but manages to provide a reasonable set of controls in a fairly usable manner.

Stability was a little shaky, with numerous scans crashing out even when not dealing with anything out of the ordinary.

Scanning speeds were reasonable, overheads distinctly high, and our set of activities took a fair while to complete, although resource use was low.

Detection was very strong in the reactive set with cloud access available, and not bad in the offline proactive sets either. The WildList sets were well covered, and most of the clean sets were fine too, but a single false alarm cropped up, with a generic detection spoiling Cyren’s hopes of a VB100 award this month by a whisker.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 1

Stability: Fair

Defenx Security Suite 8.1.1

Main version: 8.1.1

Update versions: 4319.687.1936

Last 6 tests: 3 passed, 0 failed, 3 no entry

Last 12 tests: 3 passed, 1 failed, 8 no entry

Sibling to Agnitum’s Outpost, Defenx also took a few months off testing last year while ownership of the underlying engine was being transferred to new hands, but has done well since its return. The product has had a few updates to its look and feel. It still takes a few minutes to get set up, thanks to the bundling of updates and quick checks as well as with firewall set-up into the process, and the interface looks crisp and clean, with a good set of controls within easy reach.

Stability was pretty good, with just a little wobbliness under very heavy pressure in the on-access bombardment; we also noted that the updater reports that it has updated on the current day, regardless of whether or not the update succeeded.

Scanning speeds weren’t the fastest initially, but sped up hugely in the warm runs; similarly, overheads were rather high to start off with, but improved quickly. Our set of tasks did take quite some time to complete, with fairly high resource consumption.

Detection was no more than reasonable, with decent levels achieved in the reactive sets but not much to boast about in the offline proactive parts. However, the WildList was fully covered, and with no false alarms in the clean sets, a VB100 award is earned.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Emsisoft Anti-Malware

Main version: 8.1.0.40

Update versions: N/A

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 7 passed, 3 failed, 2 no entry

Emsisoft is another product that includes the Bitdefender engine, and generally picks up some good scores. Its largely decent recent record was slightly dented by some stability issues in recent tests. This time, set-up was a little slow, but the interface remains bright and cheerful with friendly wording and slick, professional styling. A reasonable set of configuration options are available.

Once again, stability was shaky, with our on-access test freezing up repeatedly – on occasion locking up the entire system. Much coaxing was required to get it to complete the required tests, but these issues only showed up under heavy stress.

Scanning speeds were OK, and overheads very light indeed, thanks to defaulting to ignoring file read operations. Our set of activities showed a little slowdown, but not too much, and we recorded low resource use.

Detection was pretty strong, although perhaps not quite as high as we might have expected, and with a little gentle assistance the certification tests were passed successfully, earning Emsisoft a VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Buggy

eScan Internet Security Suite (server)

Main version: 14.0.1400.1602 DB

Update versions: N/A

Last 6 tests: 5 passed, 1 failed, 0 no entry

Last 12 tests: 11 passed, 1 failed, 0 no entry

The eScan product range is a very familiar sight on our test bench, having been a regular for more than a decade. It usually performs well, with just a single blip in the last two years. The set-up process is rather protracted, and updates aren’t the fastest either, with the task of getting it up and running taking more than 10 minutes on most occasions.

The interface tries to combine Windows 8 sharp corners with a few softer curves, and does a reasonable job, while the colour scheme has gradually been brightening, now adding some lime green to the original dark and murky greys. Configuration is excellent, with a full set of options sensibly laid out, and stability wasn’t bad either, with just a couple of scans locking up for a time.

Scanning speeds were pretty impressive, overheads a touch high perhaps, but our set of tasks wasn’t too badly hit and resource use was low.

Detection was very strong, helped along by that ever popular Bitdefender engine, and there were no issues to report in the certification sets, with another VB100 award comfortably earned by eScan.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

ESET Endpoint Antivirus

Main version: 5.0.2228.1

Update versions: 9711, 9711, 9820, 9855

Last 6 tests: 6 passed, 0 failed, 0 no entry

Last 12 tests: 12 passed, 0 failed, 0 no entry

ESET’s record in our comparatives is pretty much unchallengeable, with a pass in every test since May 2003 (when it skipped one). The product sets up fairly quickly, and the interface is simple and clean, with an excellent set of controls available but kept under the covers for those willing to dig into them. Stability was flawless, with no issues to report.

Scanning wasn’t super-quick initially, but sped up hugely after the first preparatory run, and overheads were very light indeed. Our set of tasks didn’t take too long to complete, and resource use was low.

Detection was very strong, only tailing off into the very last parts of the RAP sets. Once again, the certification sets presented no difficulty and a VB100 award is comfortably earned, keeping that huge chain of passes going.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

ESTsoft ALYac Enterprise 2.5.0.23

Main version: 2.5.0.23

Update versions: 13.3.21.1/528420.2014042215/7.54309/11657838.20140423, 13.3.21.1/529620.2014051318/7.54665/11750661.20140512, 13.3.21.1/529948.2014051918/7.54800/11796345.20140520, 13.3.21.1/530430.2014052815/7.54969/11835421.20140528

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 6 passed, 2 failed, 4 no entry

ESTsoft is a relative newcomer to our tests, but has strung together a good run of passes in the last few years. The product’s set-up was rapid, but updates took an age to complete, making for a lengthy overall install time. The interface is fairly busy, with a lot of components to cover, and is adorned with cutesy cartoonish touches, but it provides a decent level of fine-tuning options. Stability was a little shaky, with a number of scans crashing and some oddities with the interface freezing and even vanishing on occasion.

Scanning speeds were sluggish at first, but sped up nicely after that. Lag times were likewise heavy during initial runs but improved later. Our set of activities did take a little while to complete, but resource use was low.

With the Bitdefender engine underpinning things, detection scores were predictably excellent with high scores everywhere, and the core certification sets were handled well, earning ESTsoft another VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Fortinet FortiClient

Main version: 5.0.7.333

Update versions: 5.147/22.059, 5.0.9.347 5.147/22.189, 5.0.9.347 5.147/22.213, 5.0.9.347 5.152/22.239

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 9 passed, 1 failed, 2 no entry

Fortinet’s history in our tests dates back more than a decade. The latest product is fairly minimalist, the small installer completing rapidly but taking a long time to update, and the interface mainly presenting information, with few actual controls for the end-user. Once again, we had a number of stability issues, some of them quite severe: blue screens occurred on several occasions – mainly when handling our malware sets, but at least once when simply scanning the local system partition.

Scanning speeds were reasonable, a little slow over executables, with overheads rather high initially but improving in the warm runs. Our set of tasks wasn’t slowed down too much, and we noted reasonable CPU use but rather higher than average memory consumption.

Detection was very impressive indeed, with a very high score even for the later parts of the proactive sets, and there were no false alarms in the clean sets. In the WildList sets, though, a single item was ignored by the on-demand scanner in all three test runs, while the on-access component picked it up in all but the first run. Initial investigations have been unable to determine a reason for this, with the developers insisting that specific detection had been in place for quite some time – but with the problem cropping up repeatedly, no VB100 award can be granted this month, upsetting a pretty strong recent history for Fortinet.

ItW on demand: 99.95%

ItW on access: 99.98%

False positives: 0

Stability: Buggy

Ikarus anti.virus

Main version: 2.7.19

Update versions: 1.5.6/ 87333, 2.7.20/ 1.6.1 /87572, 87659, 87746

Last 6 tests: 3 passed, 1 failed, 2 no entry

Last 12 tests: 4 passed, 5 failed, 3 no entry

Ikarus first appeared in a VB comparative as long ago as 2001, but has only become a regular entrant within the last few years. The product has become familiar over time, with the installer requiring the .NET framework, which as always adds a fair bit of time to the process.

The interface is reasonably clear, with a fair set of controls available, and stability was very good, with good responsiveness throughout and no problems to report.

Scanning speeds were on the slow side to start with but very rapid in the warm runs, while overheads were similarly heavy, becoming lighter. Resource use was a touch high, but our set of tasks got through in good time.

Detection was very good indeed, with very little missed, and with no problems in the certification sets a VB100 award is comfortably earned.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

iSheriff Endpoint Antivirus

Main version: 5.0.9.0000

Update versions: 5.0.8/12.163, 5.0.9.0506/ 5.0.9, 5.0.9.0530/ 5.0.9.2, 5.0.9.0527/ 5.0.9.1

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

Formed from the division of Total Defense along business and consumer lines, iSheriff keeps the same business solution with a little light rebranding, and thus maintains the same strand in our test records. The set-up involves visiting a complex portal site and ferreting out the appropriate installer download, which runs through in good time. The control system is fairly complex too, with some components open to the end-user and others only accessible from the online portal, all of them displayed in the browser.

Stability was a little suspect, with a number of errors and alerts during installation, on-access scanning appearing not to work very well on numerous occasions, and logging frequently failing to produce usable or reliable data – which is considered a more serious issue than usual on a server platform.

Working around these problems, we recorded some pretty decent scanning speeds, average and fairly consistent overheads, and a reasonable time taken to complete our set of activities, with lowish resource consumption.

Detection was very strong indeed thanks to the Bitdefender engine included within, and after a little patient coaxing we did manage to get a full set of data for the certification sets – there were no problems here, and iSheriff duly earns a VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Kaspersky Anti-Virus 8.0 for Windows Servers Enterprise Edition Service Pack 1

Main version: 8.0.1.916

Update versions: N/A

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 10 passed, 1 failed, 1 no entry

Kaspersky’s VB100 test history is one of the most complete, with only three comparatives not entered in the entire history of the VB100 – and a very high proportion of passes have been achieved throughout that time. The current server product installs very quickly, with both the server and client components in place within a couple of minutes, but updating took rather a long time – most likely due to the central management module fetching data for a wider range of products than required for our testing. The interface uses the MMC system, but looks much slicker and more friendly than many others we’ve seen built on the same framework, with the wealth of controls presented sensibly.

Stability was almost perfect, with just a single fairly trivial error encountered which didn’t upset the operation of the product noticeably.

Scanning speeds were a little slow to start with, but blazing fast in the warm runs thanks to some good optimization. Overheads were perhaps a touch high initially, but again they sped up nicely, and were only really slow with the options turned up to the max. Our set of tasks took a little long to complete, but not excessively so, with RAM use around average but CPU use a little high.

Detection was pretty good, dropping away fairly noticeably in the proactive sets without access to cloud look-ups, but the WildList and clean sets were properly handled and Kaspersky earns another VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Kaspersky Endpoint Security 10 for Windows

Main version: 10.2.1.23

Update versions: 14.0.0.4651(f)

Last 6 tests: 2 passed, 0 failed, 4 no entry

Last 12 tests: 3 passed, 0 failed, 9 no entry

Kaspersky submitted a pair of products this month, with its ‘Endpoint’ solution up next. This one installed in reasonable time but again took an age to update – more than half an hour on at least one occasion. The interface has a typical Kaspersky look and feel, with the vendor’s trademark green tones and customary wealth of configuration options, all nicely laid out with the occasional fun bit of creativity in the design. Stability was impeccable, with no issues to report.

Scanning speeds weren’t bad at first and improved dramatically, in the warm runs. Overheads were perhaps a touch high in the first run but barely detectable after that, at least with the default settings. Our set of tasks ran through in good time with low resource use.

Detection was pretty good too, dropping off quite a bit in the proactive sets. There were no issues in the core sets, and Kaspersky earns a second VB100 award this month.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Kingsoft Antivirus

Main version: 2013.SP6.0.021400

Update versions: 2013.SP6.0.051211, 2013.SP6.0.052013, 2013.SP6.0.052716

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 8 passed, 0 failed, 4 no entry

Kingsoft’s revamp a couple of years ago, including the adoption of the popular Avira engine, has brought it a good run of passes ever since. The current solution installed in reasonable time, with the interface (which is only available in Chinese) looking clean and clear with indications that it includes a very wide selection of components beyond the basic anti-malware protection.

Stability was decent, with just a few instances of updates failing to complete properly first time.

Scanning was a little on the slow side, overheads very high initially but improving later on, while our set of tasks was a little slower than usual and showed fairly high use of resources.

Detection was very strong indeed, indicating the presence of some extras on top of the base engine, and it came as no surprise that the WildList was fully covered. In the clean sets, however, a single item was flagged as malicious, a driver from hardware firm Realtek, and although the developers had already spotted and fixed this before we informed them, it was enough to deny Kingsoft a VB100 award this month despite a pretty good showing generally.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 1

Stability: Stable

Maya PremiumIS Internet Security

Main version: 1.1.59.115

Update versions: 1.1.63.119

Last 6 tests: 0 passed, 2 failed, 4 no entry

Last 12 tests: 0 passed, 2 failed, 10 no entry

Maya’s offerings have been submitted for testing several times over the last year or so, but have only once before made it all the way to a final report. The set-up process takes an average amount of time and effort, and the product’s interface has a simple, boxy look that feels a little home-made but presents its controls clearly.

The available options are fairly basic, and stability was pretty dismal. We had repeated error messages during installs, although it seemed to complete happily in the end, and attempts at on-demand scans frequently produced further errors, freezes and crashes – mostly (but not only) when covering large malware sets.

Those scans which did run to completion did so in pretty decent time, although executables took a while; our lag times measure shows the absence of on-read protection. Our set of activities completed in reasonable time though, and while CPU use was fairly high, RAM use was very low.

Detection was almost respectable in the reactive sets, very low in the proactive part of the RAP test, and the WildList was barely covered at all, with a mismatch between on-demand and on-access scores reflecting further problems with the stability of the on-write protection. There were also a handful of false detections in our clean sets, but not too many, and there are some signs here that Maya may have some hope of reaching VB100 standard in the future.

ItW on demand: 34.4%

ItW on access: 9.2%

False positives: 10

Stability: Buggy

Microsoft System Center Endpoint Protection

Main version: 4.5.218.0

Update versions: 1.1.10502.0/ 1.173.35.0, 1.173.2187.0, 1.173.2460.0, 1.1.10600.0/ 1.175.703.0

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 6 passed, 0 failed, 6 no entry

Microsoft’s business offering has become the vendor’s default choice for VB100 testing lately, picking up a clean sweep of VB100 awards for the Windows comparatives in the last year, with less frequent participation before that. As one would expect, the installation process is very slick, and pretty fast too. The product interface is crisp and angular these days, with a fairly easy-to-navigate layout and a reasonable set of basic configuration options. Stability was fine apart from a few failures to update, which were quickly remedied by re running the task.

Scanning speeds were reasonable, on-access overheads not too bad initially and fading into the background in subsequent re-runs. Our set of tasks did take a fair while to complete, with CPU use not too high and RAM use very low.

Detection was decent in the reactive sets, a little below par in the proactive sets, but the core certification sets were handled impeccably, and Microsoft earns another VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

MSecure Malware Secure

Main version: 1.1.107.0

Update versions: 87333, 87572, 87659, 87746

Last 6 tests: 0 passed, 4 failed, 2 no entry

Last 12 tests: 1 passed, 5 failed, 6 no entry

MSecure’s products, based on the Ikarus engine, have been appearing in our tests fairly regularly over the last year or so, but have had a run of rather poor showings after initial success. The set-up process is very rapid indeed, and updates also fast. The product GUI has gone for the typical Windows 8 boxy look, but feels a little unbalanced, with minimal configuration options.

Stability was not good, with a few scans crashing out and one causing an unexpected reboot – which few server admins would be happy about. We also observed some severe problems with the on-access protection, which appears to ignore the extension list displayed (and editable) in the interface, covering only a very limited set of file types which do not include several of those most commonly used by malware (setting the option to ‘All files’ and rebooting rectifies this problem though).

Detection was excellent, with very good scores everywhere, and the WildList sets were covered flawlessly on demand, with no problems in the clean sets either. However, with the on-access component seriously unreliable, MSecure cannot be given a VB100 award.

ItW on demand: 100.00%

ItW on access: N/T

False positives: 0

Stability: Buggy

Norman Endpoint Protection

Main version: 9.10

Update versions: 7.03.02, 7.04.04

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 7 passed, 4 failed, 1 no entry

Norman is another vendor that has maintained very regular participation in the VB100 tests since the very beginning, with a preponderance of green (for good) in our long-term history showing a fairly strong pass rate over the years – a trend which has wobbled a little lately, but seems to be rebalancing itself towards success. Installation took a little while, but once up the interface looked clean and well laid out in its browser setting, with a simpler GUI available for manual scanning. Stability was unshakeable throughout testing.

Scanning speeds were decent, overheads a little heavy first time round, but soon speeding up very well. Our set of activities got through very rapidly indeed, with very low use of resources.

Detection was very good too, with pretty decent scores everywhere, and with no issues in the certification sets, a VB100 award is well deserved.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Panda Endpoint Protection

Main version: N/A

Update versions: N/A, 6.81.11/ 6.81.12

Last 6 tests: 3 passed, 2 failed, 1 no entry

Last 12 tests: 7 passed, 2 failed, 3 no entry

Since Panda’s return to our tests after a lengthy break, it has put in a good run of performances, with the only bad spots occasioned by rather unlucky FPs in a couple of tests.

The vendor’s business solution is not quite as fast to install as its consumer offerings, but doesn’t take too long, and like many these days offers a split in controls between a local console and a cloud-based management tool. Stability was only impacted by a couple of errors during scans of large malware sets.

Scanning speeds were a little on the slow side, with fairly heavy impact on file accesses and on our set of activities. Use of RAM was rather high throughout.

Detection was decent in the reactive sets, with no score in the proactive parts as the product cannot operate when the Internet connection is down. The WildList sets were handled perfectly, with no false alarms in the clean sets, and a VB100 award is easily earned by Panda.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Qihoo 360 Internet Security

Main version: 4.9.0.4133(x64)

Update versions: 4.9.0.4116(x64)

Last 6 tests: 4 passed, 1 failed, 1 no entry

Last 12 tests: 9 passed, 1 failed, 2 no entry

With a VB100 history dating back just shy of five years, Qihoo’s record is pretty strong. As usual, installation and initial updates were completed very rapidly, the whole process rarely taking more than two minutes, and the interface is clear and breezy with a decent basic set of controls. Stability was good throughout, with just a minor issue with one or two updates to report.

Scanning speeds started fairly slow and remained that way, with our lag time measures showing another product without full on-read protection in real time (some detections are noted and reported retrospectively after the file has been opened). Our set of tasks completed quickly, and with low resource use recorded.

Detection, helped along by the Bitdefender engine, was excellent throughout, and there were no problems in the core sets, earning Qihoo another VB100 award.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Quick Heal Server Edition

Main version: 15.00 (8.0.6.1)

Update versions: N/A

Last 6 tests: 5 passed, 1 failed, 0 no entry

Last 12 tests: 9 passed, 2 failed, 1 no entry

Quick Heal is another firm with well over a decade of VB100 comparatives under its belt, and our records show a good ratio of green (for good) to red (for danger). The current version of the product sets up fairly quickly, with updates not too sluggish either. Stability was good, although one large scan did crash out.

Scanning speeds weren’t bad, with some file types handled much more quickly on second and subsequent viewings. Our set of activities took a little extra time to complete, with high CPU use, but memory use was very low.

Detection was on the low side in the reactive sets, heading towards poor in the proactive parts, but there were no issues in the core sets and a VB100 award is earned.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Stable

Roboscan Enterprise Solution

Main version: 2.5.0.23

Update versions: 13.3.21.1/528420.2014042215/7.54309/11657838.20140423, 13.3.21.1/529620.2014051318/7.54665/11750661.20140512, 13.3.21.1/529948.2014051918/7.54800/11796345.20140520, 13.3.21.1/530430.2014052815/7.54969/11835421.20140528

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 6 passed, 2 failed, 4 no entry

Roboscan, sibling of ESTsoft, also includes the Bitdefender engine in its defensive line-up. It has a good run of passes in our tests now, with a clean sweep in the last year’s worth of Windows tests. Installation is rapid, but initial updates are very slow. The interface is indistinguishable from that of ALYac, with a lot going on, most of which consists of usable configuration options.

Stability was a little shaky, with a number of scan crashes and also some problems handling its own logging – but most of these issues occurred only when dealing with unusually large sets of detections.

Scanning wasn’t too slow from the off, and was blazing fast in the warm runs, with overheads starting off reasonable and also benefiting from some potent optimization. Our set of tasks took a fair bit of time to get through, but resource use was very low.

Detection was splendid, with nothing to complain about anywhere, including in the core sets, earning Roboscan another VB100 award fairly comfortably.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Fair

Tencent PC Manager

Main version: 8.9.25002.501

Update versions: N/A

Last 6 tests: 5 passed, 0 failed, 1 no entry

Last 12 tests: 9 passed, 0 failed, 3 no entry

Tencent has put in a strong first two years in our testing, with a pass in every test entered so far. The product is another with a very fast basic install, but a rather slow update process – the sluggish updating may be due to the distance of our test lab from its main user locations, though. The interface is available in Chinese only, and looks crisp and clear with a number of tabs for different modules, although it was hard to determine how deep the configuration options went. Stability was impeccable.

Scanning speeds were reasonable, maintaining very even pacing throughout, and once again our lag time measures reflect an absence of on-read protection by default. Our set of activities reveal a bit of an impact, but resource use remained low.

Detection, provided in part by Avira, was superb, only dropping off a little into the final days of the RAP sets, and the certification tests raised no issues, earning Tencent another VB100 award and maintaining its impressive run.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Tencent PC Manager (TAV version)

Main version: 8.10.25201.501

Update versions: N/A

Last 6 tests: 1 passed, 0 failed, 5 no entry

Last 12 tests: 1 passed, 0 failed, 11 no entry

Finally this month, a second product from Tencent, referred to as the ‘TAV’ version and using, as we understand it, only the company’s in-house detection capabilities, leveraging cloud look-ups to supplement local techniques. The set-up process was very rapid indeed, and the interface indistinguishable from that of the usual PC Manager product. Once again, stability was excellent.

Scanning speeds seemed fairly close to those of the main product, and again, on-read protection was absent, rendering the lag time score irrelevant, but our set of tasks wasn’t slowed down too much, and once again resource use was very low.

Detection scores were a little on the low side in the reactive sets, with no proactive numbers as the product relies on the cloud, but the WildList was properly covered and there were no issues in the clean sets either, earning Tencent another VB100 award this month, this one all on its own.

ItW on demand: 100.00%

ItW on access: 100.00%

False positives: 0

Stability: Solid

Results tables

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the table)

Performance graph.

(Click for a larger version of the table)

On-demand throughput graph 1.

On-demand throughput graph 2.

(Click for a larger version of the table)

File access lag time graph 1.

File access lag time graph 2.

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the table)

(Click for a larger version of the chart)

(Click for a larger version of the chart)

Conclusions

Another good set of passes this month, with most products putting in decent performances. Of the few that didn’t quite make it, most were scuppered by single issues which are likely to have been fairly momentary, but a couple had some more serious problems which will take a little work to eradicate. We also noted a few rather unstable products, with blue screens and reboots unexpectedly common this month.

We gathered a fair amount more information from our speed and performance tests this month, with much more granularity being recorded, and will be working on ways of making better use of this data going forward.

The publication of this test report and results comes rather later than scheduled, but much of the extra time taken has been given over to keeping the next comparative running smoothly – a considerably larger set of products were coming towards the end of testing on Windows 8.1 as the finishing touches were being put to this report. With the annual VB conference fast approaching, we hope to get those results out after the team returns from Seattle.

Technical details

Test environment. All tests were run on identical systems with AMD A6-3670K Quad Core 2.7GHz processors, 4GB DUAL DDR3 1600MHz RAM, dual 500GB and 1TB SATA hard drives and gigabit networking, running Microsoft Windows Server 2012 R2.

Any developers interested in submitting products for VB's comparative reviews, or anyone with any comments or suggestions on the test methodology, should contact john.hawes@virusbtn.com. The current schedule for the publication of VB comparative reviews can be found at http://www.virusbtn.com/vb100/about/schedule.xml.

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest reviews:

VB100 Certification Report - February 2019

Users are right to expect anti-malware products to satisfy a minimum standard of blocking malicious executables that have recently been seen in the wild, while blocking few to no legitimate programs. This report details the performance of 30…

VBWeb Comparative Review - Winter 2019

In the Winter 2019 VBWeb report we detail the performance of two web security products against live web threats and look at the current state of the web-based threat landscape.

VB100 Certification Report - December 2018

For more than two decades, Virus Bulletin has set a minimum standard for anti-virus (or anti-malware) products, checking whether products live up to expectation and providing those that do with the VB100 ‘stamp of approval’. This report details the…

VBSpam Comparative Review - December 2018

In this test – which forms part of Virus Bulletin’s continuously running security product test suite – 11 full email security solutions and eight blacklists of various kinds were assembled on the test bench to measure their performance against…

VBWeb Comparative Review - Autumn 2018

The Autumn 2018 VBWeb test measured the effectiveness of three web security products.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.