The dawn of the ‘rogue AV testers’


Costin Raiu

Kaspersky Lab, Romania
Editor: Helen Martin


‘Some of the new testing labs that have appeared recently mimic the tactics of rogue AV products.’ Costin Raiu, Kaspersky Lab

Recently, I was sitting with some colleagues, discussing everybody’s favourite subject (OK, second favourite subject after the malware naming mess): the state of AV testing. During the discussion, somebody brought up the name of a new, obscure testing organization in the Far East. Nobody else had ever heard of it, so my colleague jokingly dubbed it a ‘rogue Andreas Marx’.

It then occurred to us that some of the new testing labs that have appeared recently mimic the tactics of rogue AV products. As we know, the rogue AV business model is based on selling a false sense of security; we professionals know it is fake, but the victims don’t. People buy a rogue AV product because they believe it will solve their security problems, but at best the products don’t do anything, and at worst, they install additional malware.

Rogue AV testers display similar behaviour. In this case, the business model is not based on a false sense of security, but instead on a false sense of insecurity. So, how do they operate? Well, it seems to start with a number of tests which look legitimate, and which mimic real-world conditions. The tests then slowly become more ‘complicated’, and security products perform increasingly poorly. Finally, the main idea emerges: that all security products are useless.

Hence, the false sense of insecurity is promoted through the tests: you are insecure, the money you paid for AV software was misspent. Rogue AV testers also often fail to disclose product names in published test results and attempt to sell their results for significant sums of money.

The following are some characteristics we identified as being specific to rogue AV testers:

  1. They are not affiliated with any serious testing organization, such as AMTSO. Rogue AV testers may also show fake affiliations or even falsely display, say, the AMTSO logo on their website.

  2. They publish free public reports, but charge for the ‘full’ reports. In general, the public reports are made to look as bad as possible for all participants, to maximize the profits from selling the full reports.

  3. The public reports are full of charts that look complicated and clever, but which sometimes reveal amusing mistakes. Although exact numbers are not usually available, the charts can provide useful information about the errors in the tests.

  4. They claim that all AV products are useless. This is the foundation stone of any business based on the ‘false sense of insecurity’.

  5. They charge (usually large sums of money) for samples and methodology to make sure the flawed methodology and samples cannot be reviewed externally. Reputable testers will make samples and methodology freely available to the developers of the products they test.

  6. Should a company or individual agree to pay the large sums to obtain the methodology, the fees escalate, revealing new, previously hidden costs. The main idea here is that the rogue AV testers do not want to provide access to samples and methodology, because it would reveal gross errors in their tests – by escalating their prices they hope that many will be deterred or prevented from accessing them.

There are other characteristics, but I think everybody gets the point.

Just as rogue AV products exploded and became one of the most profitable categories of crimeware, I suspect rogue AV testers will follow. In the process, they will also become extremely profitable and have a negative impact on the industry.

So, if you are trying to compare security solutions, I recommend sticking to established testing organizations such as Virus Bulletin, and AV-Comparatives or reputable magazines with a good history behind them.

Do not become a victim of the rogue AV testers!



Latest articles:

Cryptojacking on the fly: TeamTNT using NVIDIA drivers to mine cryptocurrency

TeamTNT is known for attacking insecure and vulnerable Kubernetes deployments in order to infiltrate organizations’ dedicated environments and transform them into attack launchpads. In this article Aditya Sood presents a new module introduced by…

Collector-stealer: a Russian origin credential and information extractor

Collector-stealer, a piece of malware of Russian origin, is heavily used on the Internet to exfiltrate sensitive data from end-user systems and store it in its C&C panels. In this article, researchers Aditya K Sood and Rohit Chaturvedi present a 360…

Fighting Fire with Fire

In 1989, Joe Wells encountered his first virus: Jerusalem. He disassembled the virus, and from that moment onward, was intrigued by the properties of these small pieces of self-replicating code. Joe Wells was an expert on computer viruses, was partly…

Run your malicious VBA macros anywhere!

Kurt Natvig wanted to understand whether it’s possible to recompile VBA macros to another language, which could then easily be ‘run’ on any gateway, thus revealing a sample’s true nature in a safe manner. In this article he explains how he recompiled…

Dissecting the design and vulnerabilities in AZORult C&C panels

Aditya K Sood looks at the command-and-control (C&C) design of the AZORult malware, discussing his team's findings related to the C&C design and some security issues they identified during the research.

Bulletin Archive

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.