Changes to the VB test sets

2006-03-01

Matt Ham

Virus Bulletin, UK
Editor: Helen Martin

Abstract

VB's product reviewer Matt Ham describes some changes to the test sets used for VB's comparative reviews.


Introduction

This month's reviewer activity has been concentrated behind the scenes, with some labour-intesive changes being made in the test sets used for VB's comparative reviews.

Clean test set

Over the years VB's clean test sets have consisted of a reasonably representative selection of files. Recently, however, there has been some concern over the inclusion of a large number of dynamically compressed files. In most cases these are installers, which contain multiple executables under potentially proprietary encryption or compression algorithms. While a small number of these would be expected in everyday on-demand scanning, the test sets contain a far larger percentage than one would expect to encounter in a real-world situation.

With the number of scanners that contain routines for delving into such files on the increase, it has become apparent that such in-depth investigation has a severe impact on speed of scanning. Therefore, the inclusion of a large number of installer files in the clean test sets was putting the more thorough products at a disadvantage in terms of the scanning speeds we reported.

To resolve the situation, the clean executables test set has been split into two. One set contains 'pure' executables, while the other contains these dynamically compressed executables. It is hoped that this will enable a better breakdown of any future scanning speed issues.

Spyware test set

The second major change comes in the form of a new test set – a spyware test set. Currently, there are no plans to make the detection of samples in this test set a requirement for achieving a VB 100% award.

While recognising that it is not the most stringent of test methodologies, the current plans are not to look at spyware detection capabilities on machines that are already compromised. Instead, it is envisaged that the files included in the test set will be the initial vector of the malware in question. Thus the downloaded file of a spyware application, Trojanised software with spyware functionality, backdoor servers and the like will make up the bulk of samples in the test set. As the compilation of both the spyware test set and testing methodology is still a work in progress, I would be very pleased to receive comments and suggestions at

twitter.png
fb.png
linkedin.png
googleplus.png
reddit.png

 

Latest articles:

EternalBlue: a prominent threat actor of 2017–2018

At the centre of last year's infamous WannaCry ransomware attack was an NSA exploit leaked by the Shadow Brokers hacker group, known as ‘EternalBlue’. The worm-like functionality of the exploit made a deadly impact by propagating to interconnected…

VB99 paper: Giving the EICAR test file some teeth

There are situations that warrant the use of live viruses. There are also situations where the use of live viruses is unwarranted. Specifically, live viruses should not be used when safer and equally effective methods can be used to obtain the…

Powering the distribution of Tesla stealer with PowerShell and VBA macros

Since their return more than four years ago, Office macros have been one of the most common ways to spread malware. In this paper, Aditya K Sood and Rohit Bansal analyse a campaign in which VBA macros are used to execute PowerShell code, which in…

VB2017 paper: Android reverse engineering tools: not the usual suspects

In the Android security field, all reverse engineers will probably have used some of the most well-known analysis tools such as apktool, smali, baksmali, dex2jar, etc. These tools are indeed must‑haves for Android application analysis. However, there…

VB2017 paper: Exploring the virtual worlds of advergaming

As adverts in gaming (‘advergaming’) ecosystems continue to become more sophisticated, so the potential complications grow for parents, children and gamers, who just want to play without having to worry about where their data is going (and how it is…


Bulletin Archive

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.