In its bi-monthly VBWeb tests, Virus Bulletin will test the ability of network-based solutions to prevent malicious websites from serving end-users the payload (typically a malicious file).
The first test started on Friday 4 December 2015. The results of this test can be read here. Hereafter, tests will be run every two months.
Products submitted to either kind of test will be tested under exactly the same conditions. The only differences are that the results for a privately tested product will not be published (either by Virus Bulletin or by the vendor) and the product will not be eligible for the VBWeb certification logo.
For reasons of transparency we would prefer to see vendors submit to public testing, so we charge lower fees for participating publicly.
Live malicious traffic is requested by VB’s test network and the responses are stored in a local cache. They are subsequently (and immediately) replayed to the various products. Assuming the original, unfiltered request resulted in a malicious payload being served, a product is considered to have blocked the request if the payload is not served after it filtered the traffic.
Our focus is on URLs (or ‘cases’; see below) that result in malware being downloaded, either through a direct download or via drive-by downloads. We will likely include other kinds of malicious traffic, e.g. scams and phishing, but these won’t count towards certification.
The certification requirements will be decided based on standards achieved in trials and initial tests, and will be detailed in each test report.
We will regularly visit legitimate, non-malicious sites to verify they aren’t being blocked. In the future, we may be able to add a more rigorous false positive test.
We are working on this at the moment. For now, the traffic will be plain HTTP only.
Yes, products will have full Internet connectivity throughout the test.
We are looking at ways to do this. For now, our focus is on products that run on the gateway as an explicit or transparent proxy.
In principle we intend to include such products, but there are some potential issues here. Please contact us to discuss how your particular product is set up.
We will provide PCAPs of the network traffic.
A URL is a text string, such as http://www.example.com/foo?id=bar
When a URL is entered into a browser, an HTTP request is made to the web server. The response sent by the web server usually results in the browser making further requests: images, CSS libraries, etc. A 'case' is anything that happens as a consequence of the URL being entered, that is the total collection of requests and responses that are generated.
Each case starts with a URL being entered into the browser, but the responses usually don't just depend on this URL. A different browser, a different IP address, or a different time can also result in different responses and this is especially an issue with malicious websites. For that reason, we don't refer to 'bad URLs' but instead to 'bad cases’. The only way to find out whether a case is malicious is by actually making the request.
So one initial URL results in many URLs being requested - the total collection of these requests and responses are the resulting case. A case can be 'good' or 'bad' (or something in between).