ICSA Labs report reveals testing details

Posted by   Virus Bulletin on   Nov 18, 2009

Certification issues unmasked in 20-year lab survey.

Renowned testing and certification group ICSA Labs has released a fascinating report drawing back the curtain on its security product tests - the detailed results of which are generally kept quiet as products go through the process of achieving certification.

Among the most widely noted facts in the report is the large number of products that require multiple cycles through the test programme before they manage to achieve certifiable standards.

As with many certification bodies, ICSA generally reports only the status of those products that have made the grade and been awarded certification, while failure to reach the required standards usually results in a private report to the vendor involved so that they can fix any problems with the product and have another go at the test.

According to the report, only 4% of all products tested achieve certification on the first attempt - although 27% of anti-malware solutions are found to be certifiable from the off, the average is heavily dented by the VPN and web application firewall categories, where no product has ever made the grade without receiving advice and assistance from the certifiers.

While these figures have been picked up by the sensational press as indicators of general inadequacy across the security industry, the report aims to highlight the tough and challenging nature of the certification process, and the efforts gone to by certification labs to work with vendors to improve their solutions, providing expertise and cross-industry cooperation.

The report also provides an overview of the history of ICSA and the security industry and offers advice to both product developers and potential buyers, with a lengthy checklist of criteria to consider when selecting a security solution. The report was created in collaboration with ICSA owner Verizon Business, and was issued as part of the inauguration of a new blog. The first entry of the blog, penned by contributing writer and ICSA MD George Japak, is available here. The report itself can be found here, with comments on the ESET blog here, and in The Register here.

Posted on 11 November 2009 by Virus Bulletin

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest posts:

VB2018 video: The Big Bang Theory by APT-C-23

Today, we release the video of the VB2018 presentation by Check Point researcher Aseel Kayal, who connected the various dots relating to campaigns by the APT-C-23 threat group.

VB2019 London - join us for the most international threat intelligence conference!

VB calls on organisations and individuals involved in threat intelligence from around the world to participate in next year's Virus Bulletin conference.

VB2018 paper: Tracking Mirai variants

Today, we publish the VB2018 paper by Qihoo 360 researchers Ya Liu and Hui Wang, on extracting data from variants of the Mirai botnet to classify and track variants.

VB2018 paper: Hide'n'Seek: an adaptive peer-to-peer IoT botnet

2018 has seen an increase in the variety of botnets living on the Internet of Things - such as Hide'N'Seek, which is notable for its use of peer-to-peer for command-and-control communication. Today, we publish the VB2018 paper by Bitdefender…

New paper: Botception: botnet distributes script with bot capabilities

In a new paper, Avast researchers Jan Sirmer and Adolf Streda look at how a spam campaign sent via the Necurs botnet was delivering the Flawed Ammyy RAT. As well as publishing the paper, we have also released the video of the reseachers' VB2018…

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.