Banking sector confuses users over online safety

Posted by   Virus Bulletin on   Apr 22, 2009

Leftfield AV detection scores used to promote ID theft tool.

With online banking an increasingly important way of interacting with our banks, the security of online transactions is a huge issue for users. Even more important for banks is the perception of security, as users worried by scare stories about stolen passwords and plundered accounts are likely to retreat to older, safer, but for the banks much more costly methods of doing business: mail, phone and face-to-face interaction.

It seems obvious, therefore, that banks should have a vested interest in reassuring their users that standard security measures - such as desktop anti-malware software - provide a decent level of protection. Some, however, have begun promoting additional tools which may help ensure secure communications - such as Trusteer's Rapport anti-phishing system, which provides extra security by countering hijacked connections and communications.

To strengthen the case for its use, Trusteer has found itself some test results which appear to show extremely low detection capabilities in leading anti-malware products - which the company claims demonstrate the need for additional protection.

In the set of test results published by Trusteer, the two biggest names in anti-malware, Symantec and McAfee, tie with a paltry score of 23%, while the highest reported score, from Kaspersky, is just 44%. The data - which totally contradicts ratings measured by numerous independent and respected testing bodies - is presented without context or further information, and some banks, heedless of the possibility of terrifying their customers, have proceeded to republish the same figures.

The lack of message clarity from the banks is compounded by the source of the detection scores. The results are produced by a group called the 'Okie Island Trading Company', generally referred to by the much more official and serious-sounding acronym OITC. Among other activities, the group publishes details of anti-malware research on its website, and does at least provide details of the methodology used in its tests, claiming that samples come from trustworthy sources and are validated.

However, the methodology has several fairly obvious flaws, not least of which is a sample selection method based on excluding samples detected by more than 25% of products - assuming that any such samples cannot represent new threats. The main objection to the methodology, however, is the use of the VirusTotal multi-scanner system to produce the results. This simplistic method of determining scanner detection capabilities not only ignores the wealth of additional protections offered by modern security software, but also presents numerous issues regarding the accuracy of the results. Since each product vendor provides VirusTotal with its own bespoke set of settings, which may or may not represent the product's best possible scanning capability, the detection results produced by the system will not be an accurate reflection of the protection provided in real-world use, even when limited to simple static scanning.

None of these potential issues are raised in the test methodology, let alone in the writeups from Rapport and at least one bank has been seen quoting the figures as fact.

The test figures suffer from several other confidence-denting problems, including conflicting dates applied by various sites quoting identical figures, and the use of click-through advertising of selected security products on the OITC site itself.

"Anti-malware testing is a complex and difficult business, which presents a lot of pitfalls for amateurs," said John Hawes, Technical Consultant at Virus Bulletin. "We work hard to ensure Virus Bulletin's tests provide valid and useful reflections of the real world, and in conjunction with groups like AMTSO we are trying to define basic standards of testing which will improve the output of all kinds of testing bodies."

Hawes continued: "It's clearly equally important to encourage people who make use of test results to select their sources carefully, and when republishing them to present data fairly and clearly. For the struggling banking sector to restore its customers' trust, and for the global economy to evolve into an online future, people's perceptions of their security when online are vital. Presenting such unreliable data, without context or explanation, can only damage those perceptions and harm the banks' business. I really don't know what their thinking is here."

More comment on the OITC tests is provided by Sophos blogger and AMTSO board member Stuart Taylor here. The OITC results and methodology can be found on its website here, with more details on the specific test quoted by Trusteer here. Trusteer's use of the figures can be seen here, with the same data presented on a troubled UK bank's website here.

Posted on 22 April 2009 by Virus Bulletin



Latest posts:

The spam that is hardest to block is often the most damaging

We see a lot of spam in the VBSpam test lab, and we also see how well such emails are being blocked by email security products. Worryingly, it is often the emails with a malicious attachment or a phishing link that are most likely to be missed.

Throwback Thursday: We're all doomed

Mydoom turns 15 this month, and is still being seen in email attachments. This Throwback Thursday we look back to March 2004, when Gabor Szappanos tracked the rise of W32/Mydoom.

VB2019 call for papers - now open!

Have you analysed a new online threat? Do you know a new way to defend against such threats? Are you tasked with securing systems and fending off attacks? The call for papers for VB2019 is now open and we want to hear from you!

VB2018 paper: Unpacking the packed unpacker: reversing an Android anti-analysis library

Today, we publish a VB2018 paper by Google researcher Maddie Stone in which she looks at one of the most interesting anti-analysis native libraries in the Android ecosystem. We also release the recording of Maddie's presentation.

VB2018 paper: Draw me like one of your French APTs – expanding our descriptive palette for cyber threat actors

Today, we publish the VB2018 paper by Chronicle researcher Juan Andres Guerrero-Saade, who argues we should change the way we talk about APT actors.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.