Stuck between a ROC and a hard place

Friday 6 October 11:00 - 11:30, Green room

Holly Stewart (Microsoft)

All protection products strive for perfection – we all want customers that are never infected and never experience incorrect detections (false positives). However, the era of combating malware with precise, static signatures is long gone. Anti-malware vendors must leverage next-gen methodologies, automation, and machine learning to combat the threats our customers face. Anyone who has studied machine learning knows that no model, no matter how good, will ever be a perfect reflection of the real world (it is, by definition, just a model). The trade-off between advance detection and false positives is inevitable. So, how do you make it? How do you strike the right balance for your customers?

This talk and paper will walk the audience through market data on hundreds of millions of Windows anti-malware customers and the impact of false positives and false negatives on market share. It will answer the following questions: If you have a malware miss, how likely are you to lose a customer? If you accidentally detect a clean application, are you more likely or less likely to lose a customer in comparison to a false negative? Does this vary by geography? For example, are customers in Spain more sensitive to false positives than, say, customers in China? What about customer type? Are gamers more sensitive to false positives than college students? The talk and paper will explain our methodology and insights from this in-depth, empirical research on the customer impact of false positives and false negatives.



Holly Stewart

Holly has worked in the security industry since 1997. She's held many types of roles, from technical writing in the early days, to product and program management, incident response, communications, and, for the past few years, data science. She started working for Microsoft in 2010. Currently, she works for the Windows Defender team where she manages researchers and data scientists focused on applying machine learning, automation, and other next generation capabilities to malware detection.


   Download slides






Other VB2017 papers

XAgent: APT28 cyber espionage on macOS

Tiberius Axinte (Bitdefender)

This paper provides an in-depth analysis of the macOS version of the APT28 component known as XAgent. We will dissect the…

Walking in your enemy's shadow: when fourth-party collection becomes attribution hell

Juan Andres Guerrero-Saade (Kaspersky Lab)
Costin Raiu (Kaspersky Lab)

Attribution is complicated under the best of circumstances. Sparse attributory indicators and the possibility of overt…

Mariachis and jackpotting: ATM malware from Latin America

Thiago Marques (Kaspersky Lab)

Fabio Assolini (Kaspersky Lab)

Of all the forms of attack against financial institutions in the world, the ones that are most likely to combine traditional…

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.