Stuck between a ROC and a hard place

Friday 6 October 11:00 - 11:30, Green room

Holly Stewart (Microsoft)

All protection products strive for perfection – we all want customers that are never infected and never experience incorrect detections (false positives). However, the era of combating malware with precise, static signatures is long gone. Anti-malware vendors must leverage next-gen methodologies, automation, and machine learning to combat the threats our customers face. Anyone who has studied machine learning knows that no model, no matter how good, will ever be a perfect reflection of the real world (it is, by definition, just a model). The trade-off between advance detection and false positives is inevitable. So, how do you make it? How do you strike the right balance for your customers?

This talk and paper will walk the audience through market data on hundreds of millions of Windows anti-malware customers and the impact of false positives and false negatives on market share. It will answer the following questions: If you have a malware miss, how likely are you to lose a customer? If you accidentally detect a clean application, are you more likely or less likely to lose a customer in comparison to a false negative? Does this vary by geography? For example, are customers in Spain more sensitive to false positives than, say, customers in China? What about customer type? Are gamers more sensitive to false positives than college students? The talk and paper will explain our methodology and insights from this in-depth, empirical research on the customer impact of false positives and false negatives.



Holly Stewart

Holly has worked in the security industry since 1997. She's held many types of roles, from technical writing in the early days, to product and program management, incident response, communications, and, for the past few years, data science. She started working for Microsoft in 2010. Currently, she works for the Windows Defender team where she manages researchers and data scientists focused on applying machine learning, automation, and other next generation capabilities to malware detection.


   Download slides






Other VB2017 papers

The state of cybersecurity in Africa: Kenya

Tyrus Kamau (Euclid Consultancy)

The cyber threats Kenya faces range from basic hacking such as website defacements, financial fraud, social media account…

Mariachis and jackpotting: ATM malware from Latin America

Thiago Marques (Kaspersky Lab)

Fabio Assolini (Kaspersky Lab)

Of all the forms of attack against financial institutions in the world, the ones that are most likely to combine traditional…

Keynote address: Inside Cloudbleed

John Graham-Cumming (Cloudflare)

In February 2017, Cloudflare was revealed to have been leaking private information including HTTP headers, cookies and POST data…

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.