VB2017 preview: Stuck between a ROC and a hard place

Posted by   Martijn Grooten on   Aug 24, 2017

Authors of security software in general, and anti-virus software in particular, have always needed to find the right balance between a high detection rate and a low false positive rate – something that has become even more important with advances in machine-learning detection technologies. Making the model too strict will result in false positives, while making it too relaxed will result in missed malware – and in both cases unhappy customers.

To help guide the decision as to where to pitch that balance, Microsoft researchers Holly Stewart and Joe Blackbird came up with the interesting idea of using the wisdom of the crowd: they decided to use Microsoft's telemetry to look at what kind of errors (false positives or false negatives) had prompted end-users to switch anti-virus solutions.

In their paper, which Holly will present at VB2017 in Madrid, the researchers discuss what the data showed; they also looked at how different user types, or users in different geographic locations, may respond differently to errors made by their anti-virus solutions. It is an interesting approach that will likely be useful to other security software developers who have to make similar decisions when it comes to applying their machine-learning models.

stewart_vb2017_roccurve.png
Receiver Operating Characteristic (ROC) curve that plots the true positive (good detection) rate against the false positive (incorrect detection) rate. 

 

You can still register for VB2017 to hear more about this research and to learn from the work of more than 50 security researchers from all over the world!

VB2017-325w.jpg

twitter.png
fb.png
linkedin.png
hackernews.png
reddit.png

 

Latest posts:

VB2019 paper: APT cases exploiting vulnerabilities in region-specific software

At VB2019, JPCERT/CC's Shusei Tomonaga and Tomoaki Tani presented a paper on attacks that exploit vulnerabilities in software used only in Japan, using malware that is unique to Japan. Today we publish both their paper and the recording of their…

New paper: Detection of vulnerabilities in web applications by validating parameter integrity and data flow graphs

In a follow-up to a paper presented at VB2019, Prismo Systems researchers Abhishek Singh and Ramesh Mani detail algorithms that can be used to detect SQL injection in stored procedures, persistent cross-site scripting (XSS), and server‑side request…

VB2020 programme announced

VB is pleased to reveal the details of an interesting and diverse programme for VB2020, the 30th Virus Bulletin International Conference.

VB2019 paper: Cyber espionage in the Middle East: unravelling OSX.WindTail

At VB2019 in London, Jamf's Patrick Wardle analysed the WindTail macOS malware used by the WindShift APT group, active in the Middle East. Today we publish both Patrick's paper and the recording of his presentation.

VB2019 paper: 2,000 reactions to a malware attack – accidental study

At VB2019 cybercrime journalist and researcher Adam Haertlé presented an analysis of almost 2000 unsolicited responses sent by victims of a malicious email campaign. Today we publish both his paper and the recording of his presentation.

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.