VB2017 preview: Stuck between a ROC and a hard place

Posted by   Martijn Grooten on   Aug 24, 2017

Authors of security software in general, and anti-virus software in particular, have always needed to find the right balance between a high detection rate and a low false positive rate – something that has become even more important with advances in machine-learning detection technologies. Making the model too strict will result in false positives, while making it too relaxed will result in missed malware – and in both cases unhappy customers.

To help guide the decision as to where to pitch that balance, Microsoft researchers Holly Stewart and Joe Blackbird came up with the interesting idea of using the wisdom of the crowd: they decided to use Microsoft's telemetry to look at what kind of errors (false positives or false negatives) had prompted end-users to switch anti-virus solutions.

In their paper, which Holly will present at VB2017 in Madrid, the researchers discuss what the data showed; they also looked at how different user types, or users in different geographic locations, may respond differently to errors made by their anti-virus solutions. It is an interesting approach that will likely be useful to other security software developers who have to make similar decisions when it comes to applying their machine-learning models.

stewart_vb2017_roccurve.png
Receiver Operating Characteristic (ROC) curve that plots the true positive (good detection) rate against the false positive (incorrect detection) rate. 

 

You can still register for VB2017 to hear more about this research and to learn from the work of more than 50 security researchers from all over the world!

VB2017-325w.jpg

twitter.png
fb.png
linkedin.png
googleplus.png
reddit.png

 

Latest posts:

Test your technical and mental limits in the VB2017 foosball tournament

As has become tradition, VB2017 will once again see a security industry table football tournament. Register your team now for some great fun and adrenaline-filled matches in between sessions in Madrid!

The case against running Windows XP is more subtle than we think it is

Greater Manchester Police is one of many organizations still running Windows XP on some of its systems. This is bad practice, but the case against running XP is far more subtle than we often pretend it is.

Hot FinSpy research completes VB2017 programme

Researchers from ESET have found a new way in which the FinSpy/FinFisher 'government spyware' can infect users, details of which they will present at VB2017 in Madrid.

Transparency is essential when monitoring your users' activities

Activity monitoring by security products in general, and HTTPS traffic inspection in particular, are sensitive issues in the security community. There is a time and a place for them, VB's Martijn Grooten argues, but only when they are done right.

VB2017 preview: Android reverse engineering tools: not the usual suspects

We preview the VB2017 paper by Fortinet researcher Axelle Apvrille, in which she looks at some less obvious tools for reverse engineering Android malware.