Tony Lee Microsoft
Jimmy Kuo Microsoft
download slides (PDF)
We will focus on the following related subjects:
The anti-malware has long history of collaboration, from monthly to daily/top threat sharing, incidence response working groups, and now meta-data sharing. The evolution of this collaboration is driven by challenges from the threat landscape which demands threat visibility and efficient analysis, that motivates collaboration on data sharing. Its progress was marked by the IEEE ICSG industry working group works, and its common data exchange schema.
However, the effort has not seen the growth and adoption as expected both in number of participants and the level of sharing. The challenges can be broken down into several key areas,
We will closely examine these underlying challenges and propose a set of actions that industry can take to drive forward the data sharing initiative.
Telemetry data sharing also has a significant role in quality and meaningful industry testing.
Analysis and observation of a number of main industry tests reveals a pattern of test sets dominated by samples with low to zero threat prevalence in the field, which bears little user impact. This test practice incentivizes vendors to spend significant resources on producing less quality detections on malware of little ItW significance, at the same time, resulting in higher FP risks.
Some testers attempt to leverage telemetry data from vendors for sample selection and test score calculation that differentiate samples by prevalence, but all run into similar obstacles,
We will lay out a set of principles based on threat telemetry data that support meaningful test methodologies, as well as conduct case studies on test sets, compare and contrast different selection strategies, and evaluate impact with anonymous product results.
History has shown that industry testing is a collaborative effort by both testers and vendors. While testers leverage both samples and data from vendors they test on, test practice they employ incentivizes and motivates vendor practices. We will also propose industry guidelines that support and promote effective telemetry data sharing and its principle application in industry testing.
