Let's translate firewall/endpoints for you: XAI on security products

Thursday 3 October 11:30 - 12:00, Red room

Tongbo Luo (JD.com)
Jimmy Su (JD.com)
Kailiang Ying (Syracuse University)
Xinyu Ma (Flappypig Team)
Zhaoyan Xu (Palo Alto Networks)

In the past year, we have witnessed a dramatic rise in the number of platforms and apps based on machine learning and artificial intelligence. AI technology has impacted from software and the Internet industry to other verticals such as healthcare, legal, manufacturing, automobile and agriculture. This trend has also applied to security industry, with all types of security products (e.g. malware detection engines, firewall IPS/IDS engines and endpoint sandboxes) beginning to leverage AI technology to escalate their power for better detection rates and quicker response speeds. However, at the same time, due to the increasingly complex model, especially in some deep neural network-based engines, it is extremely hard to understand the decision made by the products. Typical concerns include reactions such as "How on earth did the endpoint product quarantine my document?" from a frustrated customer, and complaint such as "How did the firewall miss such a native C&C traffic?" from a Dev-Ops team. As a result, an accountable explanation is becoming a necessary part along with the "binary" malicious-or-benign result.

In this talk, we want to ask questions of these AI-powered products: Why does this SVM-based scanner report certain binary files as malware? What caused the RNN-based firewall to reset the traffic to this IP address? With the emerging of XAI (eXplainable AI), we are able to peek inside the complicated machine learning "Black-Box". The core part of this talk is to demonstrate how to explain the result from AI-powered security products and what kind of answers we can get from them. A further investigation is "is the answer convincible and constructive?".

With concrete examples shown in the talk, we expect to make the audience aware of the importance of XAI to the current security industry, and to revisit existing products to check whether the current model is really effective or whether it happened to be a "happy accident".



Tongbo Luo

Tongbo Luo is Chief AI Security Scientist at JD.com and was most recently Principal Security Researcher at Palo Alto Networks. He obtained his M.S. and Ph.D. in computer science from Syracuse University in 2014. He actively researches docker security, cybersecurity, IoT security, and Adversarial AI for security problems. He has regularly shared his research with the security community at BlackHat USA, BlackHat EU, BlackHat Asia, VB and BSildes in the past few years.



Jimmy Su

Dr. Jimmy Su leads the JD Security Research Center in Silicon Valley.  JD Security Research Center focuses on eight areas: account security, APT detection, bot detection, data security, AI applications in security, Big Data applications in security, blockchain-based identify management, and IoT security.  Before joining JD, he was the Director of Advanced Threat Research at FireEye Labs. He led the research and development of multiple world-leading security products at FireEye, including network security, email security, mobile security, fraud detection, and end-point security.



Kailiang Ying

Kailiang is an independent researcher and technologist specializing in security. He received his Ph.D. degree from Syracuse University major in computer science in 2018. His research focuses on AI security, mobile security and Trusted Execution Environment. Most recently, Kailiang was a security researcher intern at JD.COM Silicon Valley Research Center in 2019.



Xinyu Ma

Xinyu Ma is a security researcher at the JD Silicon Valley Research Center. He is a man of the capture the flag competitions and a core member of the "Flappypig" and "R3kapig" team, where he leads his partners participating in the Def Con, HITB, NUIT Do Hack and Drgaon CTF competitions. Xinyu has also hosted the HITB CTF competitions in Amsterdam and Singapore. He is a researcher who focuses on web security and memory forensics, and also has a strong curiosity to learn novel techniques. Before joining JD he worked at Qihoo 360.



Zhaoyan Xu

Zhaoyan Xu is a research engineer at Palo Alto Networks, CA, United States. He joined Palo Alto Networks in 2014 and worked in the area of Internet security. He earned his Ph.D. degree from Texas A&M University, College Station in 2014. His research interests include web security, malware analysis, detection and system security.

Back to VB2019 Programme page

Other VB2019 papers

2,000 reactions to a malware attack - accidental study

Adam Haertle (BadCyber.com / ZaufanaTrzeciaStrona.pl)

The Bagsu banker case

Benoît Ancel (CSIS)

DNS on fire

Warren Mercer (Cisco Talos)
Paul Rascagneres (Cisco Talos)

We have placed cookies on your device in order to improve the functionality of this site, as outlined in our cookies policy. However, you may delete and block all cookies from this site and your use of the site will be unaffected. By continuing to browse this site, you are agreeing to Virus Bulletin's use of data as outlined in our privacy policy.