Thursday 25 September 16:00 - 16:30, Green room
Mohamed Nabeel & Alex Starov (Palo Alto Networks)
According to Gartner, more than 70% of organizations will have integrated AI models in their workflows by the end of 2025. In order to reduce cost and foster innovation, it is often the case that pretrained models are fetched from model hubs like Hugging Face or TensorFlow Hub. However, this introduces a security risk where attackers can inject malicious code into the models they upload to these hubs, leading to various kinds of attacks including remote code execution (RCE), sensitive data exfiltration, and system file modification when these models are loaded or executed (predict function). Since AI models play a critical role in the digital transformation, this would, unfortunately, drastically increase the number of software supply chain attacks.
While there are several efforts at detecting malware when deserializing Pickle-based saved models (hiding malware in model parameters), the risk of abusing DL APIs (e.g. TensorFlow APIs) is understudied. Specifically, we show how one can abuse hidden functionality of TensorFlow APIs, such as file read/write and network send/receive, along with their persistence APIs, to launch attacks.
It is concerning to note that existing scanners in model hubs like Hugging Face and TensorFlow Hub are unable to detect some of the stealthy abuse of such APIs. This is because scanning tools like Flicking and ModelScan face challenges in semantic-level analysis and in-depth taint analysis. We also show how one may identify potentially abusable hidden API capabilities using LLMs and build scanners to detect such abuses.
Mohamed Nabeel Mohamed Nabeel, Ph.D., is a principal security researcher at Palo Alto Networks where he investigates and develops solutions for open problems in web and DNS security using ML/AI/GenAI, providing advanced capabilities to protect internet users and AI security. He has authored and presented 20+ US patents and 25+ papers at top security conferences. He has presented his work in top industry conferences including RSA Conference and Virus Bulletin.
|
|
Alex Starov Alex Starov is a senior manager of the web security research team at Palo Alto Networks. His research focuses on proactive and data-driven web security and malicious URL detection and he manages several of the brightest researchers and engineers on protecting web users against sophisticated cyber attacks. He obtained his Ph.D. in computer science from Stony Brook University. He has published his work in top security venues as well as authored several patents. |
Back to VB2025 conference page