The three levels of exploit testing

Thursday 25 September 11:30 - 12:00, Red room.

Richard Ford Florida Institute of Technology
Marco Carvalho Florida Institute of Technology

Many different client-side security products claim to provide protection against exploits from known and unknown vulnerabilities. However, to date, tests of exploit detection solutions have been lacking, and those that have been conducted are of limited utility.

In this talk, we explore three different types of tests that could be used to measure the efficacy of products that claim to provide protection, and discuss the properties that these tests would actually measure. We argue that each test measures a different property of the protection provided. We further illustrate how existing tests can provide misleading information about the actual efficacy of measured products.

At the simplest level, exploit detection can be measured by testing machines using known-vulnerable versions of software against exploits taken from Metasploit. This test, however, fails to distinguish between products that detect known exploits targeting known vulnerabilities and products that simply detect the presence of known exploit code. At the next level, tests could be conducted using new exploits for known vulnerabilities. These could be created by building new exploits for well-documented vulnerabilities; however, this test does not discriminate between products that rely on prior knowledge of the exploit conditions and those which do not. Finally, we propose a new test that we believe raises the bar in exploit detection testing: the measurement of new exploits targeting new vulnerabilities. We provide a methodology that allows testers to ethically and efficiently leverage new vulnerabilities, and consider how an unscrupulous vendor might attempt to 'game' this new methodology. We demonstrate how these tests can be used in a cost-effective manner, and show how to further enhance this test with minimal effort, by adding techniques that allow testers to discriminate between post-exploitation behavioural detection and actual detection of the exploit.

Click here for more details about the conference.

Richard Ford

Richard Ford

Dr Richard Ford graduated from the University of Oxford in 1992 with a D.Phil. in quantum physics. Since that time, he has worked extensively in the area of computer security and malicious mobile code prevention. Previous projects include work on the Computer Virus Immune System at IBM Research, and development of the world's largest web hosting system whilst Director of Engineering for Verio. Ford is currently the Head of the Computer Sciences & Cybersecurity Department, Assistant Director of the Harris Institute for Assured Information, and the Harris Professor of Assured Information at the Florida Institute of Technology. His research interests include biologically inspired security solutions, rootkit detection, novel exploit paths, resilience, security metrics, and malware prevention. Ford is a consulting editor of Virus Bulletin, and co-editor of a column in IEEE Security & Privacy. Ford is a member of CARO, and the President/CEO of AMSTO, the Anti-Malware Testing Standards Organization. In addition to carrying out research in computer security, Ford is an instrument rated private pilot, and a three time winner of the National Flute Association's Big Band Jazz competition.