Aditya Sood and Richard Enbody discuss some of the different techniques that are used by present-day malware to circumvent protection mechanisms.
Copyright © 2012 Virus Bulletin
In this paper, we discuss some of the different techniques that are used by present-day malware to circumvent protection mechanisms.
With the advent of Windows x64 systems, the x86 emulator has been added to provide backward compatibility. WOWx64 is an x86 emulator that allows 32-bit Windows applications to run on 64-bit Windows. Malware writers use an x86 emulator detection routine to get detailed information about the environment in which the malware is going to be executed. This is a critical step from the attacker’s perspective because in order to trigger successful DLL injection, a 32-bit process has to load a 32-bit DLL, thereby avoiding collisions with 64-bit DLLs. Malware writers harness the power of inbuilt APIs to call ‘IsWOW64Process()’ to detect the x86 environment. This function is called in conjunction with ‘CreateEnvironmentBlock()’, which is present in userenv.dll, to retrieve environmental information for a specific user. The extracted information is passed to the ‘CreateProcessAsUser()’ function to create a process within the security context of the targeted user. Figure 1 shows a code snippet extracted from ICE bot.
This technique has been used widely by malware writers to detect the presence of virtual machines. The primary aim is to make analysis of the malware harder by shutting down some of its functionality if a virtual machine is detected. There are several techniques that can be used to detect the presence of a Virtual Machine Environment (VME), as follows:
Memory-specific techniques include Red Pill, which is a proof of concept that utilizes the Store Interrupt Descriptor Table (SIDT) to collect information about the Interrupt Descriptor Table Register (IDTR). The IDTR points directly to the Interrupt Descriptor Table (IDT) and, based on the memory address, Red Pill can detect the presence of a virtual machine. ScoopyNG  is another proof of concept that scrutinizes the location of the Local Descriptor Table (LDT), Global Descriptor Table (GDT), Interrupt Descriptor Table (IDT) and Store Task Register (STR) to determine the presence of a virtual machine. It also runs additional checks using VMware commands such as ‘get version’, ‘get memory size’ and ‘emulation check’. Any of these techniques can easily be deployed by malware to detect whether the code is inside a virtual machine. Figure 2 shows the output of ScoopyNG.
VMDetect  uses an invalid opcode mechanism that acts as a backdoor code to detect a virtual machine. It uses the privileged ‘IN’ (reading from communication ports) instruction to check if an exception occurs as ‘EXCEPTION_PRIV_INSTRUCTION’, and uses this information to verify whether the code is executing under VMware. However, these protections can easily be subverted by disabling all the protection flags in the VM configuration files, as shown in Figure 3.
Several samples of malware have been found using one of these memory-based techniques to design an anti-virtual-machine routine to subvert detection. (More details about virtual machine detection and analysis can be found at .)
Virtual machines make a number of adjustments in the Windows registry and create certain specific processes that can be utilized to detect the presence of a virtual machine environment. We have come across several registry-based settings that can be used to harness information about virtual machines. One of these is very critical as it is very hard for analysts to work around, as tampering with this key information could interfere with the booting state of the virtual machine. Figure 4 shows the VMware detection check based on SCSI/Disk info.
VMware can easily be detected based on the Media Access Control (MAC) address. This is not a widely used technique because it is not difficult to tweak the MAC address of a system. VMware can be detected in this way because the first 24 bits of the MAC address define the manufacturer of the machine. Generally, MAC addresses for VMware machines always start with ‘00-05-69-xx-xx-xx’, ‘00-0c-29-xx-xx-xx’ or ‘00-50-56-xx-xx-xx’. If the MAC address matches any of the 24 bits discussed above then it is a VMware machine. Figure 5 shows VMware detection using this method.
The Virtual Machine Communication Interface (VMCI)  is another target that can provide details about the running state of a virtual machine. VMCI provides an effective communication interface between the virtual machine and the host operating system. To detect whether code is running inside a virtual machine, malware writers can trace the installed VMCI device on the system. Simply, the malware can open a handle to the VMCI device(s) present on the system to verify the presence of a virtual machine. Table 1 presents the information that is required to query the VMCI interfaces on Linux and Windows operating systems.
|Operating system||VMware VMCI details|
|Linux||Host machine: /dev/vmmon • Guest machine: /dev/vmci|
|Windows||Host machine: \\.\vmx86 • Guest machine: \\.\VMCI|
Table 1. VMCI details of VMware.
Malware writers typically look for ‘\\.\VBoxGuest’ to determine if a virtual box is present on the system.
DLL injection has been around for several years and is used very effectively by malware writers. This technique is used to inject an unauthorized DLL into the target process at runtime to hook specific functions so that execution flow can be redirected. Until now, malware writers have explicitly used three standard techniques for performing DLL injection: ‘CreateRemoteThread’, ‘SetWindowsHook’ and ‘Appinit_dlls’. However, recently APC-based DLL injection has been seen in the wild. Both user- and kernel-mode Asynchronous Procedure Calls (APCs) ,  are used to build robust malware. All the APC-based routines require the _KAPC structure, which is called using the ‘nt!KeInitializeApc’ call. The details are shown in Figure 6.
The kernel-mode and user-mode functions executed through the APC procedure are termed kernel-mode and user-mode routines, respectively. APC-based DLL injection can be used by both user-land and kernel-land rootkits, as discussed in the following sections.
Malware writers define a custom APC function that is allowed to execute asynchronously in the context of the target thread, provided that the thread is in a waiting (alertable) state. User-mode rootkits use APC techniques extensively to inject unauthorized code into target processes. Generally, in every process the thread has its own APC queue. Rootkits queue a malicious APC for an alertable thread in the process. When the thread receives a queued APC, its waiting state is over and it processes the queued request, resulting in execution of the malicious APC procedure. Before executing the APC routine, a thread triggers one of the four waiting functions: KeWaitForSingleObject, KeWaitForMultipleObjects, KeWaitForMutexObject, or KeDelayExecutionThread. In user-mode APCs, the primary calling routine is defined in user mode so the APC procedure (implementation) has to switch back to ring 3 for successful execution.
Kernel-mode APC injection is categorized into two types:regular kernel-mode APC and special kernel-mode APC. In regular kernel-mode APC, the target kernel-mode routine is executed at passive interrupt request level (IRQL), whereas special kernel-mode APC triggers the target kernel-mode routine at APC IRQL. Both special and regular kernel-mode APCs are asynchronous events that have the ability to direct the flow of execution in threads from normal state to the target kernel routine by taking them out of their waiting states. The only difference is that regular kernel-mode APC is executed in more restricted conditions.
The complete details of kernel-mode and user-mode APC can be found in . ZeroAccess  (and see p.4) is an example of malware that has shown the usage of code execution through APC. Figure 7 shows a simple prototype of APC injection in action.
Many malware writers use mutex-based detection techniques to determine whether an operating system has any security programs installed on it. A mutex  is typically a mutual exclusion lock and is used to protect the different resources and data from being accessed concurrently. Malware writers define the mutex routine in the main entry point of the malware. The primary aim is to detect whether any other installed program is using that mutex. Generally, malware writers have knowledge of the mutexes (unique mutex names) that are used by different protection programs or anti-virus software that may be installed on the system. In Windows-based malware, the CreateMutex() API is used extensively to detect the presence of any type of mutex in the system. The entry routine defined in the malware code triggers this API to scrutinize whether the mutex is already present in the system. If the mutex exists, the API returns an error message – which shows that protection programs have already been installed on the running machine. Based on this information, the malware stops its execution and becomes dormant. Zeus, SpyEye, ICEX and several other bots use this technique.
Mutexes are also used in bot wars. Based on mutex information, one bot can kill another to increase its kingdom of infections. In this case, the OpenMutex() API is used to access the running mutex in the system. The kind of API used for collecting mutex information from the system depends on the malware writer’s choice. This functionality has been seen in earlier versions of SpyEye, which had an inbuilt Zeus-killing routine that used named pipes and designated commands to kill the Zeus bot in the system.
To detect the presence of security programs in the Windows operating system, malware writers use the de facto standard of runtime dynamic linking of system DLLs. This technique allows malware writers to design a generic routine that calls the LoadLibrary() API to dynamically load the target library into the address space of the calling process. The GetProcAddress() API is used afterwards to resolve the address of the loaded library in the system. The detection routine is very simple. Since the malware writers have information about the specific set of DLLs used in sandbox programs, anti-virus software and many others, if the required DLL is loaded through the LoadLibrary() API, it means the system is equipped with the protection software and the malware stops its execution and does not interact with the system. If the required DLL is not found in the system, then the malware starts the infection process. Figure 8 shows the idea behind this detection technique.
In the first part of this article, we have presented some of the tactics used by malware writers to design code that is resistant to the detection routines used by malware analysts. We will continue the discussion in part two of the article, in which we will look at advanced anti-debugging, polymorphism, tactical encryption routines, subverting client-side protection software, bypassing anti-virus solutions, etc.
 ScoopyNG – The VMware detection tool. http://www.trapkit.de/research/vmm/scoopyng/index.html.
 Thwarting Virtual Machine Detection. http://handlers.sans.org/tliston/ThwartingVMDetection_Liston_Skoudis.pdf.
 VMCI SDK. http://pubs.vmware.com/vmci-sdk/.
 Asynchronous Procedure Calls. http://msdn.microsoft.com/en-us/library/windows/desktop/ms681951%28v=vs.85%29.aspx.
 Almeida, A. Inside NT’s Asynchronous Procedure Call. http://www.ddj.com/windows/184416590.
 Windows Vista APC Internals. http://www.opening-windows.com/techart_windows_vista_apc_internals.htm.
 ZeroAccess Malware Part 3: The Device Driver Process Injection Rootkit. http://resources.infosecinstitute.com/zeroaccess-malware-part-3-the-device-driver-process-injection-rootkit/.