Alex Shipp MessageLabs
One recurring theme that malware writers have been using for many years is to use various different PE compression engines to disguise their creations. These engines typically take a Windows PE file and create a smaller file, which uncompresses itself in memory at runtime. Examples of such compression engines are UPX and ASPack, but the total list comprises many hundreds. Packing presents two problems for anti-malware engines. Firstly, to detect known malware, it is unworkable to create signatures for every different packer, and so the file must be unpacked first in order to get to the point where signature matching can occur. Secondly, to detect unknown malware, it is also usually necessary to unpack the file so that strong code analysis heuristics can be applied.
There are various different strategies which can be employed to unpack files to the point at which they can be analysed further. This paper looks at these various strategies, examining the strengths and weaknesses of each.
