Exploit's new technology trick dodges memory protection
A hacker who goes by the name "JDuck" has discovered the first malicious PDF files which use the relatively new Return Oriented Programming (ROP) technology to bypass Data Execution Prevention (DEP). This means that the days for providing reliable protection via DEP are numbered even before this technology has become a mainstream feature.
Initially, JDuck only intended to integrate the PDF exploit into his metasploit vulnerability testing platform. When doing so, he noticed that the exploit worked flawlessly against Adobe Reader 9.3 although DEP is enabled by default in this version. Further examination revealed that the exploit contained a list of memory addresses that each pointed at the tail end of a function – that is, at a few machine code instructions followed by a return command. This characterises a rather cunning new exploit technology which has so far not been observed in the wild.
Exploits typically involve injecting arbitrary code as payloads and manipulating the memory in such a way that the program accesses the injected code. As a protective measure, modern systems set the access rights for data storage areas so that the areas can be read but not executed; in Windows, this is called Data Execution Prevention (DEP).
The classic way of bypassing DEP is called return-to-libc. Rather than requiring dedicated code, it uses already loaded system functions to trigger actions such as a program start. This only requires the stack to be specially crafted in such a way that, for instance, a return call in the program jumps to the exec function, and that the exec function then finds the parameters on the stack that make it execute a program such as notepad.exe.
In 2007, Hovav Shacham was among those who presented a generalised version of this concept which involved assembling the exploit from code fragments that already existed in memory and were each immediately followed by a return command. In a paper that is worth reading, the developer explains that this programming technique is Turing complete in contexts such as the Libc – which means that it allows arbitrary programs to be executed. In principle, the required code fragments can even be gathered via a special ROP compiler. In 2009, researchers demonstrated that even voting machines with a consistent Harvard architecture can be hacked; at this year's RSA conference, Dino Dai Zovi gave a presentation on "Practical Return Oriented Programming".
Until now, it was safe to assume that the majority of malicious files trying to exploit program vulnerabilities to install malware are blocked by the DEP mechanism. That this new exploit technology has appeared in malware files doesn't bode well. There is a chance that DEP could become ineffective even before it has become a mainstream technology. DEP is currently only active by default in 64-bit systems, which introduced the "no execute" bit for Intel architectures; on the 32-bit systems still in widespread use, every program is allowed to determine whether to run with or without DEP. This also includes 32-bit versions of Windows on 64-bit hardware.
- Understanding DEP as a mitigation technology part 1, Microsoft SRD blog post.