A denial-of-service attack on the Java bytecode verifier
This page gives information about how to perform a denial-of-service attack on Java’s bytecode verifier, if this verifier loads bytecode Java 1.0-1.5 bytecode. This attack is possible because the verifier’s type inference algorithm has quadratic complexity. When loading a class that resembles exactly the worst case this can lead to type inference times of multiple seconds to hours, depending on how large the loaded class file is.
Java 6 works around this issue by means of proof-carrying code. Every Java 6 compliant compiler must generate the so-called StackMapTable attribute, which is a clue (or a proof) for the verifier. Instead of inferring types, the verifier can just check this clue, which is much cheaper. IT people can therefore secure their Java 6 systems by installing a security manager that only allows Java 6 bytecode to be loaded, which can then be efficiently verified.
COMP 523 Report
You can download my project report for COMP 523 (language based security) here:
Efficient Java bytecode verification by the means of proof-carrying code
The report explains the exploit in detail.
Demonstration
Click on this link for a demonstration of the exploit.
Warning! If you click this link for the first time, this may freeze your browser for several seconds, up to minutes on slow machines!
The page loads an applet that dynamically loads a class with worst-case complexity. The class was compiled with a Java 1.5 compiler and hence lacks StackMapTable attributes.
Credits for this exploit go to Michael Franz and his colleagues, who first described such an exploit in their technical report.
Andreas Gal, Christian W. Probst, Michael Franz: A Denial of Service Attack on the Java Bytecode Verifier. Technical Report No. 03-23, University of California, Irvine, School of Information and Computer Science, November 2003.
