Jevgeni has posted 19 posts at DZone. You can read more from them at their website. View Full User Profile

ClassLoaderLocal: How to Avoid ClassLoader Leaks on Application Redeploy

06.19.2009
| 11675 views |
  • submit to reddit

"OutOfMemoryError: PermGen" is a very common message to see after a few redeploys. The reason why it's so common is that it's amazingly easy to leak a class loader. It's enough to hold a single outside reference to an object instantiated from a class loaded by the said class loader to prevent that class loader from being GC-d.

In this post I'll review how we solved this problem in JavaRebel, and share the solution with you. It's not a magical solution, but it will help alleviate some of the problems introduced in both libraries and applications in Java EE.

The most common way to leak is to register some kind of a callback object and never deregister it. E.g. look at the following code:

Core.addListener(new MyListener());
If Core is a part of the framework/platform/container then it will hold to MyListener long after the application was redeployed and the class loader left hanging.

 

Let's see if we can do anything to solve this. The Core implementation looks something like this:

public class Core {
List listeners = new ArrayList();

void addListener(Listener l) {
listeners.add(l);
}

void fireListeners() {
// Exercise for the reader!
}
}

The problem is that listeners provides a strong reference to the Listener object. What if we replace it by a weak one?

public class Core {
List listeners = new ArrayList();

void addListener(Listener l) {
listeners.add(new WeakReference(l));
}

void fireListeners() {
// Exercise for the reader!
}
}

Unfortunately although this does solve the problem of GC-ing the class loader, it doesn't really work. The Listener behind the weak reference will be GC-d at first opportunity and after that it'll no longer receive any callbacks. To illustrate why it's a problem the code above is basically equivalent to throwing the reference away altogether:

public class Core {
List listeners = new ArrayList();

void addListener(Listener l) {
// Listener is ignored and GC-d
}
}

Replacing weak reference with a soft one doesn't improve the situation, just delays the inevitable a bit further. Both are useful for caches, where objects can be recreated at will, but not in this case where we have an externally created object.

So what do we do? What we'd like to do is have the Listener reference to depend on the class loader somehow. Unfortunately, to the best of my knowledge, there isn't a ready-made solution for that, and there's no way to achieve it with any combinations of weak references without causing problems.

What we'd like to have is an ability to add a strong reference to the class loader: in other words have it carry a custom property:

void addListener(Listener l) {
ClassLoader cl = l.getClass().getClassLoader();
List lls = (List) cl.getProperty("CoreListeners");
if (lls == null) {
lls = new ArrayList();
cl.putProperty("CoreListeners", lls);
}
lls.add(l);
}

That would work, wouldn't it? Well, not quite. We also need to save a reference to the class loaders, so that we could later go over all of them. Here the WeakHashMap is useful:

Map classLoaders = new WeakHashMap();

void addListener(Listener l) {
//...
classLoaders.put(cl, Boolean.TRUE);
}

There's not WeakHashSet in Java, so we're just using a boolean flag as the value.

So this would probably work, but unfortunately class loaders don't have a getProperty()/putProperty() API. However, it turns out that with a bit of a hack we can simulate it, by generating a unique class per class loader to hold the properties for us. Let's see how it's done!

We start with a little boilerplate:

class ClassLoaderLocalMap {
private static Method defineMethod;
private static Method findLoadedClass;

static {
try {
defineMethod = ClassLoader.class.getDeclaredMethod(
"defineClass",
new Class[] {
String.class,
byte[].class,
int.class,
int.class });
defineMethod.setAccessible(true);

findLoadedClass =
ClassLoader.class.getDeclaredMethod(
"findLoadedClass",
new Class[] { String.class});
findLoadedClass.setAccessible(true);
}
catch (NoSuchMethodException e) {
throw new RuntimeException(e);
}
}
}

This will give us access to ClassLoader protected methods defineClass() and findLoadedClass() later on. Now let's setup the basic API:

public static void put(
ClassLoader cl,
Object key,
Object value) {
// Synchronizing over ClassLoader is safest
synchronized (cl) {
getLocalMap(cl).put(key, value);
}
}

public static Object get(
ClassLoader cl,
Object key) {
// Synchronizing over ClassLoader is safest
synchronized (cl) {
return getLocalMap(cl).get(key);
}
}

getLocalMap() method should return a map of entries associated with the class loader. How should that work?

Next we introduce a map from class loaders to unique holder class names. We also introduce a nextHolderName() method that generates unique names:

private static final Map classLoaderToHolderClassName = 
Collections.synchronizedMap(new WeakHashMap())
private static int counter = 1;

private static synchronized String nextHolderName() {
return "ClassLoaderLocalMapHolder$$GEN$$" + counter++;
}

Finally we can implement the getLocalMap() method (to save space I removed all exception handling):

private static Map getLocalMap(ClassLoader cl) {
String holderClassName =
(String) classLoaderToHolderClassName.get(cl);
if (holderClassName == null) {
holderClassName= nextHolderName();
classLoaderToHolderClassName.put(
cl, holderClassName);
}

Class holderClass =
(Class) findLoadedClass.invoke(
cl,
new Object[] {propertiesClassName});

if (holderClass == null) {
byte[] classBytes =
buildHolderByteCode(holderClassName);

holderClass = (Class) defineMethod.invoke(cl,
new Object[] {
holderClassName,
classBytes,
new Integer(0),
new Integer(classBytes.length)});
}

return (Map) holderClass
.getDeclaredField("localMap").get(null);
}

The last method to implement is buildHolderByteCode. It's quite trivial and builds the following class renamed to the unique name:

public class ClassLoaderLocalMapHolder$$GEN$$X {
public static final Map localMap = new HashMap();
}

The code can be derived using ASMifier with just a little customization, you can look it up in the full source code.

Although we could now easily implement the original example it makes sense to do just a little bit extra effort and introduce a ClassLoaderLocal, with behavior similar to the ThreadLocal:

public class ClassLoaderLocal {
private Object key = new Object();

public Object get(ClassLoader cl) {
if (!ClassLoaderProperties.containsKey(cl, key))
return null;
return ClassLoaderProperties.get(cl, key);
}

public void set(ClassLoader cl, Object value) {
ClassLoaderProperties.put(cl, key, value);
}
}

So the original example now becomes:

Map classLoaders = new WeakHashMap();
ClassLoaderLocal cll = new ClassLoaderLocal();

void addListener(Listener l) {
ClassLoader cl = l.getClass().getClassLoader();
List lls = (List) cll.get(cl);
if (lls == null) {
lls = new ArrayList();
cll.set(cl, lls);
}
lls.add(l);

classLoaders.put(cl, Boolean.TRUE);
}

In this code if any listener comes from a freed class loader, then it will be GC-d from both Core.classLoaders and ClassLoaderProperties.classLoaderToHolderClassName, as both are WeakHashMaps and there are no strong references to the class loaders. The generated ClassLoaderLocalMapHolder$$GEN$$X class will also be GC-d along with the class loader, so we have effectively eliminated a class loader leak without explicit cleanup calls from the user.

I hope this code will be useful for someone. I cannot give any guarantees whether it will work or not and it's clearly a hack (though a solid hack). Please use it if you actually understand what is happening. If you see a bug in the code or have a good suggestion, please be sure to comment. There could be a free JavaRebel license in it for you :) Full source code: ClassLoaderLocalMap.java, ClassLoaderLocal.java.

See original post and discussion at dow.ngra.de.

Published at DZone with permission of its author, Jevgeni Kabanov.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Konstantin Scheglov replied on Fri, 2009/06/19 - 5:19am

I don't understand why not use just WeakHashMap<ClassLoader, List<Listener>>.

Jevgeni Kabanov replied on Fri, 2009/06/19 - 6:36am in response to: Konstantin Scheglov

Because Listener instance holds a strong reference to its ClassLoader thus it will never be collected. WeakHashMap works _only_ if the values do not strongly reference the keys.

Josef Sin replied on Fri, 2009/06/19 - 7:25am

I hope that Apache Tomcat developers are reading java.dzone :-)

Thomas Mueller replied on Fri, 2009/06/19 - 8:49am

Tomcat and Glassfish 3 set most static fields (final or non-final) to null when unloading a web application (using reflection, setAccessible(true)). This can cause a NullPointerException. In Tomcat >= 6.0 this behavior can be disabled by setting the system property org.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES to false, however Tomcat may then run out of memory.

Andrew Perepelytsya replied on Fri, 2009/06/19 - 9:29am

@Thomas, Tomcat 6.0.x still suffers from this problem, apparently what you described above is only a partial fix.

@Eugene, great article, now only need time to wrap my mind around it ;) I faced (and solved) a similar problem in Mule 3.x, albeit differently. In my case I had a registry lifecycle to bind to, so I resorted to a simpler (for my use-case at least) solution of storing a hard-ref to the object and disposing it on redeploy, just before app shutdown, thus giving way to GC. I don't think it is applicable to every app though, and given your JavaRebel background you can expect reload happening at any point in time, not necessarily bound to app lifecycle. But at least I got away without bytecode instrumentation ;)

Andrew

Jevgeni Kabanov replied on Fri, 2009/06/19 - 9:34am

@Andrew Actually if you check out discussion at dow.ngra.de, there's a way to do the same without doing any bytecode instrumentation whatsoever.

WildDonkey replied on Wed, 2009/06/24 - 4:36pm

Isn't the simplest thing to somehow ensure that the construction of MyListener registers itself somewhere in its original classloader (for example simply by adding itself to a static Listeners list field of the MyListener, or maybe another class expressly for that purpose).

That way you can then just pass the weak reference to the Core.addListener() and it wont immediately be garbage collected I guess because the original classloader still has a reference to it through the static field.

 

john green green replied on Mon, 2009/10/26 - 3:27am

Isn't the simplest thing to somehow ensure that the construction nike shoes russiaof MyListener registers itself somewhere in its original classloader (for example simply by adding itself to a static Listeners list field of the MyListener, or maybe another class expressly for that purpose).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.