One of the trickiest and hardest-to-understand memory problems in Java is the ClassLoader memory leak.
This type of memory leak is most likely to be found in Webapp servers, frameworks that use dependency injection, and applications that use scripting languages such as Groovy and Jython.
This article looks at how the JVM works with class loaders and how they’re handled by the garbage collector (GC). It also explores some of the reasons why class loaders may cause memory leaks, and includes some hints on troubleshooting the problem.
Class Loaders in Java
Class loaders are responsible for loading classes into memory. When an object is created, the class loader checks whether its class is already in memory. If not, it retrieves it from storage and loads it into the Metaspace area of memory.
The JVM has a hierarchy of four types of class loaders:
- The Bootstrap class loader: This loads classes that form part of the Java language;
- The Extension class loader: This loads extension classes that are stored in the ext directory;
- The Application class loader: This loads classes that are found in directories listed in the CLASSPATH environment variable;
- Custom class loaders: These are user-defined class loaders. Typically, they’re used for plugins, loading from non-standard locations, or where non-standard loading behavior is required.
There is a parent-child relationship between these loaders, and loaders at the bottom of the hierarchy will first pass loading requests up to the parent. Only if the parent is unable to find the class will the child loader process the request. This ensures that no non-standard Java classes override the core Java.
Custom class loaders add flexibility to an application, but if used incorrectly, they can cause a memory leak by creating a proliferation of classes. For more information on memory leaks, see this article: Common Memory Leaks in Java.
Use Cases for Custom Class Loaders
Why might we need custom class loaders? Let’s look at some typical uses.
- To allow us to modify existing byte code, or to create the byte code for a new class on the fly. We can do this using Java Reflection, and it’s often used to translate scripting languages such as Groovy to Java byte code.
- To choose classes dynamically at run time. One example is to be able to configure the driver to be used for a database connection, so an application can be configured to use any database.
- To allow different versions of software to be run.
- In WebApps, such as TomCat, they’re used to ensure each application is run in isolation, without the danger of mixing classes from different applications.
- To allow dependency injection.
- To allow new versions to be deployed without bringing down the JVM.
- To load plugins at run time.
Class Loaders and Garbage Collection
To effectively debug memory issues related to class loaders, we need to understand how GC deals with them. If you’re not familiar with how GC works, you may like to read this article: What is Garbage Collection?
Classes reside in the Metaspace, whereas objects reside in the heap. The Metaspace is only cleaned during a full GC event. A class can only be garbage collected when:
- No objects of that class have live references pointing to them AND
- The class loader that created them owns no other live classes AND
- The class loader has no live references pointing to it.
ClassLoader memory leaks can happen if any live object holds a reference to any variable belonging to an object created from one of the child classes, or to the class loader. Since the class loader can’t be garbage collected, none of the classes that belong to it can be garbage collected. To summarize:
- A live reference to any object of a class keeps the class alive, which
- Keeps the class loader alive, which
- Keeps all its classes alive.
It’s therefore really important when we’re using dynamic class loading to make sure references are released, either by ensuring they go out of scope or by explicitly setting them to null.
Common Causes of ClassLoader Memory Leaks
- Forgotten references to objects created from dynamic classes. This is especially dangerous if they are static references, since these will only be garbage collected when the class is released.
- Retaining pointers to the class loaders. An example of this is a web server, which uses a different class loader for each WebApp, and may retain pointers to them within a collection in order to reference the right class loader for each request. The sample program shown below simulates this behavior.
- ThreadLocal variables that reference dynamic classes, since these are retained for the duration of the thread unless they are explicitly released.
It’s possible that the leak may be in third-party libraries. If so, you may need to check the documentation for the library to see if there are configuration choices that may prevent the leak. Upgrading to the latest version may also be an option.
Not all ClassLoader memory issues are leaks. If the application really does need to create a very large number of classes, it may be necessary to increase the size of the Metaspace using the JVM argument -XX:MaxMetaspaceSize=<size>.
You may be interested in reading a case study that describes troubleshooting Metaspace issues in a microservice: Troubleshooting Microservices OutOfMemoryError in the Metaspace. The problem turned out to be a third-party library that created new classes for every request.
The sample program below simulates a server that retains pointers to class loaders in an ArrayList. This prevents the class loaders, and all their child classes, from being garbage collected. Since it loops indefinitely, it will eventually throw a Metaspace-related OutOfMemoryError.
import java.net.URLClassLoader;
import java.net.URL;
import java.util.ArrayList;
public class BuggyProg11 {
// Simulates Classloader leak
// ==========================
private static ArrayList list = new ArrayList();
public static void main(String[] args){
int i=1;
// Loops indefinitely
while(true)
{
loadClass();
try{Thread.sleep(50);} catch(Exception e){}
System.out.println("Num of objects "+i++);
}
}
private static void loadClass() {
// Creates a class loader, which loads a class
try {
ClassLoader loader = new URLClassLoader(new URL[] { new URL("file:.") });;
Class<?> myClass = loader.loadClass("Test11Class");
Test11Class obj = (Test11Class)myClass.newInstance();
list.add(loader);
}
catch (Exception e) {
System.out.println(e.toString());
}
}
}
// Class to be loaded
class Test11Class {
public Test11Class() {
}
}In the next section, we’ll look at how we might troubleshoot this program.
Troubleshooting ClassLoader Memory Leaks
The most obvious symptom of a ClassLoader leak is a program crash throwing the error java.lang.OutOfMemoryError: Metaspace. However, occasionally, the memory error may occur in other areas, such as the heap, if the affected classes hold static references to other objects. In fact, there are nine types of OutOfMemoryError that can occur in Java.
It’s also possible that the program may not actually crash, but may have performance issues, such as a spike in CPU usage or slow response times. If no maximum size is specified for the Metaspace, it could degrade the performance of other applications on the device.
Since the memory issues are likely to affect the Metaspace rather than the heap, the GC log will probably show an interesting pattern. The image below is taken from the GC log analyzer tool GCeasy. The red triangles indicate full GC events, and the graph shows heap usage over time.
Fig: GCeasy Graph Typical of Native Memory Leak
Since GC events are running back-to-back, even though the heap usage is low, it indicates that the problem is outside of the heap, in other words, it is in native memory. The next step in troubleshooting is to determine which part of native memory is affected. This article shows step by step how to do this: Understanding Native Memory Tracking in Java.
Following the steps described in the above article, the Metaspace tracking report from GCeasy may look like this, showing that the Metaspace is not being reclaimed.
Fig: GCeasy Native Memory tracking: Metaspace
Having established that the problem is in the Metaspace, we next need to see a list of classes that have been loaded. One way to do this is to set the JVM to verbose class loading by using the command line switch -verbose:class.
This can be tedious to monitor, and it’s easier to use the class histogram from a heap dump analyzer such as HeapHero or Eclipse MAT. HeapHero has the additional advantage that it prints useful statistics at the front of the report. Let’s run the sample program above, take a heap dump and submit it to HeapHero. The statistics at the front of the report show that there are a very large number of class loaders, as shown in the image below.
Fig: HeapHero Statistics Indicating a Large Number of Class Loaders
This immediately indicates that we have a class loader leak. We can then look at the class histogram:
Fig: Class Histogram Produced by HeapHero
For more information on how to use HeapHero to diagnose memory issues related to the Metaspace, this video demonstrates a worked example: Troubleshooting Metaspace Memory Issues.
In the case of the sample program, exploring the heap shows elements of the ArrayList holding references to the class loaders, preventing them from being garbage collected.
Fig: Exploring Objects in HeapHero
In the case of programs that use Java reflection, you may see a proliferation of classes with similar naming patterns.
Conclusion
ClassLoader memory leaks can be difficult to diagnose.
We need a good understanding of how class loaders work in Java, and how custom class loaders and their children are handled by the garbage collector.
With the help of tools such as GCeasy and HeapHero, we can speed up the troubleshooting process when dealing with this kind of error.