OutOfMemoryErrors in Java are a common cause of system downtime. Although many of them are simple to diagnose and fix, they disrupt production, and obscure errors can take hours or days to troubleshoot.
Part of the problem is that many troubleshooting guides concentrate solely on the Java heap space. Although the issue frequently relates to the heap, this is not always the case. It helps to have a broader understanding of the JVM memory model, and how different types of OutOfMemoryError correspond to it.
Why Do OutOfMemory Errors Occur?
Primarily, OutOfMemory errors occur because:
- The device or container has insufficient memory;
- The JVM is incorrectly configured to match the application’s needs;
- The application has a memory leak;
- The application is wasting memory.
For more information on memory leaks, see this article: Common Memory Leaks and How to Fix Them.
What Information Does the JVM Provide When Throwing OutOfMemoryErrors?
Let’s look at the output from a program that runs out of memory.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.base/java.lang.Long.valueOf(Long.java:1211)
at BuggyProg9a.<init>(BuggyProg9a.java:27)
at BuggyProg9a.main(BuggyProg9a.java:9)The first line tells us two things:
- The name of the thread where the error occurred, in this case, main;
- The type of error, in this case ‘Java Heap Space’. We’ll look at this in more detail later in the article.
The subsequent lines are the stack trace, starting with the point where the error occurred, and working backwards through the code path followed to arrive at this point. Each line includes the name of the file that contained the original source code, and the line number within that file. This makes it easy to trace where the error occurred, and how the application arrived at that point.
The JVM Memory Model
The JVM runtime memory is divided into several different spaces, each with a specific purpose. Although the Java heap space is an important memory area, it’s not the only part of memory that can cause an OutOfMemory error. To successfully troubleshoot Java memory issues, we need to understand the JVM memory model. We can visualize it like this:
Fig: The JVM Memory Model
The two main areas are heap memory, which is managed by the JVM, and native memory, which is managed by the operating system.
The heap is a central storage area, shared by all classes that make up the program. Heap memory is subdivided into the Young Generation and the Old Generation. This speeds up the work of the garbage collector (GC) by keeping newly created items in a smaller area, and items that have survived several cycles of GC in the larger Old Generation.
Native memory is split as follows:
- Metaspace (or PermGen in older JVM versions) holds class metadata;
- Thread Space holds the stack space for each running thread;
- Code Cache stores pre-compiled code for frequently-used methods;
- Direct Buffer Space holds fast direct I/O buffers;
- GC is an area reserved for the garbage collector;
- JNI is used by the Java Native Interface;
- Misc is reserved for the internal use of the JVM, holding information such as symbol tables.
For a deeper understanding of the JVM internal memory, you may like to watch this video: JVM in 10 minutes.
The first step in troubleshooting an OutOfMemoryError is to determine which of the above areas of memory has run out of space. We can establish this by looking at the type of error.
In the example in the previous section, the error message looked like this:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap spaceThe error type in this message is ‘Java Heap Space’. There are, in fact, nine different types of OutOfMemoryErrors that can occur in Java. Only two of them relate to the heap. We’ll take a brief look at each of them in the next section. The diagram below is an overview of the nine types, showing which of the memory areas they relate to.
Fig: Types of OutOfMemoryErrors Showing Related Memory Spaces
Types of OutOfMemoryErrors in Java
The first step in troubleshooting OutOfMemoryErrors in Java is to determine the error type. For example, it’s no good examining a heap dump if the error relates to direct buffers, which are not part of the heap.
Let’s briefly look at the error types, and what artefacts and tools help us solve them. It’s worth mentioning here that a really useful tool for any type of troubleshooting is yCrash, which extracts and analyzes a wide range of troubleshooting artefacts. There is also a free CLI-based script that captures 360° artefacts relating to the JVM and its environment.
1. Java Heap Space
This type of error indicates the heap has run out of space. This may be due to a memory leak, incorrect configuration of the heap size, excessive wasted memory or a surge in traffic.
The most useful artifact for troubleshooting this type of error is a heap dump, which can be analyzed using tools such as HeapHero or Eclipse MAT.
Read more about this type of error: Java Heap Space.
2. GC Overhead Limit Exceeded
This error type also relates to the heap, indicating that the garbage collector is unable to free enough Java heap space. The causes and troubleshooting strategies are the same as the ‘Java Heap Space’ error type.
Read more about this type of error: GC Overhead Limit Exceeded.
3. Requested Array Size
This error occurs if the program is trying to create an array that is larger than the JVM limit. This is roughly 231 bytes, the maximum that can be indexed by an integer, less a small amount for system overheads. This can happen if the array size is specified by the result of a faulty calculation. The best artefact for diagnosing this type of error is the stack trace.
Read more about this type of error: Requested Array Size.
4. Metaspace
This occurs when the metaspace grows too large. This could be because too many classes are created dynamically, or because the metaspace size is under-configured. Tools such as HeapHero, MAT and VisualVM include a class histogram taken from a heap dump, allowing us to see what classes are being held in memory.
Read more about this type of error: Metaspace.
5. Permgen Space
Older JVMs used a space known as the PermGen, or Permanent Generation, in place of the metaspace. The causes and troubleshooting strategies are the same as for the previous error type.
Read more about this type of error: Permgen Space.
6. Unable to Create New Native Threads
This error occurs when a Java program tries to create more threads than the underlying operating system is able to provide. It’s often caused by a bug, where the application is creating threads within a loop that doesn’t terminate. To troubleshoot this type of error, we need a thread dump. Tools such as fastThread analyze thread dumps to assist with diagnosis.
Read more about this type of error: Unable to Create New Native Threads.
7. Direct Buffer Memory
Direct buffers in Java are created using the method ByteBuffer.allocateDirect(). They create buffers in native memory rather than the heap, which results in faster I/O. If the buffers are larger than the maximum specified by the command line argument -XX:MaxDirectMemorySize, this error will be thrown. It may be the result of a memory leak.
The stack trace is the best troubleshooting artifact for these errors.
Read more about this type of error: Direct Buffer Memory.
8. Kill Process or Sacrifice Child
This happens on Linux systems when the device or container runs dangerously low on memory. The operating system uses an algorithm to choose an application to terminate in order to free up memory.
The error will appear in the kernel log rather than the application log, and the application may look as though it’s simply disappeared without trace. The yCrash script is the best way of recovering the artefacts we need for troubleshooting, since we’ll need to look at the system as a whole.
Read more about this type of error: Kill Process or Sacrifice Child.
9. Reason Stack_Trace_With_Native_Method
This type of error only occurs in applications that use the Java Native Interface (JNI). It’s most often seen in older applications. A possible cause is recursive calls to native methods.
The best starting point is to follow the stack trace. If this doesn’t provide the answer, you may need to use operating system tools for further diagnostics.
Read more about this type of error: Reason Stack_Trace_With_Native_Method.
Conclusion
To effectively diagnose OutOfMemoryErrors in Java, we need to understand the JVM memory model, and pay attention to the error message, which contains an error type.
Some error types relate to Java heap space, and can be diagnosed using a heap dump. Others require a different troubleshooting strategy.
With the right artefacts and tools, OutOfMemoryErrors can be diagnosed successfully.
Share your Thoughts!