OutOfMemory errors are annoying in the testing phase, but they become a nightmare if they happen in production. All too often, an application works perfectly in development, runs well in the live environment – and then crashes without warning.
How do you diagnose and fix OutOfMemoryError? Java provides several JDK tools that are useful for analyzing run-time issues, but making sense of the information can be time-consuming. This article looks at how to understand the different types of OutOfMemoryErrors, gather and analyze information to identify the problem, and decide on an appropriate fix in different situations.
Understanding the JVM Memory Model
The JVM has several memory pools. When we get an OutOfMemoryError, we need to know which area of memory it relates to. The diagram below illustrates these memory pools.
Fig: JVM Memory Model
Briefly, these pools are used as follows:
- The Heap: This area is a central storage for all objects created by the application. It’s managed by the JVM rather than the device’s operating system. To make garbage collection (GC) more efficient, it’s split into:
- Young Generation: Newly-created objects. This area is again subdivided into Eden Space, for very new objects, and S0 and S1, which hold objects that have survived at least one cycle of GC.
- Old Generation: Objects that have survived several cycles of GC.
- Native Memory: This area is managed by the operating system. It contains several areas for specific uses:
- Metaspace: Class metadata, byte code for methods etc. In older versions of Java, these were stored in the PermGen, which was JVM-managed.
- Threads: Stacks for each active thread,
- Code Cache: Pre-compiled machine code for frequently-used methods (hot spots.)
- Direct Buffers: Buffers used for fast I/O, including file and network buffers.
- GC: Reserved for the garbage collector.
- JNI: Used by the JNI (Java Native Interface).
- Misc: Reserved for various JVM activities.
For more information, you may like to watch Understanding JVM Memory.
Types of Java OutOfMemoryErrors, and Related Memory Areas
Java applications can throw 9 different types of OutOfMemoryErrors. The first step in troubleshooting the problem is to establish the reason by reading the error message, and see which memory area it relates to. This is summarized in the table below.
| Error Message | Related memory area |
| java.lang.OutOfMemoryError:Java heap space | Heap |
| java.lang.OutOfMemoryError:GC Overhead Limit Exceeded | Heap |
| java.lang.OutOfMemoryError:Requested Array Size | Heap |
| java.lang.OutOfMemoryError:Metaspace | Metaspace |
| java.lang.OutOfMemoryError:Permgen Space | PermGen |
| java.lang.OutOfMemoryError:Unable to create new native threads | Threads |
| java.lang.OutOfMemoryError:Direct buffer memory | Direct Buffers |
| java.lang.OutOfMemoryError:Stack trace with native method | JNI |
| Out of memory: Kill process <pid> (java) score <score> or sacrifice child | Device/Container |
Strictly speaking, the last-mentioned error is thrown by the operating system, rather than the JVM, and relates to the total memory within the hosting device or container.
For more information, and links to specific articles dealing with each type of error, see 9 Types of OutOfMemoryError in Java.
These 9 error messages, and the related memory areas, are illustrated in the diagram below.
Fig: Types of OutOfMemoryErrors and Related Memory Areas
Troubleshooting Artifacts
The table below summarizes the troubleshooting artifacts you need to collect in order to identify the cause of the error. It shows which artifacts relate to which type of errors. Where appropriate, it contains links to more information, including instructions for retrieving the artifact.
| Artifact | Useful information | Related Out of Memory Errors |
| Stack trace | Trace the program path to the point of the crash | All Errors |
| Heap Dump | How much memory is used? By what objects? | Java heap space |
| GC Overhead Limit Exceeded | ||
| Metaspace | ||
| Permgen Space | ||
| Direct buffer memory | ||
| Thread Dump | What threads are running? What are they doing? | Unable to create new native threads |
| GC Logs | Is GC performing as it should? | Java heap space |
| GC Overhead Limit Exceeded | ||
| Metaspace | ||
| Permgen Space | ||
| Direct buffer memory | ||
| Dmesg(Linux)/Event log (Windows) | Operating System Error Messages | Kill process <pid> (java) score <score> or sacrifice child |
| Application log | Stack trace, application messages | All errors |
Overview of JDK Troubleshooting Tools
The JDK includes a range of troubleshooting tools, which allow us to get information about a running process, take heap dumps and carry out other analytical tasks. These are summarized in the table below, together with links to their online documentation.
| Tool | Purpose |
|---|---|
| jcmd | Sends diagnostic commands to a running Java process |
| jstack | Prints thread stack (Experimental) |
| jvisualvm | Visual monitoring and troubleshooting tool |
| jconsole | Visual monitoring of performance and resource usage |
| jstat | Displays performance statistics for a running Java process |
| jmap | Memory-related statistics for a running Java application |
| jps | Show PIDs of all running Java applications |
However, retrieving meaningful analytics with these tools can be very time-consuming, although they’re great for taking a quick look at how an application is performing.
Many people prefer to use third party tools, such as the yCrash suite, to quickly diagnose application issues.
The yCrash Suite
yCrash is a comprehensive set of analytical utilities that allow you to diagnose OutOfMemory and other Java issues easily. Most of the tools can either work in the cloud, as an on-premise tool, or as an API.
Heap dumps can contain confidential information: the memory contents could include sensitive data such as credit card numbers or social security numbers. If this is the case, the on-premise version allows you to analyze the dump without uploading it to the cloud.
These tools consolidate diagnostics into meaningful graphs and charts, provide memory usage statistics, and include AI-generated tailored troubleshooting and performance tips.
Let’s take a look at the tools that make up this set, and see how they can help with diagnosis.
- HeapHero
HeapHero analyzes a heap dump, and, among other things, produces:
- Highlights of any potential issues found;
- Overall usage statistics;
- Interactive charts of objects in memory, highlighting the largest objects. It’s possible to see details of each object, such as its parents, children and actual contents;
- A histogram of classes that are currently loaded;
- Information about running threads.
The list of objects, sorted from largest to smallest, is useful for diagnosing OutOfMemoryErrors that relate to the heap. In most cases, OutOfMemoryErrors are caused by one of the few objects at the top of this list.
Although direct buffers are not included in the heap, objects that relate to them, such as cleaners, will appear in the heap and give clues to errors that relate to the Direct Buffer area.
The histogram is useful to see what classes are loaded in the case of errors relating to the Metaspace or PermGen.
The image below is an extract from a HeapHero report showing the largest objects.
Fig: List of Largest Objects Produced by HeapHero
- GCeasy
GCeasy analyzes GC logs. The GC cleans not only the heap, but other areas as well, so GC performance is relevant to most Java OutOfMemoryErrors.
The GCeasy report contains, among other things,
- Highlights of any potential issues found, as well as tuning suggestions;
- Memory usage in each heap area;
- Key performance indicators;
- Interactive graphs of heap usage before and after GC;
- Breakdown of GC times.
Memory leaks are one of the primary causes of OutOfMemoryErrors, and it’s possible with GCeasy to spot GC performance patterns that indicate this type of problem. The image below shows healthy GC patterns compared to the pattern indicating memory leaks.
Fig: Identifying Memory Leak Patterns from GCeasy Report
- fastThread
This tool is excellent for troubleshooting thread-related errors such as java.lang.OutOfMemoryError: Unable to create new native thread.
The report contents include:
- Highlights of any potential issues found;
- Interactive graphs of threads currently in use;
- Thread pools;
- Threads with identical stack trace;
- Threads with high CPU.
The image below shows fastThread statistics for a program that has a bug where it repetitively creates hanging threads.
Fig: fastThread Graph for Program with Hanging Threads
- yCrash
This is a central utility that gives access not only to the three tools listed above, but also to 360° diagnostic data taken from both the JVM and the operating system.
It can be installed on-premise as an agent/server that constantly collects real time diagnostics from a running application. The agent has a very low footprint, so it’s feasible to run it in the background on production systems. It can also be accessed via API, or via a downloadable script.
Fixing OutOfMemoryError in Java
Once we’ve diagnosed the problem, we can decide on a fix.
Some issues require fixing bugs in the program code. These include:
- Memory Leaks;
- Recursive Method Calls;
- Buffers Created Too Large;
- Excessive number of threads.
Consider using pools for resources such as numerous threads and large buffers.
In some cases, the solution is to make the heap size or metaspace bigger, or to adjust the sizes of spaces within the heap. We’d use JVM command line switches to do this. We can also use a JVM switch to limit the size of thread stacks in the case of errors related to threads. This is not always recommended, however, as it could result in a StackOverflowError.
Tuning the garbage collector may sometimes be the solution, or switching to a different GC algorithm.
In some cases, where other solutions are not sufficient, we may need to add more RAM to the device or the container.
Conclusion
When we’re faced with the need to fix OutOfMemoryErrors in Java, we need a systematic strategy. This can be summarized as:
- Identify the reason from the error message;
- Identify the affected area of memory;
- Gather the necessary diagnostic artifacts;
- Use diagnostic tools to find the likely cause of the problem;
- Decide on and implement a suitable fix.
When you’re faced with live system crashes, you need to work fast to get the system back up and running. I hope this article helps!
Share your Thoughts!