How to Solve OutOfMemoryError: Requested Array Size

There are 9 types of java.lang.OutOfMemoryError, each signaling a unique memory-related issue within Java applications. Among these, ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’ is a rare type of error & seldom occurs. In this post, we’ll delve into the root causes behind this error, explore potential solutions, and discuss effective diagnostic methods to troubleshoot this problem. Let’s equip ourselves with the knowledge and tools to conquer this common adversary.

---

Here’s a video summary of the article:

---

JVM Memory Regions

To better understand OutOfMemoryError, we first need to understand different JVM Memory regions. Here is a video clip that gives a good introduction about different JVM memory regions. But in nutshell, JVM has following memory regions:

1. Young Generation: Newly created application objects are stored in this region.
2. Old Generation: Application objects that are living for longer duration are promoted from the Young Generation to the Old Generation. Basically this region holds long lived objects.
3. Metaspace: Class definitions, method definitions and other metadata that are required to execute your program are stored in the Metaspace region. This region was added in Java 8. Before that metadata definitions were stored in the PermGen. Since Java 8, PermGen was replaced by Metaspace.
4. Threads: Each application thread requires a thread stack. Space allocated for thread stacks, which contain method call information and local variables are stored in this region.
5. Code Cache: Memory areas where compiled native code (machine code) of methods is stored for efficient execution are stored in this region.
6. Direct Buffer: ByteBuffer objects are used by modern framework (i.e. Spring WebClient) for efficient I/O operations. They are stored in this region.
7. GC (Garbage Collection): Memory required for automatic garbage collection to work is stored in this region. 
8. JNI (Java Native Interface): Memory for interacting with native libraries and code written in other languages are stored in this region.
9. misc: There are areas specific to certain JVM implementations or configurations, such as the internal JVM structures or reserved memory spaces, they are classified as ‘misc’ regions.

What is ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’?

‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’ occurs when your application attempts to create an array that exceeds the maximum allowable size imposed by the JVM, which is Integer.MAX_VALUE (2,147,483,647). Even if you have sufficient heap memory available, this error will still be thrown if you try to create an array more than the imposed size limit.

What causes ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’?

JVM throws ‘OutOfMemoryError: Requested array size exceeds VM limit’ under the following conditions:

1. Parsing/Loading Large Files: Trying to load or parse very large files (e.g., reading an entire file into a byte array) without chunking can push array size beyond the safe limits.
2. Data Structure Pre-Allocation: Some frameworks or poorly written utilities may try to pre-allocate massive arrays assuming they will be used to store large datasets in-memory, which may not be practical.
3. Incorrect Calculations for Array Size: A bug or miscalculation in the code, such as multiplying large values, can cause the array size to exceed the valid integer range.

Solutions for ‘OutOfMemoryError: Requested array size exceeds VM limit’

Following are the potential solutions to fix this error:

1. Increase Heap Size: You can increase the maximum heap size (-Xmx) when running your Java application. This allows more memory to be allocated to your application, potentially allowing larger arrays to be created. However, be cautious as increasing heap size may have performance implications and may not always be a feasible solution, especially in memory-constrained environments.
2. Reduce array size: Try to see whether you can reduce the array size. If creating such a large array is unavoidable, consider alternative approaches such as using different data structures or processing data in smaller chunks.

Sample Program that generates ‘OutOfMemoryError: Requested array size exceeds VM limit’

To better understand ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’, let’s try to simulate it. Let’s leverage BuggyApp, a simple open-source chaos engineering project. BuggyApp can generate various sorts of performance problems such as Memory Leak, Thread Leak, Deadlock, multiple BLOCKED threads, … Below is the program from the BuggyApp project that attempts to create a very large integer array, which exceeds the VM limit:

public class OOMRequestedArraySizeExceedsVMLimit {
	
    public static void main(String[] args) {      

        // Attempt to create an array of the size (which will exceed VM limit)
        int[] arr = new int[Integer.MAX_VALUE];
    }
}

In the above program ‘OOMRequestedArraySizeExceedsVMLimit‘ class’s ‘main()’ method tries to create a very large ‘int’ array (i.e., new int[Integer.MAX_VALUE]). When this program is executed, you will see the ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’ reported in the console. 

How to troubleshoot ‘OutOfMemoryError: Requested array size exceeds VM limit’?

It is very easy to identify the root cause of ‘OutOfMemoryError: Requested array size exceeds VM limit’. This error would be printed in the std error log (or console), with the stack trace of the code that is causing it. Example when the above program is executed, you will see the following stack trace to be printed:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
   at com.buggyapp.oom.OOMRequestedArraySizeExceedsVMLimit.main(OOMRequestedArraySizeExceedsVMLimit.java:12)

It clearly points out that problem originates on the line #12 in the OOMRequestedArraySizeExceedsVMLimit.java. Equipped with this information, one can easily isolate the root cause of the problem and fix it.

Conclusion

In this post, we’ve covered a range of topics, from understanding JVM memory regions to diagnosing and resolving ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’. We hope you’ve found the information useful and insightful. However, our discussion doesn’t stop here. Your experiences and insights are invaluable to us and to your fellow readers. We encourage you to share your encounters with ‘java.lang.OutOfMemoryError: Requested array size exceeds VM limit’ in the comments below. Whether it’s a unique solution you’ve discovered, a best practice you swear by, or even just a personal anecdote, your contributions can enrich the learning experience for everyone.

FAQ