How does the JVM allocate memory? How is it organized, and when is it released?
These are all important concepts for troubleshooting, and for creating memory-efficient code. In this article, we’ll look closely at Java heap space, which is one of the JVM runtime memory areas, and the most common source of memory-related issues.
JVM Architecture: The Memory Model
Java memory is, at the highest level, divided into two main areas: heap memory and native memory. The heap is managed by the JVM, whereas native memory is managed by the operating system.
We can visualize it like this.
Fig: JVM Memory Model
The Java heap is a large pool of memory, shared between all threads and classes that make up the application. All objects are stored in the heap. Primitive variables are stored in the stack if they are local variables; otherwise, they are also stored in the heap. Heap memory is managed by the JVM.
To some, the name may be misleading, as the Java heap is not organized as a computer-science-defined heap data structure. It’s named that way because it conforms to the broader computer science definition of a large pool of dynamically allocated memory.
We’ll discuss the subdivisions of the heap in a later section.
Native memory is managed by the operating system. It consists of several special-purpose memory pools:
- Metaspace is used to store class definitions, including bytecode for each method, and metadata describing the class. In Java versions prior to Java 8, class definitions were stored in the Permgen (permanent generation), which was part of the heap.
- Thread Space contains a stack for each running thread.
- Code Cache contains pre-compiled machine code for frequently-used methods.
- Direct Buffers are used for fast input/output buffers.
- GC is a work area used by the garbage collector.
- JNI is used by the Java Native Interface.
- Misc holds miscellaneous data needed by the JVM, such as symbol tables.
What is Stored in the Java Heap Space?
Simply put, all variables except for Java primitive local variables are stored in the heap. The stack stores the Java primitives as part of the stack frame for each method. For objects defined as local variables, the object is stored in the heap, and the stack stores pointers to them.
To visualize this clearly, I suggest watching this video:
How is the Heap Space Managed by the JVM?
The heap space is managed by the garbage collector (GC). This is a background process that runs at intervals to make sure there is enough free memory for new allocations.
Its job on each cycle, known as a GC event, begins with identifying GC roots: items in memory that are known to be still in use. Primarily, these include:
- References within a stack frame that point to objects in the heap. Since frames are popped off the stack as soon as the method completes, references contained in existing frames are guaranteed to be in use.
- Static Variables. These remain in memory, unless the loader that created their class is discarded. Static variables may reference child variables, in which case, the child variable is seen as being still in use.
Once these roots are identified, the GC marks them and all their children as having live references, and therefore still required. It then takes each child variable, and marks its children as being in use, and recursively works through the entire parent-child dominator tree.
Fig: Garbage Roots and the Dominator Tree
At the end of this process, any items that have not been marked are labeled as finalizable. If they have no finalize() method, they are immediately removed from memory; otherwise, they are added to a finalizer queue, and will be removed during a future cycle once the method has been executed.
As you can imagine, this process is quite CPU intensive, and also, part of this work can’t be done while other threads are working. Any new variables created could corrupt the system, and references may need to be updated by the GC to point to the object’s new location. Application threads are therefore stopped while the GC is in critical stages. These pauses are known as stop-the-world events, and the time during which threads are paused is referred to as latency.
To speed up the GC process, most GC algorithms split the heap into the Young Generation (YG) and Old Generation (OG). This is always true for the Serial, Parallel, CMS and G1 algorithms. The theory behind this is that most objects in memory ‘die young.’ For example, when a transaction request is received, several variables will be created to hold relevant data. However, once the transaction is complete, these variables can be released.
The heap is therefore arranged like this.
Fig: Heap Space in Generational GC
The young generation is quite small, and it’s cleaned frequently, whereas the old (tenured) generation is much bigger, and cleaned less often. Minor GC events clean only the YG, whereas major GCs clean the OG as well, and also may free memory in the metaspace.
The YG is also subdivided into Eden space, and S1 and S2. New objects are always created in Eden. S1 and S2 (Survivor Generations) are used alternately during a minor GC event as described below. We’ll refer to them as Current Survivor and Previous Survivor areas in the description.
- Objects in Eden that have live references are moved to the current survivor area, and aged as 0;
- Eden space is cleared, thus removing unreferenced objects.
- Objects in the previous survivor area that have live references are moved to the current survivor areas, and their ages are incremented.
- The previous survivor area is cleared.
- Any objects in the current survivor area that are aged above a given limit are removed from the YG and promoted into the OG.
When the OG fills up beyond a given threshold, a major GC event is triggered. This removes any objects that have no live references from the OG. Most GC algorithms then compact the OG.
This method is known as Generational GC.
Two of the newer GC algorithms, Shenandoah and ZGC, work differently. They split the heap into pages, and pages are cleaned individually, prioritizing those with a high percentage of garbage. Newer versions of these two algorithms allow us to configure them to use generational GC on top of this paging system, if we require it.
From a coder’s point of view, when are items released to the GC? It’s important to understand this to prevent memory leaks.
We can explicitly release an object to the GC by setting it to null. Otherwise:
- Local variables are defined inside a method or code block. They are eligible for GC when they go out of scope. This happens when the method completes and is therefore popped off the stack.
- Instance (member) variables are defined within a class, but not inside any code block. These variables form part of an object created from a class, and they are eligible for GC when the object itself is released to the GC.
- Static (class) variables are defined using the keyword static. They are only eligible for GC if the class loader that created the class becomes eligible for GC, allowing the class to be unloaded. Unless we’re using custom class loaders by design, this won’t happen.
- Variables declared using the ThreadLocal feature are eligible for GC when the thread completes, or when they are explicitly removed.
What Causes Heap Space Issues and How Do We Troubleshoot Them?
The GC is always on standby to attempt to clear memory whenever more space is needed in the YG or the OG. If contention for memory resources is high, the GC runs more and more frequently, negatively impacting performance. The system may experience long pauses, response time may be slow, and CPU usage high. If it’s unable to clear enough space for new memory allocations, the JVM throws an OutOfMemoryError. If you want to learn about these OutOfMemoryErrors and its types, then this blog is a must read where we also discuss its causes, how to diagnose each type and remedy them.
Heap space problems can be grouped into three main causes:
- Under-configuration. The heap itself may not be optimally configured, or the device or container may be short of memory. If traffic increases over time, we often need to increase the heap size accordingly.
- Memory wastage. In many applications, most of the memory is not serving a useful purpose, but instead is wasted by poor coding practices.
- Memory Leak. If the application isn’t releasing variables to the GC when they’re no longer needed, they build up over time, eventually causing performance problems and OutOfMemoryErrors. To find out more, see Java Memory Leaks: The Definitive Guide to Causes, Detection & Fixes.
If we’re experiencing Java heap space problems, the first thing we need is a heap dump, which is a snapshot of the heap at a given moment. Since this is a large binary file, we need a Java heap dump analyzer, such as HeapHero or Eclipse MAT, to explore its contents. For a demonstration of how to use HeapHero to troubleshoot heap problems, see How to Analyze a Heap Dump Fast.
Prevention is always better than cure, so it’s important both in testing and in production to keep an eye on how well the GC is working. By enabling GC logs and monitoring them from time to time, we can detect issues proactively before they affect production.
Coding For Efficient Heap Space Usage
Modern programmers are too inclined to treat memory as if it were an unlimited resource. RAM is cheap, so why bother to save it?
With the modern trend towards cloud computing, microservices in containers, and small devices, memory efficiency is once again a high priority. Cloud providers charge by resources used, so memory-hungry applications can be costly to run.
Here are a few hints for making sure our applications don’t turn into memory hogs.
- Make sure variables are defined in the right scope, so they will be released when they’re no longer needed.
- Beware of memory wastage. A few areas to look at are:
- Always define a sensible initial size for collection objects. If they need to expand at run time, most of them do so by doubling their current size, which can result in a lot of wasted space.
- Avoid using an object when you could use a primitive. Each object has a header that occupies from 8 to 16 bytes, depending on the JVM. Let’s look at what is called a boxed number. A variable defined as the primitive int occupies 4 bytes, whereas if it’s defined as new Int() (a boxed number) it could occupy from 12 to 20 bytes. This can make a big difference if we have huge arrays of numbers.
- Duplicate strings often waste a lot of memory. To learn how to avoid them, see this article: String Deduplication in Java.
- At the planning stage, decide what actually needs to be in memory at a given time. Some developers happily load an entire file or query result into memory at once, when they could instead use streams or cursors to process the data sequentially.
- Never use caches without size limits: they could grow indefinitely. Caches should also have a working eviction policy to remove least-used items from the cache. It’s worthwhile using a tried and tested open source library such as Google Guava, rather than re-inventing the wheel.
Configuring the Heap
Many heap-related parameters can be used as arguments when invoking the JVM. For a list of most-used JVM arguments and their syntax, see JVM Arguments Master Sheet.
The overall heap size is configured using these two arguments:
- -Xmx: The maximum space the heap is allowed to occupy.
- -Xms: The initial heap size.
If we’re working within a container, we would usually use these arguments instead:
- -XX: MaxRAMPercentage: Set the maximum size as a percentage of available memory.
- -XX: InitialRAMPercentage: Set the initial size as a percentage of available memory.
There are strong arguments for making the initial and maximum heap size the same.
If the heap is configured too big, the unused memory will be wasted, as the operating system reserves this memory for the JVM. If we’re running in the cloud, we’ll be charged for it. On the other hand, if the heap space is configured too small, we run the risk of performance problems and OutOfMemoryErrors.
We can also configure other properties of the heap, although in most cases, the JVM makes its own good decisions if we don’t set the parameters. These include:
- -XX:+Use<GC name>: Select the GC algorithm;
- -XX:MaxTenuringThreshold: The tenuring threshold controls how many cycles of GC an object must survive before being promoted to the OG.
There are other options specific to the garbage collector we’re using. You can learn more in this video below:
Conclusion
A good understanding of the Java heap space is essential for both developers and troubleshooters.
In this article, we’ve looked at what’s stored in the heap and how it’s managed. We’ve also looked briefly at efficient coding strategies, and the basics of heap configuration.
Efficient heap usage is essential for high-performance, critical applications.
Share your Thoughts!