Coding for Memory Efficiency: Best Practices to Avoid Heap Overflow

Why is memory efficiency important? Isn’t RAM so cheap now that we don’t need to worry how much our applications use?

At one time, this thinking was common in the IT industry, but now the pendulum is swinging the other way. Minimizing Java heap space requirements cuts cloud computing costs, avoids memory-related performance issues, and reduces disruptions due to system crashes. It also lets us develop applications for tiny devices, which are very much in demand.

What should we avoid to prevent our applications from becoming memory hogs? This article looks at what to do and what not to do to reduce memory requirements when we program in Java.

What Happens When a Java Application Runs Short of Heap Space?

When heap memory gets too full, the JVM initiates a garbage collection (GC) cycle to try to free up space. If there’s still not enough memory, it then checks the current heap size against the maximum heap size configured by the -Xmx parameter (or the default maximum if this wasn’t configured.) If the heap hasn’t yet reached the maximum, the JVM requests more memory from the operating system to make room for new memory allocations.

If there’s still not enough free heap memory to satisfy incoming requests, the JVM fails with one of these two errors:

• OutOfMemoryError: Java Heap Space: See How to Solve Out of Memory: Java Heap Space.
• OutOfMemoryError: GC Overhead Limit Exceeded. See How to Solve Out of Memory: GC Overhead Limit Exceeded.

Before the application actually crashes, we’re likely to see performance dropping drastically. GC cycles run more and more frequently as the JVM tries to keep memory at an acceptable level. Requests may time out, sessions may drop, and the device as a whole may slow down, because the GC process is very CPU-hungry.

What causes the Java heap space to fill up? There are three high-level causes:

• Memory leaks;
• Memory wastage;
• Under-configuration.

This article covers the first two causes, which can be solved by tightening up the code. For more information on correctly configuring the heap size, see Sizing Your Heap.

Memory Leaks: Constantly Draining Java Heap Space 

Memory leaks occur when objects that are no longer needed aren’t released to the GC in good time. The application runs fine to begin with, but over time, the unused objects build up until they cause performance problems, and eventually result in Out of Memory errors.

In a healthy application, the GC is able to consistently bring memory back down to a similar level, as shown in this graph of memory usage over time, with GC events shown as red triangles.

Fig: Healthy GC Pattern Graphed by GCeasy

If we have a memory leak, we’ll see a different pattern.

Fig: Memory Leak Pattern

Although the GC still clears memory on each cycle, the heap size never goes back to the same level. Objects build up over time, and the GC runs more and more frequently to try to clear heap space. As time goes on, GC events run almost back-to-back, affecting performance. Eventually, the system crashes due to heap overflow.

To prevent memory leaks, we need a good understanding of how memory is released for GC. 

An object can be garbage collected if it has no strong references pointing to it (unless otherwise specified by the programmer, all references are strong). References are removed either when they go out of scope, they are explicitly set to null or, in the case of ThreadLocal variables, when they are removed.

Let’s revise what we mean by going out of scope. The table below shows the scope of variables depending on how and where they were declared, and when they become eligible for GC if no other live references point to them.

Variables declared within a method, therefore may be short-lived, since they only remain while the method block is active. On the other hand, static variables may never be garbage collected, since classes are seldom unloaded.

It’s therefore highly important to declare variables within the correct scope, otherwise they may be kept much longer than needed. Most memory leaks are caused by objects that aren’t declared in the right place. 

We need to be especially careful of static variables:

• Declaring large objects, such as collections, as static is never good practice;
• Be careful of making static variables refer to local variables. This is likely to result in local variables being kept indefinitely, unless the reference is explicitly set to null;

Other causes of memory leaks include:

• Collections that are allowed to grow indefinitely. Unfortunately, most collection structures don’t have the option to set a maximum size. We can easily check the size, however, and take action if they grow too big.
• Rogue loops that add objects to collections.
• Caches that aren’t properly managed. Caches should always have a maximum size, and an eviction policy to remove items that haven’t been recently used.
• Slow finalize() methods. This method is invoked by the GC immediately before the object is removed from memory. It’s handled by a finalizer queue, which is single-threaded. This means if an object’s finalize() method is slow or hangs, perhaps waiting for network, I/O or locks, other objects build up behind it in the queue. None of these objects can be removed from memory in the meantime, and they may accumulate to the point of causing an Out of Memory error. In fact, the finalize() method has been deprecated since Java 9 for this very reason.
• Listeners that aren’t deregistered can prevent a listening object from being garbage collected, since the event source object still holds a reference to it. The same problem can occur with callbacks. To prevent this, it’s good practice to always use weak references when registering listeners.

Coding Practices That Waste Java Heap Space

It’s surprising how much memory is simply wasted by most Java programs. You may be interested in this case study, where heap dump analysis showed that the popular Spring Boot sample application, Pet Clinic, actually wastes 65% of memory.

Wasted memory increases cloud computing costs, impacts performance and can cause Out of Memory errors during peak processing times.

Analyzing GC logs is likely to show a pattern similar to this one:

Fig: High Heap Usage

Although there is no leak, GC needs to run frequently, since the heap is using close to its maximum allocation.

What can we tighten up on to prevent wasted heap space? Here are a few things to look at:

• Loading entire data sets into memory at one time. This is seldom necessary, unless the data set is very small, for example, a configuration file. Instead, wherever possible, use:
  • Streams to process data in smaller chunks;
  • Database cursors to filter large datasets. Select only the relevant items, summarize if necessary, and use cursor commands to traverse the data.
• Duplicate Strings.  It’s very common for a program to store identical text in many different places. Company data, product descriptions, logging messages and much more may be duplicated across many threads dealing with incoming requests. For a detailed description on how to easily prevent this, see String Deduplication in Java.
• String Concatenation. Take this code snippet:

String concatData="";
for (int i=0;i< arraySize;i++)
  concatData=concatData+arrayString[i];

For each iteration, a new String object is created to hold the result, since Strings can’t be mutated. If the array of strings is large, we would create thousands of Strings, each with its own object header taking up space. If instead we code this as shown in the second snippet, only one StringBuffer is created, saving a lot of memory.

StringBuffer concatData = new StringBuffer();
for (int i = 0; i < arraySize; i++) {
    concatData.append(arrayString[i]);
}
String result = concatData.toString();

• Inefficient Collections. Many programmers declare Collection objects, such as HashMaps and ArrayLists, without specifying an initial size. Each class has its own defaults, but in the case of HashMaps, it defaults to starting with 16 buckets, and doubling in size every time the current size is close to being exceeded. For tiny collections that occupy less than the default, the remaining space is wasted. A more serious problem is that if a collection currently holds, say, 2²⁰ items, and one more is added, the collection would be resized for 2²¹ items. If no more items were ever added, that would be a lot of wasted space. It’s better, therefore, to decide on a reasonable size for the collection, and create it at this size. To allow for growth, we could make the size parameterized so it can be configured.
• Holding Separate Caches for Each Thread. This can waste a lot of space. 
• Duplicate Objects and Arrays. Make sure large objects are stored only once and shared.
• Inefficient Arrays. As with Collections, we should decide on a reasonable size for arrays. We can make this configurable for expansion if we need to.
• Object Header Overheads. Each object in Java has a header, which may occupy 12 or 16 bytes, depending on the JVM. Java primitives, such as byte, float, long etc. have no such overhead. We should always take this into account, and remember:
  • A variable defined as byte occupies a single byte, whereas if it’s defined as new Byte() it occupies at least 13 bytes, including the object header. This can make a big difference if we have a large array of numbers. Numbers stored in objects such as Byte, Long, Float etc. are known as boxed numbers, and we should avoid using them for data storage.
  • Grouping several small objects together into a single larger object can save space, since less object headers are stored.
• Classes that Inherit Lots of Fields That Aren’t Needed.  It’s tempting to simply extend an existing class to inherit its methods, but if the parent class stores a lot of data that we don’t need, it becomes wasteful.
• Using the Wrong Data Structures. Some data structures use a lot of space, and if we use them when a simpler structure would work just as well, it can add big memory overheads.

Diagnosing Java Heap Space Issues

How can we see exactly how much space our objects are using? This is an important part of testing large systems, and it’s also essential for troubleshooting memory-related problems.

The first step is to take a heap dump, which is a snapshot of the contents of the heap at a given moment. Next, we need a heap dump analyzer such as HeapHero or Eclipse MAT, so we can explore the contents of the dump.

HeapHero has the added advantage that it includes a detailed, interactive breakdown of any wasted memory it detects.

Fig: HeapHero Wastage Report

The GC logs are also a source of valuable information. By monitoring these logs using a tool such as GCeasy, we can proactively detect memory problems before they impact production. They can also let us detect patterns, so we can tell if we have a memory leak, or simply need more memory.

Conclusion

Saving memory is becoming more and more important, with microservices, cloud computing and small devices becoming popular. Using too much Java heap space can be costly, and impacts performance.

From the planning stage, through coding and testing, we need to keep memory usage in mind.

By using diagnostics such as heap analyzers and GC log analyzers during performance testing, we can save a lot of heartaches in production.

But most of all, we need to develop good coding habits, keeping wastage to a minimum.