Performance matters: more and more so as the world increasingly relies on critical computer systems to facilitate everyday tasks.
In applications such as the automobile and flight industries, as well as robotics and fast-moving forex trading, every microsecond counts. Not all applications are quite this high-pressured, but even small business systems must pay attention to performance, or their costs will go up and their customer base will go down.
In Java, a high percentage of performance problems are memory-related. In this article, we’ll look at how a Java memory analyzer tool can help us find the cause of poor performance, which is, of course, the first step towards solving the problem.
Java Memory Analyzer Tools
The first and most important tool for solving memory issues is a good heap dump analyzer. In this article, we’ll be using HeapHero, but if you prefer, there are other first-class tools available, such as eclipse MAT.
Not all memory issues are heap-related, however. We’ll also look at how garbage collection (GC) logs can help us determine whether the problem lies in the heap, and what type of problem it’s likely to be. We’ll use GCeasy to analyze the logs, although again, you may prefer a different tool.
Lastly, for issues that don’t originate from the heap, we’ll take a brief look at Native Memory Tracking, using the JDK tool jcmd in conjunction with GCeasy.
Why Do Memory Issues Have Such a High Impact on Performance?
Both the JVM and the underlying operating system make every effort to provide memory for each request, to prevent the application crashing with an OutOfMemoryError. If either of these is having to work too hard to meet memory demands, performance degrades.
In the JVM, there are two primary areas in memory, known as heap memory and native memory. Heap memory is managed by the JVM, whereas native memory is managed by the underlying operating system. We can visualize it like this:
Fig: JVM Memory Model
Heap memory is used for storing all objects created by the application, whereas native memory is split into several areas, each with a specific purpose. For a detailed discussion of the JVM memory model, see this video: JVM Explained in 10 Minutes.
Ensuring there is enough heap memory to store objects as they are created is the job of the garbage collector (GC). This is a process that runs in the background within the JVM, and its job is to clean out any objects that are no longer reachable from the garbage roots. Garbage roots include static variables and references within the stack. Although much of its work can be done concurrently with application threads, critical tasks require all other threads to be paused to prevent heap corruption. These are known as Stop the World (STW) events.
If the GC is overworking because there’s not enough memory, it becomes extremely resource-hungry. CPU usage shoots up, and STW events happen more and more frequently. This in turn leads to poor response time, as user requests must wait until the GC event finishes.
GC efficiency can be defined by three Key Performance Indicators (KPI):
- Throughput: the percentage of time spent on processing application tasks as opposed to GC tasks. We need to aim for around 98% throughput.
- Latency: The duration of STW pauses. Both average latency and maximum latency are important.
- Footprint: The resources, such as CPU time, used by the GC.
Managing native memory is the job of the operating system. If native areas, such as the metaspace, grow too large, the operating system itself may run low on memory. Memory paging may become frequent, slowing down the system. Eventually, the entire device can become unresponsive, and any other applications running on the same device may be affected. Under Linux, it’s possible that the OS may kill the Java application to prevent a system crash. If the application is running under a load balancer, the users may lose their sessions and have to log in again.
As we can see, efficient memory usage is critical for high performance.
What Causes Memory Problems?
To find the right fix for a problem, we first need to accurately identify its cause.
Memory issues can be caused by:
- Memory leaks. These happen when objects build up in memory without becoming eligible for garbage collection. There could be many reasons for this. The most common is that some object is holding references to memory that’s no longer needed. It can also happen if a loop doesn’t terminate when it should, creating more and more objects as it iterates. For more information, see these articles:
- Wasted memory due to poor coding practices. These can result in memory hogs such as duplicate objects, collections that consist mostly of empty space, and unnecessary object headers. For more information, see How Much Memory is My Application Wasting? You may also be interested in a case study: Memory Wasted by a Spring Boot Application.
- An incorrectly configured heap size. If the heap size is inadequate for the task, the GC is going to be overworked, and performance will drop. For some hints on how to find the optimum heap size for an application, see Sizing Your Heap Correctly.
- Inefficient GC settings. Tuning garbage collection is often the fastest and most inexpensive way of improving performance, since the GC is extremely resource-hungry. We need to choose the best GC algorithm for the task and the environment, and ensure it’s optimally configured. You may like to watch this video: GC Tuning and Troubleshooting Crash Course.
- Inadequate memory in the device or container. If all other causes have been eliminated, and we’re sure the problem is memory-related, we have no option but to add more RAM to the device or container.
Using a Java memory analyzer, we can determine the real cause of memory issues, and find a permanent fix for the problem.
Diagnosing Problems Using Java Memory Analyzer Tools
Let’s look at how we may use some of the available tools to see whether we have memory issues, and if so, why.
1. Establish Whether We Have a Memory Problem, and What Type
The best way to do this is to analyze the GC logs. They can answer questions like:
- Are the key performance indicators acceptable? If not, we definitely can improve performance by addressing memory issues.
- Is the heap usage consistently high? If so, look for wasted memory, and ensure the GC is well-tuned. If these strategies don’t help, increase the heap size, always remembering we need to ensure we don’t increase it to the point where the device as a whole has memory problems.
- Is the heap usage increasing over time, even though the GC is running regularly? If so, we probably have a memory leak.
- If the heap usage is not excessive, yet GC is running frequently without clearing much memory, we probably have space issues within native memory rather than the heap.
Fig: Key Performance Indicators Extracted by GCeasy
2. For Heap-Related Issues, Use a Heap Dump Analyzer
If we’ve established that the problem relates to the heap, the next step is to take a heap dump, and submit it to a heap dump analyzer such as HeapHero.
Perhaps the most important section of the HeapHero report is the Largest Object report. The top three or four largest objects are always the most likely to be the culprit when we have heap-related issues. From this report, we can interactively explore the dominator tree, which is invaluable for finding memory leaks.
Fig: Largest Object Report Produced by GCeasy
For a video demonstration of this process see How to Analyze a Heap Dump Fast.
We can also explore upwards through the dominator tree from the Garbage Roots section.
A class histogram shows us whether we have an excessive number of objects of the same class, which can be a clue to finding objects created by a rogue loop, or issues with dynamic classes.
The duplicate object report is also useful, as is the list of objects awaiting finalization. If an object is created from a class that overrides the finalize() method, it can cause problems when this method hangs or is slow for any reason. Not only that object, but any behind it in the finalizer queue, won’t be garbage collected until the method completes.
Most applications can benefit by looking for wasted memory. HeapHero contains an interactive memory wastage report, where we can dig down to find exact details of the affected objects.
3. Native Memory Tracking
If GC log analysis indicates that the problem lies in native memory, this becomes a little harder to track down. The heap dump may not be helpful, although the class histogram is useful for finding memory leaks within the metaspace.
HotSpot Java distributions include a facility for native memory tracking (NMT).
To enable it, we would first invoke the JVM with the following switches on the command line:
java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=detailWe could also keep track of the summary only:
java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summaryAt any time when the program is running with this argument, we can request a dump of the NMT information to a file using the CLI utility jcmd supplied with the JDK.
jcmd <pid> VM.native_memory summary > filename.txtWe can either examine this text file manually, or, to save time, we can submit it to GCeasy, which displays the information in the format of useful graphs. This lets us discover which part of native memory is causing the problem.
You can find more information here: Understanding Native Memory Tracking in Java.
Conclusion
Poor performance could have many causes, such as a slow network or badly-designed thread locking, but memory problems are by far the most common. Excessive use of heap memory overloads the GC, whereas high native memory usage can degrade operating system performance. Either one can eventually cause the system to crash with an OutOfMemoryError.
It makes sense, therefore, to regularly check our applications for memory problems. Monitoring GC logs and submitting heap dumps periodically to a Java memory analyzer such as HeapHero can proactively prevent performance problems.
A few simple tweaks often turn a sluggish application into the digital equivalent of an Olympic sprinter.
Share your Thoughts!