In many cases, Java applications work reasonably well using the JVM defaults for run-time parameters.
Many developers are not aware that there are definite advantages to controlling the initial and maximum size of the heap. Using simple command line arguments when invoking the JVM, it’s often possible to greatly improve performance, and sometimes cut costs dramatically when running in the cloud.
In this article, we’ll look at the kind of problems that can be caused by incorrect heap sizing, how to tell if the heap size is optimal, and how to adjust it as needed.
What is the Heap, and Why Does its Size Matter?
Java runtime memory is split into several different areas, each with its own purpose. This is illustrated in the diagram below.
Fig: The JVM Memory Model
The heap, consisting of the young and old generations, is an all-purpose storage area shared between all classes that make up the application. For a full description of the JVM memory model, you may like to watch this short video: JVM Explained.
JVM memory is cleaned regularly by the garbage collector, a background process that removes any objects no longer in use.
If too much memory is allocated to the heap, the excess memory is wasted: it can’t be used for anything else. If we’re running in the cloud, we’ll still be billed for the amount of memory allocated, even though we’re not using it. If we’re running on a local server, we may find that other applications sharing the same machine may perform badly, since they may not have enough memory to run efficiently.
On the other hand, if too little memory is allocated to the heap, we’ll see poor performance, intermittent unresponsiveness and, on occasion, system crashes. We may also find that other applications on the same device run slower, since our application is likely to use excessive CPU time.
Think about what happens when a company books a venue for a conference. It’s important to have a good estimate beforehand of how many delegates to expect. If we book too small a venue, the conference won’t work well, as there will not be enough resources. If we book too large a venue, the delegates will be comfortable, but we’ll incur a big bill for resources we’re not using. Sizing the heap is exactly like this.
How Can We Tell Whether the Heap is Sized Correctly?
The easiest way to do this is to analyze how well garbage collection (GC) is performing.
To do this, we need to enable GC logging from the command line when we invoke the JVM.
For Java versions 8 and earlier, the argument looks like this:
-XX:+PrintGCDetails -Xloggc:<gc-log-file-path>For Java versions 9 and later, the argument is:
-Xlog:gc*:file=<gc-log-file-path>In fact, it’s always a good idea to enable GC logging in production, since it adds little overhead to the system, and the logs are invaluable for monitoring and troubleshooting.
It’s best to monitor the logs over a period of at least 24 hours, so we can see what’s happening during peak and non-peak hours.
The logs are large and cumbersome to analyze manually, but there are several excellent tools available for gaining insights from the logs. They include:
In this article, we’ll be using GCeasy for sample monitoring.
What Happens If the Heap is Under-Allocated?
If we haven’t allocated enough memory to the heap, we’re likely to see:
- Poor response times, especially during peak hours;
- The system becoming intermittently unresponsive;
- CPU usage spiking;
- Intermittent crashes with Out of Memory errors.
This can, in fact, also happen if other areas of the JVM are under-allocated.
When the GC is overloaded, CPU usage spikes and the system becomes unresponsive. It runs more and more frequently, trying to clear memory, and it’s extremely resource-hungry.
Let’s have a look at what we may expect to see in GC log analysis when the heap is under-allocated. One of the first things to look at is the KPI (Key performance Indicators) report:
Fig: KPI Report Produced by GCeasy
- Throughput is the percentage of time spent during actual work, as opposed to time spent in garbage collection. Throughput should be high. 95%, for example, may sound good, but if 5% of time is spent on GC, it means we’re wasting a whole hour of every day where the application is doing no work. We should aim for 98 or 99%.
- Latency is the time during which all application threads are paused while the GC is in a critical stage.
If throughput is poor or latency is high, it indicates that there may be memory issues.
The next thing to look at is a graph of how often GC is run, and how much memory is cleared each time. A healthy GC pattern looks like this:
Fig: Healthy GC Pattern Shown on GCeasy Graph
The red triangles indicate full GC events. GC is running at regular intervals, and it’s able to reduce memory to a similar level each time.
If the heap is under-allocated, we may instead see something like this:
Fig: GCeasy Graph Showing Back-to-back GC Events
Notice that at certain times, GC events are running almost back-to-back, without clearing a significant amount of memory. In this graph, the problem is occurring at a certain time, and then resolving itself. This usually happens when there is not enough heap space to deal with peak traffic.
It’s also possible that we may see a graph like the one below.
Fig: GCeasy Graph Showing Progressive Memory Issues
Here the situation is deteriorating over time, and is not resolving. This is most often seen where there is a memory leak, but it can also happen when the heap is under-allocated.
GCeasy includes warnings at the front of the report if GC events are running back-to-back, throughput or latency are unacceptable, or any other issues are detected.
Wherever we see GC events running back-to-back, it indicates memory issues, and, if we’re certain the program doesn’t have memory leaks, we need to make the heap size larger. Alternatively, if the problem is only occurring at peak times, we may consider additional JVMs when demand is high.
What Happens if the Heap is Over-Allocated?
If this is the case, it won’t be obvious. The system is likely to perform well, and GC patterns are probably healthy. However, if we’re running on our own servers, we may see overall system performance degraded, since there may not be enough memory left for the operating system and other applications to run well. If we’re running in the cloud, our bills will be much too high.
Over-allocation is one of the issues GCeasy is able to diagnose. It does this by modeling, where it simulates the effect of reducing the heap size, and checks whether this would cause performance problems. If it detects that memory is over-allocated, it displays a message at the front of the report:
Fig: GCeasy Indicates Memory May Be Over-Allocated
This should be taken as an indication only, as it may not be 100% accurate, especially if the logs don’t span a long enough period.
Arriving at the absolute optimum heap size generally involves a bit of experimenting. If the GCeasy report indicates that memory may be over-allocated, or even if GC is looking too ‘healthy’, it’s worth trying the following:
- Ensure GC logging is in place;
- Pick a time period that covers both peak and non-peak traffic as the trial period;
- Reduce the heap memory size by a small amount;
- Analyze the logs at the end of the period, and compare them to the pre-trial statistics.
- If throughput has not reduced, latency has not increased, and GC patterns are still healthy, keep the new settings. If not, revert to the previous settings: the memory was not overallocated.
We can repeat this until we arrive at the optimum heap size.
For more information, see this post: How to Optimize Memory Allocation.
How Do We Adjust the Heap Size?
Changing the heap size is very simple: the initial heap size and the maximum heap size are controlled by command line arguments when the JVM is invoked.
The initial heap size, when the JVM starts up, is controlled by the argument -Xms. The maximum size to which the heap is allowed to grow is controlled by the argument -Xmx. Up to the value of -Xmx, the JVM can request more memory from the operating system as needed.
For example, to set the initial heap size to 512 MB and the maximum heap size to 1 GB when running a program named myprog, the command line would look like this:
java -Xms512m -Xmx1g myprogThe arguments –Xms and –Xmx are followed immediately by the required size suffixed by either k, m or g to indicate kilobytes, megabytes or gigabytes.
If these arguments aren’t provided, the JVM calculates default initial and maximum heap sizes, depending on the version of Java and the memory size of the device. The initial size is usually a small percentage of the available memory, and the maximum may be around 25% of available memory. This calculation doesn’t work properly when using Java versions prior to Java 8 within containers.
It’s also possible to set initial, minimum and maximum heap sizes as a percentage of the available memory.Tthis is especially useful to make the application scalable when running in a container. The command line switches are: -XX: InitialRAMPercentage, -XX: MaxRAMPercentage and –XX: MinRAMPercentage. This facility is only available in Java versions 9 and later.
Many system administrators like to set -Xms to a lower value than -Xmx, but this is not usually a good idea. Unlike thread pools and connection pools, there is no real advantage to setting the initial heap size lower than the maximum. In fact, setting the two parameters to different values has several disadvantages:
- The application may not be stable if the entire device becomes short of memory. If the operating system is unable to supply additional memory when requested, it could result in the application throwing an Out of Memory Error. On the other hand, if the operating system runs dangerously low on memory, it uses an algorithm to decide which application it views as the culprit. Any application that repeatedly asks for more memory may be seen as a rogue process and killed by the operating system.
- There is some overhead involved in requesting memory from and releasing memory to the operating system. For memory-intensive applications, this can result in a significant loss of performance.
- The GC may run more frequently, as it is generally triggered before the JVM requests more memory.
- If the initial memory is not sufficient, it may increase the application’s start-up time.
Setting -Xms lower than -Xmx doesn’t lower cloud computing costs, so there is really no good reason to do this.
For a full cover of this topic, and some benchmark tests, see this article: Benefits of Setting Initial and Maximum Heap Sizes the Same.
Conclusion
Understanding heap sizing is important for systems administrators and developers. If the heap is set to the optimum size, we avoid performance issues and crashes on the one hand, and reduce running costs on the other.
Garbage collection monitoring is key to making sure the heap is set to the best possible size for the application. In this article, we’ve used GCeasy to analyze the logs and identify key performance indicators. With regular monitoring and adjusting two simple command line arguments, we can keep our applications running efficiently without incurring unnecessary costs.