Memory Leaks are challenging performance problems to solve, especially when it happens in a production environment. In the earlier post, we have discussed common types of Memory Leaks and not so common types of Memory Leaks. Memory Leaks exhibits interesting symptoms, such as:
- Gradual Memory Increase Over Time
- CPU Spike
- Degraded Response Time/Timeouts
- OutOfMemoryError
In this post, let’s discuss the symptoms of a Memory Leak in detail.
Video
In this video, our Architect Ram Lakshmanan has explained about the symptoms of memory leaks.
1. Gradual Memory Increase Over Time
Fig: Memory consumption pattern of a healthy application
Above is the memory consumption pattern of a healthy application. You can notice that memory consumption grows till the peak point, then the Full Garbage Collection event (i.e., red triangle) gets triggered. After that memory consumption drops all the way to the bottom. You can see this saw-tooth pattern continuously repeating.
Now let’s review the memory consumption pattern of an application suffering from the memory leak.
Fig: Gradual Memory Increase in the application suffering from Memory Leak
In the above graph, you can observe that when the first Full Garbage Collection event ran, memory size dropped to 4 GB. But as time progresses, memory size keeps gradually increasing and never drops back to 4 GB (despite Full GC events). On the contrary, towards the right end of the graph, the Full Garbage Collection event keeps running with any memory reclamation at all. It’s a classic memory pattern to indicate that an application is suffering from memory leak.
2. CPU Spike
When memory leaks worsen, garbage collectors will start to run frequently to recover the memory space (which you can notice at the right end of the above graph). As you might be aware, Garbage Collection is a heavy CPU intensive operation. Because it must scan tons of objects in memory, their references hierarchy all the way till their GC roots to identify whether objects have active references or not. Then all the unreferenced objects must be evicted from the memory. Thus garbage collection requires a lot of CPU cycles. When memory leak worsens, garbage collection has to run continuously. In such circumstances, application’s CPU consumption will start to consume up to 100% CPU. Thus it’s a golden rule: Whenever memory leak worsens CPU consumption will rise up to 100%.
3. Degraded Response Time/Timeouts
As mentioned in the previous section, when memory leaks worsen, garbage collectors will start to run frequently to recover the memory space. Garbage Collector besides consuming a lot of CPU cycles, it also pauses your application threads. GC will not allow application threads to run, as they will keep modifying the object references in the memory and it will hinder the Garbage Collector’s functionality. When application threads are paused, no customer transactions will be processed when Garbage Collector runs.
It means if Garbage collection continuously runs, then application threads will be paused continuously. It means the overall application’s transaction response time will degrade; and transactions will start to have timeouts, which will cause the application’s health checks to fail.
4. OutOfMemoryError
When memory leak becomes very serious, no more objects can be created in the memory. At this point JVM will throw java.lang.OutOfMemoryError. There are 9 types of OutOfMemoryError. Based on the JVM memory region in which the leak is happening, an appropriate OutOfMemoryError will be thrown.
Conclusion
In this post we discussed interesting symptoms of Memory Leak. Whenever you observe such symptoms you can take corrective actions (such as decommissioning the JVM from taking traffic, capturing heap dumps,..) before the problem worsens and causes negative customer impact.