OutOfMemoryError Unable to create new native threads in Jenkins: Root Causes, Diagnostics & Production Fixes

Jenkins – popular CI/CD pipeline is used for several critical operations in the organization such as building applications, conducting automated tests, deployments in pre-prod and prod environments, … If Jenkins is down, engineers’ productivity will be severely hampered. Thus, extra care is given to major organizations to keep them up 24 x 7. 

While Java has 9 types of OutOfMemoryError, Jenkins is susceptible to 8 of them. In this blog series, we systematically walk through each of those 8 types, helping you identify, diagnose, and fix them. This post covers one of them.

Occasionally it can experience java.lang.OutOfMemoryError: Unable to create new native threads, which would disrupt entire Jenkins availability. In this post let’s discuss what does ‘java.lang.OutOfMemoryError: Unable to create new native threads’ mean, how to isolate the root cause of it quickly, what are its temporary and permanent fixes, and even better, how to prevent them from happening.

Immediate Stabilization Steps – OutOfMemoryError Unable to create new native threads in Jenkins 

When Jenkins experience ‘java.lang.OutOfMemoryError: Unable to create new native threads’ here are the options one can take to stabilize the Jenkins immediately (basically first-aid):

1. Restart the JVM: When ‘java.lang.OutOfMemoryError: Unable to create new native threads’ happens in Jenkins, it will put JVM into an unstable state. It’s dangerous to run Jenkins in this setting, as it can result in erroneous behavior. Thus, it’s highly recommended to restart the JVM, so that it will come back in a clean slate.

2. Terminate Unnecessary processes: This error happens when there is insufficient RAM capacity in the container/device in which Jenkins is running. Sometimes new cronjob or home-grown script will crop up & they will consume memory, which will leave less room for the Jenkins application to run. Terminate those newly cropped up processes, if they aren’t critical.

Why OutOfMemoryError Unable to create new native threads Happens in Jenkins?

To better understand OutOfMemoryError Unable to create new native threads, we first need to understand different JVM Memory regions. Here is a video clip that gives a good introduction about different JVM memory regions. But in nutshell, JVM has following memory regions:

Fig: JVM Memory Regions

1. Young Generation: Newly created application objects are stored in this region.
2. Old Generation: Application objects that are living for longer duration are promoted from the Young Generation to the Old Generation. Basically this region holds long lived objects.
3. Metaspace: Class definitions, method definitions and other metadata that are required to execute your program are stored in the Metaspace region. This region was added in Java 8. Before that metadata definitions were stored in the PermGen. Since Java 8, PermGen was replaced by Metaspace.
4. Threads: Each application thread requires a thread stack. Space allocated for thread stacks, which contain method call information and local variables are stored in this region.
5. Code Cache: Memory areas where compiled native code (machine code) of methods is stored for efficient execution are stored in this region.
6. Direct Buffer: ByteBuffer objects are used by modern framework (i.e. Spring WebClient) for efficient I/O operations. They are stored in this region.
7. GC (Garbage Collection): Memory required for automatic garbage collection to work is stored in this region. 
8. JNI (Java Native Interface): Memory for interacting with native libraries and code written in other languages are stored in this region.
9. misc: There are areas specific to certain JVM implementations or configurations, such as the internal JVM structures or reserved memory spaces, they are classified as ‘misc’ regions.

Fig: ‘java.lang.OutOfMemoryError: Unable to create new native threads’

java.lang.OutOfMemoryError: Unable to create new native threads occurs in Jenkins when the JVM attempts to create more threads than the available RAM capacity in the native memory region can support. In a Jenkins environment, this pressure builds quickly, parallel build executors, concurrent pipeline stages, agent connections, and plugin activity all spin up threads simultaneously. When the cumulative thread count pushes beyond what the OS can allocate in native memory, Jenkins hits this limit and the JVM throws this error. 

This video covers OutOfMemoryError: Unable to create new native threads from a Java perspective, the same underlying concept that applies to Jenkins.

Root Causes of OutOfMemoryError Unable to create new native threads in Jenkins 

‘java.lang.OutOfMemoryError: Unable to create new native threads’ in Jenkins is potentially caused because of the following reasons:

1. Thread Leak due to Buggy Code: Due to the bug in the Jenkins code or Plugin, application can inadvertently create a lot of new threads, it can lead to buildup of unused threads in memory, eventually exhausting the available native memory, resulting in OutOfMemoryError.
2. Lack of RAM capacity: When there is a lack of RAM capacity in the container/device in which the Jenkins is running. 
3. More processes in Memory: When other processes are running on the container/device, it leaves less room for the threads to be created in the native memory. 
4. Kernel Limit: By default, Kernel sets a limit on the number of threads each process can create. When the Jenkins application creates more threads than the allowed kernel limit.

How to Diagnose the OutOfMemoryError Unable to create new native threads Problem in Jenkins (Step-by-Step)

In order to troubleshoot this problem and other Jenkins Production problems, you can leverage the yCrash monitoring tool. This tool is capable of predicting outages before it surfaces in the production environment. Once it predicts outage in the environment, it captures 360° troubleshooting artifacts from your environment, analyses them and instantly generates a root cause analysis report. Artifacts it captures include Garbage Collection log, Thread Dump, Heap Substitute, netstat, vmstat, iostat, top, top -H, dmesg, kernel parameters, disk usage…. 

You can register here and start using the free-tier of this tool.

Below is the snapshots from report generated by the yCrash tool reporting the thread leaking issue:

Fig:  yCrash reporting 1,200+ are created and they can cause ‘OutOfMemoryError: unable to create new native thread’

Fig:  yCrash reporting the line of code in which 1,200+ threads are stuck

From the report, you can notice that yCrash points out that 1,200+ threads are created, and they have the potential to cause ‘OutOfMemoryError: unable to create new native thread’ problem. Besides the thread count, the tool is also reporting the line of code, i.e. ‘com.ycrash.springboot.buggy.app.service.threadleak.ForeverThread.run(ForeverThread.java:12)‘ in which all the 1,200 threads are stuck. Equipped with this information, one can quickly identify the problematic code.

Solutions for OutOfMemoryError Unable to create new native threads in Jenkins 

Here are the potential solutions to address java.lang.OutOfMemoryError: Unable to create new native threads in Jenkins:

1. Identify & Fix the Memory Leak in Jenkins: Using the diagnostic steps described in the above section find the leaking objects in the memory and fix it.

2. Remove the recently added Plugins: Whenever you add new plugins, it will occupy space in the Metaspace. Sometimes you might end up adding poorly implemented, memory inefficient plugins. Remove the recently added plugins and restart the JVM and see whether Jenkins stabilizes. 

3. Revert to Previous Jenkins Installation: If you have recently upgraded to latest version of Jenkins installation and Unable to create new native threads OutOfMemoryError started to surface after it, consider reverting to previous Jenkins installation. 

4. Fix Thread Leak: Analyze the thread dump of your application and identify the leaking threads. Instrument fix to ensure that threads are properly terminated after it completed executing its tasks.
   
5. Increase RAM capacity: Try to run your application on a container/device which has larger RAM capacity. 
   
6. Reduce other processes: Terminate (or move) other processes that are running on the container/device, so that there is more room for the java application to create new threads.
   
7. Reduce thread stack size: When you reduce the thread’s stack size (by using -Xss JVM argument), your application can create a number of threads within the same amount of memory. However, be cautious when you pursue this option, as reducing thread stack size can result in StackOverflowError.
   
8. Change Kernel setting per process thread limit: By default, Kernel sets a limit on the number of threads each process can create. If OutOfMemoryError is happening because of this limit, then you can increase this limit by using ‘limit -u’ command.

How to Prevent OutOfMemoryError Unable to create new native threads in Jenkins

Before you upgrade to new release of Jenkins or install a new Jenkins plugin in the production environment, you might be studying following key metrics in your performance lab:

• CPU Utilization
• Memory Utilization
• Response Time of key transactions

These are wonderful metrics that highlight the performance characteristics of the new release. However, several performance problems slowly build over the period of time, for example for most applications OutOfMemoryError happens only if it runs for more than 1 week. In the performance lab, we don’t run such long endurance tests. 

Above mentioned metrics are more reactive indicators that don’t indicate the silently lurking problem in the environment. We recommend studying below mentioned Micro-metrics along with above reactive indicators in the performance lab and certify the release. These Micro-Metrics are good at predicting/forecasting performance problems even if they act at acute scale.

• GC Behavior Pattern: Detects memory leaks, poor GC configuration, or excessive object promotion causing GC pauses.
• Object Creation Rate: Identifies allocation surges that can trigger frequent GCs or memory pressure.
• GC Throughput: Highlights apps spending too much time in GC instead of work—can lead to CPU spikes or slowdowns.
• GC Pause Time: Surfaces stop-the-world GC events affecting responsiveness or causing thread backlogs.
• Thread Patterns: Flags CPU spikes, thread starvation, bursty load, and thread buildup from backend slowness.
• Thread States: Detects BLOCKED, DEADLOCKED, or WAITING threads due to DB chattiness, config limits, or locking.
• Thread Pool Behavior: Identifies thread exhaustion, request rejections, or poor pooling thresholds in backend services.
• TCP/IP Connection Count & States: Catches backend connection leaks, TIME_WAIT surges, or slow/unresponsive downstream services.
• Error Trends in Application Logs: Detects hidden runtime errors, JDBC leaks, logging misconfigurations, or disk issues.

yCrash tool facilitates you reporting these Micro-Metrics which will unearth several performance problems well in advance, before they silently surface in production. You can find the details on how to source and study these Micro-Metrics through yCrash from here. 

Business Impact & ROI

Isolating and fixing OutOfMemoryError in Jenkins will have considerable business impact to your organization:

1. Engineering Time Savings: yCrash dramatically reduces the time engineers spend analyzing Heap Dumps and pinpointing root causes in complex, multi-threaded applications.
   • Suppose your organization is analyzing 10 incidents per month & each analysis traditionally will take around 40 hours. 
   • With a Performance Engineer’s hourly rate at USD $100, yCrash can save approximately $480,000 annually (10 incidents x 40 hours/dump x $100/hour x 12 months) by automating root cause analysis and reducing troubleshooting time.

2. Rapid Deployments & Increased Productivity: yCrash minimizes prolonged downtime of Jenkins that can lead to delayed deployment, degeneration of engineers productivity and reputational damage of the organization. By quickly diagnosing issues, yCrash helps to prevent such large-scale impacts, protecting revenue and brand reputation.

3. Protection from Escalated Operational Consequences: Certain Jenkins outages can have severe repercussions, including escalated consequences like organizational changes or job losses. yCrash’s rapid problem isolation capabilities prevent such disruptions, allowing teams to resolve issues before they escalate to crisis levels. By maintaining operational continuity and team stability, yCrash supports a steady, resilient organizational environment and protects against the high stakes impacts that can result from unmanaged production outages.

Conclusion

Jenkins is the backbone of your organization’s CI/CD pipeline. Keeping it stable is not optional, it’s rather essential. Even though ‘java.lang.OutOfMemoryError: Unable to create new native threads’ is not a common error, it can hurt your entire Jenkins platform availability, when it surfaces. Hopefully this post has given you enough light on how to troubleshoot this problem effectively & efficiently.