HEAP DUMP ANALYSIS API

Android, JVM Heap dump analysis doesn’t have to be done manually (painfully) anymore. You can programmatically analyze Heap dumps through our REST API. Below are the few use cases where our heap dump analysis REST APIs used by major enterprises.

Use Case 1: CI/CD pipeline

As part of continuous integration pipeline, several mature engineering organizations are executing performance tests. As part of this stress tests, they capture heap dumps from the application. Captured heap dumps are analyzed through our heap dump analysis REST API. If API identifies any memory vulnerabilities or certain values (like Object count, or wasted memory size) goes beyond a threshold, then the entire build is failed.

Use Case 2: Production root cause analysis

Whenever application experiences any memory problems or OutOfMemoryError, then heap dumps are captured from the application to analyze. Captured heap dumps can be programmatically analyzed through heap dump analysis REST API. Root cause analysis can be instantly identified.

Use Case 3: Analyzing multiple dumps instantly

Enterprises have multiple applications. It’s hard to analyze each applications heap dump manually on periodic basis, both in production and test environment. It’s both time consuming and tedious process. In such circumstances, they configure cron job that will capture heap dumps on a periodic basis and analyze heap dumps through our REST API. They configure alerts in case if heap dump thresholds drop below certain values.

How to invoke Heap Dump Analysis API?

Invoking the Heap Dump API is very simple:

  1. You will have to Register with us. We will email you the API key. This is the one-time setup process. Note: if you have purchased enterprise version with API, you don’t have to worry about registration. The API Key will be provided to you as part of installation instructions.
  2. POST HTTP request to https://api.heaphero.io/analyze-hd-api?apiKey={YOUR_API_KEY}
  3. The body of the HTTP request should contain the Heap Dump that needs to be analyzed.
  4. HTTP response will be sent back in JSON format.

CURL command

Assuming your Heap Dump file is located in ./my-heap-dump.hprof, then below is the CURL command to invoke the API:

Curl –X POST --data-binary @./my-heap-dump.hprof https://api.heaphero.io/analyze-hd-api?apiKey={YOUR_API_KEY} --header “Content-Type:text”

It cannot get any simpler than that? Isn’t it?

Compression

Heap Dump are quite large in size. For fast and efficient processing, we recommend you to compress and send the heap dump files. When you are compressing the heap dump, you need to pass ‘Content-Encoding’ element in the HTTP Header element or in the URL parameter.

Say suppose you are compressing heap dump file into ‘zip’ format, then you can invoke the API with HTTP header element

curl -X POST --data-binary @./my-heap-dump.zip "https://api.heaphero.io/analyze-hd-api?apiKey={YOUR_API_KEY}" --header "Content-Type:text" --header "Content-Encoding:zip"

or you can also invoke the API with ‘Content-Encoding’ element in the URL parameter

curl -X POST --data-binary @./my-heap-dump.zip "https://api.heaphero.io/analyze-hd-api?apiKey={API_KEY_SENT_IN_EMAIL}&Content-Encoding=zip" --header "Content-Type:text"

We support following compression formats :

zip, gz, xz, z, lzma, deflate, sz, lz4, zstd, bz2, tar

You may use the one of your choice. Whatever compression format you used for compressing the heap dump should be passed in ‘Content-Encoding’ element.

Downloading Heap Dump from remote location

Say suppose, you have stored your heap dump files in a remote location like AWS S3 bucket. In those cases, if you want to analyze your heap dump then you will have to download files to your local machine and then invoke the API with heap dump file. To make it simple and automation easier we have introduced ‘location’ parameter in our API. You can pass http(s) URL location to the ‘location’ parameter and invoke the API. When you do it, API will download file from the remote location, analyze the data and return back the response. For more details refer to this article.

curl -X POST "https://api.heaphero.io/analyze-hd-api?apiKey={YOU_API_KEY}&location={HEAP-DUMP-HTTP-URL}" --header "Content-Type:text"

 

Postman

You can also invoke API using the POSTMAN, SOAP UI,… sort of tools. Steps to invoke HeapHero API through Postman is outlined in this article.

Sample Response:

{
    "totalSize": "235.94kb",
    "objectCount": "4,324",
    "classCount": "470",
    "threadCount": 3,
    "wastedMemoryPercentage": 0,
    "largeClasses": [
        {
           "name": "String",
           "percentage": "39.35761589403973",
           "size": "92.86kb",
           "count": "1,121"
        },
        {
           "name": "char[]",
           "percentage": "15.794701986754967",
           "size": "37.27kb",
           "count": "1,135"
        },
        {

           "name": "byte[]",
           "percentage": "10.341059602649006",
           "size": "24.4kb",
           "count": "8"
       },
       {
           "name": "Object[]",
           "percentage": "10.188741721854305",
           "size": "24.04kb",
           "count": "511"
       },
       {
           "name": "java.lang.reflect.Field",
           "percentage": "2.3543046357615895",
           "size": "5.55kb",
           "count": "79"
       }
  ],
  "largeObjects": [
      {
          "percentage": 17.589403,
          "size": "41.5kb"
      },
      {
          "objectName": "java.lang.System",
          "percentage": 10.370861,
          "size": "24.47kb"
      },
      {
          "objectName": "sun.launcher.LauncherHelper",
          "percentage": 10.370861,
          "size": "24.47kb"
      },
      {
          "objectName": "java.nio.charset.Charset",
          "percentage": 8.390729,
          "size": "19.8kb"
      },
      {
          "objectName": "sun.launcher.LauncherHelper",
          "percentage": 5.8245034,
          "size": "13.74kb"
      }
  ],
  "objectsHeaderSummary": {
     "objectHeaderSize": "12b",
     "allHeadersSize": "50.67kb",
     "allHeadersPercentage": "21.5%",
     "objectsHeader": [
          {
             "className": "char[]",
             "percentage": "5.6%",
             "totalHeaderSize": "13.3kb",
             "avgObjSize": 93,
             "count": "1,135"
         },
         {
            "className": "String",
            "percentage": "5.6%",
            "totalHeaderSize": "13.14kb",
            "avgObjSize": 24,
            "count": "1,121"
         },
         {
            "className": "Object[]",
            "percentage": "2.5%",
            "totalHeaderSize": "5.99kb",
            "avgObjSize": 49,
            "count": "511"
        },
        {
            "className": "j.l.Integer",
            "percentage": "1.3%",
            "totalHeaderSize": "3kb",
            "avgObjSize": 16,
            "count": "256"
        },
        {
            "className": "j.u.Hashtable$Entry",
            "percentage": "0.6%",
            "totalHeaderSize": "1.32kb",
            "avgObjSize": 32,
            "count": "113"
        }
     ]
  },
  "duplicateStringsSummary": {
    "totalCount": "1,121",
    "uniqueCount": "1,009",
    "wastedMemorySize": "10.52kb",
    "wastedMemoryPercentage": "4.5%",
    "duplicates": [
        {
            "name": "C:\\Program Files\\Java\\jdk1.8.0_151\\jre\\lib\\ext",
            "percentage": "0.2%",
            "size": "408b",
            "count": "4"
        },
        {
            "name": "C:\\Program Files\\Java\\jdk1.8.0_151\\jre\\lib\\ext\\access-bridge-64.jar",
            "percentage": "0.1%",
            "size": "352b",
            "count": "3"
        },
        {
            "name": "C:\\Program Files\\Java\\jdk1.8.0_151\\jre\\lib\\ext\\sunjce_provider.jar",
            "percentage": "0.1%",
            "size": "352b",
            "count": "3"
        },
        {
            "name": "C:\\Program Files\\Java\\jdk1.8.0_151\\jre\\lib\\ext\\localedata.jar",
            "percentage": "0.1%",
            "size": "336b",
            "count": "3"
        },
        {
            "name": "C:\\Program Files\\Java\\jdk1.8.0_151\\jre\\lib\\ext\\cldrdata.jar",
            "percentage": "0.1%",
            "size": "320b",
            "count": "3"
        }
     ]
  },
  "duplicateObjectsSummary": {
      "totalCount": "58",
      "wastedMemorySize": "2.27kb",
      "wastedMemoryPercentage": "1.0%",
      "duplicates": [
         {
            "name": "j.l.r.SoftReference",
            "percentage": "0.8%",
            "size": "1.8kb",
            "count": "47"
         },
         {
            "name": "j.l.r.SoftReference",
            "percentage": "<0.1%",
            "size": "120b",
            "count": "4"
        },
        {
            "name": "j.l.r.SoftReference",
            "percentage": "<0.1%",
            "size": "80b",
            "count": "3"
        },
        {
            "name": "j.l.r.SoftReference",
            "percentage": "<0.1%",
            "size": "80b",
            "count": "3"
        },
        {
           "name": "j.l.r.SoftReference",
           "percentage": "<0.1%",
           "size": "40b",
           "count": "2"
        }
     ]

  },
  "duplicatePrimitveArraysSummary": {
     "totalCount": "97",
     "wastedMemorySize": "34.5kb",
     "wastedMemoryPercentage": "14.6%",
     "duplicates": [
        {
           "name": "byte[8192]",
           "percentage": "6.8%",
           "size": "16.03kb",
           "count": "3"
        },
        {
           "name": "char[8192]",
           "percentage": "6.8%",
           "size": "16.02kb",
           "count": "2"
       },
       {
           "name": "int[0]",
           "percentage": "0.5%",
           "size": "1.25kb",
           "count": "81"
       },
       {
           "name": "short[32]",
           "percentage": "0.2%",
           "size": "560b",
           "count": "8"
       },
       {
           "name": "int[32]",
           "percentage": "0.2%",
           "size": "432b",
           "count": "4"
       }
     ]
  },
  "inefficientCollectionsSummary": {
     "total": "50",
     "inefficientCount": "39",
     "wastedMemorySize": "3.89kb",
     "wastedMemoryPercentage": "1.7%",
     "inefficientEntities": [
          {
             "problem": "29% of j.u.HashMap contains 1 element only",
             "wastedMemorySize": "800b",
             "wastedMemoryPercentage": "0.3%"
         },
         {
             "problem": "11% of j.u.HashMap contains 2 - 4 elements only",
             "wastedMemorySize": "360b",
             "wastedMemoryPercentage": "0.1%"
         },
         {
             "problem": "66% of j.u.WeakHashMap contains no elements",
             "wastedMemorySize": "352b",
             "wastedMemoryPercentage": "0.1%"
         },
         {
             "problem": "35% of j.u.HashMap contains no elements",
             "wastedMemorySize": "288b",
             "wastedMemoryPercentage": "0.1%"
         },
         {
             "problem": "50% of j.u.Vector contains no elements",
             "wastedMemorySize": "264b",
             "wastedMemoryPercentage": "0.1%"
         }
      ]
  },
  "inefficientObjectArraysSummary": {
      "total": "648",
      "inefficientCount": "216",
      "wastedMemorySize": "13.14kb",
      "wastedMemoryPercentage": "5.6%",
      "inefficientEntities": [
          {
              "problem": "13% of Object[] contains no elements",
              "wastedMemorySize": "4.18kb",
              "wastedMemoryPercentage": "1.8%"
          },
          {
              "problem": "2% of Object[] contains half empty elements",
              "wastedMemorySize": "4.07kb",
              "wastedMemoryPercentage": "1.7%"
          },
          {
              "problem": "10% of Object[] contains 1 element only",
              "wastedMemorySize": "1.91kb",
              "wastedMemoryPercentage": "0.8%"
          },
          {
              "problem": "100% of java.lang.invoke.MethodHandle[] contains no elements",
              "wastedMemorySize": "1.04kb",
              "wastedMemoryPercentage": "0.4%"
          },
          {
              "problem": "7% of Object[] declared with 1 length",
              "wastedMemorySize": "936b",
              "wastedMemoryPercentage": "0.4%"
          }
      ]
  },
  "inefficientPrimitiveArraysSummary": {
      "total": "1,239",
      "inefficientCount": "105",
      "wastedMemorySize": "59.03kb",
      "wastedMemoryPercentage": "25.0%",
      "inefficientEntities": [
           {
              "problem": "< 0.1%of char[] contains no elements",
              "wastedMemorySize": "32.03kb",
              "wastedMemoryPercentage": "13.6%"
           },
           {
              "problem": "62%of byte[] contains no elements",
              "wastedMemorySize": "24.33kb",
              "wastedMemoryPercentage": "10.3%"
           },
           {
              "problem": "93%of int[] declared with 0 length",
              "wastedMemorySize": "1.27kb",
              "wastedMemoryPercentage": "0.5%"
           },
           {
              "problem": "100%of short[] contains no elements",
              "wastedMemorySize": "640b",
              "wastedMemoryPercentage": "0.3%"
           },
           {
              "problem": "4%of int[] contains no elements",
              "wastedMemorySize": "576b",
              "wastedMemoryPercentage": "0.2%"
           }
       ]
  },
  "boxedNumbersSummary": {
        "totalBoxedObjects": "258",
        "wastedMemorySize": "4.04kb",
        "wastedMemoryPercentage": "1.7%",
        "boxedNumbers": [
            {
               "name": "j.l.Integer",
               "percentage": "1.7%",
               "size": "4kb",
               "count": "256"
            },
            {
               "name": "j.l.Boolean",
               "percentage": "<0.1%",
               "size": "38b",
               "count": "2"
           },
           {
               "name": "j.l.Long",
               "percentage": "0.0%",
               "size": "n/a",
               "count": "0"
           },
           {
               "name": "j.l.Short",
               "percentage": "0.0%",
               "size": "n/a",
               "count": "0"
           },
           {
               "name": "j.l.Byte",
               "percentage": "0.0%",
               "size": "n/a",
               "count": "0"
          }
      ]
  },
  "objectsWaitingFinalization": {
      "percentage": "<0.1%",
      "size": "48b"
  },
  "heapSizeRecommendation": "No major recommendations."
  "GraphURL":"http://150.100.100.67:8080/tier1app/heap-report2_0.jsp"
} 

JSON Response Elements

countStringNumber of duplicate strings which is of this particular type

ElementsData TypeDescription
totalSizeStringSize of the heap when heap dump was captured. It will be specified in kb, mb, gb, tb
objectCountStringTotal number of objects that are present in the heap
classCountStringTotal number of classes that are present in the heap
threadCountStringTotal number of threads that are present in the application
wastedMemorySizeStringAmount of memory wasted due to inefficient programming practises. It will be specified in kb, mb, gb, tb
wastedMemoryPercentageStringPercentage of memory wasted due to inefficient programming practises.
largeClassesArrayList of classes who are occupying large amount of memory
> nameStringName of the class
> percentageStringPercentage of the memory occupied by this class
> sizeStringSize of memory occupied by objects of this type class
> countStringNumber of object instances of this type of class
largeObjectsArrayList of large size objects occupying the memory
> objectNameStringName of the object
> percentageStringPercentage of memory occupied by this type of object
> sizeStringSize of the memory occupied by the object
> problemStringIf this object is causing memory leak in the application, it will be reported in this element.
objectsHeaderSummaryObject 
> objectHeaderSizeStringObject Header size
> allHeadersSizeStringTotal of memory occupied by all object headers
> allHeadersPercentageStringTotal percentage of memory occupied by all object headers
> objectsHeaderArrayList of the top object headers that are present in the application.
>> classNameStringValue of this particular class type
>> percentageStringPercentage of this particular class type
>> totalHeaderSizeStringSize of this particular class type. It will be specified in kb, mb, gb, tb
>> avgObjSizeString 
>> countString 
duplicateStringsSummaryObject 
> totalCountStringTotal number of duplicate strings in the application
> uniqueCountStringNumber of unique strings in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of duplicate strings. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of duplicate strings.
> duplicatesArrayDetailed list of duplicate strings in the application
>> nameStringValue of this particular string type
>> percentageStringPercentage of memory wasted because of this particular string type
>> sizeStringAmount of memory wasted because of this particular string type. It will be specified in kb, mb, gb, tb
duplicateObjectsSummaryObject 
> totalCountStringTotal number of duplicate objects in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of duplicate objects. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of duplicate objects.
> duplicatesArrayDetailed list of duplicate objects in the application
>> nameStringValue of this particular object type
>> percentageStringPercentage of memory wasted because of this particular object type.
>> sizeStringAmount of memory wasted because of this particular object type. It will be specified in kb, mb, gb, tb
>> countStringNumber of duplicate objects which is of this particular type
duplicatePrimitveArraysSummaryObject 
> totalCountStringTotal number of duplicate primitive arrays in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of primitive arrays. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of duplicate primitive arrays.
> duplicatesArrayDetailed list of duplicate primitive arrays in the application
>> nameStringValue of this particular primitive array type
>> percentageStringPercentage of memory wasted because of this particular primitive array type
>> sizeStringAmount of memory wasted because of this particular primitive array type. It will be specified in kb, mb, gb, tb
>> countStringNumber of duplicate primitive array which is of this particular type
inefficientCollectionsSummaryObject 
> totalStringTotal number of collection objects present in the application
> inefficientCountStringTotal number of collection objects that are inefficiently implemented in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of inefficient collections. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of inefficient collections.
> inefficientEntitiesArrayDetailed list of inefficient collection implementations in the application
>> descriptionStringProblem description on the inefficient collection implementation. You can learn about the different types of inefficient collection implementation from here: https://blog.heaphero.io/2018/04/13/heaphero-user-manual-2/#Ineff
>> wastedMemorySizeStringAmount of memory wasted in the application because of this particular type of inefficient collection. It will be specified in kb, mb, gb, tb.
>> wastedMemoryPercentageStringPercentage of memory wasted in the application because of this particular type of inefficient collection.
inefficientObjectArraysSummaryObject 
> totalStringTotal number of collection object arrays present in the application
> inefficientCountStringTotal number of collection object arrays that are inefficiently implemented in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of inefficient object arrays. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of inefficient object arrays.
> inefficientEntitiesArrayDetailed list of inefficient object arrays implementations in the application
>> descriptionStringProblem description on the inefficient object arrays implementation. You can learn about the different types of inefficient arrays implementation from here: https://blog.heaphero.io/2018/04/13/heaphero-user-manual-2/#IneffObj
>> wastedMemorySizeStringAmount of memory wasted in the application because of this particular type of inefficient object arrays. It will be specified in kb, mb, gb, tb.
>> wastedMemoryPercentageStringPercentage of memory wasted in the application because of this particular type of inefficient object arrays.
inefficientPrimitiveArraysSummaryObject 
> totalStringTotal number of collection primitive arrays present in the application
> inefficientCountStringTotal number of collection primitive arrays that are inefficiently implemented in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of inefficient primitive arrays. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of inefficient primitive arrays.
> inefficientEntitiesArrayDetailed list of inefficient primitive arrays implementations in the application
>> descriptionStringProblem description on the inefficient primitive arrays implementation. You can learn about the different types of inefficient primitive arrays implementation from here: https://blog.heaphero.io/2018/04/13/heaphero-user-manual-2/#Ineffprim
>> wastedMemorySizeStringAmount of memory wasted in the application because of this particular type of inefficient primitive arrays. It will be specified in kb, mb, gb, tb.
>> wastedMemoryPercentageStringPercentage of memory wasted in the application because of this particular type of inefficient primitive arrays.
boxedNumbersSummaryObject 
> totalBoxedObjectsStringTotal number of boxed numbers present in the application
> wastedMemorySizeStringAmount of memory wasted in the application because of boxed numbers. It will be specified in kb, mb, gb, tb.
> wastedMemoryPercentageStringPercentage of memory wasted in the application because of boxed number.
> boxedNumbersArrayDetailed list of boxed number implementations in the application
>> nameStringValue of this particular boxed number type
>> percentageStringPercentage of memory wasted because of this particular boxed number type
>> sizeStringAmount of memory wasted because of this particular boxed number type. It will be specified in kb, mb, gb, tb
>> countStringNumber of boxed numbers which is of this particular type
objectsWaitingFinalizationObject 
> percentageStringPercentage of objects waiting for the finalization
> sizeStringSize of objects waiting for the finalization
heapSizeRecommendationStringShows recommendations. 

Tagging the report

By default API response in sent in JSON format. However if you would like to get response in XML format, you can pass ‘accept’ HTTP header element with value ‘xml’

Curl –X POST --data-binary @./my-heap-dump.hprof https://api.heaphero.io/analyze-hd-api?apiKey={YOUR_API_KEY} –-header "accept:xml"

or you can also invoke the API with ‘accept’ element in the URL parameter

Curl –X POST --data-binary @./my-heap-dump.hprof https://api.heaphero.io/analyze-hd-api?apiKey={YOUR_API_KEY}&accept=xml

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: