Friday, December 27, 2013

10 Tips for using the Eclipse Memory Analyzer

Analyzing and understanding the memory use of an application is challenging. A subtle logic error can result in listeners never being diposed, ultimately leading to the dreaded OutOfMemory error. Even if your application is properly disposing of all unused objects, you application may still be requiring 10 or 100 times more memory than necessary.
Lucky for us, the Eclipse Memory Analyzer (MAT) can help provide details of an application’s memory use. The tool is useful for both tracking memory leaks and for periodically reviewing the state of your system. In this tutorial I’ll outline 10 tips to help you use the MAT more effectively. If you’re a Java developer, the Eclipse Memory Analyzer Tool  should certainly be in your debugging toolbox.
The Memory Analyzer can be installed using the Install New Software dialog or through the Eclipse MarketPlace Client.
In this example, we are using a very simple program that allocates 100,000 Listeners, and stores them in 4 different lists. The application then sleeps without removing or clearing these lists.

1. Acquiring a memory snapshot (Heap Dump)

There are several ways you can get started with MAT. You can:
  1. Configure an application to dump the contents of its memory when an out of memory error occurs,
  2. Connect the MAT to an existing java process, or
  3. Manually acquire a heap dump and load it into the MAT
In all cases it’s important to keep in mind that this is a snapshot of memory at a point in time. The MAT cannot tell you why an object was created, nor can it show you objects that have already been garbage collected. However, if you use the MAT alongside other debugging tools and techniques, you can often conquer memory leaks pretty quickly.
To configure your application to produce a heap dump whenever an OutOfMemory Error is thrown, add the following vm argument:
Alternatively, you can use jstack to acquire a Heap dump from a currently running java process.
jmap -dump:file=heap.bin 
Finally, you can use the MAT’s Acquire Heap Dump action to choose an existing Java process on your local machine.
Screen Shot 2013 01 17 at 12.08.57 PM 10 Tips for using the Eclipse Memory Analyzer
When you load a Heap dump for the first time, the MAT will index it. This make take a few minutes, but the results will be persisted so subsequent loads will be quick.

 2. Understanding the Histogram

When you first acquire your heap dump, the MAT will show you an overview of the applications memory use.
Screen Shot 2013 01 17 at 12.57.28 PM 10 Tips for using the Eclipse Memory Analyzer
The pie chart in the middle shows you the biggest objects by retained size. That means if we could dispose a particular instance of java.lang.Thread we would save 11.2Mb, and over 90% of the memory used in this application. While that might look interesting, java.lang.Thread is unlikely the real problem here. To get a better sense of what objects currently exist, you can use the Histogram.
Screen Shot 2013 01 17 at 12.59.18 PM 10 Tips for using the Eclipse Memory Analyzer
The histogram shows the number of instances of a particular class and how much memory each one uses. Of course, char[]String andObject[] are unlikely the problem. To help better organize this view, you can group by classloader or package. This will allow you to focus on your Objects.
Screen Shot 2013 01 17 at 12.27.34 PM 10 Tips for using the Eclipse Memory Analyzer
The histogram can also be filtered using a regular expression. For example, we can show only classes that match the pattern com.example.mat.*
Screen Shot 2013 01 17 at 1.01.48 PM 10 Tips for using the Eclipse Memory Analyzer
With this view, we can now start to see that 100,000Listener Objects are alive in the system. We can also see the amount of memory each Object is using. There are two calculations, Shallow Heap and Retained Heap.  Shallow heap is the amount of memory consumed by one object. An Object requires 32 (or 64 bits, depending on the architecture) for each reference. Primitives such as integers and longs require 4 or 8 bytes, etc… While this can be interesting, the more useful metric is the Retained Heap.

3. Understanding the Retained Heap

The retained heap shows the sum of the shallow heap size of all objects that would be removed when this object is garbage collected. For example, if an ArrayList held 100,000 items, and each item required 16 bytes, then removing the ArrayList would free 16 x 100,000 + X, where X is the shallow size of the ArrayList. (Note: this assumes that these objects are only being referenced by theArrayList, and not elsewhere).
The retained heap is computed by adding the size all the objects in the retained set. retained set of X is the set of objects which would be removed by the GC when X is collected.
The retained heap can be calculated in two different ways, using the quick approximation or the precise retained size.
Screen Shot 2013 01 17 at 12.38.46 PM 10 Tips for using the Eclipse Memory Analyzer
Screen Shot 2013 01 17 at 1.06.24 PM 10 Tips for using the Eclipse Memory Analyzer
By calculating the Retained Heap we can now see that com.example.mat.Controller is holding the majority of the memory, even though it’s only 24 bytes itself. By finding a way to free up the Controller, we can certainly get our memory problem under control.

4. The Dominator Tree

The key to understanding your retained heap, is looking at thedominator tree. The dominator tree is a tree produced by the complex object graph in your system. The dominator tree allows you to identify the largest memory graphs. An Object X is said to dominate an Object Y if every path from the Root to Y must pass through X. Looking at the dominator tree for our example, we can start to see where the bulk of our memory is leaking.
Screen Shot 2013 01 17 at 1.22.02 PM 10 Tips for using the Eclipse Memory Analyzer
By looking at the dominator tree, we can easily see that it’s not thejava.lang.Thread that’s the problem, but rather the Controller and the Allocator that hold the memory. All 100,000 Listeners are being retained by the Controller. By either removing freeing these objects, or freeing the lists that they contain, we can likely improve our situation. There are a few useful properties of the dominator tree:
  • All Objects that belongs to a subtree of X (com.example.mat.Controller in this case) are said to be in the retained set of X
  • If X is the immediate dominator of Y (Controller is the immediate dominator of the Allocator), then the immediate dominator of X (which is java.lang.Thread in our example) also dominates Y.
  • The parent child relationship in the tree do not necessarily correspond to the relationships in the Object graph
From the Histogram you can also choose a particular class and find all the objects that dominate the instances of this class.
Screen Shot 2013 01 17 at 1.32.47 PM1 10 Tips for using the Eclipse Memory Analyzer

5. Exploring Paths to the GC Roots

Sometimes you have a large collection of Objects that you’re certain your disposing of. Finding the the dominator may help, but often you want the exact path from that Object to the roots. For example, if I now properly dispose of my Controller, then certainly my memory problem should go away,  unfortunately they didn’t.  If I now choose an instance of the Listener, and look at Paths To GC Roots, I can see that the Controller class (Note: the class, not an Object) reference to a list of Listeners. This is because one of the list was declared static.
Screen Shot 2013 01 17 at 1.54.55 PM 10 Tips for using the Eclipse Memory Analyzer
You can also explore all the incoming and outgoing references to an Object. This is very useful if you want to see all the references to a particular Object in the Object graph.

6. The ‘Inspector’

The Inspector provides detailed information about the currently selected Class or Object. In this example we can see that the currently selected ArrayList contains 100,000 elements and references an object array at memory location 0x7f354ea68.
Screen Shot 2013 01 17 at 1.37.53 PM 10 Tips for using the Eclipse Memory Analyzer
Keeping the Inspector and the Snapshot linked will provide you with important statistics about all your selections.

7. Common Memory Anti-Patterns

The MAT provides reports for common memory use anti-patterns. These can be used to get an idea of where memory leaks are occurring, or by looking for some low hanging fruit which can be cleaned up to help improve performance.
Screen Shot 2013 01 17 at 2.13.51 PM 10 Tips for using the Eclipse Memory AnalyzerThe Heap Dump Overview will show you detailed information about your Heap Dump and provide links to common tools (like the Histogram). Information such as the threads which were running, total number of Objects in the system, size of the heap, and are also shown.
Screen Shot 2013 01 17 at 2.16.30 PM 10 Tips for using the Eclipse Memory Analyzer
The Leak Suspects report displays possible memory leaks, and provides links to the tools and charts to analyze these findings.
Screen Shot 2013 01 17 at 2.18.29 PM 10 Tips for using the Eclipse Memory AnalyzerAnother common anti-pattern is the use of a large number of collections, with very few entries in each one. For example, if our listeners each had an array of notifiers (items that need to be notified of certain events), but these notifiers were only used occasionally, we would end up wasting a lot of space. The Java Collections tools can help with these problems.
Screen Shot 2013 01 17 at 2.27.19 PM 10 Tips for using the Eclipse Memory Analyzer
Screen Shot 2013 01 17 at 2.31.25 PM 10 Tips for using the Eclipse Memory Analyzer
Screen Shot 2013 01 17 at 2.28.41 PM 10 Tips for using the Eclipse Memory Analyzer
By running a Collection -> Fill Ratio Report we can see that there are 100,000 ArrayLists that are empty. If were allocated these in a lazy manner (only when needed), we would save almost 8Mb.
We can also use Collection Analysis to see array fill ratios,collection size statistics and map collision ratios.

8. Java Tools

The MAT has a number of built in tools to generate reports tailored to the details of the Java runtime. For example, the Threads and Stacks report will show details about all the treads in the system. You can see the local variables which are currently kept alive on each stack.
Screen Shot 2013 01 17 at 3.26.48 PM 10 Tips for using the Eclipse Memory Analyzer
You can Find all the Strings in the system that match a particular pattern:
Screen Shot 2013 01 17 at 3.30.49 PM1 10 Tips for using the Eclipse Memory Analyzer
Or even find Strings in the system which contain wasted space in their character arrays (often due to repeated use of substring).
Screen Shot 2013 01 17 at 4.18.14 PM 10 Tips for using the Eclipse Memory Analyzer

9. Object Query Language

As we’ve shown, the Eclipse Memory Analyzer has a lot of tools to help track both memory leaks and excessive memory use. While most memory problems can likely be addressed using the techniques described above, a Heap Dump contains much more information. The Object Query Language  (OQL) allows you to build your own reports based on the results of a Heap Dump.
The OQL is an SQL-like language. Just think of Classes as tables,Objects as rows and Fields as columns. For example. to show all Objects of type com.example.mat.Listener, you would simply write:
select * from com.example.mat.Listener
Screen Shot 2013 01 17 at 4.23.57 PM 10 Tips for using the Eclipse Memory Analyzer
Columns can be configured using different fields, such as:
SELECT toString(l.message), l.message.count FROM com.example.mat.Listener l
Screen Shot 2013 01 17 at 4.28.07 PM 10 Tips for using the Eclipse Memory AnalyzerAnd finally, the WHERE clause can be used to specify particular criteria, such as all the Strings in the system which are not of the format “message:.*”
SELECT toString(s), s.count FROM java.lang.String s WHERE (toString(s) NOT LIKE "message.*")

Screen Shot 2013 01 17 at 4.34.53 PM 10 Tips for using the Eclipse Memory Analyzer

10. Exporting your results

The memory analyzer is a great tool for creating reports about the state of an application’s memory. A Heap Dump contains valuable information about the state of your system, and the MAT provides the tools needed to access this data. However, like with many open tools, if something is missing you’re not locked in, nor are you out of luck. With the MAT you can export your results to several different formats including HTML, CSV or even Plain Text. You can then use your favorite spreadsheet program (or your own tool) to continue your analysis.
Screen Shot 2013 01 17 at 4.39.34 PM 10 Tips for using the Eclipse Memory Analyzer
The Eclipse Memory Analyzer is a powerful tool, one all Java Developers should be familiar with. Tracking memory leaks and other memory related problems is often challenging, but hopefully with the MAT you can get to the root of your problems relatively quickly.


oakleyses said...

louis vuitton handbags, oakley sunglasses, louboutin, longchamp outlet, nike shoes, louis vuitton outlet stores, chanel handbags, burberry outlet, prada outlet, jordan shoes, tiffany and co, michael kors outlet, tory burch outlet, louis vuitton outlet, longchamp handbags, nike free, true religion jeans, michael kors outlet, kate spade outlet, polo ralph lauren outlet, tiffany and co, prada handbags, polo ralph lauren outlet, michael kors outlet, michael kors outlet, longchamp handbags, oakley sunglasses, ray ban sunglasses, kate spade handbags, burberry outlet, louis vuitton outlet, louboutin outlet, louboutin, coach factory outlet, air max, air max, coach outlet, gucci outlet, christian louboutin shoes, michael kors outlet, coach purses, ray ban sunglasses, michael kors outlet, louis vuitton, coach outlet store online, true religion jeans, oakley sunglasses cheap

oakleyses said...

ralph lauren, lululemon, air max, hollister, north face, nike air max, polo lacoste, vanessa bruno, timberland, vans pas cher, louboutin, louis vuitton, oakley pas cher, air max pas cher, nike roshe run, air max, true religion outlet, barbour, sac longchamp, air force, hollister, sac louis vuitton, nike free, polo ralph lauren, nike trainers, louis vuitton uk, nike roshe, sac hermes, longchamp, michael kors, sac burberry, sac guess, mulberry, new balance pas cher, converse pas cher, sac louis vuitton, hogan outlet, nike tn, north face, true religion outlet, ray ban pas cher, michael kors, air jordan, nike blazer, nike free pas cher, michael kors pas cher, abercrombie and fitch, ray ban sunglasses

oakleyses said...

mac cosmetics, mont blanc, marc jacobs, canada goose outlet, nike huarache, vans shoes, soccer jerseys, hollister, giuseppe zanotti, beats by dre, abercrombie and fitch, longchamp, insanity workout, celine handbags, bottega veneta, ghd, nfl jerseys, north face outlet, chi flat iron, ugg boots, birkin bag, ugg australia, canada goose, herve leger, ugg pas cher, rolex watches, valentino shoes, canada goose uk, canada goose, ferragamo shoes, canada goose, ugg boots, uggs outlet, north face jackets, soccer shoes, asics running shoes, new balance shoes, p90x, lululemon outlet, canada goose jackets, mcm handbags, instyler, babyliss pro, ugg, wedding dresses, jimmy choo outlet, reebok outlet, nike roshe run

oakleyses said...

parajumpers, karen millen, air max, converse, pandora charms, moncler, louboutin, moncler, links of london, lancel, juicy couture outlet, oakley, hollister, pandora charms, supra shoes, thomas sabo, canada goose, gucci, wedding dresses, timberland boots, swarovski crystal, air max, coach outlet store online, moncler, ray ban, canada goose, moncler, ugg, louis vuitton, swarovski, hollister, montre homme, moncler, hollister clothing store, ralph lauren, rolex watches, moncler outlet, moncler, iphone 6 cases, baseball bats, juicy couture outlet, toms shoes, vans, pandora jewelry, ugg, converse shoes

Anna said...

Great and Useful Article.

Online Java Course

Java Online Training

Java Course Online

Best Recommended books for Spring framework

Java Interview Questions

Java Training Institutes in Chennai

Java Training in Chennai

J2EE Training in Chennai

java j2ee training institutes in chennai

Java Course in Chennai

Zheng junxai5 said...

jordan 3 powder blue
coach outlet store online clearances
toms wedges
louis vuitton purses
coach outlet
cheap jordans
retro 11
oakley outlet
nike free 5.0
coach factory outlet
louis vuitton purses
gucci handbags
louis vuitton outlet online
adidas superstar trainers
louis vuitton handbags
coach outlet online
michael kors outlet
louis vuitton outlet
ralph lauren sale
michael kors purses
michael kors outlet
cheap toms shoes
nfl jerseys
cheap toms
ray ban sunglasses uk
louboutin femme
michael kors handbags
coach outlet store online
coach factory outlet
louis vuitton purses
nike sb dunks
nike blazers uk
coach outlet store
nike air max uk
nike uk
polo shirts
gucci bags
adidas running shoes
true religion outlet online
michael kors handbags

raybanoutlet001 said...

adidas stan smith uk
jordan retro
michael kors handbags
tiffany and co
kobe basketball shoes
ray ban sunglasses
michael kors outlet store
fitflops clearance
cheap rolex watches
fitflops outlet
adidas tubular UK
cheap nfl jerseys
yeezy boost 350
jordans for cheap
tiffany and co jewellery
cheap jordan shoes
nike huarache
adidas tubular
cheap oakleys
adidas nmd runner
michael kors handbags clearamce
coach outlet online
jordans for cheap