Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 324970 - Provide a mechanism to continue manual analysis from MAT's automatic reports
Summary: Provide a mechanism to continue manual analysis from MAT's automatic reports
Status: RESOLVED FIXED
Alias: None
Product: MAT
Classification: Tools
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: All Windows XP
: P3 enhancement (vote)
Target Milestone: 1.11.0   Edit
Assignee: Andrew Johnson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 536920
Blocks:
  Show dependency tree
 
Reported: 2010-09-10 10:24 EDT by Krum Tsvetkov CLA
Modified: 2020-10-09 11:13 EDT (History)
2 users (show)

See Also:


Attachments
Links from Pie Chart (120.48 KB, image/png)
2020-10-09 11:11 EDT, Andrew Johnson CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Krum Tsvetkov CLA 2010-09-10 10:24:16 EDT
MAT provides various reports (and the possibility to create new ones) as a mechanism to execute a series of queries, structure the results and present them in a compact form. However, if the user identifies a useful set of objects somewhere in the report (say a huge number of redundant Strings, or a huge number of empty HashMaps), then it is difficult to reuse the results from the report and continue the analysis (for example to analyze who is referencing the objects in these interesting sets). The user has to 1) know what are the steps which the report has executed to get to the data, and 2) run the same steps manually.
It would be much easier for the end user to have the possibility (while in the tool) to somehow reuse these results and continue the analysis.

I don't have the concrete proposal how to solve this, but I'd like to use this "enhancement" entry as a discussion forum on the topic.

Let us consider different approaches and try to come up with some reasonable solution. Below are some ideas:

1) identify which are the most important and most frequently used features (e.g. the memory waste part of the component report) and provide a wizard or similar to guide the user through the steps. This would make some steps easier, but is not a general solution, so there will be always some unhappy users.

2) try to add more and more details to the reports, for example information about the dominators of empty collections. This is easy to achieve, but is again a limited improvement - there will always be some tasks which need manual analysis

3) Find an alternative way to render the reports - MAT is already able to render the tables and trees produced by queries both in HTML and SWT. HTML was used for the reports, because it is easy to structure a large amount (and a mixture) of tables, trees, text, etc...
Probably we may also find a way to render the same data in a well structured way using SWT means, and by this allow the user to continue the analysis manually from the "leaves" of the report - be it histograms or something else.

Let us collect some further ideas and discuss the above mentioned suggestions.
Comment 1 Andrew Johnson CLA 2011-01-17 08:03:06 EST
One minor enhancement would be to ensure we use IContextObjectSet where appropriate not IContextObject e.g. in LeakHunter getHistogramOfDominated
then modify HTMLOutputer to have an additional query listing say the first 10 objects or the OQL command if appropriate.
Comment 2 Andrew Johnson CLA 2011-01-18 09:19:10 EST
The HTMLOutputter could be enhanced to show more information from a StructuredResult

If there is a list of objects from IContextObjectSet.getOQL then return a link for this, else if IContextObjectSet.getObjects then return a query for the first 10 objects.

first 10 of 30 objects
or
7 objects

If there are alternate contexts from ResultMetaData.getContextProviders() then we can provide this information too.

Perhaps we should skip a context if it returns the same data as the base IContext.
For example, ClassLoaderExplorer has Class Loader and Defined Classes contexts, but the Class Loader context is the same as the base context.

Should there by a URI for multiple objects? The query context menu would then appear for them, rather than running a query to display them all.

Is the MAT protocol suitable for when there are multiple snapshots? It doesn't have the snapshot name, so this must be obtained from elsewhere.
Comment 3 Andrew Johnson CLA 2011-01-22 17:15:29 EST
I've updated HtmlOutputter

If there are multiple contexts then these are also displayed, presuming they
are not simple contexts with the same object id as the base context.

If there is a context object set with at least one object then if there are <= 10 then all the objects are linked via a list_objects query
 All {0} objects
else if there is an OQL this is used via the OQL query
 All objects
else via the list_objects query
 First 10 of {1} objects

For example a immediate dominators report can now say:

sun.security.provider.Sun
Objects
All 1 objects
Dominated Objects
First 10 of 23 objects


Is this a useful first step?
Comment 4 Andrew Johnson CLA 2011-04-13 13:01:11 EDT
We could show more commands in the report, if available, as links to rerunning a part of the report.

If there is a QuerySpec with a command and a result of a SectionSpec then the report generation code converts the SectionSpec to a SectionPart based on the QuerySpec, but the command is lost. The command cannot then be shown in the report.

Perhaps SectionPart should also hold a command in this situation.

If the command contains a ${} expansion for OQL then this gets lost by the params conversion.

E.g. a command such as
list_objects SELECT OBJECTS s FROM OBJECTS ${snapshot}.getImmediateDominatedIds(-1) s;

goes to
list_objects SELECT OBJECTS s FROM OBJECTS null.getImmediateDominatedIds(-1) s;

Perhaps unrecognized vars should not be substituted, or replaced with the empty string, and the substitution only done for executed commands.
Comment 5 Andrew Johnson CLA 2011-04-14 10:00:35 EDT
Adding getCommand to a SectionPart generated from a QuerySpec gives more links in the leak suspects report:

Leaks

System Properties
Thread Overview
Top Consumers
Class Histogram

all then have links.

We can avoid problems with ${snapshot} being substituted by changing it to $ {snapshot} with a space between $ and {. The OQL parser handles that, the parameter substitution in the report generation ignores that.
Comment 6 Andrew Johnson CLA 2014-05-13 13:08:11 EDT
The leak suspects report could do with more commands from the different parts - possibly by adding QuerySpec wrappers to the results.
Comment 7 Eclipse Genie CLA 2020-10-07 08:29:15 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170435
Comment 8 Eclipse Genie CLA 2020-10-07 08:34:28 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170437
Comment 10 Eclipse Genie CLA 2020-10-07 09:19:44 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170442
Comment 12 Andrew Johnson CLA 2020-10-07 10:27:23 EDT
I've added some more links to the Top Consumers report when running on the whole heap.

I have also improved the HTML charts. There was an <area> section for the pie chart showing the labels in the legend.
It wasn't always in the right place with Windows high res displays - though rescaling coords in <area> doesn't seem the right solution
as there should be a better way in BIRT.
I have now added an mat:// link if the pie slice has an associated object ID and given alternative text for the labels <area> and the whole image.
I have improved the tooltip/hover text for pie chart labels to show the size too.

For collapsible sections I change the words from 'hide / unhide' when it is open to 'unhide / hide' when it is closed as that
is useful when using a screen reader.

Do we need to do any more - or are the links in the existing HTML reports sufficient?
Comment 13 Eclipse Genie CLA 2020-10-08 13:09:05 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170515
Comment 15 Krum Tsvetkov CLA 2020-10-09 06:24:47 EDT
>Do we need to do any more - or are the links in the existing HTML reports sufficient?
I think they are sufficient.
Comment 16 Eclipse Genie CLA 2020-10-09 08:39:22 EDT
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170543
Comment 18 Andrew Johnson CLA 2020-10-09 11:11:22 EDT
Created attachment 284409 [details]
Links from Pie Chart

Showing a area hotspot link from the legend of a pie chart in an HTML report
Comment 19 Andrew Johnson CLA 2020-10-09 11:13:12 EDT
Marking this as done.
For new and noteworthy, note the extra links on reports and the first 10 objects available from IContextObjectSet.

Also the tooltips and mat:// links now work on Pie Chart images on HTML reports