Bugzilla – Full Text Bug Listing
| Summary: | Performance problems opening Test Result Reports from the Summary View | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Technology] Jubula | Reporter: | Alexandra Schladebeck <alexandra.schladebeck> | ||||
| Component: | Core | Assignee: | Project Inbox <jubula.core-inbox> | ||||
| Status: | CLOSED FIXED | QA Contact: | Oliver Goetz <Oliver.Goetz> | ||||
| Severity: | critical | ||||||
| Priority: | P2 | CC: | Achim.Loerke, dennis.schladebeck+eclipse, Marcel.Hein, markus.tiede, zanetu, zeb.ford-reitz | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 4.0 | ||||||
| Hardware: | PC | ||||||
| OS: | Windows Vista | ||||||
| See Also: | https://bugzilla.bredex.de/show_bug.cgi?id=333 | ||||||
| Whiteboard: | |||||||
| Bug Depends on: | 469489 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
|
Description
Alexandra Schladebeck
This looks to be a case of poor Entity Manager (http://download.oracle.com/javaee/5/api/javax/persistence/EntityManager.html) management on our part, although I'm not sure how correct management would look. When opening a Test Result, Jubula stores all related information acquired from the database in the so-called Master Session, which is basically the main Entity Manager (containing information related to the current active Project, Test Result Summaries, etc). On the one hand, this is great, because it allows us to cache all of that information "somewhere" (i.e. the Entity Manager's 2nd level cache), which speeds up cross-linking (i.e. Show Specification) and opening the Test Result again after having closed it. It is also fairly simple to implement and not very error-prone. On the other hand, this is terrible, because all of that cached information uses a lot of memory, which of course has a negative effect on performance, and the cache only gets cleared during certain operations (none of which involve opening or closing Test Results). I don't think we have time to correct this in the near future, but I can offer a workaround. When performance degrades to such a degree, you can reconnect to the database using "Test"->"Select Database...". This should clear the cache mentioned in the paragraph above. Setting target milestone to 1.0 because the bug definitely won't be resolved by M7. Resetting assignee to default because I have no plans to look into this in the near future. Looks like a problem with our internal workflow with a lot of test results. Closed as won´t fix I'm going taking another look at this. Caching of Test Results was disabled (by default) with http://git.eclipse.org/c/jubula/org.eclipse.jubula.core.git/commit/?id=71b0183738226bc7b9087e991e96e759f575c47e. This should resolve the issue of degrading performance over time. I've also added a database index with http://git.eclipse.org/c/jubula/org.eclipse.jubula.core.git/commit/?id=b5d664a02300d46e158f0b755f99731b3e1b8a73, which should improve the overall performance of fetching Test Results. In my simple (and highly *un*rigorous) benchmarks for a somewhat large Test Result, the fetch time was reduced from 37 seconds to 26 seconds. Note that the database index change will not have any effect for database tables built by Jubula before commit b5d664a02300d46e158f0b755f99731b3e1b8a73! The fixes mentioned above are only for the opening and closing of Test Result Viewers. I've filed https://bxapps.bredex.de/bugzilla/show_bug.cgi?id=333 for performance degradation when exporting Test Result Details. Reopening this ticket because the performance of opening test result details did not significantly increase when there are already some test result details opened. The initial opening of test result details however got much faster but when there are around 5-6 other test result details already opened the performance decreases significantly, e.g. opening the test result details for ITE-FULLTEST took around 1.23 min when opening them as first item to look at; trying to open them as 6th item took around 5 min and resulted in two UEs ("An internal error occurred during: "Compacting resource model". Java heap space"; see attachment). A workaround for this problem is to close the other test result details.
Created attachment 213487 [details]
Two error messages when trying to open test result details
Open Test Results require a lot of memory. Opening many causes the garbage collector to run frequently. The Java heap space error tells you that you have opened one to many test result. Only possible solution: increase the Java heap space size by modifying the jubula.ini file. Closing this ticket due to comment 10 In my eyes, the performance for this operation is once again teetering on "unacceptable". A test with the following properties took slightly over a minute to open: * Duration - 1:35:07 * Executed Test Steps - 9662 * Total keywords (queried from database) - 28238 Re-assigning to default assignee as I'm not planning on working on this in the near future. My ideal value for opening a test result from the database in this way would be less than 10 seconds. That's obviously a value which is far off that which we currently have. (I measured in excess of a minute to open our Swing FULLTEST). Two options come to mind: 1. Suggestions and estimations for reducing the time drastically (i.e. to my ideal value) 2. Suggestions and estimations for reducing the time to less than 30 seconds. *** Bug 406776 has been marked as a duplicate of this bug. *** Marking as P1 for this to be looked at with a high priority. I would like the analysis to find out: * What options are available for reducing the time to open the test result reports and what improvement they could bring I would like to know: * How much time we would have to plan to achieve a significant improvement in this area (as close to 10 seconds for opening as possible). This might be interesting if we decide to improve the performance: http://java-persistence-performance.blogspot.de/2011/06/how-to-improve-jpa-performance-by-1825.html The most relevant classes here are - the entity "org.eclipse.jubula.client.core.model.TestResultPO" - see [1] - and - the transient test result node tree generating operation - see [2] [1] git.eclipse.org/c/jubula/org.eclipse.jubula.core.git/tree/org.eclipse.jubula.client.core/src/org/eclipse/jubula/client/core/model/TestResultPO.java#n47 [2] org.eclipse.jubula.client.ui.editors.TestResultViewer.GenerateTestResultTreeOperation.run(IProgressMonitor pMonitor) Combining the changes performed for bug 469489 comment 1 and the following: http://git.eclipse.org/c/jubula/org.eclipse.jubula.core.git/commit/?id=1d9beb20ac28828777c71c7740cfc8871d3d6d45 I've been able to speed up the time taken for opening the result viewer by ~20-30 times. This is now meeting e.g. the requirements stated in comment 14 for opening the ITE nightly FULLTEST result. The second change is a rather radical (but currently performed in a surgical way) one as we do no longer load (which in my opinion has also been done pretty dirty) backing nodes at all. I'd like to see whether anyone is from now on "missing" anything within that editor context and if not perform a proper clean-up of this mechanism of using backing nodes. Verified. I created [1] as a result from testing this. [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=472028 |