Bugzilla – Full Text Bug Listing
| Summary: | [perf] Huge Performance regression in ApiDescriptionTests#testCleanVisit() | ||
|---|---|---|---|
| Product: | [Eclipse Project] PDE | Reporter: | Satyam Kandula <satyam.kandula> |
| Component: | API Tools | Assignee: | PDE API Tools Inbox <pde-apitools-inbox> |
| Status: | RESOLVED WORKSFORME | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | ankur_sharma, curtis.windatt.public, daniel_megert, Michael_Rennie |
| Version: | 3.7 | Keywords: | performance |
| Target Milestone: | 3.8 | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Whiteboard: | |||
| Bug Depends on: | |||
| Bug Blocks: | 348419 | ||
|
Description
Satyam Kandula
I don't see any changes in API Tools in that timeframe that would cause this. The test works with the jdt.ui project. I ran this test for M5 and M6 and following were the results after 100 iterations I20110127-2034 (M5) Used Java Heap: 3.92M (95% in [-33.12M, 40.96M]) Measurable effect: 74.66M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) Working Set: 459.08K (95% in [-875.85K, 1.75M]) Measurable effect: 2.63M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) Committed: 117.84K (95% in [-1.13M, 1.36M]) Measurable effect: 2.51M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) Working Set Peak: 521.48K (95% in [145.43K, 897.53K]) Measurable effect: 758.1K (0.4 SDs) (required sample size for an effect of 5% of stdev: 6401) Elapsed Process: 662ms (95% in [647ms, 678ms]) Measurable effect: 30ms (0.4 SDs) Kernel time: 45ms (95% in [39ms, 50ms]) Measurable effect: 11ms (0.4 SDs) (required sample size for an effect of 5% of mean: 2512) Page Faults: 1.22K (95% in [870, 1.58K]) Measurable effect: 712 (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) CPU Time: 814ms (95% in [758ms, 870ms]) Measurable effect: 112ms (0.4 SDs) (required sample size for an effect of 5% of mean: 771) GDI Objects: 0 (95% in [0, 0]) I20110310-1119 (M6) Used Java Heap: -620.12K (95% in [-38.33M, 37.12M]) Measurable effect: 76.05M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) Working Set: 318.64K (95% in [-771.21K, 1.38M]) Measurable effect: 2.15M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) Committed: 39.04K (95% in [-1,010.91K, 1.06M]) Measurable effect: 2.07M (0.4 SDs) (required sample size for an effect of 5% of stdev: 6401) Working Set Peak: 506.64K (95% in [172.56K, 840.72K]) Measurable effect: 673.48K (0.4 SDs) (required sample size for an effect of 5% of stdev: 6401) Elapsed Process: 660ms (95% in [647ms, 673ms]) Measurable effect: 25ms (0.4 SDs) Kernel time: 43ms (95% in [38ms, 48ms]) Measurable effect: 10ms (0.4 SDs) (required sample size for an effect of 5% of mean: 2395) Page Faults: 1.18K (95% in [881, 1.48K]) Measurable effect: 605 (0.4 SDs) (required sample size for an effect of 5% of stdev: 6400) CPU Time: 816ms (95% in [764ms, 867ms]) Measurable effect: 103ms (0.4 SDs) (required sample size for an effect of 5% of mean: 643) GDI Objects: 0 (95% in [0, 0]) The two results are quite close. I suspect the real reason might be outside API Tools. (In reply to comment #2) I couldn't reproduce either :(. I tried running the full test suite also but couldn't reproduce. I tried running the suites using Windows 7 64bit and also got consistent results (no regression). Closing as worksforme, since we cannot reproduce the regression. reopening, Ankur wants to do some additional investigations. Though we are still pursuing it, there is not much hope for any breakthrough during 3.7 Returning to WORKSFORME, no additional investigation planned. |