Bug 103089 - [JUnit] Running JUnit tests constrained by package takes minutes to run on jumbos
Summary: [JUnit] Running JUnit tests constrained by package takes minutes to run on ju...
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P3 normal with 5 votes (vote)
Target Milestone: 3.1.1   Edit
Assignee: JDT-UI-Inbox CLA Friend
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-07-07 17:36 EDT by Adam Malter CLA Friend
Modified: 2005-09-02 10:44 EDT (History)
4 users (show)

See Also:


Attachments
Stack trace of worker thread (34.05 KB, text/plain)
2005-07-07 17:38 EDT, Adam Malter CLA Friend
no flags Details
A patch that still relies on the region API but computes a more preceise region (2.44 KB, patch)
2005-07-28 12:38 EDT, Dirk Baeumer CLA Friend
no flags Details | Diff
Additional test cases (8.31 KB, patch)
2005-07-28 12:38 EDT, Dirk Baeumer CLA Friend
no flags Details | Diff
An improved patch (2.44 KB, patch)
2005-08-26 08:46 EDT, Dirk Baeumer CLA Friend
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Malter CLA Friend 2005-07-07 17:36:10 EDT
Behavior is that running JUnit tests constrained by a subpackage (by either
right clicking the package in the Package Explorer and running as JUnit or
manually creating the Run profile) takes many minutes to run with the progress
monitor displaying "Searching for errors in <project>"

Further analysis during discussion of bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=82852 [82852] reveals that
excessive IO is occurring during creation of the JavaProject.newTypeHierarchy(..)
Comment 1 Adam Malter CLA Friend 2005-07-07 17:38:45 EDT
Created attachment 24447 [details]
Stack trace of worker thread

Trace shows the Type Hierarchy being created. Trace is over about 90 seconds of
wall time.
Comment 2 Adam Malter CLA Friend 2005-07-07 17:40:58 EDT
Please read previous comments from bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=82852 [82852] for further information.

Project contains about 20k resources, with about 13k of them being .java files.

Machine is WinXP, Dual Xeon 3.6ghz, 2gig. Eclipse is running on Sun Java 1.5_04
with mx heap of 768m.
Comment 3 Philipe Mulet CLA Friend 2005-07-07 18:55:20 EDT
Using region-based hierarchy to find all test cases just doesn't scale, as
described in bug 82852.

Why not instead using a standard search query with suitable search scope ?
Comment 4 Dirk Baeumer CLA Friend 2005-07-08 10:53:03 EDT
Philippe, we are using a type hierarchy since we need to get all sub types of
TestCase in a certain packages. I don't see how this can be done using the
search engine. 

Can you please comment.
Comment 5 Philipe Mulet CLA Friend 2005-07-08 15:23:50 EDT
IType.newTypeHierarchy(...) should do the trick. You could constrain on a given
project, and we could add an API using a searchscope to narrow the subtype
search to a given set of elements (like a package here).

The problem with a region is that it doesn't query indexes, and iterates all the
types to connect them all. Then you likely only exploit a small subset of it.
IRegion hierarchies got somewhat obsoleted once subtypes hierarchies got introduced.
Comment 6 Dirk Baeumer CLA Friend 2005-07-28 09:07:44 EDT
Philippe,

the call we are using is

ITypeHierarchy newTypeHierarchy(
		IType type,
		IRegion region,
		IProgressMonitor monitor)
		throws JavaModelException;

where the type is TestCase.

So the type hierarchy is already constraint to subclasses of TestCase and a
given region. So how different is this from creating a type hierarchy on
TestCase can constrain it to a region. 

Instead of adding new API or use doing the filtering can't we change the method
above to query/use the index since the "root" of the hierarchy is known here.
Comment 7 Dirk Baeumer CLA Friend 2005-07-28 09:08:42 EDT
The method is defined on IJavaProject.
Comment 8 Dirk Baeumer CLA Friend 2005-07-28 12:38:03 EDT
Created attachment 25418 [details]
A patch that still relies on the region API but computes a more preceise region
Comment 9 Dirk Baeumer CLA Friend 2005-07-28 12:38:29 EDT
Created attachment 25419 [details]
Additional test cases
Comment 10 Dirk Baeumer CLA Friend 2005-07-28 12:41:53 EDT
I have released the patches to HEAD. 

The final fix needs some further discussion with JDT/Core (the type hierarchy
API on IType doesn't allow to constrain the hierarchy to a package).
Comment 11 Dirk Baeumer CLA Friend 2005-07-28 12:45:17 EDT
Adam, can you please take the next integration build and see if the performance
is better in your use case.
Comment 12 Adam Malter CLA Friend 2005-07-29 14:12:54 EDT
Dirk,

I'm having a bit of trouble with the patch. I successfully applied the patch to
my codebase and it indeed seems much much faster. However, it now appears to
only 'shallowly' run tests in a package. It does not recursively go down package
children looking for tests, it only executes ones on the absolute branch you select.

So, for example, I have:
test/com.tradecard.framework.EntityTest.java
test/com.tradecard.framework.util.DBUnitPOCTest.java

and I click the framework package, it will run EntityTest, but not the
DBUnitPOCTest.

I think perhaps you did too good of a job of constraining the package.
Comment 13 Adam Malter CLA Friend 2005-07-29 14:17:20 EDT
Oh, I should comment that I manually applied the patch supplied here, and did
not take the integration build. 

So if this patch relies on API changes elsewhere in the codebase, perhaps that
is why I am seeing this functionality as broken.
Comment 14 Dirk Baeumer CLA Friend 2005-08-02 08:41:47 EDT
Adam, I tested it with 3.1 and there executing tests in a package didn't go down
into "subpackages" as well (language wise something like a subpackage doesn't
exist anyways). So in the example below:

Test
+ p
  + Test1.java
+ p.q
  + Test2.java

Executing JUnit tests on p only runs Test1 (I tested it with 3.1). Can you
provide a test case (or a workspace) where subpackages are considered ?
Comment 15 Adam Malter CLA Friend 2005-08-02 19:09:47 EDT
You know what, you are correct. I have not been running JUnit tests through
Eclipse for so long because of this bug I didn't realize that was the old
functionality.

However, I would make an argument that the 'subpackage' recursive functionality
is correct and what people would expect. I know that if I use an ant task to run
JUnit tests and specify a package, I will get the recursive functionality.

But, perhaps this is a new bug.
Comment 16 Dirk Baeumer CLA Friend 2005-08-03 05:30:27 EDT
The sub package bug is https://bugs.eclipse.org/bugs/show_bug.cgi?id=46433.

Adam, the patch went into yesterdays intergration build and if we are happy with
it I will backport it to 3.1.1. 
Comment 17 Dirk Baeumer CLA Friend 2005-08-26 08:45:01 EDT
Markus, Tobias can you please have a look at the patch and review it. Since the
preformance improvements are significant I would like to back port it to 3.1.1.

The code is basically borrowed from TypeHierarchyLifeCycle

Comment 18 Dirk Baeumer CLA Friend 2005-08-26 08:46:04 EDT
Created attachment 26525 [details]
An improved patch
Comment 19 Tobias Widmer CLA Friend 2005-08-26 09:55:15 EDT
Patch looks good to me, +1 for 3.1.1
Comment 20 Markus Keller CLA Friend 2005-08-26 10:55:17 EDT
Patch looks good to me, +1 for 3.1.1

Comment 21 Dirk Baeumer CLA Friend 2005-08-29 06:33:55 EDT
I will mark this bug as fixed for 3.1.1

Opened bug 108254 and 108255 to track this issue in the 3.2 stream.
Comment 22 Markus Keller CLA Friend 2005-09-02 09:40:22 EDT
Start verifying...
Comment 23 Markus Keller CLA Friend 2005-09-02 10:44:52 EDT
Verified in M20050831-1200 + ZRH plugins from R3_1_maintenance.