Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 401627 - [1.8][testing] RunAllJava8Tests should run TestAll's in 1.8 mode (only)
Summary: [1.8][testing] RunAllJava8Tests should run TestAll's in 1.8 mode (only)
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.3   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: BETA J8   Edit
Assignee: shankha banerjee CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-24 12:35 EST by Srikanth Sankaran CLA
Modified: 2013-04-24 04:56 EDT (History)
3 users (show)

See Also:
jarthana: review+


Attachments
A hack but works (1.84 KB, patch)
2013-02-26 15:22 EST, Jay Arthanareeswaran CLA
no flags Details | Diff
Patch (10.72 KB, patch)
2013-04-22 10:09 EDT, shankha banerjee CLA
no flags Details | Diff
Patch: 23 April (8.28 KB, patch)
2013-04-23 06:53 EDT, shankha banerjee CLA
no flags Details | Diff
New Patch (7.42 KB, patch)
2013-04-24 02:24 EDT, shankha banerjee CLA
no flags Details | Diff
New Patch (5.89 KB, patch)
2013-04-24 04:36 EDT, shankha banerjee CLA
shankhba: review?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Srikanth Sankaran CLA 2013-02-24 12:35:01 EST
BETA_JAVA8
----------

The smoke test RunAllJava8Tests should also run RunCompilerTests, but only
in 1.8 mode. We are missing some failures otherwise sometimes, which cause
needless disruption to other committers.
Comment 1 Srikanth Sankaran CLA 2013-02-24 12:35:34 EST
Jay, can you follow up on this on a priority basis ? Thanks.
Comment 2 Srikanth Sankaran CLA 2013-02-26 07:30:18 EST
Jesper, are you interested in taking this up ? Jay is busy juggling half
a dozen things:

Basically, RunAllJava8Tests should run:

(1) org.eclipse.jdt.core.tests.compiler.regression.TestAll (only in 1.8 mode)
(2) org.eclipse.jdt.core.tests.compiler.parser.TestAll (only in 1.8 mode)
(3) org.eclipse.jdt.core.tests.eval.TestAll (only in 1.8 mode)

(4) RunAllJava8Tests already runs encodes some Java8 tests which are
mentioned in one of these TestAlls. Such references should be removed
from RunAllJava8Tests so there is no duplication (not sure if junit
itself will eliminate duplicates, but still it is a good idea to clean
this up)

(5) ComplianceDiagnosticsTest should run in all modes 1.3 - 1.8.
Comment 3 Jesper Moller CLA 2013-02-26 08:04:05 EST
OK, I'll take this
Comment 4 Srikanth Sankaran CLA 2013-02-26 08:13:11 EST
(In reply to comment #3)
> OK, I'll take this

Thanks, please ignore the references to RunCompilerTests in comment#0,
Comment#2 describes what is requested.
Comment 5 Srikanth Sankaran CLA 2013-02-26 08:14:20 EST
Take a look at AbstractCompilerTest.buildMinimalComplianceTestSuite(Class, int)
This is useful to restrict the compliance levels.
Comment 6 Jay Arthanareeswaran CLA 2013-02-26 15:22:17 EST
Created attachment 227631 [details]
A hack but works

Just thought we could use this for the time being. This is more of a hack but works alright.

Jesper, if you have started working on this and have a cleaner fix, feel free to post the patch. Otherwise, we can live with this - the RunAllJava8Test is a temporary suite anyway.
Comment 7 Srikanth Sankaran CLA 2013-02-26 17:47:30 EST
This won't work for all : RunCompilerTests is a wrapper not publicly available.
Jay, besides I am trying to get Jesper to resolve a number of bugs so we can
set the ball in motion for committer election process. So let him work on
this and come up with a solution that would benefit all.
Comment 8 Jesper Moller CLA 2013-03-07 16:40:43 EST
I hope no-one is blocked by this, I have a number of other bugs to clear up before this one (currently focused on master / Kepler)
Comment 9 Srikanth Sankaran CLA 2013-03-07 22:15:43 EST
(In reply to comment #8)
> I hope no-one is blocked by this, I have a number of other bugs to clear up
> before this one (currently focused on master / Kepler)

Sounds good. This present bug is nice to fix, but can be taken up in due
course when some cycles can be stolen.
Comment 10 Srikanth Sankaran CLA 2013-04-06 05:21:57 EDT
Shankha, Please take Jay's help to wrap this up. Thanks.
Comment 11 shankha banerjee CLA 2013-04-22 10:09:26 EDT
Created attachment 229962 [details]
Patch

I have run the test cases individually to make sure we are running all the test cases which we ought to run. 

Some test cases which were run directly under RunAllJava8 tests and which belong to org.eclipse.jdt.core.tests.compiler.regression have been removed as these tests were only run on Compliance 1.8 and we are already running them as part of org.eclipse.jdt.core.tests.compiler.regression suite.
Comment 12 shankha banerjee CLA 2013-04-22 10:11:12 EDT
Please review. The changes have been uploaded.
Comment 13 Jay Arthanareeswaran CLA 2013-04-22 10:58:24 EDT
(In reply to comment #11)
> Created attachment 229962 [details]
> Patch

I haven't reviewed this patch in details but I found the following problems.

1. With the patch, RunCompilerTests only runs about 2797 tests. It should be over 10,122, which is what I see without the patch. Even after the fix, the existing tests and suites should continue to function the way they used to.

2. As per above number, RunAllJava8Tests should have the current no of tests in RunAllJava8Tests + 10,122 - whatever common tests these two might have. But I see only about 3400 tests when RunAllJava8Tests  is run with the patch.

3. Method setComplianceLevel() doesn't use the parameter, instead hard codes the compliance level.

Please address these issues and post a new patch.
Comment 14 shankha banerjee CLA 2013-04-23 06:53:11 EDT
Created attachment 230014 [details]
Patch: 23 April

The review comments have been taken care of. Changes needed to be made in parser.TestAll as ComplianceDaignosticTest have to be run across all complaince levels. 

The total number of tests were : 12821. The Regression test and eval test are run in 1.8 mode. The parser tests except ComplianceDaignosticTest are run in 1.8 mode. 

Thanks.
Comment 15 shankha banerjee CLA 2013-04-24 02:24:52 EDT
Created attachment 230059 [details]
New Patch

Made changes to make it more clearer how the changes help in making the paresr test runs without compliance daigonsitic test. 

The copyright information was updated for few files. 

We need not change the compliance level at the workspace level.
Comment 16 shankha banerjee CLA 2013-04-24 04:36:08 EDT
Created attachment 230062 [details]
New Patch

New Patch uploaded so that can be applied through egit. Have checked the number of tests being run and results. They do not ave any discrepancies. 

Have renamed the function setTestClasses in TestAll.java under parser tests to getTestClassess. 

Thanks
Comment 17 Jay Arthanareeswaran CLA 2013-04-24 04:56:30 EDT
Patch looks good and released to BETA_JAVA8 via commit:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=25d70eac1f59d17fd35f15454804710ecb835662