Bugzilla – Full Text Bug Listing
| Summary: | tycho integration tests are too slow | ||
|---|---|---|---|
| Product: | z_Archived | Reporter: | Jan Sievers <jan.sievers> |
| Component: | Tycho | Assignee: | Jan Sievers <jan.sievers> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P3 | CC: | igor, t-oberlies |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Bug Depends on: | 365012 | ||
| Bug Blocks: | |||
|
Description
Jan Sievers
example for in-process maven calls ( see profile "embedded"): http://svn.apache.org/repos/asf/maven/core-integration-testing/trunk/core-it-suite/pom.xml I tried running ITs (test classes, not methods) in parallel with surefire 2.9: <parallel>classes</parallel> <threadCount>2</threadCount> on my machine, this speeds up total IT test execution time by a factor of 4 to 5. (down from ~20 min to ~4-5 min) What we would need to do for parallel test class execution to work reliably: - tests need to be truly independent of each other => separate a few test projects which are shared between several test classes => check for statics/singletons used from concurrent tests - local repo needs to be pre-filled with p2 caches (and any needed maven deps) (For p2 I have the idea of using the p2 mirror mojo before running ITs to fill a local p2 repo with minimal content, say org.eclipse.rcp feature. All ITs would then use the local p2 repo and no longer go on the network) - concurrent access to tycho index files in local repo should be protected (bug 347963 ) Especially the last point is some effort but will pay off in the long run IMHO. (In reply to comment #2) > => check for statics/singletons used from concurrent tests > > - local repo needs to be pre-filled with p2 caches (and any needed maven deps) > (For p2 I have the idea of using the p2 mirror mojo before running ITs to > fill a local p2 repo with minimal content, say org.eclipse.rcp feature. > All ITs would then use the local p2 repo and no longer go on the network) > > - concurrent access to tycho index files in local repo should be protected (bug > 347963 ) > > Especially the last point is some effort but will pay off in the long run IMHO. What is the point in "pre-filling" p2 local cache? If the cache is not thread/process -safe, I think it makes far more sense to fix the cache than to find ways to pre-populate it. After all, running multiple builds concurrently _is_ expected to work. (In reply to comment #3) > What is the point in "pre-filling" p2 local cache? If the cache is not > thread/process -safe, I think it makes far more sense to fix the cache than to > find ways to pre-populate it. After all, running multiple builds concurrently > _is_ expected to work. sure. I mixed up 2 things here: For my local experiment I pre-filled the local repo so I would run into concurrency problems with less probability. The point here was just to find out the potential speed-up. You are right that fixing the concurrency issue in bug 347963 would make this obsolete. For maven core, I understood that the multi-process safe local repo will only come with Tesla so up to then we need to make sure maven (plugin) dependencies of test projects are pre-filled before ITs run. http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=cc0a1231c5ea46dec13541a43a31a0f72363381d adds experimental support for running ITs in parallel (factor 4-5 speedup on my machine) Do I need to do anything special to run ITs beyond adding -Pparallel? I am getting a few IT failures on my macbookpro Tests in error: testP2DataGeneratedForPomDependency(org.eclipse.tycho.test.pomDependencyConsider.p2Data.generate.PomDependencyOnNonTychoArtifactTest): Exit code was non-zero: 1; command line and log = testSigning(org.eclipse.tycho.test.jarsigning.JarSigningTest): Exit code was non-zero: 1; command line and log = eeFromBREE(org.eclipse.tycho.test.bug364095_dependencyResolverBREE.DependencyResolverEETest): Exit code was non-zero: 1; command line and log = test(org.eclipse.tycho.test.bug363331_extraTargetPlatformRequirements.ExtraTargetPlatformRequirementsTest): Exit code was non-zero: 1; command line and log = testFeaturePatch(org.eclipse.tycho.test.featurePatch.build.FeaturePatchTest): Exit code was non-zero: 1; command line and log = (In reply to comment #6) > Do I need to do anything special to run ITs beyond adding -Pparallel? I am > getting a few IT failures on my macbookpro > > > > > Tests in error: > > testP2DataGeneratedForPomDependency(org.eclipse.tycho.test.pomDependencyConsider.p2Data.generate.PomDependencyOnNonTychoArtifactTest): > Exit code was non-zero: 1; command line and log = > testSigning(org.eclipse.tycho.test.jarsigning.JarSigningTest): Exit code was > non-zero: 1; command line and log = > > eeFromBREE(org.eclipse.tycho.test.bug364095_dependencyResolverBREE.DependencyResolverEETest): > Exit code was non-zero: 1; command line and log = > > test(org.eclipse.tycho.test.bug363331_extraTargetPlatformRequirements.ExtraTargetPlatformRequirementsTest): > Exit code was non-zero: 1; command line and log = > testFeaturePatch(org.eclipse.tycho.test.featurePatch.build.FeaturePatchTest): > Exit code was non-zero: 1; command line and log = It's experimental because I suspect issues e.g. with tycho-p2-runtime when used by several concurrent processes (configuration/ folder cannot be shared by several processes I think) Are your errors reproducible every time? If so please attach detailed surefire reports. Errors I am seeing are rare and not reproducible. Even though I can imagine more changes to the integration tests to speed up execution time, parallel test execution with bug 365012 fixed solves this problem for me. FYI, I am having problems running -Pparallel with p2 juno M4 build. At least on macosx. Did not have a chance to try other platforms yet. Please disregard my last comment, I was not able to reproduce failures with -Pparallel once I fully updated to juno m4+ embedded runtime. |