Community
Participate
Working Groups
The entire CBI has a lot of moving parts. Many are on the infra side (authentication, servers, storage, Nexus, Signing, Git, k8s). Many more exist on the project side (size of repo, the many tasks in the job, ant, maven, tycho, dependencies, version mismatches, Xmx settings, external slaves, ...) We're reaching a point where it feels that build slowness/breakage leads to a bug, where the Eclipse infra team must effectively prove that the infra is running correctly. Sometimes the infra is broken. Most often it's not. Please provision a typical 80-percent job that runs a build: checkout from Gerrit, checkout from GitHub, pull a dependency from maven, build stuff, sign, copy to download, upload to Nexus, update a web page, tag a repo, then exit. Run this job every... hour? Then never modify the underlying code -- this will provide benchmarks and trends for the time it takes to run the various tasks, and also demonstrate when there is something broken with the infra -- and when there is not.
(In reply to Denis Roy from comment #0) > The entire CBI has a lot of moving parts. Many are on the infra side > (authentication, servers, storage, Nexus, Signing, Git, k8s). Many more > exist on the project side (size of repo, the many tasks in the job, ant, > maven, tycho, dependencies, version mismatches, Xmx settings, external > slaves, ...) > > > We're reaching a point where it feels that build slowness/breakage leads to > a bug, where the Eclipse infra team must effectively prove that the infra is > running correctly. > > > Sometimes the infra is broken. Most often it's not. > > > Please provision a typical 80-percent job that runs a build: checkout from > Gerrit, checkout from GitHub, pull a dependency from maven, build stuff, > sign, copy to download, upload to Nexus, update a web page, tag a repo, then > exit. Run this job every... hour? Then never modify the underlying code -- > this will provide benchmarks and trends for the time it takes to run the > various tasks, and also demonstrate when there is something broken with the > infra -- and when there is not. you may consider to run the build of an existing project using a fixed version of that project to serve as this infra test
I agree that we need a set of test jobs that help us to distinguish between dev and infra related issues. This won't be a/the silver bullet though. For example, if one node in the cluster has an issue, we won't necessarily find this by running test jobs, since they may be scheduled on a different node. We also can't recreate all particularities of projects that have a higher chance of causing issues (lots of I/O, high CPU demands, tons of files, etc).
May be you can try using https://ci.eclipse.org/platform/job/eclipse-aggregator-master/ This is the minimal build set up and this does the following 1. Git operations to get latest source from git. 2. run maven build on the source. This job without -Peclipse-sign profile should complete within an hour. with -Peclipse-sign maven profile it will take another 20 minutes extra. Here are some of the features 1. 501 maven projects in a single reactor 2. There will be lots of disk I/O and lot of input from eclipse p2 repositories. If you add eclipse-sign profile this will test the signing services as well. I believe this will test k8s, nexus and signing.
We have https://ci.eclipse.org/cbi/job/bench-jgit/ and https://ci.eclipse.org/cbi/job/bench-guava/ for a while now with build time trends (e.g., https://ci.eclipse.org/cbi/job/bench-guava/buildTimeTrend).
