Bug 288779

Comment 1 Nick Boldt 2009-09-08 13:20:07 EDT

Two thoughts...

a) Oisin, as a member of Calisto-dev, can you not create your own Hudson jobs? Who set up the existing STP jobs in the first place?

b) If the goal is to build the same bits from the same source/config but use a different JDK, then you can set up a multi-configuration build which will, in parallel, run your job against both JDK5 and JDK6. 

  "Suitable for projects that need a large number of different 
  configurations, such as testing on multiple environments, 
  platform-specific builds, etc."
   -- https://build.eclipse.org/hudson/view/Athena%20CBI/newJob

Would that be a better solution? Note that these multi-config builds can be multi-dimensional, so if your individual stp.* builds can run in parallel, you could set them to all build from a single job. Thus 6 builds x 2 JDKs = 12 configs, all run from a single entry point.

Here's a sample job that was set up to run Query, Validation, and Transaction in parallel. (It was later decided that serial was better than parallel.)

https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-emf-qvt-1.4/

Comment 2 Oisin Hurley 2009-09-08 13:42:26 EDT

(In reply to comment #1)
> a) Oisin, as a member of Calisto-dev, can you not create your own Hudson jobs?
> Who set up the existing STP jobs in the first place?

I set them up, but for some reason I can't find the "Make New Job",
or at least I couldn't at 11pm last night when I was looking for it.
 
BTW if I hit the URL

https://build.eclipse.org/hudson/view/STP/newJob

I get the message

"ohurley is missing the Create permission"

Can I fix this myself, or does someone else have to?

> b) If the goal is to build the same bits from the same source/config but use a
> different JDK, then you can set up a multi-configuration build which will, in
> parallel, run your job against both JDK5 and JDK6. 

*That's* the kind of thing I am looking for - thanks for the info.

I'll check the sample job you've linked to and get back to again
later tonight. Really what I'm looking to create is N independent
builds, with a roll-up build that is kicked off when all of the N
succeed.

Comment 3 Nick Boldt 2009-09-08 15:29:56 EDT

(In reply to comment #2)
> https://build.eclipse.org/hudson/view/STP/newJob
> I get the message
> "ohurley is missing the Create permission"
> Can I fix this myself, or does someone else have to?

Ping webmaster@eclipse.org - you should be a Hudson admin. I've added 'em to cc:
 
> > you can set up a multi-configuration build which will, in
> > parallel, run your job against both JDK5 and JDK6. 
> *That's* the kind of thing I am looking for - thanks for the info.
> I'll check the sample job you've linked to and get back to again
> later tonight. Really what I'm looking to create is N independent
> builds, with a roll-up build that is kicked off when all of the N
> succeed.

You can chain builds together in series, or have things run in parallel against different configurations. So, 

6pm (build A + build A') run on JDK 5 and 6; when both done (either green or yellow), build B is told it can start.
6:30pm (build B + build B') run on JDK 5 and 6; when both done, build C is told it can start.
...
8:30pm (build Z) runs on JDK 5 to aggregate the update site zips from builds A - Y; it was told it could build by the last build in the stack

Note that if you have parallel builds which don't depend on each other (eg., A and B), you can have them BOTH notifying downstream that it's time to build; however, you may get extra builds you don't strictly need:

A builds for 30 mins, tells C it can build when A is done
B builds in parallel for 35 mins, tells C it can build when B is done
C builds (because of A's invocation) for 30 mins
C builds again (because of B's invocation) immediately thereafter for 30 mins

So the "stack" is done after the first C build, but you get a second one because B also requested it. And then, you can also have scheduled CVS/SVN checks, such that if C notices its repo has been updated, it'll build by itself too. Depending on the timing, you may end up with 2 or 3 back-to-back C builds.

Upshot is lots of extra builds, but also lots of cross-build permutation testing - which is certainly a good thing.

---

One additional feature of Athena you might like (assuming you're using Athena, not pure PDE or Buckminster) is that you can create "snapshot" zips instead of named zips, which allows downstream builds to depend on the output of upstream builds. http://wiki.eclipse.org/Athena_Progress_Report#2009-08-07

Comment 4 Nick Boldt 2009-09-08 15:37:00 EDT

One final thought.

If you have a well-defined build stack order (A must build before B because B depends on A, etc.) you might want to introduce a little bit of clarity/sanity into the build names themselves. Thus, instead of:

stp.bpmn-modeler.trunk.jdk6
stp.intermediate-model.trunk.jdk6
stp.policy.trunk.jdk6
stp.sca-tools.trunk.jdk6
stp.sca-tools.trunk.amalgam.jdk6
stp.toplevel.trunk.jdk6

you might want:

stp.0.toplevel
stp.1.intermediate-model
stp.2a.bpmn-modeler
stp.2b.policy
stp.3.sca-tools
stp.4.sca-tools.amalgam

which suggests that policy and bpmn can run in parallel.

Or, you might want something like:

stp.0.toplevel
stp.1-0.intermediate-model
stp.2-1-0.bpmn-modeler
stp.3-0.policy
stp.4-3-2-1-0.sca-tools
stp.5-4.sca-tools.amalgam

where the first # is an identifier, and the -# suffices are builds on which that build depends.

Comment 5 David Carver 2010-04-27 08:24:39 EDT

Oisin just pinging you on this...do you need this now for SOA or what is the current status?

Comment 6 Denis Roy 2011-03-09 14:24:11 EST

Old bug, closing.  Please reopen if the missing info can be provided.