Bugzilla – Full Text Bug Listing
| Summary: | [releng] Run a build from any platform | ||
|---|---|---|---|
| Product: | [Eclipse Project] e4 | Reporter: | Paul Webster <pwebster> |
| Component: | UI | Assignee: | Project Inbox <e4.ui-inbox> |
| Status: | RESOLVED FIXED | QA Contact: | Paul Webster <pwebster> |
| Severity: | normal | ||
| Priority: | P3 | CC: | aniefer, darin.eclipse, dj.houghton, hendy, henrik.lindberg, Lars.Vogel, nboldt, thomas |
| Version: | 0.9 | Keywords: | helpwanted |
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Paul Webster
Vote +1 for this bug :) From the initial mailing list post: Right now org.eclipse.e4.builder/scripts/masterBuild.sh launches the e4 builds. It uses org.eclipse.e4.builder/builder/general as the builder (we build against a master feature, org.eclipse.e4.master) and copies/modifies templates from org.eclipse.e4.builder/templates to generate the published download site. But a number of artifacts are generated by the script, the tests are run from the script, the difference between test builds and published builds are controlled by the scripts, etc. That makes this very much a linux only build. Plus, it also must consume 4 zipped repos (using repo2runnable) that must be provided, for example: eclipse-R-3.5-200906111540.zip GEF-Update-3.5.0.zip emf-xsd-Update-2.5.0.zip wst-sdk-buildrepo-R-3.1-20090616035105.zip I'd like help to convert most of what the bash script does to an ant driven build. The goal would be that a test build could be launched from within eclipse on any of the platforms. Hi Paul, May I propose another solution? Use Buckminster. It is capable of configuring a target platform from external repositories and it can publish both binary and source bundles (in different categories if needed) to a new p2 repository. Optionally it will also do signing, and pack200. Buckminster executes the exact same build from within the IDE as it does if you run it headlessly. If you're interested, I'd be happy to help you get started. The harness for Athena is shell which wraps ant, or just ant. So, it can be run from Hudson (shell -> ant) or within Eclipse (ant). See http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started/Build_In_Eclipse It supports both SDKs and p2 repo zips as input, and will run your headless (UI or core) JUnits on a linux server. I would suggest evaluating both Athena and Buckminster and deciding which solution works better for your needs. One day, the two projects may very well merge, but for now they approach this requirement from different angles. :) I've written out some of the information http://wiki.eclipse.org/E4/Builds#Build_Requirements (mostly) from the masterBuild.sh script. We have a number of variables related to where the builds and supporting files live (basebuilder, targets, a base java, where to build, where those results go, etc). Also some related to what releng projects to check out. Then there are variables that describe the environment in the build.properties, some other base locations, where to get certain downloadable things for the target platform (WST, GEF, EMF, etc). There's manual setup involved (getting the JREs, the basebuilder, the builder, the zipped repos, etc). Some of these are unavoidable, but some should perhaps be driven by properties. Then there's a list of steps that are currently driven by the masterBuild.sh linux script. This is the "requirements" statement, since the build has to continue to produce what is provided in this script. PW The goal of this bug is to turn what is currently a linux script and a complex directory structure into something that can easily launch a test build from a workspace or a real build on build.eclipse.org. Slowly converting the script functions into ant tasks that pull them all into the PDE pre/post build cycle seems far and away the simplest next step (probably followed by a way to determine if it's a test build or real build). I'm open to suggestions for Athena/CBI or Buckminster. I can help with reviewing patches and/or integrating any changes into the directory structure/build.eclipse.org, however I don't have the time or expertise to re-write the linux script functions into ant tasks or to re-write the linux script functions into Buckminster CSPEC/MSPEC/or code. PW (In reply to comment #2) > Right now org.eclipse.e4.builder/scripts/masterBuild.sh launches the > e4 builds. It uses org.eclipse.e4.builder/builder/general as the > builder (we build against a master feature, org.eclipse.e4.master) and > copies/modifies templates from org.eclipse.e4.builder/templates to > generate the published download site. Having had a quick look through the script, it looks a lot like what I do in Hudson via start.sh, or in Eclipse via build.xml. In Eclipse it's assumed you'll have your e4.builder project already checked out (along with basebuilder and common.releng (but there are .psfs for that so it's like two clicks)). In Hudson, I have basebuilder and common.releng cached on disk, but if you need a different version than what's there, the script will fetch them like yours does. If found, you get a symlink; if not, a fresh checkout folder. Makes setup as fast as possible. Your buildMasterFeature method looks a lot like my start.sh: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/start.sh?root=Technology_Project&view=markup (see line 465) So aside from some extra flags you'd have to pass through (for flex, for untransformedRepo?) and conversion of the generateSwtZip method to an equivalent step in buildExtra.xml, if looks like a pretty straighforward migration. (Mail notification can be set in Hudson instead of as part of the script so your local builds won't spam the e4-dev list.) To run start.sh in Hudson, you need a boilerplate wrapper like this one: http://dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/releng/trunk/org.eclipse.linuxtools.releng/tools/linuxtools_build.eclipse.org_hudson.sh But to run the same build locally in Eclipse, you need only run the build.xml (right-click > run as > ant build) file in your .releng (or .builder) project. It too is boilerplate, as it's really just loading build.properties and running the common builder's build.xml http://dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/releng/trunk/org.eclipse.linuxtools.releng/build.xml http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/buildAll.xml?root=Technology_Project The only thing you CANNOT do with Athena in Hudson on build.eclipse.org is tagging, because the hudsonbuild user doesn't have commit rights to your repo. And you can't publish directly to download.eclipse because hudsonbuild doesn't have write perms there. (Publishing is solved here: http://wiki.eclipse.org/Common_Build_Infrastructure/Publishing) Tagging is still TBD. (In reply to comment #7) > Having had a quick look through the script, it looks a lot like what I do in > Hudson via start.sh, or in Eclipse via build.xml. > This should look familiar :-) I swiped your start.sh to get started. PW (In reply to comment #8) > (In reply to comment #7) > > Having had a quick look through the script, it looks a lot like what I do in > > Hudson via start.sh, or in Eclipse via build.xml. > This should look familiar :-) I swiped your start.sh to get started. Excellent. Now swipe one of the several .releng projects here and get started running in Eclipse. Once that works, porting it to run in Hudson is trivial. http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/?root=Technology_Project GEF is the easiest example to get working; in fact, it's the example in the wiki. http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started/Build_In_Eclipse The build.properties in there is used everywhere in the PDE-based build so you can set anything there and PDE *should* respect it all the way through building the update site zip & tests, running the tests, and any downstream repackaging you might want to do thereafter. Fixed with Platform and e4 now using CBI |