Community
Participate
Working Groups
Background: Traditionally, buildable source distributions do not include any binary files. This ensures that builds are reproducible, auditable, and there is a choice of tools used in the building process. Current situation: Currently, there are a few jars in the srcIncluded zips that I don't think should be there. Specifically, jdtcore.jar and jdtCompilerAdapter.jar. These jars can be built using the source that is part of the zip. I understand that these are needed to build the rest of Eclipse, but in a compiler bootstrapping situation such as this, it is customary to build all parts necessary from source. I've attached a patch to the main build.xml in the srcIncluded zip which builds all parts necessary for the aforementioned jars and places them in the root of the exploded zip. They can then be used to build the rest of Eclipse (including themselves).
Created attachment 17826 [details] patch to build.xml to build jdtcore.jar and jdtCompilerAdapter.jar Note that when this patch is applied, jdtcore.jar and jdtCompilerAdapter.jar should be removed from srcFetch and srcIncluded zips as they will no longer be necessary.
If possible, it'd be great if we could get this in for 3.1M5.
Created attachment 17828 [details] New patch for CVS head from Ben Konrath
The jdtcore jars included in the src build are from a stable build, in this case 3.1 M4. We want the source build to replicate the real eclipse build process so it should use the same compiler. Thus it will not use the compiler source from the current build, but rather the compiler that was actually used in the build. In order implement this, we'll need to compile the compiler using javac using the resulting binary to recompile the compiler source use the resulting binary to compile eclipse We'll incorporate this as time permits.
Hi Kim, I'm a bit confused by the second paragraph in comment #4. Could you please provide an example to help explain? IMHO replicating the build process by providing a stable compiler does not offer a signification advantage. If someone wants to build a stable Eclipse, I think that they should build from an M-build or a true stable release. The way I understand it is that even if Eclipse is built with a compiler from a stabilized build, the projects that are compiled when using Eclipse will still use the unstable compiler - this includes runtime workbenches if developing plugins. Also, this change would not affect the eclipse.org builds since the srcIncluded and srcFetch zips are not used in the build process (as far as I can tell). Andrew and I talked about adding a conditional variable to the "bootstrap" target so that it is only run if the user sets conditional variable to be true. The bootstrap target would have to be changed to delete the jdt jars, build them and then move them to correct place. The idea is that only people who know what they are doing would set this conditional variable. Personally I don't really like this method because it keeps the binary jars included unnecessarily in the source distribution. Comments?
Eclipse is built using a stable jdtcore compiler. The second paragraph in comment #4 refers to the fact that we use the jdtcore compiler to compile eclipse. However, in order to create the jdtcore compiler from source, we compile the compiler souce using javac. This will give use a binary jdtcore compiler. But it's not the same as the one that was used to create eclipse because it wasn't created using jdtcore, but javac. So we have to take the jdtcore compiler created using javac, and apply it against the compiler source again to create a new jdtcore compiler that was created using the first jdtcore compiler. This binary can be used to compile the eclipse source build. This is the recommended approach by the jdtcore team. We are taking this approach because we want the source builds to use the same process that is used in the regular build. Using this methodology, if people open bugs with respect to their source builds we can be sure that they are valid bugs and not problems arising as a result of the way that eclipse was compiled. Also, we want to ensure that external contributors can use the same process for producing ports of eclipse that can be included on eclipse.org. For instance, HP provided a hpux-motif-ai64 drop for 3.0.1. We won't add a conditional variable to the build script to build the compiler from source. It would be better to change the scripts to compile the compiler from source by default and remove the binary jars.
Created attachment 17939 [details] double stage bootstrap patch Hi Kim, I changed the patch to bootstrap the compiler in the following way: * use javac to build jdtcore.jar and jdtCompilerAdapter.jar * use these two "freshly-built" jars to build jdtcore.jar and jdtCompilerAdapter.jar again * move the new jars in place to build eclipse The patch requires that jdtcore.jar and jdtComplierAdapter.jar not be present in the root of the unpacked zip. The CLASSPATH should still be set to include these so that the ECJ is picked up during the second stage of the bootstrap. I also remove the "-encoding ISO-8859-1" from the compilerArgs as I don't think that it is required and it made the patch a little easier. It would be great if you take a look at this after 3.1M5 is released. Thanks.
Comment on attachment 17828 [details] New patch for CVS head from Ben Konrath Ben's patch is newer than this.
Bootstrap source builds will be created as of builds > N20050303 I used a different approach from the patch because I was having a lot of classpath issues within ant. Suggestions for improvement are always welcome :-) The changes are in the HEAD stream of org.eclipse.releng.eclipsebuilder/sdk
Created attachment 18570 [details] patch to fix some issues with the current bootstrap Hi Kim, I just looked at your solution and most of it looks good. The attached patch addresses the issues I found. Here is a changelog of the patch: * export ANT_OPTS on a separate line which makes it more portable (build) * removed unnecessary export ORIGCLASSPATH (build) * some formatting fixes (all) * removed duplicate javac line (compilejdtcore.xml) * removed the <and> statements within the <condition> statements because they not necessary now that there is only one jar * compilejdtcore.xml and the main build.xml now fail if org.eclipse.jdt.core.JDTCompilerAdapter cannot be found * copy the ECJ (Eclipse Complier for Java) from current build instead of using pre-zipped source * change jdtcore.jar to ecj.jar because it's technically not jdtcore.jar Comments?
The jar line in compilejdtcore.xml differs from that in compilejdtcorewithjavac.xml in this way (ignoring the ../): - <jar jarfile="../jdtcore.jar" basedir="${destdir}"> + <jar jarfile="jdtcore.jar"> This seems to be causing things to be added twice to the jar. Removing the "basedir" property seems to fix it. I'll attached a patch.
Created attachment 18864 [details] patch to remove unnecessary "basedir" in jar ant task
reopening
Added all patches except the parts that use the compiler from the build. We will continue to use the stable compiler to build eclipse.
Hi Kim, Thanks for applying those changes. As far as this compiler thing goes, I really think that it's important for the included compiler to build Eclipse only because Eclipse is the perfect test case for that compiler. But I guess we'll have to agree to disagree on this issue :)
