Community
Participate
Working Groups
Using new bundles to take advantage of functionality in bug 243582.
Created attachment 161461 [details] patch
Test build running here https://build.eclipse.org/hudson/job/eclipse-equinox-test/132/
Another note, when this is implemented in the build, all the bundles will have to be retagged to include the changes to the manifest. This is because we use the comparator and new bundles are discarded if they have the same name and version as existing bundles in the repository.
I think we should wait until after M6 to release this, although the test builds are helpful to make sure the M6-based builder will be able to handle it.
In the interest of stability, I also agree we should wait until M7 to release the M6 builder. There will still be I-builds before EclipseCon that could be used to demo this new feature.
Looks like Hudson at Eclipse died and the build along with it. I'll try another build once it's working again.
The test build made it far enough, the manifest entries are missing from the bundles. I will need to run a test build locally to figure out what went wrong.
I have released a change for the next IBuild. Kim, you will want to pick that up for the next test build.
(In reply to comment #4) > I think we should wait until after M6 to release this, although the test builds > are helpful to make sure the M6-based builder will be able to handle it. Fully agree with this. Running the test builds now is extremely helpful to find issues and fix them, such that the feature can be properly adopted by others out of M6. Having M6 ship with the modified manifests is less important to me. I found the test build results hard to consume from Hudson. The single archive.zip was > 4 Gig in size and after a day of downloading the download stopped, missing the central ZIP directory :( I eventually managed to get build artifacts right from my shell on build.eclipse.org but the Hudson job should be improved to provide artifacts broken into smaller sizes rather than just an archive.zip On retagging: is there a chance to avoid this by clearing the reference repo that you compare against? Perhaps starting the M7 cycle with the fresh builder would help us avoid the need to retag all?
Martin, you don't have to download the entire archive.zip. That isn't useful and is way too large. It's just produced by Hudson convention. You can just look at the repository that's produced from the build. For example https://build.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test/lastSuccessfulBuild/artifact/builds/transfer/files/testUpdates-N/ Here's why we need to retag: We want to ensure that the bundles that users get via p2 update are the same as the ones in the zips on the build page. If we just clean out the repository and don't retag, the bundles that are produced with the new header in the manifest will have different content than the ones in existing user installs but have the same name and id. Consequently, users will not be prompted to update. The end result is that the source repository information will be missing in bundles that don't get updated during the rest of the 3.6 cycle. Andrew had a talk about this issue at EclipseCon last year. http://aniefer.blogspot.com/2009/08/versioning-p2-slides-from-eclipsecon.html
(In reply to comment #10) > Martin, you don't have to download the entire archive.zip. Good to know. I had started from the link posted in bug 243582 comment 79, (build #131) and it seemed to me as if only archive.zip was available at the time I tried. I tried a couple of navigations without success until I went to the shell. Of course there's no way proving any more that the artifacts were not accessible, since everything except for the logs seems to be cleared by now. > Here's why we need to retag: > We want to ensure that the bundles that users get via p2 update are the same Sure, this argument makes sense. I guess that strictly speaking, for a bundle that has not been changed since Eclipse 3.5 (not sure if any such old-timer exists), you'll also have to rev up the micro version then.
(In reply to comment #11) > Sure, this argument makes sense. I guess that strictly speaking, for a bundle > that has not been changed since Eclipse 3.5 (not sure if any such old-timer > exists), you'll also have to rev up the micro version then. Here are some ones that don't appear to have change between a run-of-the-mill Eclipse 3.5 and Eclipse.3.6M5 install I have lying around: org.eclipse.core.boot_3.1.100.v20080218.jar org.eclipse.core.runtime.compatibility.auth.source_3.2.100.v20090413.jar org.eclipse.core.runtime.compatibility.auth_3.2.100.v20090413.jar org.eclipse.core.runtime.compatibility.source_3.2.0.v20090413.jar org.eclipse.core.runtime.compatibility_3.2.0.v20090413.jar org.eclipse.equinox.concurrent_1.0.0.v20090520-1800.jar org.eclipse.equinox.http.jetty.source_2.0.0.v20090520-1800.jar org.eclipse.equinox.http.jetty_2.0.0.v20090520-1800.jar org.eclipse.equinox.jsp.jasper.registry.source_1.0.100.v20090520-1800.jar org.eclipse.equinox.jsp.jasper.registry_1.0.100.v20090520-1800.jar org.eclipse.equinox.jsp.jasper.source_1.0.200.v20090520-1800.jar org.eclipse.equinox.jsp.jasper_1.0.200.v20090520-1800.jar org.eclipse.equinox.util.source_1.0.100.v20090520-1800.jar org.eclipse.equinox.util_1.0.100.v20090520-1800.jar org.eclipse.help.appserver.source_3.1.400.v20090429_1800.jar org.eclipse.help.appserver_3.1.400.v20090429_1800.jar org.eclipse.jdt.launching.macosx.source_3.2.0.v20090527.jar org.eclipse.jdt.launching.macosx_3.2.0.v20090527.jar org.eclipse.jdt.launching.ui.macosx.source_1.0.0.v20090527.jar org.eclipse.jdt.launching.ui.macosx_1.0.0.v20090527.jar org.eclipse.team.cvs.ssh2.source_3.2.200.I20090508-2000.jar org.eclipse.team.cvs.ssh2_3.2.200.I20090508-2000.jar org.eclipse.ui.presentations.r21.source_3.2.100.I20081007-0800.jar org.eclipse.ui.presentations.r21_3.2.100.I20081007-0800.jar org.eclipse.ui.views.properties.tabbed.source_3.5.0.I20090429-1800.jar org.eclipse.ui.views.properties.tabbed_3.5.0.I20090429-1800.jar org.eclipse.ui.workbench.compatibility.source_3.2.0.I20090429-1800.jar org.eclipse.update.core.source_3.2.300.v20090525.jar org.eclipse.update.core_3.2.300.v20090525.jar org.eclipse.update.scheduler.source_3.2.200.v20081127.jar org.eclipse.update.scheduler_3.2.200.v20081127.jar org.eclipse.ui.workbench.compatibility_3.2.0.I20090429-1800: I had a look at the nightlies and didn't see the Eclipse-SourceReferences tag generated on the individual JARs. Is it post-processed when packing up into contents.jar or artifacts.jar?
Alex, we didn't implement this in the build yet. This would require changing all the bundles in the builder and this was too risky of a change to implement during milestone week. We will have new bundles in the builder next week. I was just running test builds earlier.
(In reply to comment #12) > I had a look at the nightlies and didn't see the Eclipse-SourceReferences tag > generated on the individual JARs. Is it post-processed when packing up into > contents.jar or artifacts.jar? The last test build at comment #7 was unsuccessfull in that the headers were missing from the manifest. I have since fixed a bug here (comment #8) and got it working for me locally, but there has not been time to perform another full releng test.
I ran a test build on Friday with new bundles in the builder. It looks like the manifests have the source headers. Example: Eclipse-SourceReferences: scm:cvs:pserver:dev.eclipse.org:/cvsroot/rt: org.eclipse.equinox/framework/bundles/org.eclipse.osgi;tag=HEAD Here's the test build https://build.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test/lastSuccessfulBuild/artifact/builds/transfer/files/testUpdates-N/N20100312-1851/ Let me know if this looks good and I'll release this to the real build.
The headers look good to me. I successfully imported a few bundles. It's questionable as to whether we should be putting an explicit "HEAD" tag in the headers (that should be the default when unspecified), but we can track that in a different bug, and clean it up if needed.
Andrew, I ran a build with the latest M6 bundles and the source references weren't generated. The build directory is here... /builds/N201003151329 on eclipsebuildserv. The earlier test build was with bundles from a build last week.
(In reply to comment #16) > The headers look good to me. I successfully imported a few bundles. It's > questionable as to whether we should be putting an explicit "HEAD" tag in the > headers (that should be the default when unspecified), but we can track that in > a different bug, and clean it up if needed. They look good to me too. Added bug 306637 to track the addition of HEAD.
(In reply to comment #17) > Andrew, I ran a build with the latest M6 bundles and the source references > weren't generated. The build directory is here... > > /builds/N201003151329 > > on eclipsebuildserv. > > The earlier test build was with bundles from a build last week. Kim, can you try this again, N201003151329 is gone from the build machine. I ran a test on my own machine and it worked.
(In reply to comment #19) > (In reply to comment #17) > Kim, can you try this again, N201003151329 is gone from the build machine. I > ran a test on my own machine and it worked. Andrew, can you commit the patch in bug 306637 to stop generating the HEAD tag?
Andrew, I ran it again and it didn't have the source references Here is the repo /builds/transfer/files/testUpdates-N2/N20100331-1018/ Or the build directory /builds/N201003311018
Kim, is any portion of the fetching happening in parallel? Particularly, the examples vs the master? The properties file that is supposed to contain all the references has only examples in it. The fetch generator is supposed to append to any content that was already there instead of replacing it.
Created attachment 164434 [details] patch patch to add extssh to cvsuser to source references are generated correctly. Also, update generated sourceReferences.properties to reflect pserver:anonymous instead of extssh:kmoir postFetch
Note: The reason we didn't run into bug 308696 before was that I was running the test builds from Hudson, where we can use a live copy of the CVS pserver repo. On the build machine here, the build updates the maps to use :extssh:kmoir to check out source via extssh and avoid any latency in updates to the pserver copy of the repo available to the general public.
Created attachment 164634 [details] patch
Ran a build with bundle's from yesterday's I-build to address bug 308696 Darin, can you take a look at it and see if it is as you expect http://eclipsebuildserv.ottawa.ibm.com/downloads/bogus/downloads/drops/N20100415-0804/index.php If the bundles look okay to you, I'll release for tonight's build.
Roger Houston. Looks good.
Okay, released for N20100415-2000. We'll have to ask everyone to tag all their bundles if we want this information to be in all the manifests for M7. As I mentioned before, we compare bundles with the other child repos in the repository, and toss the new ones if they have the same version as existing ones in the repo. This ensures that the user that installs the bundles from the repo and the one who installs them from a zip get the same content.
