Bugzilla – Full Text Bug Listing
| Summary: | Many "Unable to resolve partial IU" errors for Helios SR1 aggregation | ||
|---|---|---|---|
| Product: | Community | Reporter: | David Williams <david_williams> |
| Component: | Cross-Project | Assignee: | Thomas Hallgren <thomas> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | ahunter.eclipse, filip.hrbek, laurent.goubet, nicolas.bros, thomas |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Windows 7 | ||
| Whiteboard: | |||
|
Description
David Williams
Anthony, I notice that at your repo site, ~/downloads/modeling/gmp/updates/milestones that you do have a site.xml file. Can you try renaming it? I think site.xml is supposed to be read only as a last resort .... but wonder if something (else) is broken and causing it to be read instead of the content.jar, thus causing this indirect error? Actually, now that I look at it ... I don't see a content.jar/xml file at your repo site, ~/downloads/modeling/gmp/updates/milestones Is there a step missing? Laurent, I know you were having this problem too, and I see too there is no "content.jar" at your repository. modeling/m2t/updates/milestones/ I bet you have a common build with GMF? :) David, Made a last try this morning by replacing the "feature id" automatically generated for the b3aggrcon (org.eclipse.acceleo.sdk.feature.group") which doesn't correspond to anything in our update site by a "real" feature id (org.eclipse.acceleo.sdk) corresponding to what we had in the old .build files. As I can't seem to trigger a launch of the helios build, I'll wait and see what the next build says. If it fails anew, I'll simply re-disable the contribution and let it lie until someone understand what this error message mean. I lost 13 hours yesterday trying to get the build to pass ... I guess providing a way for projects to contribute the old "build" file (that worked wonderfully for us) is impossible? Unsurprinsingly, that failed again. Disabling Acceleo contribution from the build until someone finds a meaning to that cryptic error message. The aggregator does support old legacy sites (i.e. site.xml). What happens is that when p2 encounters such sites, it creates incomplete proxys for the meta-data. These proxies are the "partial IU's" that the aggregator later attempts to resolve and replace with fully fledged ones. Judging form the log, the source of the problem here is not that it is a legacy site, it is that the site contains bundles that contain no manifest (seems to be generated source bundles). Thomas, The update site Acceleo uses (http://download.eclipse.org/modeling/m2t/updates/milestones/) is the aggregate update site for modeling/m2t, we don't have a custom site. It may indeed contain bundles with no manifest, but I know for sure (downloaded and checked the SDK 3 minutes ago) that every single one of the Acceleo plugins and bundles contain a META-INF/MANIFEST.MF entry, including generated source bundles. Perhaps your site is not the culprit? The offending bundle is org.apache.batik.bridge.source. Someone contributes that through a legacy update site. The error handling in the aggregator could be improved to find the culprit contribution and send proper emails for this kind of error. I added bug 322480 to cover that. (In reply to comment #8) > Perhaps your site is not the culprit? > > The offending bundle is org.apache.batik.bridge.source. Someone contributes > that through a legacy update site. Thomas, that's for the latest builds; for build 265 (https://build.eclipse.org/hudson/job/helios.runAggregator/265/console) the offending bundle was "org.eclipse.acceleo.common" and earlier builds (262 or 263, can't remember) failed because of "org.eclipse.emf.compare.core" (which is on yet another update site) because of this very same "no bundle manifest" reason. I don't find org.eclipse.acceleo.common, version: 3.0.0.v201008110958 in the plugins folder of your site. Perhaps the error message is caused by a missing bundle? (In reply to comment #11) > I don't find org.eclipse.acceleo.common, version: 3.0.0.v201008110958 in the > plugins folder of your site. Perhaps the error message is caused by a missing > bundle? Nope, it's just that I contributed a new Acceleo build since then to try and fix this. build 268 is the latest (I think) that failed explicitely because of me : https://build.eclipse.org/hudson/job/helios.runAggregator/268/console . the "real qualifier is 3.0.1 v201008111222 which is present on the site When I explore the update site http://download.eclipse.org/modeling/emf/updates/milestones (from helios.b3aggr), I see that its bundles have a property org.eclipse.equinox.p2.partial.iu = true. And when trying to build MoDisco, the bundles from this update site are not found by Buckminster. I believe this is related to the Helios aggregation errors. OK, I think I have this nailed down. There was a regression introduced in the aggregator while fixing a bug related to how properties were copied between artifact descriptors during mirroring. This lead to some confused behavior when mirroring the partial IU's. so that the packed artifact was fed into the logic that extracted the manifest. That didn't play out well. This bug is now fixed and a new build of the aggregator is in progress. https://build.eclipse.org/hudson/view/Modeling/job/emft-b3-build/159/console David, can you please install that and try it when the build is finished? (In reply to comment #13) > I believe this is related to the Helios aggregation errors. Nicolas, is the b3 aggregator used in the MoDisco build? I don't think that the fact that Buckminster doesn't find the bundles are related to this error so if the b3 aggregator isn't used, can you please file a Bugzilla on Buckminster for that? Preferable with a sample that reproduces the problem. (In reply to comment #15) > Nicolas, is the b3 aggregator used in the MoDisco build? No, MoDisco uses Buckminster but not the b3 aggregator. I created Bug 322506 - buckminster fails to find bundle in update site > > David, can you please install that and try it when the build is finished? I tried the repos we were using, but seemed to be same aggregator: [exec] Installing org.eclipse.b3.aggregator.engine.feature.feature.group 0.1.0.r01053. [exec] -metadataRepository http://download.eclipse.org/modeling/emft/b3/headless-3.6/ [exec] -artifactRepository http://download.eclipse.org/modeling/emft/b3/headless-3.6/ Should I switch to repo at http://download.eclipse.org/modeling/emft/b3/headless-3.7milestones Will that work with Eclipse 3.6? Or do I need to update to 3.7 M1 as well? Ah, sorry. We don't auto promote the build anymore. Can you please try this site: https://build.eclipse.org/hudson/view/Modeling/job/emft-b3-build/lastSuccessfulBuild/artifact/site/headless/ (In reply to comment #18) > Ah, sorry. We don't auto promote the build anymore. Can you please try this > site: > > https://build.eclipse.org/hudson/view/Modeling/job/emft-b3-build/lastSuccessfulBuild/artifact/site/headless/ Actually, I see same build/feature ID from either repo ... [exec] Installing org.eclipse.b3.aggregator.engine.feature.feature.group 0.1.0.r01247. [exec] -metadataRepository http://download.eclipse.org/modeling/emft/b3/headless-3.7milestones [exec] -artifactRepository http://download.eclipse.org/modeling/emft/b3/headless-3.7milestones [exec] Installing org.eclipse.b3.aggregator.engine.feature.feature.group 0.1.0.r01247. [exec] -metadataRepository https://build.eclipse.org/hudson/view/Modeling/job/emft-b3-build/lastSuccessfulBuild/artifact/site/headless/ [exec] -artifactRepository https://build.eclipse.org/hudson/view/Modeling/job/emft-b3-build/lastSuccessfulBuild/artifact/site/headless/ Yes, they are the same. What I meant was that we don't promote to the 3.6 site. We haven't officially spawned off a 3.6 branch yet and the build is still based on Helios. This seems to have resolved the "Unable to resolve ..." problem. The build got past the verification phase and is now mirroring. (I re-enabled acceleo even, but had to disable linuxtools, for apparently unrelated problem). My guess is this didn't show up before because teams were not contributing legacy update sites before. In either case, I think they certainly _should_ be contributing P2 repos instead of legacy sites, so Anthony and Laurent, please investigate that (or, state your intent that you only contribute legacy update sites ... tsk tsk :) But we'll count this as fixed and move on. (In reply to comment #21) > My guess is this didn't show up before because teams were not contributing > legacy update sites before. In either case, I think they certainly _should_ be > contributing P2 repos instead of legacy sites, so Anthony and Laurent, please > investigate that (or, state your intent that you only contribute legacy update > sites ... tsk tsk :) I am using a common modeling promote script that updates the P2 repo, or at least what I think is a P2 repo. How do I tell if it is a "real" P2 repo based on the output below? Is just the presence of the site.xml indicate it is not a "real" P2 repo? [update] Started 16:10:51. Executing with the following options: -f /tmp/tmp.eXFLo28991/promoteToEclipse.gmf-runtime.properties [loading, sub = gmf-runtime ... done] -user ahunter -Q -branch 1.4.1 -buildID S201008111427 -promote -debug 0 -parent modeling [update] [16:10:51] Check out www/modeling/gmp/updates/* from :ext:ahunter@dev.eclipse.org:/cvsroot/org.eclipse [update] [16:10:54] Unpack gmf-runtime-Master-1.4.1RC1.zip ... [update] [16:10:54] Generate site XML for S201008111427 (use -debug 1 for details). [update] [16:11:05] Copy new jars & site/* to /var/www/html/modeling/gmp/updates/milestones/gmp-gmf-runtime-1.4-S_rN ... [update] [16:11:06] Purge old versions in /var/www/html/modeling/gmp/updates/milestones [compose] [16:11:06] Delete existing features & plugins in /var/www/html/modeling/gmp/updates/milestones ... [compose] [16:11:06] Merge 3 components' jars into /var/www/html/modeling/gmp/updates/milestones/ [compose] [16:11:07] Merge 1 components' category*.xml into single site.xml [fix zip] [16:11:07] Generate Update zip ... [ p2 ] [16:11:07] Generate /var/www/html/modeling/gmp/updates/milestones/gmp-gmf-runtime-1.4-S_rN/artifacts.jar and content.jar ... [ p2 ] [16:11:23] 1.4.1/S201008111427/gmf-runtime-Update-1.4.1RC1.zip [compose] [16:11:24] Merge 3 components' category*.xml into single site.xml [compose] [16:11:24] Run siteOptimizer to create /var/www/html/modeling/gmp/updates/milestones/digest.zip ... [ p2 ] [16:11:27] Generate /var/www/html/modeling/gmp/updates/milestones/artifacts.jar and content.jar ... [promote] [16:11:53] Check in site.xml as /updates/milestones/site.xml ... [promote] [16:11:55] Create /var/www/html/modeling/gmp/updates/gmp-gmf-runtime-1.4-S-S201008111427.zip ... [promote] Zip contains 317 files and is 27M. [promote] [16:11:57] Promote zip to ahunter@build.eclipse.org:~/gmp-gmf-runtime-1.4-S-S201008111427.zip ... [promote] [16:12:08] Replace content in REMOTE update site (/home/data/httpd/download.eclipse.org/modeling/gmp/updates/milestones) ... [promote] Connected to build.eclipse.org. Unpack ~/gmp-gmf-runtime-1.4-S-S201008111427.zip ... [promote] Remove old jars ............................................ [compose] [16:14:05] Merge 6 components' category*.xml into single site.xml [compose] [16:14:06] Run siteOptimizer to create /home/data/httpd/download.eclipse.org/modeling/gmp/updates/milestones/digest.zip ... [ p2 ] [16:14:07] Generate /home/data/httpd/download.eclipse.org/modeling/gmp/updates/milestones/artifacts.jar and content.jar ... [promote] Remove temp zip & jarlist*.txt ... done. [promote] Fix perms ... done. [promote] [16:14:08] SCP Update zip & md5 to remote box... done [16:14:16]. [compose] [16:14:16] Local update site is: /var/www/html/modeling/gmp/updates/milestones
>
> I am using a common modeling promote script that updates the P2 repo, or at
> least what I think is a P2 repo. How do I tell if it is a "real" P2 repo based
> on the output below? Is just the presence of the site.xml indicate it is not a
> "real" P2 repo?
>
Its not so much a site.xml file is present. That's actually ok, and some people do that on purpose, if they want a site that works with old update manager.
But to have a "real" P2 repo, you need a content.jar or content.xml file at the URL that you point P2 at.
(I'm not sure how to tell that from the output you have .... I just looked on the file system and didn't see a content.jar/xml file there.)
I am also using a common modeling promote script for both EMF Compare and Acceleo, this will go on for the next couple milestones/RC until we've completed our migration to bucky builds. If memory serves, we'll also have our own update sites once we've finished moving, thus this problem shouldn't happen again. |