Bugzilla – Full Text Bug Listing
| Summary: | several "can not unpack" errors ween when running aggregator | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Technology] CBI | Reporter: | David Williams <david_williams> | ||||
| Component: | CBI p2 Repository Aggregator | Assignee: | Project Inbox <b3.aggregator-inbox> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | P3 | CC: | filip.hrbek, jcayne, slamers, thomas | ||||
| Version: | unspecified | ||||||
| Target Milestone: | --- | ||||||
| Hardware: | PC | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Bug Depends on: | 312175, 312181 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
Hi Dave, I'm not sure, but I think I recognize this as issues with J9 and unpack200. Can you please try with another VM by passing: -Dorg.eclipse.update.jarprocessor.pack200=<path to JVM installation>/jre/bin The buckyBuilder at build.eclipse.org explicitly use a very old unpack200 from /shared/common/ibm-java2-ppc-50/jre/bin to circumvent problems with more recent JVM's. The new 64 bit 1.6 SR7 is OK but I wouldn't trust anything in between. Perhaps you could also try running the old Helios builder locally? I wouldn't be surprised if it shows the same symptoms. I base that assumption on that the unpack failure occurs deep down in the p2 code and AFAIK, that part of the code has not undergone any changes for a very long time. To sum it up, I think this is caused by a difference in environment rather than between the b3 aggregator and the old buckyBuilder. When looking more closely at the problem with the TPTP artifact repository I found that it was caused by a repository reference in the TPTP Helios contribution. The reference is not enabled (it's status is 0) and that puzzled me for a while until I discovered the root cause (bug 312175). This generates the stack-trace but shouldn't affect the end result. The exception is caught, logged, and then forgotten. The following information might be useful to the TPTP team. The reason p2 logs a stack-trace is that: 1. No artifact.(jar|xml) or compositeArtifact.(jar|xml) was found at http://download.eclipse.org/tptp/updates/. A site.xml is present so p2 falls back to that. 2. The site.xml contains a reference to the feature named org.eclipse.tptp.test version 4.3.0.v200610310100--4hNDDhHNFUEMEM 3. p2 assumes that it is placed under the folder 'features' and in a file called: org.eclipse.tptp.test_4.3.0.v200610310100--4hNDDhHNFUEMEM.jar 4. That file doesn't exist. So there are two problems with the TPTP contribution: 1. The tptp helios repo contains a reference to a repo that has meta-data but no artifacts 2. The site.xml at the appointed locatoin is in error. Update! No need to try with another JVM. Filip just got the same errors. I just realized why this happens. When running the b3 aggregator we don't include the 'trustedContributions' option which means that the platform artifacts are copied and unpacked. The buckyBuilder that builds Helios runs with "-trustedContributions Eclipse,Equinox", and thus, does not copy or unpack these artifacts. This means two things: 1. This option should be used when running the b3 aggregator. 2. It's highly likely that we discovered a bug in some of the platform binaries. They can obviously not be unpacked by p2. This deserves a bugzilla of its own. We have our own Hudson where we run tests in parallel. A test using -trustedContributions is in progress... One thing that should be noted is that --trustedContributions is mutually exclusive to ---mavenResult. When we want the maven artifacts, we must make a decision to either: - Mirror everything, including the platform, into the new repository (where artifacts are stored according to Maven layout rules) or - Use the platform as a validation repository only. This means that it will only be referenced so categories etc. cannot change. This approach implies that the platform repository is somehow mavenized too in order for the Helios repository to be usable by maven consumers. Hi Thomas, Thanks for the info about the TPTP repository. We currently use an incremental update site which contains an artifacts.(jar/xml) file in each individual TPTP release directory, but not in the main "root" level. We are looking into creating a composite repository so that we have a compositeArtifacts.jar at the root level. With regards to the site.xml, we are using it for backwards compatibility with old releases. I am a little confused about the missing jar you mentioned. That jar is from an old TPTP release (Eclipse 3.2). It exists in the updates/4.3.0/features directory (which is where the site.xml points to in the "url" attribute). Should we be doing something differently (given that we are using an incremental update site and that there is no root level "features" directory)? Hi Sean, My previous conclusion was wrong. Your site.xml is probably ok. The problem lies in the actual feature. Take a look at this file: http://download.eclipse.org/tptp/updates/4.3.0/features/org.eclipse.tptp.test_4.3.0.v200610310100--4hNDDhHNGZEMEM.jar Pay special attention to the qualifier used in the filename (and also in the site.xml). It is: v200610310100--4hNDDhHNGZEMEM Now open the jar file and look at the qualifier inside of the feature.xml. It's almost the same but not quite:: v200610310100--4hNDDhHNFUEMEM That is probably the root of the problem. Thanks Thomas. We can't modify the jar since it has already been released as part of TPTP 4.3.0 (Eclipse 3.2.1). Recent versions of the org.eclipse.tptp.test feature do not have this problem. Other than showing up in the logs, is this causing any problems? It might cause problem yes. The reason is that since the artifacts.(jar|xml) is missing from the root, p2 is forced to use it's backward compatibility layer for reading legacy sites. This layer actually opens the feature jar and looks at the feature.xml in order to find the plugins that it references. The solution might be simple though. Just place a compositeArtifacts.xml file in the root and have it include all children artifact repositories. That way you will ensure that the backward compatitility layer never comes into play. I wouldn't be surprised if it also makes accesses to tptp a whole lot faster. I'm closing this a fixed since I don't think we have any outstanding issues that relates to the problem described here. [Bookkeeping change only. Moving bugs to the new "home" of aggregator, CBI. No change to assignee for resolved and verified bugs.] |
Created attachment 167596 [details] log showing some errors I'll attach log, but shortly after doing a successful build with buckyBuilder on eclipse.org, I tried running batch aggregator on my local machine, and got several "could not unpack" errors in the log. Several from platform plugins ... which may be do not go through the same verification in buckyBuilder as with aggregator? I've not look at it in detail, and the job is still running, so there might be other issues ... maybe easy explainations ... but, I'm going to bed soon :) and with time zone differences, thought I'd open a bug with what I know, and see if it means anything to you. I'm also not sure why it'd say the tptp file wasn't found (since ran on eclipse.org shortly before) ... but that I'm more willing to believe is a glitch. signed files with invalid content are a bit scarier. Insights/advice welcome.