Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 279056

Summary: SignatureVerifier does not like VE signed jars, GEF jars OK
Product: z_Archived Reporter: Nick Boldt <nboldt>
Component: VEAssignee: VE Bugzilla inbox <ve-inbox>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: aniefer, dash.commonbuilder-inbox, kim.moir, pascal, ve-inbox
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on:    
Bug Blocks: 270849, 272109    
Attachments:
Description Flags
console log w/ p2 errors
none
build log none

Description Nick Boldt CLA 2009-06-04 01:06:38 EDT
Created attachment 138239 [details]
console log w/ p2 errors

Signed jars appear to be problematic for p2, and while I don't want to blame it unnecessarily, I'm not sure where else to turn. Consider this a cry for help. :)

We're using the R35_M7 tag of basebuilder to run builds at the moment, but VE 1.4 is built against Eclipse 3.4.2, not 3.5.

Linux Tools 

--

Using the Athena build process, I can build GEF, sign it, and install from the resulting update site (zip or unpacked). No problem.

https://build.eclipse.org/hudson/job/cbi-gef-3.5.x-nightly/81/artifact/build/I200905301628/

Then, using the identical process, I can build Linux Tools or VE, but if I sign them, I get errors such as this:

org.eclipse.osgi.signedcontent.InvalidContentException: The file "org/eclipse/ve/internal/swt/GridLayoutComponentPage.class" in the jar "/tmp/signatureFile4177882853734188736.jar" has been tampered!
	at org.eclipse.osgi.internal.signedcontent.DigestedInputStream.verifyDigests(DigestedInputStream.java:102)
	at org.eclipse.osgi.internal.signedcontent.DigestedInputStream.read(DigestedInputStream.java:129)
	at org.eclipse.osgi.baseadaptor.bundlefile.BundleEntry.getBytes(BundleEntry.java:102)
	at org.eclipse.osgi.internal.signedcontent.SignedContentImpl$SignedContentEntryImpl.verify(SignedContentImpl.java:167)
	at org.eclipse.osgi.internal.signedcontent.SignedContentFile.getSignedEntries(SignedContentFile.java:51)
	at org.eclipse.equinox.internal.p2.artifact.repository.SignatureVerifier.verifyContent(SignatureVerifier.java:82)
	at org.eclipse.equinox.internal.p2.artifact.repository.SignatureVerifier.verify(SignatureVerifier.java:59)
	at org.eclipse.equinox.internal.p2.artifact.repository.SignatureVerifier.close(SignatureVerifier.java:100)
	at org.eclipse.equinox.internal.provisional.p2.artifact.repository.processing.ProcessingStep.close(ProcessingStep.java:83)

Here's the complete list of classes with an InvalidContentException:

org/eclipse/ve/internal/jfc/core/ComponentProxyAdapter.class
org/eclipse/ve/internal/jfc/core/NonBoundsBeanPropertySource.class
org/eclipse/ve/internal/jfc/core/JFCPreferencePageContents.class
org/eclipse/ve/internal/jfc/core/JFCVisualPlugin.class
org/eclipse/ve/internal/jfc/core/BorderLayoutConstraintsPropertyEditor.class
org/eclipse/ve/internal/jfc/core/ComponentProxyAdapter$2.class
org/eclipse/ve/internal/jfc/core/ChoiceProxyAdapter.class
org/eclipse/ve/internal/jfc/core/LayoutManagerCellEditor.class
org/eclipse/ve/internal/jfc/core/ComponentManager$3.class
org/eclipse/ve/internal/jfc/core/ComponentProxyAdapter$1.class
org/eclipse/ve/internal/jfc/core/JTableProxyAdapter.class
org/eclipse/ve/internal/swt/GridLayoutComponentPage.class
org/eclipse/jem/internal/proxy/swt/DisplayManager$DisplayRunnable.class
org/eclipse/ve/internal/rcp/WorkbenchPartGraphicalEditPart$ConstraintHandler$VisualComponentListener.class
org/eclipse/ve/internal/swt/ImageController.class
org/eclipse/ve/internal/jface/codegen/ViewerConstructorDecoderHelper.class
org/eclipse/ve/internal/swt/ImageController$ImageFileFilter.class

The full Eclipse console log is attached.

Perhaps related to this, I get this error during the signing process:

[Fatal Error] :1:347: An invalid XML character (Unicode: 0x13) was found in the element content of the document.
Comment 1 Nick Boldt CLA 2009-06-04 01:11:54 EDT
Created attachment 138240 [details]
build log 

The attached log shows 19 hits for 

[Fatal Error] :1:347: An invalid XML character (Unicode: 0x13) was found in the element content of the document.

where "347" may be a number between 335 and 349 (mostly 347).
Comment 2 Pascal Rapicault CLA 2009-06-04 11:07:05 EDT
Did you try to manually download the jar, and run jarsigner -verify. Also did you try downloading the pack.gz, unpack and run jarsigner -verify on it. What do you get?
Comment 3 Nick Boldt CLA 2009-06-04 12:11:17 EDT
Correction. Managed to install LT sans probleme last night; seems this is now localized to VE. Removing GEF and LT from cc: list; renaming bug.

Could this be related to:

line endings?
file encoding?
special chars in the class files (?!?) which the signer can't grok?

Does anyone in p2 have any idea what to look for? 

VE folks, can you look at the classes listed below and see if there's something weird in there that could be causing the problem? non-UTF or non-ASCII chars, maybe? 

(In reply to comment #2)
> Did you try to manually download the jar, and run jarsigner -verify. Also did
> you try downloading the pack.gz, unpack and run jarsigner -verify on it. What
> do you get?

After unpacking, I get: 

$ jarsigner -verify org.eclipse.ve.jfc_1.4.0.v20090505-2035.jar
jarsigner: java.lang.SecurityException: SHA1 digest error for org/eclipse/ve/internal/jfc/core/BorderLayoutConstraintsPropertyEditor.class

That's one of the classes with an InvalidContentException, quelle surprise. :)

You don't think there'd be a difference between the jar and the packed jar, do you?
Comment 4 Kim Moir CLA 2009-06-04 12:34:44 EDT
This looks similar to bug 277834 and bug 232008.  There are issues with packing using a 1.6 vm so we force our build to use a 1.5 vm by setting this system property -Dorg.eclipse.update.jarprocessor.pack200=$builderDir/jdk/linux/jdk1.5.0_14/bin in our bootstrap script. 

We have to do this because our build is invoked using a 1.6 vm.

See org.eclipse.releng.eclipsebuilder\bootstrap.sh in /cvsroot/eclipse

Comment 5 Andrew Niefer CLA 2009-06-04 12:45:43 EDT
The platform in the past has occasionally encountered jars where pack200/unpack screws up on particular class files.  The fix for this has usually been to lower the effort level used for packing.  The platform is using an effort level of 4 (instead of the default 5).

Do this either by specifying a META-INF/eclipse.inf in the bundle with 
pack200.args=-E4
or using a pack.properties file with
pack200.default.args=-E4

See http://wiki.eclipse.org/JarProcessor_Options for details

Based on comment #3 this is not a problem with p2, so sending to VE
Comment 6 Andrew Niefer CLA 2009-06-04 13:01:06 EDT
Re comment #4,
The pack200 backwards compatibility is based on the presence of 1.5 .class files in the jar.  If you are using a 1.6 vm to pack, and these jars contain 1.6 .class files, then they likely won't be unpackable by 1.5.  (We hit this problem on source bundles that don't contain .class files, and on a couple of jdt bundle that have 1.6 classes)

Comment 7 Nick Boldt CLA 2009-06-04 13:10:53 EDT
Thanks for the suggestions, but we're already packing w/ 5.0 and -E4.

(In reply to comment #4)
> This looks similar to bug 277834 and bug 232008.  
> we force our build to use a 1.5 vm by setting this system
> property
> -Dorg.eclipse.update.jarprocessor.pack200=$builderDir/jdk/linux/jdk1.5.0_14/bin
> in our bootstrap script. 

We're using 5.0 JVM to pack.

[echo] Pack with /opt/public/common/ibm-java2-ppc-50/jre/bin/pack200

(https://build.eclipse.org/hudson/job/cbi-ve-1.4-integration-Ganymede/lastSuccessfulBuild/consoleFull)

> The platform in the past has occasionally encountered jars where pack200/unpack
> screws up on particular class files. 
> Do this either by specifying a META-INF/eclipse.inf in the bundle with 
> pack200.args=-E4
> or using a pack.properties file with
> pack200.default.args=-E4

Been there, doing that. As we've been packing for a couple years and I adopted the Platform's approach, I use:

@pack200@ -v -l packlog.txt -E4 -J-Xmx768m $*

And in pack.properties:

pack200.default.args=-E4
pack.excludes=eclipse/plugins/@excludejars@

---

I wonder if the problem could be nested packed jars in the plugin? 

org.eclipse.ve.jfc_1.4.0.v200906021203.jar
  \vm\jfcvm.jar.pack.gz
  \vm\jfcbeaninfo.jar.pack.gz

Should this plugin be refactored to remove the nested jars? Would that help?

---

Otherwise, I see two options here:

a) enhance Athena to support a mix of packed and unpacked jars when creating the Update zip - currently it's all packed or none packed (if skipPack=true).

b) disable packing entirely for VE builds (with skipPack property) - tested this with a local build and I can confirm that packing screws up the jar.

---

Meanwhile, even with an unpacked build, I still get those "An invalid XML character (Unicode: 0x13)" errors...
Comment 8 Andrew Niefer CLA 2009-06-04 13:43:43 EDT
I don't think this would be a nested jar problem.

I'm guessing the :1:347: means character 1 on line 347.
I'm not sure who is producing those.

Do you have the jars from after signing?  Do they fail to verify as well?
Comment 9 Nick Boldt CLA 2009-06-04 21:22:27 EDT
(In reply to comment #8)
> I don't think this would be a nested jar problem.

Solved locally by suppressing packing and signing of nested jars, and packing of the containing jar.

I've added this to eclipse.inf for ve.jfc and ve.swt plugins:

jarprocessor.exclude.pack=true
jarprocessor.exclude.children=true

Need to verify in a Hudson build, so I've changed the map to use HEAD instead of the May 5 tag. 

Still getting the "(Unicode: 0x13)" thing, but if that doesn't prevent users from installing the goods, I'm not worried. 
Comment 10 Nick Boldt CLA 2009-06-05 01:18:10 EDT
> enhance Athena to support a mix of packed and unpacked jars when creating
the Update zip - currently it's all packed or none packed (if skipPack=true).

This extra granularity is now supported - individual jars can be set to not get packed and will still end up in the Update zip.

Verified that VE can now be installed from update site on download.eclipse and from downloaded p2 repo zip.