Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 378234

Summary: CBI build should produce a comparable build output repo
Product: [Technology] CBI Reporter: Paul Webster <pwebster>
Component: artifact repositoryAssignee: Thanh Ha <thanh.ha>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: andrea.ross, contact, david_williams, igor, john.arthorne, loskutov, t-oberlies, thanh.ha, tjwatson
Version: 1.0   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on: 385529, 386266, 386646, 386649    
Bug Blocks: 372792    
Attachments:
Description Flags
3.8 M6 p2 repo file list
none
Juno_M6_R3 p2 repo file list
none
Files, most missing from the CBI m6 repo
none
Java project that can compare 2 jars
none
Diff of 4.2 fina repo against the Juno_RC4_R4 branch
none
diff of javap -v output for EclipseStarter.class
none
diff of javap -v output for EclipseStarter.class, conditioned
none
core.commands BREE patch
none
classes different from baseline
none
classes different from baseline
none
proposed changes to p2 pom.xml files none

Description Paul Webster CLA 2012-05-02 08:00:50 EDT
The current CBI build produces a p2 repo with 461 files.  The 3.8 M6 repo has 1348 files.  That means there's still a lot of stuff not being built.
Comment 1 Paul Webster CLA 2012-05-02 08:01:31 EDT
Created attachment 214915 [details]
3.8 M6 p2 repo file list
Comment 2 Paul Webster CLA 2012-05-02 08:02:12 EDT
Created attachment 214916 [details]
Juno_M6_R3 p2 repo file list
Comment 3 Paul Webster CLA 2012-05-02 08:11:12 EDT
(In reply to comment #1)
> Created attachment 214915 [details]
> 3.8 M6 p2 repo file list

This has a lot of .pack.gz.  Without those it only has 772 files in it.

PW
Comment 4 Paul Webster CLA 2012-05-02 08:16:24 EDT
Created attachment 214918 [details]
Files, most missing from the CBI m6 repo

After removing versions with sed 's/[0-9]*\.[0-9]*\.[0-9]*\..*$//g' this is the diff.

PW
Comment 5 Paul Webster CLA 2012-05-02 09:41:14 EDT
Created attachment 214924 [details]
Java project that can compare 2 jars

I've modified a version of the p2 jar comparator using java and bash scripts so we can run it now against our CBI p2 build repo and the M6 repo.

It reports differences in the manifests, as well as class files.

bash$ /bin/bash \
/opt/pwebster/workspaces/build42x/z.ex.compare/scripts/comparePlugins.sh \
-platformRepo $(pwd)/S-3.8M6-201203141800 \
-cbiRepo $(pwd)/cbi-m6 \
-eclipseRoot /opt/local/e4-self/I20120430-1800/eclipse

PW
Comment 6 Paul Webster CLA 2012-05-02 09:47:58 EDT
I've generated a report of the jar comparisons of the plugins of the 3.8 M6 repo against the build output repo of Juno_M6_R3

http://build.eclipse.org/eclipse/cbi/binaryCompare-v01.txt

These 3 files are related to signing AFAIK so it makes sense they're missing: META-INF/ECLIPSEF.SF META-INF/ECLIPSEF.RSA META-INF/eclipse.inf

Other files listed as missing are truly missing, like the .api_description file, or plugin.properties in the source plugins.

===

The class org/eclipse/ui/editors/text/StorageDocumentProvider.class is different

This message indicates their is a difference in disassembled byte codes, signatures, or class file versions between the 2 classes.

The 3.8 M6 repo was conditioned for signing, so hopefully when we condition and sign our CBI build repo most of these will go away ... but it will be nice to know for sure.

The next step would be to turn on conditioning/signing and then re-run the comparison to see how close we are.

PW
Comment 7 Thanh Ha CLA 2012-06-15 13:33:00 EDT
I ran the compare script against the R4 JunoRC4 platform with signing enabled. The results can be found here: 

  http://build.eclipse.org/eclipse/cbi/binaryCompare-RC4_R4.txt
Comment 8 Andrea Ross CLA 2012-07-09 21:55:24 EDT
Marking this bug as CBI 1.0
Comment 9 Paul Webster CLA 2012-07-19 12:03:23 EDT
Created attachment 218941 [details]
Diff of 4.2 fina repo against the Juno_RC4_R4 branch

There are a couple of categories of differences:

There are 805 bundles in 4.2, compared to 505 in the aggregator.

The tests are missing from the aggregator repo.
The examples are missing from the aggregator repo.
These 2 account for approximately 70 and 50 bundles respectively.

It only picks up 5 SWT platforms instead of all of them.
It only picks up 5 launcher fragments instead of the ~13.
It only picks up some of the core.filesystem fragments instead of all of them.

Most of org.eclipse.equinox.executable_root* is missing.

Groups of bundles like org.eclipse.equinox.weaving* and org.eclipse.equinox.transforms* and org.eclipse.equinox.server/servlet* are missing.

org.eclipse.osgi.configuration_root* is missing.

org.eclipse.pde.api.tools* is missing.

org.eclipse.rcp.configuration_root* is missing.
Comment 10 Paul Webster CLA 2012-07-19 12:08:25 EDT
Created attachment 218942 [details]
diff of javap -v output for EclipseStarter.class

Running Juno/R42 with -X we get a lot of "baseline and reactor have same version but different contents".

ex, org.eclipse.osgi.  I did a javap of EclipseStarter, and there are significant differences in the disassembly.  Maybe because our repo was (mostly) conditioned.

[WARNING] MavenProject: org.eclipse.equinox.framework:org.eclipse.osgi:3.8.0-SNAPSHOT @ /opt/pwebster/lts/R4_platform-aggregator/rt.equinox.framework/bundles/org.eclipse.osgi/pom.xml: baseline and reactor have same version but different contents
   null: different
      .api_description: present in baseline only
      META-INF/ECLIPSE_.RSA: present in baseline only
      META-INF/ECLIPSE_.SF: present in baseline only
      META-INF/eclipse.inf: present in baseline only
      org/eclipse/core/runtime/adaptor/EclipseStarter.class: different
      org/eclipse/core/runtime/adaptor/package.html: not present in baseline
      org/eclipse/core/runtime/internal/adaptor/BasicLocation.class: different
      org/eclipse/core/runtime/internal/adaptor/ClasspathManifest.class: different
Comment 11 Paul Webster CLA 2012-07-19 12:09:19 EDT
Is there a way to make sure that maven conditions the bundles as it produces the jars?

Or is there a way to condition one of the output jars so I can compare it after the fact?  Even command line would be good.

PW
Comment 12 Paul Webster CLA 2012-07-19 12:26:00 EDT
Created attachment 218944 [details]
diff of javap -v output for EclipseStarter.class, conditioned

I conditioned the osgi jar with java6 pack200 -r and then ran my javap and diff.

Looks much better.  There is still a diff, however, and I'm not sure what to make of it.

PW
Comment 13 Paul Webster CLA 2012-07-19 12:46:27 EDT
I looked at both source jars, and EclipseStarter.java was the same in both.

Does that mean there's a compiler option difference?

PW
Comment 14 Paul Webster CLA 2012-07-19 13:33:57 EDT
David, are there global compiler options set for our build?

in org.eclipse.osgi/build.properties the only things I see are:
jre.compilation.profile = J2SE-1.5
javacSource=1.5
javacTarget=jsr14
javacWarnings..=-deadCode,-raw,-unchecked


In eclipsebuilder eclipse/buildConfigs/master/build.properties
javacDebugInfo=on
javacVerbose=true
compilerArg=-inlineJSR -enableJavadoc -encoding ISO-8859-1

In http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/tree/eclipse/buildConfigs/sdk/build.properties

javacDebugInfo=on
javacVerbose=true
compilerArg=-inlineJSR -enableJavadoc -encoding ISO-8859-1

Looks like we usually use the above compilerArg everywhere ... Can we make sure that's set for the CBI build?

PW
Comment 15 David Williams CLA 2012-07-19 23:30:07 EDT
(In reply to comment #14)
> David, are there global compiler options set for our build?
> 

There is not one global value if that's what you are asking. I see 10 build.properties where they are set, with what I happen to have loaded. And, yes, those compilerArgs are pretty standard (7 of those 10), two test ones are "looser" and I think one (p2?) are "tighter". I think in the past we've recommended at least that standard set, but then let each component team decide other settings.
Comment 16 Paul Webster CLA 2012-07-20 10:27:05 EDT
(In reply to comment #14)
> In
> http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/tree/eclipse/buildConfigs/sdk/build.properties
> 
> javacDebugInfo=on
> javacVerbose=true
> compilerArg=-inlineJSR -enableJavadoc -encoding ISO-8859-1


Thanh, can we start by setting these compilerArg options in the eclipse-parent pom?

PW
Comment 17 Paul Webster CLA 2012-07-23 16:19:36 EDT
I deleted my local repo and re-ran Juno_RC4_R4 build with updated compiler args. 

              <compilerArguments>
                <inlineJSR/>
                <enableJavadoc/>
                <encoding>ISO-8859-1</encoding>
              </compilerArguments>



for org.eclipse.osgi I still get a difference as below.  I picked org/eclipse/osgi/framework/internal/core/StartLevelManager.class.  The javap difference is huge, but if I java6 pack200 -r the jar file, then the diff resolves to a good result:

1,2c1,2
< Classfile jar:file:/opt/pwebster/lts/buildRepo_R42/R-4.2-201206081400/plugins/org.eclipse.osgi_3.8.0.v20120529-1548.jar!/org/eclipse/osgi/framework/internal/core/StartLevelManager.class
<   Last modified May 29, 2012; size 16439 bytes
---
> Classfile jar:file:/opt/pwebster/lts/org.eclipse.osgi-3.8.0-SNAPSHOT.jar!/org/eclipse/osgi/framework/internal/core/StartLevelManager.class
>   Last modified Jul 23, 2012; size 16439 bytes



Output of comparator during run:
   null: different
      .api_description: present in baseline only
      META-INF/ECLIPSE_.RSA: present in baseline only
      META-INF/ECLIPSE_.SF: present in baseline only
      META-INF/eclipse.inf: present in baseline only
      org/eclipse/core/runtime/adaptor/package.html: not present in baseline
      org/eclipse/core/runtime/internal/adaptor/PluginConverterImpl.class: different
      org/eclipse/osgi/framework/console/package.html: not present in baseline
      org/eclipse/osgi/framework/eventmgr/CopyOnWriteIdentityMap$Snapshot$EntryIterator.class: different
      org/eclipse/osgi/framework/eventmgr/CopyOnWriteIdentityMap$Snapshot$EntrySet.class: different
      org/eclipse/osgi/framework/eventmgr/CopyOnWriteIdentityMap$Snapshot$SnapshotIterator.class: different
      org/eclipse/osgi/framework/eventmgr/CopyOnWriteIdentityMap$Snapshot.class: different
      org/eclipse/osgi/framework/eventmgr/EventManager$EventThread$Queued.class: different
      org/eclipse/osgi/framework/eventmgr/EventManager$EventThread.class: different
      org/eclipse/osgi/framework/eventmgr/package.html: not present in baseline
      org/eclipse/osgi/framework/internal/core/StartLevelManager.class: different
      org/eclipse/osgi/framework/log/package.html: not present in baseline
      org/eclipse/osgi/framework/util/Headers.class: different
      org/eclipse/osgi/framework/util/SecureAction$1$1.class: different
...
Comment 18 Paul Webster CLA 2012-07-23 16:20:48 EDT
Igor, did you say you included something in the latest 0.16 SNAPSHOT to try and compare against normalized files?

PW
Comment 19 Paul Webster CLA 2012-07-23 21:27:51 EDT
In addition to comment #17 I also ran with  -Peclipse-sign, and just deleted the signing plugin from that profile in eclipse-parent/pom.xml.  Now I get something much more promising :-)

[WARNING] MavenProject: org.eclipse.equinox.framework:org.eclipse.osgi:3.8.0-SNAPSHOT @ /opt/pwebster/git/lts/R4_platform-aggregator/rt.equinox.framework/bundles/org.eclipse.osgi/pom.xml: baseline and reactor have same version but different contents
   null: different
      .api_description: present in baseline only
      META-INF/ECLIPSE_.RSA: present in baseline only
      META-INF/ECLIPSE_.SF: present in baseline only
      META-INF/eclipse.inf: different
      org/eclipse/core/runtime/adaptor/package.html: not present in baseline
      org/eclipse/osgi/framework/console/package.html: not present in baseline
      org/eclipse/osgi/framework/eventmgr/package.html: not present in baseline
      org/eclipse/osgi/framework/log/package.html: not present in baseline
      org/eclipse/osgi/launch/package.html: not present in baseline
      org/eclipse/osgi/service/datalocation/package.html: not present in baseline
      org/eclipse/osgi/service/debug/package.html: not present in baseline
      org/eclipse/osgi/service/environment/package.html: not present in baseline
      org/eclipse/osgi/service/localization/package.html: not present in baseline
      org/eclipse/osgi/service/resolver/package.html: not present in baseline
      org/eclipse/osgi/service/runnable/package.html: not present in baseline
      org/eclipse/osgi/service/security/package.html: not present in baseline
      org/eclipse/osgi/signedcontent/package.html: not present in baseline
      org/eclipse/osgi/storagemanager/package.html: not present in baseline
      org/eclipse/osgi/util/package.html: not present in baseline

+ there were more changes in the source bundles.

PW
Comment 20 Paul Webster CLA 2012-07-23 21:31:22 EDT
(In reply to comment #19)
> In addition to comment #17 I also ran with  -Peclipse-sign, and just deleted
> the signing plugin from that profile in eclipse-parent/pom.xml.  Now I get
> something much more promising :-)

oh, but many differences in org.eclipse.core.commands.

PW
Comment 21 Paul Webster CLA 2012-07-24 08:53:25 EDT
(In reply to comment #20)
> 
> oh, but many differences in org.eclipse.core.commands.

Looks like it's being compiled to the wrong BREE:

1,3c1,3
< Classfile jar:file:/opt/pwebster/lts/buildRepo_R42/R-4.2-201206081400/plugins/org.eclipse.core.commands_3.6.1.v20120521-2329.jar!/org/eclipse/core/commands/AbstractHandler.class
<   Last modified May 22, 2012; size 2226 bytes
<   MD5 checksum ebf6e7ca59cc2719645a1cf8e4117e0c
---
> Classfile jar:file:/opt/pwebster/lts/R4_platform-aggregator/eclipse.platform.ui/bundles/org.eclipse.core.commands/target/org.eclipse.core.commands-3.6.1-SNAPSHOT.jar!/org/eclipse/core/commands/AbstractHandler.class
>   Last modified Jul 23, 2012; size 2052 bytes
>   MD5 checksum 9bce15b2005e988f829cc9bf7ecc295c
7,8c7,8
<   minor version: 3
<   major version: 45
---
>   minor version: 0
>   major version: 46
Comment 22 Paul Webster CLA 2012-07-24 11:12:09 EDT
(In reply to comment #21)
> > Classfile jar:file:/opt/pwebster/lts/R4_platform-aggregator/eclipse.platform.ui/bundles/org.eclipse.core.commands/target/org.eclipse.core.commands-3.6.1-SNAPSHOT.jar!/org/eclipse/core/commands/AbstractHandler.class
> >   Last modified Jul 23, 2012; size 2052 bytes
> >   MD5 checksum 9bce15b2005e988f829cc9bf7ecc295c
> 7,8c7,8
> <   minor version: 3
> <   major version: 45
> ---
> >   minor version: 0
> >   major version: 46

http://www.rgagnon.com/javadetails/java-0544.html

major  minor Java platform version 
45       3           1.0
45       3           1.1
46       0           1.2
47       0           1.3
48       0           1.4
49       0           1.5
50       0           1.6

PW
Comment 23 Igor Fedorenko CLA 2012-07-24 14:05:12 EDT
(In reply to comment #21)
> (In reply to comment #20)
> > 
> > oh, but many differences in org.eclipse.core.commands.
> 
> Looks like it's being compiled to the wrong BREE:
> 


This is caused by default BREE 1.4 configured in eclipse-parent/pom.xml. You should be able to force whatever BREE is needed by org.eclipse.core.commands in project pom.xml. We may need to revisin eclipse-parent/pom.xlm default if too many projects require such override.
Comment 24 Thanh Ha CLA 2012-07-24 14:26:09 EDT
Created attachment 219130 [details]
core.commands BREE patch

I've attached a patch to set the BREE to CDC-1.0/Foundation-1.0. Checking the javap output I am now getting the following:


javap -v -classpath org.eclipse.core.commands-3.6.1-SNAPSHOT.jar org/eclipse/core/commands/AbstractHandler

Compiled from "AbstractHandler.java"
public abstract class org.eclipse.core.commands.AbstractHandler extends org.eclipse.core.commands.common.EventManager implements org.eclipse.core.commands.IHandler2
  SourceFile: "AbstractHandler.java"
  minor version: 3
  major version: 45
Comment 25 David Williams CLA 2012-07-24 15:00:37 EDT
(In reply to comment #24)
> Created attachment 219130 [details]
> core.commands BREE patch
> 
> I've attached a patch to set the BREE to CDC-1.0/Foundation-1.0. Checking the
> javap output I am now getting the following:
> 
> 
> javap -v -classpath org.eclipse.core.commands-3.6.1-SNAPSHOT.jar
> org/eclipse/core/commands/AbstractHandler
> 
> Compiled from "AbstractHandler.java"
> public abstract class org.eclipse.core.commands.AbstractHandler extends
> org.eclipse.core.commands.common.EventManager implements
> org.eclipse.core.commands.IHandler2
>   SourceFile: "AbstractHandler.java"
>   minor version: 3
>   major version: 45

Is that the right value? I've lost track :) 

Just for additional information, 1.4 is still most frequently used BREE in the platform (barely) so it should probably be the default, but LOTS of others will need to override it. From one of my "repo" reports, here is the distribution of brees for Eclipse Project repository: 


   Checked 680 of 697 jars.


  Bundles with appropriate use of Bundle-RequiredExecutionEnvironment (BREE):
   Java with BREE: 330
   Non Java without BREE:317

   Distribution of BREEs in Java Bundles 

    1	CDC-1.0/Foundation-1.0,J2SE-1.2
   32	CDC-1.0/Foundation-1.0,J2SE-1.3
   17	CDC-1.1/Foundation-1.1,J2SE-1.4
    2	J2SE-1.2
    6	J2SE-1.3
    1	J2SE-1.3,CDC-1.0/Foundation-1.0
    1	J2SE-1.3,CDC-1.1/Foundation-1.1
  117	J2SE-1.4
    7	J2SE-1.4,CDC-1.0/Foundation-1.0,J2SE-1.3
    6	J2SE-1.4,CDC-1.1/Foundation-1.1
    2	J2SE-1.4,J2SE-1.3,CDC-1.0/Foundation-1.0
   80	J2SE-1.5
    2	J2SE-1.5,J2SE-1.4
   27	J2SE-1.5,J2SE-1.4,CDC-1.1/Foundation-1.1
    5	J2SE-1.5,OSGi/Minimum-1.2
    1	J2SE-1.5,OSGi/Minimum-1.2,CDC-1.1/Foundation-1.1,J2SE-1.4
   12	JavaSE-1.6
    2	OSGi/Minimum-1.0
    2	OSGi/Minimum-1.1
    2	OSGi/Minimum-1.2
    5	OSGi/Minimum-1.2,CDC-1.1/Foundation-1.1,J2SE-1.4
Comment 26 Thanh Ha CLA 2012-07-24 15:09:46 EDT
(In reply to comment #25)
> (In reply to comment #24)
> >   minor version: 3
> >   major version: 45
> 
> Is that the right value? I've lost track :) 

It appears to be the correct one. Paul and I took a look at the comparator with the patch and the result is a lot better. I went ahead and pushed the patch into the repos.

R3 eclipse.platform.ui:
http://git.eclipse.org/c/cbi/eclipse.platform.ui.git/commit/?h=Juno_RC4_R3&id=57130131533cfc726570c1a46adca9a15276bfec

R4 eclipse.platform.ui:
http://git.eclipse.org/c/cbi/eclipse.platform.ui.git/commit/?h=Juno_RC4_R4&id=c9a2a5ed6262a6f149b5299ab911f0c60c9feb6b
Comment 27 David Williams CLA 2012-07-24 15:13:48 EDT
Ok, good. 

Also ... probably more than you wanted/needed to know, here's the exact list of bree-to-library associations we use for the platform's PDE build. Not sure if you've gotten this data before, already, or if you have your own "associations" ... but, ... I had it handy, so thought I'd post here in case you find it helpful.  


    bootclasspath="/shared/common/j2sdk1.4.2_19/jre/lib/rt.jar:/shared/common/j2sdk1.4.2_19/jre/lib/jsse.jar:/shared/common/j2sdk1.4.2_19/jre/lib/jce.jar"
    bootclasspath_15="/shared/common/jdk-1.5.0_16/jre/lib/rt.jar:/shared/common/jdk-1.5.0_16/jre/lib/jsse.jar:/shared/common/jdk-1.5.0_16/jre/lib/jce.jar"
    bootclasspath_16="/shared/common/jdk1.6.0_27.x86_64/jre/lib/rt.jar:/shared/common/jdk1.6.0_27.x86_64/jre/lib/jsse.jar:/shared/common/jdk1.6.0_27.x86_64/jre/lib/jce.jar"
    bootclasspath_foundation="/shared/common/org.eclipse.sdk-feature2/libs/ee.foundation-1.0.jar"
    bootclasspath_foundation11="/shared/common/org.eclipse.sdk-feature2/libs/ee.foundation.jar"
    # https://bugs.eclipse.org/bugs/show_bug.cgi?id=375976, and 
    # https://bugs.eclipse.org/bugs/show_bug.cgi?id=376029
    OSGiMinimum11="/shared/common/org.eclipse.sdk-feature2/libs/ee.minimum.jar"
    OSGiMinimum12="/shared/common/org.eclipse.sdk-feature2/libs/ee.minimum-1.2.0.jar"
Comment 28 Thanh Ha CLA 2012-07-24 17:49:01 EDT
Created attachment 219138 [details]
classes different from baseline

I'm not sure what the best way to get a list of bundles that were compiled with the wrong BREE is but I came up with a little hack using egrep.

    egrep '(baseline and reactor have same version but different contents|class: different)' compare.dump > class-different.log

The idea here is that any line starting with [WARNING] that has lines following it that contain "class: different" need further inspecting. [WARNING] lines that are followed by another [WARNING] line can be ignored.

At first glance it seems like the bundles with class issues are indeed due to incorrect BREE. I will take a closer look and set the correct BREE for the problem bundles.
Comment 29 Thanh Ha CLA 2012-07-25 12:52:45 EDT
Created attachment 219175 [details]
classes different from baseline

I pushed into CBI R4 branch some modifications to BREE of certain packages which cleaned up some of the class differences.

R4 eclipse.jdt.debug:
http://git.eclipse.org/c/cbi/eclipse.jdt.debug.git/commit/?h=Juno_RC4&id=dd04456568aed75b275ad0998ce82c0bf41c4374

R4 eclipse.platform.runtime:
http://git.eclipse.org/c/cbi/eclipse.platform.runtime.git/commit/?h=Juno_RC4&id=793276a173a30c3c1e409c973e9c5e6049d85838

R4 eclipse.platform.ui:
http://git.eclipse.org/c/cbi/eclipse.platform.ui.git/commit/?h=Juno_RC4_R4&id=6c1c6c4cb590ccc9f4160da95e12034f2d8e610b

R4 rt.equinox.bundles:
http://git.eclipse.org/c/cbi/rt.equinox.bundles.git/commit/?h=Juno_RC4&id=8a0a8ae71cd51b2c03931ddcf524f7c59b6a467f


BREE changed to CDC-1.0/Foundation-1.0:
eclipse.platform.runtime - org.eclipse.core.runtime
eclipse.platform.ui - org.eclipse.jface.databinding
eclipse.platform.ui - org.eclipse.ui
eclipse.platform.ui - org.eclipse.ui.views.properties.tabbed
rt.equinox.bundles - org.eclipse.equinox.http.servlet
rt.equinox.bundles - org.eclipse.equinox.concurrent
rt.equinox.bundles - org.eclipse.equinox.http.registery
rt.equinox.bundles - org.eclipse.equinox.jsp.jasper
rt.equinox.bundles - org.eclipse.equinox.device
rt.equinox.bundles - org.eclipse.equinox.ip
rt.equinox.bundles - org.eclipse.equinox.servletbridge
rt.equinox.bundles - org.eclipse.equinox.http.servletbridge
rt.equinox.bundles - org.eclipse.equinox.wireadmin (OSGi/Minimum-1.0, cleaned up but still some differences)


BREE changed to J2SE-1.5:
eclipse.jdt.debug - org.eclipse.jdt.debug



There are still the bundles below which changing the BREE did not clear up the class differences. We need to understand what's causing the class differences in these cases.


not sure (CDC-1.1/Foundation-1.1):
eclipse.jdt.core - org.eclipse.jdt.compiler.apt (JavaSE-1.6)
eclipse.pde.build - org.eclipse.pde.build.tests (J2SE-1.4)
eclipse.platform - org.eclipse.update.core
eclipse.platform - org.eclipse.ant.ui
eclipse.platform.runtime - org.eclipse.core.jobs
eclipse.platform.team - org.eclipse.team.examples.filesystem
eclipse.platform.text - org.eclipse.jface.text (J2SE-1.4)
eclipse.platform.ua - org.eclipse.ua.tests (J2SE-1.5)
eclipse.platform.ua - org.eclipse.ua.tests.doc (J2SE-1.4)
eclipse.platform.ui - org.eclipse.core.databinding.observable
eclipse.platform.ui - org.eclipse.core.databinding
eclipse.platform.ui - org.eclipse.jface.tests.databinding (J2SE-1.4)
eclipse.platform.ui - org.eclipse.ui.tests.harness
eclipse.platform.ui - org.eclipse.ui.tests.views.properties.tabbed
rt.equinox.bundles - org.eclipse.equinox.common
rt.equinox.bundles - org.eclipse.equinox.preferences
rt.equinox.bundles - org.eclipse.equinox.app
rt.equinox.bundles - org.eclipse.equinox.ds
rt.equinox.bundles - org.eclipse.equinox.bidi (OSGi/Minimum-1.2)
rt.equinox.bundles - org.eclipse.equinox.cm
rt.equinox.bundles - org.eclipse.equinox.useradmin (OSGi/Minimum-1.1)
rt.equinox.framework - org.eclipse.equinox.launcher (OSGi/Minimum-1.2)
rt.equinox.p2 - org.eclipse.equinox.frameworkadmin.equinox
rt.equinox.p2 - org.eclipse.equinox.p2.transport.ecf (J2SE-1.5)
rt.equinox.p2 - org.eclipse.equinox.p2.discovery (J2SE-1.5)
rt.equinox.p2 - org.eclipse.equinox.p2.discovery.compatibility (J2SE-1.5)
rt.equinox.p2 - org.eclipse.p2.tests.verifier (J2SE-1.5)
rt.equinox.p2 - org.eclipse.equinox.p2.tests.ui (J2SE-1.5)
eclipse.platform.releng - org.eclipse.releng.tools (J2SE-1.4)
rt.equinox.bundles - org.eclipse.equinox.wireadmin (OSGi/Minimum-1.0, cleaned up but still some differences)
Comment 30 Igor Fedorenko CLA 2012-07-29 01:45:25 EDT
Created attachment 219295 [details]
proposed changes to p2 pom.xml files

I had a closer look at differences in .class files.  Most of the differences were in rt.equinox.p2, so I've updated/cleaned pom.xml files in that git repository. I'd appreciate if somebody with p2 commit privileges could review and apply the patch.

There are still differences in 109 class files and they all represent the same problem. For reasons I don't quite understand, Tycho and PDE/Build produce different bytecode for some overridden method invocations. For example, the following expression

   "".equals(someString)

Tycho generates "INVOKEVIRTUAL java/lang/String.equals" while baseline version has "INVOKEVIRTUAL java/lang/Object.equals" (note different target class).

Does anybody know what can trigger this difference?

Can somebody paste here exact version of jdt compiler and exact jdt compiler parameters used to compile org.eclipse.core.databinding bundle in Juno?
Comment 31 Andrey Loskutov CLA 2012-07-29 11:03:38 EDT
(In reply to comment #30)
> There are still differences in 109 class files and they all represent the
> same problem. For reasons I don't quite understand, Tycho and PDE/Build
> produce different bytecode for some overridden method invocations. For
> example, the following expression
> 
>    "".equals(someString)
> 
> Tycho generates "INVOKEVIRTUAL java/lang/String.equals" while baseline
> version has "INVOKEVIRTUAL java/lang/Object.equals" (note different target
> class).
> 
> Does anybody know what can trigger this difference?

Can you point to the specific class/method please? Firth I thought it would be an ancient compiler target but even setting to 1.1 doesn't produce "java/lang/Object.equals" for a sample above. If compiler resolves call "".equals() as Object.equals() then something really weird must happen - like wrong String class found by compiler on classpath? Wrong target JRE definition set?
Comment 32 Igor Fedorenko CLA 2012-07-29 12:29:40 EDT
If you run the build with -Dtycho.debug.artifactcomparator , Tycho will dump to filesystem disassembly of all class files where baseline and build do not match, search for files with -baseline and -build name suffixes. You can then diff the files to see the delta. For a specific example, have a look at StringToBooleanConverter from org.eclipse.core.databinding bundle. It's quite small and the difference is easy to spot.

You are right about JRE libraries, however. Looks like all bundles that have this problem target one of restricted BREE, so I configured [1] org.eclipse.core.databinding bundle to compile against CDC-1.1/Foundation-1.1 libraries and this indeed produced classes identical to what is available in Juno.


[1] http://wiki.eclipse.org/Tycho/Release_Notes/0.14
Comment 33 Thanh Ha CLA 2012-07-30 14:07:22 EDT
(In reply to comment #32)
> You are right about JRE libraries, however. Looks like all bundles that have
> this problem target one of restricted BREE, so I configured [1]
> org.eclipse.core.databinding bundle to compile against
> CDC-1.1/Foundation-1.1 libraries and this indeed produced classes identical
> to what is available in Juno.

Hi Igor, can you explain what you did here to get this working? I had a look at the link but it's not clear to me what you mean by compile against CDC-1.1/Foundation-1.1 libraries. Is this a separate JDK that needs to be downloaded and add to the toolchain?
Comment 34 Igor Fedorenko CLA 2012-07-30 15:52:09 EDT
There are several bundles that I believe do not properly declare minimal bundle runtime execution environment. This needs to be cleaned up first and I opened separate bug 386266 to track this work. Once this is done, I'll update CBI parent pom.xml and provide instructions how to use BREE-specific system libraries.
Comment 35 Paul Webster CLA 2012-09-17 13:46:16 EDT
I've updated eclipse.platform.repository to mirror in some of the other repositories created for modules like platform and sdk.

PW
Comment 36 Paul Webster CLA 2013-01-08 09:48:39 EST
David, you can also use https://github.com/irbull/p2diff to compare the p2 repos, and get an idea.

I'm not sure how good it is if the IUs have different qualifiers.

PW
Comment 37 Thanh Ha CLA 2014-03-31 13:21:28 EDT
We've long since moved Platform over to CBI build so I think we are done here. Feel free to reopen if anyone has any specific action items we still need to resolve.