Community
Participate
Working Groups
Both last night's N build, and today's I-build failed with compile errors when jface.databinding and jface.text try to import SWT. At first, it was thought this might have been related to related to a change in the way SWT exports its packages. (exporting from fragments, bug declaring "Eclipse-ExtensibleAPI" ... even though there had been a few successful N-builds with that change. The other change is SWT changed to require "1.5" (instead of 1.4/1.3/Foundation). It appears jface.databinding and jface.text are failing because they require only 1.3/Foundation level BREEs. I say this due to the investigations by Silenio, who found that other bundles that require SWT seemed to compile ok, but they (already) require "1.5". This limitation on "matching BREES" did not used to exist in PDE builds, but on the surface, appears that Tycho does not handle it as well (or, heaven forbid, JDT compiler no longer handles is well!).
Forgot to mention ... we tried a test build, with the exports added by to SWT and the build still fails in the same way. The test build on build.eclipse.org is still running, but my local build failed so expect build.eclipse.org test build will as well.
BTW, I'm also trying a local test build where I've removed the possibly bogus "1.3 JDK" I added to our toolchains.xml file. See bug 413014.
(In reply to comment #2) > BTW, I'm also trying a local test build where I've removed the possibly > bogus "1.3 JDK" I added to our toolchains.xml file. See bug 413014. Made no difference. Still failed, in same way. FWIW, the test M build did succeed (whew). The most recent test build on build.eclipse.org machine is at http://build.eclipse.org/eclipse/builds/4N/gitCache/eclipse.platform.releng.aggregator/ if anyone wanted to study it.
Why is jface.databinding so far behind jface? If I'm not mistaken, jface 3.2 has a BREE of 1.4 and the current, jface 3.9, has a BREE of 1.5. Changing jface.databinding to a BREE of 1.5 would pave the way for future enhancements that take advantage of generics, IMO would be very powerful.
Here's an intriguing thought ... though a long shot ... according to bug 401141 the signing of "inner jars" were turned off on 7/15 ... the same day that our (new) builds started to fail. At the same time, I "forgot" and left signing turned "on" for main jars, in nightlies. Are any of the SWT jars/fragments involved "inner jars". Even if not, are any of the SWT or JFace bundles "split packages"? I believe there are cases that "hosts" and "fragments" have to be signed by the same certificate, to be "loaded" by class loader ... same for bundles that have "split packages". I know that normally this makes no difference for "running equinox", but ... not sure the "class loading" during Tycho builds would be following those same rules or not ... I wonder if it does its own "custom classloading"? Just an idea in case helps explain this confusing situation. In the mean time, I am trying a test "patched build", with (only) jface.databinding changed to "1.5" ... just to see if that one case solves the issue. It was the only one using "foundation level". jface.text was using "1.4" ... so reluctant to change it without trying the one first. And, the results? (just finished) ... build now fails similarly, but at org.eclipse.ui. ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.18.0:compile (default-compile) on project org.eclipse.ui: Compilation failure: Compilation failure: [ERROR] /data/shared/eclipse/builds/4N/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.ui/src/org/eclipse/ui/internal/UIPreferenceInitializer.java:[25,0] [ERROR] import org.eclipse.swt.SWT; (That is with main signing turned off and using "N-Build qualifiers" so should not be any replacement from comparator operation ... so, don't see how signing is an issue, after all).
For tonight's 8 PM N build, I am going to "turn off" all signing (like N builds normally are not signed). Plus, I'm going to revert the "fix" to toolchains.xml file, just to remove as many changes as possible. But, I'm going to leave in the "patch" for jface.databinding. So, will expect the nightly to fail at org.eclipse.ui ... thus question if we can have any "Foundation Classes" depend on a bundle at 1.5 level ... though seems a 1.4 bundle can depend on a bundle at 1.5 level?
(In reply to comment #6) > For tonight's 8 PM N build, I am going to "turn off" all signing (like N > builds normally are not signed). > > Plus, I'm going to revert the "fix" to toolchains.xml file, just to remove > as many changes as possible. > > But, I'm going to leave in the "patch" for jface.databinding. > > So, will expect the nightly to fail at org.eclipse.ui ... thus question if > we can have any "Foundation Classes" depend on a bundle at 1.5 level ... > though seems a 1.4 bundle can depend on a bundle at 1.5 level? Failed "as expected". From what I can easily see in my workspace, Only the following "UI related" bundles still specify Bundle-RequiredExecutionEnvironment: CDC-1.0/Foundation-1.0, J2SE-1.3 org.eclipse.ui org.eclipse.ui.views.properties.tabbed org.eclipse.ui.tests.views.properties.tabbed I don't believe we use the tests.view.properties.tabbed (at least right now?) and org.eclipse.ui.views.properties.tabbed does not have a "direct" dependency on SWT ... only indirect, through org.eclipse.ui. It seems a reasonable "test build" to see if a patched version of "org.eclipse.ui" would be enough to get the build to complete.
(In reply to comment #7) > (In reply to comment #6) > > It seems a reasonable "test build" to see if a patched version of > "org.eclipse.ui" would be enough to get the build to complete. And, actually, I decided to try both jface.databinding and o.e.ui at the BREE level of "1.4" ... simply because that is, technically, a "smaller change", and jface.text seemed ok at 1.4 level, but my all means don't intend it to be "final" ... if the committers for those bundles think that "1.5" is the better way to go. I'll test "1.4" on my local test machine, and if it works, rebuild a Nightly ... so we can get some unit testing done (hopefully) by Wednesday AM. I'll wait for final decision from committers before producing an I-build for this week. [BTW, I'm sure at some level this is a Tycho bug ... but ... it'd be a hard sell, I think, into putting much effort into supporting this ... unless of course those that are smarter than me know the reasons that we need to.]
(In reply to comment #8) > (In reply to comment #7) > > (In reply to comment #6) > > > > > It seems a reasonable "test build" to see if a patched version of > > "org.eclipse.ui" would be enough to get the build to complete. > > And, actually, I decided to try both jface.databinding and o.e.ui at the > BREE level of "1.4" ... simply because that is, technically, a "smaller > change", and jface.text seemed ok at 1.4 level, ... Blow by blow ... that didn't work (same jface.databinding compile errors), so trying both at 1.5 level now.
Created attachment 233546 [details] list of failed and skipped bundles This is not leading to an easy solution. By "enabling" some to compile ... allows others (previously skipped) to be compiled, and many of them too have compile errors due to the SWT. The first to fail was "ui.forms" and it currently specifies BREE of 1.4. All together there were 13 bundles that failed (with the two patches to 1.5) (I assume due to SWT issues) and (still) 118 "skipped" presumably because their prereqs did not compile. Full list is attached. But ... seems like, off hand, we are faced with decision for all "UI related" bundles to move to 1.5 if SWT does ... or, for SWT to revert that change and go back to "lower level". I suggest for July for SWT to revert back, I'll remove my "build patches" of course, and then have larger team discussion over several weeks, perhaps deciding, say ... mid-August? I'm not sure if there are "adopter requirements" that we stay on lower level ... or, if it just violates our knowledge (or historical biases :) of good architecture?
(In reply to comment #10) > But ... seems like, off hand, we are faced with decision for all "UI > related" bundles to move to 1.5 if SWT does ... or, for SWT to revert that > change and go back to "lower level". -100. What if one of the UI bundles wants to use 1.7? Then all must raise This is a no go and needs to be fixed at the root.
Going on the working theory of a mismatch of BREEs. I have no idea how anything is compiling with a BREE less than JavaSE-1.5 since the core framework (org.eclipse.osgi) has moved up to BREE JavaSE-1.5. I am aware that CBI seems to have a limitation that requires host and their fragments match BREEs, but I did not think the limitation extended to any clients of a bundle.
(In reply to comment #12) > Going on the working theory of a mismatch of BREEs. I have no idea how > anything is compiling with a BREE less than JavaSE-1.5 since the core > framework (org.eclipse.osgi) has moved up to BREE JavaSE-1.5. > > I am aware that CBI seems to have a limitation that requires host and their > fragments match BREEs, but I did not think the limitation extended to any > clients of a bundle. I'm just guessing, but I wonder if part of the problem in this this case is that some bundles "reexport" SWT: org.eclipse.jface org.eclipse.ui and in turn, some probably reexport jface and eclipse ui? I know there is an implied assumption that a reexport is "providing" the re-exported API ... I wonder if Tycho assumes that means "reexporting the BREE level" too (though, I know if should not mean that).
Adding Igor and Jan to CC. I think we'll need some Tycho expertise to help untangle this string. Any suggestions on what might be going wrong? Or how we can debug?
Yes, many basic UI bundles reexport some required bundles. E.g. o.e.jface reexports o.e.swt and o.e.ui reexports o.e.swt, o.e.jface, and o.e.ui.workbench. But in general, bundles don't have to update their BREE if a required bundle does. E.g. org.eclipse.osgi and org.eclipse.core.runtime have been at 1.5 for quite a while, but org.eclipse.core.expressions requires org.eclipse.core.runtime and is still at J2SE-1.4 (or CDC-1.0/Foundation-1.0 + Import-Package: org.w3c.dom). For the moment, I'm OK with reverting SWT to not require 1.5 any more (revert bug 411356). Bug 412067 is now fixed in a way that doesn't require 1.4 nor 1.5.
Also adding JDT-core to CC list. I know once before there was a "optimization" put in compiler that basically assumed "highest level bree" (bug 360164) ... that was for a pretty special case (involving enums) but just wondering if some similar optimization made its way back into JDT? (I do not think so ... just for completeness wanted JDT Core involved). Later today, we may have to re-create (or find in failed builds) the "effective POMs" and "assumed classpath" at the time of the failure.
(In reply to comment #14) > Adding Igor and Jan to CC. > > I think we'll need some Tycho expertise to help untangle this string. Any > suggestions on what might be going wrong? not sure I'm following. links to full build logs with errors and/or steps to reproduce appreciated. Also, I'm missing details of expected vs. observed behaviour. > Or how we can debug? If the problem seems BREE-related, maven debug mode (-X) will dump the effective BREE used for compilation of each bundle. In general, see docs [1] on the effects of BREE during build. In particular, any bundles required by a given bundle do not affect effective BREE of the given bundle. re-exports are also not taken into account for this. As Thomas mentioned, there might be a bug when it comes to compiling fragments, I'm not sure how the effective BREE of fragment and host should be computed during build (should fragment BREE be merged into host and vice versa? From the merged BREEs, is the most restrictive BREE the effective one?). A small demo project would help to focus the discussion and track down the problem. [1] http://wiki.eclipse.org/Tycho/Execution_Environments
The problem is happening because the SWT fragment classes are not added to the classpath when the BREE of the dependent bundle is more restrictive. It seems to only happen with fragments. For example, org.eclipse.ui (CDC1.0/J2SE1.3) depends on org.eclipse.jface (J2SE1.5) without problems. I was able to compile further by changing the BREE of the following bundles, but the build still did not finish (failed on org.eclipse.ui.ide). Basically, direct or indirect dependents of SWT would have to move to 1.5. org.eclipse.jface.databinding CDC1.0/1.3 to 1.5 org.eclipse.ui.forms 1.4 to 1.5 org.eclipse.ui CDC1.0/1.3 to 1.5 org.eclipse.help.ui 1.4 to 1.5 org.eclipse.ui.cheatsheets 1.4 to 1.5 org.eclipse.ui.intro 1.4 to 1.5 org.eclipse.jface.text 1.4 to 1.5 org.eclipse.ui.views 1.4 to 1.5 Another test I made was to change the swt BREE to 1.4. In this case, the whole build succeed with changes in the following bundles: org.eclipse.jface.databinding CDC1.0/1.3 to 1.4 org.eclipse.ui CDC1.0/1.3 to 1.4 org.eclipse.ui.views.properties.tabbed CDC1.0/1.3 to 1.4 Given that the problem only happens with fragments, this really seems like a bug in tycho.
Created attachment 233555 [details] main DEBUG log for jface.databinding compile error This is no "small test case" :) but is the full log from when we first saw the error in a "nightly" build. (For those on CC who might not know how to navigate such large logs) the type of searches I do (in vi) are like ... /[ERROR to find first compile error /Reactor Build Order shows the "planned order" of building /--- tycho.*compiler.*jface.databinding will take you to where the compiler "starts" for that bundle. Which is sort of "in the middle" of the effective POM (I think). /Reactor Summary shows the "overall results" (what failed, and what was skipped) I'm hoping someone can spot something obvious we are forgetting or doing wrong (or, else, I fear it is a subtle Tycho bug, when one bundle requires another bundle that requires fragments).
Since this seems to be related to other fragments I think it boils down to the previous limitation I was mentioning. But the limitation seems more broad than I originally thought. For some reason the build seems to ignore the fragments if they cannot 'resolve' against the minimum BREE of the client bundles. I suspect this has to do with the way tycho configures the profiles for the OSGi resolver state. PDE-Build would setup the resolver state such that it contained all the configured EE profiles. This would allow everything to resolve as long as an EE profile existed that satisfied the bundle's BREE. What is most mysterious is why this issue in tycho only seems to effect fragments. It looks as if the pre-built SWT fragments are being run through the resolution again with a limited BREE profile for the bundle we are currently compiling but for non-fragments this does not seem to effect them. As Silenio points out, org.eclipse.ui (CDC1.0/J2SE1.3) depends on org.eclipse.jface (J2SE1.5) without problems. Can someone point me to the code in tycho that deals with setting up the OSGi resolver state?
(In reply to comment #20) > Can someone point me to the code in tycho that deals with setting up the > OSGi resolver state? http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-core/src/main/java/org/eclipse/tycho/core/osgitools/EquinoxResolver.java#n64
Tycho configures OSGi resolver (and P2 resolver, btw) according to the project effective build BREE. So if you are building a project that has BREE=1.4 and SWT fragment has BREE=1.5, OSGi resolver will not include SWT fragment in project dependencies. The build failure highlighted the fact that the project will not run in its declared BREE, which I believe is correct and desired behaviour.
(In reply to comment #19) > Created attachment 233555 [details] > main DEBUG log for jface.databinding compile error see below relevant snippet from the log. You can see that the compile classpath does not contain any of the SWT fragments which actually bring the classes (as the SWT host bundle is an empty shell). Since there is no reverse lookup host -> fragment(s) in p2 metadata, there has always been a hack [1] in the tycho code to work around SWT bug 361901 . Not sure yet how exactly this relates to the sceanrio here since SWT is being built in the same reactor (and not resolved from remote p2 repos), but it seems this hack just broke. [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-bundles/org.eclipse.tycho.p2.resolver.impl/src/main/java/org/eclipse/tycho/p2/resolver/ProjectorResolutionStrategy.java#n110 --- snip --- [INFO] --- tycho-compiler-plugin:0.18.0:compile (default-compile) @ org.eclipse.jface.databinding --- [DEBUG] Configuring mojo org.eclipse.tycho:tycho-compiler-plugin:0.18.0:compile from plugin realm ClassRealm[plugin>org.eclipse.tycho:tycho-compiler-plugin:0.18.0, parent: sun.misc.Launcher$AppClassLoader@17e1225d] [DEBUG] Configuring mojo 'org.eclipse.tycho:tycho-compiler-plugin:0.18.0:compile' with basic configurator --> [DEBUG] (f) basedir = /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding [DEBUG] (f) buildDirectory = /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target [DEBUG] (f) compilerArgument = -warn:-deprecation,raw,unchecked [DEBUG] (f) compilerArguments = {enableJavadoc=null, encoding=UTF-8, inlineJSR=null, log=/opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/@dot.xml, proceedOnError=null, verbose=null} [DEBUG] (f) compilerId = jdt [DEBUG] (f) debug = true [DEBUG] (f) encoding = UTF-8 [DEBUG] (f) excludeResources = [**/package.html] [DEBUG] (f) fork = false [DEBUG] (f) generatedSourcesDirectory = /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/generated-sources/annotations [DEBUG] (f) optimize = false [DEBUG] (f) outputFileName = org.eclipse.jface.databinding-1.6.200-SNAPSHOT [DEBUG] (f) project = MavenProject: org.eclipse.jface:org.eclipse.jface.databinding:1.6.200-SNAPSHOT @ /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/pom.xml [DEBUG] (f) requireJREPackageImports = false [DEBUG] (f) session = org.apache.maven.execution.MavenSession@70fd4aaf [DEBUG] (f) showDeprecation = false [DEBUG] (f) showWarnings = true [DEBUG] (f) staleMillis = 0 [DEBUG] (f) strictCompilerTarget = false [DEBUG] (f) useJDK = BREE [DEBUG] (f) verbose = false [DEBUG] -- end configuration -- [DEBUG] Manifest minimal BREE: OSGi profile 'CDC-1.0/Foundation-1.0' { source level: 1.3, target level: 1.1} [DEBUG] Effective EE: OSGi profile 'CDC-1.0/Foundation-1.0' { source level: 1.3, target level: 1.1} [DEBUG] Effective source/target: 1.3/1.1 [DEBUG] Using compiler 'jdt'. [DEBUG] Source directories: [/opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/src] [DEBUG] Classpath: [/opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/classes /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.swt/bundles/org.eclipse.swt/.[+org/eclipse/swt/*:+org/eclipse/swt/accessibility/*:+org/eclipse/swt/awt/*:+org/eclipse/swt/browser/*:+org/eclipse/swt/custom/*:+org/eclipse/swt/dnd/*:+org/eclipse/swt/events/*:+org/eclipse/swt/graphics/*:+org/eclipse/swt/layout/*:+org/eclipse/swt/opengl/*:+org/eclipse/swt/printing/*:+org/eclipse/swt/program/*:+org/eclipse/swt/widgets/*:~org/eclipse/swt/internal/*:~org/eclipse/swt/internal/image/*:~org/eclipse/swt/internal/theme/*:~org/eclipse/swt/internal/cocoa/*:~org/eclipse/swt/internal/accessibility/gtk/*:~org/eclipse/swt/internal/cairo/*:~org/eclipse/swt/internal/cde/*:~org/eclipse/swt/internal/gnome/*:~org/eclipse/swt/internal/gtk/*:~org/eclipse/swt/internal/mozilla/*:~org/eclipse/swt/internal/opengl/glx/*:+org/eclipse/swt/ole/win32/*:~org/eclipse/swt/internal/gdip/*:~org/eclipse/swt/internal/ole/win32/*:~org/eclipse/swt/internal/win32/*:~org/eclipse/swt/internal/opengl/win32/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface/target/classes[+org/eclipse/jface/*:+org/eclipse/jface/action/*:+org/eclipse/jface/action/images/*:+org/eclipse/jface/bindings/*:+org/eclipse/jface/bindings/keys/*:+org/eclipse/jface/bindings/keys/formatting/*:+org/eclipse/jface/commands/*:+org/eclipse/jface/contexts/*:+org/eclipse/jface/dialogs/*:+org/eclipse/jface/dialogs/images/*:+org/eclipse/jface/fieldassist/*:+org/eclipse/jface/fieldassist/images/*:+org/eclipse/jface/images/*:~org/eclipse/jface/internal/*:~org/eclipse/jface/internal/provisional/action/*:+org/eclipse/jface/layout/*:+org/eclipse/jface/menus/*:+org/eclipse/jface/operation/*:+org/eclipse/jface/preference/*:+org/eclipse/jface/preference/images/*:+org/eclipse/jface/resource/*:+org/eclipse/jface/util/*:+org/eclipse/jface/viewers/*:+org/eclipse/jface/viewers/deferred/*:+org/eclipse/jface/window/*:+org/eclipse/jface/wizard/*:+org/eclipse/jface/wizard/images/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.commands/target/classes[+org/eclipse/core/commands/*:+org/eclipse/core/commands/common/*:+org/eclipse/core/commands/contexts/*:+org/eclipse/core/commands/operations/*:+org/eclipse/core/commands/util/*:~org/eclipse/core/internal/commands/operations/*:~org/eclipse/core/internal/commands/util/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/rt.equinox.bundles/bundles/org.eclipse.equinox.common/target/classes[~org/eclipse/core/internal/boot/*:~org/eclipse/core/internal/runtime/*:+org/eclipse/core/runtime/*:+org/eclipse/equinox/events/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding.observable/target/classes[+org/eclipse/core/databinding/observable/*:+org/eclipse/core/databinding/observable/list/*:+org/eclipse/core/databinding/observable/map/*:+org/eclipse/core/databinding/observable/masterdetail/*:+org/eclipse/core/databinding/observable/set/*:+org/eclipse/core/databinding/observable/value/*:+org/eclipse/core/databinding/util/*:~org/eclipse/core/internal/databinding/identity/*:~org/eclipse/core/internal/databinding/observable/*:~org/eclipse/core/internal/databinding/observable/masterdetail/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding.property/target/classes[+org/eclipse/core/databinding/property/*:+org/eclipse/core/databinding/property/list/*:+org/eclipse/core/databinding/property/map/*:+org/eclipse/core/databinding/property/set/*:+org/eclipse/core/databinding/property/value/*:~org/eclipse/core/internal/databinding/property/*:~org/eclipse/core/internal/databinding/property/list/*:~org/eclipse/core/internal/databinding/property/map/*:~org/eclipse/core/internal/databinding/property/set/*:~org/eclipse/core/internal/databinding/property/value/*:?**/*] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding/target/classes[+org/eclipse/core/databinding/*:+org/eclipse/core/databinding/conversion/*:+org/eclipse/core/databinding/validation/*:~org/eclipse/core/internal/databinding/*:~org/eclipse/core/internal/databinding/conversion/*:~org/eclipse/core/internal/databinding/validation/*:?**/*]] [DEBUG] Output directory: /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/classes [DEBUG] Classpath: [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/classes [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.swt/bundles/org.eclipse.swt/.[+org/eclipse/swt/*:+org/eclipse/swt/accessibility/*:+org/eclipse/swt/awt/*:+org/eclipse/swt/browser/*:+org/eclipse/swt/custom/*:+org/eclipse/swt/dnd/*:+org/eclipse/swt/events/*:+org/eclipse/swt/graphics/*:+org/eclipse/swt/layout/*:+org/eclipse/swt/opengl/*:+org/eclipse/swt/printing/*:+org/eclipse/swt/program/*:+org/eclipse/swt/widgets/*:~org/eclipse/swt/internal/*:~org/eclipse/swt/internal/image/*:~org/eclipse/swt/internal/theme/*:~org/eclipse/swt/internal/cocoa/*:~org/eclipse/swt/internal/accessibility/gtk/*:~org/eclipse/swt/internal/cairo/*:~org/eclipse/swt/internal/cde/*:~org/eclipse/swt/internal/gnome/*:~org/eclipse/swt/internal/gtk/*:~org/eclipse/swt/internal/mozilla/*:~org/eclipse/swt/internal/opengl/glx/*:+org/eclipse/swt/ole/win32/*:~org/eclipse/swt/internal/gdip/*:~org/eclipse/swt/internal/ole/win32/*:~org/eclipse/swt/internal/win32/*:~org/eclipse/swt/internal/opengl/win32/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface/target/classes[+org/eclipse/jface/*:+org/eclipse/jface/action/*:+org/eclipse/jface/action/images/*:+org/eclipse/jface/bindings/*:+org/eclipse/jface/bindings/keys/*:+org/eclipse/jface/bindings/keys/formatting/*:+org/eclipse/jface/commands/*:+org/eclipse/jface/contexts/*:+org/eclipse/jface/dialogs/*:+org/eclipse/jface/dialogs/images/*:+org/eclipse/jface/fieldassist/*:+org/eclipse/jface/fieldassist/images/*:+org/eclipse/jface/images/*:~org/eclipse/jface/internal/*:~org/eclipse/jface/internal/provisional/action/*:+org/eclipse/jface/layout/*:+org/eclipse/jface/menus/*:+org/eclipse/jface/operation/*:+org/eclipse/jface/preference/*:+org/eclipse/jface/preference/images/*:+org/eclipse/jface/resource/*:+org/eclipse/jface/util/*:+org/eclipse/jface/viewers/*:+org/eclipse/jface/viewers/deferred/*:+org/eclipse/jface/window/*:+org/eclipse/jface/wizard/*:+org/eclipse/jface/wizard/images/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.commands/target/classes[+org/eclipse/core/commands/*:+org/eclipse/core/commands/common/*:+org/eclipse/core/commands/contexts/*:+org/eclipse/core/commands/operations/*:+org/eclipse/core/commands/util/*:~org/eclipse/core/internal/commands/operations/*:~org/eclipse/core/internal/commands/util/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/rt.equinox.bundles/bundles/org.eclipse.equinox.common/target/classes[~org/eclipse/core/internal/boot/*:~org/eclipse/core/internal/runtime/*:+org/eclipse/core/runtime/*:+org/eclipse/equinox/events/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding.observable/target/classes[+org/eclipse/core/databinding/observable/*:+org/eclipse/core/databinding/observable/list/*:+org/eclipse/core/databinding/observable/map/*:+org/eclipse/core/databinding/observable/masterdetail/*:+org/eclipse/core/databinding/observable/set/*:+org/eclipse/core/databinding/observable/value/*:+org/eclipse/core/databinding/util/*:~org/eclipse/core/internal/databinding/identity/*:~org/eclipse/core/internal/databinding/observable/*:~org/eclipse/core/internal/databinding/observable/masterdetail/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding.property/target/classes[+org/eclipse/core/databinding/property/*:+org/eclipse/core/databinding/property/list/*:+org/eclipse/core/databinding/property/map/*:+org/eclipse/core/databinding/property/set/*:+org/eclipse/core/databinding/property/value/*:~org/eclipse/core/internal/databinding/property/*:~org/eclipse/core/internal/databinding/property/list/*:~org/eclipse/core/internal/databinding/property/map/*:~org/eclipse/core/internal/databinding/property/set/*:~org/eclipse/core/internal/databinding/property/value/*:?**/*] [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.core.databinding/target/classes[+org/eclipse/core/databinding/*:+org/eclipse/core/databinding/conversion/*:+org/eclipse/core/databinding/validation/*:~org/eclipse/core/internal/databinding/*:~org/eclipse/core/internal/databinding/conversion/*:~org/eclipse/core/internal/databinding/validation/*:?**/*] [DEBUG] Source roots: [DEBUG] /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/src [INFO] Compiling 165 source files to /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/target/classes [DEBUG] Using javaHome: /shared/common/cbi/ee/CDC-1.0_Foundation-1.0 [DEBUG] Boot classpath access rules: [+java/**:+javax/microedition/io/*:?**/*] [DEBUG] Using boot classpath: [Classpath for jar file /opt/public/common/cbi/ee/CDC-1.0_Foundation-1.0/lib/ee.foundation-1.0.jar] [DEBUG] Using ext dirs: [] [DEBUG] Using endorsed dirs: [] [DEBUG] Original compiler output: ---------- 1. ERROR in /opt/public/eclipse/builds/4I/gitCache/eclipse.platform.releng.aggregator/eclipse.platform.ui/bundles/org.eclipse.jface.databinding/src/org/eclipse/jface/internal/databinding/swt/ControlVisibleProperty.java (at line 14) import org.eclipse.swt.widgets.Control; ^^^^^^^^^^^^^^^^^^^^^^^
from what I can tell from the build timestamps, this is a candidate suspect for the commit that broke the build http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=eb4a566fe3bbefedab594865c6cfba14ed29c7d7
(In reply to comment #22) > ... The build failure highlighted the fact > that the project will not run in its declared BREE, which I believe is > correct and desired behaviour. Not sure this is what you mean ... but, we in the Platform need to leave open the possibility that others may want to "mix and match" plugins ... sometimes even from different releases ... so that's why we have BREE's that don't make sense if we were building "one application". I suppose "the maven way" to do that is then to build all those pieces separately? ... but ... not sure we can do that any time soon.
(In reply to comment #25) > (In reply to comment #22) > > ... The build failure highlighted the fact > > that the project will not run in its declared BREE, which I believe is > > correct and desired behaviour. > > Not sure this is what you mean ... but, we in the Platform need to leave > open the possibility that others may want to "mix and match" plugins ... > sometimes even from different releases ... so that's why we have BREE's that > don't make sense if we were building "one application". I suppose "the maven > way" to do that is then to build all those pieces separately? ... but ... > not sure we can do that any time soon. The "maven way" is to configure project build target platform to include SWT fragment(s) compatible with the project BREE. Alternatively, you can override project BREE with explicit pom.xml configuration, which will essentially bypass any BREE validation for the project, but this appears to be what you want.
(In reply to comment #22) > The build failure highlighted the fact > that the project will not run in its declared BREE, which I believe is > correct and desired behaviour. If that is the case, why org.eclipse.ui (CDC1.0/J2SE1.3) which depends on org.eclipse.jface (J2SE1.5) compiles without any problems?
(In reply to comment #27) > (In reply to comment #22) > > The build failure highlighted the fact > > that the project will not run in its declared BREE, which I believe is > > correct and desired behaviour. > > If that is the case, why org.eclipse.ui (CDC1.0/J2SE1.3) which depends on > org.eclipse.jface (J2SE1.5) compiles without any problems? Good question. I have not build Platform in very long time, but from a cursory look I think org.eclipse.ui build should indeed fail... unless there is BREE override somewhere I can't see or the build is able to pull compatible jface version from one of enabled remote repositories. Bug in tycho is also a possibility.
(In reply to comment #24) > from what I can tell from the build timestamps, this is a candidate suspect > for the commit that broke the build > > http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/ > ?id=eb4a566fe3bbefedab594865c6cfba14ed29c7d7 Yes, the changes done for bug#411356 broke the build. I have reverted these changes so that we can have an successful integration build. http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=a93fce3f5105854eeb8341437779e5e3e2bb72f1 http://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/?id=7a969c0b205c07b32c882f6dbf3d2fd379534c8e
(In reply to comment #17) > As Thomas mentioned, there might be a bug when it comes to compiling > fragments, > I'm not sure how the effective BREE of fragment and host should be computed > during build (should fragment BREE be merged into host and vice versa? From > the merged BREEs, is the most restrictive BREE the effective one?). > No merging should be done from the perspective of the OSGi resolver for the BREE. The BREE requirement is what OSGi calls a non-payload fragment requirement. This means it is not merged into the requirements from the host perspective and should in no way change the effective class space (or classpath) of the host. I see tycho is using the OSGi resolver in devmode [1] [1] - http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-core/src/main/java/org/eclipse/tycho/core/osgitools/EquinoxResolver.java#n142 This should allow the target project to resolve as long the properties configured with the resolver state include the necessary EE profile. This is true even if one of its dependencies depends on a higher EE profile (unless there is some bug in the OSGi resolver). I believe the target project must be resolving, because we don't see any resolver errors. So this indicates that either the fragments of SWT are not being allowed to attach to their host when in dev mode, or something else is detecting that the fragments are not really resolved. I will try to see if there is a bug in the OSGi resolver that is preventing the attachment of the fragments. What version of org.eclipse.osgi is being used during the build by tycho?
(In reply to comment #30) > I will try to see if there is a bug in the OSGi resolver that is preventing > the attachment of the fragments. What version of org.eclipse.osgi is being > used during the build by tycho? I tried a few tests with the Equinox resolver directly and have not been able to find anything suspicious with development mode and attachent of fragments to their host when either the fragment or the host has a BREE that does not match. I would like to confirm that development mode is always used by tycho for resolution. Also can you confirm that non-incremental resolves are always performed like in [1]? [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-core/src/main/java/org/eclipse/tycho/core/osgitools/EquinoxResolver.java#n91
(In reply to comment #30) > What version of org.eclipse.osgi is being > used during the build by tycho? kepler RC3 for tycho 0.18.0
(In reply to comment #31) > I would like to confirm that development mode is always > used by tycho for resolution. confirmed. > Also can you confirm that non-incremental > resolves are always performed like in [1]? confirmed.
not sure if directly related but there is one thing definitely fishy about SWT commit http://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/?id=5fba4321d581bbd674b16968240b2ba2ff039c2c it has several MANIFEST files with no newline at EOF like: +Bundle-RequiredExecutionEnvironment: J2SE-1.5 \ No newline at end of file which means the last line of the MANIFEST (in this case specifying BREE) will be ignored AFAIK.
(In reply to comment #34) Silenio already added the terminating newlines again in the next commit, but that also didn't help. The Java Tutorials [1] also tell the MANIFEST.MF must end with a newline, but if you closely check the grammar in the spec [2], then you see it must even end with two newlines. The spec also contains a list of "additional restrictions and rules", which are inconsistent with [1], since they say that the terminating newlines should not be necessary at all: "Before parsing: [..] Two newlines are appended [..]". There's already something in the build that re-formats MANIFEST.MF files (applies line length limit, converts newlines to CRLF, and adds another newline if there was just a single one at the end), so I guess the missing newline would also have been taken care of by that tool. [1] http://docs.oracle.com/javase/tutorial/deployment/jar/modman.html [2] http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html
(In reply to comment #35) > (In reply to comment #34) > Silenio already added the terminating newlines again in the next commit, but > that also didn't help. right, missing line endings at EOF are not the root cause here. Just to let you know I have been trying to strip this down to a small sample project but still not quite there yet, will continue. I am suspecting that the SWT fragments are added to the equinox resolver state but due to their J2SE-1.5 BREE (which is higher than the CDC-1.0/Foundation-1.0 BREE of the consuming bundle org.eclipse.jface.databinding in this case), they are not attached to their host, thus missing on the compile classpath.
(In reply to comment #35) > There's already something in the build that re-formats MANIFEST.MF files > (applies line length limit, converts newlines to CRLF, and adds another > newline if there was just a single one at the end), so I guess the missing > newline would also have been taken care of by that tool. sligtly off-topic, but this is just a java.util.jar.Manifest read() from project root/write() to target/ folder (with write using normalized MANIFEST format). So if something gets lost during read, it will still be lost after write. See tycho bug 377249
Created attachment 233667 [details] stripped down sample project small(er) sample project to more easily reproduce the problem. steps to reproduce: 1. unzip sample 2. mvn clean verify build is expected to succeed. Now change Bundle-RequiredExecutionEnvironment in both org.eclipse.swt and org.eclipse.swt.gtk.linux.x86 from CDC-1.0/Foundation-1.0 to J2SE-1.5. Build fails now with compilation errors due to missing swt fragment on classpath. I have tracked it down to DependencyComputer.addDependency() [1] for (int i = 0; i < fragments.length; i++) { if (fragments[i].isResolved() && !isPatchFragment(fragments[i])) { addDependency(fragments[i], added, map, entries, useInclusion); } } fragments[i].isResolved() returns false for project org.eclipse.jface.databinding when BREE of SWT is changed to J2SE-1.5. This is why it's not added to the dependencies and subsequently the build fails. The OSGi resolver for project org.eclipse.jface.databinding is using minimum declared BREE of the this bundle, i.e. CDC-1.0/Foundation-1.0, so it seems correct to me that in this execution environment, the fragment is not resolved. OTOH the SWT host bundle is added to the dependencies even if it also has BREE J2SE-1.5 now, so at least hosts and fragments are not treated consistently here. Not sure if we should restore consistency by 1. ignoring isResolved() for fragments or 2. by requiring matching BREEs for normal dependencies too I think 2. would have a huge ripple effect on all SWT consumers. Or maybe there is another way to solve it? @Igor I'm wondering if the fragment isResolved() check was actually meant to filter out the non-matching os/ws/arch fragments. If that is the case we could probably find other ways to filter on os/ws/arch only. --- How to debug the sample project build? mvnDebug clean verify attach remote debugger to port 8000. Also see [2] for project setup (check out tag 'tycho-0.18.0') [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-core/src/main/java/org/eclipse/tycho/core/osgitools/DependencyComputer.java?h=tycho-0.18.x#n190 [2] http://wiki.eclipse.org/Developing_Tycho
(In reply to comment #38) > 2. by requiring matching BREEs for normal dependencies too That would mean the BREE is considered re-exported, something like a Require-Bundle: <BREE>;visibility:=reexport There are good reasons why Require-Bundle doesn't re-export dependencies by default, and the same reasons apply to the BREE as well. Even if SWT decides to use a 1.5 BREE in Luna, it still makes sense to allow org.eclipse.jface.databinding to require a different BREE. As long as SWT doesn't break APIs (i.e. doesn't increase the major version), it's OK to reference it from code that is at an earlier BREE. If someone want to run in an earlier EE, they just have to find a version of SWT that satisfies all their constraints (acceptable EE; minimum version of required bundles and imported packages). That version of SWT can either be an earlier version or a version from a different provider.
(In reply to comment #39) > (In reply to comment #38) > > 2. by requiring matching BREEs for normal dependencies too > > That would mean the BREE is considered re-exported, something like a > Require-Bundle: <BREE>;visibility:=reexport > > There are good reasons why Require-Bundle doesn't re-export dependencies by > default, and the same reasons apply to the BREE as well. > > Even if SWT decides to use a 1.5 BREE in Luna, it still makes sense to allow > org.eclipse.jface.databinding to require a different BREE. As long as SWT > doesn't break APIs (i.e. doesn't increase the major version), it's OK to > reference it from code that is at an earlier BREE. If someone want to run in > an earlier EE, they just have to find a version of SWT that satisfies all > their constraints (acceptable EE; minimum version of required bundles and > imported packages). That version of SWT can either be an earlier version or > a version from a different provider. makes sense to me. looks like we need to find a way to ignore isResolved() for fragments but still make sure we only add the fragment with matching os/ws/arch.
FYI, Databinding is also planning to introduce generics hence they plan to move to Java 1.5. See Bug 335792 for details. Currently this work is done in the e4 repos so it may take a while before this is merged.
(In reply to comment #40) > (In reply to comment #39) > > (In reply to comment #38) > > > 2. by requiring matching BREEs for normal dependencies too > > > > That would mean the BREE is considered re-exported, something like a > > Require-Bundle: <BREE>;visibility:=reexport > > > > There are good reasons why Require-Bundle doesn't re-export dependencies by > > default, and the same reasons apply to the BREE as well. > > > > Even if SWT decides to use a 1.5 BREE in Luna, it still makes sense to allow > > org.eclipse.jface.databinding to require a different BREE. As long as SWT > > doesn't break APIs (i.e. doesn't increase the major version), it's OK to > > reference it from code that is at an earlier BREE. If someone want to run in > > an earlier EE, they just have to find a version of SWT that satisfies all > > their constraints (acceptable EE; minimum version of required bundles and > > imported packages). That version of SWT can either be an earlier version or > > a version from a different provider. > > makes sense to me. looks like we need to find a way to ignore isResolved() > for fragments but still make sure we only add the fragment with matching > os/ws/arch. How does this make sense? Build target platform contains SWT fragment that cannot be resolved given project BREE. I don't think Tycho should make any assumptions about other SWT fragment versions that may or may not be available at runtime. If project declares BREE=1.4 I believe it is totally reasonable, and in fact expected and desired, to fail the build if all available and required SWT fragments declare 1.5 as their BREE.
(In reply to comment #42) > How does this make sense? Build target platform contains SWT fragment that > cannot be resolved given project BREE. I don't think Tycho should make any > assumptions about other SWT fragment versions that may or may not be > available at runtime. If project declares BREE=1.4 I believe it is totally > reasonable, and in fact expected and desired, to fail the build if all > available and required SWT fragments declare 1.5 as their BREE. The use of Java 5 by SWT is an implementation detail of SWT that is not exposed to the client. It should not force all consumers to bump up their BREE. A BREE is not considered part of the API signature of a bundle. I think you are mixing the ability to build a bundle and the ability to deploy that bundle with all of its dependencies and resolve them at runtime. If someone wants to deploy the bundle that has BREE J2SE-1.4 which depends on SWT to a system running on J2SE-1.4 then they need to find a version of SWT which also supports BREE J2SE-1.4. PDE-Build gets around this issue by configuring every BREE profile with the resolver State [1] [1] http://git.eclipse.org/c/pde/eclipse.pde.build.git/tree/org.eclipse.pde.build/src/org/eclipse/pde/internal/build/site/PDEState.java#n461
(In reply to comment #43) > (In reply to comment #42) > > How does this make sense? Build target platform contains SWT fragment that > > cannot be resolved given project BREE. I don't think Tycho should make any > > assumptions about other SWT fragment versions that may or may not be > > available at runtime. If project declares BREE=1.4 I believe it is totally > > reasonable, and in fact expected and desired, to fail the build if all > > available and required SWT fragments declare 1.5 as their BREE. > > The use of Java 5 by SWT is an implementation detail of SWT that is not > exposed to the client. It should not force all consumers to bump up their > BREE. A BREE is not considered part of the API signature of a bundle. I > think you are mixing the ability to build a bundle and the ability to deploy > that bundle with all of its dependencies and resolve them at runtime. If > someone wants to deploy the bundle that has BREE J2SE-1.4 which depends on > SWT to a system running on J2SE-1.4 then they need to find a version of SWT > which also supports BREE J2SE-1.4. > > PDE-Build gets around this issue by configuring every BREE profile with the > resolver State [1] > > > [1] > http://git.eclipse.org/c/pde/eclipse.pde.build.git/tree/org.eclipse.pde. > build/src/org/eclipse/pde/internal/build/site/PDEState.java#n461 As a build tool, Tycho is expected to detect and report as many runtime problems as reasonably possible. In this particular case, combination of bundles available at build time will not resolve under java 1.4 and there is no information available at build time that Tycho can use to determine that 1.4-compatible SWT fragment will indeed be available available at runtime. I strongly believe current Tycho behaviour is correct and desired. A project with BREE=1.4 must have compatible SWT fragment included in its build target platform or the build must fail.
(In reply to comment #42) > I don't think Tycho should make any > assumptions about other SWT fragment versions that may or may not be > available at runtime. Addition to my original explanation and to comment 43: Our build combines these versions of SWT and jface.databinding, but that doesn't mean every build has to use the same combinations. Tycho could emit a warning, but it must not abort the build, since this combination is perfectly valid. The goal of APIs is to reduce coupling to the essential parts, and the BREE is not an essential property of a bundle that should restrict modular reuse. Note that we're not only building a product, but also a set of reusable bundles.
Last time I checked, SWT did not have an API, only implementation provided by SWT fragments.
(In reply to comment #46) > Last time I checked, SWT did not have an API, only implementation provided > by SWT fragments. I'd be interested in why you say that ... it does have API ... are you talking about "Interfaces" vs. "Concrete" classes? Or, just some mistake in our documentation?
There is no common set of classes/interfaces shared by SWT on all (supported) platforms. SWT bundle is essentially empty, all code comes from platform-specific fragments. You can argue that they all are produced from same sources, so are the same, but as a consumer I can't tell if SWT public classes/interfaces on windows are actually the same on linux.
A few recent SWT builds didn't declare Export-Package in the o.e.swt bundle, but that's fixed now (bug 411672 comment 7). But anyway, that SWT ships implementations in fragments doesn't mean they are not API. For OSGi, it doesn't matter whether Export-Package or implementations are in the main bundle or in a fragment. The problem here is that Tycho fails with SWT's specific setup, but it works correctly in other cases where a bundle's dependencies includes another bundle with a higher BREE. OSGi defines how bundles work together, and it doesn't preclude such dependencies.
To work around this in SWT you may be able to move the SWT host bundle up to J2SE-1.5 BREE and remove the BREEs from the fragments. The fragments depend on the latest version of their host so they can depend on running on Java 5 if they are resolved to the latest host at runtime. You may just need to add some pom magic to make sure the fragments are compiled against Java 5, but I'm not really sure how the SWT fragments are built since they are not in separate projects.
(In reply to comment #50) > To work around this in SWT you may be able to move the SWT host bundle up to > J2SE-1.5 BREE and remove the BREEs from the fragments. workaround confirmed
I believe this is best viewed as a Tycho bug or (serious) limitation. Rather than open a new bug, I will move this one to Tycho since it has so much good discussion here. SWT might make use of the work around in comment 50, but not sure that's ideal long term, and not sure Equinox can (bug 415116). There are obviously different views on "the correct way" to do a build -- and in my opinion they each "make sense" for different purposes or use case. So, the solution to that "build philosophy" problem is easy ... make it a optional preference ... keep current behavior by default, but allow users to specify <devmode>true</devmode>, or similar. Hopefully that would allow the fix outlined in comment 40 to be feasible. Note: I do consider this a "bug", not a "feature request" since if nothing else, Tycho is inconsistent ... having different behavior for plugins vs. fragments. (Which is one reason we had to spend so much time just figuring out what the problem was). [I suppose a possible deluxe solution would allow for different preferences for bundles and fragment resolution ... but, not sure we in Platform (or anyone) would ever need that ... I suspect most projects either want "devmode" throughout, or not at all? In fact, at Eclipse, recent Architecture Council discussion did conclude that BREE is an implementation detail and that the one in a prereq should not effect your own BREE setting. See http://wiki.eclipse.org/Architecture_Council/Meetings/September_12_2013#Minimum_BREE. (And, I wasn't even at that meeting :) ] And, I'll be explicit, I do consider this a "blocker" in the sense that it blocks us from getting a correct build. While there are "work arounds" suggested, those work arounds simply allow the build to continue without failing ... not to get a correct build. Appreciate your help.
I don't see how it makes sense to include bundles that require J2SE1.5 in build dependencies of a project that targets J2SE-1.4 or earlier/restricted BREE. Even if this is an option and turned off by default, I am sure somebody will turn it on and then come to us for support. What may make sense, is to allow earlier SWT versions to be considered during dependency resolution. This way projects that target J2SE1.5 will be built with SWT from the current build and projects that require earlier/lesser BREE will be built with an older SWT.
(In reply to Igor Fedorenko from comment #53) > I don't see how it makes sense to include bundles that require J2SE1.5 in > build dependencies of a project that targets J2SE-1.4 or earlier/restricted > BREE. Even if this is an option and turned off by default, I am sure > somebody will turn it on and then come to us for support. > > What may make sense, is to allow earlier SWT versions to be considered > during dependency resolution. This way projects that target J2SE1.5 will be > built with SWT from the current build and projects that require > earlier/lesser BREE will be built with an older SWT. I appreciate the constructive reply, and do not know how to explain it better, but I feel something is not getting across, or else I am not understanding you? In the few concrete cases I know of where people wanted to try and "mix and match", they deliberately want the NEW version of something like swt (or fragments) due to fixes or improvements made, but not "all of Eclipse" (usually due to their use of highly custom code that makes use of "non api" that would be hard to migrate). While we don't encourage people to do that, since it is more "testing burden" on them, and may or may not work in the end ... we don't want to stand in their way of even trying, and they would not be happy to hear "we can no longer provide that flexibility because the new new builder we moved to does not allow it." I definitely don't understand what you mean by "come to us for support". Support for what? Clearly not SWT. Do you mean you are concerned someone would complain because they'd say "you should have failed the build, even though I used this option that said it would not fail the build"? What am I missing? Again, our OSGi goals are all about "componentization" ... not "app building".
Lets consider a build with two bundles, bundleA and bundleB, bundleB depends on bundleA, both bundles have version 1.1 and both require J2SE-1.4. In version 1.2 bundleA moved to J2SE1.5 and introduced some new API, while bundleB stayed on J2SE-1.4. There are several ways to build this. 1. Use BREE=J2SE1.5 during bundleB build and manually set compiler source/target to 1.4. Tycho will use bundleA 1.2 from the same build during the build. This will produce correct bundleB but runs the risk that bundleB can accidentally use new APIs introduced in Java 1.5 and bundleA 1.2. This should already be possible as of Tycho 0.18.1. 2. Use BREE=J2SE-1.4 and bundleA 1.1 to build bundleB. This is my recommendation because it validates the bundle only uses APIs available when running under java 1.4. This is not possible now and will require changes to Tycho. 3. What I believe you propose is to use BREE=J2SE-1.4 and force bundleA 1.2 during bundleB build. Although this will validate bundleB only uses java 1.4 API, it will not detect use of new APIs introduced in bundleA 1.2 and this will result runtime errors for users who actually try running bundleB under java 1.4. It can also result in hard to explain and workaround build failures if bundleA 1.2 exposes java 1.5 classes through its APIs. As a side note, I find it strange that java compiler even allows building classes that target java 1.4 against dependencies compiled with target=1.5+. I understand why it is necessary for dependencies coming JDK, but not normal dependencies. I just don't see why introduce option #3 when #1 is already possible. Both will require manual validation that bundleB runs on BREE=J2SE-1.4, but #3 uses impossible BREE/bundleA combo which will inevitably cause confusion and result in questions/bugreports. And if you really care about BREE=J2SE-1.4, then #2 is the only way to go IMO. It will properly validate BREE=J2SE-1.4 compatibility at compile time and it will also allow BREE=J2SE-1.4 when running bundleB tests, which I do not believe will be possible with #1 or #3.
(In reply to Igor Fedorenko from comment #55) I think I better see where you are coming from ... that you are "thinking ahead" ... how to test each bundle "as is is built" ... which we in the Eclipse Platform do not currently do, so have never faced that complication. (If we did "test while we built", we'd likely want to "manually" set the JRE to test against so it could be tested with a different level than what it was built with). I agree our desired "mix and match" approach can in theory can lead to complications, if one bundle "leaks" API from the JRE. And, yes, the JDT compiler does have to be careful in what it assumes about "pre-req" JRE levels (bug 360164). While a "different case", remember that not all bundles come from the same place, and in many cases (in large "enterprise" settings, like Eclipse Foundation :) may be already built and coming from a repo, so we have no (direct) control over what BREE they set. It is very important that we not force "everyone in the stack" to react, simply because a "lower level" bundle found it necessary to change their BREE level. But, let me be quick to point out, as far as I know, we can currently -- with existing Tycho 0.18.1 -- "do what we want to do" with bundles, and are only finding this limitation with fragments. So a) hope you are not suggesting you "fix" Tycho so bundles are no longer "working" for us :) and b) does this change your mind any about making a "small" change to loosen resolution requirements for fragments?
I believe existing limitation only applies for bundles together. It is already possible to have multiple versions of dependencies coming from repositories, so projects just need specify repositories with dependency versions that match their BREE. I am not aware of any "looseness" in Tycho dependency resolution, so don't know if there is anything to change/fix. Do you have an example of bundles that build with dependencies that are incompatible with build effective BREE? For bundles that fail due to SWT bundle fragment incompatible with J2SE-1.4, why can't you force build-time BREE to J2SE-1.5 but keep compile source/target at 1.4? This shouldn't require any changes to Tycho and seems like the most pragmatic approach.
(In reply to Igor Fedorenko from comment #57) > I believe existing limitation only applies for bundles together. It is > already possible to have multiple versions of dependencies coming from > repositories, so projects just need specify repositories with dependency > versions that match their BREE. Not sure that's always possible or desirable. As in bug 360164 XSDImpl (in WTP) was at 1.4, but EMF moved up to 1.5, and XSDImpl did not want to build (or run) against an "old" version of EMF. (And, in fact, it was working fine, and continues to work fine, and was just a momentary problem since JDT made some optimizations that mistakenly made similar assumptions that that "all prereqs were at same or lower JRE level ... but, they were eventually convinced :) > > I am not aware of any "looseness" in Tycho dependency resolution, so don't > know if there is anything to change/fix. Do you have an example of bundles > that build with dependencies that are incompatible with build effective BREE? > org.eclipse.osgi (bug 415116) is the problematic case, if that's what you mean. But, suspect you mean "cases that work"? I think there are several, but a quick peek shows o.e.pde.runtime specifies BREE of 1.4, but depends on o.e.jdt.ui which uses BREE of 1.5 and that all seems to work fine. > For bundles that fail due to SWT bundle fragment incompatible with J2SE-1.4, > why can't you force build-time BREE to J2SE-1.5 but keep compile > source/target at 1.4? This shouldn't require any changes to Tycho and seems > like the most pragmatic approach. This MIGHT work for SWT ... since they are using 1.5 to "get" generics, which should produced compatible binaries? ... I'll have to let component lead comment, I'm not sure. Silenio, can you clarify? But, won't work for org.eclipse.osgi since they want to use 1.6, specifically for libraries that are not in 1.5. But, again, component lead may need to clarify. Tom?
(In reply to David Williams from comment #58) > But, won't work for org.eclipse.osgi since they want to use 1.6, > specifically for libraries that are not in 1.5. But, again, component lead > may need to clarify. Tom? If this fragment/host restriction is not lifted I'm just going to force all the fragments of org.eclipse.osgi to move up to Java 6. I'm not really interested in any more pom meta-data that is different than the specified bundle manifest meta-data for the execution environment. To me that just causes future confusion. Its really a non-issue for me since these fragments are so tightly coupled with org.eclipse.osgi that their minimum host-range will never allow them to run on with older org.eclipse.osgi that did require older BREE. > org.eclipse.osgi (bug 415116) is the problematic case, if that's what you > mean. > But, suspect you mean "cases that work"? I think there are several, but a > quick peek shows o.e.pde.runtime specifies BREE of 1.4, but depends on > o.e.jdt.ui which uses BREE of 1.5 and that all seems to work fine. > There are loads of cases that work fine for normal bundles since the org.eclipse.osgi states JavaSE-1.5 is the minimum BREE yet we are still building lots of bundles with J2SE-1.4 as the minimum BREE in our build and these bundles depend on org.eclipse.osgi in one form or another. It is only the case where building a fragment which does not match the BREE of its host that does not "work".
(In reply to David Williams from comment #58) > > For bundles that fail due to SWT bundle fragment incompatible with J2SE-1.4, > > why can't you force build-time BREE to J2SE-1.5 but keep compile > > source/target at 1.4? This shouldn't require any changes to Tycho and seems > > like the most pragmatic approach. > > This MIGHT work for SWT ... since they are using 1.5 to "get" generics, > which should produced compatible binaries? ... I'll have to let component > lead comment, I'm not sure. Silenio, can you clarify? > SWT does not have any pressing reason to move to 1.5 other than the fact that that is what the community would like to happen (bug#411356). At some point, we introduced a call to URLDecoder.decode(java.lang.String, java.lang.String) which was only available on 1.4 (bug#412067), but that is no longer the case. I think what Igor suggested does not really affect SWT. If I understood correctly, SWT would move to BREE 1.5 and bundles that depend on SWT and still have BREE 1.4 or less (see comment#18 for a list) would have to change their compiling settings (i.e source/target level). It seems like we would be pushing the problem on those bundles anyways. Frankly, there is a inconsistency in the way tycho enforces BREE prereqs depending on host bundles vs fragments. At the very least this inconsistency should be resolved. It could either apply the same restrictions to host bundles or remove the restrictions for fragments. I believe the latter would cause quite a bit of grief in the Eclipse Platform build.
I can't tell why Tycho allows dependencies that are incompatible with build effective BREE. It shouldn't. If somebody provides a small standalone example that demonstrates this behaviour, I can explain what happens and suggest the next steps. I agree Tycho should handle bundles and fragments consistently but I don't believe ignoring BREE during dependency resolution is the desired behaviour. I also want to reiterate my recommendation to split monolith 'aggregator' Platform build into a number of smaller component builds. This way each component will be able to choose dependencies that make the most sense for the component and not try to find one universal set that works for entire Platform.
(In reply to Igor Fedorenko from comment #61) > ... I don't believe ignoring BREE during dependency resolution > is the desired behaviour. It is the desired behavior ... sorry to just keep repeating that, but it is ... at least desired by "us" (and other) projects during builds. Perhaps some one who is more of an expert than I, can better describe OSGi "devmode" resolution but think Tom does a pretty good job in comment 30. What more needs to be said? > I also want to reiterate my recommendation to split monolith 'aggregator' > Platform build into a number of smaller component builds. Yeah, we thought someone was going to do that for us, but then we heard them say it was impossible. :) If you have concrete suggestions on how to make incremental progress towards that goal, bug 398613 is the place to suggest them. And, it would be appreciated. But, that would not change the issue this bug is about. We'd still not want to force one bundle to "move up" its bree level, for the simple reason that some pre-req moved up its bree level. > If somebody provides a small standalone > example that demonstrates this behaviour, ... You've said this in response to a few bugs. I'm sorry you don't find the Eclipse SDK build small enough for you to debug ... but "small" is relative (i.e. it could be bigger :) ... what exactly is the problem in using the Eclipse SDK build to debug or understand this problem better? After all, we (all) went to a lot of pain so "anyone could run the build" and now it just sounds like you are saying "that's not enough". [I mean, if its just machine memory or something, I'd understand ... but, that's why I'm asking, I want to understand.]
(In reply to David Williams from comment #62) > (In reply to Igor Fedorenko from comment #61) > > ... I don't believe ignoring BREE during dependency resolution > > is the desired behaviour. > > It is the desired behavior ... sorry to just keep repeating that, but it is > ... at least desired by "us" (and other) projects during builds. Perhaps > some one who is more of an expert than I, can better describe OSGi "devmode" > resolution but think Tom does a pretty good job in comment 30. What more > needs to be said? There is obviously a disconnect here, quite likely due to some misunderstanding on my part. If other Tycho committers "see it" and know how to support this usecase without unwanted side-affects, I have no problems with that, but I can't implement something I don't fully understand myself. This is why I ask for a small bundleA/bundleB/fragmentB illustration I can use to understand the desired behaviour "in general" first and then see how it can be applied to specific case of SWT.
Not sure when or if I could provide a good example (other than eclipse SDK build, which is understandably hard to work with for this wort of debugging) but at status call today, the two primary components that this was "blocking" thought they had "workarounds" of a sort ... Tom can move equinox fragments up to 1.6 so they 'match' and Silenio can simply leave SWT at 1.4 so it "matches" in the opposite way. So, changing severity to "major" (meaning "missing function") to be more accurate of current impact.
(In reply to comment #62) > (In reply to Igor Fedorenko from comment #61) > > If somebody provides a small standalone > > example that demonstrates this behaviour, ... > > You've said this in response to a few bugs. I'm sorry you don't find the Eclipse > SDK build small enough for you to debug ... but "small" is relative (i.e. it > could be bigger :) ... what exactly is the problem in using the Eclipse SDK > build to debug or understand this problem better? I second Igor's request for a small, standalone example for the following reasons: * The standalone example project ensures that we are talking about the same issue. I have had the experience that other effects in real projects covered up the problem that should have been discussed. (This argument alone disqualifies the Eclipse SDK build.) * Creating the minimal, standalone example brings valuable experience: You think you know that a bug causes your build failure? Then reproduce the bug in a minimal project. Often it turns out that some other configuration is actually responsible or relevant for the observed problem. By cutting down the configuration to the minimum to reproduce the problem, you often learn much about what you have actually configured in your build. * Creating a minimal, standalone example is part of the work needed to fix the problem (for projects that take testing seriously). If the standalone project is contributed, this is part of the work done, and this may increase the willingness of the committers to do the rest of the work.
Created attachment 236757 [details] stripped down to absolute minimum, self-contained sample project steps to reproduce: 1. unzip sample project 2. mvn clean verify => build fails (after ~10 seconds) with error below. Now remove BREE entries from MANIFEST of both bundle org.eclipse.swt and fragment org.eclipse.swt.gtk.linux.x86 => build succeeds --- error log snippet --- [ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.18.0:compile (default-compile) on project org.eclipse.jface.databinding: Comp ilation failure: Compilation failure: [ERROR] C:\tmp\swt-bree-sample\org.eclipse.jface.databinding\src\org\eclipse\jface\internal\databinding\swt\ButtonImageProperty.java:[3,0] [ERROR] import org.eclipse.swt.graphics.Image; [ERROR] ^^^^^^^^^^^^^^^ [ERROR] The import org.eclipse.swt cannot be resolved [ERROR] C:\tmp\swt-bree-sample\org.eclipse.jface.databinding\src\org\eclipse\jface\internal\databinding\swt\ButtonImageProperty.java:[7,0] [ERROR] Image getImage() { [ERROR] ^^^^^ [ERROR] Image cannot be resolved to a type [ERROR] 2 problems (2 errors)
https://git.eclipse.org/r/26728
(In reply to comment #67) > https://git.eclipse.org/r/26728 This proposed change added the option to specify a different execution environment for the dependency resolution. While this would help, it may not yet solve all transitive EE resolution problems. Instead of specifying one EE for dependency resolution, the PDE build provided all (known) EEs to the resolver. Doing it in the same way as the PDE build seems to be the preferred option of our users, so we'll probably go for this option instead.
(In reply to Tobias Oberlies from comment #68) > While this would help, it may not yet solve all transitive EE resolution > problems. Instead of specifying one EE for dependency resolution, the PDE > build provided all (known) EEs to the resolver. Doing it in the same way as > the PDE build seems to be the preferred option of our users, so we'll > probably go for this option instead. https://git.eclipse.org/r/#/c/26728/
http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=b699e6c4b09203c4a87c03b8085af441087b2c71 with <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <resolveWithExecutionEnvironmentConstraints>false</resolveWithExecutionEnvironmentConstraints> </configuration> </plugin> the p2 and Equinox resolver will use all known execution environments
(In reply to Jan Sievers from comment #70) > http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/ > ?id=b699e6c4b09203c4a87c03b8085af441087b2c71 > > with > > <plugin> > <groupId>org.eclipse.tycho</groupId> > <artifactId>target-platform-configuration</artifactId> > <version>${tycho-version}</version> > <configuration> > <resolveWithExecutionEnvironmentConstraints>false</ > resolveWithExecutionEnvironmentConstraints> > </configuration> > </plugin> > > > > the p2 and Equinox resolver will use all known execution environments Just to clarify, the "default behavior" is as it was before, and to get the "all known EEs", one has to specify "false", for resolveWithExecutionEnvironmentConstraints. Right?
(In reply to David Williams from comment #71) > Just to clarify, the "default behavior" is as it was before, and to get the > "all known EEs", one has to specify "false", for > resolveWithExecutionEnvironmentConstraints. Right? yes
