Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 381192 - SWT is not ready for Tycho builds
Summary: SWT is not ready for Tycho builds
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.2   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: Krzysztof Daniel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 382486 (view as bug list)
Depends on: 382344 382470 382480 382483 382486
Blocks: 372794
  Show dependency tree
 
Reported: 2012-05-31 09:20 EDT by Krzysztof Daniel CLA
Modified: 2014-01-16 06:23 EST (History)
9 users (show)

See Also:


Attachments
Mavenizing linux x86_64 fragment. (5.99 KB, patch)
2012-06-01 08:17 EDT, Krzysztof Daniel CLA
no flags Details | Diff
change in org.eclipse.swt build files that enables fragment building (1.77 KB, patch)
2012-06-01 08:30 EDT, Krzysztof Daniel CLA
no flags Details | Diff
org.eclipse.swt.gtk.linux.x86_64/build.properties (630 bytes, text/plain)
2012-06-05 08:51 EDT, Krzysztof Daniel CLA
no flags Details
org.eclipse.swt/pom.xml (1.37 KB, application/xml)
2012-06-11 04:19 EDT, Krzysztof Daniel CLA
no flags Details
org.eclipse.swt.gtk.linux.x86_64/pom.xml (3.10 KB, application/xml)
2012-06-11 04:23 EDT, Krzysztof Daniel CLA
no flags Details
eclipse.platform.swt.binaries/pom.xml (699 bytes, application/xml)
2012-06-11 04:25 EDT, Krzysztof Daniel CLA
no flags Details
Patch for the binaries repo (21.09 KB, patch)
2012-06-12 08:27 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Using pkg-config instead of hardcoded paths to build swt on linux (1.65 KB, patch)
2012-06-13 04:20 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Building all platforms. (13.12 KB, patch)
2012-06-13 10:52 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Cumulative patch for eclipse.platform.swt.binaries (31.61 KB, patch)
2012-06-14 06:20 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Cumulative patch for eclipse.platform.swt (4.81 KB, patch)
2012-06-14 06:20 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Cumulative patch for aggregation repo (does NOT include submodules) (1.45 KB, patch)
2012-06-14 06:24 EDT, Krzysztof Daniel CLA
no flags Details | Diff
Disable xulrunner building (1.93 KB, patch)
2012-06-14 07:08 EDT, Krzysztof Daniel CLA
no flags Details | Diff
macosx xulrunner issue (64.49 KB, application/octet-stream)
2012-07-17 16:44 EDT, Thanh Ha CLA
no flags Details
copyright fix for 4.2.2 and 3.8.2 branches (7.51 KB, patch)
2012-09-14 13:27 EDT, Paul Webster CLA
no flags Details | Diff
copyright fix for master (7.50 KB, patch)
2012-09-14 13:27 EDT, Paul Webster CLA
no flags Details | Diff
swt.binaries prolog and copyright fix for 4.2.2 and 3.8.2 (27.73 KB, patch)
2012-09-14 13:51 EDT, Paul Webster CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Krzysztof Daniel CLA 2012-05-31 09:20:35 EDT
This very cryptic bug title refers to the fact that org.eclipse.swt and related fragments use custom builds and have
    custom=true
in build.properties. 

This is a problem for Tycho because it is not able to determine which files should be included and it generates empty jars by default. What's more, further Tycho versions will fail if they will not find the bin.includes and src.includes.

I think it's time for SWT to move to customBuildCallbacks :-)
Comment 1 Igor Fedorenko CLA 2012-05-31 09:41:09 EDT
Tycho does not support customBuildCallbacks.
Comment 2 Alexander Kurtakov CLA 2012-05-31 09:46:08 EDT
I think that the idea is to start using customBuildCallback for the developers that still depend on pde.build. This will allow having proper bin.includes which can be used by tycho because currently there is no bin.includes and tycho doesn't know what to put in the fragment.
Comment 3 Krzysztof Daniel CLA 2012-05-31 09:50:18 EDT
Alex is right.
custombBuildCallback means a valid build.properties file that could be used by tycho. I can write a pom.xml that will build the fragment, but if build.properties is empty, tycho will fail to generate proper bundle .
Comment 4 Krzysztof Daniel CLA 2012-05-31 10:41:08 EDT
Yet another problem is that without proper build.properties there is no source bundle generated...
Comment 5 Krzysztof Daniel CLA 2012-06-01 08:17:50 EDT
Created attachment 216651 [details]
Mavenizing linux x86_64 fragment.
Comment 6 Krzysztof Daniel CLA 2012-06-01 08:30:26 EDT
Created attachment 216653 [details]
change in org.eclipse.swt build files that enables fragment building
Comment 7 Krzysztof Daniel CLA 2012-06-01 08:55:25 EDT
Explanation of changes:

buildFragment.xml:
have a common place from which all unix-like fragments can invoke a build

build.properties:
tycho does not support custom builds nor customBuildCallback. If we want to have properly packaged plugins, we have to point tycho where it should look for sources and binaries. All those things are prepared by ant script. Similar script will be required for other platforms, or they need to be platform specific. That's future plans.

pom.xml - the one in the swt.binaries repo:
just say that you have such fragment available

fragment pom.xml:
it has a default build which creates a jar that want.
in addition, it has a separate profile that can be enabled from command line, and when it is enabled, it builds all the native libraries.
The pom.xml is simple because it delegates all the work to existing scripts (and therefore all preconditions must be met, f.e. relative position of the fragment and master swt).
Comment 8 Silenio Quarti CLA 2012-06-01 12:35:02 EDT
(In reply to comment #6)
> Created attachment 216653 [details]
> change in org.eclipse.swt build files that enables fragment building

I believe these changes should go into buildSWT.xml. buildSWT.xml already knows the list of files (C source) needed to build the native libraries for a given platform (src_${ws} properties).  If possible we would like to avoid duplicating that list.

Note that the fragment build.xml includes both buildFragment.xml and buildSWT.xml. I do not see any problem moving the tasks from buildFragment.xml to buildSWT.xml.
Comment 9 Silenio Quarti CLA 2012-06-01 12:50:36 EDT
(In reply to comment #5)
> Created attachment 216651 [details]
> Mavenizing linux x86_64 fragment.

If I understand this correctly, the fragment pom.xml calls the current fragment build.xml to compile the code and then goes on with the maven build (which looks in the build.properties file to decide what files are included). It seems fine to me. The only problem I see is that it seems that there are some files missing (fragment.properties,about.html,about_files,META_INF/) which are added to the bundle in "gather.bin.parts".  I might be off here, since I do not understand maven/tycho builds well yet.
Comment 10 Alexander Kurtakov CLA 2012-06-01 12:54:13 EDT
I think tycho determines what to include from bin.includes so probably these entries needs to be added.
Comment 11 Krzysztof Daniel CLA 2012-06-05 08:51:50 EDT
Created attachment 216829 [details]
org.eclipse.swt.gtk.linux.x86_64/build.properties

about/ fragment.properties, META-INf is inside the built plugin.
Comment 12 Krzysztof Daniel CLA 2012-06-11 04:19:37 EDT
Created attachment 217134 [details]
org.eclipse.swt/pom.xml
Comment 13 Krzysztof Daniel CLA 2012-06-11 04:23:03 EDT
Created attachment 217135 [details]
org.eclipse.swt.gtk.linux.x86_64/pom.xml
Comment 14 Krzysztof Daniel CLA 2012-06-11 04:25:19 EDT
Created attachment 217136 [details]
eclipse.platform.swt.binaries/pom.xml
Comment 15 Krzysztof Daniel CLA 2012-06-11 04:31:01 EDT
(In reply to comment #8)
> (In reply to comment #6)
> > Created attachment 216653 [details]
> > change in org.eclipse.swt build files that enables fragment building
> 
> I believe these changes should go into buildSWT.xml. buildSWT.xml already knows
> the list of files (C source) needed to build the native libraries for a given
> platform (src_${ws} properties).  If possible we would like to avoid duplicating
> that list.
Done. It actually looks like I was able to build SWT using only SWT scripts without any additional modifications :-).
Comment 16 Silenio Quarti CLA 2012-06-11 11:43:52 EDT
The changes look good. (In reply to comment #15)
> (In reply to comment #8)
> > (In reply to comment #6)
> > > Created attachment 216653 [details]
> > > change in org.eclipse.swt build files that enables fragment building
> > 
> > I believe these changes should go into buildSWT.xml. buildSWT.xml already knows
> > the list of files (C source) needed to build the native libraries for a given
> > platform (src_${ws} properties).  If possible we would like to avoid duplicating
> > that list.
> Done. It actually looks like I was able to build SWT using only SWT scripts
> without any additional modifications :-).

Great. "build_libraries" is prepared to build on remote machines, but it does the work locally if the $machine var is not specified.

The changes seem good. I just want to confirm that the ".api.description" file is generated in the tycho build. This is is done by the "apitools.generation" task in the p2 build. The file lives at the root of the bundle.

What is the next step? Are you writing the pom.xml files for the other platforms? They should be quite similar. I wonder if there is a way of sharing what is common (same idea of buildFragment.xml). I am not sure this is worth in this case.
Comment 17 Krzysztof Daniel CLA 2012-06-12 08:27:41 EDT
Created attachment 217205 [details]
Patch for the binaries repo

I think that's it :-)
Comment 18 Krzysztof Daniel CLA 2012-06-12 08:52:26 EDT
to activate building of a certain swt  fragment, please specify f.e.
-Dnative=gtk.linux.x86_64
Past way of enabling the native build (-P native-build) is no longer advised.
Comment 19 Krzysztof Daniel CLA 2012-06-13 04:15:35 EDT
Silenio, 
I need your opinion here:

Attempts to build mozilla support 
<property name="targets" value="make_mozilla make_xulrunner make_xpcominit install"/>
instead of just
<property name="targets" value="install"/>
results (in the end) in the following (amongst others) error:
     [exec] xpcom.cpp:85:15: error: ‘nsStaticModuleInfo’ was not declared in this scope
I have found [1] that 
"Notably, the nsStaticModuleInfo struct seems to be long gone and that was an entry point for any previous Xulrunner application.".

Bug 327696 has a lot of information about support for xulrunner - especially that 
"SWT.NONE-style Browsers now default to using WebKit on Linux, so
attempting to support every XULRunner release is no longer necessary, nor is
reasonable"

Given all those informations, can we drop the xulrunner support and use webkit only?

[1] http://www.j5live.com/2007/08/02/webkit-and-xulrunner-mozilla-side-by-side-on-the-xo/
Comment 20 Krzysztof Daniel CLA 2012-06-13 04:20:31 EDT
Created attachment 217255 [details]
Using pkg-config instead of hardcoded paths to build swt on linux
Comment 21 Krzysztof Daniel CLA 2012-06-13 04:37:37 EDT
Yet another comment about xulrunner support:
Eclipse has one year development cycle. Mozilla in that time ships about 10 xulrunner versions. Xulrunner changes api/abi in x.y.z.w+1 release, which means that not only Eclipse M7 working fine may stop working after it is released, but we are doomed in terms of long term support.
Comment 22 Krzysztof Daniel CLA 2012-06-13 10:52:20 EDT
Created attachment 217280 [details]
Building all platforms.
Comment 23 Grant Gayed CLA 2012-06-13 10:53:44 EDT
(In reply to comment #19)
> Given all those informations, can we drop the xulrunner support and use webkit
> only?

You could do this and base Browser functionality would work, assuming that
WebKitGTK >= 1.2.x is found at runtime.  SWT's xulrunner library is needed if a
Browser is created with style SWT.MOZILLA.  Eclipse itself never does this, but
some other apps do.  Such apps must ship a XULRunner binary and point the
Browser at it (they cannot assume that a supported XULRunner will be detected
on the OS).  However if SWT's xulrunner library is not found then of course
this will not work for them.

The XULRunner versions that are supported by each of SWT's releases are at
http://www.eclipse.org/swt/faq.php#browserlinux .  SWT compiles its xulrunner
library against the XULRunner 1.8 SDK, so it's not surprising that compiling it
against a more modern XULRunner release fails.  I think this is the only way to
get this library built with swt's current content; to compile against a newer
XULRunner SDK would require changes in SWT.
Comment 24 Krzysztof Daniel CLA 2012-06-13 14:37:29 EDT
(In reply to comment #23)
> SWT's xulrunner library is needed if a
> Browser is created with style SWT.MOZILLA.  Eclipse itself never does this, but
> some other apps do.  


Thanks for the explanations! Would it be possible to refactor SWT and move the xulrunner support to a separate plugin/fragment?
Comment 25 Thanh Ha CLA 2012-06-13 15:39:20 EDT
(In reply to comment #22)
> Created attachment 217280 [details]
> Building all platforms.

I think there's a patch missing for the following module:

  <module>bundles/org.eclipse.swt.win32.wce_ppc.arm</module>
Comment 26 Grant Gayed CLA 2012-06-13 17:07:34 EDT
(In reply to comment #24)
> Would it be possible to refactor SWT and move the
> xulrunner support to a separate plugin/fragment?

I don't think so, and I'm not sure that this would help.  The dependence on the XULRunner 1.8.0.x SDK would not change, and I assume that there's already a way to not compile SWT's xulrunner library without making a change like this.  (Maybe I'm missing something?)
Comment 27 Krzysztof Daniel CLA 2012-06-14 06:20:26 EDT
Created attachment 217348 [details]
Cumulative patch for eclipse.platform.swt.binaries
Comment 28 Krzysztof Daniel CLA 2012-06-14 06:20:57 EDT
Created attachment 217349 [details]
Cumulative patch for eclipse.platform.swt
Comment 29 Krzysztof Daniel CLA 2012-06-14 06:24:05 EDT
Created attachment 217350 [details]
Cumulative patch for aggregation repo (does NOT include submodules)
Comment 30 Krzysztof Daniel CLA 2012-06-14 06:38:26 EDT
*** Bug 382486 has been marked as a duplicate of this bug. ***
Comment 31 Krzysztof Daniel CLA 2012-06-14 07:08:13 EDT
Created attachment 217352 [details]
Disable xulrunner building

As it was said earlier - proper version of xulrunner is needed to built swt on linux. But it is also possible to apply this patch and forget about xulrunner :-).

Note that native builds will not be triggered until the build is invoked like this:
mvn clean install -Dnative=linux.gtk.x86_64 -Dmaven.repo.local=/var/tmp/lts/R4_localrepo -Dmaven.test.skip=true

where -Dnative= says which fragment should be build.
Comment 32 Thanh Ha CLA 2012-07-17 16:44:57 EDT
Created attachment 218839 [details]
macosx xulrunner issue

I tried building on macosx x86_64 with the -Dnative=cocoa.macosx.x86_64 now that I have a compiler installed on my system and found there is issue with xulrunner. See attached log.

I think a similar workaround as the one done for Linux needs to be implemented to workaround this.
Comment 33 Markus Keller CLA 2012-09-14 11:54:45 EDT
Most of the pom.xml files that have recently been released to the SWT repo generate CHKPII errors, since they're missing the prolog at offset 0 of the file:

<?xml version="1.0" encoding="UTF-8"?>

Please add this in all branches.

Furthermore, the first line of the copyright messages should end with:
[..] and others.
Comment 34 Paul Webster CLA 2012-09-14 13:15:30 EDT
(In reply to comment #33)
> Most of the pom.xml files that have recently been released to the SWT repo
> generate CHKPII errors, since they're missing the prolog at offset 0 of the
> file:
> 
> <?xml version="1.0" encoding="UTF-8"?>

I had a look at the poms in org.eclipse.swt and they all appear to have the prolog ... which ones are missing it?


> 
> Furthermore, the first line of the copyright messages should end with:
> [..] and others.

I can provide a patch to update that

PW
Comment 35 Markus Keller CLA 2012-09-14 13:26:42 EDT
Sorry, I forgot that there are two SWT repos. The problems are in eclipse.platform.swt.binaries, e.g. here:

git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/?id=8ae3a1752943789e19374929e69bf46bc7537537

CHKPII doesn't like the pom.xml files in these directories in R3_8_maintenance:
binaries-parent/
org.eclipse.swt.carbon.macosx/
org.eclipse.swt.cocoa.macosx.x86_64/
org.eclipse.swt.cocoa.macosx/
org.eclipse.swt.gtk.aix.ppc/
org.eclipse.swt.gtk.aix.ppc64/
org.eclipse.swt.gtk.hpux.ia64/
org.eclipse.swt.gtk.hpux.ia64_32/
org.eclipse.swt.gtk.linux.ia64/
org.eclipse.swt.gtk.linux.ppc/
org.eclipse.swt.gtk.linux.ppc64/
org.eclipse.swt.gtk.linux.s390/
org.eclipse.swt.gtk.linux.s390x/
org.eclipse.swt.gtk.linux.x86_64/
org.eclipse.swt.gtk.solaris.sparc/
org.eclipse.swt.gtk.solaris.x86/
org.eclipse.swt.motif.aix.ppc/
org.eclipse.swt.motif.hpux.PA_RISC/
org.eclipse.swt.motif.hpux.ia64_32/
org.eclipse.swt.motif.linux.x86/
org.eclipse.swt.motif.solaris.sparc/
org.eclipse.swt.photon.qnx.x86/
org.eclipse.swt.win32.wce_ppc.arm/
org.eclipse.swt.win32.win32.ia64/
org.eclipse.swt.win32.win32.x86_64/
org.eclipse.swt.wpf.win32.x86/
Comment 36 Paul Webster CLA 2012-09-14 13:27:04 EDT
Created attachment 221102 [details]
copyright fix for 4.2.2 and 3.8.2 branches
Comment 37 Paul Webster CLA 2012-09-14 13:27:24 EDT
Created attachment 221103 [details]
copyright fix for master
Comment 38 Paul Webster CLA 2012-09-14 13:51:15 EDT
Created attachment 221104 [details]
swt.binaries prolog and copyright fix for 4.2.2 and 3.8.2
Comment 39 Paul Webster CLA 2012-09-14 13:52:12 EDT
Silenio, can I get you to review and commit the last 3 patches?

PW
Comment 43 Alexander Kurtakov CLA 2013-12-10 02:53:12 EST
Can we close this one now? Not perfect now but separate bug reports for specific issues would be better if one wants to improve it.
Comment 44 Alexander Kurtakov CLA 2014-01-16 06:23:59 EST
Noone interested so I'm closing it as there are too many things discussed here. Separate bug reports per issue would be better if there is still smth left.