Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 400956

Summary: Generated source plugins always use "." as source root inside source jar
Product: z_Archived Reporter: Paul Webster <pwebster>
Component: TychoAssignee: Jan Sievers <jan.sievers>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: david_williams, igor, jan.sievers, t-oberlies, thanh.ha
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on:    
Bug Blocks: 400740, 403256    

Description Paul Webster CLA 2013-02-15 14:06:51 EST
When PDE generates org.eclipse.jdt.debug.source it uses build.properties and keeps the source in subdirectories:

source.jdi.jar = jdi interfaces/
output.jdi.jar = jdi-bin/
source.jdimodel.jar = model/,\
                      jdi/,\
                      eval/
output.jdimodel.jar = bin/                   

generates:
about.html  jdimodelsrc/  jdisrc/  META-INF/  plugin.properties  schema/

When tycho generates the source plugin, it bundles all source at the root:

about.html  com/  META-INF/  org/  OSGI-INF/  schema/

This breaks PDE import of source bundles (for all bundles with inner jars).

PW
Comment 1 Jan Sievers CLA 2013-02-18 03:42:59 EST
is this import wizard behaviour documented anywhere?

can you provide a sample and steps to reproduce?
Comment 2 Jan Sievers CLA 2013-02-18 03:50:41 EST
sources have the package root in the zip root.

while the proposed change may make PDE UI happy, it will break all other tools that simply attach source zips to the corresponding jars.
Comment 3 Jan Sievers CLA 2013-02-18 03:51:24 EST
(In reply to comment #2)
> sources have the package root in the zip root.

s/sources/sources zips/g
Comment 4 Jan Sievers CLA 2013-02-18 05:02:04 EST
I could imagine a "distinctSourceFolders" configuration switch like

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-source-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <distinctSourceFolders>true</distinctSourceFolders>
        </configuration>
      </plugin>

(defaults to false)
Comment 5 Igor Fedorenko CLA 2013-02-18 08:09:05 EST
Why not change PDE to be able import of source bundles produced by Tycho?
Comment 6 Paul Webster CLA 2013-02-18 09:15:06 EST
(In reply to comment #2)
> while the proposed change may make PDE UI happy, it will break all other
> tools that simply attach source zips to the corresponding jars.

These are PDE source bundles, though, and have been this way forever.  It's not likely that there are tools that are failing to read them, as they've always been this way.

PW
Comment 7 Paul Webster CLA 2013-02-18 09:16:06 EST
(In reply to comment #5)
> Why not change PDE to be able import of source bundles produced by Tycho?

The tycho source bundles have lost information (which source belongs to which jar library), so PDE can't match them up any more.

PW
Comment 8 Jan Sievers CLA 2013-02-18 10:32:38 EST
(In reply to comment #7)
> (In reply to comment #5)
> > Why not change PDE to be able import of source bundles produced by Tycho?
> 
> The tycho source bundles have lost information (which source belongs to
> which jar library), so PDE can't match them up any more.

From the example

http://git.eclipse.org/c/jdt/eclipse.jdt.debug.git/tree/org.eclipse.jdt.debug

I can reverse-engineer that in case of several source roots per nested jar, they are collapsed into one source root inside the source jar.

since it can't be the original source folder name, what's the algorithm for the name of the source root directory inside the source jar (jar name minus ".jar") ?
what happens in case of nested dirs (like WEB-INF/classes/) ?

pointers to docs or code on how this is supposed to work in PDE appreciated.
Comment 9 Jan Sievers CLA 2013-02-18 10:42:29 EST
(In reply to comment #6)
> These are PDE source bundles, though, and have been this way forever.  It's
> not likely that there are tools that are failing to read them, as they've
> always been this way.

For those bundles that want to be interoperable with plain maven usecases (e.g. jetty or similar), it's a good idea to provide source jars in the same way as any other maven artifact provides source jars (so that e.g. m2eclipse can attach them automatically in its classpath container).
That said, these kind of bundles would not use nested jars to start with.
Comment 10 Jan Sievers CLA 2013-02-18 16:15:00 EST
https://git.eclipse.org/r/#/c/10458/
Comment 11 Jan Sievers CLA 2013-02-19 04:33:29 EST
http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=e0a74423888b26d8ae968c5757c3a46874c5ac89

you will have to configure distinctSourceFolders=true
Comment 12 Jan Sievers CLA 2013-02-19 04:35:20 EST
(In reply to comment #11)
> you will have to configure distinctSourceFolders=true
meant to say distinctSourceRoots
Comment 13 Tobias Oberlies CLA 2013-02-19 09:31:12 EST
(In reply to comment #9)
> For those bundles that want to be interoperable with plain maven usecases (e.g.
> jetty or similar), it's a good idea to provide source jars in the same way as
> any other maven artifact provides source jars (so that e.g. m2eclipse can attach
> them automatically in its classpath container).
> That said, these kind of bundles would not use nested jars to start with.

So, doesn't this mean that everyone would be happy with a distinctSourceRoots=true default?
Comment 14 Jan Sievers CLA 2013-02-19 10:11:50 EST
(In reply to comment #13)
> (In reply to comment #9)
> > For those bundles that want to be interoperable with plain maven usecases (e.g.
> > jetty or similar), it's a good idea to provide source jars in the same way as
> > any other maven artifact provides source jars (so that e.g. m2eclipse can attach
> > them automatically in its classpath container).
> > That said, these kind of bundles would not use nested jars to start with.
> 
> So, doesn't this mean that everyone would be happy with a
> distinctSourceRoots=true default?

not so sure.

1. it would be an incompatible change in tycho
2. it's based on (as far as I got it from Paul and Curtis) undocumented behaviour of PDE UI where the jar name is magically mapped to the corresponding source root

One could question that PDE import wizard tries to re-create the original project from the source jar because information is missing anyway:

1. multiple source roots per jar are mappped to one source root
2. jars which are not on Bundle-ClassPath fail to be reconstructed by the import wizard today (example: org.eclipse.core.ant)

Once you start working on a project, just sync the sources (which should be easy now with Eclipse-SourceReferences header and EGit support for it in place)
the other usecase I see is binary projects with atached source jars. this should work just fine with all sources in the source jar root.

overall I still find it more intuitive and simple that source jars contain all sources at the root of the jar by default.
Comment 15 Tobias Oberlies CLA 2013-02-19 11:11:53 EST
So if distinctSourceRoots=true, you'd get the sources in the source zip under src/ for normal JARs? This would in fact be strange and enough reason for keeping the default to false.
Comment 16 Jan Sievers CLA 2013-02-20 03:26:48 EST
(In reply to comment #15)
> So if distinctSourceRoots=true, you'd get the sources in the source zip
> under src/ for normal JARs? 
http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-its/src/test/java/org/eclipse/tycho/test/sourcebundles/SourceBundlesNestedJarsTest.java#n44