Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 397852

Summary: Be able to specify the jdt.core used for compiling - from non "public" repository
Product: [Eclipse Project] Platform Reporter: David Williams <david_williams>
Component: RelengAssignee: David Williams <david_williams>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: daniel_megert, david_williams, igor, jan.sievers, pwebster, thanh.ha
Version: 3.8   
Target Milestone: 4.3 M5   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on: 375554, 397850    
Bug Blocks: 372792    

Description David Williams CLA 2013-01-10 07:54:49 EST
+++ This bug was initially created as a clone of Bug #375554 +++

As mentioned in original bug (as some of the final comments) and on cbi-dev list, we in the platform need the ability to specify a relatively arbitrary version of jdt.core to use for compiling. 

AND, we often specifically do not want these in a "public" repository, since they have not been released yet. That is, we do not want to give the impression they are "released" in the Eclipse Foundation sense of the word (though, of course, they are not exactly private, since they are in our I or S builds). 

This is required for our Platform/Sim. Release process to give adequate testing to the next/future released version of the compiler. Put another way, we wouldn't necessarily say EVERYONE in Sim. Release should use these non-released compilers, but the Platform definitely needs to, and often others with "big compiling" needs do also, such as WTP. But, we need the ability to revert if we find a bad one, or move up to an an arbitrarily more recent one, if some bad bug has been fixed.
Comment 1 Paul Webster CLA 2013-01-10 08:06:27 EST
*** Bug 397854 has been marked as a duplicate of this bug. ***
Comment 2 Paul Webster CLA 2013-01-10 08:07:18 EST
tycho allows pluging the compiler in, but it must exist at visible maven co-ordinates somewhere.

We need to place the .jar and some .pom in a location where it can be consumed.

1) in a location on the file system, a semi-permanent build location so we can use file://

2) in a semi-permanent location in our /shared/eclipse so we can use http:// ... then anybody can use maven to build our intermediate.

3) somehow put it in maven.eclipse.org so that anybody can use mavne to build our intermediate builds.
Comment 3 Jan Sievers CLA 2013-01-10 08:33:11 EST
(In reply to comment #2)
> tycho allows pluging the compiler in, but it must exist at visible maven
> co-ordinates somewhere.
> 
> We need to place the .jar and some .pom in a location where it can be
> consumed.
> 
> 1) in a location on the file system, a semi-permanent build location so we
> can use file://
> 
> 2) in a semi-permanent location in our /shared/eclipse so we can use http://
> ... then anybody can use maven to build our intermediate.
> 
> 3) somehow put it in maven.eclipse.org so that anybody can use mavne to
> build our intermediate builds.

please avoid any local file:// /not publicly visible URLs which make CBI build non-portable. The benefit of anyone being able to build eclipse anywhere would be lost.
another option would be to mvn deploy to a file:// URL and copy the resulting maven repo and serve it from a publicly visible but not "officially released/advertised" place on download.eclipse.org. Ideally this should just be on maven.eclipse.org though.
Comment 4 Paul Webster CLA 2013-01-10 09:50:55 EST
Igor had some specific steps in bug 375554 comment #11 :

Here is what I recommend you do.

* Setup a hosted maven repository. This does not have to be anything fancy, a stable location somewhere on download.eclipse.org is sufficient. For example, I use technology/m2e/maven to host CBI signing plugin.

* Deploy jdt.core version that you need to use to the hosted repository you setup using file:// protocol. See exact command below.

* Add new <pluginRepository> element to CBI parent pom.xml with http:// URL of the hosted repository

* Add new <pluginManagement> plugins/plugin element with configuration similar to comment #1 to CBI parent pom.xml file.



JDTVERSION=3.9.0.v20121208-144805
mvn deploy:deploy-file -Durl=file:///tmp/cbi-jdt \
  -DgroupId=org.eclipse.cbi.tycho \
  -DartifactId=org.eclipse.jdt.core \
  -Dversion=${JDTVERSION} \
  -Dfile=org.eclipse.jdt.core_${JDTVERSION}.jar \
  -Dsources=org.eclipse.jdt.core.source_${JDTVERSION}.jar
Comment 5 Igor Fedorenko CLA 2013-01-10 09:55:59 EST
I completely agree with comment #3 -- do not use file:// URLs in pom.xml files, use publicly accessible http:// or https:// URLs.
Comment 6 David Williams CLA 2013-01-10 15:32:27 EST
Thanks all for such constructive tips and reminders. 

I plan to make this change (at least in master, to start, to try it out) and thought I'd comment here that I'd like the URL to point to "build machine", it'd be something like 

http://build.eclipse.org/eclipse/cbi-jdt

so that way it is more obvious it is not a "released" version ... could/would eventually perhaps disappear. I know that's not "normal" for the maven culture but hope its acceptable for this kind of case. 

Also, rather than a groupId=org.eclipse.cbi.tycho, I assume groupId=org.eclipse.jdt would be slightly more correct? 

Thanks again,
Comment 7 David Williams CLA 2013-01-10 17:41:33 EST
I've made a guess ... following the existing examples ... of how the parent pom should be changed to get this "build.eclipse.org" repo: 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=dca74e84a1431e0b74eabf1c33873262f47dd1f1
Comment 8 Paul Webster CLA 2013-01-10 19:03:38 EST
The only thing I think is that you have to use http: instead of https: ... build.eclipse.org won't serve https up.

PW
Comment 9 David Williams CLA 2013-01-10 19:59:15 EST
I mistakenly put "https" in the first version, should have been "http". 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=bc2d317b3b8299ec0d64f3a63751a5a94275f3c8

jinks :)
Comment 10 David Williams CLA 2013-01-11 01:34:51 EST
I've put this version of the compiler in all three streams, since that's what we are doing in our current PDE based builds. 

G o o d  l u c k ! :)
Comment 11 Thanh Ha CLA 2013-01-11 10:12:07 EST
(In reply to comment #7)
> I've made a guess ... following the existing examples ... of how the parent
> pom should be changed to get this "build.eclipse.org" repo: 
> 
> http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/
> commit/?id=dca74e84a1431e0b74eabf1c33873262f47dd1f1

This change breaks the build for me.

I don't think putting the repository in the <pluginRepositories> section is correct. If I'm not wrong I think maven calculates the dependencies from the <repositories> section. Unfortunately I haven't been able to get this to work either.

I moved the section to the <repositories> section as follows:

    <repository>
      <id>cbi-jdt</id>
      <url>${cbi-jdt-repo.url}</url>
      <releases>
        <enabled>true</enabled>
      </releases>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>

http://build.eclipse.org/eclipse/cbi-jdt/ looks like a maven repo to me. However it doesn't look like maven's using this additional repository to resolve dependencies:


[INFO] Adding repository http://download.eclipse.org/tools/orbit/downloads/drops/S20121207152029/repository
[INFO] Adding repository http://download.eclipse.org/tools/orbit/downloads/drops/S20121207152029/repository
[INFO] Adding repository http://download.eclipse.org/rt/ecf/3.5.6/site.p2
[INFO] Adding repository http://download.eclipse.org/rt/ecf/3.5.6/site.p2
[INFO] Adding repository http://www.eclipse.org/external/jetty/updates/jetty-bundles-8.x/8.1.3.v20120522
[INFO] Adding repository http://www.eclipse.org/external/jetty/updates/jetty-bundles-8.x/8.1.3.v20120522
[INFO] Fetching p2.index from http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones/ (0B of 99B at 0B/s)
[INFO] Fetching p2.index from http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones/ (134B of 99B at 0B/s)
[INFO] Adding repository http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones
[INFO] Fetching p2.index from http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones/ (0B of 99B at 0B/s)
[INFO] Fetching p2.index from http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones/ (134B of 99B at 0B/s)
[INFO] 1 operation remaining.
[INFO] Adding repository http://download.eclipse.org/modeling/emf/emf/updates/2.9milestones
[INFO] Adding repository http://download.eclipse.org/egit/updates
[INFO] Adding repository http://download.eclipse.org/egit/updates
[INFO] Adding repository http://download.eclipse.org/tools/ajdt/aspectj/update
[INFO] Adding repository http://download.eclipse.org/tools/ajdt/aspectj/update
[INFO] Resolving dependencies of MavenProject: eclipse.platform.ua:org.eclipse.help.webapp:3.6.200-SNAPSHOT @ /home/user/Downloads/eclipse2/eclipse.platform.releng.aggregator/eclipse.platform.ua/org.eclipse.help.webapp/pom.xml



Jan, is there any additional settings we need to add to make maven use this repo?
Comment 12 Paul Webster CLA 2013-01-11 10:14:08 EST
I thought it had to be in the pluginRepositories section in order for maven to pick it up for use by tycho?

PW
Comment 13 Paul Webster CLA 2013-01-11 10:17:16 EST
(In reply to comment #12)
> I thought it had to be in the pluginRepositories section in order for maven
> to pick it up for use by tycho?

i.e. the tycho repo is in pluginRepositories section, so it makes sense this would as well.  What if you leave it there, but mark it as releases/enable = true

PW
Comment 14 Thanh Ha CLA 2013-01-11 10:28:33 EST
(In reply to comment #13)
> (In reply to comment #12)
> > I thought it had to be in the pluginRepositories section in order for maven
> > to pick it up for use by tycho?
> 
> i.e. the tycho repo is in pluginRepositories section, so it makes sense this
> would as well.  What if you leave it there, but mark it as releases/enable =
> true
> 
> PW

That was it! I pushed the change into R3_8_maintenance, R4_2_maintenance, and master.

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=f4055fecee1bfb16902d3a6cf6af291f673001c3
Comment 15 Paul Webster CLA 2013-01-11 10:39:54 EST
I think it's just the convention: where anything without SNAPSHOT counts as a release.  In the tycho repo above our jdt repo, we only want to consider SNAPSHOTs from it as any released versions of tycho would come from maven central.

PW
Comment 16 David Williams CLA 2013-05-30 16:45:13 EDT
mass change to 'verified', as these bugs are either routine or obviously fixed build breaks.