Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 365594

Summary: add code recommenders to java package
Product: [Technology] EPP Reporter: Marcel Bruch <marcel.bruch>
Component: java-packageAssignee: Steffen Pingel <steffen.pingel>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: eclipse-bugs, mknauer, steffen.pingel, wayne.beaton
Version: unspecified   
Target Milestone: 1.4.0   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:

Description Marcel Bruch CLA 2011-12-05 09:14:30 EST
Code Recommenders recently decided to join the annual release train for Juno. We are now evaluating whether there are packages which may include Code Recommenders.


What does Code Recommenders offer for the annual release? 
We have several components that can be packaged separately:

* Intelligent Method Call Completion & Method Overrides Completion
* Call Chain Completion
* Snipmatch 
* Extdoc Platform 
* Subwords Completion
* Usage Sharing Client to build continuously growing knowledge bases for intelligent completion + extdoc
* (maybe) new code search client + engine for Eclipse

A (not self-learning) version of Call Chain and Subwords Completion may already be handed over to JDT 3.8 until Juno. All tools work for Java only.


A video showing the above tool suite in action is available here:
http://code-recommenders.blogspot.com/2011/11/do-you-still-read-source-code-to-learn.html

PMC conversation about joining the release train:
http://dev.eclipse.org/mhonarc/lists/technology-pmc/msg03741.html

latest branch of the Code Recommenders source code repository:
http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/?h=v0.5-extdoc


I answered the questions listed on
http://wiki.eclipse.org/EPP/How_to_create_a_package#Adding_components_to_a_package below.


> What is the impact on the download size of the Java package if the new component is included?
The whole Code Recommenders plugin set will be below 1 MB. Models are loaded on-the-fly from a remove server.


> Does the new component depend on components (e.g. PDE) that are not already part of the Java
> package?
It depends on JDT and Platform only.


> Does the new component or any of its dependencies have incubation status?
Code Recommenders itself has currently incubation status. There are no other components it relies on that have incubator status.

> Does the new component require additional components that would not be shipped as part
> of the Java package (e.g. an external runtime)?
There are several third party dependencies to Google Guice, Google Guava, several Apache Commons libs. As the only extraordinary dependency we use the IBM Wala Bytecode Toolkit (IP checked already). But we are currently discussing to deliver Code Recommenders in a JDT-Only version w/o dependencies to any bytecode toolkit. 


> Does the new component log any output to the console?
At least it should not. We use SLF4J.


> Does the new component contribute a startup extension?
Yes. For model loading and index building, we run an initialization job triggered by an EarlyStartup hook.


> Does the new component contribute any (modal) dialogs that are visible on startup?
No.


> What is the runtime impact if the new component is not used?
Low. Components will integrate with code completion and deliver views. If no view is opened and completion engines are deactivated only the minimal model indexes will be loaded and the DI modules will exist. There is no need to load big amounts of data into memory (w/ JDT-only version).


> Which UI contributions if any does the new component add to the Java perspective?
Optionally: an "Extended Javadoc" View.
If the code search client would be ready until release, there will be a few more screens on that. But they don't have to be part of the (predefined) Java perspective.


Let me know what else is of interest for you.

Thanks,
Marcel
Comment 1 Steffen Pingel CLA 2011-12-08 05:06:07 EST
Thanks for the detailed information!

(In reply to comment #0)
> > What is the impact on the download size of the Java package if the new
> component is included?
> The whole Code Recommenders plugin set will be below 1 MB. Models are loaded
> on-the-fly from a remove server.

That sounds reasonable.

> > Does the new component or any of its dependencies have incubation status?
> Code Recommenders itself has currently incubation status. There are no other
> components it relies on that have incubator status.

That might be a show stopper for the Java package. So far we have only included matured projects to avoid labeling the package as Incubation. Do you have any plans to graduate Code Recommenders by Juno?

> > Does the new component require additional components that would not be shipped
> as part
> > of the Java package (e.g. an external runtime)?
> There are several third party dependencies to Google Guice, Google Guava,
> several Apache Commons libs. As the only extraordinary dependency we use the IBM
> Wala Bytecode Toolkit (IP checked already). But we are currently discussing to
> deliver Code Recommenders in a JDT-Only version w/o dependencies to any bytecode
> toolkit.

Is any of this still under parallel IP?

> > Does the new component contribute a startup extension?
> Yes. For model loading and index building, we run an initialization job
> triggered by an EarlyStartup hook.

How noticeable is that startup job? Is there any way to postpone it until Code Recommenders is actually used.
 
> > What is the runtime impact if the new component is not used?
> Low. Components will integrate with code completion and deliver views. If no
> view is opened and completion engines are deactivated only the minimal model
> indexes will be loaded and the DI modules will exist. There is no need to load
> big amounts of data into memory (w/ JDT-only version).

Do you have a sense how much heap is consumed by these models if Code Recommenders is not used?

> > Which UI contributions if any does the new component add to the Java
> perspective?
> Optionally: an "Extended Javadoc" View.
> If the code search client would be ready until release, there will be a few more
> screens on that. But they don't have to be part of the (predefined) Java
> perspective.

As long as we don't end up with two Javadoc views visible by default that sounds reasonable.
Comment 2 Marcel Bruch CLA 2011-12-08 06:01:42 EST
(In reply to comment #1)
> > > Does the new component or any of its dependencies have incubation status?
> > Code Recommenders itself has currently incubation status. There are no other
> > components it relies on that have incubator status.
> 
> That might be a show stopper for the Java package. So far we have only included
> matured projects to avoid labeling the package as Incubation. Do you have any
> plans to graduate Code Recommenders by Juno?
I've to check that with Technology PMC. I did not ask how/when to graduate since there was no need. 
Spontaneously, I could imagine to request the graduate review in ~February. 


> > > Does the new component require additional components that would not be shipped
> > as part of the Java package (e.g. an external runtime)?
> > There are several third party dependencies to Google Guice, Google Guava,
> > several Apache Commons libs. As the only extraordinary dependency we use the IBM
> > Wala Bytecode Toolkit (IP checked already). But we are currently discussing to
> > deliver Code Recommenders in a JDT-Only version w/o dependencies to any bytecode
> > toolkit.
> 
> Is any of this still under parallel IP?
No.


> > > Does the new component contribute a startup extension?
> > Yes. For model loading and index building, we run an initialization job
> > triggered by an EarlyStartup hook.
> 
> How noticeable is that startup job? Is there any way to postpone it until Code
> Recommenders is actually used.
We can work on such a solution if required.


> > > What is the runtime impact if the new component is not used?
> > Low. Components will integrate with code completion and deliver views. If no
> > view is opened and completion engines are deactivated only the minimal model
> > indexes will be loaded and the DI modules will exist. There is no need to load
> > big amounts of data into memory (w/ JDT-only version).
> 
> Do you have a sense how much heap is consumed by these models if Code
> Recommenders is not used?
No. Which number would be critical? To give a reliable answer I have to do some memory analysis. If desired, I could add this to the todo list for 0.6 or 0.7.


My todo list: I'll
* check with Technology PMC what is needed to graduate and (if allowed to) and come up with a roadmap.
* request to schedule a release review for 0.6 (early 2012).
* check whether we need the startup hook.
* try to minimize memory consumptions as much as possible
* try to provide some memory usage statistics after all modules are started and indexes are populated.
Comment 3 Steffen Pingel CLA 2011-12-08 06:57:24 EST
That sounds great!

(In reply to comment #2)
> My todo list: I'll
> * try to minimize memory consumptions as much as possible
> * try to provide some memory usage statistics after all modules are started and
> indexes are populated.

I am just looking for a rough ballpark number to avoid surprised. It should be sufficient to start the Java package with and without Code Recommenders installed and compare the heap usage (after garbage collection).
Comment 4 Wayne Beaton CLA 2011-12-08 09:44:56 EST
(In reply to comment #2)
> I've to check that with Technology PMC. I did not ask how/when to graduate
> since there was no need. 
> Spontaneously, I could imagine to request the graduate review in ~February. 

The PMC is watching ;-)

You can work toward graduation in February, if desired.

You can also just graduate with the Juno review (we can combine the project's Juno release review with a graduation review).

Make sure that you understand what graduation means. There are some maturity requirements in terms of APIs, code quality, and community development, for example.
Comment 5 Marcel Bruch CLA 2012-04-26 05:04:08 EDT
(In reply to comment #2)
> My todo list: I'll
> * check with Technology PMC what is needed to graduate and (if allowed to) and
> come up with a roadmap.
since technology PMC is listening...

> * request to schedule a release review for 0.6 (early 2012).
0.5 went through a review. We had several bug fix releases since then and a major rework on the model delivery component that fixes the issues that arose with 0.5.

> * check whether we need the startup hook.
Code Recommenders now runs without startup hook now. I manually verified that no class is loaded by analyzing the heap dump after startup. However, all plugins are started as soon as any completion engine or view gets activated.

> * try to minimize memory consumptions as much as possible
> * try to provide some memory usage statistics after all modules are started and
> indexes are populated.
You asked for a rough ballpark number. I can't yet give a number of how much memory is retained by recommenders' objects but the size of all objects in org.eclipse.recommenders namespace (running Recommenders in an Eclipse workspace with 40 projects) is lower than 1 MB. I assume that the overall memory consumption is lower than 5MB. If you need a more detailed listing I've to dig deeper into MAT to find these numbers.

Side note to memory consumption:
For many data structures that potentially grow large but can be recomputed on demand, we use weak HashMaps and limited object pools to allow the GC to free resources when needed.


Other notes:
There is one open bug that indicates a potential problem with M2E: Bug 375743 
I'm carefully tracking this but wasn't able to reproduce yet. Thus, I can't say whether it still exists. Tests on a fairly large M2E workspace should be done.


Is there anything else I should provide?
Comment 6 Steffen Pingel CLA 2012-04-26 05:23:26 EDT
Thanks for the detailed information. That all sounds great and we should give this a go for M7. Which features should I include in the Java package?
Comment 7 Marcel Bruch CLA 2012-04-26 07:55:32 EDT
Feature id is 'org.eclipse.recommenders.feature.rcp' - it's the same as in the RAP/RCP package.

Should I disable Recommenders' completion engines by default?
Call-, Overrides-, and Subwords completion are activated by default. Templates- and Chain completion are not activated by default due to some performance issues and minor bugs we are currently addressing.
Comment 8 Marcel Bruch CLA 2012-04-26 07:57:42 EDT
(In reply to comment #7)
> 
> Should I disable Recommenders' completion engines by default?

Also: The models are yet loaded from a university server, i.e., an external non-eclipse resource.
Comment 9 Steffen Pingel CLA 2012-05-07 08:23:48 EDT
I have added the feature for M7.

(In reply to comment #7)
> Should I disable Recommenders' completion engines by default?

I'll leave that up to you. My sense is that if features are disabled by default users won't get any benefit as the vast majority of users will use defaults.

(In reply to comment #8)
> Also: The models are yet loaded from a university server, i.e., an external
> non-eclipse resource.

It sounds like that could be an issue. Is the server capable of handling requests for all users downloading the Java package (potentially 100.000s once released)? I am also wondering if that is acceptable for EPP packages or did the PMC specifically approve that?
Comment 10 Markus Knauer CLA 2012-05-07 12:50:25 EDT
(In reply to comment #9)
> (In reply to comment #8)
> > Also: The models are yet loaded from a university server, i.e., an external
> > non-eclipse resource.
> 
> It sounds like that could be an issue. Is the server capable of handling
> requests for all users downloading the Java package (potentially 100.000s once
> released)? I am also wondering if that is acceptable for EPP packages or did
> the PMC specifically approve that?

I see this as an issue, too. From my perspective it is something that must be clarified by the Code Recommenders project with the PMC and Eclipse Legal whether this is acceptable or not - my guess based on similar discussions is that it is not acceptable. I remember that it has been brought to the Technology PMC but I've never seen a final decision.

As a user (i.e. nobody from EPP or eclipse.org) I wouldn't like the idea of installing a bit of software that connects to remote servers at all, and in that case it is even more complicated because it is a server not located at the Eclipse Foundation. To me, the minimum requirement would be an approval from Eclipse Legal + Technology PMC and a move of the server backend to eclipse.org. Otherwise I would strongly recommend to remove this feature.
Comment 11 Steffen Pingel CLA 2012-05-07 13:03:29 EDT
(In reply to comment #10)
> As a user (i.e. nobody from EPP or eclipse.org) I wouldn't like the idea of
> installing a bit of software that connects to remote servers at all, and in that
> case it is even more complicated because it is a server not located at the
> Eclipse Foundation. To me, the minimum requirement would be an approval from
> Eclipse Legal + Technology PMC and a move of the server backend to eclipse.org.
> Otherwise I would strongly recommend to remove this feature.

I agree. A similar discussion may have happened around the m2e project already which retrieves index files and artifacts from the Maven central which is also not Eclipse.org infrastructure.
Comment 12 Wayne Beaton CLA 2012-05-07 13:08:56 EDT
(In reply to comment #10)
> I see this as an issue, too. From my perspective it is something that must be
> clarified by the Code Recommenders project with the PMC and Eclipse Legal
> whether this is acceptable or not - my guess based on similar discussions is
> that it is not acceptable. I remember that it has been brought to the
> Technology PMC but I've never seen a final decision.

We (the Technology PMC) asked the project to make their case. We need to know where they want to host the server, impact on bandwidth, who will be responsible for maintaining the server, what processes are in place to ensure that private data is not compromised, etc. 

AFAIK, we have not yet received any proposal.
Comment 13 Steffen Pingel CLA 2012-05-09 07:06:08 EDT
Marcel, can you help me understand what the desired settings for the content assist are? I read through bug 378918 but I am not quite sure what the expected behavior is. You also need to be aware that Mylyn disables the default JDT (Java Proposals) if you activate a task and use content assist for the first time. We need to make sure that this doesn't interfere with the code recommenders content assist. In Mylyn we also do a check if the default proposal computer is active and don't return any proposals in that case to avoid duplicates.
Comment 14 Steffen Pingel CLA 2012-05-09 07:13:28 EDT
Would it be possible to modify the Subwords proposal computer to only return results that are not also returned by the default proposal computer to avoid duplicates?
Comment 15 Marcel Bruch CLA 2012-05-09 07:56:05 EDT
as jdt does not allow to filter, highlight or manipulate any existing proposal, the only reasonable action is to not-enable code recommenders' subwords completion per default. I'm working on this tonight.
Comment 16 Marcel Bruch CLA 2012-05-09 07:58:07 EDT
(In reply to comment #13)
> In Mylyn we also do a check if the default
> proposal computer is active and don't return any proposals in that case to
> avoid duplicates.

I think, this is the way to go. Can you point me to the class where you do this check?
Comment 17 Steffen Pingel CLA 2012-05-09 08:24:56 EDT
Would it be possible to modify the Subwords proposal computer to only return results that are not also returned by the default proposal computer to avoid duplicates?

Mylyn's implementation is in shouldReturnResults() in https://git.eclipse.org/c/mylyn/org.eclipse.mylyn.context.git/tree/org.eclipse.mylyn.java.ui/src/org/eclipse/mylyn/internal/java/ui/editor/FocusedJavaAllProposalComputer.java .
Comment 18 Steffen Pingel CLA 2012-05-09 15:41:43 EDT
Did any fixes make it into M7? If not we may need to exclude Code Recommenders for M7 and re-try for RC1.
Comment 19 Marcel Bruch CLA 2012-05-09 15:47:14 EDT
I'm currently testing the latest build. In it's current implementation subwords does not make any completions if JDT_ALL is activated. Are there still a few minutes left?
Comment 20 Steffen Pingel CLA 2012-05-09 15:57:50 EDT
Looks like you should have another hour but i would inform cross-projects to make sure that another aggregation build is run.
Comment 21 Marcel Bruch CLA 2012-05-09 16:11:50 EDT
All pieces are in place in time.
Comment 22 Steffen Pingel CLA 2012-05-10 19:32:54 EDT
Looking at M7 I see few major problems in the Java package:

* Content assist entries are duplicated: Both Java Subwords Completions and Java Proposals (Task-Focused) contribute default Java completions which leads to duplicate entries. Marcel, unless Subwords Completions returns a disjoint set of completions we can't have both computers active at the same time.
* Code Recommenders bundles are not signed.
* Some bundles contained in the non-orbit dependencies libraries feature don't list Eclipse as the provider. Should they?
* The 3rd party non-orbit dependencies feature includes com.google.gson. Isn't that in Orbit now?
* Code Recommenders doesn't contribute a branding icon to the About dialog. That would be nice to have.

What are the plans in terms of graduating code recommenders?

I also noticed that the Mylyn Gerrit connector doesn't work in the Java package. It fails with this error:

java.lang.LinkageError: loader constraint violation: loader (instance of org/eclipse/osgi/internal/baseadaptor/DefaultClassLoader) previously initiated loading for a different type with name "com/google/gson/GsonBuilder"
	at org.eclipse.mylyn.internal.gerrit.core.client.JSonSupport.<init>(JSonSupport.java:122)
	
It likely related to having multiple versions of com.google.gson in the runtime. I'll investigate whether this needs to be addressed in the Gerrit connector.
Comment 23 Steffen Pingel CLA 2012-05-10 19:39:25 EDT
It indeed looks like the Gerrit connector either needs a uses constraint to bind the com.google.gwtjsonrpc library to com.google.gson 1.6 or move up to com.google.gson 2.1.

osgi> p com.google.gson
com.google.gson; version="1.6.0"<com.google.gson_1.6.0.v201101131530 [5]>
  org.eclipse.mylyn.hudson.core_1.0.0.I20120509-1226 [269] imports
  org.eclipse.mylyn.gerrit.core_0.9.1.v20120425-0100 [413] imports
  org.eclipse.mylyn.gerrit.ui_0.9.1.v20120425-0100 [414] imports
com.google.gson; version="2.1.0"<com.google.gson_2.1.0.201204241625 [6]>
  org.eclipse.recommenders.completion.rcp_0.5.2.v20120509-1951-b101 [308] imports
  org.eclipse.recommenders.completion.rcp.calls_0.5.2.v20120509-1951-b101 [309] imports
  org.eclipse.recommenders.completion.rcp.chain_0.5.2.v20120509-1951-b101 [310] imports
  org.eclipse.recommenders.completion.rcp.overrides_0.5.2.v20120509-1951-b101 [312] imports
  org.eclipse.recommenders.completion.rcp.subwords_0.5.2.v20120509-1951-b101 [313] imports
  org.eclipse.recommenders.completion.rcp.templates_0.5.2.v20120509-1951-b101 [314] imports
  org.eclipse.recommenders.extdoc_0.5.2.v20120509-1951-b101 [316] imports
  org.eclipse.recommenders.extdoc.rcp_0.5.2.v20120509-1951-b101 [317] imports
  org.eclipse.recommenders.jayes.serialization_0.5.2.v20120509-1951-b101 [320] imports
  org.eclipse.recommenders.rcp_0.5.2.v20120509-1951-b101 [321] imports
  org.eclipse.recommenders.utils_0.5.2.v20120509-1951-b101 [322] imports
  org.eclipse.recommenders.utils.rcp_0.5.2.v20120509-1951-b101 [323] imports
  org.eclipse.recommenders.webclient_0.5.2.v20120509-1951-b101 [324] imports
  com.google.gwtjsonrpc_1.2.5.v201112241444 [409] imports
Comment 24 Steffen Pingel CLA 2012-05-10 20:07:32 EDT
Regarding content assist my sense is that the duplicates are very annoying but the bug is not a show stopper since there is a simple work-around: Either disable Subwords or Task-Focused completion or enable Java completions (i.e. restore defaults).

We should leave Code Recommenders in the package for M7 to give the community an opportunity to detect further problems. We can still make a call for RC1 or RC2 to revert if we can't fix the problems.
Comment 25 Eric Rizzo CLA 2012-05-10 22:51:19 EDT
RC1 seems rather late for a project that has yet to gain legal and PMC approval, as discussed in Comment 10 and Comment 12.
Also, the issue of duplicate content assist proposals MUST be fixed before RC1 or this feature should NOT be shipped, IMO. From a community perspective, Eclipse can not afford to compromise it's hard-earned reputation for consistent quality delivery, if such a fundamental, visible feature would require a work-around.
Comment 26 Marcel Bruch CLA 2012-05-11 01:19:57 EDT
(In reply to comment #25)
> Also, the issue of duplicate content assist proposals MUST be fixed before RC1
> or this feature should NOT be shipped, IMO.

I configured our completion to not contribute to code completion if JDT completion is enabled. I missed the fact that Mylyn disables and replaces JDT completion on startup - which lead to this worse behavior. I'll disable code recommenders completion engines per default for RC1.


> RC1 seems rather late for a project that has yet to gain legal and PMC
> approval, as discussed in Comment 10 and Comment 12.

The server URL issue will be discussed until RC1. 

> From a community perspective,
> Eclipse can not afford to compromise it's hard-earned reputation for consistent
> quality delivery, if such a fundamental, visible feature would require a
> work-around.

We feel the same. Such an issue should not arise in Juno RC1 and later.
Comment 27 Marcel Bruch CLA 2012-05-11 01:24:55 EDT
(In reply to comment #22)
> Looking at M7 I see few major problems in the Java package:
> 
> * Content assist entries are duplicated: Both Java Subwords Completions and
> Java Proposals (Task-Focused) contribute default Java completions which leads
> to duplicate entries. Marcel, unless Subwords Completions returns a disjoint
> set of completions we can't have both computers active at the same time.

I can't make them disjoint. The way content assist is implemented does not allow this. Subwords need to be disabled on startup.


> * Code Recommenders bundles are not signed.
Surprising. All bundles should be signed. I'll look at that.

> * Some bundles contained in the non-orbit dependencies libraries feature don't
> list Eclipse as the provider. Should they?
> * The 3rd party non-orbit dependencies feature includes com.google.gson. Isn't
> that in Orbit now?
I'll fix that immediately.

> * Code Recommenders doesn't contribute a branding icon to the About dialog.
> That would be nice to have.

Never thought about it before. I can add one after fixing the other issues.


> What are the plans in terms of graduating code recommenders?

I've to discuss this with Technology PMC.
Comment 28 Marcel Bruch CLA 2012-05-13 05:27:40 EDT
(In reply to comment #27)
> (In reply to comment #22)
> > * Content assist entries are duplicated
> I can't make them disjoint. The way content assist is implemented does not
> allow this. Subwords need to be disabled on startup.

I've disabled them on first startup and added a simple preference page to allow users to (de-) activate it by a single click (http://goo.gl/RWLol shows an early version of it).

Steffen, you have a filter in the Mylyn's completion proposal computer that only one of both (JDT or Mylyn) can be active at the same time. It looks like you assume that users may accidentally enable both engines (e.g., by pressing "restore defaults")? If this is the intension our approach to deactivate once on first startup is not sufficient enough, right? In that case should Subwords check if one of both, JDT and Mylyn, is active and and if so disable itself completely? 

> > * Code Recommenders bundles are not signed.
> Surprising. All bundles should be signed. I'll look at that.

Fixed. The wrong folder was copied into the release folder.

> > * Some bundles contained in the non-orbit dependencies libraries feature don't
> > list Eclipse as the provider. Should they?

Fixed. All bundles not hosted at Orbit use Eclipse.org as provider name.

> > * The 3rd party non-orbit dependencies feature includes com.google.gson. Isn't
> > that in Orbit now?
> I'll fix that immediately.

Fixed.

> > * Code Recommenders doesn't contribute a branding icon to the About dialog.
> > That would be nice to have.
> Never thought about it before. I can add one after fixing the other issues.

Added.

> > What are the plans in terms of graduating code recommenders?
> I've to discuss this with Technology PMC.

Discussion initiated.

> 

(In reply to comment #10 and comment #12) -  external URL:

The discussion where to place the repository has been initiated. 

Just to make sure that everyone knows that this URL issue not difficult to fix:
Models are delivered from a standard http server (like maven repositories are).
Changing the predefined URL to any URL, e.g. http://download.eclipse.org/recommenders/models/juno/, will work.

I think we'll have a solution for this until RC1.

Anything else considered critical?
To be clear:
I'd like to see Code Recommenders in the Java Package. But if any serious concerns/issues exist that may damage the reputation of Eclipse and/or the Java Packaging - drop it in RCx (or add a subset of features only).
Comment 29 Steffen Pingel CLA 2012-05-14 04:52:25 EDT
(In reply to comment #28)
> I've disabled them on first startup and added a simple preference page to allow
> users to (de-) activate it by a single click (http://goo.gl/RWLol shows an early
> version of it).

Are you disabling the Task-focused proposals when the proposals are enabled?

> Steffen, you have a filter in the Mylyn's completion proposal computer that only
> one of both (JDT or Mylyn) can be active at the same time. It looks like you
> assume that users may accidentally enable both engines (e.g., by pressing
> "restore defaults")? 

Yes, we have to guard against that case.

> If this is the intension our approach to deactivate once on
> first startup is not sufficient enough, right? In that case should Subwords
> check if one of both, JDT and Mylyn, is active and and if so disable itself
> completely?

That sounds right to me. Are you disabling Subwords dynamically, i.e. no results are returned, or by actually modifying the preference?

We should sync up at some point if there are changes that we could drive within JDT to handle overlapping proposal computers more gracefully.

> Anything else considered critical?

I'll take a look at the next package build. Is there a repository where I could consume the fixed build already?

> I'd like to see Code Recommenders in the Java Package. But if any serious
> concerns/issues exist that may damage the reputation of Eclipse and/or the Java
> Packaging - drop it in RCx (or add a subset of features only).

Let's try to fix any problems that come up first. I look at it this way: If we are no longer able to innovate the negative impact could be even greater.
Comment 30 Marcel Bruch CLA 2012-05-15 02:35:08 EDT
(In reply to comment #29)
> (In reply to comment #28)
 
> Are you disabling the Task-focused proposals when the proposals are enabled?

No, only JDT_ALL. While adding MYLYN_ALL wouldn't be a problem, re-activating could be inconsistent. To make it work 'in 80% of all cases'  I would have to store whether MYLYN_ALL was active before. 

It seems that the checkbox may fall too short. Maybe it's better to delegate users to "Java > Editor > Content Assist > Advanced" preference page and explain them what to change there...

> > If this is the intension our approach to deactivate once on
> > first startup is not sufficient enough, right? In that case should Subwords
> > check if one of both, JDT and Mylyn, is active and and if so disable itself
> > completely?
> 
> That sounds right to me. Are you disabling Subwords dynamically, i.e. no
> results are returned, or by actually modifying the preference?


Both. If MYLYN_ALL or JDT_ALL is enabled we return an empty list but also schedule a UIJob to alter the preferences after content assist finished.


> We should sync up at some point if there are changes that we could drive within
> JDT to handle overlapping proposal computers more gracefully.

FWIW, Dani told me that JDT team has no probably no resources for making or reviewing larger modifications to JDT in 3.9.

> > Anything else considered critical?
> 
> I'll take a look at the next package build. Is there a repository where I could
> consume the fixed build already?

The latest Juno build is always available here: 

http://download.eclipse.org/recommenders/updates/train/e42/


I still have a signing problem. The final archives contain ECLIPSE_.RSA and ECLIPSE_.SF files with a underscore which makes the verifier complain. I've asked Jesse for assistance on this. If you know what caused this, I'd be glad to hear about it.



[1] http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/dist/org.eclipse.recommenders.repository.rcp.train.e42/pom.xml
[2] https://hudson.eclipse.org/hudson/job/recommenders.release-e42/ws/dist/org.eclipse.recommenders.repository.rcp.train.e42/target/
Comment 31 Steffen Pingel CLA 2012-05-16 19:05:06 EDT
(In reply to comment #30)
> Both. If MYLYN_ALL or JDT_ALL is enabled we return an empty list but also
> schedule a UIJob to alter the preferences after content assist finished.

So for the Java package MYLYN_ALL will get disabled after first use of content assist? Did you check what happens if you use content assist and then activate a task and use content assist again?

> I still have a signing problem. The final archives contain ECLIPSE_.RSA and
> ECLIPSE_.SF files with a underscore which makes the verifier complain. I've
> asked Jesse for assistance on this. If you know what caused this, I'd be glad to
> hear about it.

I have no idea... we have a custom signing script for Mylyn. Were you able to fix it in the meantime? Would be great to get an updated build contributed to the train to kick of another package build. I addressed the GSon incompatibilities in Mylyn and would like to do another round of testing.
Comment 32 Marcel Bruch CLA 2012-05-17 00:35:48 EDT
(In reply to comment #31)
> (In reply to comment #30)
> > Both. If MYLYN_ALL or JDT_ALL is enabled we return an empty list but also
> > schedule a UIJob to alter the preferences after content assist finished.
> 
> So for the Java package MYLYN_ALL will get disabled after first use of content
> assist? Did you check what happens if you use content assist and then activate
> a task and use content assist again?

No. It's recommenders that disables itself if MYLYN_ALL ||  JDT_ALL is active.
See 5 lines in shouldReturnResults() in http://goo.gl/TfiL1 for exact details.

Regarding shortcut in  preferences:
See line 65ff in http://goo.gl/KWlWS If subwords gets disabled in our preference page, we activate mylyn if installed and jdt otherwise.

> > I still have a signing problem.

It seems that it was a weird caching problem rather than a signing problem.
jarsigner -verify says files are ok.

> Were you able to
> fix it in the meantime? Would be great to get an updated build contributed to
> the train to kick of another package build. I addressed the GSon
> incompatibilities in Mylyn and would like to do another round of testing.

new files are in place. I failed to run the juo aggregator. It terminated with an exception immediately. But I'd like to test it tonight. I'll ask David what's wrong with aggregator.
Comment 33 Steffen Pingel CLA 2012-05-25 04:53:28 EDT
I did some more testing with Juno SR1 and it looks good now. Only open item is the location of the model repository. We have to have a definite answer on this by RC3 otherwise we'll have to revert.

It's unfortunate that Subwords proposals are now disabled by default. It'd be nice if we could come up with an extension point to provide task-focused sorting for the proposals for a future release and use Subwords completion by default.
Comment 34 Marcel Bruch CLA 2012-05-30 12:31:21 EDT
(In reply to comment #33)
> I did some more testing with Juno SR1 and it looks good now. Only open item is
> the location of the model repository. We have to have a definite answer on this
> by RC3 otherwise we'll have to revert.

We received the ok to use download.eclipse.org for delivering recommendation models. A preliminary repository is hosted already there [1] and is used as default in rc2. There should be no references to any external URL.

> It's unfortunate that Subwords proposals are now disabled by default. It'd be
> nice if we could come up with an extension point to provide task-focused
> sorting for the proposals for a future release and use Subwords completion by
> default.

Sounds great. Our own completion engines (calls, overrides, chain) don't make use of subwords yet but should. Let's see what we can do after Juno to make this more flexible.

[1] http://download.eclipse.org/recommenders/models/juno-2012-05-15
Comment 35 Markus Knauer CLA 2012-05-31 04:08:30 EDT
(In reply to comment #34)
> (In reply to comment #33)
> > I did some more testing with Juno SR1 and it looks good now. Only open item is
> > the location of the model repository. We have to have a definite answer on this
> > by RC3 otherwise we'll have to revert.
> 
> We received the ok to use download.eclipse.org for delivering recommendation
> models. A preliminary repository is hosted already there [1] and is used as
> default in rc2. There should be no references to any external URL.

Great! From my point of view we are clean now. Thanks.
Comment 36 Eric Rizzo CLA 2012-06-28 10:32:51 EDT
Can we close this bug now that Juno has shipped?
Comment 37 Markus Knauer CLA 2012-06-28 11:23:46 EDT
(In reply to comment #36)
> Can we close this bug now that Juno has shipped?

Closing as FIXED and released! Thanks!