Community
Participate
Working Groups
Build Identifier: 20110218-0911 As we bring new developers in to our group one enhancement request from devs coming from visual studio or intelliJ is the matching of identifiers based on substrings-- not just prefixes. I thought for sure this had to have been opened before, but I couldn't find this in bugzilla. If this is a duplicate, then please disregard. For example: I have a class: class A { private String connectionsStringInformation; private String getCnxString() { return Info[ctrl+space] In VS and IDEA connectionStringInformation shows up in the autocompletion list, whereas in Eclipse the list seems to only work based on prefixes. I looked around through the source a little and I (have no idea if I'm in the right place) but BinaryTypeEncoding has getFields and getMemberType where I see it doing a binary search on a lazily sorted list of field names -- which makes sense. Obviously you wouldn't want the cost of linearly scanning everything. However, perhaps they are "indexing" the prefixes and all of the camelCase boundaries so that a match on any of those would add that to the completion list. I see that the idea of "prefix" matching seems pretty pervasive through the autocomplete code (param names, etc.). Has this been something you have considered? It does seem like a potentially large effort, but if it's not as bad as it might seem, I would love to be able to contribute a plugin that could do this. Mylyn messes around with the autocomplete list today -- so obviously its possible... Reproducible: Always
Marcel Bruch has blogged about this idea - http://code-recommenders.blogspot.com/2011/05/subword-matching-completion-engine-for.html. There is also a prototype available (details on the blog), maybe you can try it out and provide feedback, if any.
There is currently a patch under review which adds this functionality to Eclipse. (https://bugs.eclipse.org/bugs/show_bug.cgi?id=347397) The IP review should be done in a week. Afterwards we will announce the first version of the subwords completion engine via forum and blog.
(In reply to comment #2) > There is currently a patch under review which adds this functionality to > Eclipse. (https://bugs.eclipse.org/bugs/show_bug.cgi?id=347397) So the feature will be available in Code Recommenders. Marcel, do you plan to contribute this to JDT ?
A "plain" subwords engine clearly would be a great fit for JDT and should be part of JDT. I'm currently thinking about leveraging the Usage Data Collector to continuously build recommender models based on what proposals developers pick from code completion. So, having some hooks into the JDT code completion system to collect this information would be great. Then it would be straight-forward to build new completion systems for JDT but allowing us to learn how people use these engines. What do you think? Is something like this possible: few more listeners and hook methods for code completion?
(In reply to comment #4) > A "plain" subwords engine clearly would be a great fit for JDT and should be > part of JDT. Yes, exactly. > I'm currently thinking about leveraging the Usage Data Collector to > continuously build recommender models based on what proposals developers pick > from code completion. So, having some hooks into the JDT code completion system > to collect this information would be great. Then it would be straight-forward > to build new completion systems for JDT but allowing us to learn how people > use these engines. > > What do you think? Is something like this possible: few more listeners and hook > methods for code completion? You refer to http://www.eclipse.org/epp/usagedata/ right ? I don't think this can collect any data about which content assist proposal (or even quick fix/assist) is chosen by the user. In the past I have thought about tracking user actions (see bug 330357) so I am open to ideas on adding hooks and listeners in general. Why don't you open a bug for this and specify what sort of data you would like to capture, what sort of listeners/hooks you have in mind etc. (Let's keep this bug for subwords engine)
(In reply to comment #2) Steve, Deepak, 1st beta of subwords completion engine is available for testing at Code Recommenders HEAD update site: http://download.eclipse.org/recommenders/updates/head/ Most things work quite nice with JDT - although some parts of the code look more like a hack since JDT's completion wasn't built for that kind of completions. For example, updating the relevance while a developer continues typing is not possible yet since JDT sorts proposals by relevance only once when the completion popup is created. Except from this, the engine works quite nice... It comes with match highlighter, is based on regular expressions and leverages a new "bigrams relevance + prefix match ranking strategy". We'll see how this one works out. Anyway, comments on this are welcome. Deepak, sources are available here: http://goo.gl/APqVI Maybe we could made some changes to JDT's proposals system to support things like rescoring etc. Best, Marcel
Does your engine require to recompute the proposals by JDT Core when typing to narrow down the list?
(In reply to comment #7) > Does your engine require to recompute the proposals by JDT Core when typing to > narrow down the list? No. It computes the proposal once when ctrl+space is triggered. Afterwards it just performs an additional check in isPrefix() similar to the CamelCase match code; every new character further restricts the filtering. It's currently implemented like this: @Override protected boolean isPrefix(final String prefix, final String completion) { // ... return subwordsContext.isRegexMatchButNoPrefixMatch(); } See http://goo.gl/cSGVF for an example. The classes have some glue code (e.g., the constructor) that wouldn't be needed if this code would move into JavaProposal Hierarchy.
If find a solution for bug 350991, then the next step would be to integrate the code into the existing proposal classes with a preference to enable sub-word matching (off by default).
We run into trouble when triggering code completion at certain positions. Currently, if a developer triggers code completion as, e.g., method completion as in composite.ld<^Space> we run JDTs CompilationUnit.codeComplete function as if code completion has been triggered immediately after the dot: composite.<^space>ld Doing it like this, we get all proposals JDT would usually offer and then start the regex filtering. Unfortunately, this works only in those cases where Eclipse makes proposals with empty prefixes. Triggering completion for instance on a TypeName completion: public tp<^Space> --> public <^Space>tp doesn't work since JDT expects that a developer enters at least one character - otherwise the result list would be huge. There are other locations in which the subwords approach doesn't work out well (but I need to look them up again) One solution could be to expect the developer to enter at least the first character and trigger code completion like this: public tp<^Space> --> public t<^Space>p This way, we could get all JDT completions starting with "t" and then could start filtering on 'p'. However, this doesn't feel like subwords completion for method calls. Is there any API method we could use to get a list of all available types? We then could test each type whether it matches the subwords regex *t*p* ?
> method calls. Is there any API method we could use to get a list of all > available types? We then could test each type whether it matches the subwords > regex *t*p* ? Marcel, perhaps you can try using JDT/Core's org.eclipse.jdt.core.search.SearchEngine.searchAllTypeNames(char[], int, char[], int, int, IJavaSearchScope, TypeNameRequestor, int, IProgressMonitor)
(In reply to comment #11) > Marcel, perhaps you can try using JDT/Core's > org.eclipse.jdt.core.search.SearchEngine.searchAllTypeNames(char[], int, > char[], int, int, IJavaSearchScope, TypeNameRequestor, int, IProgressMonitor) If this is the best/most performant way that's the way to go. However, we now get into the situation that we need to know much more about the completion context. For instance, we need to know what kind of completion was requested, i.e, we need the CompletionOn*** Nodes from JDT internals. When we should respond to such completion events outside JDT, it would be good to have a more open completion context as proposed in bug 340945 to safe computation efforts. Currently, we have to rerun the completion to get the information JDT uses to compute its proposals.
After some discussions it turned out that a completion engine should take into account the left-hand-side varname when ranking its proposals. E.g., given: interface Person { String getName(); String getAddress(); int getAge(); } //--- Person p; String name = p.<CTRL-SHIFT-SPACE> should return p.getName with highest relevance score because (i) it has an appropriate return type and (ii) similar method name. There is an early prototype that ranks proposals based on the lhs varname (ignoring return type) is available here: Update Site: http://download.eclipse.org/recommenders/updates/head/e37 Feature Subwords2 Source Code: http://goo.gl/Nvvv1
(In reply to comment #9) > If find a solution for bug 350991, then the next step would be to integrate the > code into the existing proposal classes with a preference to enable sub-word > matching (off by default). Dani, it would be great if this can be taken up in M1. (Some reasons why the feature is awesome - http://blog.deepakazad.com/2012/05/subwords-completion.html). The preference could probably be implemented for all Java* Proposal engines (I know one challenge is mentioned in comment 10) Another question is whether the the feature have to be tied to a completion engine OR can it be a purely filtering mechanism so that all completion engines benefit from it e.g. in the SDK alone we have the Java* engines, the template engines, the word proposal engine.
(In reply to comment #14) > Dani, it would be great if this can be taken up in M1. Maybe some of the code in code recommenders could be used? At any rate the work would have to be done with M1.
(In reply to comment #15) > Maybe some of the code in code recommenders could be used? At any rate the work > would have to be done with M1. The code needs a little polishing and may contain room for performance improvements but the Matcher is encapsulated in a single class, and thus, reusing/moving it should be simple. The polishing can be done before M1. It would be great if the matching logic can be made flexible/exchangeable (CamelCase, Prefix, Subwords matching), as we have a few more completion matchers in mind for the future. FWIW, Steffen expressed his interest to use subwords matching with Mylyn's proposals. I've no clue whether anything special is needed to support Mylyn's proposals or if this would work out of the box when implemented in Java* proposals.
added steffen to keep him posted for mylyn related details.
*** Bug 415807 has been marked as a duplicate of this bug. ***
Gabor plans to implement this as part of GSOC 2015.
Created attachment 254203 [details] Screenshot from IntelliJ Substring matching is really nicely done in Intellij. Screenshot attached.
New Gerrit change created: https://git.eclipse.org/r/50074
Created attachment 254445 [details] Test project The current patch does IMHO not work correctly. Here a small test project for testing the new code completion. Import it as File -> Import -> General -> Existing project and Open the SamplePart class. Position your cursor in the line 32 (after) button.setText("Press me"); Tests which should work IMHO but currently do not work: Test scenario 1: Type "button.selection" -> Press CTRL+Space, addSelectionListener is available in the popup but if I select it via keyboard or mouse the suggestion is not taken over into my code. Test scenario 2: Type "button.", wait 200ms and continue to type selection -> The "addSelectionListener" method is not in the result list.
(In reply to Lars Vogel - Out-of-Office until 13 July 2015 from comment #19) > Gabor plans to implement this as part of GSOC 2015. Very good to hear! I have implemented my own version of substring matching that I've been reasonably happy with using for a while now. In short, it supports two modes of operation: lowercase which matches characters forwards only and UPPERCASE which matches characters anywhere an unmatched character might remain. E.g. 'selectionlistener' finds the add and remove variants and 'listenerADD' finds all listeners that can be added. Maybe this could serve as a starting point? The code is in https://github.com/Overruler/eclipse.jdt.core/blob/master/org.eclipse.jdt.core/search/org/eclipse/jdt/core/search/SearchPattern.java and you can follow the new R_SIMPLE_MATCH constant to see where this feature is used.
Timo, thank you very much for sharing your work! I'll check and test your changes. I'm sure it will be a big help.
Timo, thank you very much for the patch again. I've reviewed it and found it very useful. I've talked to JDT developers and we would like to use your contribution with some small modification. Most notably, (1) the uppercase mode has not yet been considered so that should be left out for now and (2) I would like to try avoiding String instantiation in the matching logic to reduce the overhead. However, so that we can use your contribution in Eclipse, it is legally required that you submit it in one of the official ways: https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#Contributing_a_patch Could you please do that for us? Thank you very much in advance.
New Gerrit change created: https://git.eclipse.org/r/51184
(In reply to Eclipse Genie from comment #27) > New Gerrit change created: https://git.eclipse.org/r/51184 Thank you, Timo, for sharing the patch through Gerrit. However, it seems you have only pushed one file instead of the whole change set. We still need the rest so that we can use it. Apart from this, I've discovered a strange problem. In some cases, when the token is not a prefix of the proposal, proposals are shown in the list but the code is not completed when selected. For example, calling content assist after "elem" offers "otherElement" but if I choose it, it will not be completed. Earlier, you mentioned that you had been using this patch without problems. Have you experienced the same? Are you able to reproduce it?
(In reply to comment #28) > Thank you, Timo, for sharing the patch through Gerrit. However, it seems you > have only pushed one file instead of the whole change set. We still need the > rest so that we can use it. Well, the thing is that subwords matching isn't the /only/ thing I've changed in content assist, and these other goodies don't fall under this bug or maybe even any bug at all. Now, I've tried to keep my Git history on Github somewhat sensible so the changes that integrate this feature might be easy to extract too. But truth be told I haven't actually tested with only the subwords changeset included. Because I have all those other goodies too, why would I give them up? > Apart from this, I've discovered a strange problem. In some cases, when the > token is not a prefix of the proposal, proposals are shown in the list but the > code is not completed when selected. For example, calling content assist after > "elem" offers "otherElement" but if I choose it, it will not be completed. > Earlier, you mentioned that you had been using this patch without problems. Have > you experienced the same? Are you able to reproduce it? Hmm, I can't say for certain one way or other, but I'll keep an eye on that.
(In reply to comment #26) > Most notably, (1) the uppercase mode > has not yet been considered so that should be left out for now I had forgotten I even implemented that before I read the code again to be able to describe what it does. Which goes to show how discoverable that feature really was... So no big loss there. I still think being able to enter parts of words in any order is an important feature that should be supported. Perhaps the search filter could match first with a "no backwards jumps, minimum 1 characters in sequence" algorithm and when that fails try once more with a "all jumps allowed, only after longest sequence found, minimum 2 characters in sequence" algorithm. In any case the implementation has to strike a good balance between allowing many desirable items through while having every additional character filter enough unwanted items to keep the search progressing. It's an interesting problem with no single right solution and I wish you good luck :)
(In reply to Timo Kinnunen from comment #29) > (In reply to comment #28) > > Thank you, Timo, for sharing the patch through Gerrit. However, it seems you > > have only pushed one file instead of the whole change set. We still need the > > rest so that we can use it. > > Well, the thing is that subwords matching isn't the /only/ thing I've > changed in content assist, and these other goodies don't fall under this bug > or maybe even any bug at all. Now, I've tried to keep my Git history on > Github somewhat sensible so the changes that integrate this feature might be > easy to extract too. But truth be told I haven't actually tested with only > the subwords changeset included. Because I have all those other goodies too, > why would I give them up? Yes, I see that your changes have a broader scope but there are important bits in CompletionEngine.java as well, and the functionality in SuperOrSubtypesCompletionHelper seems related too, since we also want substring code completion for types. If you could push your complete changes, I could later sort stuff out but without having it in Gerrit, I have legal barriers using your patch. :) Don't you mind sending us the whole one?
Fine, if you insist. The amended Gerrit change should show up soon.
New Gerrit change created: https://git.eclipse.org/r/51914
WARNING: this patchset contains 1076 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Current status of the JDT/Core and JDT/UI patches provided by Gabor as part of GSoC: Substring completion works for methods and fields. Type completion support needs to be added with corresponding tests. I will be reviewing the JDT/UI patch: https://git.eclipse.org/r/#/c/51914/ Manoj, please review the JDT/Core patch: https://git.eclipse.org/r/#/c/50074/
(In reply to Noopur Gupta from comment #35) > Current status of the JDT/Core and JDT/UI patches provided by Gabor as part > of GSoC: Substring completion works for methods and fields. Type completion > support needs to be added with corresponding tests. > > I will be reviewing the JDT/UI patch: https://git.eclipse.org/r/#/c/51914/ > > Manoj, please review the JDT/Core patch: https://git.eclipse.org/r/#/c/50074/ From the perspective of methods and fields, the patch looks good.
New Gerrit change created: https://git.eclipse.org/r/58135
(In reply to Noopur Gupta from comment #35) > I will be reviewing the JDT/UI patch: https://git.eclipse.org/r/#/c/51914/ JDT/UI patch looks good and can be committed. JDT/Core patch results in 7 test failures in org.eclipse.jdt.text.tests.contentassist.ContentAssistTestSuite. I have updated the tests here: (In reply to Eclipse Genie from comment #37) > New Gerrit change created: https://git.eclipse.org/r/58135
Gerrit change https://git.eclipse.org/r/50074 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=ae317766273ccd74fe0e1bfd1f3ccc3925356a13
(In reply to Eclipse Genie from comment #39) > Gerrit change https://git.eclipse.org/r/50074 was merged to [master]. > Commit: > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/ > ?id=ae317766273ccd74fe0e1bfd1f3ccc3925356a13 I see "WIP" in the commit summary - Is there still work to be committed as part of this?
(In reply to Jay Arthanareeswaran from comment #40) > (In reply to Eclipse Genie from comment #39) > > Gerrit change https://git.eclipse.org/r/50074 was merged to [master]. > > Commit: > > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/ > > ?id=ae317766273ccd74fe0e1bfd1f3ccc3925356a13 > > I see "WIP" in the commit summary - Is there still work to be committed as > part of this? Jay: See comment 35. Type completion support is expected to be added later.
Gerrit change https://git.eclipse.org/r/58135 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.ui.git/commit/?id=b727c4c8f0e9596358fb8e0a8dc95e2eb4febeb8
Gerrit change https://git.eclipse.org/r/51914 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.ui.git/commit/?id=dc2a60fbf2f942a819906d44b23c20e7e86a0b34
Reported bug 481323 for substring completion of types.
How to disable this feature? The ranking is off; it's offering ".contentEquals()" before ".equals()" when typing ".equals"
(In reply to Rustam Abdullaev from comment #45) > How to disable this feature? The ranking is off; it's offering > ".contentEquals()" before ".equals()" when typing ".equals" I don't see this issue. Please open a new bug report with steps using the latest Eclipse build.
