Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 473787

Summary: [Assist] Allow contributors more control over content assist filtering
Product: [ECD] Orion Reporter: Curtis Windatt <curtis.windatt.public>
Component: EditorAssignee: Olivier Thomann <Olivier_Thomann>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: Carolyn_MacLeod, gujiman, libingw, Michael_Rennie, Olivier_Thomann, steve_northover
Version: 9.0   
Target Milestone: 12.0   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Attachments:
Description Flags
Proposed patch
none
Proposed patch
none
Proposed patch none

Description Curtis Windatt CLA 2015-07-28 16:34:42 EDT
When content assist is activated, when contributors are called for proposals a possible prefix is provided.  Contributors are free to ignore the provided prefix and instead use the source/offset to determine what the user has typed and what context they are in.  However, once proposals are returned, contentassist.js controls the filtering as the user types.

1) In HTML assist, we want tag proposals to show up whether the user has typed a '<' or not.  However, content assist would always close as the user types '<' because it only recognizes alphanumeric characters as valid prefixes.
(We worked around this by never including the '<' in the prefix or the proposal)

2) In JS assist, we can support 'loose' match proposals (ex different casing).  This works fine when content assist is activated.  However, if the user tries typing while content assist is open the loosely matching proposal will be filtered out as content assist does an exact match on the prefix.
Comment 1 Curtis Windatt CLA 2015-08-05 14:29:13 EDT
3) In JS assist, it would be helpful for specific keys (ex. '.') to select the current selected proposal, rather than filter it out.
Comment 2 guji muun CLA 2015-09-06 22:49:16 EDT
i had a similar problem that i managed to fix it:

open the file 'contentAssist.js' and search and replace the following 4 lines of code:

search:       proposalString.indexOf(prefixText + this._filterText)
replace with: proposalString.toLowerCase().indexOf(prefixText.toLowerCase() + this._filterText.toLowerCase())

search:       proposal.name.indexOf(prefixText + this._filterText)
replace with: proposal.name.toLowerCase().indexOf(prefixText.toLowerCase() + this._filterText.toLowerCase())

search:       proposal.proposal.indexOf(this._filterText)
replace with: proposal.proposal.toLowerCase().indexOf(this._filterText.toLowerCase())

search:       proposal.indexOf(this._filterText)
replace with: proposal.toLowerCase().indexOf(this._filterText.toLowerCase())


What i have done here is told contentAssist to read what i have typed in to lower case and search fo proposals in lower case.
Comment 3 Curtis Windatt CLA 2015-09-09 16:48:21 EDT
(In reply to guji muun from comment #2)
> What i have done here is told contentAssist to read what i have typed in to
> lower case and search fo proposals in lower case.

This would force all content assist filtering to ignore casing.  This fixes (2), though other languages might not support loosely matching on case.  Also might have benefits for HTML.
Comment 4 Curtis Windatt CLA 2015-12-09 15:18:21 EST
Another case

4) Some HTML attributes have dashes - in them, you can't get content assist for aria-.
Comment 5 Steve Northover CLA 2016-01-05 15:13:19 EST
Sorry, what is the issue here?  Is it that each different kind of content assist needs to provide a different filter?
Comment 6 Curtis Windatt CLA 2016-01-05 16:35:57 EST
(In reply to Steve Northover from comment #5)
> Sorry, what is the issue here?  Is it that each different kind of content
> assist needs to provide a different filter?

Each language treats special characters differently.  In HTML you can have - dashes as part of element/attribute names, but in other languages you can't.  In JS we don't want the filter to be case sensitive.  In HTML < is the start of an identifier, while in JS . can be at the start.  We can modify the content assist window to handle these cases, but the behaviour will apply to all languages.  It would be better if the content assist providers could specify language specific behviour.
Comment 7 Steve Northover CLA 2016-01-06 12:34:24 EST
Right.  Provide a hook.  Each content assist contributer provides a regex or likely better we call something that performs magic to return yes/no and we kick off a content assist.

CAR probably should not blog without this fixed.  CAR, you up for fixing this?
Comment 8 Carolyn MacLeod CLA 2016-01-06 13:13:41 EST
Actually, I started looking into fixing this about an hour ago. :)
Curtis, I assume it's ok if I take ownership.
Comment 9 Olivier Thomann CLA 2016-04-07 13:58:09 EDT
After discussing with Carolyn, my understanding of this issue was to provide a function to find out what could be part of a prefix for a possible completion.
In html, you do want '-' to be part of the prefix, but in js you don't want that.
Right now the code assist (from codeassist.js in orion.editor) doesn't let you specify what you want to be part of the prefix. The regex is hard-coded line 303. 
We could pass a function inside the code assist constructor that could be used inside the getPrefixStart function to compute what can be part of a prefix. This can simplify the code that we have right now in the html code assist for aria completions.

Curtis, let know if this is what we want to do for this issue or if you are expecting more changes.
Comment 10 Curtis Windatt CLA 2016-04-07 14:29:52 EDT
(In reply to Olivier Thomann from comment #9)
> After discussing with Carolyn, my understanding of this issue was to provide
> a function to find out what could be part of a prefix for a possible
> completion.
> In html, you do want '-' to be part of the prefix, but in js you don't want
> that.
> Right now the code assist (from codeassist.js in orion.editor) doesn't let
> you specify what you want to be part of the prefix. The regex is hard-coded
> line 303. 
> We could pass a function inside the code assist constructor that could be
> used inside the getPrefixStart function to compute what can be part of a
> prefix. This can simplify the code that we have right now in the html code
> assist for aria completions.
> 
> Curtis, let know if this is what we want to do for this issue or if you are
> expecting more changes.

Yes this is what I was expecting.  Here is one more case to try.  This case isn't a requirement to get working, as allowing whitespace in content assist prefixes will likely cause a lot of problems.

5) In JS we have templates for libs (mongodb, express, etc.).  The templates have names with spaces. ex: 'express - Create an app'.  An empty space can't be part of the prefix so there is no way to type to filter content assist.  You also don't get any assist if you type 'express - C'.
Comment 11 Olivier Thomann CLA 2016-04-14 11:54:42 EDT
Created attachment 260960 [details]
Proposed patch

Curtis, please review
Comment 12 Olivier Thomann CLA 2016-04-18 16:35:48 EDT
Created attachment 261052 [details]
Proposed patch

New patch using an editor context instead of the text directly. It implements the new function for html code assist provider and tern assist code assist provider.
Comment 14 Olivier Thomann CLA 2016-04-18 18:32:53 EDT
Created attachment 261056 [details]
Proposed patch

Removed a bogus test.
Comment 15 Olivier Thomann CLA 2016-04-19 11:04:14 EDT
Delivered.
Comment 16 Olivier Thomann CLA 2016-04-19 11:07:27 EDT
Reopening to look at filtering issues.
Comment 17 Olivier Thomann CLA 2016-04-19 19:07:24 EDT
For filtering issues, I don't really see how we can customize it per code assist provider. The code assist framework could be improve to do a filtering similar to what is done for the outliner: usage of '*', '?', case insensitive/sensitive,...
Curtis, do you think this is enough? We could also implement the Camel matching if the pattern contains only uppercase letters.
Comment 18 Curtis Windatt CLA 2016-04-20 10:07:19 EDT
(In reply to Olivier Thomann from comment #17)
> For filtering issues, I don't really see how we can customize it per code
> assist provider. The code assist framework could be improve to do a
> filtering similar to what is done for the outliner: usage of '*', '?', case
> insensitive/sensitive,...
> Curtis, do you think this is enough? We could also implement the Camel
> matching if the pattern contains only uppercase letters.

- Make filtering case insensitive always
- Allow special characters, specifically dashes. The only char that should close content assist before all proposals are filtered is space/tab.

- No wildcards: Typing wildcards would insert them into your editor which doesn't seem right.
- No camel casing: At least for now. It would be super cool to have JDT like filtering where you can throw a mix of uppercase and lowercase letters to quickly find the proposal you are looking for. But on top of having to add support to content assist for this, you would also have to fix the prefix checking in JS, WebTools and anywhere else you want to support it.  If you think it's easier than that, I'd love to try a demo.
Comment 19 Olivier Thomann CLA 2016-04-20 11:13:45 EDT
I added case insensitive matching. Delivered. Closing as FIXED.
If we want to add different types of filtering then we should open new bug reports.
Comment 20 Curtis Windatt CLA 2016-04-20 17:13:59 EDT
Reopening for one other case
5) In Javascript, open content assist with multiple results.  Type the first letter to filter (Still with multiple results).  Now type '*', '?', '.' or pretty much any character not supported in the prefix.  Result: The assist window does not close or filter.

We should allow all characters in the filter so HTML can use '-' and such.  However, it must actually perform filtering on the results so that the window closes when there are no valid proposals.
Comment 21 Curtis Windatt CLA 2016-04-20 17:16:57 EDT
(In reply to Curtis Windatt from comment #20)
> 5) In Javascript, open content assist with multiple results.  Type the first
> letter to filter (Still with multiple results).  Now type '*', '?', '.' or
> pretty much any character not supported in the prefix.  Result: The assist
> window does not close or filter.

This is actually (6).  You can use (5) from Comment #10 to test the behaviour.  Current fix is an improvement as you can actually filter with spaces, the problem is you can filter with 'expres*******aappppppffeejjeelljjeell'.
Comment 22 Olivier Thomann CLA 2016-04-21 15:22:34 EDT
ok, I found one of the problems. Do you want to accept '*' and '?'? At the location where we apply the filter, we don't know what is the prefix computation function? I don't see why you only want to be able to type what could be accepted in a prefix as this would exclude spaces.
Also the space handling is a problem. I don't see how you can expect to get a proposal for 'express - C<code assist>'. Now you can type 'express<code assist>' and then filter the results using spaces.
Comment 23 Olivier Thomann CLA 2016-04-21 15:33:15 EDT
Delivered.
Comment 24 Curtis Windatt CLA 2016-04-25 13:16:46 EDT
Verified that (6) is fixed while self hosting on master.  We are done here, thanks Olivier.