Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 321049 - Resource Filters not working if you use / (slash) oon the include or exclude rules.
Summary: Resource Filters not working if you use / (slash) oon the include or exclude ...
Status: RESOLVED INVALID
Alias: None
Product: Platform
Classification: Eclipse Project
Component: IDE (show other bugs)
Version: 3.6   Edit
Hardware: PC Windows 7
: P3 major with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Platform UI Triaged CLA
QA Contact: Serge Beauchamp CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-27 12:57 EDT by Sorin Sbarnea CLA
Modified: 2010-07-29 06:47 EDT (History)
2 users (show)

See Also:


Attachments
sample not-working configuration (8.12 KB, image/png)
2010-07-29 05:34 EDT, Sorin Sbarnea CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sorin Sbarnea CLA 2010-07-27 12:57:56 EDT
Build Identifier: I20100608-0911

Imagine the following structure
aaa/xxx
aaa/yyy

You want to exclude "aaa/xxx" from the project or to include "aaa/yyy" in the project (considering you already added an exclude for aaa).



Reproducible: Always

Steps to Reproduce:
1. create the file structure from the details
2. go to resource filters and add exclude filter
If you add "aaa" it will work as expected but if you try to add "aaa/xxx" it will fail.

Detail on how to use the dialog:
filter type: exclude all
applies to: folders (also you can choose files and folders)
all children:  checked or not
file and folder attributes:
PROJECT RELATIVE PATH: MATCHES: AAA/WWW/*

In fact if you use "/" the match patter will not work at all. I tried all possible conbinations but each time you try to match subfolder it will just fail.
Comment 1 Serge Beauchamp CLA 2010-07-29 04:40:26 EDT
If you have the folder "xxx" and "yyy" under the folder "aaa", and add a resource filter (on the project) excluding "www", then of course it won't affect the existing folders.

Here's the case I tested:

I have the following structure:

project 'test':
   aaa
       xxx
       yyy

Then, on project 'test', I add the following resource filter:

Exclude all
folder
All Children (recursive)
Project Relative Path matches "aaa/xxx" (case sensitive)

Then "aaa/xxx" is properly excluded from the workspace tree.
Comment 2 Serge Beauchamp CLA 2010-07-29 04:43:07 EDT
Another case that works:

In the same structure above, you add a file in "aaa/xxx/foo.txt"

Then add a resource filter on project 'test':

Exclude all
files and folders
All Children (recursive)
Project Relative Path matches "aaa/xxx/*" (case sensitive)

You will see foo.txt is properly excluded from the workspace.

By the way, note that you can add resource filters on any container, not only on the project, so you can add a resource filter on "xxx" excluding "*", which will have the same effect, but will be less cumbersome to setup.
Comment 3 Sorin Sbarnea CLA 2010-07-29 05:33:05 EDT
I reopened it because I to see at least once case that is still not working: when using both Include and Exclude. Please take a look at the screenshot, in this case the alf/whole_system does not appear in the project tree even if it is properly included.

As a remark, the description form the edit dialog states that * is to be used to do wildcard matches. 

Bases on the description text a normal user would assume that it *has* to add the wildcard after the directory path in order to work so we assumed that we have to enter "/aaa/xxx*" or "aaa/xxx/*" but not "aaa/xxx". The *MATCHES* word also sustain this expected behavior, in fast it is not "starts with" or "contains"

In addition to that the help does not contain any examples of usage.

The next part may not be the subject of this bug, but I think it may be useful in order to understand the original issue: Yesterday I spend 15 minutes with another colleague at work trying to figure out how to use the filtering. As we were unsuccessful we decided to file a bug (we also tried the help and google search)

I don't want to make feature requests here but if several programmers are not able to define some basic filters this clearly indicates some bad UI design/architecture.

Personally I would prefer to use a simple text area where I could add patching pattern. Using a text area will also add additional features like commenting and by default we can have the some sample filters as comments.
Comment 4 Sorin Sbarnea CLA 2010-07-29 05:34:25 EDT
Created attachment 175462 [details]
sample not-working configuration
Comment 5 Serge Beauchamp CLA 2010-07-29 05:52:26 EDT
(In reply to comment #3)
> I reopened it because I to see at least once case that is still not working:
> when using both Include and Exclude. Please take a look at the screenshot, in
> this case the alf/whole_system does not appear in the project tree even if it
> is properly included.
> 

First, the screenshot you display, the rule "include only project relative path matches alf/whole_system" makes sense only if it is set on the 'alf' container.  

If you set that resource filter on the project container, then 'alf' (since it's project relative path doesn't match 'alf/whole_system'), and therefore, none of its children can be included.

This is not a bug, this is by design.
Comment 6 Serge Beauchamp CLA 2010-07-29 06:00:27 EDT
(In reply to comment #3)
> As a remark, the description form the edit dialog states that * is to be used
> to do wildcard matches. 
> 
> Bases on the description text a normal user would assume that it *has* to add
> the wildcard after the directory path in order to work so we assumed that we
> have to enter "/aaa/xxx*" or "aaa/xxx/*" but not "aaa/xxx". The *MATCHES* word
> also sustain this expected behavior, in fast it is not "starts with" or
> "contains"

Are you referring to when the match key is set to 'Project Relative Path'?  In such case it seems clear to me that the path (which is of form "dir/dir/entry") must match the string, which can contain wildcards.

Because the user can already set in the 'Applies to' field whether the entry is a file or folder, the 'Project Relative Path' of a folder does not include a tailing slash.  Nor this format used elsewhere in Eclipse either (see the 'Resource' property page "Path" field).

The user could setup a filter that prevent files ending with *.c included in a *.bak folder to be displayed, for example, with the filter:

Exclude all [Files] Project Relative Path matches folder/*.bak/*.c (recursive)

I don't see any inconsistency there.
Comment 7 Sorin Sbarnea CLA 2010-07-29 06:11:05 EDT
I think that there are something not clear here regarding the file structure:

PROJECT_ROOT:
alf/
alf/databases/
alf/foo/
alf/whole_system/
installer/
photoshop/

* Every directory from these has files in it and other subdirectories..
* exclusion of "installer" and "photoshop" worked.
* exclusion of "alf/databases" worked (relative, recursive)

Now I wanted to add an inclusion filter to include only files from "alf/whole_system/", I did but the effect is that the entire "alf" folder is removed from the project tree. I was expecting to keep seeing "alf/whole_system".

Please can you explain how am I supposed to achieve this without modifying the other rules (the ones from exclude all group)?.
Comment 8 Serge Beauchamp CLA 2010-07-29 06:19:21 EDT
(In reply to comment #3)
> In addition to that the help does not contain any examples of usage.
> 

That's true, this could be improved.

> The next part may not be the subject of this bug, but I think it may be useful
> in order to understand the original issue: Yesterday I spend 15 minutes with
> another colleague at work trying to figure out how to use the filtering. As we
> were unsuccessful we decided to file a bug (we also tried the help and google
> search)
> 
> I don't want to make feature requests here but if several programmers are not
> able to define some basic filters this clearly indicates some bad UI
> design/architecture.
> 
> Personally I would prefer to use a simple text area where I could add patching
> pattern. Using a text area will also add additional features like commenting
> and by default we can have the some sample filters as comments.

First, I'd would say that filtering and pattern matching is an advanced feature, and we spent a great deal of time discussing about all the ways it should be presented, how the rules would apply, etc...

Then, still, people can confused if they misunderstand how it works, especially that there's not really a 'test' feature that allows the user to see which files or folders would be filtered in or out by the rules.  We can always improve the help documentation.

I think what you are expecting, is that the whole workspace tree would be built, and then afterwards each file path would be compared against the string pattens you specified, and include or exclude the files based on the result of this comparison.

This is not how it works for a simple reason that one main goal of the resource filtering feature is to allow projects to scale with file hierarchies containing thousands of items.

As such, before even creating and evaluating files for inclusion or not, their parent directory is tested against the filters, and a file can't be included if its parent is not.

So if you have a filter that includes only "folder/file.txt", Eclipse can't decide that, when "folder" is evaluated, it needs first to fetch all of its content to see if any children is included by the filters, then conclude that after all, "folder" should be included even though filters exclude it, because one of its children is explicitly included.

When you consider that the filters types are extensible by 3rd parties (core.resources doesn't know the details of the filter matching), you'll realize that there's no shortcuts that Eclipse could do to optimize this.

Therefore, if a folder is excluded by a rule, none of its children will be fetched, period.

Doing the opposite would incur a massive performance penalty (think when the user has directories with hundred of thousands of files), and would defeat a major goal of the resource filters.  Then, it would be next to impossible for the user to figure out why, even though he explicitly excluded a folder, Eclipse is still extremely slow at loading and refreshing his project.

The Eclipse filtering feature is not a static pattern matching against a list of paths, it is a dynamic filtering that evaluate each file system object during the refresh operation and determines at that point whether it should be included in the workspace.

I hope this make sense for you.
Comment 9 Serge Beauchamp CLA 2010-07-29 06:31:14 EDT
(In reply to comment #7)
> I think that there are something not clear here regarding the file structure:
> 
> PROJECT_ROOT:
> alf/
> alf/databases/
> alf/foo/
> alf/whole_system/
> installer/
> photoshop/
> 
> * Every directory from these has files in it and other subdirectories..
> * exclusion of "installer" and "photoshop" worked.
> * exclusion of "alf/databases" worked (relative, recursive)
> 
> Now I wanted to add an inclusion filter to include only files from
> "alf/whole_system/", I did but the effect is that the entire "alf" folder is
> removed from the project tree. I was expecting to keep seeing
> "alf/whole_system".
> 
> Please can you explain how am I supposed to achieve this without modifying the
> other rules (the ones from exclude all group)?.

You can do:

1) Right click on the "alf" folder, select "Properties..."
2) Select the "Resource/Resource Filters" section.
3) All the following filter:

   Include only [Folder] Name matches whole_system

If, for some reason, you really want to create your filter on the project itself, and not on the 'alf' folder, you can make a more complex include rule:

  (OR Group) Include only [Folders] OR recursive
      Project relative Path matches alf/whole_system
      Project relative Path matches alf

You can create the "OR" group by selecting the "Add Group..." button.  You can use drag and drop to move items in and out of groups too.

Note that by specifying your include only [folders] rule on the project, the other exclude all rules are superfluous.
Comment 10 Sorin Sbarnea CLA 2010-07-29 06:47:32 EDT
Thanks! Now I do know how to setup this - even in two modes. It is far more complicated than I expected and probably deserves a blog post/guide or a question on stackoverflow. 

I'm sure that I'm will not be the only one fooled by a not working "include only" filter this will not work due to "xxx/yyy". Maybe I will spend some time documenting this for others.