Community
Participate
Working Groups
Our customers have to work with large source trees, where creating projects with those trees can be cumbersome. Also, often the user wants to deal only with a subset of files in those hierarchies, but each file system object gets created an a resource in the resource tree and causes performance and scalability problems. With this patch, the Core Resources now support adding resource filters onto IFolder objects that limits at a low level which file system objects are available to the resource tree so large source tree can scale easily when filters are used. How do it work: - The .project file now contains <filteredResource> tags that specify which filter exist on which folder. - A folder can contain a list of Filters (from 0 to n) - Filters can be optionally "inheritable", in which case they apply not only to the folder's children, but also to it's sub-folder children as well. - Each filter has the following attributes: a) The project relative path of the folder it applies. b) The id of the filter type provider (ex: "org.eclipse.core.resource.regexFilterProvider"). c) The arguments of the filter. This is filter dependent. (ex: ".*\.c") d) The filter type. This is a integer that can contain any of the following flags: Filters can be set to include only files or folders matching a criteria, or exclude all file and folders matching a criteria. The two following flags choose between the two modes: public static final int INCLUDE_ONLY = 1; public static final int EXCLUDE_ALL = 2; Note that multiple filters on the same folder can be of a mix of "include_only" and "exclude_all" types. Filters can be applicable to files or folders or both, the following flags indicate the preference: public static final int FILES = 4; public static final int FOLDERS= 8; Filters can be inheritable to sub-folders, the following flag indicate that: public static final int INHERITABLE = 16; Note that all those flags are declared in the org.eclipse.core.resources.IFilter interface. Public API: The new APIs are as follow: IFolder.createFilter() IFolder.removeFilter() IFolder.getFilters() IResource.isFiltered() The IFilter interface And the org.eclipse.core.resource.filterProviders extension point. Some interesting behavior: - If the user creates a new file from the "File | New..." wizard that is actually filtered out by its parent resource filters, it will show up just after creation, but it will disappear from the resource tree when the user manually refreshes the view. I believe this is the proper behavior. - Deleting folders that contain filtered out children work as expected, but might cause the user to delete files that he's not aware are present on the file system. This should be addressed by the UI showing a warning in such case. How to try it out: 1) Apply the patch to the org.eclipse.core.resources project. 2) import the test project provided in this patch 3) Observe that the files "folder/foo.h" and "folder/sub_folder/output.c" are not visible to the resource tree. Soon to come: junit tests patches. To come a little later: Actual UI to allow the user to add filters.
Created attachment 116630 [details] org.eclipse.core.resources patch
Created attachment 116631 [details] Test Project
(In reply to comment #0) > How to try it out: > 1) Apply the patch to the org.eclipse.core.resources project. > 2) import the test project provided in this patch > 3) Observe that the files "folder/foo.h" and "folder/sub_folder/output.c" are > not visible to the resource tree. More details, Note that the full list of objects being hidden are: "folder/foo.h" (the include_only *.c filter on "folder") "folder/bar.h" (the include_only *.c filter on "folder") "folder/sub_folder/file.h" (the inherited include_only *.c filter on "folder") "folder/sub_folder/output.c" (the exclude_all "output.c" on "sub_folder")
A description of this patch can be found at the following blog entry as well: http://sergebeauchamp.blogspot.com/2008/10/new-resource-filters-for-eclipse-core.html
Created attachment 116644 [details] org.eclipse.core.resources patch Updated version with bug fixes when deleting, moving and coping filtered resources and parents.
Some questions on the patch: - addFilter/removeFilter is on IFolder. Did you consider putting it on IContainer? Specifically, wouldn't filters be useful at the IProject level to exclude immediate children of the project? - isFiltered - this could be interpreted as "is filtered out", i.e., is this resource omitted from the resource tree due to presence of a filter in some parent? This might be useful functionality, but I think this is not what the method means. A clearer method name might be "hasFilters". - The filtering implementation would need optimizing for this to scale. There is a lot of creation and manipulation of LinkedList objects each time filtering occurs. This is always something that can be improved later since it doesn't affect the shape of the API.
(In reply to comment #6) > Some questions on the patch: > > - addFilter/removeFilter is on IFolder. Did you consider putting it on > IContainer? Specifically, wouldn't filters be useful at the IProject level to > exclude immediate children of the project? > Good point! the updated patch (https://bugs.eclipse.org/bugs/attachment.cgi?id=116644) has it on IContainer instead, so it works for IProject too :) > - isFiltered - this could be interpreted as "is filtered out", i.e., is this > resource omitted from the resource tree due to presence of a filter in some > parent? This might be useful functionality, but I think this is not what the > method means. A clearer method name might be "hasFilters". > Good idea, I'll change it to "hasFilters" > - The filtering implementation would need optimizing for this to scale. There > is a lot of creation and manipulation of LinkedList objects each time filtering > occurs. This is always something that can be improved later since it doesn't > affect the shape of the API. > I now reimplemented that code path so that there is only 1 linked list created (for the list of filters) and x number of IFileInfo[] for each filtering pass. Hopefully much more efficient resource-wise.
Created attachment 116653 [details] org.eclipse.core.resources patch Latest version of the patch, including all changes requested.
Created attachment 116675 [details] org.eclipse.core.resources patch org.eclipse.test.performance Bug fixes, update of the extension point schema, add junit tests.
Created attachment 117264 [details] 116675: org.eclipse.core.resources patch org.eclipse.test.performance Includes minor changes to support the UI.
Created attachment 117266 [details] org.eclipse.ui.ide Includes UI to add, change, remove and re-order resource filters. To use, simply right-click on a container to see its property page and select the "Resource Filters" page.
John/Boris, can you make sure this gets looked at by the proper parties in resources/ide?
Szymon! :)
(In reply to comment #13) > Szymon! :) Chris and Serge, I'm off till next Wednesday. When I am back, I'll review both parts. I talked to Boris and he is ok with that.
I have reviewed the patch already... see comment #6 and 7. We have been discussing this feature in the context of e4, so it's not clear to me at this point if this is something we should put in e4 first to experiment with before backporting to 3.x. We don't want to introduce major new concepts in 3.x that will conflict with or soon be replaced by e4 concepts.
*** Bug 84988 has been marked as a duplicate of this bug. ***
Created attachment 117339 [details] org.eclipse.ui.ide Minor UI fixes, and included a "String Matcher" filter type provider, so that filters can be specified with simple wildcards "* ?".
I posted a blog entry showing what the UI looks like at: http://sergebeauchamp.blogspot.com/2008/11/resource-filter-ui.html
The UI seems very powerful but not so user-friendly... is this what everybody wants? A UI like the Java Build Path : Source filters seems easier to understand for the user (i.e. what really gets included / excluded?). Similar UI is also used for Java Build Path : Libraries : Access Rules, and CDT C/C++ General : Paths and Symbols : Source Location. Admittedly, the Resource Filters will likely be used by Administrators much more than normal users. But is the separation between files and folders really needed? I love regular expressions, but our customers don't love them to that extent... are they needed here, or would patterns like **/CVS/** also work sufficiently well? With respect to performance... is it realistic to hide resources based on content type, when analyzing the content type requires opening the file? That would likely make everything slower rather than faster. I like the idea of being able to plug in "dynamic" filter kinds programmatically such that client code can filter-out stuff in not-yet-forseeable ways. But for the UI exposed to end users, I'd vote for allowing the well-known Eclipse Path Filters only, just like they are already known from other places in Eclipse.
(In reply to comment #15) > I have reviewed the patch already... see comment #6 and 7. > > We have been discussing this feature in the context of e4, so it's not clear to > me at this point if this is something we should put in e4 first to experiment > with before backporting to 3.x. We don't want to introduce major new concepts > in 3.x that will conflict with or soon be replaced by e4 concepts. So maybe this issue should be moved to e4 now?
Thanks for your comments :) (In reply to comment #19) > The UI seems very powerful but not so user-friendly... is this what everybody > wants? A UI like the Java Build Path : Source filters seems easier to > understand for the user (i.e. what really gets included / excluded?). Similar > UI is also used for Java Build Path : Libraries : Access Rules, and CDT C/C++ > General : Paths and Symbols : Source Location. > From the UI examples you mentioned, I assume you would prefer the UI to be restrained to expose the "mode" and "arguments" as parameter that the user can edit. I think those UI are simplier because the scope of the filtering is smaller too. For instance, the "target" (files, folders, or filers and folders) is always folders, there's no extensibility, and the inheritable flag doesn't apply. > Admittedly, the Resource Filters will likely be used by Administrators much > more than normal users. But is the separation between files and folders really > needed? I love regular expressions, but our customers don't love them to that > extent... are they needed here, or would patterns like **/CVS/** also work > sufficiently well? > Regarding regular expressions, the last UI patch (https://bugs.eclipse.org/bugs/attachment.cgi?id=117339) also includes a "String Matcher" filter type that the user can choose (it is actually displayed as default) instead of the Regular Expression. The String Matcher accepts as argument the familiar Eclipse syntax "* = any string, ? = any character, \ escape literals: * ? \", so it makes it easier for users. From my experience with the feature, the ability to discriminate between files and folders is very powerfull, and really helps prevents unintended filering. > With respect to performance... is it realistic to hide resources based on > content type, when analyzing the content type requires opening the file? That > would likely make everything slower rather than faster. > I guess it depends, but the filter can be used for convenience reason as well, to allow building automatically the project file structure (for example, eliminating "*(Copy).java", etc...). The possiblities are interesting. > I like the idea of being able to plug in "dynamic" filter kinds > programmatically such that client code can filter-out stuff in > not-yet-forseeable ways. But for the UI exposed to end users, I'd vote for > allowing the well-known Eclipse Path Filters only, just like they are already > known from other places in Eclipse. >
I think what I find most confusing in the UI is the organization as a table, where I need to mentally analyze all columns of the table in order to understand what a filter entry does. If you want different kinds of filters (regex, string, others...) what about looking at a UI similar to the Launch Configuration Source Path Mappings? - Each kind of mapping is visually represented by a separate icon, and a single label is sufficient to understand what the filter does. "Include" and "Exclude" kinds of filters separated into separated branches of the filter tree distinguishable by icons again. Perhaps I'm misunderstanding the meaning of the "inheritable" and "isFolder" properties, but I'd think that these can also be expressed by the patterns: isFolder: CVS/ !isFolder: CVS inheritable: **/*.c !inheritable: *.c
Created attachment 117374 [details] org.eclipse.core.resources org.eclipse.core.tests.resources remove dependencies on JDK 1.6
Created attachment 117375 [details] New simplified UI
(In reply to comment #22) > I think what I find most confusing in the UI is the organization as a table, > where I need to mentally analyze all columns of the table in order to > understand what a filter entry does. > > If you want different kinds of filters (regex, string, others...) what about > looking at a UI similar to the Launch Configuration Source Path Mappings? - > Each kind of mapping is visually represented by a separate icon, and a single > label is sufficient to understand what the filter does. "Include" and "Exclude" > kinds of filters separated into separated branches of the filter tree > distinguishable by icons again. > > Perhaps I'm misunderstanding the meaning of the "inheritable" and "isFolder" > properties, but I'd think that these can also be expressed by the patterns: > > isFolder: CVS/ !isFolder: CVS > inheritable: **/*.c !inheritable: *.c > I think I understand your point. I attached (https://bugs.eclipse.org/bugs/attachment.cgi?id=117375) to this bug a screenshot of a simplified UI, let me know what you think. Many columns are removed (type and target) and the "inherited" one shows an "*" only if it is set (and blank if not). Instead of showing a generic "filter" icon, it now show the target (file, folder or file and folders) so it becomes very easy to read from left to right: Exclude All [target] *.c The settings are still available in the "Edit..." and "Add..." dialog, so the user can still change the filter type and target, but it makes it, I think, much easier to read and understand. Does that address your comment? P.S. I think grouping the filters as in the LC source mapping UI would interfere with the ability to reorder the filters in an arbitrary order.
Created attachment 117380 [details] New simplified UI Changes to make the UI easier to read.
Created attachment 117383 [details] org.eclipse.ui.ide New simplified UI patch (includes icons)
Serge, in your blog (http://sergebeauchamp.blogspot.com/2008/10/new-resource-filters-for-eclipse-core.html) you say: > Where linked resources allow the user to build the project > structure by explicitly including files and folders, resource > filters allows the user to build the project structure by > explicitly excluding files and folders. One reasons why some people prefer linked resources is because they do not add a .project file into their directory. And with linked resources plus filters you could have different "views" on the same directory tree. So, I think it is not totally complementary. Is there a technical reason that filters are not supported for linked resources?
(In reply to comment #28) > Is there a technical reason that filters are not > supported for linked resources? I'll let Serge weigh in here, but in terms of Alias Management, it seems problematic that linked resources A, B, C would all open up a directory tree below /foo/bar but each with different resource filters. The algorithm for determining what file is an alias of another file would get much more complex, because it would not only need to analyze root directories but also the filters. It would seem to me that "different views on the same directory tree" should better be accomplished on a higher level... creating a *single* linked resource on /foo/bar, with a *single* resource filter, but different application level filters for creating different views. While writing this, though, it looks like the added complexity on alias management might be inevitable anyways...
(In reply to comment #29) > I'll let Serge weigh in here, but in terms of Alias Management, it seems > problematic that linked resources A, B, C would all open up a directory tree > below /foo/bar but each with different resource filters. The algorithm for > determining what file is an alias of another file would get much more complex, > because it would not only need to analyze root directories but also the > filters. I proposed something similar in bug253912. If there was a single IResource per EFS IFileStore, with mappings (a la linked resources) to the full paths in the workspace tree, then I think the alias management problem would disappear.
Here are some more random thoughts: * We may eventually want resource filters to be in a form that allows evaluating them at a remote location, in case the resources are actually stored remotely. LDAP filters (org.osgi.framework.Filter) have been proposed for this: http://dev.eclipse.org/mhonarc/lists/eclipse-incubator-e4-dev/msg00893.html For advanced kinds of filters that only support local evaluation, it would still be good if at least a subset of the filter can be evaluated remotely. (e.g. filter-out 60% of the stuff remotely, then filter-out the remaining 40% locally). * I'm not in favor of allowing the user to re-order the filters. In terms of performance, the Filter implementers will likely have a better understanding of what's the best ordering than any user. Taking into account that we're striving to allow more parallelism across the board, I prefer the filter semantics to be logical semantics, which the implementation can choose to re-order as needed for best (parallel) execution and performance. * In terms of semantics, ONE well-defined semantics is likely better than having the order influence it. What's the semantics of this filter: INCLUDE /foo/*.c (inheriting) EXCLUDE /foo/bar Will /foo/bar/*.c be included or not? I'd propose that logically, the EXCLUDE filters would always have precedence over the INCLUDE ones. I'd think that not allowing for user-defined filter ordering would allow the UI to be organized in a cleaner way. On another note, (I've mentioned this before), I think it's important that filtered-out resources be still visually represented in the resource tree, such that users understand why certain elements are not visible. In the simplest form, a label decorator should decorate any folders that have children filtered-out.
(In reply to comment #29) > (In reply to comment #28) > It would seem to me that "different views on the same directory tree" should > better be accomplished on a higher level... creating a *single* linked resource > on /foo/bar, with a *single* resource filter, but different application level > filters for creating different views. > > While writing this, though, it looks like the added complexity on alias > management might be inevitable anyways... We have to keep in mind that "application level" filters already exist: View filters. The user can choose to have different views on the same resource tree by configuring explorers with different filters and root. The resource filter scope is limited intentionally, it is not meant to fulfil all the resource organizational requirement of the end user, it is meant only to filter the file system resources that are made available through the refreshLocal() mechanism.
(In reply to comment #28) > Serge, > in your blog > (http://sergebeauchamp.blogspot.com/2008/10/new-resource-filters-for-eclipse-core.html) > you say: > > > Where linked resources allow the user to build the project > > structure by explicitly including files and folders, resource > > filters allows the user to build the project structure by > > explicitly excluding files and folders. > > One reasons why some people prefer linked resources is because they do not add > a .project file into their directory. And with linked resources plus filters > you could have different "views" on the same directory tree. So, I think it is > not totally complementary. Is there a technical reason that filters are not > supported for linked resources? > You can still use a folder linked resource (especially with the patch described here https://bugs.eclipse.org/bugs/show_bug.cgi?id=229633) and put resource filters on that linked resource folder to define what children are seen by refreshLocal(). So whether customers prefer or not to have project sources under the project directory or not (i.e. having a ".project" file in their source directory) is not relevant to this feature: it can be done either way. The reason why resource filters do not filter out linked resource is by design, is not a technical limitation. Linked resources have to be created explicitly, if the user doesn't want to have the linked resource in the project, he can simply delete it, and it won't appear again. Real files under a folder (or linked resource folder) is a different matter, and unless they are filtered out by resource filters, will appear in the project whether the user wants them or not (not taking into consideration some complicated work-around).
(In reply to comment #29) > (In reply to comment #28) > > Is there a technical reason that filters are not > > supported for linked resources? > > I'll let Serge weigh in here, but in terms of Alias Management, it seems > problematic that linked resources A, B, C would all open up a directory tree > below /foo/bar but each with different resource filters. The algorithm for > determining what file is an alias of another file would get much more complex, > because it would not only need to analyze root directories but also the > filters. > > It would seem to me that "different views on the same directory tree" should > better be accomplished on a higher level... creating a *single* linked resource > on /foo/bar, with a *single* resource filter, but different application level > filters for creating different views. > > While writing this, though, it looks like the added complexity on alias > management might be inevitable anyways... > You are right, the optimization that the Alias Manager is doing is problematic right now. When resource filters are put on directory A, all linked resource folders that have the same location as A will inherit the filters as well. It gets problematic with different linked resources pointing to the same location but having different filters. This need to be fixed at the Alias Manager, in my opinion, which need to cause folders pointing to the same location but with different filters to have actually different children.
(In reply to comment #31) > Here are some more random thoughts: > > * We may eventually want resource filters to be in a form that allows > evaluating > them at a remote location, in case the resources are actually stored > remotely. > LDAP filters (org.osgi.framework.Filter) have been proposed for this: > http://dev.eclipse.org/mhonarc/lists/eclipse-incubator-e4-dev/msg00893.html > > For advanced kinds of filters that only support local evaluation, it would > still be good if at least a subset of the filter can be evaluated remotely. > (e.g. filter-out 60% of the stuff remotely, then filter-out the remaining > 40% locally). > I think it's an interesting idea, but it would need to be supported at the EFS level, so that the resource layer could apply the filters to the EFS plugin, and it turns it could the the proper low level magic. I think the current design could still apply, the only difference would be that: 1) "Remote Filter Types" would be available (through the existing extension mechanism) 2) The resource layer would apply the filters onto each folder that is loaded if the EFS plugin supports it. > * I'm not in favor of allowing the user to re-order the filters. In terms > of performance, the Filter implementers will likely have a better > understanding of what's the best ordering than any user. Taking into account > that we're striving to allow more parallelism across the board, I prefer > the filter semantics to be logical semantics, which the implementation can > choose to re-order as needed for best (parallel) execution and performance. > I agree with you that reordering filters isn't as useful as I originally though. I'm not sure how any parallelization applies here (the FS calls ultimately being the bottle neck), but I'll change it to the following: 1) The UI won't allow filters to be re-ordered anymore 2) Filter types extension will be able to specify how "fast" they are, so that the resource layer can sort the filters automatically from the fastest to the slowest. > * In terms of semantics, ONE well-defined semantics is likely better than > having the order influence it. What's the semantics of this filter: > INCLUDE /foo/*.c (inheriting) > EXCLUDE /foo/bar > Will /foo/bar/*.c be included or not? I'd propose that logically, the > EXCLUDE filters would always have precedence over the INCLUDE ones. > > I'd think that not allowing for user-defined filter ordering would allow the UI > to be organized in a cleaner way. It is to resolve this confusion, that the rules are "Include ONLY" and "Exclude ALL" (as displayed in the UI), not simply "include" and "exclude". Also, the user puts filters on a specific folder, so the argument typically matches the file or folder name, not the path (although it really up to the file type implementation to decide). Furthermore, the user can decide whether a filter applies to only files, only folders, or both. To reformulate the rules with the current design and implementation, it becomes: Filters on folder "foo": INCLUDE_ONLY *.c [files] (inheritable) EXCLUDE_ALL bar [folders] So the files in "/foo/bar/*.c" will be excluded, since a rule "exclude all" exist that matches the folder "bar". Does that make sense? > On another note, (I've mentioned this before), I think it's important that > filtered-out resources be still visually represented in the resource tree, such > that users understand why certain elements are not visible. In the simplest > form, a label decorator should decorate any folders that have children > filtered-out. Good idea, I'll look into it.
I'm moving this bug to E4, since this discussion is central to the E4 focus of creating more flexible resources. Once we have a solution we all like we can back-port to 3.x stream if needed.
(In reply to comment #35) "Remote Filters" as an extension of the extendable filter type mechanism are a great idea. That way, an EFS implementation that has enhanced filtering support (beyond the standard EFS API) could also contribute the resource filter in order to give access to its capabilities. > I agree with you that reordering filters isn't as useful as I originally So, would this re-open a possibility to have a UI with separate "include" and "exclude" as separate branches (like the existing JDT/CDT filters)? > It is to resolve this confusion, that the rules are "Include ONLY" and > "Exclude ALL" (as displayed in the UI), not simply "include" and "exclude". Hm... strictly speaking, applying "INCLUDE_ONLY *.c" and "INCLUDE_ONLY *.h" after each other would return an empty set, which I assume is not what you want. I guess that your rules for "include" filters are different than for "exclude" filters. Another reason for organizing the "include" filters separate from the "exclude" filters in the UI. > Also, the user puts filters on a specific folder, so the argument typically I'm a bit concerned about "hiding" the exclude filters by having them potentially dispersed across multiple folders on which they are applied. That way a person who has not created the filters is having a hard time understanding what gets filtered out in total. Maintenance of the filters becomes harder. I think it is better to have all filtering defined in a single place -- the project properties -- like CDT and JDT are doing it. And have the filters defined by project-relative paths. An additional advantage of doing this is, that the filters can be declared already on project creation (new project wizard). Given that you're persisting the filters in the .project file, this should be possible? And here is yet another idea: assume that I'm looking at huge shared project X, but I want to concentrate on some portion of it only. It would be nice if I had "user private" resource exclusion filters that are not persisted in the .project file but in a separate file (e.g. userfilters.prefs). See bug 194414 for related discussions about sharing settings.
(In reply to comment #37) > (In reply to comment #35) > > I agree with you that reordering filters isn't as useful as I originally > > So, would this re-open a possibility to have a UI with separate "include" and > "exclude" as separate branches (like the existing JDT/CDT filters)? > See my comment to the next item: > > It is to resolve this confusion, that the rules are "Include ONLY" and > > "Exclude ALL" (as displayed in the UI), not simply "include" and "exclude". > > Hm... strictly speaking, applying "INCLUDE_ONLY *.c" and "INCLUDE_ONLY *.h" > after each other would return an empty set, which I assume is not what you > want. I guess that your rules for "include" filters are different than for > "exclude" filters. Another reason for organizing the "include" filters separate > from the "exclude" filters in the UI. > That is right, if the user puts two "include_only" filters that are mutually exclusive, the resulting list is empty. I assume what the user would want, is to "include_only" (*.c or *.h). The way to do it currently is to use the regular expression filter type (it's not possible with the default string matcher) and to type in the argument for a single "include_only" filter: ".*\.(c|h)". This is because the filter list is an AND logical operation between the filters, not an OR. While I understand this is not very user friendly, and I was thinking of providing an "OR" and "AND" filter type that would allow the user to group other filters in the UI, so that he could arbitrarily build complex filter statements. For the example above, the user could do, instead of writing a regular expression, the following: OR Filter - INCLUDE_ONLY *.c - INCLUDE_ONLY *.h So the filters would not be grouped in "include" or "exclude" groups, but in logical hierarchy (with "or" and "and" groups). But I was wondering if that would be over engineering, and if such sophistication would actually be used. > > Also, the user puts filters on a specific folder, so the argument typically > > I'm a bit concerned about "hiding" the exclude filters by having them > potentially dispersed across multiple folders on which they are applied. That > way a person who has not created the filters is having a hard time > understanding what gets filtered out in total. Maintenance of the filters > becomes harder. > > I think it is better to have all filtering defined in a single place -- the > project properties -- like CDT and JDT are doing it. And have the filters > defined by project-relative paths. An additional advantage of doing this is, > that the filters can be declared already on project creation (new project > wizard). Given that you're persisting the filters in the .project file, this > should be possible? > One thing, is that while the Core Resource API works on the IContainer itself, the resource need not to exist in order to add filters on it. So whether putting the UI at the project level to edit all the filters or at each folder level is a UI preference only. So a new project wizard can already add filters even on containers that do not exist yet, if need be. While I see that editing all filters in one location is useful, I still see benefits to have the filters editable directly at the container properties, rather than only at the project properties. For example, the user can just right click on the folder and add a resource, as opposed to go to the project properties, select back the folder that he wants to edit, then change the properties there. The CDT has a lot of per file/folder settings that are edited in their local properties pages too. One way to reconcile both abilities is to have a checkbox below the filter table that says "show all filters on container below this resource". What about that? > And here is yet another idea: assume that I'm looking at huge shared project X, > but I want to concentrate on some portion of it only. It would be nice if I had > "user private" resource exclusion filters that are not persisted in the > .project file but in a separate file (e.g. userfilters.prefs). See bug 194414 > for related discussions about sharing settings. > It's an interesting idea. There's also drawbacks in terms of management of those private files.
Created attachment 117656 [details] org.eclipse.core.resources org.eclipse.core.tests.resources Includes minor changes to support the updated UI.
Created attachment 117657 [details] org.eclipse.ui.ide Updated UI: Adds support for AND, OR and NOT logical proposition, support drag and drop, etc... An updated screenshot can be seen at http://sergebeauchamp.blogspot.com/2008/11/here-is-screenshot-of-resource-filter.html
(In reply to comment #37) > (In reply to comment #35) > Hm... strictly speaking, applying "INCLUDE_ONLY *.c" and "INCLUDE_ONLY *.h" > after each other would return an empty set, which I assume is not what you > want. I guess that your rules for "include" filters are different than for > "exclude" filters. Another reason for organizing the "include" filters separate > from the "exclude" filters in the UI. > With the latest patch update, the user can specify filters with "OR", "AND" and "NOT" logical groups, so he can structure the filters in the following manner to effectively include only both *.c and *.h: INCLUDE_ONLY <Or Group> [files] [inheritable] - String Matcher *.h - String Matcher *.c
Hm... while very powerful, again I'm concerned that such an AND/OR construct is extremely hard to understand for the user. I've been spending lots of time with similar constructs in plugin.xml... Again, I'm suggesting to allow full flexibility of the filters in the API, but a limit in the UI. I propose to explicitly create two separate groups "INCLUDE" and "EXCLUDE". Any filter in "INCLUDE" would build an implicit UNION ("OR") of include-only filters. The result of all INCLUDE's would be run against the EXCLUDE's then. This sounds difficult, but it's easy to understand. For the example from your screenshot, it would look like this: INCLUDE **/*.c **/*.h EXCLUDE **/CVS/ **/orange*.c Note how I'm using the "**/" construct to indicate inheritable filters, and the CVS/ construct to indicate folders (icons would be nicer to do file/folder differentiation). See how the "*.c" and "*.h" INCLUDE filters add up, and how the EXCLUDE filters take away from the result. For conflict handling, again I propose that EXCLUDE has precedence over INCLUDE, so orange*.c gets excluded. Users implicitly understand this when reading the construct top down (i.e. "include x,y,z but NOT a,b,c". I'm using "String Matcher" kinds of filters here, but the INCLUDE/EXCLUDE logic could be equally applied for any kinds of contrinbuted filters (regex, contenttype, remote...). I believe that this proposal should be familiar to users of existing similar dialogs in JDT, CDT, PDE; that it's easy to understand yet powerful.
(In reply to comment #42) > Hm... while very powerful, again I'm concerned that such an AND/OR construct is > extremely hard to understand for the user. I've been spending lots of time with > similar constructs in plugin.xml... > > Again, I'm suggesting to allow full flexibility of the filters in the API, but > a limit in the UI. I propose to explicitly create two separate groups "INCLUDE" > and "EXCLUDE". Any filter in "INCLUDE" would build an implicit UNION ("OR") of > include-only filters. The result of all INCLUDE's would be run against the > EXCLUDE's then. > > This sounds difficult, but it's easy to understand. For the example from your > screenshot, it would look like this: > > INCLUDE > **/*.c > **/*.h > EXCLUDE > **/CVS/ > **/orange*.c > > Note how I'm using the "**/" construct to indicate inheritable filters, and the > CVS/ construct to indicate folders (icons would be nicer to do file/folder > differentiation). See how the "*.c" and "*.h" INCLUDE filters add up, and how > the EXCLUDE filters take away from the result. For conflict handling, again I > propose that EXCLUDE has precedence over INCLUDE, so orange*.c gets excluded. > Users implicitly understand this when reading the construct top down (i.e. > "include x,y,z but NOT a,b,c". > > I'm using "String Matcher" kinds of filters here, but the INCLUDE/EXCLUDE logic > could be equally applied for any kinds of contrinbuted filters (regex, > contenttype, remote...). I believe that this proposal should be familiar to > users of existing similar dialogs in JDT, CDT, PDE; that it's easy to > understand yet powerful. > I think I understand your point of view. What about that: By default, the core resource will filter folder contents based on an OR operation or all "include" filters (assuming the default "include all" if no include filters are present) and an AND operation for all "exclude" filters, if present. That way, a single: "include *.h" will still is interpreted as "include only *.h entries" but the following: "include *.h" "include *.c" will be interpreted as "include only (*.h or *.c)". If the user wants to specify something like include_only "*.c" AND include_only "date > 10/7/2006" Then he would have to actually use an "AND" group to group to include filters. Note that this change only changes the default logical operations between "include" filters, it simplifies things only as far as the default matches what the user wants to do. That being said, I'm not in favor of having the UI be limited in comparison to the API, since it would allow code to build filters programatically that the user would then be unable to view or edit in the UI. Besides that, I don't think grouping in the UI the include and exclude filters in two groups "include" and "exclude" adds much since the vast majority of cases the user would have only 1 or 2 filters, and it makes the groups superfluous, but if people think it's important, it's easy to add. I'd like to bring also arguments why we shouldn't try to mimic the way the JDT library access rules UI too closely: 1) The JDT UI is not the only UI to use filters, since the most direct comparison to the user are the package view filters, which use another UI metaphor, along with the CDT source lookup UI (which uses a mixed of grouped an non-grouped entries). 2) As opposed to the JDT UI, the resource filter doesn't work with a static hierarchy. Folder resource can be moved around in the project and between projects, where the filters need to follow the resource, which makes a UI project level list inconsistent with the resource-level properties perspective (the arguments would need to be updated, which may be or not be what the user expects). 3) The path that would be to be specified at the project level to add filters on a folder is confusing, since the filters are really applied on a folder location, not on a project path. For example for the user to put filter on "foo/bar/*.c" implies that the filter is on the project hierarchy, which isn't true, since other resources can be present under "foo/bar" that won't be filtered (i.e. linked resources and groups) and mixes up the logical hierarchy and the physical hierarchy. By having filters on a folder, it is consistent with the concept that filters work on the file system entries that are retrieved by the refreshLocal() mechanism, but not on the logical hierarchy, which is built explicitly by the user. P.S. if you look at the screenshot I posted, the file/folder differentiation is already represented with an icon ;) (See more screenshot at http://sergebeauchamp.blogspot.com/)
Created attachment 117712 [details] org.eclipse.core.resources org.eclipse.core.tests.resources The Include_only filters list are grouped by default with an OR logical operation.
(In reply to comment #43) > include_only "*.c" AND include_only "date > 10/7/2006" What about INCLUDE "*.c" EXCLUDE "date <= 10/7/2006" instead? Sometimes, less is more ... limiting possible input but gaining ease of understanding. I'm quite sure that for the sake of resource filtering, a simple "Include X except Y" should be sufficient. I see your point about specifying filters on the folder level instead of the project level, especially related to rename/move/refactoring. I guess with label decorators indicating what folders carry filtering information, this should be OK. Even if you initially specify filters on the folder level, and persist the data on the folder level, you should still be able to combine the information into a "review only" UI on the project level if you should want at some point. Note, though, that inheritance can make things more complex and harder to understand: /foo/parent: INCLUDE_ONLY "test*" inheriting /foo/parent/child: INCLUDE_ONLY "*.c" --> will "foo.c", "bar.c" be included or filtered-out due to the inherited INCLUDE_ONLY "test*" pattern? See, here's a scenario that I'm a bit afraid of: user _knows_ that foo.c should be included and has a hard time finding the filter that hides it because that very filter may be at any parent node of the current folder. That's why I'm in favor of a single place where all filter specs are collected, instead of having it only dispersed across the file system. Inheritance is hard to get right, take the Windows NTFS security/permission/ACL UI in Windows Explorer as an example. It's usable only because it shows inherited permissions along with the locally set ones.
(In reply to comment #45) > (In reply to comment #43) > > include_only "*.c" AND include_only "date > 10/7/2006" > > What about INCLUDE "*.c" EXCLUDE "date <= 10/7/2006" instead? > > Sometimes, less is more ... limiting possible input but gaining ease of > understanding. I'm quite sure that for the sake of resource filtering, a simple > "Include X except Y" should be sufficient. That's true, but some filter types are not as easily negated as simply changing the equality operator. Another example using "include_only "org.*" is simply not possible to express in the negative using a String Matcher but by using the "NOT" filter type group. I think the way it is now, it complies with your idea of how it should work by default (includes being resolved with OR, and excludes with AND). The only thing is that the possibility is there for the user to create more complex prepositions. If the user doesn't want to have the complexity of AND, OR, NOT groups, he doesn't have to use them at all, and can simply add a set of include_only/exlude_all statements, and it will do what he requires for a lot of cases. > Note, though, that inheritance can make things more complex and harder to > understand: > /foo/parent: INCLUDE_ONLY "test*" inheriting > /foo/parent/child: INCLUDE_ONLY "*.c" > > --> will "foo.c", "bar.c" be included or filtered-out due to the inherited > INCLUDE_ONLY "test*" pattern? > The way it is implemented right now, all inherited filters get inserted in the current folder as if they are simply listed as regular filters. That means that under /foo/parent/child, it will behave as if the user explicitly put the filters: INCLUDE_ONLY "test*" INCLUDE_ONLY "*.c" So the files "/foo/parent/child/foo.c" and "/foo/parent/child/bar.c" will be included by virtue of the "*.c" include filter, and its OR logical relation with other include filters on the same folder. I believe this is the correct behavior. > See, here's a scenario that I'm a bit afraid of: user _knows_ that foo.c should > be included and has a hard time finding the filter that hides it because that > very filter may be at any parent node of the current folder. That's why I'm in > favor of a single place where all filter specs are collected, instead of having > it only dispersed across the file system. > > Inheritance is hard to get right, take the Windows NTFS security/permission/ACL > UI in Windows Explorer as an example. It's usable only because it shows > inherited permissions along with the locally set ones. As I stated above, "foo.c" will get included. I think the general rule can be stated as follows: - If the user adds a filter on a folder (either "include_only" or "exclude_all"), he is guaranteed that his filter will be reflected in final result no matter what are the other filters of the same mode already existing on the folder. For example, if the user adds an "include_only *.c" on a folder, he is guaranteed that all *.c files will be included no matter what are the other include filters are, inherited or not. (of course, exclude filters can always overwrite that). If the user adds an "exclude_all *.d", he is guaranteed that all "*.d" files will be excluded, no matter what are the other filters. In your example above, the user might get unexpected result if a parent folder has an inheritable "exclude_all test*" filter, and its child folder attempts to have an "include_only *.c" filter, and then wonders why the "test.c" file isn't visible. I believe that it is reasonable to expect users to cautiously put inheritable exclude_all filters, since they can't be overridden down the hierarchy. The "filter" overlay in the navigator is pretty clear, though, and looking at the chain of parents for the overlay icon is pretty quick.
Sounds good to me. Thanks for bearing with me! Let's see it in action now, I hope you'll get provisioned as e4 committer soon. Or would you like somebody else to commit your patches into the e4 Repository?
Actually, ip-wise, I think you'll need to commit yourself or we'll need to pass this through EMO review...
(In reply to comment #48) > Actually, ip-wise, I think you'll need to commit yourself or we'll need to pass > this through EMO review... > I will. I faxed the forms today, it shouldn't be long I hope to get the all clear.
This patch is now committed to the /cvsroot/eclipse/e4/org.eclipse.e4.resources CVS repository, along with minor change for 3.5, general cleanup and UI updates.
Since the code has all been released I think this can be marked FIXED. We can open new bugs for any outstanding issues with filters.
