Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 328464

Summary: Workspace#validFiltered results in I/O via fetchFileInfo => stat()
Product: [Eclipse Project] Platform Reporter: James Blackburn <jamesblackburn+eclipse>
Component: ResourcesAssignee: Malgorzata Janczarska <malgorzata.tomczyk>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: john.arthorne, malgorzata.tomczyk, Szymon.Brandys
Version: 3.7Flags: Szymon.Brandys: review+
Target Milestone: ---   
Hardware: PC   
OS: All   
Whiteboard:
Attachments:
Description Flags
Allways creating the new file info
Szymon.Brandys: iplog+
Conains test for case sensitive search Szymon.Brandys: iplog+

Description James Blackburn CLA 2010-10-22 08:34:17 EDT
Because of Bug 311189 and then knock on Bug 317783, we use the API IWorkspace#validateFiltered to work out whether a resource has really been removed from the workspace (some team providers first remove, then re-add Resources, resulting in model element being incorrectly removed from the CDT model unless we test for existence).

I noticed this API taking a very long time when our NFS filer is being slow. The backtrace looks like:

UnixFileNatives.lstat(byte[], StructStat) line: not available [native method]	
UnixFileNatives.fetchFileInfo(String) line: 73	
LocalFileNativesManager.fetchFileInfo(String) line: 31	
LocalFile.fetchInfo(int, IProgressMonitor) line: 144	
LocalFile(FileStore).fetchInfo() line: 277	
File(Resource).isFilteredWithException(boolean) line: 2060	
Workspace.validateFiltered(IResource) line: 2578	
ResourceChangeHandler$RcMoveHandler$1.run(IProgressMonitor) line: 212	

Must this API call perform I/O?
Comment 1 John Arthorne CLA 2010-11-02 13:42:16 EDT
From a quick look, the I/O doesn't appear to be strictly needed. It just checks if the file exists on disk, but if it doesn't it creates a dummy file info with just the name and directory attributes. Perhaps it could filter on a dummy file info all the time. I can see this being expensive because the loop walks up the parent hierarchy, so there are D native calls, where D is the depth of the resource in the workspace tree.
Comment 2 Malgorzata Janczarska CLA 2010-11-04 06:44:38 EDT
Created attachment 182360 [details]
Allways creating the new file info

This patch does not fetch file info. Instead new file info is always created base on FileStore name. I've tested in on my local filesystem and is works exactly the same.
However I have one concern. The javadoc to IFileStore.getName() says that the name may differ in case for case insensitive filesystems. In order to make sure of the file case in filesystem you have to fetch the file info, so do exactly what we've been trying to eliminate. I'm not sure in what situations the difference in case may occur, but I made few tests on my windows case-insensitive file system and never came across. Do you know any resolution of this problem without performing I/O?
Comment 3 John Arthorne CLA 2010-11-05 11:27:37 EDT
The case mismatch occurs when the local file system resource doesn't match the case of the resource handle. Steps:

1) Create IFile on local file "abc.txt"
2) In the file system, rename the file to "ABC.txt".

-> Now the IFileInfo will return name of "ABC.txt" while resource name is "abc.txt". This difference is mainly interesting while performing refresh local so we can detect case mismatch properly. In this particular situation we can probably just live with the resource name. Any difference in case should get resolved when the next local refresh occurs, but until then the filter will just apply on the resource name in memory rather than the file name on disk.
Comment 4 Malgorzata Janczarska CLA 2010-11-08 05:43:10 EST
Thanks for clarification John. So I think this won't cause any inconsistency in the work of the validFiltered function, because the filter would not be applied to the resource until it's refreshed anyway.
Comment 5 Szymon Brandys CLA 2010-11-08 07:17:10 EST
The patch looks good. 

(In reply to comment #4)
> Thanks for clarification John. So I think this won't cause any inconsistency in
> the work of the validFiltered function, because the filter would not be applied
> to the resource until it's refreshed anyway.

Gosia, could you attach a test for that?
Comment 6 Malgorzata Janczarska CLA 2010-11-08 10:20:08 EST
Created attachment 182621 [details]
Conains test for case sensitive search

Attached patch contains test for case sensitive search.
The test changes the name of the file to uppercase and then synchronizes. The validateFiltered function will reflect the file name change only after the file is synchronized. Before that fix was applied validateFiltered function reflected the file name change even before it was refreshed in workspace, so there is a small change here. However I think that the way it works now is better, as it is consistent with the way filters are actually applied.
Comment 7 Szymon Brandys CLA 2010-11-08 13:58:10 EST
Gosia's patch looks good, however there is an issue with it on Win. Steps:

1) On a project create a regex filter filtering 'a.txt'
2) Create 'a.txt' resource. It will be filtered out. Both old and new #validateFiltered would say the resource is filtered out.
3) Rename file 'a.txt' to 'A.txt' in the file system.
4) Old validateFiltered says that resource 'a.txt' is not filtered out. What is true, since resource 'a.txt' points to file 'A.txt' in the filesystem.
5) New validateFiltered says that resource 'a.txt' is still filtered out, what is not true in this case.

Thus I'm not sure if we can get rid of the i/o call.
Comment 8 Szymon Brandys CLA 2010-11-08 14:01:43 EST
(In reply to comment #1)
> From a quick look, the I/O doesn't appear to be strictly needed. It just checks
> if the file exists on disk, 

Well. it doesn't just check whether the file exist on disc. On Win it gets the real file name to check against filters. It doesn't matter on case sensitive OSes, but does on Win.
Comment 9 Szymon Brandys CLA 2010-11-09 05:02:28 EST
I looked at it again today. 

Going through steps from comment 7 I found something confusing after file 'a.txt' is renamed to 'A.txt' (step 3).
IResource#exists for resource 'a.txt' returns false and for 'A.txt' returns true, what is fine.
However if you get IFileStores for both resources, IFileInfo#exists() for both return true, what confused me.

IWorkspace#validateFiltered method indicates whether a resource is filtered out when it EXISTS. We should check filters against stores that are or would be used by existing resources. And in this particular case the store for existing resource 'a.txt' would match the filter and the resource would be filtered out on creation. 

Thus the patch is fine and is released to HEAD.