Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 311189 - external settings are changed & lost after project set import replaces existing project
Summary: external settings are changed & lost after project set import replaces existi...
Status: RESOLVED FIXED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-core (show other bugs)
Version: 7.0   Edit
Hardware: PC All
: P3 major (vote)
Target Milestone: 7.0   Edit
Assignee: James Blackburn CLA
QA Contact: Doug Schaefer CLA
URL:
Whiteboard:
Keywords:
Depends on: 312430
Blocks:
  Show dependency tree
 
Reported: 2010-04-30 10:33 EDT by James Blackburn CLA
Modified: 2010-07-28 15:26 EDT (History)
0 users

See Also:


Attachments
patch + test (28.94 KB, patch)
2010-05-05 10:39 EDT, James Blackburn CLA
jamesblackburn+eclipse: iplog-
Details | Diff
Tweak (1.08 KB, patch)
2010-05-05 12:39 EDT, James Blackburn CLA
jamesblackburn+eclipse: iplog-
Details | Diff
patch 2 (10.62 KB, patch)
2010-05-11 07:21 EDT, James Blackburn CLA
jamesblackburn+eclipse: iplog-
Details | Diff
patch 3 (5.47 KB, patch)
2010-05-12 06:10 EDT, James Blackburn CLA
jamesblackburn+eclipse: iplog-
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description James Blackburn CLA 2010-04-30 10:33:58 EDT
This is an odd one.

Doing a clean import of a projectset containing a bunch of CDT projects works fine. However subsequently re-importing the team project set (at some point in the future) causes paths to be replaced / change in a non-obvious way.

For example one .cproject went from:

...
<externalSetting>
  <entry flags="VALUE_WORKSPACE_PATH" kind="includePath" name="/TRDriver/src"/>
  <entry flags="VALUE_WORKSPACE_PATH" kind="libraryPath" name="/TRDriver/Debug"/>
  <entry flags="" kind="libraryFile" name="TRDriver"/>
</externalSetting>
...

to:
...
<externalSetting>
  <entry flags="VALUE_WORKSPACE_PATH|RESOLVED" kind="includePath" name="TRDriver/src"/>
</externalSetting>
...

There's something really weird going on here as:
  - The .cproject file hasn't changed on HEAD between the project set first being imported and the subsequent replace
  - 2 paths have been lost
  - The flags of the src includePath, and the value of the path have *both* changed.  
    + For export purposes, this external setting is now wrong and will be treated as relative to the referencing project's Project.

Not yet sure what could cause these to spontaneously change... Any bright ideas?

In the MBS the paths are also missing, and I also see random deltas in the option values:

from:
<option id="gnu.c.link.option.other.10961892" name="Other options (-Xlinker [option])" superClass="gnu.c.link.option.other" valueType="stringList"/>
to:
<option id="gnu.c.link.option.other.10961892" name="Other options (-Xlinker [option])" superClass="gnu.c.link.option.other"/>
Comment 1 James Blackburn CLA 2010-04-30 10:47:57 EDT
(In reply to comment #0)
> In the MBS the paths are also missing, and I also see random deltas in the
> option values:
> 
> from:
> <option id="gnu.c.link.option.other.10961892" name="Other options (-Xlinker
> [option])" superClass="gnu.c.link.option.other" valueType="stringList"/>
> to:
> <option id="gnu.c.link.option.other.10961892" name="Other options (-Xlinker
> [option])" superClass="gnu.c.link.option.other"/>

Overriding and updating the .cproject, opening properties and pressing OK shows the optionValue change; it's likely due to some tidy in MBS since the last time the .cproject was checked in.
 
The paths don't spontaneously break though which suggests something bad happening in an event handler on checkout.
Comment 2 James Blackburn CLA 2010-05-04 10:27:46 EDT
Delta events are fired during the project set import. The CDT core model responds to the source being deleted by rejigging (&breaking) the path entries:

ResourceChangeHandler$RcMoveHandler.handleResourceRemove(IResource) line: 208	
ResourceChangeHandlerBase$DeltaVisitor.visit(IResourceDelta) line: 105	
ResourceDelta.accept(IResourceDeltaVisitor, int) line: 68	
ResourceDelta.accept(IResourceDeltaVisitor, int) line: 79	
ResourceDelta.accept(IResourceDeltaVisitor, int) line: 79	
ResourceDelta.accept(IResourceDeltaVisitor) line: 48	
ResourceChangeHandler(ResourceChangeHandlerBase).doHandleResourceMove(IResourceChangeEvent, ResourceChangeHandlerBase$IResourceMoveHandler) line: 172	
ResourceChangeHandler.doHandleResourceMove(IResourceChangeEvent, ResourceChangeHandlerBase$IResourceMoveHandler) line: 357	
ResourceChangeHandler(ResourceChangeHandlerBase).resourceChanged(IResourceChangeEvent) line: 131	
NotificationManager$2.run() line: 291	
SafeRunner.run(ISafeRunnable) line: 42	
NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], IResourceChangeEvent, boolean) line: 285	

This is exacerbated by very frequent notifications as the import action isn't wrapped in a workspace operation (bug 311526).

The way the team providers hook in and work (taking CVS as the canonical example), when replacing existing projects:
  - Lock the projects being replaced
  - Delete all the resources under the project 
     + They don't delete the project itself
  - From a resource change handler perspective all the resources are removed, then re-added.

It seems wrong that CDT automatically deletes paths added by the user. The same looks to be true for per-File and per-Folder resource configurations.

Ideally we should delete paths and symbols created by the user. Certainly it's wrong that the changes persist even after the contents of the project metadata has been replaced.
Comment 3 James Blackburn CLA 2010-05-04 10:42:16 EDT
Hmm that back-trace never does getProjectDescription but never does a set. AFAICS it doesn't so anything useful but burn CPU...

The thing loss of paths comes from here - which schedules a job with an incorrect updated set of path entries:
PathEntryManager.updatePathEntryFromDeleteSource(ISourceRoot) line: 1452	
PathEntryManager.processDelta(ICElementDelta) line: 1410	
PathEntryManager.processDelta(ICElementDelta) line: 1419	
PathEntryManager.processDelta(ICElementDelta) line: 1419	
PathEntryManager.elementChanged(ElementChangedEvent) line: 1338	
CModelManager$1.run() line: 1108	
SafeRunner.run(ISafeRunnable) line: 42	
CModelManager.notifyListeners(ICElementDelta, int, IElementChangedListener[], int[], int) line: 1101	
CModelManager.firePostChangeDelta(ICElementDelta, IElementChangedListener[], int[], int) line: 1058	
CModelManager.fire(ICElementDelta, int) line: 1022	
CModelManager.fire(int) line: 979	
CModelManager.resourceChanged(IResourceChangeEvent) line: 898

The spawned job updates the paths setting the project description (with the WR rule held):

CProjectDescriptionManager.setProjectDescription(IProject, ICProjectDescription, int, IProgressMonitor) line: 815	
CProjectDescriptionManager.setProjectDescription(IProject, ICProjectDescription, boolean, IProgressMonitor) line: 804	
CProjectDescriptionManager.setProjectDescription(IProject, ICProjectDescription) line: 799	
CoreModel.setProjectDescription(IProject, ICProjectDescription) line: 1389	
ConfigBasedPathEntryStore.setRawPathEntries(IPathEntry[]) line: 153	
PathEntryStoreProxy.setRawPathEntries(IPathEntry[]) line: 105	
PathEntryManager.saveRawPathEntries(ICProject, IPathEntry[]) line: 1009	
SetPathEntriesOperation.executeOperation() line: 55	
SetPathEntriesOperation(CModelOperation).execute() line: 338	
SetPathEntriesOperation(CModelOperation).run(IProgressMonitor) line: 603	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1975	
SetPathEntriesOperation(CModelOperation).runOperation(IProgressMonitor) line: 635	
PathEntryManager.setRawPathEntries(ICProject, IPathEntry[], IProgressMonitor) line: 636	
PathEntryManager$2$1.run(IProgressMonitor) line: 1469	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1975	
Workspace.run(IWorkspaceRunnable, IProgressMonitor) line: 1957	
PathEntryManager$2.run(IProgressMonitor) line: 1463	
Worker.run() line: 54	

The upside of this is that the resources are locked during the replace. So if we do the path updating / reconciling while the resource rule is held we won't end up removing paths that are about to reappear...
Comment 4 James Blackburn CLA 2010-05-04 10:51:02 EDT
(In reply to comment #3)
> AFAICS it doesn't so anything useful but burn CPU...

Ah, I see that it might make changes on #done()...
Comment 5 James Blackburn CLA 2010-05-05 10:39:40 EDT
Created attachment 167148 [details]
patch + test

This patch adds a test for, and fixes the issue with source entries disappearing during resource replace in a team operation.

Both the backtraces in comment 2 and comment 3 are relevant: 
  - the CModelManager is firing a model event causing PathEntryManager to delete the source entry, 
  - the settings model ResourceChangeHandler removes the same source paths on resource change events (as well as removing resource specific configurations).

The fix is two-fold:
  - PathEntryManager#updatePathEntryFromDeleteSource checks that the source entry resource doesn't exist when it actually performs the path entry removal
  - Moved all the ResourceChangeHandler checking / update logic from the resource change handler itself to the workspace job scheduled from #done.

ResourceChangeHandler was a complete mess, so I've tidied and documented it. Fundamentally it responds / handles these events:
  - Project close / move, notifies the CProjDescManager
  - Resource Delete: it removes any corresponding SourceEntries, and any resource specific configurations
  - Resource move: if the destination is still within the same project, it updates any source entries / resource specific configurations. If the destination is in a different project, it eats the event (this is probably not the best move, but this is what it did before...).

The code to do the above was duplicated in two call-backs #handleResourceRemove() and #handResourceMove(). I've removed the duplication so modified resources are now put in a map fMovedResouces[from -> to] where to can be null. The source entry / resource config fix up is moved to #done()'s workspace modification job.

Other tests look good.
Comment 6 James Blackburn CLA 2010-05-05 11:58:36 EDT
Committed first patch.

Still not tracked down why / where the leading '/' of the absolute path get to. Leaving open for a bit...
Comment 7 James Blackburn CLA 2010-05-05 12:39:47 EDT
Created attachment 167173 [details]
Tweak

Add a small tweak, refresh the resource in question as it's possible to enter the handler when the resource does exist on the filesystem, but the workspace isn't aware of it yet.
Comment 8 James Blackburn CLA 2010-05-11 07:21:24 EDT
Created attachment 167894 [details]
patch 2

Another test + patch.

The logic we were using to discover whether the project description was externally modified, was wrong. We cache the core.resources modificationStamp for this needn't always go forwards. In fact if the .cproject is deleted and re-created, the count starts from at 1.
Correctly clear the cached cproject if the external file changes in any way.

Test added for the issue.
Comment 9 James Blackburn CLA 2010-05-12 06:10:51 EDT
Created attachment 168110 [details]
patch 3

The platform doesn't honour the API contract for IResource#getModificationStamp() (bug 160728)

The result is that subsequent generations of IResource share the same modification stamp starting from 0.

The suggested solution is to create a modification stamp = getModificationStamp() + getLocalTimeStamp().  Both of these are cached so should be fast.

Added a method #getModificationStamp(IResource) to XmlProjectDescriptionStorage to handle this.
Comment 10 James Blackburn CLA 2010-05-12 06:15:17 EDT
Fixed in HEAD.
Comment 11 CDT Genie CLA 2010-07-28 15:25:48 EDT
*** cdt cvs genie on behalf of jblackburn ***
Bug 311189 settings are changed & lost after project set import replaces existing project

[*] PathEntryManager.java 1.105 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/internal/core/model/PathEntryManager.java?root=Tools_Project&r1=1.104&r2=1.105

[*] ResourceChangeHandlerBase.java 1.7 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/core/settings/model/util/ResourceChangeHandlerBase.java?root=Tools_Project&r1=1.6&r2=1.7

[*] ResourceChangeHandler.java 1.14 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/internal/core/settings/model/ResourceChangeHandler.java?root=Tools_Project&r1=1.13&r2=1.14

[*] AllCoreTests.java 1.21 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core.tests/model/org/eclipse/cdt/core/model/tests/AllCoreTests.java?root=Tools_Project&r1=1.20&r2=1.21
[+] Bug311189.java  http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core.tests/model/org/eclipse/cdt/core/model/tests/Bug311189.java?root=Tools_Project&revision=1.1&view=markup

[*] ResourceChangeHandler.java 1.15 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/internal/core/settings/model/ResourceChangeHandler.java?root=Tools_Project&r1=1.14&r2=1.15
Comment 13 CDT Genie CLA 2010-07-28 15:26:51 EDT
*** cdt cvs genie on behalf of jblackburn ***
Bug 311189 Settings changed / lost after project set re-import.  IResource#getModificationStamp doesn't honour the API contract. It may return an identical stamp even if the underlying resource has been removed and re-added.

[*] XmlProjectDescriptionStorage2.java 1.4 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/internal/core/settings/model/xml2/XmlProjectDescriptionStorage2.java?root=Tools_Project&r1=1.3&r2=1.4

[*] XmlProjectDescriptionStorage.java 1.8 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.core/model/org/eclipse/cdt/internal/core/settings/model/xml/XmlProjectDescriptionStorage.java?root=Tools_Project&r1=1.7&r2=1.8