Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 526180

Summary: ExternalFoldersManager.createLinkFolder is not 'synchronized'
Product: [Eclipse Project] JDT Reporter: Ed Willink <ed>
Component: CoreAssignee: Andrey Loskutov <loskutov>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: error-reports-inbox, loskutov
Version: 3.8.2   
Target Milestone: 4.8 M4   
Hardware: All   
OS: All   
See Also: https://bugs.eclipse.org/bugs/show_bug.cgi?id=527776
https://bugs.eclipse.org/bugs/show_bug.cgi?id=527819
https://git.eclipse.org/r/112314
https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=13eb6669806171ab554d7a112e84deacfa14d6e9
Whiteboard:
Bug Depends on:    
Bug Blocks: 527776    

Description Ed Willink CLA 2017-10-17 17:05:32 EDT
While debugging a Tycho test of an editor I got some fails due to 

Resource '/.org.eclipse.jdt.core.external.folders/.link1' already exists.

which I recognize as a previous hard to reproduce. The stack trace below shows two concurrent paths to
createLinkFolder one of which loses and reports the duplicate. The refreshIfExistAlready suggests that a re-use
is expected, but double creation seems unguarded.

Rerun org.eclipse.ocl.examples.test.xtext.CompletionProposalTests [JUnit Plug-in Test]	
	org.eclipse.equinox.launcher.Main at localhost:56510	
		Thread [main] (Suspended (breakpoint at line 177 in ExternalFoldersManager))	
			owns: RunnableLock  (id=682)	
			ExternalFoldersManager.createLinkFolder(IPath, boolean, IProject, IProgressMonitor) line: 177	
			ExternalFoldersManager.createLinkFolder(IPath, boolean, IProgressMonitor) line: 169	
			ExternalFolderChange.updateExternalFoldersIfNecessary(boolean, IProgressMonitor) line: 49	
			DeltaProcessor.resourceChanged(IResourceChangeEvent) line: 2132	
			DeltaProcessingState.resourceChanged(IResourceChangeEvent) line: 473	
			NotificationManager$1.run() line: 299	
			SafeRunner.run(ISafeRunnable) line: 42	
			NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], ResourceChangeEvent, boolean) line: 289	
			NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 152	
			Workspace.broadcastBuildEvent(Object, int, int) line: 360	
			Workspace.aboutToBuild(Object, int) line: 303	
			Project$1.run(IProgressMonitor) line: 556	
			Workspace.run(ICoreRunnable, ISchedulingRule, int, IProgressMonitor) line: 2240	
			Project.internalBuild(IBuildConfiguration, int, String, Map<String,String>, IProgressMonitor) line: 544	
			Project.build(int, IProgressMonitor) line: 112	
			CompletionProposalTests.doSetUp(String, Injector, String, String) line: 220	
			CompletionProposalTests.testEditor_OCLstdlib_Completions() line: 246	
			NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
			NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
			DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
			Method.invoke(Object, Object...) line: 498	
			CompletionProposalTests(TestCase).runTest() line: 176	
			CompletionProposalTests(TestCase).runBare() line: 141	
			TestResult$1.protect() line: 122	
			TestResult.runProtected(Test, Protectable) line: 142	
			TestResult.run(TestCase) line: 125	
			CompletionProposalTests(TestCase).run(TestResult) line: 129	
			TestSuite.runTest(Test, TestResult) line: 252	
			TestSuite.run(TestResult) line: 247	
			JUnit38ClassRunner.run(RunNotifier) line: 86	
			JUnit4TestReference.run(TestExecution) line: 86	
			TestExecution.run(ITestReference[]) line: 38	
			RemotePluginTestRunner(RemoteTestRunner).runTests(String[], String, TestExecution) line: 539	
			RemotePluginTestRunner(RemoteTestRunner).runTests(TestExecution) line: 761	
			RemotePluginTestRunner(RemoteTestRunner).run() line: 461	
			RemotePluginTestRunner.main(String[]) line: 181	
			PlatformUITestHarness.lambda$0() line: 43	
			1363860802.run() line: not available	
			RunnableLock.run(Display) line: 37	
			UISynchronizer(Synchronizer).runAsyncMessages(boolean) line: 182	
			Display.runAsyncMessages(boolean) line: 4213	
			Display.readAndDispatch() line: 3820	
			PartRenderingEngine$5.run() line: 1150	
			Realm.runWithDefault(Realm, Runnable) line: 336	
			PartRenderingEngine.run(MApplicationElement, IEclipseContext) line: 1039	
			E4Workbench.createAndRunUI(MApplicationElement) line: 153	
			Workbench.lambda$3(Display, WorkbenchAdvisor, int[]) line: 680	
			2127407451.run() line: not available	
			Realm.runWithDefault(Realm, Runnable) line: 336	
			Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 594	
			PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 148	
			IDEApplication.start(IApplicationContext) line: 151	
			UITestApplication(NonUIThreadTestApplication).runApp(Object, IApplicationContext, String[]) line: 52	
			UITestApplication.runApp(Object, IApplicationContext, String[]) line: 43	
			UITestApplication(NonUIThreadTestApplication).start(IApplicationContext) line: 46	
			EclipseAppHandle.run(Object) line: 196	
			EclipseAppLauncher.runApplication(Object) line: 134	
			EclipseAppLauncher.start(Object) line: 104	
			EclipseStarter.run(Object) line: 388	
			EclipseStarter.run(String[], Runnable) line: 243	
			NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
			NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
			DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
			Method.invoke(Object, Object...) line: 498	
			Main.invokeFramework(String[], URL[]) line: 653	
			Main.basicRun(String[]) line: 590	
			Main.run(String[]) line: 1499	
			Main.main(String[]) line: 1472	
		Thread [Active Thread: Equinox Container: 90162e18-a21c-4497-98d6-7ca18be4cce1] (Running)	
		Daemon Thread [Framework Event Dispatcher: org.eclipse.osgi.internal.framework.EquinoxEventPublisher@4ddced80] (Running)	
		Daemon Thread [Start Level: Equinox Container: 90162e18-a21c-4497-98d6-7ca18be4cce1] (Running)	
		Daemon Thread [Refresh Thread: Equinox Container: 90162e18-a21c-4497-98d6-7ca18be4cce1] (Running)	
		Daemon Thread [Bundle File Closer] (Running)	
		Daemon Thread [SCR Component Actor] (Running)	
		Thread [Worker-JM] (Running)	
		Daemon Thread [EMF Reference Cleaner] (Running)	
		Daemon Thread [Java indexing] (Running)	
		Daemon Thread [EventAdmin Async Event Dispatcher Thread] (Running)	
		Thread [Worker-2] (Running)	
		Thread [WorkbenchTestable] (Running)	
		Thread [Worker-3] (Suspended (breakpoint at line 177 in ExternalFoldersManager))	
			ExternalFoldersManager.createLinkFolder(IPath, boolean, IProject, IProgressMonitor) line: 177	
			ExternalFoldersManager.createLinkFolder(IPath, boolean, IProgressMonitor) line: 169	
			ExternalFolderChange.updateExternalFoldersIfNecessary(boolean, IProgressMonitor) line: 49	
			SetContainerOperation(ChangeClasspathOperation).classpathChanged(ClasspathChange, boolean) line: 59	
			SetContainerOperation.executeOperation() line: 111	
			SetContainerOperation(JavaModelOperation).run(IProgressMonitor) line: 724	
			Workspace.run(ICoreRunnable, ISchedulingRule, int, IProgressMonitor) line: 2240	
			Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 2267	
			SetContainerOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 795	
			JavaCore.setClasspathContainer(IPath, IJavaProject[], IClasspathContainer[], IProgressMonitor) line: 5679	
			JREContainerInitializer.initialize(IPath, IJavaProject[]) line: 111	
			LaunchingPlugin$VMChanges.rebind(IProgressMonitor, IJavaProject[]) line: 365	
			LaunchingPlugin$VMChanges.access$0(LaunchingPlugin$VMChanges, IProgressMonitor, IJavaProject[]) line: 299	
			LaunchingPlugin$VMChanges$1.run(IProgressMonitor) line: 285	
			BatchOperation.executeOperation() line: 39	
			BatchOperation(JavaModelOperation).run(IProgressMonitor) line: 724	
			Workspace.run(ICoreRunnable, ISchedulingRule, int, IProgressMonitor) line: 2240	
			Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 2267	
			JavaCore.run(IWorkspaceRunnable, ISchedulingRule, IProgressMonitor) line: 5609	
			LaunchingPlugin$VMChanges.doit(IProgressMonitor) line: 289	
			LaunchingPlugin$JREUpdateJob.run(IProgressMonitor) line: 386	
			Worker.run() line: 56	
		Thread [Worker-4] (Running)	
		Thread [Worker-5] (Running)	
		Thread [Worker-6] (Running)	
		Thread [ReaderThread] (Running)	
		Daemon Thread [com.google.inject.internal.util.$Finalizer] (Running)	
	C:\Program Files\Java\jdk1.8.0_102\bin\javaw.exe (17 Oct 2017, 21:53:53)
Comment 1 Andrey Loskutov CLA 2017-11-27 03:28:18 EST
We observe same issue in our tests, randomly we get this:

!ENTRY EWC.TEST 1 0 2017-11-26 04:06:09.516
!MESSAGE Job (Update Installed JREs class org.eclipse.jdt.internal.launching.LaunchingPlugin$JREUpdateJob) during test testReceiveDifferentNotifications encountered resource exception

!ENTRY org.eclipse.core.resources 4 374 2017-11-26 04:06:09.516
!MESSAGE Resource '/.org.eclipse.jdt.core.external.folders' already exists.
!STACK 0
java.lang.Exception: Resource '/.org.eclipse.jdt.core.external.folders' already exists.
	at org.eclipse.core.internal.resources.ResourceException.provideStackTrace(ResourceException.java:39)
	at org.eclipse.core.internal.resources.ResourceException.<init>(ResourceException.java:35)
	at org.eclipse.core.internal.resources.Resource.checkDoesNotExist(Resource.java:314)
	at org.eclipse.core.internal.resources.Resource.checkDoesNotExist(Resource.java:301)
	at org.eclipse.core.internal.resources.Project.assertCreateRequirements(Project.java:52)
	at org.eclipse.core.internal.resources.Project.create(Project.java:263)
	at org.eclipse.jdt.internal.core.ExternalFoldersManager.createExternalFoldersProject(ExternalFoldersManager.java:320)
	at org.eclipse.jdt.internal.core.ExternalFoldersManager.createExternalFoldersProject(ExternalFoldersManager.java:262)
	at org.eclipse.jdt.internal.core.ExternalFoldersManager.createLinkFolder(ExternalFoldersManager.java:168)
	at org.eclipse.jdt.internal.core.ExternalFolderChange.updateExternalFoldersIfNecessary(ExternalFolderChange.java:49)
	at org.eclipse.jdt.internal.core.ChangeClasspathOperation.classpathChanged(ChangeClasspathOperation.java:59)
	at org.eclipse.jdt.internal.core.SetContainerOperation.executeOperation(SetContainerOperation.java:111)
	at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:724)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2240)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2267)
	at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:795)
	at org.eclipse.jdt.core.JavaCore.setClasspathContainer(JavaCore.java:5679)
	at org.eclipse.jdt.internal.launching.JREContainerInitializer.initialize(JREContainerInitializer.java:111)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin$VMChanges.rebind(LaunchingPlugin.java:365)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin$VMChanges.access$0(LaunchingPlugin.java:299)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin$VMChanges$1.run(LaunchingPlugin.java:285)
	at org.eclipse.jdt.internal.core.BatchOperation.executeOperation(BatchOperation.java:39)
	at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:724)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2240)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2267)
	at org.eclipse.jdt.core.JavaCore.run(JavaCore.java:5609)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin$VMChanges.doit(LaunchingPlugin.java:289)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin$JREUpdateJob.run(LaunchingPlugin.java:386)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56)
Comment 2 Andrey Loskutov CLA 2017-11-27 03:37:40 EST
*** Bug 467364 has been marked as a duplicate of this bug. ***
Comment 3 Andrey Loskutov CLA 2017-11-27 03:53:32 EST
(In reply to Andrey Loskutov from comment #1)
> We observe same issue in our tests, randomly we get this:
> !ENTRY org.eclipse.core.resources 4 374 2017-11-26 04:06:09.516
> !MESSAGE Resource '/.org.eclipse.jdt.core.external.folders' already exists.
> !STACK 0
> java.lang.Exception: Resource '/.org.eclipse.jdt.core.external.folders'
> already exists.

Oops, looking into the code this is a bit different. Current bug is about linked folders *inside* the external folders project. Our issue is the project itself. Looking at the code, the fiddling with linked folders is really not nice, and I don't think I can fix it in the time I have now, so I give the bug back to the team.

For the comment 1 issue I've created bug 527776, which is easier to fix :-).
Comment 4 Andrey Loskutov CLA 2017-11-27 18:12:34 EST
Gerrit seem to not post comments on new patches since the bugzilla upgrade, so here is the link to the patch: https://git.eclipse.org/r/112314

Assigning to me because I can't see this misery anymore.
Comment 9 Sasikanth Bharadwaj CLA 2017-12-06 03:33:40 EST
Verified for 4.8M4 (by inspection)