Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 453769

Summary:	Workspace change notifications consume too much CPU
Product:	[ECD] Orion	Reporter:	Mark Macdonald <mamacdon>
Component:	Server	Assignee:	Anthony Hunter <ahunter.eclipse>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	P3	CC:	ahunter.eclipse
Version:	7.0
Target Milestone:	8.0
Hardware:	PC
OS:	Windows 7
Whiteboard:

Description Mark Macdonald

2014-12-01 10:26:01 EST

Running a local Orion server with HEAD = commit 7dac1da [1]. My computer has a 4-core processor, so 25% CPU usage measured by Windows Task Manager is basically 1 core running flat out.

When I launch the server, 
* Right after launch, the server consumes 25% CPU for a few minutes.
* Then it settles down into a steady constant CPU usage of ~13%.
* Every few minutes, CPU spikes up to 25% again for a bit, then back down to 13%.

What concerns me is the steady state of 13%. I expected the server to eventually become idle, but it never does.

If I pause the JVM, I can see org.apache.commons.io.FileAlterationObserver methods in one of the thread stacks.

Using Process Monitor [2], I can observe it constantly walking my user workspace [3]. Aside from that, it also seems to be scanning the server workspace directory as well, which surprised me:
 C:\Users\mamacdon\dev\foo\.archive\
 C:\Users\mamacdon\dev\foo\.metadata\

^ AFAIK these folders are not visible to a user so I expected them not to be scanned.

My local server has a single active user, so I'm concerned about how this might scale.


[1] http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=7dac1da
[2] http://technet.microsoft.com/en-ca/sysinternals/bb896645.aspx
[3] C:\Users\mamacdon\workspace\dev\foo\ma\mamacdon

Comment 1 Mark Macdonald

2014-12-01 10:41:28 EST

My workspace contains a lot of linked projects, so perhaps it has induced a cycle in the directory scanner?

I can provide a zip of my server workspace via e-mail.

Comment 2 Anthony Hunter

2014-12-01 11:25:21 EST

Szymon activated the IFileChangeNotificationService in the CFActivator which has caused this problem.

It should not be started at all unless you intended to do cf deployments.

Comment 3 Anthony Hunter

2014-12-01 11:49:49 EST

(In reply to Anthony Hunter from comment #2)
> Szymon activated the IFileChangeNotificationService in the CFActivator which
> has caused this problem.
> 
> It should not be started at all unless you intended to do cf deployments.

I added Bug 453792 to disable the cf liveupdate feature by default.

Comment 4 Mark Macdonald

2014-12-01 12:45:39 EST

Adding more info about my workspace:

Server workspace dir:
> C:\Users\mamacdon\workspace\dev\foo

User dir:
> C:\Users\mamacdon\workspace\dev\foo\ma\mamacdon
(2,398 Files, 802 Folders, 25 MB)

Linked project locations: (Note that some of these overlap)
> "ContentLocation": "file:/C:/Users/mamacdon/code/beta3super/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/blort/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/devworks/"
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/Markdown-Plugin/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/cfdebugtest/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/deferred-fs/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/orion-codemirror/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/orion-plugins/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/github/orion-tools/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/myworkspace/"
> "ContentLocation": "file:/C:/Users/mamacdon/code/orion/IDS.Web.IDE/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/orion/IDS.Web.IDE/bundles/com.ibm.team.scm.orion.client/web/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/orion/client/",
> "ContentLocation": "file:/C:/Users/mamacdon/code/orion/client/bundles/org.eclipse.orion.client.users/"
> "ContentLocation": "file:/C:/Users/mamacdon/code/orion/client/modules/orionode/",
(approx. linked content size: 67,899 files, 11,684 folders, 2.68 GB )

Comment 5 Anthony Hunter

2014-12-02 16:01:32 EST

(In reply to Mark Macdonald from comment #0)
> Using Process Monitor [2], I can observe it constantly walking my user
> workspace [3]. Aside from that, it also seems to be scanning the server
> workspace directory as well, which surprised me:
>  C:\Users\mamacdon\dev\foo\.archive\
>  C:\Users\mamacdon\dev\foo\.metadata\

For now I also added filters for .archive and .metadata. with commit:
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=3a8a0ae8472b292a4b202516ccbb2f0d2e92de06

Comment 6 Anthony Hunter

2014-12-04 14:45:58 EST

(In reply to Mark Macdonald from comment #0)
> My local server has a single active user, so I'm concerned about how this
> might scale.

The current IFileChangeNotificationService is sufficient for proof of concept testing but cannot be enabled as is on a production server.

I looked into how org.apache.commons.io was implemented and I was positive it was advertised as "integrated with the core OS" for very good performance.

On Linux, the FileAlterationObserver simply gets a list of all the files and compares it to the list from the previous interval. This means the entire directory tree from the serverworkspace is crawled every second, which is ridiculously bad.

We need to update the cf liveupdate feature so it only listens for changes in a specific project only when the DevMode is turned On for that specific project. We do not need to listen to anything else otherwise.

Comment 7 Anthony Hunter

2014-12-09 15:12:27 EST

Testing of the IFileChangeNotificationService introduced with bug 451383 revealed that the apache commons io file monitoring feature will not work.

IFileChangeNotificationService is removed with the commits in Bug 454643.