Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 417124

Summary:	New files are not added to the search index (Lucene)
Product:	[ECD] Orion	Reporter:	Silenio Quarti <Silenio_Quarti>
Component:	Server	Assignee:	John Arthorne <john.arthorne>
Status:	RESOLVED FIXED	QA Contact:
Severity:	major
Priority:	P3	CC:	gheorghe, john.arthorne
Version:	4.0
Target Milestone:	4.0 M2
Hardware:	PC
OS:	Windows 7
Whiteboard:

Description Silenio Quarti

2013-09-12 14:49:02 EDT

Running on orion.eclipse.org.

I have created (and commit) new files over a week ago, but I cannot see these files in the "Find File Named" dialog and in search results yet.  It does not seem to be the usual delay to get them added to the search index.

Comment 1 Silenio Quarti

2013-09-12 14:51:47 EDT

Some of the files in org.eclipse.orion.client not indexed yet in my workspace:

bundles/org.eclipse.orion.client.ui/web/orion/folderView.js
bundles/org.eclipse.orion.client.ui/web/orion/editorView.js

Comment 2 Silenio Quarti

2013-09-13 11:08:48 EDT

I cloned org.eclipse.orion.client in another folder yesterday and none of the files in the new clone can be found/searched today.

Comment 3 John Arthorne

2013-09-13 11:39:49 EDT

(In reply to Silenio Quarti from comment #2)
> I cloned org.eclipse.orion.client in another folder yesterday and none of
> the files in the new clone can be found/searched today.

orion.eclipse.org is quite seriously busted. There are 600MB of errors in the log and it has 2000 open file handles which is the ulimit for the process.I wonder if it is caused by bug 417139. I will spin a new build with that fix and see if it helps.

Comment 4 John Arthorne

2013-09-13 13:51:01 EDT

I have deployed a build with the fix for the file leak. I think there might be another error that is breaking the indexer though. On the fresh install I found this in the log:

java.lang.RuntimeException: java.net.URISyntaxException: Illegal character in path at index 0: [HBDI]four
        at org.eclipse.orion.internal.server.search.Indexer.indexProject(Indexer.java:197)
        at org.eclipse.orion.internal.server.search.Indexer.indexWorkspace(Indexer.java:166)
        at org.eclipse.orion.internal.server.search.Indexer.indexUser(Indexer.java:150)
        at org.eclipse.orion.internal.server.search.Indexer.run(Indexer.java:307)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)

This is thrown as a RuntimeException which is aborting the whole indexing pass. So there are two problems here:

1) Why is it failing on this project name
2) It should not abort indexing other projects in this case

Comment 5 John Arthorne

2013-09-13 14:46:15 EDT

The problem occurs when there is a project name starting with a sequence within square brackets. When creating a URI containing such a project name, it is interpreted by java.net.URI as an IPv6 address in the host name. This ends up causing the RuntimeException which kills the search indexer. I have released a fix for the encoding problem, and a second fix to avoid aborting the indexing. This commit also contains a regression test for this case:

http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=161bbb370bbbf7f393f4a9cff743241b7dfabc9d