Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 370484 - Upgrade to Lucene/Solr 3.5
Summary: Upgrade to Lucene/Solr 3.5
Status: RESOLVED FIXED
Alias: None
Product: Orion
Classification: ECD
Component: Server (show other bugs)
Version: 0.4   Edit
Hardware: PC Windows 7
: P2 normal (vote)
Target Milestone: 0.5 M2   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 369792 376069
Blocks: 334711 358151
  Show dependency tree
 
Reported: 2012-02-02 14:30 EST by John Arthorne CLA
Modified: 2012-05-04 13:39 EDT (History)
1 user (show)

See Also:


Attachments
Change required for Lucene/Solr 3.5 (1.44 KB, patch)
2012-03-20 10:16 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff
Patch with manifest changes (2.98 KB, patch)
2012-03-21 03:23 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff
Patch with all changes (3.51 KB, text/plain)
2012-04-02 10:49 EDT, John Arthorne CLA
no flags Details
Warnings in console (9.78 KB, text/plain)
2012-04-02 10:51 EDT, John Arthorne CLA
no flags Details
Patch for server repo (9.91 KB, patch)
2012-04-04 12:57 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff
Patch for the launch config (1.03 KB, patch)
2012-04-04 13:05 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff
Server patch with minor update (9.91 KB, patch)
2012-04-04 13:32 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description John Arthorne CLA 2012-02-02 14:30:07 EST
We need to upgrade to Lucene/Solr 3.5. We are currently on 1.4.
Comment 1 John Arthorne CLA 2012-02-02 14:32:04 EST
I have entered all the CQ's. For reference:

Library			Orbit CQ	Orion CQ

Lucene Analysis  	6034		6196
Lucene Core		6033		6197
Lucene Highlighter	6036		6198
Lucene Memory		6037		6199
Lucene Misc		6038		6200
Lucene Queries		6039		6201
Lucene Spellchecker	6041		6202

Solr Java Client	6043		6204 
Solr WebApp		6042		6203
Comment 2 Jay Arthanareeswaran CLA 2012-03-20 10:16:59 EDT
Created attachment 212916 [details]
Change required for Lucene/Solr 3.5

I have raised a question on bug 369792. Assuming the answer to that doesn't involve further changes in Orion projects, the changes in this patch is all that we would need to make Orion work with Solr 3.5. 

I have only tested few simple scenarios. John, is there an automated test that I can run and check if the changes are fine?
Comment 3 John Arthorne CLA 2012-03-20 14:57:07 EDT
Thanks Jay! No we unfortunately don't have automated tests for search.
Comment 4 Jay Arthanareeswaran CLA 2012-03-21 03:23:16 EDT
Created attachment 212970 [details]
Patch with manifest changes

This patch includes the manifest changes as well. Only change I am not very sure about is the inclusion of the org.apache.lucene.spatial in the import section of the manifest. I will update after clarifying with Gunnar on bug 369792.
Comment 5 Jay Arthanareeswaran CLA 2012-03-26 05:20:12 EDT
(In reply to comment #4)
> Created attachment 212970 [details]
> Patch with manifest changes
> 
> This patch includes the manifest changes as well. Only change I am not very
> sure about is the inclusion of the org.apache.lucene.spatial in the import
> section of the manifest. I will update after clarifying with Gunnar on bug
> 369792.

There is still some discussion going on on bug 369792 on dependencies and such. While at that, what would be the next step to consume the new solr & lucene bundles and the code changes I have submitted?
Comment 6 John Arthorne CLA 2012-04-02 10:49:18 EDT
Created attachment 213453 [details]
Patch with all changes

Here is an update with a patch for the manifest, the SearchServlet.java imports, and the changes to the search feature.
Comment 7 John Arthorne CLA 2012-04-02 10:51:20 EDT
Created attachment 213454 [details]
Warnings in console

There are many warnings in the console when running with Lucene/Solr 3.5, looks like there is more work to do here to be using 3.5 properly.
Comment 8 John Arthorne CLA 2012-04-02 10:53:42 EDT
Jay, the compile error in help with new lucene is a blocker because we use this same help bundle in Orion. Could you take a look at bug 340563?
Comment 9 John Arthorne CLA 2012-04-04 10:04:44 EDT
Based on the resolution of but 376069, we need to add this system property in Orion when running on Lucene 3.5:

-Dhelp.lucene.tokenizer=standard
Comment 10 Jay Arthanareeswaran CLA 2012-04-04 12:57:55 EDT
Created attachment 213595 [details]
Patch for server repo

This patch should do. Only question remaining is about the dependency on org.apache.lucene.spatial, which is being discussed on bug 369792.

Another patch will follow on the client repo for the launch config.
Comment 11 Jay Arthanareeswaran CLA 2012-04-04 13:05:04 EDT
Created attachment 213597 [details]
Patch for the launch config

This patch contains the new VM argument for the tokenizer switch.
Comment 12 Jay Arthanareeswaran CLA 2012-04-04 13:32:34 EDT
Created attachment 213601 [details]
Server patch with minor update

Just the order of the VM arguments has changes in this patch, in order to keep the existing ones upfront.
Comment 13 John Arthorne CLA 2012-04-05 13:20:41 EDT
Thanks for all your work Jay. I can take this one from here, but I'll let you know if any other problems come up. We depend on the fix in Help so we will need to consume that before changing our build to consume latest Solr. I believe I also have some more CQ's to open because this pulled in new versions of some libraries.
Comment 14 John Arthorne CLA 2012-04-09 16:54:42 EDT
I have entered CQ's for some new Lucene bundles we need, and for updated prerequisites that need to be upgraded to work with latest lucene. Here is a list, along with Orbit and Orion CQ numbers.

org.apache.lucene.spatial 3.5.0   Orbit: 6040 Orion: 6412 
org.apache.lucene.grouping 3.5.0  Orbit: 6035 Orion: 6413

org.apache.commons.io 2.0.1  Orbit: 4791 Orion: 6414 

ch.qos.logback.core    1.0  5885 Orion: 6415 
ch.qos.logback.slf4j   1.0  5886 (part of classic)
ch.qos.logback.classic 1.0  5886 Orion: 6416
Comment 15 John Arthorne CLA 2012-05-04 13:39:05 EDT
Released in I20120504-1158.