Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 415796 - [server] provide migration of legacy metadata storage to simple metadata storage
Summary: [server] provide migration of legacy metadata storage to simple metadata storage
Status: RESOLVED FIXED
Alias: None
Product: Orion
Classification: ECD
Component: Server (show other bugs)
Version: 4.0   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: 4.0 M2   Edit
Assignee: Anthony Hunter CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 412995
  Show dependency tree
 
Reported: 2013-08-23 16:12 EDT by Anthony Hunter CLA
Modified: 2013-10-03 16:21 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Anthony Hunter CLA 2013-08-23 16:12:02 EDT
In from Bug 412995 we have introduced a new simple metadata storage.

We need to provide a migration from the legacy metadata storage (Orion 3.0 CompatibilityMetaStore) to the new simple metadata storage (Orion 4.0 SimpleMetaStore)

We plan to have a command line utility to migrate the storage.
Comment 1 Anthony Hunter CLA 2013-09-06 11:37:38 EDT
I have added the ProjectMigration utility to the org.eclipse.orion.tools bundle.

There is a ProjectMigration.jardesc to create the ProjectMigration.jar

The migration is run from the command line:
java -jar /workspace/orion-dev/ProjectMigration.jar -usertree -root /opt/data/serverworkspace/

ProjectMigration: Usage: java -jar ProjectMigration.jar [ -flat | -usertree ] -root folder

There is option for flat or usertree, depending on the need based on the value of orion.file.layout for the Orion server.

Note that the migration does not handle the existing userId in the IOrionCredentialsService. This needs to be done.

A secondary important topic is that I need to write JSON files so I needed to use the org.json library. I could not figure out how to get the org.json.jar into a classpath to run from the command line, so for now I just put the JSON sources in the org.eclipse.orion.tools bundle. This likely needs to be resolved, I am not sure if it is legal to consume the org.json source within the project.
Comment 2 Anthony Hunter CLA 2013-09-06 15:48:13 EDT
(In reply to Anthony Hunter from comment #1)
> Note that the migration does not handle the existing userId in the
> IOrionCredentialsService. This needs to be done.

This is now handled in commit
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=28af8424d08717260e7b0213b5977c1f3267a428

You can now completely migrate a server from the old to new.

You should first shutdown and backup the server.
Next run the migration from the command line

Add the setting orion.core.metastore=simple to your orion.conf so that you use the simple metadata storage

You can also remove .metadata/.plugins/org.eclipse.orion.server.core/.settings, none of the prefs files are used anymore

Start the server.
Comment 3 John Arthorne CLA 2013-09-06 16:23:42 EDT
Cool! Let's try a test migration of orion.eclipse.org next week.
Comment 4 Anthony Hunter CLA 2013-09-16 16:44:30 EDT
With commit 
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=bd438ff0160cf0757eb16be1cfd4654178b34101

I have completed the migration to the fourth draft of the design on Bug 412995 .
Comment 5 Anthony Hunter CLA 2013-10-03 16:21:45 EDT
This delivery completes the feature:
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=5d15dfdd411abce1afc8f9514b2afc6aea79f2ef

The migration is now done live on server startup.

The order of operations is:

1) backup the server, since the project data is renamed and moved, the largest change being the project data moved from /serverworkspace/A/A/B to /serverworkspace/an/anthony/Orion Content/Project Two/

2) Add the property to the orion.conf
orion.core.metastore=simple

3) Start the server.

The orion log will indicate that migration has started and completed, as well as indicating the migration log location, .metadata/migration.log