Community
Participate
Working Groups
As of Orion 2.0, the server metadata is stored in Equinox preferences. There are four preference trees for the various types of metadata: Users, Workspaces, Projects, and Sites. These are currently persisted in four files, one per preference tree. There are a number of problems with this implementation: 1) Storing large amounts of metadata in a single file that needs to be read on most requests is a severe bottleneck. 2) Migration of metadata format across builds/releases is ad-hoc and error prone 3) Individual preference node access/storage is thread safe, but many Orion API calls require multiple read/writes to multiple nodes. There is no way to synchronize access/update across nodes. 4) Metadata file format does not lend itself to administrative tools for managing users, migrating, etc. 5) There is no clean API to the backing store and the interaction between the request handling services and the backing store is tangled. We need a clean backing store API, and a scalable implementation of that API. The actual backing storage should be pluggable, for example raw files, database. Interaction with the API needs to be performant and transactional. There needs to be an easy way to administer the metadata, for example deleting inactive accounts, migrating account data, etc.
Current state: I have defined a new API for metadata storage in Orion: IMetaStore. I have an implementation of this API that stores metadata in the current Orion format, to support compatibility with existing Orion installs. This work is found in a branch but I plan to move it into master next week. http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/log/?h=johna/metastore
This work has been released in master. There is now a simple metadata API, called IMetaStore, which encapsulates all metadata persistence on the server. It consists of 12 methods: create/read/update/delete for users, workspaces, and projects. All of the Orion implementation has been migrated to use this new API. I have left the old code in place in deprecated form to give a transition period, but I aim to delete them in the future (WebUser, WebProject, WebWorkspace, etc). Anybody using that old API will not work with any alternate back end metadata store that gets plugged in. There are still some minor details that I am not happy about, such as the lifecycle of the IMetaStore service and how clients obtain it, but I will open new bugs for any follow-on work.
Any changes to the internal format (file content wise) used by the default implementation, John? IOW, is the metadata format on disk still the same?
Metadata on disk is still the same. I have a "compatibility" implementation of the API that writes to the current disk format. The intent is to change that though, in particular breaking up the metadata to be stored separately per user. I'm not sure if that will happen for 3.0 though, because there are a lot of details about how to handle user data migration on orionhub, etc. My current estimate is we will have the existing file format for one last release (3.0 in June), and then migrate to something new immediately after that.
The follow on work for Orion 4.0 is tracked at Bug 412995 .
