Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 360503 - Alternate storage for jobs history
Summary: Alternate storage for jobs history
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: gyrex (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Gunnar Wagenknecht CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 391401
Blocks:
  Show dependency tree
 
Reported: 2011-10-11 03:41 EDT by Gunnar Wagenknecht CLA
Modified: 2018-03-19 11:59 EDT (History)
1 user (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Gunnar Wagenknecht CLA 2011-10-11 03:41:34 EDT 
The jobs history is currently stored together with jobs definition in ZooKeeper using cloud preferences. This seems to be a scalability issue especially with  growing number of jobs executed repeatably.

We should look at abstracting the job history storage so that alternate implementations could be provided.
Comment 1 Gunnar Wagenknecht CLA 2011-10-11 03:58:29 EDT 
As part of this we should also investigate if moving job definitions out of the preferences into the alternate storage is also an option. 

Another question that needs to be answered is whether scheduling and execution coordination should stay in ZooKeeper or if the alternate storage provides atomic ways of dealing with job states. FWIW, the ephemeral nodes in ZooKeeper are great for detected failing workers and lost jobs. However, it's not trivial to manage states in parallel with an underlying store. One system should be preferred.
Comment 2 Gunnar Wagenknecht CLA 2013-02-06 09:55:00 EST 
FWIW, the commit initially targeted at this bug actually fixed bug 391401.
http://git.eclipse.org/c/gyrex/gyrex-platform.git/commit/?id=6421cb3d15a5a8948d5aa2a58600e72603727357

Now we have to think about what we should do with this one.
Comment 3 Gunnar Wagenknecht CLA 2013-02-07 16:21:48 EST 
No alternate storage will be provided in Gyrex. The storage provider interface created in bug 391401 is all for now. It allows to plug-in custom storage providers.