Bugzilla – Full Text Bug Listing
| Summary: | [plan item] Support workspace checkpoint and rollback | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Jim des Rivieres <jeem> | ||||||
| Component: | Resources | Assignee: | DJ Houghton <dj.houghton> | ||||||
| Status: | RESOLVED WONTFIX | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | P4 | CC: | andre_weinand, brian.schultheiss, chris, daniel_megert, eclipse-bugs, eclipse, erich_gamma, fsauer, gunnar, j.c.yip, john.arthorne, nikolaymetchev, sheldon.robinson | ||||||
| Version: | 2.1 | Keywords: | plan | ||||||
| Target Milestone: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| Whiteboard: | |||||||||
| Attachments: |
|
||||||||
|
Description
Jim des Rivieres
Erich points out that this is not the only way to address the exploration scenario. Another approach is to extend the local history to include folders and allow to tag the local history entry with a user understandable tag (e.g. "refactoring: renamed foo"). The local history based approach has the benefit that you can rollback more than one transaction. If we take this appropach, we could also add support for correlating the local history with version activities, so that you can see in the local history when a version operation happened. I've reworded this plan item to make it clear that a transaction based approach is only one possibility: Support workspace checkpoint and rollback. Sometimes a set of resource changes must be explored to see if they are possible or worthwhile. When the changes do not pan out, there needs to be a way to roll back the workspace to the prior state. Eclipse should support something like long workspace transactions with the ability to checkpoint and rollback. The facility should be available to users and programmatically (e.g., JDT refactoring). [Platform UI, Platform Core, JDT Core, JDT UI] [Theme: User experience] Please assign this bug to me for exploration. After discussions with John W, John A, and Jeff M, I propose that we keep a number of "checkpoints" that are created manually or automatically. Each checkpoint is just a label and a timestamp (last modified as per local history). The label is definable by UI or API. When it is desired to roll back, the contents of local history are restored back to the checkpoint time. Since there may be a mixture of "keeper" and "discard" work since the last checkpoint, we should consider using the synchronizer view UI to decide which changes are rolled back. This is just like CVS, FTP, and WebDAV. API rollback probably just rolls everything back to the checkpoint, but perhaps we need a way to specify subsets of local history to restore (if you think so, please indicate). There are some issues with observable and unobservable data loss that must be considered. Local history has a file size limit, over which files are not saved. A maximum number of versions of a file are kept; and the history is limited in total size. Incomplete restore sets could result from garbage collection of old/large local history files, for which the user would be unaware of a lossy rollback. Flags on resources can specify that local history should not be kept; such resources will not be able to be restored. Finally, this proposal only addresses resources, not meta data such as settings. I have implemented the basic file storage and UI for adding and selecting
checkpoints. Some issues arose that I would like comments for, before starting
the Sychronizer work (daunting).
1. As the local history is gargage collected, it deletes older and larger files
first. Do we want to change this model with Checkpoints such that we attempt to
keep file states that associate with specific checkpoints, preferentially over
states in-between checkpoints? You can imagine that several versions (states) of
a file are kept in local history between checkpoints. For rollback, the only
important states are the ones that occur just after a checkpoint because that
state represents the "undo" needed to rollback the file. We could "compress" all
states between. I think in practice all this means is keeping any state that
occurs just after a checkpoint in preference to the next younger state. So,
instead of collecting in this order: { n, n-1, n-2, ..., x, x-1, x-2, ... 1 },
we keep 'n' and 'x' and collect in this order: { n-1, n-2, ..., x-1, x-2, ...
1}, where 'n' and 'x' are states with timestamps most proximal and greater than
corresponding checkpoints N and X. This seems more likely to behave in
human-friendly ways, but I don't know what other ramifications it may have.
2. As an implementation detail, I have created a new class that lives in
org.eclipse.core.resources that just serves to make API for a few calls needed
to the HistoryStore instance obtained via the real Workspace class since
IWorkspace does not provide access to it. If I understand correctly, the API for
HistoryStore is presented via IContainer and IResource on a per-resource basis.
Since Checkpoints want to know about everything in the workspace, we need a
wider interface. I'll include a patch of current ideas if needed, but I'm just
curious if I'm on the right track.
Thanks for your comments. Cheers, Chris
Chris, it sounds like you're making good progress. A couple of answers/comments... 1. I agree the local history store will want to be "checkpoint aware", so that it can be smart about what old states get cleaned up. Note that the current state of most files is not stored in local history, we just store old versions as they are replaced by new content or deleted. I see two general approaches to implementing checkpoint on top of local history: a) When a checkpoint occurs, copy the current contents of *all* files in the checkpointed project into the local history, and mark them with some checkpoint tag. b) After a checkpoint occurs, lazily mark the first version of each file copied to the local history after that point. Rollback would then just revert the files that have changed. The disadvantage of b) is that there are a number of ways that a file can change in which the old state will not be recorded in local history. If a file is changed directly in the file system after the checkpoint for example, there will be no way to roll back that file. All of our API for modifying and deleting files provide a flag for whether local history should be recorded, so these can cause other old checkpointed states to be lost. Of course the disadvantage of a) is that it makes the checkpoint operation much slower, and requires much more local history space. I think I prefer b), although it would require some user education that checkpoint and rollback are not a real substitute for a versioning system (i.,e there are conditions under which it can be lossy). 2. That sounds fine for now. I imagine if this becomes real then it will be tightly integrated into the resources plug-in anyway, so new API is certainly an option. John, thank you very much for your thoughts. I believe that I understand your options a) and b) as storage strategies. My proposals have all been aimed at b) for the very reasons you site - space and speed. And I agree with the limitations you mention as well as others sited in my first comment in this PR. My principal concern is that "user education" is a tough road. We will raise expectations with "rollback" being as strong as database functionality. Still, I prefer b). Over time, I think we should merge all of the UI for local history/replace/checkpoints/etc. to present a clear model. I don't have a proposal for that yet, though. I'd like to prototype the checkpoint/rollback first and see how it feels. However, all ideas are welcomed at any stage in the game. Cheers, Chris I think that the more robust implementation choice a) is better from the Users point of view. If b) is chosen, education will be difficult, definitely. Plus there will almost certainly be follow-up bug reports/RFEs that want it to behave like a). I think it would be easier to educate about the cost of a) since it can be presented at the time the user invokes the feature without a lot of complicated wording that a similar explanation of the limitations of b) would require. IOW, practice the Principle of Least Surprise - b) is more surprising to the average user, IMHO, so choose a) instead. Tangent thought: perhaps the trickiness of this issue (a vs. b) is a sign that the overall strategy of coupling tightly to the existing Local History framework should be reconsidered. I am by no means familiar with the underlying code or Eclipse architecture (and thus am not qualified to make design decisions), but whenever I see a design choice stir up such tricky logical/symantic questions as this, I always go back and re-evaluate, looking for cleaner & simpler design alternatives. I should qualify that I preferred b) given the assumption that local history store was being used. My real preference is to not implement this at all, since this is functionality already provided by repositories such as CVS. If we go down the checkpoint/rollback path, and want to give satisfying user-level semantics, we'll end up having to implement a full blown repository anyway. However, it depends on how this is exposed to the end user. If it is just used to implement "undo" for refactorings (i.e., in a highly controlled manner), then I think the semantics of b) are acceptable. This is a good point. The intent for pursuing this feature was to make use of "undo" easier across multiple file changes. I agree that a robust implememtation is redundant. The question is wether this feature makes sense in it's less-than-perfect implementation. However, I like the idea of using it primarily for programmatic rollback, e.g. with refactoring. One more comment before moving on with it. The storage strategy a) has a serious space penatly, because checkpoint/rollback is scoped across the whole workspace. The user does not choose a scope (e.g. project). A major advantage is to provide "undo" across projects without having to think first "what I am going to change?", since refactoring, e.g., can span several projects. Requesting comments on a few more issues, please. If I understand correctly, Local History does not maintain deleted projects. If not, is the reason for space? If a project is accidently deleted, is there a way back? Rollback (or as DarinS suggested, "Fallback") will need to restore deleted projects. I think we should save deleted projects, but perhaps we should do something different for projects that are under repository control? This might get messy, though. I'd like to increase the default local history size (not file count) and cache deleted projects. Thoughts? I claim that "rollback/fallback" should delete resources and projects that are newer than the "checkpoint" time. I am presently determining this as all resources that have local timestamps greater than the checkpoint time and are not present in local history. This should be all the "new" stuff. Does the name "fallback" have any bad/misleading meanings internationaly that anyone is aware of? Thanks, Chris You are correct that local history does not maintain deleted projects. This is
part of a general resources plugin policy to discard all state on project
deletion, largely because most project metadata is stored on a per-plugin basis.
It is also a performance issue, in both speed and space. Deletion performance in
the workspace is generally terrible because we have to copy everything into the
local history first... we wanted to avoid this hit in the project deletion case.
If we consider changing this, it would be preferable to only have project
history saved if a checkpoint exists.
I agree that rollback should delete resources created since the checkpoint (of
course putting them in the local history).
Do you have any ideas for how to keep this sustainable in a large workspace?
Keep in mind the user who has 50,000 resources in their workspace, never shuts
the workspace down, checkpoints frequently, and never rolls back. Is there a
limit on the number of checkpoints? (in fact a limit of 1 user defined
checkpoint might be sufficient, with additional checkpoints for programmatic
clients). Will there be a way to discard checkpoints? The local history policy
settings would need to be expanded to incorporate a flexible set of checkpoint
policies.
> increase the default local history size
Which setting do you want to increase? Note the "max file state size" setting
means a limit on the size of individual files that are stored, not a limit on
the sum of the file sizes stored.
John, good comments. This will be a tricky performance tradeoff. The user probably doesn't want to worry about checkpoints until it's too late. The idea is late-binding decision making. In regards to the number of checkpoints, my current implementation creates one on every Eclipse boot and then also allows any number of user-defined checkpoints (via an action/dialog). Programmatic checkpoints would probably need fewer, but the issue isn't space or speed for checkoint storage since they are just labels with timestamps. On project deletion, it makes a lot of sense what you said. Project deletion is a major decision, but also one that would be very painful if accidental. However, we don't want to punish the smart user who has a repository and is just cleaning house to start fresh. Perhaps this is worth the effort of copying a project into local history IFF there is a project came from a repository (is that possible to know?). If yes, then maybe we can store just the resources that are out of sync with the repository - it seems the sync view has this information. Is that getting too complex? On Local History size, I was confused. I see that there is no limit by sum. So, I think the basic settings are ok. On checkpoint cleanup, I have suggested that we GC the checkpoints as the local history files are GC'd themselves. So, as the checkpoints become invalid due to lack of content, we GC the checkpoint. This will require, I think, a new GC policy on the Local History (see comment #3 for proposal). On checkpoint policy, I haven't though about what settings are needed. Perhaps some basic ones such as boolean:checkpoint-on-startup. And maybe we want to allow control over wether deleted projects are stored in local history. I don't want to interrupt this excellent discussion, but wanted to mention a checkpoint policy feature. It would be nice to have a checkpoint when the tests are green so that I can compare my state later with this state and rollback if needed. Eric, that sounds like a good idea. The "take a checkpoint" API is open to anyone in the platform, so JUnit could just make a fast call to this API when a test suite passes or offer a dialogue, etc. Another issue that Michael Valenta brought up is that we can not know if a resource is in or out of sync with it's repository (at least with any planned API). Also, does the user expect a rollback to replace shared projects and resources from repositories? That could get very involved and messy. Changes made to shared projects will be stored in local history as usual, but if a project is deleted, there will only be a partial set of resouces available to restore. For now, I will proceed WITHOUT copying projects to the local history when they are deleted. This means that new projects can not restored after they are deleted. This is consistent with the current local history manager. I hope we can devise a way to handle this better in the future. Comments are still welcomed, but I will proceed to putting some UI on the "fallback" action, pending the new team API submission. The "checkpoint after successful test run" idea is nice, but make sure it is optional. I tend to run JUnit tests/suites very frequently as I am first writing them and I would not want 50 checkpoints created for me one day as I was building out a group of tests. In fact, I would think it better as an option whose default was OFF so as not to put undue burden on the storage/performance. *** Bug 49388 has been marked as a duplicate of this bug. *** Created attachment 8368 [details]
Patch to org.eclipse.compare
Provides the base for checkpointing, including a checkpoint storage mechanism
and integration with the current history manager. Garbage collection is driven
by the history manager.
Missing is the restore operation, which wants to be driven by the new
synchronizer API. That new API is not available yet and so I am waiting.
Created attachment 8369 [details]
patch to org.eclipse.core.resources
OK, so the previous patch was actually some UI test code to drive the
checkpoint code in core. This patch is the real stuff that goes in core and
should have the comments from above. Sorry.
Please also note that the patch to compare does not correctly add the actions to the menubar. Three of us worked on this, including a member of the UI team, and we could not figure out why the actions are not added. It does correctly add some sub-menu items in the context menu, however. This is not intended to be the final story for Checkpoint UI; it's just a way to drive the stuff underneath until the new synchronizer API is ready. Thanks for the patches, Chris. This feature will not make it into Eclipse 3.0 but we will evaluate it for the next release. Is this still planned? 3.1 perhaps? This item is not currently on the 3.1 release plan. *** Bug 30759 has been marked as a duplicate of this bug. *** *** Bug 72933 has been marked as a duplicate of this bug. *** *** Bug 121589 has been marked as a duplicate of this bug. *** [LATER->WONTFIX] The "LATER" bugzilla resolution is being removed so reopening to mark as WONTFIX. [LATER->WONTFIX] The "LATER" bugzilla resolution is being removed so reopening to mark as WONTFIX. |