Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 238334 - There seems to be no deleted content in cvs
Summary: There seems to be no deleted content in cvs
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: CVS (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.5 M1   Edit
Assignee: Tomasz Zarna CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-06-24 16:45 EDT by David Williams CLA
Modified: 2009-06-02 04:56 EDT (History)
2 users (show)

See Also:


Attachments
Screen shot (36.40 KB, image/png)
2008-07-03 12:02 EDT, Tomasz Zarna CLA
no flags Details
msg I get :( (7.56 KB, image/gif)
2008-07-03 15:03 EDT, David Williams CLA
no flags Details
info printed to console when cvs debug (5.74 KB, text/plain)
2008-07-03 15:08 EDT, David Williams CLA
no flags Details
config from my "JEE IDE" version (2.59 MB, text/plain)
2008-07-03 15:16 EDT, David Williams CLA
no flags Details
My log for "Restore for Repository" (5.83 KB, text/plain)
2008-07-24 09:37 EDT, Tomasz Zarna CLA
no flags Details
Fix v01 (2.90 KB, patch)
2008-07-28 07:01 EDT, Tomasz Zarna CLA
no flags Details | Diff
mylyn/context/zip (72.02 KB, application/octet-stream)
2008-07-28 07:02 EDT, Tomasz Zarna CLA
no flags Details
Fix v02 (3.40 KB, patch)
2008-07-28 22:20 EDT, David Williams CLA
tomasz.zarna: iplog+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2008-06-24 16:45:27 EDT
There is functions in Eclipse that say "restore from repository". 

And ... today, at least, ... nothing ever shows up there. 

I was going to open this under Eclipse Platform Team component, but I tried a 3.3 Eclipse client (as well as Eclipse 3.4) with same result. 

In addition to testing projects which I _know_ have a long history of deleted content, I tried just deleting something, "committing" the change, and then tried ... still, nothing shows up under "restore from repository". 

I'm not exactly sure what "raw" cvs command this maps to, but assume it's something common and well known? 

I'm wondering if related to the servers being "locked down" today? 

I marked as "critical" since this is effectively "lost data".
Comment 1 Denis Roy CLA 2008-06-26 13:00:13 EDT
> I marked as "critical" since this is effectively "lost data".

I'm not accepting that. Can you tell me what/where I should be looking at in CVS?
Comment 2 David Williams CLA 2008-06-26 13:38:49 EDT
(In reply to comment #1)
> Can you tell me what/where I should be looking at in CVS?
> 

I'll pick one module that I know has content that has been deleted (from HEAD). 

webtools.maps/releng

In the /cvsroot/webtools repository

And, as I mentioned, I could "recreate" the problem on any module. 

Let's be clear on what I'm saying, though ... 

The content (a few files in this case) has been purposely deleted from HEAD. 

In the past, I could simply say, with Eclipse, Team, Restore from Repository ... and it would give me a list of things that had been deleted which I could then restore. 

So, doesn't give me that list any longer. That's what I meant my "_effectively_ missing data". It might still be there in CVS (I haven't checked) but not visible to Eclipse, for some reason. 

Comment 3 David Williams CLA 2008-06-27 13:10:45 EDT
I wonder if related to the way our modules are defined in the MODULES file? 



Another observation ... 


cvs -q -d david_williams@dev.eclipse.org:/cvsroot/webtools rlog -N webtools.maps/releng/maps/tempMap.map 


certainly seems to "find" all the "deleted" revisions. The beginning of that output says 

RCS file: /cvsroot/webtools/webtools.maps/releng/maps/Attic/tempMap.map,v
head: 1.18
branch:
locks: strict
access list:
keyword substitution: kv
total revisions: 18;    selected revisions: 18
description:
----------------------------
revision 1.18
date: 2005-11-05 14:48:18 +0000;  author: david_williams;  state: dead;  lines: +0 -0;
prep for new features -- improved map file groupings to be accurate
----------------------------
revision 1.17
date: 2005-11-05 02:40:49 +0000;  author: ledunnel;  state: Exp;  lines: +2 -2;
[115177]   Exceptions thrown when opening Database Explorer View related to loading images
----------------------------





Comment 4 David Williams CLA 2008-06-27 13:11:11 EDT
Another observation .... 

when I turn on verbose-output-for-debugging in Eclipse, it seems to get "close" to discovering deleted revisions ... it seems them in "Attic" ... not sure why it doesn't think any of these qualify, but here's the console output from running the 'restore' command in eclipse, against the releng project (which ends by telling me there is no deleted files in the project). ... 


***
cvs log -R "/releng"
    /cvsroot/webtools/webtools.maps/releng/.project,v
    /cvsroot/webtools/webtools.maps/releng/Attic/archived.txt,v
    /cvsroot/webtools/webtools.maps/releng/.settings/org.eclipse.core.resources.prefs,v
    /cvsroot/webtools/webtools.maps/releng/.settings/org.eclipse.core.runtime.prefs,v
    /cvsroot/webtools/webtools.maps/releng/.settings/org.eclipse.ltk.core.refactoring.prefs,v
    /cvsroot/webtools/webtools.maps/releng/.settings/org.eclipse.wst.html.core.prefs,v
    /cvsroot/webtools/webtools.maps/releng/.settings/org.eclipse.wst.validation.prefs,v
    /cvsroot/webtools/webtools.maps/releng/maps/build.cfg,v
    /cvsroot/webtools/webtools.maps/releng/maps/dependencies.properties,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/isv-doc.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/isv-docs.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/jst-doc.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/jst.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-I200705130851.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-I200705152217.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-I200705160545.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-R200706122135.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-R200706131308.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-R200706192011.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705021947.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705221744.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705222254.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705232222.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705240148.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705242235.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200705301823.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/orbitBundles-S200706081908.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/orbitBundles.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/tempMap.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-common.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-doc.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-server-componenet-features.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-server.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-web-thirdparty.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-web.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-webresources-tests.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-webresources.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-ws-thirdparty.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-ws.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wst-xml.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/wst.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/wtp-feature-jst-doc.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wtp-feature-jst.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/wtp-feature-wst-doc.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wtp-feature-wst.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/Attic/wtp-feature.map,v
    /cvsroot/webtools/webtools.maps/releng/maps/wtp-releng.map,v
ok (took 0:00.312)
***
Comment 5 David Williams CLA 2008-06-27 13:13:12 EDT
Since the data is there, but unaccessible via Eclipse function, I'll mark as "major", since it is a major missing function. 

I'll move over to Platform, Team ... as soon as "we" look at MODULES file to make sure it's not screwy somehow. 

Comment 6 Denis Roy CLA 2008-07-02 14:36:16 EDT
(In reply to comment #5)
> I'll move over to Platform, Team ... as soon as "we" look at MODULES file to
> make sure it's not screwy somehow. 

Who is 'we' ?  In the meantime, this 'major' bug is opened against our CVS servers, whereas in comment 3 your CVS client seems to find everything that is missing.
Comment 7 David Williams CLA 2008-07-03 00:12:26 EDT
(Sorry for the delay, Denis ... just wanted to investigate a few cases before moving ... thanks for you help). 

Moving to Platform/Team/workbench component. 

I at first thought this was an "infrastructure" problem, because even "moving back" to previous releases of Eclipse would not work (and, I _know_ I've used this function before, with those previous releases). 

But, I did just obtain another 'data point' that indicates an Eclipse 3.4 problem, and implies some sort of "workspace damage" takes place that causes current and even previous versions to not work with respect to "restore from repository".

As mentioned, I tried with the already mentioned projects on Eclipse SDK 3.3, and 3.2 (in addition to 3.4) and none of them worked (all of them said "nothing has been deleted from the project" when in fact it had been). 

So, then wanted to try other projects, other than WTP ones, so fired up an "old" workspace I had laying around (seldom used) that had the base Eclipse Platform in its workspace, and was running Eclipse 3.3. There I saw that a project I just happened to pick, org.eclipse.equinox, had "deleted content" show up when I did "restore from repository". So, to check current release against that project, I started up the 3.4 release, with that old workspace ... no function (nothing showed up under "restore from repository" ... just got a message "nothing has been deleted from the project"). 

So, _then_ I started up Eclipse 3.3 _again_ with that same old workspace, and _now_ the function no longer works! This makes me thing 3.4 is somehow "damaging" the workspace so that this function does not work. 

Hence, moving back to "critical", since ... from what I can tell ... it is "losing data" in that it is damaging a workspace. (Again, in all cases the data is not _really_ lost ... but, becomes unretrievable using Eclipse, and will force users to use some other cvs client to retrieve content that's been "deleted" from a branch. 

Hope all this makes sense and others can reproduce easily. 

I suggest to reproduce the basic problem, you should be able to use any project in your workspace (that has deleted content) and, assuming you see the same problem, go from there. 

Comment 8 Szymon Brandys CLA 2008-07-03 06:43:13 EDT
Tomasz, could you comment on it please.
Comment 9 Tomasz Zarna CLA 2008-07-03 12:02:42 EDT
Created attachment 106460 [details]
Screen shot

Works fine to me. I'm able to see some deleted files in the webtools.maps project mentioned in comment 2. It looks fine when called for a fresh project shared in a local CVS repository, ie can restore a deleted file. Am I missing something? Is there something special about your configuration David? Any exceptions in the Error Log?
Comment 10 David Williams CLA 2008-07-03 15:03:37 EDT
Created attachment 106487 [details]
msg I get :( 

Nothing special about my configuration. 
No messages in the log. 
(but, I'll attach the full CVS debug output). 
The dialog you attached is what I expect to see, but see only the one I attached.
Comment 11 David Williams CLA 2008-07-03 15:08:19 EDT
Created attachment 106492 [details]
info printed to console when cvs debug 

output I get when I turn "on" the CVS preference that says "Display detailed output to stdout (for debugging purposes)". 

I find it odd that this output seems to be listing the deleted files! 

Is your debug output the same?
Comment 12 David Williams CLA 2008-07-03 15:16:44 EDT
Created attachment 106498 [details]
config from my "JEE IDE" version 

I've seen the same thing if I try to use only "Eclipse SDK", so think it's more workspace releated, than dev. env. 

Perhaps I'll try re-creating my workspace? 

Any particular files in .metadata you think I could remove/tweak/attach to bug?
Comment 13 David Williams CLA 2008-07-03 21:15:54 EDT
I've tried a few experiments (fresh workspace, fresh platform install, etc.) and nothing worked, until .... 

I found that if I change a project's cvs "location" to 

:pserver:anonymous@dev.eclipse.org:/cvsroot/webtools

then it works as expected. If I use/change it to 

:extssh:david_williams@dev.eclipse.org:/cvsroot/webtools

then it does not work. This latter one is, of course, the one I normally use, and need to use to commit code. And, to be explicit, I _can_ commit code using that location :) can ssh to david_williams@dev.eclipse.org, etc., so the account is good. 

I sure hope this isn't because I have an underscore in my userid :) 

(I know it's rare, at Eclipse! but, has worked before :) 

HTH
Comment 14 David Williams CLA 2008-07-03 21:33:40 EDT
changing to major since there is at least a work-around path that can be used to restore data ... not very practical ... but, I personally don't use that often. 
Comment 15 Tomasz Zarna CLA 2008-07-24 09:37:50 EDT
Created attachment 108342 [details]
My log for "Restore for Repository"

(In reply to comment #11)
> I find it odd that this output seems to be listing the deleted files!

I find it odd too. Have you noticed any differences in logs between pserver and extssh? I don't have write access to webtools so can't check the way you did.

> Is your debug output the same?

Yep, it's pretty much the same. Differences between "Valid-requests" and "Valid-responses" are probably related to CVS version. The "E" messages I have don't seem to significant either. Maybe it's the "I LOVE YOU"? at the top ;)
Comment 16 David Williams CLA 2008-07-26 01:16:53 EDT
I've fired up a second instance of Eclipse, set breakpoints, and tracked this down a little ... maybe these "clues" can help you, either directly, or help you suggest other debugging tricks I can do. 

Turns out those E codes are significant. 

The relevant code comes down to AtticLogListener. In particular, the errorLine method, and the messageLine method. 

When I've set my cvs location to use extssh, I never get an E line, so errorLine is never called, so currentFolder always remains null, so even the "normal" message lines are not accumulated (due to the 'if (currentFolder != null)' check in messageLine. 

When the cvs location is set to use pserver, I get 'E lines' for every directory (in attic, I guess), starting with the directory '.' (so, there is always at least one, before any other 'M lines", and it is (only) in the errorLine method the currentFolder is set, hence when messageLine is finally called, repeatedly, then all the "deleted files" _are_ accumulated, as intended. 

So, I guess the 'E lines' are not (always) authentication errors or something, but can be an "error" I guess meaning something like "no versions are kept of folders". 

So, why are E lines sent in one case, but not another? 
Could this be related to a change in infrastructure? 
Or, just something special about my setup :) 

Are there more direct ways that 'currentFolder' could be set? I'd guess so, but may be fragile ... that is, break other use cases? 

 

 
Comment 17 David Williams CLA 2008-07-26 04:22:03 EDT
I have discovered what changed "in the infrastructure" to cause this ... so to speak. 

In my home directory at eclipse.org, I have a .cvsrc file to control some "common" cvs settings. I probably wasn't using it last year. 

The only thing I had set there was 
cvs -q 
for "quiet" operation ... so, no "error" messages (E messages) generated for "directories only". If I remove that .cvsrc file, or that -Q option, then I can restore files just fine in extssh mode. 

So, options are 1) close this as invalid, 2) close as invalid but document in a FAQ or help file that the hosts .cvsrc file can effect results if extssh is being used (probably obvious to experts?). Or ... 3) find some better way to do the algorithm so that it does not require a "verbose" mode setting on the host? 
Or 4) maybe, at least, log an error message in Eclipse (once per session) such that if "currentDirectory" was null when 'messageLine' was first called, that the user could get a message in the log, at least, warning of "unexpected sequence of responses from host cvs protocol ... perhaps for your host user ID the .cvsrc file is effecting the results". 

Thanks for your help. 

Comment 18 Tomasz Zarna CLA 2008-07-28 06:50:40 EDT
After reading your last 2 comments, I must say it does look like an invalid bug to me. However, having in mind you spent some time investigating what's going on here I think simply marking it as INVALID is not an option here. Writing documentation/faq is not my favourite way of spending time either :) I guess I will pick gate number 3, writing a patch. With the clues you gave me it should be that hard.
Comment 19 Tomasz Zarna CLA 2008-07-28 07:01:55 EDT
Created attachment 108531 [details]
Fix v01

Here it is. David would you mind checking if it solves your problem? I know you've already done much to help me with it, but double checking it on your side (with your configuration) would be great.
Comment 20 Tomasz Zarna CLA 2008-07-28 07:02:11 EDT
Created attachment 108532 [details]
mylyn/context/zip
Comment 21 David Williams CLA 2008-07-28 10:37:26 EDT
It didn't quite work. 

The files were indeed accumulated, and I got the dialog, which asks me which files and revisions I want to select to be restored, but ... 

The "Attic" directories actually appear in the list, which of course, they shouldn't since that's CVS's own internal construct. All the files to restore appear under an Attic direcotry. 

Also, if I try to select one of the files in that dialog (under an Attic directory), then I just get a null pointer exception immediately (even before the revisions of the selection are displayed). This is probably since a file with that name (with 'Attic' in the path segments) is not found. 

[sorry, not more time to look at the actual patch right now ... thought I'd post what I know.]
Comment 22 David Williams CLA 2008-07-28 22:20:07 EDT
Created attachment 108591 [details]
Fix v02


I've confirmed that just stripping out a trailing "Attic/" from the folder path makes everything work.
Comment 23 Tomasz Zarna CLA 2008-07-30 09:50:14 EDT
Thanks for you help David, I really appreciate it. The latest patch has been released to HEAD.
Comment 24 David Williams CLA 2008-07-30 10:35:08 EDT
I almost don't want to say this after you've been so kind to fix this one case ... but, I have also noticed that even "compare with history" doesn't work, if extssh is used, and the user's .cvsrc files specifies "quiet" mode. 

I'm not sure worth fixing, since apparently it's been this way for years, and no one else has complained? Seems rare that people would have .cvsrc set to quiet, with the same id they use to connect to cvs from their dev. evn. 

But, I at least wanted to mention it, in case others run into it. 

Just if you are interested, I did figure out how me myself have only seen this recently ... I only use the .cvsrc file on by "build machines" doing production builds (to avoid too much output) and most of these are done with a different build-ID, but there was a short time a few months ago I had to do some with my "real" eclipse id so I must have set it then. And, I'll add, I've long thought no one should ever use such "hidden" options, and this experience just proves the point. 

thanks again.