Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 333458 - CVS for pserver and extssh occasionally out of sync for several hours.
Summary: CVS for pserver and extssh occasionally out of sync for several hours.
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: CVS (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 335963 338040 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-01-04 06:30 EST by Tomasz Zarna CLA
Modified: 2011-08-12 14:09 EDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tomasz Zarna CLA 2011-01-04 06:30:15 EST
Yesterday evening (CET) I had committed a fix for bug 315694. This morning I tried to sync on a different machine via pserver, but the change wasn't there. I assume that usually sync'ing between pserver and extssh servers takes less time, so I'm filing this bug.
Comment 1 Tomasz Zarna CLA 2011-01-04 06:31:19 EST
Szymon has noticed the same thing with a change he made yesterday in *.core.resources.
Comment 2 Eclipse Webmaster CLA 2011-01-04 10:12:05 EST
Ok, looks like the sync got stuck.  I've restarted it and it seems happy.  Can you confirm that your changes have been copied?

-M.
Comment 3 Tomasz Zarna CLA 2011-01-04 12:43:39 EST
Yep, I can see them now.
Comment 4 Denis Roy CLA 2011-03-01 13:39:02 EST
*** Bug 338040 has been marked as a duplicate of this bug. ***
Comment 5 Denis Roy CLA 2011-03-01 13:42:38 EST
These bugs will keep cropping up occasionally until we can find a better way to sync pserver and ssh data.  Right now, I don't have any actionable ideas.
Comment 6 Denis Roy CLA 2011-03-10 14:34:57 EST
*** Bug 335963 has been marked as a duplicate of this bug. ***
Comment 7 Cole Markham CLA 2011-04-12 16:10:51 EDT
This is a bigger issue than changes being there or not. I'm assuming due to the way CVS works that changes to multiple files are not atomic. Therefore what should be a single changeset might be partially out of sync. Worse than that I am seeing files that have changed recently completely missing from the pserver server. This makes it near impossible for a non-committer to track a project with any sort of active development. I have run into this twice today and it is very frustrating.

An example if you look quickly:
http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.rap/runtime.rwt/org.eclipse.rap.rwt.q07/js/org/eclipse/rwt/?root=RT_Project
There should be a file called Border.js which was there in previous versions, but is now nowhere to be found for any version.
Comment 8 Denis Roy CLA 2011-04-12 16:35:20 EDT
> This is a bigger issue than changes being there or not. I'm assuming due to the
> way CVS works that changes to multiple files are not atomic.

This is correct.

> Therefore what > should be a single changeset might be partially out of sync. 

It *could*; however, the cvs syncer waits for commit activity to cease before launching a sync.

> There should be a file called Border.js which was there in previous versions,
> but is now nowhere to be found for any version.

Border.js is a new file that was created mere hours ago (actually, two minutes prior to the committer's comment on bug 341924), and it only contains the initial commit.

Now with that being said, it would appear the cvs syncer is picking up file updates, but not file additions -- which, for the first time, actually gives me something repeatable I can work with.  I'll investigate and see why the syncer is missing out on new files.
Comment 9 Cole Markham CLA 2011-04-12 16:40:18 EDT
(In reply to comment #8)

> Border.js is a new file that was created mere hours ago (actually, two minutes
> prior to the committer's comment on bug 341924), and it only contains the
> initial commit.

Sorry, I thought Border.js was an existing file. However the file I was having problems with earlier was this one, which definitely has history but was missing completely earlier today: http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.rap/runtime.rwt/org.eclipse.rap.rwt/src/org/eclipse/swt/internal/graphics/ColorUtil.java?root=RT_Project&view=log
 
> Now with that being said, it would appear the cvs syncer is picking up file
> updates, but not file additions -- which, for the first time, actually gives me
> something repeatable I can work with.  I'll investigate and see why the syncer
> is missing out on new files.
Thanks for taking a loook.
Comment 10 Denis Roy CLA 2011-04-14 10:23:49 EDT
> However the file I was having
> problems with earlier was this one, which definitely has history but was
> missing completely earlier today:

As much as I want to believe that what you're saying is accurate, I have a hard time understanding how files can go missing, to only reappear later, unless it's a result of a back file lock or something similar.  If that were the case, I would have thought the problem would be more widespread than it is.

If you can, please attach a screenshot of this if you happen to catch something.



> Now with that being said, it would appear the cvs syncer is picking up file
> updates, but not file additions

I'm getting nowhere fast in trying to replicate this.  I've created new files, and new files under several new subdirectory levels -- they all get replicated within the prescribed delay of a few seconds.
Comment 11 Denis Roy CLA 2011-04-14 13:39:21 EDT
FWIW, I'm also looking into upgrading our rsync package on the backend servers.  The old one that's there now does not support incremental file lists, so it's rather inefficient.  I'm thinking that with a more efficient rsync, I'll be able to do complete syncs more frequently and will less overhead on the file system.
Comment 12 Denis Roy CLA 2011-08-12 14:09:05 EDT
This is fixed.  We sync fairly often, and the auto-sync has stabilized.