Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 324116 - Move the www.eclipse.org website repository to Git
Summary: Move the www.eclipse.org website repository to Git
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Website (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: phoenix.ui CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 326588 (view as bug list)
Depends on: 293192 359737 375092 387706 387707 387708 387711 387820 387910 387964 387966 388294 388620 389480 389585 389672 390190 390194 390197 390204 390206 390208 390217 390248 390275 390316 390327 390461 390462 390568 390890 390915 390916 390933 390945 391006 391168 391169 391189 391192 391194 391264 391437 391584 391605 391606 391633 391636 391649 391682 391706 391721 391800 391867 391935 392456 392830 392869 392920 393438 393533 393931 394226 394821 394943 394990 395397 395398 395399 395423 395491 395543 395578 396032 396583 396689 397016 397221 397754 397755 401645 402523 406204 407620 409688 412184 412458 413409 416948 422608 425827
Blocks: 358980 373657 397591
  Show dependency tree
 
Reported: 2010-08-31 13:50 EDT by Denis Roy CLA
Modified: 2014-03-11 15:59 EDT (History)
39 users (show)

See Also:


Attachments
Import Script (1.47 KB, text/plain)
2010-09-30 15:11 EDT, Denis Roy CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Denis Roy CLA 2010-08-31 13:50:55 EDT
I think it would be a good thing to migrate the www.eclipse.org CVS repository to Git.  A quality team provider is an obvious pre-requisite.
Comment 1 Wayne Beaton CLA 2010-09-29 15:32:15 EDT
*** Bug 326588 has been marked as a duplicate of this bug. ***
Comment 2 Denis Roy CLA 2010-09-29 22:25:42 EDT
From bug 293192 comment 10:
(In reply to comment #9)
> > Since EGit 0.9.3 is out now, are we "adequate" enough?
> > 
> > Can we at least start transitioning some pieces like the website infrastructure
> > at Eclipse to Git?

There is also a resourcing issue. Denis, do you have any sense for what kind of
effort this will take to change over?

I think moving www.eclipse.org over to git will be trivial if we only keep one big repo.  If we split it into one repo per project that could be tedious.  The other thing we'll need to do is hack the checkout mechanism so that new commits/pushed content gets published to www as quickly as possible.

The first step would be (for us) to add the org.eclipse/www CVS repo into the Git mirror on dev.eclipse.org.

I agree that moving the website repo would be a great way to get us all using EGit.
Comment 3 Wayne Beaton CLA 2010-09-29 23:25:13 EDT
One big repo means that everybody who does work on any part of the website has to have every part of the website (in their local clone of the repo). My workspace, which has only a portion of the website, is huge.

I think that we may need to be more selective.
Comment 4 Gunnar Wagenknecht CLA 2010-09-30 02:28:47 EDT
How are permissions configured in one single repo? I thought that commit permissions apply to a whole repo but not to a subset of the repo tree.
Comment 5 Denis Roy CLA 2010-09-30 09:38:41 EDT
> How are permissions configured in one single repo? I thought that commit
> permissions apply to a whole repo but not to a subset of the repo tree.

Gah, you're right, I was forgetting that -- each project has their own space.

I guess we could create one repo for the EMO stuff, and one repo per project?
Comment 6 Wayne Beaton CLA 2010-09-30 10:45:43 EDT
We need to start prepping the committer community for this eventuality. All committers who maintain a project website at eclipse.org have to monitor this bug. I'll include a reference to it in my next note to the committers. In the meantime, we should start blogging/tweeting/communicating this.

I don't think that we're quite ready to specify a timeframe.

Denis, given that we're probably going to have to make multiple Git repos to make this work, how likely is it that we can do this piecemeal? (i.e. migrate projects in groups). I'm thinking hard/unlikely.

Is there anything else that we need to be thinking about to "Do this right"?
Comment 7 Denis Roy CLA 2010-09-30 11:08:04 EDT
> I don't think that we're quite ready to specify a timeframe.

I think we need to set a timeframe.  Otherwise, it will never be done (see: deprecate CVS)

If we are in agreement that the team provider is adequate, I can start a proof of concept sandbox repo so that we can all play around in.


> Denis, given that we're probably going to have to make multiple Git repos to
> make this work, how likely is it that we can do this piecemeal? (i.e. migrate
> projects in groups). I'm thinking hard/unlikely.

The current group ownership for each top-level directory of www clearly defines what belongs together in one repo.  I can easily see how scripting an import routing is feasible.

I'll get to work on a preliminary import.
Comment 8 Doug Schaefer CLA 2010-09-30 11:30:14 EDT
Very cool guys. +1 from me. If you want to use the cdt space as an early customer, I'd be happy to volunteer.
Comment 9 Denis Roy CLA 2010-09-30 15:11:01 EDT
Created attachment 179990 [details]
Import Script

Here's a script which will import each website top-level directory into its own Git repo, setting correct permissions as it goes along.

You can start seeing some appear on the CGit page: http://git.eclipse.org/c/

That page will become cluttered -- we're tracking that in bug 320443
Comment 10 Denis Roy CLA 2010-10-01 13:46:23 EDT
There's a few project website repos up there -- they all begin with www.eclipse.orgTEST.  CDT is one of them.

http://git.eclipse.org/

They should be fully pushable by the appropriate groups.  Those repos are all just temporary -- I'll delete them before the actual move, and the commits are not published to www.eclipse.org.
Comment 11 John Arthorne CLA 2011-06-22 13:27:14 EDT
Is this migration being done by the Eclipse Foundation or is it up to each project to migrate their web site content?
Comment 12 Denis Roy CLA 2011-06-22 14:27:15 EDT
> Is this migration being done by the Eclipse Foundation or is it up to each
> project to migrate their web site content?

My guess if that the EMO will do this.
Comment 13 Miles Parker CLA 2011-08-08 15:26:21 EDT
Hey cool, this means we can fork the entire Eclipse website!
Comment 14 Wayne Beaton CLA 2011-09-27 08:20:10 EDT
Can we set a date for this? The date, IMHO, should be well in advance of the suggested December 21/2012 CVS retirement date.

Is this something we can stage? Can we, for example, put the pieces in place to let a handful of projects do the migration voluntarily before the end of 2011?
Comment 15 Denis Roy CLA 2011-09-27 09:01:55 EDT
> Can we set a date for this?

Sure.


> Is this something we can stage? 

I tried that almost exactly one year ago, and I received zero feedback.  Please advise on how to better proceed.
Comment 16 Wayne Beaton CLA 2011-09-27 09:05:59 EDT
(In reply to comment #15)
> > Can we set a date for this?
> 
> Sure.
> 
> 
> > Is this something we can stage? 
> 
> I tried that almost exactly one year ago, and I received zero feedback.  Please
> advise on how to better proceed.

Correct me if I'm wrong, but we only go as far as creating copies of the repository to encourage experimentation: we didn't do the next step of actually driving website content from the Git repository.

How about we set a date to start moving the EF stuff? I can commit some cycles to moving over the Project management code, for example.
Comment 17 Nathan Gervais CLA 2011-09-27 09:27:57 EDT
(In reply to comment #16)
> (In reply to comment #15)
> > > Can we set a date for this?
+1 for this

Also I think I've figured out the git checkout command, i got it running on the EconTest box.  Can we use phoenix.eclipse.org as our 'git' version of the site?
Comment 18 John Arthorne CLA 2011-09-28 14:23:28 EDT
You've probably gotten past this step, but just posting this link in case it helps..

http://philsturgeon.co.uk/news/2010/02/Deploying-websites-with-Git
Comment 19 Jesse McConnell CLA 2012-03-08 10:42:29 EST
Denis,

Where are we on this?  I would really like to move jetty's website over to use git asap, we want to leverage it  for some documentation reworking we are doing.
Comment 20 Denis Roy CLA 2012-03-09 15:48:54 EST
> Where are we on this? 

Looks like I have two blockers: 

- bug 293192 Quality team provider and tooling for Git. Do committers feel that we're there?  Can I close the bug as FIXED?

- bug 359737 Migrate EF-maintained websites on www to Git.  I'm waiting for an answer from Wayne.
Comment 21 Jesse McConnell CLA 2012-03-09 15:55:48 EST
ok, I poked on those issues

would be great to get this morning and kiss off cvs once and for all :)
Comment 22 Wayne Beaton CLA 2012-03-09 16:02:41 EST
(In reply to comment #21)
> ok, I poked on those issues
> 
> would be great to get this morning and kiss off cvs once and for all :)

Did you mean "moving" or "mourning"?
Comment 23 Jesse McConnell CLA 2012-03-09 16:47:18 EST
definite 'moving' :)
Comment 24 Ed Willink CLA 2012-04-18 12:31:03 EDT
It would be helpful to establish a clear hierarchical policy.

For instance each of

www/atl
www/m2m/atl
www/modeling/m2m/atl

exists. Which are correct? Which are incorrect?
Comment 25 Wayne Beaton CLA 2012-04-18 12:52:54 EDT
(In reply to comment #24)
> It would be helpful to establish a clear hierarchical policy.
> 
> For instance each of
> 
> www/atl
> www/m2m/atl
> www/modeling/m2m/atl
> 
> exists. Which are correct? Which are incorrect?

It depends on who you're asking.

Officially, we prefer www/atl because it is the most resilient URL in the face of things like project moves and "container" project renames. We prefer to keep everything as flat as possible because it keeps the workload down and reduces the opportunity for surprises when things move around.

But the Modeling PMC tends to like things nested hierarchically. You may want to check with them.
Comment 26 Jesse McConnell CLA 2012-06-13 15:40:12 EDT
Where are we at with this issue?  Can I get a status update?  I am trying to sort out some timing for how we plan to deploy our new docbook documentation so we can migrate off of the bothersome wiki setup.
Comment 27 Konstantin Komissarchik CLA 2012-06-13 15:44:49 EDT
In case this needs saying... Please wait until after Juno so that we aren't fighting this while trying to update website for Juno release.
Comment 28 Jesse McConnell CLA 2012-07-20 14:08:51 EDT
ok, juno is out...

how about now?  any status update on this?
Comment 29 Denis Roy CLA 2012-07-20 14:14:31 EDT
Right now we're working on (amongst other things) bug 382758.
Comment 30 Jesse McConnell CLA 2012-07-20 14:16:17 EDT
ok, thanks denis
Comment 31 Wayne Beaton CLA 2012-08-21 10:55:02 EDT
I believe that we are ready to migrate project-specific websites to Git. At this point, we're looking for victims^H^H^H^H^H^Hvolunteers. Project leads can open a blocker bug on this to request migration of their project website.
Comment 32 Jesse McConnell CLA 2012-08-21 10:56:13 EDT
I'll toss on that it was a painless transition for jetty :)
Comment 33 John Arthorne CLA 2012-08-22 14:33:15 EDT
Um, our existing web site seems to be gone:

http://www.eclipse.org/eclipse/

Do we have to migrate now to get it back? :)
Comment 34 Wayne Beaton CLA 2012-08-22 14:45:26 EDT
(In reply to comment #33)
> Um, our existing web site seems to be gone:
> 
> http://www.eclipse.org/eclipse/
> 
> Do we have to migrate now to get it back? :)

That would help :-)

Migrating the eclipse.org-common directory in combination with some regular expression trouble caused this. Matt is working on it now.
Comment 35 John Arthorne CLA 2012-08-22 14:53:08 EDT
Orion volunteers to migrate. Opened bug 387820.
Comment 36 Eclipse Webmaster CLA 2012-08-22 16:17:35 EDT
(In reply to comment #34)

> Migrating the eclipse.org-common directory in combination with some regular
> expression trouble caused this. Matt is working on it now.

The checkout script has been updated, and I've forced a checkout from cvs to restore /eclipse.

-M.
Comment 37 John Arthorne CLA 2012-08-23 11:03:28 EDT
Shameless plug: once your project migrates, you can edit your project web site entirely from your browser with orionhub.org. Orion has a decent HTML editor with syntax highlighting, validation, outline, content assist, etc. If you click "Get Plugins" at the top you can install the Code Mirror plugin which does syntax highlighting of PHP files as well. The Orion Git tools let you push/pull between your Orion workspace and git.eclipse.org.
Comment 38 Mike Milinkovich CLA 2012-08-23 11:31:03 EDT
(In reply to comment #37)
> Shameless plug: once your project migrates, you can edit your project web
> site entirely from your browser with orionhub.org.

I would love to do this. Are the workflows documented anywhere?

Do I have to use orionhub, or could I use a localhost install of Orion?
Comment 39 Ian Skerrett CLA 2012-08-23 11:54:50 EDT
(In reply to comment #38)
> (In reply to comment #37)
> > Shameless plug: once your project migrates, you can edit your project web
> > site entirely from your browser with orionhub.org.
> 
> I would love to do this. Are the workflows documented anywhere?
> 
> Do I have to use orionhub, or could I use a localhost install of Orion?

+1 sign me up to use Orion for the web site work I do.  Some documentation about be nice.  :-)
Comment 40 Wayne Beaton CLA 2012-08-23 12:03:09 EDT
(In reply to comment #37)
> Shameless plug: once your project migrates, you can edit your project web
> site entirely from your browser with orionhub.org. Orion has a decent HTML
> editor with syntax highlighting, validation, outline, content assist, etc.
> If you click "Get Plugins" at the top you can install the Code Mirror plugin
> which does syntax highlighting of PHP files as well. The Orion Git tools let
> you push/pull between your Orion workspace and git.eclipse.org.

Testing PHP doesn't work. I guess that you need actual PHP support for that :-)

I'm thinking that setting up an Orion instance on a server that does support PHP would be extremely useful... Or is there some way that we can leverage orionhub for this?
Comment 41 John Arthorne CLA 2012-08-23 13:24:36 EDT
Most of the pages I edit have trivial PHP and I just test by pushing my changes, refresh the page, repeat if needed (same workflow as I had in Eclipse desktop). I'll try writing up a wiki page with more detailed instructions. Workflow would be the same with a localhost Orion server.

Enabling PHP on orionhub is also a possibility. That would allow people to deploy to a test site within orion, although often with PHP you are going to have other server side requirements such as databases, etc.
Comment 42 Wayne Beaton CLA 2012-08-23 13:35:43 EDT
(In reply to comment #41)

> Enabling PHP on orionhub is also a possibility. That would allow people to
> deploy to a test site within orion, although often with PHP you are going to
> have other server side requirements such as databases, etc.

Security is another concern. Anybody can write code on OrionHub. Letting everybody run arbitrary PHP scares me a bit...
Comment 43 Ian Skerrett CLA 2012-08-23 13:55:58 EDT
(In reply to comment #42)
> (In reply to comment #41)
> 
> > Enabling PHP on orionhub is also a possibility. That would allow people to
> > deploy to a test site within orion, although often with PHP you are going to
> > have other server side requirements such as databases, etc.
> 
> Security is another concern. Anybody can write code on OrionHub. Letting
> everybody run arbitrary PHP scares me a bit...

Wouldn't it make more sense to setup a test server for eclipse.org and we do the testing there, not at OrionHub?  OrionHub isn't going to be able to deploy every single runtime environment for all the users, so I think we should set ourselves up as a standard use case.
Comment 44 Wayne Beaton CLA 2012-08-23 14:10:28 EDT
(In reply to comment #43)
> (In reply to comment #42)
> > (In reply to comment #41)
> > 
> > > Enabling PHP on orionhub is also a possibility. That would allow people to
> > > deploy to a test site within orion, although often with PHP you are going to
> > > have other server side requirements such as databases, etc.
> > 
> > Security is another concern. Anybody can write code on OrionHub. Letting
> > everybody run arbitrary PHP scares me a bit...
> 
> Wouldn't it make more sense to setup a test server for eclipse.org and we do
> the testing there, not at OrionHub?  OrionHub isn't going to be able to
> deploy every single runtime environment for all the users, so I think we
> should set ourselves up as a standard use case.

I've opened Bug 387922. Let's take this discussion there.
Comment 45 Denis Roy CLA 2012-10-03 14:06:41 EDT
I've moved the downloads area.
Comment 46 Werner Keil CLA 2012-10-25 10:08:23 EDT
Great, how can individual projects move?
And what about Orbit? Sorry if there may be one of those Dozens or more related bugs, but I saw, Orbit guidelines mention uploading artefacts to the CVS repo so far.

Thanks
Comment 47 Wayne Beaton CLA 2012-10-29 10:57:55 EDT
(In reply to comment #46)
> Great, how can individual projects move?

http://wiki.eclipse.org/Git/Migrating_to_Git#Migrating_Your_Project_Website

> And what about Orbit? Sorry if there may be one of those Dozens or more
> related bugs, but I saw, Orbit guidelines mention uploading artefacts to the
> CVS repo so far.

Orbit has been given a special exception to keep its "source" repository in CVS for now. The website repository will still have to move.
Comment 48 John Arthorne CLA 2012-11-23 16:53:16 EST
(In reply to comment #38)
> (In reply to comment #37)
> > Shameless plug: once your project migrates, you can edit your project web
> > site entirely from your browser with orionhub.org.
> 
> I would love to do this. Are the workflows documented anywhere?
> 
> Do I have to use orionhub, or could I use a localhost install of Orion?

Here is a blog post I wrote with some directions to get started editing your site with OrionHub:

http://planetorion.org/news/2012/11/editing-eclipse-org-web-sites-with-orionhub/
Comment 49 Eclipse Webmaster CLA 2013-11-29 14:14:55 EST
This bug was started 3 years ago, and I'd like to see us close it and it's dependent children.

I suggest we either force move outstanding projects or simply remove them(with redirects possibly).

After 3 years I don't want us to wait another one so I'd like to see something on the 6 months or less scale in terms of time.

Comments?

-M.
Comment 50 Wayne Beaton CLA 2013-11-29 14:21:41 EST
(In reply to Eclipse Webmaster from comment #49)
> This bug was started 3 years ago, and I'd like to see us close it and it's
> dependent children.
> 
> I suggest we either force move outstanding projects or simply remove
> them(with redirects possibly).
> 
> After 3 years I don't want us to wait another one so I'd like to see
> something on the 6 months or less scale in terms of time.

Projects have had more than enough time to migrate their websites.

I have several concerns:

1) If a project has not migrated their website to Git, then they have not been making regular updates to their website. Does this speak to the liveliness of a project?

2) In light of the recent spate of vulnerability reports, I am concerned that these apparently abandoned web directories represent a potential security concern.

I'm in favour of setting a March deadline to migrate. After that, we archive the CVS repository and toast every part of the website that is not maintained in Git. Spring cleaning.
Comment 51 Eclipse Webmaster CLA 2014-03-11 15:59:15 EDT
It's March and that means cleanup time.  

All CVS based content has been removed from www.eclipse.org.

-M.