Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 351001 - Moving PDE code to Git repo
Summary: Moving PDE code to Git repo
Status: VERIFIED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.8 M2   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on: 286140
Blocks: 345479
  Show dependency tree
 
Reported: 2011-07-02 16:54 EDT by Ankur Sharma CLA
Modified: 2011-09-22 09:05 EDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ankur Sharma CLA 2011-07-02 16:54:34 EDT
This is a generic bug to track, discuss, plan and vote for moving pde code from CVS to Git repo.

The migration process is explained at  http://wiki.eclipse.org/Git/Migrating_to_Git

More info at http://wiki.eclipse.org/Git
Comment 1 Curtis Windatt CLA 2011-07-06 11:45:54 EDT
Kim is adding PDE to the list of projects waiting to be migrated.  Kim, please let us know when you think the migration (or test migration) will happen.  We should be able to accommodate any time.

Todo:
1) Determine any dead components that can be removed.
2) Check with all committers to set up time to stop all commits to the CVS repo to do the migration.
3) After migration, inspect the repository content for issues (especially for maintenance and Java7 branches)
4) Start developing using eGit and report any problems.
Comment 2 Curtis Windatt CLA 2011-07-06 12:15:57 EDT
- In 3.5 we moved the location of the pde projects in CVS to a folder layout.  How will that affect the git repo for maintenance branches?

- No more code changes are planned for API Tools in the Java 7 Beta branch, but changes may come up and we will be backporting the changes at the same time JDT does.

- The plan is to have separate repos for each permissions group.  In PDE this means separate repos for: PDE (core, ui, feature, runtime, api tools, etc.), BUILD, INCUBATOR, and DOC.  How difficult is it to have a separate repo with the same permissions?  API Tools has many bundles for its EE fragments and would benefit from having its own repository.

- For components to change... 
http://wiki.eclipse.org/Platform-releng/Git_Migration_Granularity

org.eclipse.pde.releng - This bundle is not listed on the wiki, it only contains psf files.
visualization - Folder with three bundles is not built by Eclipse, provided though the pde website.  Not listed in bundles to migrate.  Not actively being developed, but not something we want to lose.
org.eclipse.pde.p2.ui - Only used in 3.4.x
pde-ui-home - Old website?  Not needed
pde-build-home - Old website? Not needed
Comment 3 Curtis Windatt CLA 2011-07-06 12:21:33 EDT
One thing I just noticed on the wiki, there is no separate repository for pde doc.  Instead our doc plug-in is inside pde build.  The doc plug-in is shared among all of the components.  It should probably have its own repository.
Comment 4 Dani Megert CLA 2011-07-07 02:14:07 EDT
(In reply to comment #3)
> One thing I just noticed on the wiki, there is no separate repository for pde
> doc.  Instead our doc plug-in is inside pde build.  The doc plug-in is shared
> among all of the components.  It should probably have its own repository.

Please see bug 345471 comment 14.
Comment 5 Paul Webster CLA 2011-07-07 07:28:42 EDT
(In reply to comment #2)
> - In 3.5 we moved the location of the pde projects in CVS to a folder layout. 
> How will that affect the git repo for maintenance branches?

You probably have 2 choices here:

1) ignore the old projects.  The tool basically converts one CVS root, in your case probably /cvsroot/eclipse pde

2) convert the new repo, and the convert the old repo into a similar structure using the same pattern we did for eclipse.platform.ui.  The old repo should not create any of the newer tags/branches.  Then you can fetch the old repo into the new repo, and use commit grafting to make it look like last-old-commit is the parent of first-new-commit.  This will preserve your history, more or less, and will allow you to commit to an older maintenance branch if necessary.

> 
> - The plan is to have separate repos for each permissions group.  In PDE this
> means separate repos for: PDE (core, ui, feature, runtime, api tools, etc.),
> BUILD, INCUBATOR, and DOC.  How difficult is it to have a separate repo with
> the same permissions?  API Tools has many bundles for its EE fragments and
> would benefit from having its own repository.

We looked at that, and unless your bundles are in the 100s it's probably not worth it the hassle of multiple repos.  It's up to your component, however.



> pde-ui-home - Old website?  Not needed
> pde-build-home - Old website? Not needed

The old websites shouldn't be converted as part of this.  The foundation is going to migrate the org.eclipse CVS repo, and the old ones probably aren't used for much.

PW
Comment 6 Curtis Windatt CLA 2011-07-27 16:55:14 EDT
The org.eclipse.pde.ua.tests bundle contains a few tests that have never been included in a build. Bug 286140 describes the work to improve them.  The tests are minimal, so we could consider simply deleting the bundle when moving to git.
Comment 7 Curtis Windatt CLA 2011-07-27 17:46:47 EDT
I worked on conditioning the PDE UI repos, creating maintenance branches for all releases.  How far back do I need to do this?

org.eclipse.pde.doc.user -      3.1 and later handled by Andrew
ds/org.eclipse.pde.ds.core 	3.5 and later
ds/org.eclipse.pde.ds.tests 	3.5 and later
ds/org.eclipse.pde.ds.ui 	3.5 and later
ua/org.eclipse.pde.ua.core  	3.5 and later
ua/org.eclipse.pde.ua.ui 	3.5 and later
ui/org.eclipse.pde		3.1 and later
ui/org.eclipse.pde.core		3.1 and later
ui/org.eclipse.pde.junit.runtime 3.1 and later
ui/org.eclipse.pde.launching	3.6 and later
ui/org.eclipse.pde.runtime	3.1 and later
ui/org.eclipse.pde.ui		3.1 and later
ui/org.eclipse.pde.ui.templates	3.3 and later
ui/org.eclipse.pde.ui.tests	3.1 and later
ui/org.eclipse.ui.views.log	3.4 and later
org.eclipse.pde.ui.p2           3.4 only
ua/org.eclipse.pde.ua.tests  Never in a build, see comment #6
Comment 8 Paul Webster CLA 2011-07-27 17:50:00 EDT
It's up to your component.  I think going back to 3.4.2+ is reasonable.

PW
Comment 9 Curtis Windatt CLA 2011-07-28 15:54:19 EDT
apitools/org.eclipse.pde.api.tools 3.4 and later
apitools/org.eclipse.pde.api.tools.ee.javase17  Only in HEAD
apitools/org.eclipse.pde.api.tools.ee.* 3.5 and later
apitools/org.eclipse.pde.api.tools.tests 3.4 and later
apitools/org.eclipse.pde.api.tools.ui 3.4 and later
Comment 10 Kim Moir CLA 2011-08-05 13:59:15 EDT
Test repo for pde ui is available
ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.ui.git 

Please let me know if you have any issues.  Also, let me know when you want to schedule the real migration.
Comment 11 Curtis Windatt CLA 2011-08-05 15:57:57 EDT
Thanks Kim

Newbie question, isn't MASTER supposed to be available? When I clone the repo I get a choice of branches to clone, but then I am forced to choose a local branch to work from.

Is it possible to not migrate some of the branches? The only (existing) branches anyone should have to work with are the maintenance ones.

I appreciate that the repo retains the directory structure from the CVS repo. This makes the suggestion of separating out API Tools unecessary.
Comment 12 Paul Webster CLA 2011-08-05 16:02:13 EDT
I notice that R3_7_maintenance starts with a "delete commit" (text at the end), and there's a CVSROOT at the root of the repo :-)

Also, when I clone I get a master (which points to origin/master or remotes/origin/master) that looks correct.


Text from the delete commit:
Delete:
        ui/org.eclipse.pde/.classpath
        ui/org.eclipse.pde/.cvsignore
        ui/org.eclipse.pde/.project
        ui/org.eclipse.pde/.settings/org.eclipse.core.resources.prefs
        ui/org.eclipse.pde/.settings/org.eclipse.jdt.core.prefs
        ui/org.eclipse.pde/.settings/org.eclipse.jdt.ui.prefs
        ui/org.eclipse.pde/.settings/org.eclipse.pde.core.prefs
        ui/org.eclipse.pde/.settings/org.eclipse.pde.prefs
        ui/org.eclipse.pde/META-INF/MANIFEST.MF
        ui/org.eclipse.pde/about.html
        ui/org.eclipse.pde/about.ini
        ui/org.eclipse.pde/about.mappings
        ui/org.eclipse.pde/about.properties
        ui/org.eclipse.pde/build.properties
        ui/org.eclipse.pde/cheatsheets/helloworld-composite.xml
        ui/org.eclipse.pde/cheatsheets/helloworld/helloworld-create.xml
        ui/org.eclipse.pde/cheatsheets/helloworld/helloworld-extension.xml
        ui/org.eclipse.pde/cheatsheets/helloworld/helloworld-feature.xml
        ui/org.eclipse.pde/cheatsheets/helloworld/helloworld-install.xml
        ui/org.eclipse.pde/cheatsheets/helloworld/helloworld-update.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp-composite.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp/rcpapp-create.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp/rcpapp-customize.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp/rcpapp-export.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp/rcpapp-feature-product.xml
        ui/org.eclipse.pde/cheatsheets/rcpapp/rcpapp-plugin-product.xml
        ui/org.eclipse.pde/cheatsheets/setup-apitools-existing-projects.xml
        ui/org.eclipse.pde/eclipse32.png
        ui/org.eclipse.pde/images/topiclabel/ov_eclplugindev48.gif
        ui/org.eclipse.pde/images/topiclabel/ov_eclplugindev48_hov.gif
        ui/org.eclipse.pde/images/topiclabel/sa_samplecube48.gif
        ui/org.eclipse.pde/images/topiclabel/sa_samplecube48_hov.gif
        ui/org.eclipse.pde/images/topiclabel/sa_sampleorb48.gif
        ui/org.eclipse.pde/images/topiclabel/sa_sampleorb48_hov.gif
        ui/org.eclipse.pde/images/topiclabel/tu_createplugin48.gif
        ui/org.eclipse.pde/images/topiclabel/tu_createplugin48_hov.gif
        ui/org.eclipse.pde/images/topiclabel/tu_rcpapp48.gif
        ui/org.eclipse.pde/images/topiclabel/tu_rcpapp48_hov.gif
        ui/org.eclipse.pde/images/topiclabel/wn_pluginenviro48.gif
        ui/org.eclipse.pde/images/topiclabel/wn_pluginenviro48_hov.gif
        ui/org.eclipse.pde/intro/css/overview.css
        ui/org.eclipse.pde/intro/css/overview.properties
        ui/org.eclipse.pde/intro/css/samples.css
        ui/org.eclipse.pde/intro/css/samples.properties
        ui/org.eclipse.pde/intro/css/tutorials.css
        ui/org.eclipse.pde/intro/css/tutorials.properties
        ui/org.eclipse.pde/intro/css/whatsnew.css
        ui/org.eclipse.pde/intro/css/whatsnew.properties
        ui/org.eclipse.pde/intro/overviewExtensionContent.xml
        ui/org.eclipse.pde/intro/samplesExtensionContent.xml
        ui/org.eclipse.pde/intro/samplesExtensionContent2.xml
        ui/org.eclipse.pde/intro/tutorialsExtensionContent.xml
        ui/org.eclipse.pde/intro/whatsnewExtensionContent.xml
        ui/org.eclipse.pde/plugin.properties
        ui/org.eclipse.pde/plugin.xml
Comment 13 Kim Moir CLA 2011-08-05 16:20:44 EDT
Yes, I don't know why there is a CVSROOT.  I have a ran the script that is supposed to git rid of all the delete commits, not sure why it's still there.
Comment 14 Andrew Niefer CLA 2011-08-05 16:27:12 EDT
(In reply to comment #12)
> I notice that R3_7_maintenance starts with a "delete commit" (text at the end),
> and there's a CVSROOT at the root of the repo :-)
> 
> Also, when I clone I get a master (which points to origin/master or
> remotes/origin/master) that looks correct.
> 
> 
> Text from the delete commit:
> Delete:
>         ui/org.eclipse.pde/.classpath

Curtis, it looks like the org.eclipse.pde bundle needs to be branced for R3_7_maintenance
Comment 15 Curtis Windatt CLA 2011-08-08 10:34:18 EDT
(In reply to comment #14)
> Curtis, it looks like the org.eclipse.pde bundle needs to be branched for
> R3_7_maintenance

I have branched it for 3_5 and 3_7.  That is as far back as that bundle appears to go.
Comment 16 Kim Moir CLA 2011-08-08 11:11:42 EDT
I know what I did that caused the bogus CVSROOT in the git repo.  So I think we can fix the two issues the test repo revealed.
Comment 17 Curtis Windatt CLA 2011-08-11 16:21:46 EDT
So far so good. I've been working with local branches, merging, committing and pushing some changes upstream. Perhaps it is time to discuss when the actual migration should take place.  Timing mostly depends on when Kim thinks she will be available.

A couple questions about git:

1) Is there a standard commit message structure being used?

2) How is tagging being done? Just run the tag wizard and update the map file manually? Is the release tool been/being updated for git?  I noticed that the eGit instructions say tagging should be available from the project context menu, but I only can find it from the history view.

3) Is there a wiki or somewhere we are collecting best practices and pitfalls? I remember it being discussed on the arch call, but can't find it on google.  It would be tremendously helpful to have doc targeted towards Eclipse developers (vs a simple learning example).
Comment 18 Kim Moir CLA 2011-08-11 17:18:28 EDT
Right now I'm scheduled to do platform.resources next Thursday.  As well, I have to do UA next week when Chris G returns from vacation.  So let me know what day works for you. Friday? (Trying to avoid migrations on build days) In response to your questions:

1)  I just add the bug description and link to the bug, not sure what over people do. 

2) No, the releng tool doesn't work with git.  Paul has written some scripts to do tagging: http://wiki.eclipse.org/E4/Git

3) I find Paul's git workflows document very useful http://wiki.eclipse.org/Platform-releng/Git_Workflows#Clone_a_repo  Please update it as you find other tips :-)
Comment 19 Curtis Windatt CLA 2011-08-11 17:33:14 EDT
(In reply to comment #18)
> Friday? (Trying to avoid migrations on build days) 

Ok, we'll tentatively schedule it for Friday.  I will confirm at our Monday status call.

> 3) I find Paul's git workflows document very useful
> http://wiki.eclipse.org/Platform-releng/Git_Workflows

Thanks, exactly what I was looking for. We need to add more links to the page as searching for 'Eclipse Git Workflows' etc, doesn't bring it up.  I don't even find it using the Eclipsepedia search unless I include the word 'workflows'.
Comment 20 Paul Webster CLA 2011-08-11 18:52:56 EDT
(In reply to comment #17)
> 1) Is there a standard commit message structure being used?

In Platform UI we just copy the Grey line from the bugzilla:
Bug 351001 - Moving PDE code to Git repo
bug specific comment here

The scripts that I use are looking for bug [0-9]+ to generate the dev list message.

> 2) How is tagging being done? Just run the tag wizard and update the map file
> manually? Is the release tool been/being updated for git?  I noticed that the
> eGit instructions say tagging should be available from the project context
> menu, but I only can find it from the history view.

If you'd like to use the tagging scripts, check with Andrew Niefer.  He has some new scripts on build.eclipse.org that seem pretty reliable.


PW
Comment 21 Remy Suen CLA 2011-08-17 11:17:08 EDT
(In reply to comment #17)
> 1) Is there a standard commit message structure being used?

I copy the bug number and summary per what Paul said in comment 19 and then make two line breaks before entering my commit message. For the message myself I restrict myself to using less than 65-70 characters per line so that it can be read easily from the command line using (considering standard terminal widths).

You can see what we do in EGit for an additional point of reference.
http://wiki.eclipse.org/EGit/Contributor_Guide#Commit_message_guidelines

> 3) Is there a wiki or somewhere we are collecting best practices and pitfalls?

Besides Paul's page, there is this other page written by another team.
http://wiki.eclipse.org/MDT/OCL/Dev/EGit
Comment 22 Andrew Niefer CLA 2011-08-18 16:52:59 EDT
PDE/Build repo is now at
git://git.eclipse.org/gitroot/pde/eclipse.pde.build.git
Comment 23 Kim Moir CLA 2011-08-19 16:04:08 EDT
The pde ui migrated repo is now ready
ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.ui.git 

Please update the map files in HEAD if you'd like me to run a test build.
Comment 24 Andrew Niefer CLA 2011-08-19 16:15:25 EDT
I have updated the PDE/Build map files in HEAD
Comment 25 Curtis Windatt CLA 2011-08-25 15:07:53 EDT
Migration is complete, N, I and M builds have run with the updated map files.  Thanks Kim for looking after this.
Comment 26 Curtis Windatt CLA 2011-09-14 14:19:41 EDT
Verified.  The repo is functioning well for us.
Comment 27 Kim Moir CLA 2011-09-21 16:54:19 EDT
I just migrated the pde feature to a repo too

ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.git
Comment 28 Dani Megert CLA 2011-09-22 08:27:03 EDT
RED FLAG: The 'eclipse.pde.git' repo is in a broken state since the 'org.eclipse.pde' bundle got migrated from the OLD location (/cvsroot/eclipse) instead of /cvsroot/eclipse/pde/.
Comment 29 Dani Megert CLA 2011-09-22 08:29:27 EDT
> instead of /cvsroot/eclipse/pde/.
==> /cvsroot/eclipse/pde/ui
to be exact.
Comment 30 Kim Moir CLA 2011-09-22 08:43:46 EDT
Dani, 

The eclipse.pde.git repo is just to store the feature.  The org.eclipse.pde.ui.git repo is just used to store the other stuff.  

I just migrated the old location of org.eclipse.pde into the eclipse.pde.git repo for historical builds.  The org.eclipse.pde.ui.git repo stores the current bundles, with the exclusion of the pde feature.
Comment 31 Dani Megert CLA 2011-09-22 09:00:35 EDT
(In reply to comment #30)
> Dani, 
> 
> The eclipse.pde.git repo is just to store the feature.  The
> org.eclipse.pde.ui.git repo is just used to store the other stuff.  
> 
> I just migrated the old location of org.eclipse.pde into the eclipse.pde.git
> repo for historical builds.  The org.eclipse.pde.ui.git repo stores the current
> bundles, with the exclusion of the pde feature.

I reopened the bug because there is no 'org.eclipse.pde' in my cloned  pde.ui.git repo. I guess I just have to go through the pain and delete the repo, clone it again and import all PDE projects again.