Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 349695 - Migrate CVS to GIT
Summary: Migrate CVS to GIT
Status: CLOSED FIXED
Alias: None
Product: PTP
Classification: Tools
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Roland Schulz CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-17 10:59 EDT by Roland Schulz CLA
Modified: 2012-09-21 11:50 EDT (History)
4 users (show)

See Also:


Attachments
options for cv2git conversion (29.17 KB, application/octet-stream)
2011-06-17 11:14 EDT, Roland Schulz CLA
no flags Details
options for cvs2git conversion (29.17 KB, application/octet-stream)
2011-06-17 15:27 EDT, Roland Schulz CLA
no flags Details
Recipe to convert CVS to GIT (3.47 KB, text/plain)
2011-10-27 18:30 EDT, Roland Schulz CLA
no flags Details
Final used recipe to convert CVS to GIT (3.62 KB, text/plain)
2011-11-10 00:55 EST, Roland Schulz CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roland Schulz CLA 2011-06-17 10:59:48 EDT
On http://wiki.eclipse.org/PTP/meetings/June_2011 we decided to want to transition to GIT ~1month after CDT. This bug is meant to coordinate open issues regarding this transition.

Eclipse documentation about Git and the transition can be found at:
http://wiki.eclipse.org/Git
http://wiki.eclipse.org/Git/Migrating_to_Git

We probably want to follow their process in most parts. Their central bug for the migration is at:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=316208

The conversion is done using (assuming we use the same approach as CDT):
- http://archive.eclipse.org/arch/tools-cvs.tgz
- cvs2git (trunk, how to obtain and call see 316208)
- cvs2git.options (attached)

Open questions / TODOs:
- On the day of the conversion we need a current tools-cvs.tgz (at least the org.eclipse.ptp part). We need to get access to these files or ask the webmaster to generate the archive for us on that day.
- CDT has uploaded their test repository to http://git.eclipse.org. If we want to do that too we need to ask for permissions.
- Do we want to switch the build system (e.g. to Maven) at the same time? We first need to decide this before we can work-out our build with GIT. How CDT is doing the build with GIT is described in bug 316212.
- Do we want to split the repository into more parts than PTP and photran?
- What else is todo? Timeline?
Comment 1 Roland Schulz CLA 2011-06-17 11:14:34 EDT
Created attachment 198185 [details]
options for cv2git conversion
Comment 2 Roland Schulz CLA 2011-06-17 14:20:11 EDT
cvs2svn Statistics:
------------------
Total CVS Files:             27849
Total CVS Revisions:         92908
Total CVS Branches:          47957
Total CVS Tags:            6135232
Total Unique Tags:            1091
Total Unique Branches:          21
CVS Repos Size in KB:       742038
Total SVN Commits:           11941
First Revision Date:    Thu Dec  9 11:30:01 2004
Last Revision Date:     Thu Jun  9 17:54:55 2011

The number of unique tags is probably too high. Do we want to remove all or some of the version tags "v*"?
Comment 3 Greg Watson CLA 2011-06-17 14:28:38 EDT
(In reply to comment #2)
> cvs2svn Statistics:
> ------------------
> Total CVS Files:             27849
> Total CVS Revisions:         92908
> Total CVS Branches:          47957
> Total CVS Tags:            6135232
> Total Unique Tags:            1091
> Total Unique Branches:          21
> CVS Repos Size in KB:       742038
> Total SVN Commits:           11941
> First Revision Date:    Thu Dec  9 11:30:01 2004
> Last Revision Date:     Thu Jun  9 17:54:55 2011
> 
> The number of unique tags is probably too high. Do we want to remove all or
> some of the version tags "v*"?

These numbers seem completely wrong, unless they are for the whole of the Eclipse CVS repository. The Unique Tags/Branches are probably closer, but even these are way too high. I doubt there are more than 10 branches in PTP.
Comment 4 Roland Schulz CLA 2011-06-17 14:47:23 EDT
(In reply to comment #3)
> (In reply to comment #2)
> > cvs2svn Statistics:
> > ------------------
> > Total CVS Files:             27849
> > Total CVS Revisions:         92908
> > Total CVS Branches:          47957
> > Total CVS Tags:            6135232
> > Total Unique Tags:            1091
> > Total Unique Branches:          21
> > CVS Repos Size in KB:       742038
> > Total SVN Commits:           11941
> > First Revision Date:    Thu Dec  9 11:30:01 2004
> > Last Revision Date:     Thu Jun  9 17:54:55 2011
> > 
> > The number of unique tags is probably too high. Do we want to remove all or
> > some of the version tags "v*"?
> 
> These numbers seem completely wrong, unless they are for the whole of the
> Eclipse CVS repository. The Unique Tags/Branches are probably closer, but even
> these are way too high. I doubt there are more than 10 branches in PTP.

Are your sure? I get for the branches:
  CDT_3_Compatibility
  Callisto
  FAILOVER_RECOVERY
  FDT
  FDT_CLCore
  NG_20060523
  PERF_20060617
  PTP_TRY_PDI_20070730
  PTP_TRY_PDI_20070927
  PTP_TRY_RMR_20060628
  PTP_TRY_RMR_20070112
  PTP_TRY_RMR_20100325
* master
  photran_4_0
  photran_5_0
  ptp_1_0
  ptp_1_1
  ptp_2_0
  ptp_2_1
  ptp_3_0
  ptp_4_0
  v20100616-Photran600-All

Are some of these really not there?
I guess the PTP_TRY_* can be deleted. What about the others?

Tags (excluding v* - e.g. v20060109):
FDT_1_0_RC1
FDT_v_1_0_0
MERGE_1_NG_20060523
Merged-PTP_TRY_RMR_20100325-Branch
Merged-PTP_TRY_RMR_20100325-Branch-1
Merged-PTP_TRY_RMR_20100325-Branch-2
Merged-PTP_TRY_RMR_20100325-Branch-3
Merged-PTP_TRY_RMR_20100325-Branch-4
Merged-PTP_TRY_RMR_20100325-Branch-5
Merged-PTP_TRY_RMR_20100325-Branch-6
Merged-PTP_TRY_RMR_20100325-Branch-7
Merged-PTP_TRY_RMR_20100325-HEAD
Merged-PTP_TRY_RMR_20100325-HEAD-1
Merged-PTP_TRY_RMR_20100325-HEAD-2
Merged-PTP_TRY_RMR_20100325-HEAD-3
Merged-PTP_TRY_RMR_20100325-HEAD-4
Merged-PTP_TRY_RMR_20100325-HEAD-5
Merged-PTP_TRY_RMR_20100325-HEAD-6
Merged-PTP_TRY_RMR_20100325-HEAD-7
MergedToHEAD-PTP_TRY_RMR_20100325-Branch
MergedToHEAD-PTP_TRY_RMR_20100325-HEAD
Merged_NG_20060523_to_HEAD
PHOTRAN_6_0_0
POST_MERGE_1_NG_20060523
PRE_MERGE_1_NG_20060523
PRE_PERF_MERGE
PTP_1_0
PTP_1_0_RC1
PTP_1_0_RC2
PTP_1_0_RC3
PTP_1_0_RC4
PTP_1_0_RC5
PTP_1_1
PTP_1_1_1
PTP_1_1_RC1
PTP_1_1_RC2
PTP_1_1_RC3
PTP_1_1_RC4
PTP_2_0
PTP_2_0_1
PTP_2_1
PTP_2_1_4
PTP_3_0
PTP_4_0_0
PTP_TRY_RMR_20060628_premerge
PTP_TRY_RMR_20070112-1
Root_CDT_3_Compatibility
Root_Callisto
Root_FDT
Root_FDT_CLCore
Root_NG_20060523
Root_PERF_20060617
Root_PTP_2_0
Root_PTP_TRY_PDI_20070730
Root_PTP_TRY_PDI_20070927
Root_PTP_TRY_RMR_20060628
Root_PTP_TRY_RMR_20070112
Root_PTP_TRY_RMR_20100325
Root_photran_4_0
Root_photran_5_0
Root_ptp_1_1
Root_ptp_2_0
Root_ptp_2_1
Root_ptp_3_0
Root_ptp_4_0
V20050802
beforePhoenix
before_1_1_merge
merged-PTP_TRY_RMR_20070112
photran_5_0_0
pldt-on-CDT30
premerge-PTP_TRY_RMR_20070112
ptp4_0_0
release_0_9_0
Comment 5 Roland Schulz CLA 2011-06-17 15:27:17 EDT
Created attachment 198210 [details]
options for cvs2git conversion
Comment 6 Roland Schulz CLA 2011-06-17 15:28:19 EDT
Comment on attachment 198210 [details]
options for cvs2git conversion

corrected order in author mapping (username and email was swapped before)
Comment 7 Greg Watson CLA 2011-06-17 17:05:06 EDT
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > cvs2svn Statistics:
> > > ------------------
> > > Total CVS Files:             27849
> > > Total CVS Revisions:         92908
> > > Total CVS Branches:          47957
> > > Total CVS Tags:            6135232
> > > Total Unique Tags:            1091
> > > Total Unique Branches:          21
> > > CVS Repos Size in KB:       742038
> > > Total SVN Commits:           11941
> > > First Revision Date:    Thu Dec  9 11:30:01 2004
> > > Last Revision Date:     Thu Jun  9 17:54:55 2011
> > > 
> > > The number of unique tags is probably too high. Do we want to remove all or
> > > some of the version tags "v*"?
> > 
> > These numbers seem completely wrong, unless they are for the whole of the
> > Eclipse CVS repository. The Unique Tags/Branches are probably closer, but even
> > these are way too high. I doubt there are more than 10 branches in PTP.
> 
> Are your sure? I get for the branches:
>   CDT_3_Compatibility
>   Callisto
>   FAILOVER_RECOVERY
>   FDT
>   FDT_CLCore
>   NG_20060523
>   PERF_20060617
>   PTP_TRY_PDI_20070730
>   PTP_TRY_PDI_20070927
>   PTP_TRY_RMR_20060628
>   PTP_TRY_RMR_20070112
>   PTP_TRY_RMR_20100325
> * master
>   photran_4_0
>   photran_5_0
>   ptp_1_0
>   ptp_1_1
>   ptp_2_0
>   ptp_2_1
>   ptp_3_0
>   ptp_4_0
>   v20100616-Photran600-All

Ok, some of these are Photran branches. But I count 23, not 21. What's the difference between "CVS Branches/Tags" and "Unique Branches/Tags"?

As far as I'm concerned, the only important branches are:

 photran_4_0
 photran_5_0
 ptp_1_0
 ptp_1_1
 ptp_2_0
 ptp_2_1
 ptp_3_0
 ptp_4_0

There should now also be a ptp_4_1 and soon there will be ptp_5_0 (and presumably photran_6_0).

You should check with Jeff on the Photran branches.

I think the only important tags are the:
 v*
 PTP_[1234]_*
 Root_ptp_*
 Root_photran_* 

Again you should check with Jeff.
Comment 8 Roland Schulz CLA 2011-06-17 17:37:23 EDT
(In reply to comment #7)
> Ok, some of these are Photran branches. But I count 23, not 21. 
I count 22. The one additional is the master branch (equivalent to CVS HEAD).

> What's the
> difference between "CVS Branches/Tags" and "Unique Branches/Tags"?
I think for "CVS" branches/tags are counted per file. "Unique" is across all converted files (=the repository). 

> I think the only important tags are the:
>  v*
these are 1017. We probably don't want to keep all those. CDT keeps only https://bugs.eclipse.org/bugs/attachment.cgi?id=197677. As far as I understand this list only includes those tags which can still be downloaded from http://download.eclipse.org/tools/cdt/builds/.
Comment 9 Jeffrey Overbey CLA 2011-06-17 17:39:09 EDT
I agree with Greg on what the "important" branches and tags are.  I only recognized one or two of the other ones, and they're not important anymore.

> There should now also be a ptp_4_1 and soon there will be ptp_5_0 (and
> presumably photran_6_0).

Actually, we stopped making a separate Photran branch when joined the train last year (and started building along with PTP).  So, Photran 6 is maintained in the ptp_4_0 branch, and Photran 7 will be in the ptp_5_0 branch.

Jeff
Comment 10 Roland Schulz CLA 2011-06-17 19:10:00 EDT
The following folders are in the CVS repository but not in releng/ptp_project_sets/committer/ptp-HEAD-all-committer.psf

core/org.eclipse.ptp.proxy.tests
core/org.eclipse.ptp.rm.remote
debug/org.eclipse.ptp.debug.sdm.x10
doc/presentations
doc/tutorials
doc/whitepapers
lang
photran
rdt/org.eclipse.ptp.rdt.managedbuilder.gnu.ui
rdt/org.eclipse.ptp.rdt.releng
rdt/org.eclipse.ptp.rdt.services
rdt/org.eclipse.ptp.rdt.services.test
rdt/org.eclipse.ptp.rdt.sync.rsync.core
rdt/org.eclipse.ptp.rdt.sync.rsync.ui
rdt/org.eclipse.ptp.rdt.ui.tests
rdt/org.eclipse.ptp.rdt.xlc
releng/org.eclipse.ptp.isp-feature
releng/org.eclipse.ptp.license-feature
releng/org.eclipse.ptp.package-feature
releng/org.eclipse.ptp.perf-feature
releng/org.eclipse.ptp.perf.tau-feature
releng/org.eclipse.ptp.perf.tau.fortran-feature
releng/org.eclipse.ptp.rdt.package-feature
releng/org.eclipse.ptp.rdt.sdk-feature
releng/org.eclipse.ptp.rdt.xlc.sdk-feature
releng/org.eclipse.ptp.rm.ibm.bluegene-feature
releng/org.eclipse.ptp.rm.lsf-feature
releng/org.eclipse.ptp.rm.openmpi-feature
releng/org.eclipse.ptp.rm.orte-feature
releng/org.eclipse.ptp.rm.pbs-feature
releng/ptp_project_sets/anonymous
releng/ptp_project_sets/committer
rms/org.eclipse.ptp.rm.ibm.bluegene.core
rms/org.eclipse.ptp.rm.ibm.bluegene.proxy
rms/org.eclipse.ptp.rm.ibm.bluegene.ui
rms/org.eclipse.ptp.rm.jaxb.ui.monitor
rms/org.eclipse.ptp.rm.lsf.core
rms/org.eclipse.ptp.rm.lsf.proxy
rms/org.eclipse.ptp.rm.lsf.ui
rms/org.eclipse.ptp.rm.mpich2.core
rms/org.eclipse.ptp.rm.mpich2.proxy
rms/org.eclipse.ptp.rm.mpich2.ui
rms/org.eclipse.ptp.rm.orte.core
rms/org.eclipse.ptp.rm.orte.proxy
rms/org.eclipse.ptp.rm.orte.ui
rms/org.eclipse.ptp.rm.pbs.core
rms/org.eclipse.ptp.rm.pbs.core.server
rms/org.eclipse.ptp.rm.pbs.help
rms/org.eclipse.ptp.rm.pbs.jproxy
rms/org.eclipse.ptp.rm.pbs.proxy
rms/org.eclipse.ptp.rm.pbs.ui
rms/org.eclipse.ptp.rm.smoa.core
rms/org.eclipse.ptp.rm.smoa.ui
tools/cell
tools/etfw/org.eclipse.ptp.etfw.feedback.sample
tools/etfw/org.eclipse.ptp.etfw.tau.jars
tools/isp/org.eclipse.ptp.isp
tools/isp/org.eclipse.ptp.isp.help
tools/org.eclipse.ptp.pldt.tests
tools/perf
tools/sci/org.eclipse.ptp.sci
tools/tau
updates/org.eclipse.ptp-update

Photran we said we want to put in a separate repository. So this is easy.

But what to do with the rest? What is still needed and what is obsolete? And if it is still needed should it be in the main repository or in a separate one?

Sizes (all without "old" folders):
Workspace (without history) complete: 189MB
Only files in ptp-HEAD-all-committer.psf: 71MB
Photran: 38MB
Rest: 80MB

Since the rest is so big, it is important to clean that up prior to converting. (With Git the developer has to have the whole repository and thus the "rest" would increase e.g. download-time significantly)
Comment 11 Jeffrey Overbey CLA 2011-06-17 19:44:03 EDT
Roland, you said that you must always check out *all* the projects in a repository, right?  If so, (I'm not sure how others feel about this, but) it might be helpful to have PLDT in a separate repository as well.  We're working on integrating more Fortran MPI programming features (some of which will have to end up in PLDT, not Photran), so my day-to-day workspace contains Photran and PLDT but not the rest of PTP (to keep things small and speedy).

I think on the call there was also mention of putting RDT in a separate repo (?)
Comment 12 Albert L. Rossi CLA 2011-06-17 20:58:06 EDT
(In reply to comment #10)
> The following folders are in the CVS repository but not in
> releng/ptp_project_sets/committer/ptp-HEAD-all-committer.psf
> 
> core/org.eclipse.ptp.proxy.tests
> core/org.eclipse.ptp.rm.remote
> debug/org.eclipse.ptp.debug.sdm.x10
> doc/presentations
> doc/tutorials
> doc/whitepapers
> lang
> photran
> rdt/org.eclipse.ptp.rdt.managedbuilder.gnu.ui
> rdt/org.eclipse.ptp.rdt.releng
> rdt/org.eclipse.ptp.rdt.services
> rdt/org.eclipse.ptp.rdt.services.test
> rdt/org.eclipse.ptp.rdt.sync.rsync.core
> rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> rdt/org.eclipse.ptp.rdt.ui.tests
> rdt/org.eclipse.ptp.rdt.xlc
> releng/org.eclipse.ptp.isp-feature
> releng/org.eclipse.ptp.license-feature
> releng/org.eclipse.ptp.package-feature
> releng/org.eclipse.ptp.perf-feature
> releng/org.eclipse.ptp.perf.tau-feature
> releng/org.eclipse.ptp.perf.tau.fortran-feature
> releng/org.eclipse.ptp.rdt.package-feature
> releng/org.eclipse.ptp.rdt.sdk-feature
> releng/org.eclipse.ptp.rdt.xlc.sdk-feature
> releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> releng/org.eclipse.ptp.rm.lsf-feature
> releng/org.eclipse.ptp.rm.openmpi-feature
> releng/org.eclipse.ptp.rm.orte-feature
> releng/org.eclipse.ptp.rm.pbs-feature
> releng/ptp_project_sets/anonymous
> releng/ptp_project_sets/committer
> rms/org.eclipse.ptp.rm.ibm.bluegene.core
> rms/org.eclipse.ptp.rm.ibm.bluegene.proxy
> rms/org.eclipse.ptp.rm.ibm.bluegene.ui
> rms/org.eclipse.ptp.rm.lsf.core
> rms/org.eclipse.ptp.rm.lsf.proxy
> rms/org.eclipse.ptp.rm.lsf.ui
> rms/org.eclipse.ptp.rm.mpich2.core
> rms/org.eclipse.ptp.rm.mpich2.proxy
> rms/org.eclipse.ptp.rm.mpich2.ui
> rms/org.eclipse.ptp.rm.orte.core
> rms/org.eclipse.ptp.rm.orte.proxy
> rms/org.eclipse.ptp.rm.orte.ui


> But what to do with the rest? What is still needed and what is obsolete? 


> rms/org.eclipse.ptp.rm.pbs.core
> rms/org.eclipse.ptp.rm.pbs.core.server
> rms/org.eclipse.ptp.rm.pbs.help
> rms/org.eclipse.ptp.rm.pbs.jproxy
> rms/org.eclipse.ptp.rm.pbs.proxy
> rms/org.eclipse.ptp.rm.pbs.ui=
> rms/org.eclipse.ptp.rm.jaxb.ui.monitor

These can definitely be "archived".

Al
Comment 13 Roland Schulz CLA 2011-06-18 04:45:00 EDT
the largest files (including history and only within files part of ptp-HEAD-all-committer.psf):

507703 ./doc/org.eclipse.ptp.doc.user/html/images/03launch5.png,v
512943 ./releng/org.eclipse.ptp.macosx.x86/bin/Attic/sdm,v
542416 ./rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/09Output.png,v
551003 ./rms/org.eclipse.ptp.rm.ibm.pe.proxy/configure,v
574993 ./rms/org.eclipse.ptp.rm.ibm.pe.proxy/src/ptp_ibmpe_proxy.c,v
724277 ./releng/org.eclipse.ptp.pldt-feature/Attic/org.eclipse.ptp.pldt-1.0.0-I200607181525.zip,v
724311 ./tools/etfw/org.eclipse.ptp.etfw.feedback/html/images/Attic/feedback-wbench1.png,v
740816 ./rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/Attic/08editTemplate.jpg,v
760472 ./tools/gem/org.eclipse.ptp.gem.help/html/images/browserIntegration.png,v
871258 ./core/org.eclipse.ptp.proxy/configure,v
887749 ./core/org.eclipse.ptp.utils/configure,v
965411 ./tools/gem/org.eclipse.ptp.gem.help/html/images/consoleIntegration.png,v
1174987 ./debug/org.eclipse.ptp.debug.sdm/configure,v
1180062 ./tools/gem/org.eclipse.ptp.gem.help/html/images/analyzerIntegration.png,v
4959450 ./rdt/org.eclipse.ptp.rdt.server.dstore/Attic/rdt-server.jar,v
Comment 14 Roland Schulz CLA 2011-06-18 05:19:12 EDT
(In reply to comment #11)
> Roland, you said that you must always check out *all* the projects in a
> repository, right? 
You have to download the full repository. You don't have to import them all in your workspace. But if you want to have less than all you have to currently manually select which projects you want. The support for project sets is still work in progress (see 296082 and http://wiki.eclipse.org/EGit/User_Guide/Project_Set_File).

Thus the issues with one large repository are:
- Download/Storage size (includes full history of all files)
- Convenience if we rely on developers to manual select required projects

Issues with many small repositories:
- You can only download from one repository at a time. Thus it requires several steps to setup workspace if required projects are in different repositories
- When moving files from one repository to another it is difficult(/impossible?) to keep commit history 
- One commit can only affect one repository. Thus if your work affects projects in two different repositories you need to commit in two steps. Also the logical connection between the two commits gets lost to git and one can't see that later in the log. (This is still better than it would currently be in CVS. One advantage of Git is that it uses snapshots and not history per files. But by splitting repositories you split these snapshots and thus loose some of this advantage).

Thus we need to find the repository size which is the best balance between the advantages/disadvantages of a large repository. 

CDT (after contemplating making many small repositories) will have only one main repository and one small repository with one optional feature (EDC). 


> If so, (I'm not sure how others feel about this, but) it
> might be helpful to have PLDT in a separate repository as well.  We're working
> on integrating more Fortran MPI programming features (some of which will have
> to end up in PLDT, not Photran), so my day-to-day workspace contains Photran
> and PLDT but not the rest of PTP (to keep things small and speedy).

It might make sense to add PLDT to the Photran repository (which could be called "Photran+PLDT" or "PTP-Extras" or something else). Because as far as I know PLDT depends on Photran at compile time. Thus I assume anyone working on PLDT anyhow has PLDT checked out. But of course you don't need PLDT do work on Photran. So I don't know whether you would like this.

> I think on the call there was also mention of putting RDT in a separate repo
> (?)
I remember Beth said that. But I didn't understand whether she meant that Photran should be separate from RDT or RDT also from the rest/main of PTP. RDT (I mean all projects called org.eclipse.ptp.rdt.*) depends on: org.eclipse.ptp.remote.core, org.eclipse.ptp.remote.ui, org.eclipse.ptp.services.core, org.eclipse.ptp.services.ui.

But those projects are also required for other PTP projects. Thus if we want to split PTP (meaning here ptp-HEAD-all-committer.psf) into more than one repository we would need to split it similar to the folders i.e.: core, debug, rdt, rms, and tools. I tend to think that this would be too many repositories for the disadvantages mentioned above.
Comment 15 Roland Schulz CLA 2011-06-18 11:59:21 EDT
I've uploaded a test repository containing all projects in ptp-HEAD-all-committer.psf to:
git://github.com/rolandschulz/PTP_All_1.git

This can be used to test all git steps: clone, switch branches, commit ...

To import into eclipse:
- start with a new clean workspace (not required but recommended)
- add new software repository http://download.eclipse.org/releases/staging (egit had trouble with their RC3 so it is recommended to currently download from staging, later this will not be required)
- install egit
- file-import->"projects from git"
- paste the above URL, all other options can be left as is

The repository is also available for behind proxies at https://github.com/rolandschulz/PTP_All_1.git and can be viewed with a web-front-end at: https://github.com/rolandschulz/PTP_All_1.
Comment 16 Beth Tibbitts CLA 2011-06-18 16:46:24 EDT
OK i'm trying this but I'm a git neophyte. :)
paste the url where? Into first field and then do i need to hit the "Clone..." button?
I assumed yes, and kept hitting next to accept all defaults,
now i'll let it run for a while...
Comment 17 Roland Schulz CLA 2011-06-18 17:01:41 EDT
(In reply to comment #16)
> OK i'm trying this but I'm a git neophyte. :)
> paste the url where? Into first field and then do i need to hit the "Clone..."
> button?
yes

> I assumed yes, and kept hitting next to accept all defaults,
> now i'll let it run for a while...
it takes a while initially (some Egit performance bugs have already been filed). After the initial clone everything should be faster (all local).

Only the next update will be slow - because I just found a bug in the conversion and I updated most of the history.
Comment 18 Beth Tibbitts CLA 2011-06-18 19:21:25 EDT
OK it finished but i see nothing in my workspace.  I see in <home>/git/ the tree.
What do I do next?
Consider this the beginning of a wiki page to teach us all how to use git for PTP :)
Comment 19 Beth Tibbitts CLA 2011-06-18 19:24:25 EDT
Oh.  File>Import... Git/Projects from Git
Select the repo that is now in the pane: PTP_All_1
Next,next,Finish seems to get importing them all.
Comment 20 Roland Schulz CLA 2011-06-19 04:09:06 EDT
(In reply to comment #19)
> Oh.  File>Import... Git/Projects from Git
> Select the repo that is now in the pane: PTP_All_1
> Next,next,Finish seems to get importing them all.

This extra step should not be required. As one of the pages of the clone wizard you should have seen http://wiki.eclipse.org/EGit/User_Guide/Remote#Import_Projects. Did you not see this? Or did you not select any project there?

But you should now have it setup correct. Just in 2 steps instead of all within one wizard.

See http://wiki.eclipse.org/EGit/User_Guide (3.4-3.8,3.11-3.12) for explanation of how to do standard tasks with egit and the test repository.
Comment 21 Beth Tibbitts CLA 2011-06-20 09:30:54 EDT
(In reply to comment #10)
> The following folders are in the CVS repository but not ...

> tools/etfw/org.eclipse.ptp.etfw.feedback.sample
> tools/org.eclipse.ptp.pldt.tests

The two above projects are still active, just not included in ptp builds. feedback.sample is used on the wiki page and in the help to show users how to set up a feedback plugin, since the build doesn't include any implementations.
Comment 22 Greg Watson CLA 2011-06-21 18:05:51 EDT
(In reply to comment #10)
> The following folders are in the CVS repository but not in
> releng/ptp_project_sets/committer/ptp-HEAD-all-committer.psf
> 
> core/org.eclipse.ptp.proxy.tests
> core/org.eclipse.ptp.rm.remote
No longer used.

> debug/org.eclipse.ptp.debug.sdm.x10
This was missing.

> doc/presentations
> doc/tutorials
> doc/whitepapers
Docs, not needed.

> lang
No longer used.

> photran
Photran is separate from PTP at the moment.

> rdt/org.eclipse.ptp.rdt.managedbuilder.gnu.ui
This was missing.

> rdt/org.eclipse.ptp.rdt.releng
> rdt/org.eclipse.ptp.rdt.services
> rdt/org.eclipse.ptp.rdt.services.test
> rdt/org.eclipse.ptp.rdt.sync.rsync.core
> rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> rdt/org.eclipse.ptp.rdt.ui.tests
> rdt/org.eclipse.ptp.rdt.xlc
These are old.

> releng/org.eclipse.ptp.isp-feature
Old.

> releng/org.eclipse.ptp.license-feature
This was missing.

> releng/org.eclipse.ptp.package-feature
> releng/org.eclipse.ptp.perf-feature
> releng/org.eclipse.ptp.perf.tau-feature
> releng/org.eclipse.ptp.perf.tau.fortran-feature
> releng/org.eclipse.ptp.rdt.package-feature
> releng/org.eclipse.ptp.rdt.sdk-feature
> releng/org.eclipse.ptp.rdt.xlc.sdk-feature
> releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> releng/org.eclipse.ptp.rm.lsf-feature
> releng/org.eclipse.ptp.rm.openmpi-feature
> releng/org.eclipse.ptp.rm.orte-feature
> releng/org.eclipse.ptp.rm.pbs-feature
None of these are used.

> releng/ptp_project_sets/anonymous
> releng/ptp_project_sets/committer
Obvious.

> rms/org.eclipse.ptp.rm.ibm.bluegene.core
> rms/org.eclipse.ptp.rm.ibm.bluegene.proxy
> rms/org.eclipse.ptp.rm.ibm.bluegene.ui
> rms/org.eclipse.ptp.rm.jaxb.ui.monitor
> rms/org.eclipse.ptp.rm.lsf.core
> rms/org.eclipse.ptp.rm.lsf.proxy
> rms/org.eclipse.ptp.rm.lsf.ui
> rms/org.eclipse.ptp.rm.mpich2.core
> rms/org.eclipse.ptp.rm.mpich2.proxy
> rms/org.eclipse.ptp.rm.mpich2.ui
> rms/org.eclipse.ptp.rm.orte.core
> rms/org.eclipse.ptp.rm.orte.proxy
> rms/org.eclipse.ptp.rm.orte.ui
> rms/org.eclipse.ptp.rm.pbs.core
> rms/org.eclipse.ptp.rm.pbs.core.server
> rms/org.eclipse.ptp.rm.pbs.help
> rms/org.eclipse.ptp.rm.pbs.jproxy
> rms/org.eclipse.ptp.rm.pbs.proxy
> rms/org.eclipse.ptp.rm.pbs.ui
> rms/org.eclipse.ptp.rm.smoa.core
> rms/org.eclipse.ptp.rm.smoa.ui
> tools/cell
None of these are used.

> tools/etfw/org.eclipse.ptp.etfw.feedback.sample
I've added this.

> tools/etfw/org.eclipse.ptp.etfw.tau.jars
> tools/isp/org.eclipse.ptp.isp
> tools/isp/org.eclipse.ptp.isp.help
None of these are used.

> tools/org.eclipse.ptp.pldt.tests
I've added this.

> tools/perf
No longer used.

> tools/sci/org.eclipse.ptp.sci
I've added this.

> tools/tau
I have to find out from Wyatt which of these are still being used.

> updates/org.eclipse.ptp-update
No longer used.
Comment 23 Greg Watson CLA 2011-06-21 18:18:21 EDT
(In reply to comment #10)
A few more:

> releng/org.eclipse.ptp.rm.openmpi-feature
> rdt/org.eclipse.ptp.rdt.ui.tests
These were actually missing.

> rms/org.eclipse.ptp.rm.pbs.core
> rms/org.eclipse.ptp.rm.pbs.core.server
> rms/org.eclipse.ptp.rm.pbs.help
> rms/org.eclipse.ptp.rm.pbs.jproxy
> rms/org.eclipse.ptp.rm.pbs.proxy
> rms/org.eclipse.ptp.rm.pbs.ui
These are actually used in ptp_4_0 so must be included.
Comment 24 Roland Schulz CLA 2011-06-21 18:39:17 EDT
(In reply to comment #22)
> (In reply to comment #10)

> > doc/presentations
> > doc/tutorials
> > doc/whitepapers
> Docs, not needed.

Do you mean, it is not needed in workspace, or not needed at all? If the later, should we create a separate doc repository?
Comment 25 Greg Watson CLA 2011-06-21 19:20:21 EDT
(In reply to comment #0)
> Open questions / TODOs:
> - On the day of the conversion we need a current tools-cvs.tgz (at least the
> org.eclipse.ptp part). We need to get access to these files or ask the webmaster
> to generate the archive for us on that day.

If it's just a tar of /cvsroot/tools/org.eclipse.ptp, I can do that.

> - CDT has uploaded their test repository to http://git.eclipse.org. If we want
> to do that too we need to ask for permissions.

I've asked for a repo to be created.

> - Do we want to switch the build system (e.g. to Maven) at the same time? We
> first need to decide this before we can work-out our build with GIT. How CDT is
> doing the build with GIT is described in bug 316212.

I think we'll just transition the current PDE over. Switching to maven is a lot of work.

> - Do we want to split the repository into more parts than PTP and photran?

Do you know what CDT have done?
Comment 26 Greg Watson CLA 2011-06-21 19:21:19 EDT
(In reply to comment #24)
> (In reply to comment #22)
> > (In reply to comment #10)
> 
> > > doc/presentations
> > > doc/tutorials
> > > doc/whitepapers
> > Docs, not needed.
> 
> Do you mean, it is not needed in workspace, or not needed at all? If the later,
> should we create a separate doc repository?

Not needed in the workspace. We could put these in a separate repository.
Comment 27 Roland Schulz CLA 2011-06-21 21:51:55 EDT
(In reply to comment #25)
> (In reply to comment #0)
> > Open questions / TODOs:
> > - On the day of the conversion we need a current tools-cvs.tgz (at least the
> > org.eclipse.ptp part). We need to get access to these files or ask the webmaster
> > to generate the archive for us on that day.
> 
> If it's just a tar of /cvsroot/tools/org.eclipse.ptp, I can do that.
Yes. Just that.

> > - Do we want to split the repository into more parts than PTP and photran?
> 
> Do you know what CDT have done?

They moved EDC into its own repository. Everything else is in one repository.
Comment 28 Greg Watson CLA 2011-06-22 08:58:23 EDT
(In reply to comment #27)
> > Do you know what CDT have done?
> 
> They moved EDC into its own repository. Everything else is in one repository.

Photran is pretty independent, so it should probably go in its own repository.

Ignoring photran, I think the only other logical separation would be RDT. However RDT depends on ptp.remote.* and ptp.services.*, so having two repositories (RDT and everything else) would only benefit non-RDT developers since you'd still need both to work on RDT. I think the options therefore are:

a) 1 repo: everything
b) 3 repos: RDT, {remote,services}.*, everything else
Comment 29 Roland Schulz CLA 2011-06-30 20:44:44 EDT
(In reply to comment #28)
> (In reply to comment #27)
> > > Do you know what CDT have done?
> > 
> > They moved EDC into its own repository. Everything else is in one repository.
> 
> Photran is pretty independent, so it should probably go in its own repository.

PLDT depends on Photran for building (not for running). Thus I assume you always want to have Photran in your workspace when working on PLDT. Does it make sense to have a "Language" repository which includes PLDT and Photran? And PLDT doesn't seem to depend on anything from PTP.
 
> Ignoring photran, I think the only other logical separation would be RDT.
> However RDT depends on ptp.remote.* and ptp.services.*, so having two
> repositories (RDT and everything else) would only benefit non-RDT developers
> since you'd still need both to work on RDT. I think the options therefore are:
> 
> a) 1 repo: everything
> b) 3 repos: RDT, {remote,services}.*, everything else
I think the repository is not that huge so I would personally prefer a). But I think both are good options.

One additional option would be to say that we have one main repo and one (or a few) extra repo(s) for all parts which have external dependencies. This would be:
- debug or just debug.sdm (because of C)
- pldt & rdt.sync.ui.fortran(Photran)
- rdt.core.tests & rdt.ui.tests (CDT Test)
- rm.jaxb & rm.lml (Jaxb)
- rm.slurm & rm.ibm & sci & util (C)

Than one could get all projects from the base repository without installing additional dependency and one would not get any compile errors. This approach would not save disk/network-bandwidth (because most people would need the extra repository). But it would remove the current problem that after checking out all project one first has to close all projects which unfulfilled dependencies to get the error view to 0 errors.

Regarding the timeline: 
CDT switched June 24th. This would mean we switch around July 24th according to the proposed timeline (~1month after CDT). Maybe we can set a date on the next phone call - it seems there are few outstanding issues.
Comment 30 Beth Tibbitts CLA 2011-06-30 23:33:26 EDT
> Thus I assume you always want to have Photran in your workspace when working on PLDT.
Actually, no. I usually don't.  I usually install photran in the workbench, unless i want to live
on the bleeding edge of something Jeff is doing.
But I realize the git fences may make this complicated. So I'm willing to adjust how I do things. :)
Comment 31 Roland Schulz CLA 2011-07-01 00:01:04 EDT
(In reply to comment #30)
> > Thus I assume you always want to have Photran in your workspace when working on PLDT.
> Actually, no. I usually don't.  I usually install photran in the workbench,
> unless i want to live
> on the bleeding edge of something Jeff is doing.
> But I realize the git fences may make this complicated. So I'm willing to
> adjust how I do things. :)
I didn't think about that option. There is no need to put Photran/PLDT in one repo.
Comment 32 Roland Schulz CLA 2011-07-01 16:58:48 EDT
I have added a wiki page at http://wiki.eclipse.org/PTP/environment_setup/git. So far it includes only links to documentation. What else do we need? Do we need our own screencasts or documentation? Those who have tried the test repo, what are the open questions?
Comment 33 Roland Schulz CLA 2011-10-18 16:51:40 EDT
Latest list of directories not in project set (for HEAD). It includes some additional folders I missed last time, and reflects the changes to the project_set since the last update. It includes those already answered (with the answer) so that we have one full list of all folders.

> core/org.eclipse.ptp.proxy.tests
> core/org.eclipse.ptp.rm.remote
Greg wrote before these can be archived. Do we want to move folders which can be archived in the CVS? Or should I simply not include them in the GIT repository?

> doc
As discussed this should be its own repository

> doc/org.eclipse.ptp.help
Is this the documentation plugin? If so we said this should be part of the main repository (as the only exception of doc). What about "org.eclipse.ptp.doc.isv" and "org.eclipse.ptp.doc.user"? These are in the project-set. I suppose they should go into the main repository and not the doc repository, correct?

> lang/*
?

> photran/org.eclipse.photran.cmdline
> photran/org.eclipse.photran.core.vpg.preprocessor.c.tests
> photran/org.eclipse.photran.core.vpg.tests
> photran/org.eclipse.photran.core.vpg.tests.failing
> photran/org.eclipse.photran-dev-docs
> photran/org.eclipse.photran-feature
> photran/org.eclipse.photran.intel-feature
> photran/org.eclipse.photran.master
> photran/org.eclipse.photran.package-feature
> photran/org.eclipse.photran.releng
> photran/org.eclipse.photran-samples
> photran/org.eclipse.photran.vpg-feature
> photran/org.eclipse.photran.xlf-feature
> photran/org.eclipse.rephraserengine.core.vpg.tests
> photran/org.eclipse.rephraserengine.doc.isv
> photran/org.eclipse.rephraserengine.examples.testing
> photran/org.eclipse.rephraserengine.examples.text
> photran/org.eclipse.rephraserengine-feature
> photran/org.eclipse.rephraserengine-support
> photran/org.eclipse.rephraserengine.testing.junit3
My last list didn't include a comparison of photran. I think at that time the project-set didn't include photran. Are these projects still all required? Should they be all part of the photran git repository?

> rdt/org.eclipse.ptp.rdt.releng
> rdt/org.eclipse.ptp.rdt.services
> rdt/org.eclipse.ptp.rdt.services.test
> rdt/org.eclipse.ptp.rdt.sync.core.tests
> rdt/org.eclipse.ptp.rdt.sync.rsync.core
> rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> rdt/org.eclipse.ptp.rdt.xlc
Greg wrote these are old

> releng/org.eclipse.ptp.isp-feature
Greg: Old

> releng/org.eclipse.ptp.package-feature
> releng/org.eclipse.ptp.perf-feature
> releng/org.eclipse.ptp.perf.tau-feature
> releng/org.eclipse.ptp.perf.tau.fortran-feature
> releng/org.eclipse.ptp.rdt.package-feature
> releng/org.eclipse.ptp.rdt.sdk-feature
> releng/org.eclipse.ptp.rdt.xlc.sdk-feature
Greg: unused

> releng/org.eclipse.ptp.repo
?

> releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> releng/org.eclipse.ptp.rm.lsf-feature
> releng/org.eclipse.ptp.rm.orte-feature
> releng/org.eclipse.ptp.rm.pbs-feature
Greg: unused

> releng/ptp_project_sets
They are not needed anymore with GIT. Should I just leave it out?

> rms/org.eclipse.ptp.rm.ibm.bluegene.core
> rms/org.eclipse.ptp.rm.ibm.bluegene.proxy
> rms/org.eclipse.ptp.rm.ibm.bluegene.ui
Greg: unused

> rms/org.eclipse.ptp.rm.jaxb.openmpi
?

> rms/org.eclipse.ptp.rm.jaxb.ui.monitor
Greg: unused

> rms/org.eclipse.ptp.rm.llview.core
> rms/org.eclipse.ptp.rm.llview.ui
> rms/org.eclipse.ptp.rm.lml_jaxb.core
> rms/org.eclipse.ptp.rm.lml_jaxb.ui
?

> rms/org.eclipse.ptp.rm.lsf.core
> rms/org.eclipse.ptp.rm.lsf.proxy
> rms/org.eclipse.ptp.rm.lsf.ui
> rms/org.eclipse.ptp.rm.mpich2.core
> rms/org.eclipse.ptp.rm.mpich2.proxy
> rms/org.eclipse.ptp.rm.mpich2.ui
> rms/org.eclipse.ptp.rm.orte.core
> rms/org.eclipse.ptp.rm.orte.proxy
> rms/org.eclipse.ptp.rm.orte.ui
> rms/org.eclipse.ptp.rm.pbs.core
> rms/org.eclipse.ptp.rm.pbs.core.server
> rms/org.eclipse.ptp.rm.pbs.help
> rms/org.eclipse.ptp.rm.pbs.jproxy
> rms/org.eclipse.ptp.rm.pbs.proxy
> rms/org.eclipse.ptp.rm.pbs.ui
> rms/org.eclipse.ptp.rm.smoa.core
> rms/org.eclipse.ptp.rm.smoa.ui
Greg: unused


> tools/cell/* 
Greg: unused

> tools/etfw/org.eclipse.ptp.etfw.tau.jars
> tools/isp/org.eclipse.ptp.isp
> tools/isp/org.eclipse.ptp.isp.help
Greg: unused

> tools/org.eclipse.ptp.pldt.common
> tools/org.eclipse.ptp.pldt.doc.user
> tools/org.eclipse.ptp.pldt.help
> tools/org.eclipse.ptp.pldt.lapi
> tools/org.eclipse.ptp.pldt.mpi.analysis
> tools/org.eclipse.ptp.pldt.mpi.analysis.cdt
> tools/org.eclipse.ptp.pldt.mpi.core
> tools/org.eclipse.ptp.pldt.openmp.analysis
> tools/org.eclipse.ptp.pldt.openmp.core
> tools/org.eclipse.ptp.pldt.openmp.ui.pv
> tools/org.eclipse.ptp.pldt.tests
> tools/org.eclipse.ptp.pldt.upc
> tools/org.eclipse.ptp.pldt.wizards
?

> tools/org.eclipse.ptp.remotetools.core
> tools/org.eclipse.ptp.remotetools.environment.core
> tools/org.eclipse.ptp.remotetools.environment.generichost
> tools/org.eclipse.ptp.remotetools.environment.launcher
> tools/org.eclipse.ptp.remotetools.environment.ui
> tools/org.eclipse.ptp.remotetools.preferences
> tools/org.eclipse.ptp.remotetools.utils
?

> tools/perf/*
Greg: unused

> tools/samples/org.eclipse.ptp.pldt.sampleCDTstaticAnalysis
?

> tools/tau/org.eclipse.ptp.tau-feature
> tools/tau/org.eclipse.ptp.tau.jars
> tools/tau/org.eclipse.ptp.tau.managedbuilder.other.photran.ui
> tools/tau/org.eclipse.ptp.tau.managedbuilder.other.ui
> tools/tau/org.eclipse.ptp.tau.papiselect
> tools/tau/org.eclipse.ptp.tau.perfdmf
> tools/tau/org.eclipse.ptp.tau.performance
> tools/tau/org.eclipse.ptp.tau.selinst
> tools/tau/org.eclipse.ptp.tau.selinstfort
> tools/tau/org.eclipse.ptp.tau.toolopts
> tools/tau/org.eclpse.ptp.tau.performance.parallel
?

> updates/org.eclipse.ptp-update
Greg: unused
Comment 34 Jeffrey Overbey CLA 2011-10-18 17:07:22 EDT
> photran/org.eclipse.photran.cmdline
> photran/org.eclipse.photran.vpg-feature

We can archive these two; they are no longer used.  I think a couple of Photran's releng projects can be archived too, but Greg would have to tell you for sure.

I want to eliminate/consolidate a few more of Photran's projects, but I wasn't planning to do that right away.  I could do that after the move to Git if necessary, right?  Deleting projects from CVS wasn't handled particularly well, but I think deleting projects from a Git repository is much easier... is that right?
Comment 35 Roland Schulz CLA 2011-10-18 17:28:10 EDT
(In reply to comment #34)

> I want to eliminate/consolidate a few more of Photran's projects, but I wasn't
> planning to do that right away.  I could do that after the move to Git if
> necessary, right?  Deleting projects from CVS wasn't handled particularly well,
> but I think deleting projects from a Git repository is much easier... is that
> right?

Yes it is possible to delete files/folders/projects after everything has been converted to Git. Because Git requires the user to once download the whole repository (including the whole history), deleting files after the conversion has the disadvantage that the repository/download size is larger than it would be if files were delete before. But unless some of the folders are very large, I don't think we need to wait on you to consolidate the projects.

In theory it is possible to fully erase files from the repository (including their whole history), so that they don't take up any space anymore. But that requires extra work from every developer to resync to the repository after it was changed/cleaned. So this should be avoided.
Comment 36 Roland Schulz CLA 2011-10-18 19:03:37 EDT
Greg asked me on the call to describe the conversion process. I suggest we:
1) Finalize the list of project which should be part of the main repository
2) I will than create a test repository for the main PTP repository, Photran and Doc using the cvs2git command described before
3) We than test that the repository works as expected. If everything is OK:
4) We make CVS readonly and create the official/final set of GIT repositories.

Another question I have:
Should the folder hierarchy be as it currently is in the workspace or as it currently is as on the CVS server? In the workspace all projects are directly in one folder and on the CVS server we have the subfolders (rdt, core, ...) which in turn contain the folders. Do we want this additional layer of folders?

With GIT the two folder layouts have to be the same. If we keep that additional layer of folders we need to update a few scripts (e.g. org.eclipse.ptp.linux.x86_64/BUILD) because these scripts expect all projects to reside in the same base folder. 

Here is the list of the largest (non-image) files (mostly sorted by size only change a bit to group).

6.7M ./rms/org.eclipse.ptp.rm.jaxb.help/html/images/JAXBDemo.pdf
6.7M ./rms/org.eclipse.ptp.rm.jaxb.help/html/JAXBDemo.pdf
why twice? and can this moved to e.g. the wiki?

3.3M ./photran/org.eclipse.photran.cmdline/distrib/photran-cmdline.jar
needed?

542K ./photran/org.eclipse.photran-dev-docs/contributions/find-references/README.pdf
495K ./photran/org.eclipse.photran-dev-docs/contributions/find-references/README.odt
397K ./photran/org.eclipse.photran-dev-docs/dev-guide/dev-guide-general.pdf
259K ./photran/org.eclipse.photran-dev-docs/dev-guide/dev-guide-specialized.pdf
302K ./photran/org.eclipse.photran-dev-docs/dev-guide/images/diagrams/photran_architecture.graffle
should this be in GIT? As mentioned on the call a relative large binary file which is often updated makes the repository grow quickly.
if at all shouldn't be as both pdf and odt

540K ./tools/sci/org.eclipse.ptp.sci/configure
311K ./tools/sci/org.eclipse.ptp.sci/aclocal.m4
238K ./tools/sci/org.eclipse.ptp.sci/ltmain.sh
why is that configure so much bigger than the others? Shouldn't the configure be created by a bootstrap script?

2.1M ./photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/SemanticActions.java
1.8M ./lang/org.eclipse.ptp.lang.fortran.core/src/org/eclipse/ptp/lang/fortran/core/parser/FortranParser.java
510K ./photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/Parser.java
284K ./photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/ExecutableProgramParsingTables.java
247K ./lang/org.eclipse.ptp.lang.fortran.core/src/org/eclipse/ptp/lang/fortran/core/parser/FortranLexer.java
242K ./photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/BodyParsingTables.java
221K ./lang/org.eclipse.fdt.core/parser/org/eclipse/fdt/internal/core/parser/FortranParser.java
These are some really large java files ;-)

426K ./debug/org.eclipse.ptp.debug.sdm/libaif/patches/gdb-2002-08-11-cvs.patch
271K ./rms/org.eclipse.ptp.rm.ibm.pe.proxy/src/ptp_ibmpe_proxy.c
216K ./debug/org.eclipse.ptp.debug.sdm/libaif/configure
211K ./photran/org.eclipse.photran.doc.user/html/images/MoveSavedToCommonBlock.PNG
203K ./debug/org.eclipse.ptp.debug.sdm/configure


1.1M ./photran/org.eclipse.photran-dev-docs/dev-guide/images/photran_architecture.png
528K ./rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/09Output.png
528K ./rms/org.eclipse.ptp.rm.pbs.help/html/images/09Output.png

Some of the PNG images are not compressed very well. Some of them can be compressed more than 2x without any visible difference at all (PNG is lossless). The size of all PNG images in the project-set is currently 25M and can be compressed to 18M. 7M is not that much but it would also help the download for the end-user to get smaller. Do you want me to recompress the PNG files before converting to Git?
Comment 37 Roland Schulz CLA 2011-10-18 23:48:41 EDT
Actually my list of largest files is a bit misleading. Since GIT keeps the history it is more important how large a file including its history is than thus the latest version. Looking at the CVS file sizes is also misleading because GIT stores all files Compressed. Here is a list of the CVS files, sorted by their gzip compressed size (first column) and the uncompressed size (2nd column). 

22411403 28M rms/org.eclipse.ptp.rm.jaxb.help/html/JAXBDemo.pdf,v
7055013 9.2M photran/org.eclipse.photran.core/Attic/f95parser.jar,v
5610714 6.7M rms/org.eclipse.ptp.rm.jaxb.help/html/images/JAXBDemo.pdf,v
5103757 51M photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/Parser.java,v
4957651 4.8M rdt/org.eclipse.ptp.rdt.server.dstore/Attic/rdt-server.jar,v
1170840 1.2M tools/gem/org.eclipse.ptp.gem.help/html/images/analyzerIntegration.png,v
1070953 1.3M photran/org.eclipse.photran.core.vpg/Attic/vpg-eclipse.jar,v
994280 2.1M photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/ExecutableProgramParsingTables.java,v
957767 946K tools/gem/org.eclipse.ptp.gem.help/html/images/consoleIntegration.png,v
870706 4.0M tools/sci/org.eclipse.ptp.sci/configure,v
844092 1.8M photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/BodyParsingTables.java,v
793327 911K rms/org.eclipse.ptp.rm.jaxb.help/html/images/16composite.jpeg,v
755102 746K tools/gem/org.eclipse.ptp.gem.help/html/images/browserIntegration.png,v
731508 832K rms/org.eclipse.ptp.rm.jaxb.help/html/images/14tab-controller.jpeg,v
723516 708K releng/org.eclipse.ptp.pldt-feature/Attic/org.eclipse.ptp.pldt-1.0.0-I200607181525.zip,v
663831 709K tools/etfw/org.eclipse.ptp.etfw.feedback/html/images/Attic/feedback-wbench1.png,v
662298 724K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/Attic/08editTemplate.jpg,v
630491 811K rms/org.eclipse.ptp.rm.jaxb.help/html/images/02control.jpeg,v
542208 570K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/06Imported.png,v
524529 532K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/09Output.png,v
504739 546K photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.pdf,v
483057 503K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/02Basic.png,v
482897 499K doc/org.eclipse.ptp.doc.user/html/images/03launch5.png,v
481309 499K photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.odt,v
460043 484K doc/org.eclipse.ptp.doc.user/html/images/03launchResTab.png,v
436630 5.4M photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/ParsingTable.java,v
430685 457K doc/org.eclipse.ptp.doc.user/html/images/sync-projProperties.png,v
418617 439K doc/org.eclipse.ptp.doc.user/html/images/sync-newprojCreate.png,v
409368 437K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/Attic/14launchTabDetails.jpg,v
395118 408K doc/org.eclipse.ptp.doc.user/html/images/03launch2.png,v
393453 399K doc/org.eclipse.ptp.doc.user/html/images/Attic/04launch2.png,v
392919 396K doc/org.eclipse.ptp.doc.user/html/images/Attic/04launch5.png,v
379362 376K photran/org.eclipse.photran.doc.user/html/images/Browse
357671 350K doc/org.eclipse.ptp.doc.user/html/images/Attic/runtimePersp.gif,v
354066 356K tools/gem/org.eclipse.ptp.gem.help/html/images/gemPrefs.png,v
353469 352K tools/gem/org.eclipse.ptp.gem.help/html/images/ispPrefs.png,v
348075 396K doc/org.eclipse.ptp.doc.user/html/images/02ChooseResMgrNameAndDesc.png,v
347932 378K rms/org.eclipse.ptp.rm.jaxb.help/html/images/06Imported.png,v
343908 361K tools/pldt/org.eclipse.ptp.pldt.doc.user/html/images/mpiNotFound.png,v
332142 5.5M photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/Productions.java,v
331438 352K tools/etfw/org.eclipse.ptp.etfw.doc.user/html/images/feedback-wbench1.png,v
329609 346K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/00Preferences.png,v
325903 408K rms/org.eclipse.ptp.rm.jaxb.help/html/images/21attribute-viewer.jpeg,v
322206 1.3M tools/sci/org.eclipse.ptp.sci/aclocal.m4,v
320335 400K rms/org.eclipse.ptp.rm.jaxb.help/html/images/19widget.jpeg,v
319636 445K rms/org.eclipse.ptp.rm.jaxb.help/html/images/Table.jpg,v
319501 313K doc/org.eclipse.ptp.doc.user/html/images/Attic/ptpRunOview.gif,v
316770 310K doc/org.eclipse.ptp.doc.user/html/images/Attic/launch2.gif,v
316082 362K doc/org.eclipse.ptp.doc.user/html/images/02-RM-OpenMPI-Configuration.png,v
315813 345K doc/org.eclipse.ptp.doc.user/html/images/06ptpDebugPersp.png,v
309005 329K tools/pldt/org.eclipse.ptp.pldt.doc.user/html/images/cdtPrefsTemplates.png,v
305939 435K rms/org.eclipse.ptp.rm.ibm.ll.help/html/images/ibmll_run_basic_application_3.jpg,v
304873 426K rms/org.eclipse.ptp.rm.jaxb.help/html/images/Tree1.jpg,v
301826 296K doc/org.eclipse.ptp.doc.user/html/images/Attic/prefsOMPI.gif,v
301196 323K doc/org.eclipse.ptp.doc.user/html/images/05runtimePerspAnn.png,v

GIT stores the files different but I think this is a good estimate. The question is whether we need to keep the full history for these large files.

Also my idea of recompressing the images was only half thought through. If we recompress them and keep the history we actually increase the repository size (the repository than contains both version). So we would need to recompress and not keep the history. I'm not sure whether we want to do that.
Comment 38 Beth Tibbitts CLA 2011-10-19 08:43:45 EDT
> doc/org.eclipse.ptp.help
It's the old version of the ptp documentation plugin.
It was moved to org.eclipse.ptp.doc.user

doc.user and doc.isv are the currently-used ones
main repository, yes.  I think they should go in the project set but most folks probably don't build them.

>tools/org.eclipse.ptp.pldt.*
Everything in your list is still current

> tools/samples/org.eclipse.ptp.pldt.sampleCDTstaticAnalysis
This is a project with sample code that matches an Eclipsecon paper. I'd like to keep it. It's small.
Comment 39 Greg Watson CLA 2011-10-19 09:06:51 EDT
This list is out of date and does not reflect the latest repository contents.

Everything to be migrated (including photran) should be in their correct directories. Do not migrate any directories (or sub-directories) called 'old'.

I have moved the doc.user and doc.isv plugins to the core directory, so the doc directory can be migrated as a separate repo.

(In reply to comment #33)
> Latest list of directories not in project set (for HEAD). It includes some
> additional folders I missed last time, and reflects the changes to the
> project_set since the last update. It includes those already answered (with the
> answer) so that we have one full list of all folders.
> 
> > core/org.eclipse.ptp.proxy.tests
> > core/org.eclipse.ptp.rm.remote
> Greg wrote before these can be archived. Do we want to move folders which can be
> archived in the CVS? Or should I simply not include them in the GIT repository?
> 
> > doc
> As discussed this should be its own repository
> 
> > doc/org.eclipse.ptp.help
> Is this the documentation plugin? If so we said this should be part of the main
> repository (as the only exception of doc). What about "org.eclipse.ptp.doc.isv"
> and "org.eclipse.ptp.doc.user"? These are in the project-set. I suppose they
> should go into the main repository and not the doc repository, correct?
> 
> > lang/*
> ?
> 
> > photran/org.eclipse.photran.cmdline
> > photran/org.eclipse.photran.core.vpg.preprocessor.c.tests
> > photran/org.eclipse.photran.core.vpg.tests
> > photran/org.eclipse.photran.core.vpg.tests.failing
> > photran/org.eclipse.photran-dev-docs
> > photran/org.eclipse.photran-feature
> > photran/org.eclipse.photran.intel-feature
> > photran/org.eclipse.photran.master
> > photran/org.eclipse.photran.package-feature
> > photran/org.eclipse.photran.releng
> > photran/org.eclipse.photran-samples
> > photran/org.eclipse.photran.vpg-feature
> > photran/org.eclipse.photran.xlf-feature
> > photran/org.eclipse.rephraserengine.core.vpg.tests
> > photran/org.eclipse.rephraserengine.doc.isv
> > photran/org.eclipse.rephraserengine.examples.testing
> > photran/org.eclipse.rephraserengine.examples.text
> > photran/org.eclipse.rephraserengine-feature
> > photran/org.eclipse.rephraserengine-support
> > photran/org.eclipse.rephraserengine.testing.junit3
> My last list didn't include a comparison of photran. I think at that time the
> project-set didn't include photran. Are these projects still all required?
> Should they be all part of the photran git repository?
> 
> > rdt/org.eclipse.ptp.rdt.releng
> > rdt/org.eclipse.ptp.rdt.services
> > rdt/org.eclipse.ptp.rdt.services.test
> > rdt/org.eclipse.ptp.rdt.sync.core.tests
> > rdt/org.eclipse.ptp.rdt.sync.rsync.core
> > rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> > rdt/org.eclipse.ptp.rdt.xlc
> Greg wrote these are old
> 
> > releng/org.eclipse.ptp.isp-feature
> Greg: Old
> 
> > releng/org.eclipse.ptp.package-feature
> > releng/org.eclipse.ptp.perf-feature
> > releng/org.eclipse.ptp.perf.tau-feature
> > releng/org.eclipse.ptp.perf.tau.fortran-feature
> > releng/org.eclipse.ptp.rdt.package-feature
> > releng/org.eclipse.ptp.rdt.sdk-feature
> > releng/org.eclipse.ptp.rdt.xlc.sdk-feature
> Greg: unused
> 
> > releng/org.eclipse.ptp.repo
> ?
> 
> > releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> > releng/org.eclipse.ptp.rm.lsf-feature
> > releng/org.eclipse.ptp.rm.orte-feature
> > releng/org.eclipse.ptp.rm.pbs-feature
> Greg: unused
> 
> > releng/ptp_project_sets
> They are not needed anymore with GIT. Should I just leave it out?
> 
> > rms/org.eclipse.ptp.rm.ibm.bluegene.core
> > rms/org.eclipse.ptp.rm.ibm.bluegene.proxy
> > rms/org.eclipse.ptp.rm.ibm.bluegene.ui
> Greg: unused
> 
> > rms/org.eclipse.ptp.rm.jaxb.openmpi
> ?
> 
> > rms/org.eclipse.ptp.rm.jaxb.ui.monitor
> Greg: unused
> 
> > rms/org.eclipse.ptp.rm.llview.core
> > rms/org.eclipse.ptp.rm.llview.ui
> > rms/org.eclipse.ptp.rm.lml_jaxb.core
> > rms/org.eclipse.ptp.rm.lml_jaxb.ui
> ?
> 
> > rms/org.eclipse.ptp.rm.lsf.core
> > rms/org.eclipse.ptp.rm.lsf.proxy
> > rms/org.eclipse.ptp.rm.lsf.ui
> > rms/org.eclipse.ptp.rm.mpich2.core
> > rms/org.eclipse.ptp.rm.mpich2.proxy
> > rms/org.eclipse.ptp.rm.mpich2.ui
> > rms/org.eclipse.ptp.rm.orte.core
> > rms/org.eclipse.ptp.rm.orte.proxy
> > rms/org.eclipse.ptp.rm.orte.ui
> > rms/org.eclipse.ptp.rm.pbs.core
> > rms/org.eclipse.ptp.rm.pbs.core.server
> > rms/org.eclipse.ptp.rm.pbs.help
> > rms/org.eclipse.ptp.rm.pbs.jproxy
> > rms/org.eclipse.ptp.rm.pbs.proxy
> > rms/org.eclipse.ptp.rm.pbs.ui
> > rms/org.eclipse.ptp.rm.smoa.core
> > rms/org.eclipse.ptp.rm.smoa.ui
> Greg: unused
> 
> 
> > tools/cell/*
> Greg: unused
> 
> > tools/etfw/org.eclipse.ptp.etfw.tau.jars
> > tools/isp/org.eclipse.ptp.isp
> > tools/isp/org.eclipse.ptp.isp.help
> Greg: unused
> 
> > tools/org.eclipse.ptp.pldt.common
> > tools/org.eclipse.ptp.pldt.doc.user
> > tools/org.eclipse.ptp.pldt.help
> > tools/org.eclipse.ptp.pldt.lapi
> > tools/org.eclipse.ptp.pldt.mpi.analysis
> > tools/org.eclipse.ptp.pldt.mpi.analysis.cdt
> > tools/org.eclipse.ptp.pldt.mpi.core
> > tools/org.eclipse.ptp.pldt.openmp.analysis
> > tools/org.eclipse.ptp.pldt.openmp.core
> > tools/org.eclipse.ptp.pldt.openmp.ui.pv
> > tools/org.eclipse.ptp.pldt.tests
> > tools/org.eclipse.ptp.pldt.upc
> > tools/org.eclipse.ptp.pldt.wizards
> ?
> 
> > tools/org.eclipse.ptp.remotetools.core
> > tools/org.eclipse.ptp.remotetools.environment.core
> > tools/org.eclipse.ptp.remotetools.environment.generichost
> > tools/org.eclipse.ptp.remotetools.environment.launcher
> > tools/org.eclipse.ptp.remotetools.environment.ui
> > tools/org.eclipse.ptp.remotetools.preferences
> > tools/org.eclipse.ptp.remotetools.utils
> ?
> 
> > tools/perf/*
> Greg: unused
> 
> > tools/samples/org.eclipse.ptp.pldt.sampleCDTstaticAnalysis
> ?
> 
> > tools/tau/org.eclipse.ptp.tau-feature
> > tools/tau/org.eclipse.ptp.tau.jars
> > tools/tau/org.eclipse.ptp.tau.managedbuilder.other.photran.ui
> > tools/tau/org.eclipse.ptp.tau.managedbuilder.other.ui
> > tools/tau/org.eclipse.ptp.tau.papiselect
> > tools/tau/org.eclipse.ptp.tau.perfdmf
> > tools/tau/org.eclipse.ptp.tau.performance
> > tools/tau/org.eclipse.ptp.tau.selinst
> > tools/tau/org.eclipse.ptp.tau.selinstfort
> > tools/tau/org.eclipse.ptp.tau.toolopts
> > tools/tau/org.eclpse.ptp.tau.performance.parallel
> ?
> 
> > updates/org.eclipse.ptp-update
> Greg: unused
Comment 40 Greg Watson CLA 2011-10-19 09:09:47 EDT
(In reply to comment #34)
> > photran/org.eclipse.photran.cmdline
> > photran/org.eclipse.photran.vpg-feature
> 
> We can archive these two; they are no longer used.  I think a couple of
> Photran's releng projects can be archived too, but Greg would have to tell you
> for sure.
> 

I've moved these to an 'old' directory in the photran directory.

> I want to eliminate/consolidate a few more of Photran's projects, but I wasn't
> planning to do that right away.  I could do that after the move to Git if
> necessary, right?  Deleting projects from CVS wasn't handled particularly well,
> but I think deleting projects from a Git repository is much easier... is that
> right?

There's no need to rush into this. It can be done after the transition.
Comment 41 Greg Watson CLA 2011-10-19 09:21:24 EDT
I'd prefer not to make any significant changes prior to the migration. We can clean these up later.'

Are you planning to migrate Attic files as well?

(In reply to comment #37)
> Actually my list of largest files is a bit misleading. Since GIT keeps the
> history it is more important how large a file including its history is than thus
> the latest version. Looking at the CVS file sizes is also misleading because GIT
> stores all files Compressed. Here is a list of the CVS files, sorted by their
> gzip compressed size (first column) and the uncompressed size (2nd column).
> 
> 22411403 28M rms/org.eclipse.ptp.rm.jaxb.help/html/JAXBDemo.pdf,v
> 7055013 9.2M photran/org.eclipse.photran.core/Attic/f95parser.jar,v
> 5610714 6.7M rms/org.eclipse.ptp.rm.jaxb.help/html/images/JAXBDemo.pdf,v
> 5103757 51M
> photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/Parser.java,v
> 4957651 4.8M rdt/org.eclipse.ptp.rdt.server.dstore/Attic/rdt-server.jar,v
> 1170840 1.2M
> tools/gem/org.eclipse.ptp.gem.help/html/images/analyzerIntegration.png,v
> 1070953 1.3M photran/org.eclipse.photran.core.vpg/Attic/vpg-eclipse.jar,v
> 994280 2.1M
> photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/ExecutableProgramParsingTables.java,v
> 957767 946K
> tools/gem/org.eclipse.ptp.gem.help/html/images/consoleIntegration.png,v
> 870706 4.0M tools/sci/org.eclipse.ptp.sci/configure,v
> 844092 1.8M
> photran/org.eclipse.photran.core.vpg/parser/org/eclipse/photran/internal/core/parser/BodyParsingTables.java,v
> 793327 911K rms/org.eclipse.ptp.rm.jaxb.help/html/images/16composite.jpeg,v
> 755102 746K
> tools/gem/org.eclipse.ptp.gem.help/html/images/browserIntegration.png,v
> 731508 832K rms/org.eclipse.ptp.rm.jaxb.help/html/images/14tab-controller.jpeg,v
> 723516 708K
> releng/org.eclipse.ptp.pldt-feature/Attic/org.eclipse.ptp.pldt-1.0.0-I200607181525.zip,v
> 663831 709K
> tools/etfw/org.eclipse.ptp.etfw.feedback/html/images/Attic/feedback-wbench1.png,v
> 662298 724K
> rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/Attic/08editTemplate.jpg,v
> 630491 811K rms/org.eclipse.ptp.rm.jaxb.help/html/images/02control.jpeg,v
> 542208 570K
> rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/06Imported.png,v
> 524529 532K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/09Output.png,v
> 504739 546K
> photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.pdf,v
> 483057 503K rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/02Basic.png,v
> 482897 499K doc/org.eclipse.ptp.doc.user/html/images/03launch5.png,v
> 481309 499K
> photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.odt,v
> 460043 484K doc/org.eclipse.ptp.doc.user/html/images/03launchResTab.png,v
> 436630 5.4M
> photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/ParsingTable.java,v
> 430685 457K doc/org.eclipse.ptp.doc.user/html/images/sync-projProperties.png,v
> 418617 439K doc/org.eclipse.ptp.doc.user/html/images/sync-newprojCreate.png,v
> 409368 437K
> rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/Attic/14launchTabDetails.jpg,v
> 395118 408K doc/org.eclipse.ptp.doc.user/html/images/03launch2.png,v
> 393453 399K doc/org.eclipse.ptp.doc.user/html/images/Attic/04launch2.png,v
> 392919 396K doc/org.eclipse.ptp.doc.user/html/images/Attic/04launch5.png,v
> 379362 376K photran/org.eclipse.photran.doc.user/html/images/Browse
> 357671 350K doc/org.eclipse.ptp.doc.user/html/images/Attic/runtimePersp.gif,v
> 354066 356K tools/gem/org.eclipse.ptp.gem.help/html/images/gemPrefs.png,v
> 353469 352K tools/gem/org.eclipse.ptp.gem.help/html/images/ispPrefs.png,v
> 348075 396K
> doc/org.eclipse.ptp.doc.user/html/images/02ChooseResMgrNameAndDesc.png,v
> 347932 378K rms/org.eclipse.ptp.rm.jaxb.help/html/images/06Imported.png,v
> 343908 361K
> tools/pldt/org.eclipse.ptp.pldt.doc.user/html/images/mpiNotFound.png,v
> 332142 5.5M
> photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/Productions.java,v
> 331438 352K
> tools/etfw/org.eclipse.ptp.etfw.doc.user/html/images/feedback-wbench1.png,v
> 329609 346K
> rms/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/images/00Preferences.png,v
> 325903 408K
> rms/org.eclipse.ptp.rm.jaxb.help/html/images/21attribute-viewer.jpeg,v
> 322206 1.3M tools/sci/org.eclipse.ptp.sci/aclocal.m4,v
> 320335 400K rms/org.eclipse.ptp.rm.jaxb.help/html/images/19widget.jpeg,v
> 319636 445K rms/org.eclipse.ptp.rm.jaxb.help/html/images/Table.jpg,v
> 319501 313K doc/org.eclipse.ptp.doc.user/html/images/Attic/ptpRunOview.gif,v
> 316770 310K doc/org.eclipse.ptp.doc.user/html/images/Attic/launch2.gif,v
> 316082 362K
> doc/org.eclipse.ptp.doc.user/html/images/02-RM-OpenMPI-Configuration.png,v
> 315813 345K doc/org.eclipse.ptp.doc.user/html/images/06ptpDebugPersp.png,v
> 309005 329K
> tools/pldt/org.eclipse.ptp.pldt.doc.user/html/images/cdtPrefsTemplates.png,v
> 305939 435K
> rms/org.eclipse.ptp.rm.ibm.ll.help/html/images/ibmll_run_basic_application_3.jpg,v
> 304873 426K rms/org.eclipse.ptp.rm.jaxb.help/html/images/Tree1.jpg,v
> 301826 296K doc/org.eclipse.ptp.doc.user/html/images/Attic/prefsOMPI.gif,v
> 301196 323K doc/org.eclipse.ptp.doc.user/html/images/05runtimePerspAnn.png,v
> 
> GIT stores the files different but I think this is a good estimate. The question
> is whether we need to keep the full history for these large files.
> 
> Also my idea of recompressing the images was only half thought through. If we
> recompress them and keep the history we actually increase the repository size
> (the repository than contains both version). So we would need to recompress and
> not keep the history. I'm not sure whether we want to do that.
Comment 42 Greg Watson CLA 2011-10-19 09:29:33 EDT
(In reply to comment #36)
> Greg asked me on the call to describe the conversion process. I suggest we:
> 1) Finalize the list of project which should be part of the main repository
> 2) I will than create a test repository for the main PTP repository, Photran and
> Doc using the cvs2git command described before
> 3) We than test that the repository works as expected. If everything is OK:
> 4) We make CVS readonly and create the official/final set of GIT repositories.
> 
> Another question I have:
> Should the folder hierarchy be as it currently is in the workspace or as it
> currently is as on the CVS server? In the workspace all projects are directly in
> one folder and on the CVS server we have the subfolders (rdt, core, ...) which
> in turn contain the folders. Do we want this additional layer of folders?
> 
> With GIT the two folder layouts have to be the same. If we keep that additional
> layer of folders we need to update a few scripts (e.g.
> org.eclipse.ptp.linux.x86_64/BUILD) because these scripts expect all projects to
> reside in the same base folder.

Please keep the current CVS folder layout. I'm able to check out the CDT plugins into my workspace into one folder even though they are stored in separate folders under GIT. So I'm not sure what you're referring to here.
Comment 43 Beth Tibbitts CLA 2011-10-19 10:15:30 EDT
>I have moved the doc.user and doc.isv plugins to the core directory, so the doc
>directory can be migrated as a separate repo.
I would prefer that all plugin projects, including docs, be in the main repo.
There are some plugins that have rather small help files where the help extension is not in a separate project. Yes we should change that, but later.
Comment 44 Beth Tibbitts CLA 2011-10-19 10:16:39 EDT
Oh wait. by doc repo you meant, as we discussed last week, non-plugin projects.
To quote Emily Litella, "Never mind"
Comment 45 Jeffrey Overbey CLA 2011-10-19 12:10:52 EDT
(In reply to comment #37)
> 7055013 9.2M photran/org.eclipse.photran.core/Attic/f95parser.jar,v
> 1070953 1.3M photran/org.eclipse.photran.core.vpg/Attic/vpg-eclipse.jar,v
> photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.pdf,v
> photran/org.eclipse.photran.ui.vpg/src/org/eclipse/photran/internal/ui/search/Attic/README.odt,v
> photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/Productions.java,v

These haven't been used in a long time, so it's fine with me if you exclude them from the Git repo.

Jeff
Comment 46 Roland Schulz CLA 2011-10-19 15:51:32 EDT
(In reply to comment #39)
> This list is out of date and does not reflect the latest repository contents.
I don't understand. I'm pretty sure I have the latest version. Could you give me an example of something which doesn't reflect the repository?

(In reply to comment #41)
> I'd prefer not to make any significant changes prior to the migration. We can
> clean these up later.'
Moving files can be done later. But deleting files later is not really an option. The normal method of deleting files doesn't delete their history and thus doesn't decrease the repository size. Deleting a file including the history requires "history rewriting" which causes work for everyone who has the repository checked-out. So deleting files (with their history) later is not simple and in general not recommended. 
 
> Are you planning to migrate Attic files as well?
I was. For files deleted or moved recently I was assuming we would like to keep the history. Is that not true?

(In reply to comment #42)
> (In reply to comment #36)
> > Should the folder hierarchy be as it currently is in the workspace or as it
> > currently is as on the CVS server? In the workspace all projects are directly in
> > one folder and on the CVS server we have the subfolders (rdt, core, ...) which
> > in turn contain the folders. Do we want this additional layer of folders?
> > 
> > With GIT the two folder layouts have to be the same. If we keep that additional
> > layer of folders we need to update a few scripts (e.g.
> > org.eclipse.ptp.linux.x86_64/BUILD) because these scripts expect all projects to
> > reside in the same base folder.
> 
> Please keep the current CVS folder layout. I'm able to check out the CDT
> plugins into my workspace into one folder even though they are stored in
> separate folders under GIT. So I'm not sure what you're referring to here.

I just downloaded CDT from GIT. And while Eclipse (project explorer and navigator) shows all projects without the subfolder, all projects are stored on the disk with subfolders. E.g. the property windows shows for the location:
...\org.eclipse.cdt\cross\org.eclipse.cdt.build.crossgcc
Notice it includes the "cross" subfolder. 
This doesn't cause any problems with eclipse. But our build scripts (e.g. org.eclipse.ptp.linux.x86_64/BUILD) will not work with these subfolders.

It is simple to fix the few build scripts which are affected but I just wanted to make sure that is what we want to do.
Comment 47 Greg Watson CLA 2011-10-20 16:12:18 EDT
(In reply to comment #46)
> (In reply to comment #39)
> > This list is out of date and does not reflect the latest repository contents.
> I don't understand. I'm pretty sure I have the latest version. Could you give
> me an example of something which doesn't reflect the repository?

rdt/org.eclipse.ptp.rdt.releng
rdt/org.eclipse.ptp.rdt.services
rdt/org.eclipse.ptp.rdt.services.test

These have already been moved to rdt/old.

releng/org.eclipse.ptp.isp-feature
releng/org.eclipse.ptp.package-feature
releng/org.eclipse.ptp.perf-feature
releng/org.eclipse.ptp.perf.tau-feature
releng/org.eclipse.ptp.perf.tau.fortran-feature
releng/org.eclipse.ptp.rdt.package-feature

These have been already been moved to releng/old.

etc., etc.


> 
> (In reply to comment #41)
> > I'd prefer not to make any significant changes prior to the migration. We can
> > clean these up later.'
> Moving files can be done later. But deleting files later is not really an
> option. The normal method of deleting files doesn't delete their history and
> thus doesn't decrease the repository size. Deleting a file including the
> history requires "history rewriting" which causes work for everyone who has the
> repository checked-out. So deleting files (with their history) later is not
> simple and in general not recommended. 

We simply don't have the resources to go through and delete all these files, then check for broken links, and fixing up the problems. This will have to be done after the repository is moved.

> 
> > Are you planning to migrate Attic files as well?
> I was. For files deleted or moved recently I was assuming we would like to keep
> the history. Is that not true?

Ok.

> 
> (In reply to comment #42)
> > (In reply to comment #36)
> > > Should the folder hierarchy be as it currently is in the workspace or as it
> > > currently is as on the CVS server? In the workspace all projects are directly in
> > > one folder and on the CVS server we have the subfolders (rdt, core, ...) which
> > > in turn contain the folders. Do we want this additional layer of folders?
> > > 
> > > With GIT the two folder layouts have to be the same. If we keep that additional
> > > layer of folders we need to update a few scripts (e.g.
> > > org.eclipse.ptp.linux.x86_64/BUILD) because these scripts expect all projects to
> > > reside in the same base folder.
> > 
> > Please keep the current CVS folder layout. I'm able to check out the CDT
> > plugins into my workspace into one folder even though they are stored in
> > separate folders under GIT. So I'm not sure what you're referring to here.
> 
> I just downloaded CDT from GIT. And while Eclipse (project explorer and
> navigator) shows all projects without the subfolder, all projects are stored on
> the disk with subfolders. E.g. the property windows shows for the location:
> ...\org.eclipse.cdt\cross\org.eclipse.cdt.build.crossgcc
> Notice it includes the "cross" subfolder. 
> This doesn't cause any problems with eclipse. But our build scripts (e.g.
> org.eclipse.ptp.linux.x86_64/BUILD) will not work with these subfolders.
> 
> It is simple to fix the few build scripts which are affected but I just wanted
> to make sure that is what we want to do.

I can live with that.
Comment 48 Roland Schulz CLA 2011-10-21 19:25:04 EDT
(In reply to comment #39)
> This list is out of date and does not reflect the latest repository contents.

You were right. I didn't realize that "cvs up -P" doesn't prune directory which have been deleted.

The correct list is (this time sorted by category hopefully easier to read):

As far as I can see missing in project set:
> core/org.eclipse.ptp.doc.isv
> core/org.eclipse.ptp.doc.user

Needed but probably not required in project set:
> photran/org.eclipse.photran.core.vpg.preprocessor.c.tests
> photran/org.eclipse.photran.core.vpg.tests
> photran/org.eclipse.photran.core.vpg.tests.failing
> photran/org.eclipse.photran-dev-docs
> photran/org.eclipse.photran-feature
> photran/org.eclipse.photran.intel-feature
> photran/org.eclipse.photran.master
> photran/org.eclipse.photran-samples
> photran/org.eclipse.photran.xlf-feature
> photran/org.eclipse.rephraserengine.core.vpg.tests
> photran/org.eclipse.rephraserengine.doc.isv
> photran/org.eclipse.rephraserengine.examples.testing
> photran/org.eclipse.rephraserengine.examples.text
> photran/org.eclipse.rephraserengine-feature
> photran/org.eclipse.rephraserengine-support
> photran/org.eclipse.rephraserengine.testing.junit3
> rdt/org.eclipse.ptp.rdt.sync.core.tests
> releng/org.eclipse.ptp.rm.jaxb.contrib-feature
> releng/org.eclipse.ptp.rm.jaxb.openmpi-feature
> rms/org.eclipse.ptp.rm.jaxb.contrib
> rms/org.eclipse.ptp.rm.jaxb.openmpi
> tools/samples/org.eclipse.ptp.pldt.sampleCDTstaticAnalysis

Can be moved to old ( or left out in conversion):
> rdt/org.eclipse.ptp.rdt.sync.rsync.core
> rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> rdt/org.eclipse.ptp.rdt.xlc
> releng/org.eclipse.ptp.rdt.sdk-feature
> releng/org.eclipse.ptp.rdt.xlc.sdk-feature
> releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> releng/org.eclipse.ptp.rm.lsf-feature
> releng/org.eclipse.ptp.rm.orte-feature
> releng/org.eclipse.ptp.rm.pbs-feature
> rms/org.eclipse.ptp.rm.pbs.core
> rms/org.eclipse.ptp.rm.pbs.core.server
> rms/org.eclipse.ptp.rm.pbs.help
> rms/org.eclipse.ptp.rm.pbs.jproxy
> rms/org.eclipse.ptp.rm.pbs.proxy
> rms/org.eclipse.ptp.rm.pbs.ui
> rms/org.eclipse.ptp.rm.smoa.core
> rms/org.eclipse.ptp.rm.smoa.ui
> tools/etfw/org.eclipse.ptp.etfw.tau.jars

Can be left out (not needed for GIT):
> releng/ptp_project_sets

I don't know:
> releng/org.eclipse.ptp.repo

@Greg: For projects in the category "Can be moved to old" you wrote before that they are old. Do you want to move them? Or are some still current? Or do you want me to just ignore them when converting? What about ptp.repo?

(In reply to comment #47)
> We simply don't have the resources to go through and delete all these files,
> then check for broken links, and fixing up the problems. This will have to be
> done after the repository is moved.
What about JAXBDemo.pdf,v (both versions) and Attic/rdt-server.jar,v? These 3 files together are a significant part of the whole repository. Should I just leave out rdt-server.jar,v because it is in Attic anyhow and it is large (but keep all other Attic files)? Can we move JAXBDemo.pdf,v to somewhere else (e.g. doc repository)?
What about the images? Do we need their history? Otherwise I don't mind the effort to recompress them all to gain the 7MB (I already wrote a script to do so).
Comment 49 Greg Watson CLA 2011-10-22 16:00:06 EDT
(In reply to comment #48)
> (In reply to comment #39)
> 
> Can be moved to old ( or left out in conversion):
> > rdt/org.eclipse.ptp.rdt.sync.rsync.core
> > rdt/org.eclipse.ptp.rdt.sync.rsync.ui
> > rdt/org.eclipse.ptp.rdt.xlc
> > releng/org.eclipse.ptp.rdt.sdk-feature
> > releng/org.eclipse.ptp.rdt.xlc.sdk-feature
> > releng/org.eclipse.ptp.rm.ibm.bluegene-feature
> > releng/org.eclipse.ptp.rm.lsf-feature
> > releng/org.eclipse.ptp.rm.orte-feature
> > releng/org.eclipse.ptp.rm.pbs-feature
> > rms/org.eclipse.ptp.rm.pbs.core
> > rms/org.eclipse.ptp.rm.pbs.core.server
> > rms/org.eclipse.ptp.rm.pbs.help
> > rms/org.eclipse.ptp.rm.pbs.jproxy
> > rms/org.eclipse.ptp.rm.pbs.proxy
> > rms/org.eclipse.ptp.rm.pbs.ui
> > rms/org.eclipse.ptp.rm.smoa.core
> > rms/org.eclipse.ptp.rm.smoa.ui
> > tools/etfw/org.eclipse.ptp.etfw.tau.jars
> 
> Can be left out (not needed for GIT):
> > releng/ptp_project_sets
> 
> I don't know:
> > releng/org.eclipse.ptp.repo
> 
> @Greg: For projects in the category "Can be moved to old" you wrote before that
> they are old. Do you want to move them? Or are some still current? Or do you
> want me to just ignore them when converting? What about ptp.repo?

Some of these are required in the 4_1 branch so they need to stay. The only one that can be removed is org.eclipse.ptp.etfw.tau.jars which I've now moved to old. I'd like to keep the rest.

org.eclipse.ptp.repo is used by the maven build so is required.

> 
> (In reply to comment #47)
> > We simply don't have the resources to go through and delete all these files,
> > then check for broken links, and fixing up the problems. This will have to be
> > done after the repository is moved.
> What about JAXBDemo.pdf,v (both versions) and Attic/rdt-server.jar,v? These 3
> files together are a significant part of the whole repository. Should I just
> leave out rdt-server.jar,v because it is in Attic anyhow and it is large (but
> keep all other Attic files)? Can we move JAXBDemo.pdf,v to somewhere else (e.g.
> doc repository)?
> What about the images? Do we need their history? Otherwise I don't mind the
> effort to recompress them all to gain the 7MB (I already wrote a script to do
> so).

rdt-server.jar can be left out. Let me take a look at JAXBDemo.pdf.
Comment 50 Greg Watson CLA 2011-10-24 12:53:13 EDT
As far as I can see, the whole org.eclipse.ptp.rm.jaxb.help plugin is not used, or at least not hooked into the help system. I'll need to remove it from the build, but I think it can be moved to old.
Comment 51 Greg Watson CLA 2011-10-24 17:27:55 EDT
(In reply to comment #50)
> As far as I can see, the whole org.eclipse.ptp.rm.jaxb.help plugin is not used,
> or at least not hooked into the help system. I'll need to remove it from the
> build, but I think it can be moved to old.

Actually it is being used as Beth pointed out. I've removed the large PDF's and zip files. I'm just tarring up the cvs repository now, but as far as I'm concerned we should be good to go now.
Comment 52 Roland Schulz CLA 2011-10-25 14:37:39 EDT
Large Attic files in Photran are:

6.0M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran-dev-docs/dev-guide/Attic/dev-guide.pdf,v
62M     ./org.eclipse.ptp.photran/photran/org.eclipse.photran-dev-docs/eclipsecon-2006/Attic/eightbol.iso,v
72M     ./org.eclipse.ptp.photran/photran/org.eclipse.photran-dev-docs/eclipsecon-2006/Attic/eightbol.zip,v
1.4M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core/model/org/eclipse/photran/internal/core/f95modelparser/Attic/ParsingTable.java,v
1.2M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core/model/org/eclipse/photran/internal/core/f95modelparser/Attic/BuildModelProductionReductions.java,v
1.4M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core/model/org/eclipse/photran/internal/core/f95parser/Attic/BuildModelProductionReductions.java,v
5.4M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/ParsingTable.java,v
2.3M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core/parser/org/eclipse/photran/internal/core/parser/Attic/Parser.java,v
1.4M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.cdtinterface/model/org/eclipse/photran/internal/core/f95modelparser/Attic/ParsingTable.java,v
1.2M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.cdtinterface/model/org/eclipse/photran/internal/core/f95modelparser/Attic/BuildModelProductionReductions.java,v
2.2M    ./org.eclipse.ptp.photran/photran/org.eclipse.photran.core.vpg/parser/Attic/Parser.java.new,v

I assume I should exclude eightbol.(iso|zip). What about the others? Should I keep them or should I leave them out?
Comment 53 Jeffrey Overbey CLA 2011-10-25 14:58:21 EDT
Hi Roland -- You can leave out all of those in the attic.  They are ancient.
Comment 54 Roland Schulz CLA 2011-10-25 23:35:55 EDT
Deleted images (those in Attic) are for PTP (excluding Photran and Doc) 36MB. Should I keep all of them, or leave them all out? Or some intermediate?
Comment 55 Roland Schulz CLA 2011-10-27 18:30:54 EDT
Created attachment 206101 [details]
Recipe to convert CVS to GIT
Comment 56 Roland Schulz CLA 2011-10-27 23:38:03 EDT
The test repositories are ready at:
https://github.com/rolandschulz/org.eclipse.ptp.git
https://github.com/rolandschulz/org.eclipse.ptp.photran.git
https://github.com/rolandschulz/org.eclipse.ptp.doc.git

Please check whether everything is as it should be. I also uploaded the recipe I used.

I had to do three minor changes to be able to checkout the repository and import all project:

When one tried to import the repository, one got an error that org.eclipse.ptp.rm.jaxb.rms-feature already exists because both releng/org.eclipse.ptp.rm.jaxb.pbs-feature/.project and releng/org.eclipse.ptp.rm.jaxb.openmpi-feature/.project have that name. I suppose they should be renamed into pbs-feature/openmpi-feature respectively. I made that change in master. 

Also it wasn't possible to checkout the photran repository because a file "org.eclipse.photran.doc.user/html/images/{Replace Character* with Character(len=).png" existed. I renamed (in ptp_5 and master) that into "Replace CharacterStar with Character(len=).png" because windows doesn't support "*" in the filename.

I also renamed in the master branch (shortening the file ending): 
org.eclipse.photran.core.vpg.tests/refactoring-test-code/remove-real-and-double-precision-loop-counters/nested-do-double-inner-decrement-implicit-do-while/nested-do-double-inner-decrement-implicit-do-while.f90.result -> org.eclipse.photran.core.vpg.tests/refactoring-test-code/remove-real-and-double-precision-loop-counters/nested-do-double-inner-decrement-implicit-do-while/nested-do-double-inner-decrement-implicit-do-while.f90.r
and
org.eclipse.photran.core.vpg.tests/refactoring-test-code/remove-real-and-double-precision-loop-counters/nested-do-double-outer-increment-implicit-do-while/nested-do-double-outer-increment-implicit-do-while.f90.result -> org.eclipse.photran.core.vpg.tests/refactoring-test-code/remove-real-and-double-precision-loop-counters/nested-do-double-outer-increment-implicit-do-while/nested-do-double-outer-increment-implicit-do-while.f90.r

to stay under the 260 character limit of Windows.
Comment 57 Roland Schulz CLA 2011-11-10 00:55:37 EST
Created attachment 206755 [details]
Final used recipe to convert CVS to GIT
Comment 58 Beth Tibbitts CLA 2011-12-16 23:52:04 EST
The git repo is now in use by the PTP project.

http://wiki.eclipse.org/PTP/environment_setup_60