Bugzilla – Full Text Bug Listing
| Summary: | Import Repositories from Gitblit | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Technology] EGit | Reporter: | James Moger <james.moger> | ||||||
| Component: | UI | Assignee: | Project Inbox <egit.ui-inbox> | ||||||
| Status: | NEW --- | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | P3 | CC: | matthias.sohn, rademacher, remy.suen, sascha.scholz, selsemore, stefan.lay, steffen.pingel | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| Whiteboard: | |||||||||
| Attachments: |
|
||||||||
(In reply to comment #0) > So it would be really useful to have an "Import from Gitblit" dialog for EGit > where the user can browse the list of available repositories and then select > one (or more?) to clone. (It would be super cool to be able to clone multiple > repos at once with a ctrl-click/shift-click approach). Shouldn't Gitblit offer an Eclipse plug-in with this import wizard as a feature? That was my original thought too. I mentioned the idea to Matthias and he suggested posting an enhancement request. Do you recommend a tutorial/starting point? Will I be able to delegate cloning to EGit from my own plugin? I think I remember seeing that GitHub has something similar built-into EGit - or at least as a feature/add-on. The GitHub Mylyn plugin does have an import wizard for cloning projects that are hosted on the site. The user selects one or more projects and then it defers to EGit to actually perform the clone and register it in the Git Repositories view. Kevin that sounds exactly like what I want. Is that project hosted on GitHub somewhere or is it part of JGit/EGit? Nevermind, just found it. Maybe we could instead have only one JSON RPC implementation for this remote repository discovery and make this part of EGit itself. This would have the advantage that others who might have similar requirements could just rely on that instead of reinventing the wheel multiple times. I know that the Skalli project (implementing a project portal) discussed that they also would like to have the ability to export the list of repositories they have and integrate it with EGit so that EGit users can browse a list of repositories instead of copying around URLs manually. @Matthias: That sounds like a good idea. Enumerating the public repositories is simple. Enumerating the private/restricted repositories may be trickier for me since Gitblit only supports Basic authentication. I'm not sure what Other systems would do or propose for authentication. Since Gitblit only accepts Basic authentication its preferable to communicate with it over https, which reminds me that JGit (unlike Cgit) still performs hostname verification even with http.sslVerify=false. (In reply to comment #5) > Kevin that sounds exactly like what I want. Is that project hosted on GitHub > somewhere or is it part of JGit/EGit? The GitHub Mylyn project is an EGit subproject available here: http://egit.eclipse.org/w/?p=egit-github.git;a=summary Created attachment 207480 [details]
Mockup of EGit import wizard
Matthias Sohn, Jens Baumgart and me discussed about how to integrate imports from different kinds of servers hosting git repositories. We propose the following:
- We do not implement generic client or server code to discover repositories. Different server types have different search criteria. There are already existing APIs, and even client code exists.
- We instead focus on unifying the UI experience. We do this by creating an extension point to the Import Projects / Clone wizard. An extension would have to provide a new server category to be displayed on page "ImportSource" (see attached Mock-Up) and a Wizard Page which can be used for repository search. Our wizard expects to get a clone URI from this wizard page.
- We also propose to unify the Import Projects and the Clone wizard by this, to replace the three buttons (Add, Clone, Create) in the Repositories View by one and allow for multi-selection of repos in the wizard.
What do you think about that? Please have a look at the Mock-Up which was made by Wireframe Sketcher (an awesome tool).
Please use the interactive buttons and links in the Mockup, not the "Previous" and "Next" links. In the last days I have been working on a wizard that can clone a repository from Gerrit. With the current implementation of the GitCloneWizard, some hacking were required. If you are interested, you can have a look at the POC implementation at http://review.mylyn.org/#change,115 . Your proposal of an extension point sounds good, here are some remarks: - The Gerrit connector manages the list of Gerrit instances as task repositories in Mylyn. There's no need for EGit to add additional code to manage the import sources - An import wizard for Gerrit projects should be able to show all projects from all configured Gerrit instances, so that I can quickly search for a project and don't have to care about selecting the right instance before. - I agree with you that no additional server code for discovery should be required, because e.g. for the Gerrit connector we already have the necessary client code in place. For other systems it's probably the same. (In reply to comment #12) > In the last days I have been working on a wizard that can clone a repository > from Gerrit. With the current implementation of the GitCloneWizard, some > hacking were required. If you are interested, you can have a look at the POC > implementation at http://review.mylyn.org/#change,115 . > > Your proposal of an extension point sounds good, here are some remarks: > > - The Gerrit connector manages the list of Gerrit instances as task > repositories in Mylyn. There's no need for EGit to add additional code to > manage the import sources IMO a versioning system is not a task repository. Though it makes sense to register Gerrit as a task repository when looking at reviews. So maybe Git servers in general could appear under a new server category for version control systems in the Team Repository view. Since Gerrit is both a Git server and a code review system which is providing review tasks it could maybe appear under both "Version Control Systems" and "Task Repository". We also thought about additionally supporting searching and cloning Git repositories from a Gerrit server when the Gerrit connector is not installed. > - An import wizard for Gerrit projects should be able to show all projects from > all configured Gerrit instances, so that I can quickly search for a project and > don't have to care about selecting the right instance before. Our assumption was rather that the search is implemented on the side of the Git servers as the number of repositories might be very large (think about Github). Stefan, in your mockups there are panels for GitHub, Gerrit, Gitblit, etc. The Gerrit ones include a login panel for url and credentials. Is that panel an extension point or is that something that would be coded by the EGit team? Would url/credential storage be delegated to EGit or is that something that must be implemented by the Import Source? As you indicate in your sketches, each repository provider may provide metadata for a repository (e.g. GitHub watchers count). Would the repositories list panel be coded by the EGit team? Or is that another extension point? The general flow and idea of a unified wizard is a great idea. It is not clear to me, though, what parts I would need to provide/implement for a Gitblit Import Source, aside from the RPC logic. (In reply to comment #14) > Stefan, in your mockups there are panels for GitHub, Gerrit, Gitblit, etc. The > Gerrit ones include a login panel for url and credentials. Is that panel an > extension point or is that something that would be coded by the EGit team? > Would url/credential storage be delegated to EGit or is that something that must > be implemented by the Import Source? This is not yet clear to me. For some servers there may already be a storage for URLs and credentials. For example, the Gerrit servers are stored as Mylyn Repositories. We could do the following: - An extension point where a plugin can provide just a list of servers and credentials (for example for gerrit servers stored as Mylyn Repositories) - An extension point where a plugin can hook in a dialog which is displayed to enter URL and credentials either EGit stores the data or the plugin. In the second case the new server has to be provided to EGit by the first extension point - EGit provides such a dialog and stores URI and credentials. Which one would you prefer? > As you indicate in your sketches, each repository provider may provide metadata > for a repository (e.g. GitHub watchers count). Would the repositories list > panel be coded by the EGit team? Or is that another extension point? > I would suggest that the repository list panel should be provided by the plugin which uses the extension point. The search criteria and the data presented to the user in the list is specific for each repository. Or do you think it would be a good idea that EGit already has a default UI with a simple search field for the name and a simple result table? That's worth considering. We then should expose an extension point only for the search itself. > The general flow and idea of a unified wizard is a great idea. It is not clear > to me, though, what parts I would need to provide/implement for a Gitblit Import > Source, aside from the RPC logic. I think it is easy enough for a plugin to provide a panel for url and credentials BUT... 1. I expect most servers basically have the same requirements: url, username, password 2. I think most plugins would want to store that information using the standard Eclipse encrypted? mechanism (whatever that may be) So I would vote in favor of standard EGit url/credential entry and EGit url/credential storage. Perhaps it could be an extension point with a default implementation so that plugins could override it should they want to? I agree that the repository list panel should be provided by the plugin. That allows GitHub to have their search function with Watchers, etc. And it allows Gitblit to have a non-searchable (maybe filterable?) list of repositories with Owners, Restrictions, Last Change, etc metadata. For the Gerrit connector it would be sufficient to have a repository selection dialog provided by EGit where a user could choose to import a repository from Gerrit. The next page in the wizard could then be the repository selection implemented by the Gerrit plugin. This should basically be a replacement for the current source selection page in the clone wizard. That means, we have only minimal requirements to what should be provided by EGit in the UI. The mockup looks nice. I would consider replacing the stacked dialogs with embedded pages, i.e. if a user intents to add a new repository support that by selecting the category and showing a credentials page in the wizard rather than a second dialog. It would be unfortunate to duplicate implementation and configuration that is already in Mylyn. I agree with Sascha that an extension point which allows us to provide a selection page to hook into the wizard would be nice. Even simpler the Gerrit connector could provide a list of Gerrit repositories and their corresponding Git repositories, i.e. the integration would only work on the data model level without custom UI. This extension mechanism could of course work in addition to what Matthias described in comment 7: If Gerrit implements the RPC interface for discovering projects users would have a choice to either select from their existing Mylyn Gerrit servers or use the generic mechanism to browse. (In reply to comment #15) > This is not yet clear to me. For some servers there may already be a storage for > URLs and credentials. For example, the Gerrit servers are stored as Mylyn > Repositories. We could do the following: > - An extension point where a plugin can provide just a list of servers and > credentials (for example for gerrit servers stored as Mylyn Repositories) My sense is that if a plug-in already knows a location and credentials it is likely to also have logic for connecting to the repository. I would prefer an API where the behavior to provide the list of Git repositories is (optionally) implemented in the extension. EGit could provide a generic JSON-RPC behavior for reuse by extensions but in the case of the Mylyn Gerrit connector there is already code for retrieving the list of available Git repositories. > - An extension point where a plugin can hook in a dialog which is displayed to > enter URL and credentials either EGit stores the data or the plugin. In the > second case the new server has to be provided to EGit by the first extension > point > - EGit provides such a dialog and stores URI and credentials. That sounds similar to the implementation in org.eclipse.mylyn.commons.repositories and org.eclipse.mylyn.commons.team. These bundles provide a data model abstraction for repository locations and extensible UI for entering credentials. It's intended (and partially used) as a common abstraction for task repositories and build servers. I wonder if it makes sense to integrate the repository management in the Git import wizard with the Mylyn Team repositories view and wizard: http://eclipse.org/mylyn/new/#framework. It's intended for managing accounts for different repositories and as a single entry point for connecting to services. Ping: I plan to provide a first version of a new EGit import wizard soon, hopefully next week. Then we could use the gitblit import scenario as a proof of concept for a new extension point for repository search. I also plan to work together with Sascha on the integration of repository search on Gerrit. I proposed a first version of the extension point and its usage in a new import wizard at gerrit: http://egit.eclipse.org/r/#change,4845 http://egit.eclipse.org/r/#change,4846 http://egit.eclipse.org/r/#change,4847 http://egit.eclipse.org/r/#change,4848 In this first version there is a possibility to contribute a wizard page and a list of servers. If needed I would further add the featrue that egit itself stores a list of repository servers. Sascha Scholz and me already implemented a proof of concept on top of this in the gerrit mylyn connector project. Hi Stefan. I took a peek at your changes in Gerrit. Unfortunately... I have no idea how to use Gerrit/build EGit and with the holidays and other obligations fast approaching it will be January before I can devote time to understanding how to create and test a Gitblit extension to your revised import wizard. I don't know how fast you wanted to go with this but I'm definitely a pothole on your road to success. Sorry, I can't be more on top of this at the moment. (In reply to comment #22) > Hi Stefan. I took a peek at your changes in Gerrit. > Unfortunately... I have no idea how to use Gerrit/build EGit and with the > holidays and other obligations fast approaching it will be January before I can > devote time to understanding how to create and test a Gitblit extension to your > revised import wizard. > > I don't know how fast you wanted to go with this but I'm definitely a pothole > on your road to success. Sorry, I can't be more on top of this at the moment. Hi James, we do not need to hurry. We can first add the search on Gerrit and on Github servers. They already have EGit Wizard Pages in their Gerrit Mylyn connector projects which can now easily be added to the Import Wizard. Then we will also have holidays and a lot of other stuff to do. We would have to think about where to put the coding for the Gitblit repo search extension. Would we need to introduce dependencies? We would definitely not add code to the egit ui plugin but to a new one. Maybe we would better add a new feature which can be installed independently form the egit feature itself. This is how the Gerrit and the Github integration is done. Gitblit has a JSON RPC interface. I have implemented client code using Google GSON which is already a requirement of the GitHub Mylyn integration. If EGit already has a JSON dependency then perhaps that could be used instead. I'm not too picky and the data isn't that complex. I could even write a new servlet just for this purpose (EGit import) to dump out some format that didn't require an additional dependency to parse, if you have a suggestion. I've experimented with using this extension point to enable CollabNet TeamForge servers as Git repository providers and it works great. What is the current status of this extension point? Are there plans to graduate the API from internal/provisional status soon? Great to hear that you successfully used the extension point. We will not change anything until the 2.0 release which will happen before the juno release in june. Afterwards we may start a discussion about that question, e.g. if there are bugs or missing features which should be fixed before graduating. Do you have any issues? I would really like to have the Gitblit integration to have more examples of repository providers. The existing providers (Gerrit and Github) already use their mylyn connectors to store the servers' properties. For Gitblit we may have to create a means to add Gitblit servers, and I do not know if the extension point is useful for that use case. Thanks, Stefan. I haven't encountered any issues related to the extension point. For know, I think we will proceed and make use with it, and monitor for possible breaking changes after the 2.0 release (in addition to the CollabNet TeamForge provider, we'll contribute a CollabNet Codesion provider as well). |
Created attachment 205419 [details] Screenshot showing enumeration of repositories from Gitblit I've been missing the ability to (at least) browse the list of available projects at a remote Git server like I could do with SVN or CVS. The _upcoming_ Gitblit 0.7.0 release has a JSON RPC interface which, among other things, allows the user to authenticate and enumerate available repositories. On the Gitblit side, I used google-gson which is already a dependency of the GitHub Mylyn connector. The master branch of the Gitblit repository has a working rpc servlet/authentication filter, a static rpc client utility class, & unit tests. There is also a Swing Manager application which exercises several, though not yet all, of the rpc functions (screenshot attached). So what? So it would be really useful to have an "Import from Gitblit" dialog for EGit where the user can browse the list of available repositories and then select one (or more?) to clone. (It would be super cool to be able to clone multiple repos at once with a ctrl-click/shift-click approach). To ensure that developers don't get annoyed, account credentials should be prompted for (to list access restricted repositories) and the credentials should also be cached locally somehow, somewhere along with the url of the Gitblit instance and probably a name/description. These credentials are the same ones used for cloning, pushing, etc. While (I think) I have the motivation to hack on Eclipse - I have no idea where to begin so I'm reaching out to the EGit masters. Of course if someone wants to take this on instead of me, that would be fine too - maybe even preferable. :)