Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 360935 - Provide a Sonar instance for Eclipse projects
Summary: Provide a Sonar instance for Eclipse projects
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Cross-Project (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement with 5 votes (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 154159 (view as bug list)
Depends on: 390373 391343
Blocks:
  Show dependency tree
 
Reported: 2011-10-14 05:49 EDT by Mickael Istria CLA
Modified: 2013-09-26 09:22 EDT (History)
28 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mickael Istria CLA 2011-10-14 05:49:23 EDT
Code quality analysis is easy to set up with Sonar. Moreover, Sonar is well integrated with Hudson and Maven/Tycho.
It would be very useful for Eclipse projects to be able to use Sonar for code quality analysis, but this requires to host a Sonar instance at Eclipse.org.


Some related links to push the idea of using Sonar:
* http://www.sonarsource.org
* http://meera-subbarao.blogspot.com/2009/11/hudson-sonar-perfect-match.html
* http://wiki.hudson-ci.org/display/HUDSON/Sonar+plugin
* http://mdwhatever.free.fr/index.php/2011/09/quality-analysis-on-eclipse-plugins-with-tycho-sonar-jacoco-and-swtbot/
Comment 1 Xavier Seignard CLA 2011-10-14 06:38:52 EDT
Would be great, but there is a tiny limitation on the Sonar side :
http://jira.codehaus.org/browse/SONAR-2760

Related discussion here :
http://old.nabble.com/How-to-exclude-source-directory-from-analysis--%28not-a-package%29-to32371054.html#a32378604

Regards,

Xavier
Comment 2 Pascal Rapicault CLA 2011-10-14 09:15:34 EDT
As much as I would like to see this happening, given the limited resources, both human and machine at the EF, I would rather see the resources be invested on getting a workable and stable hudson environment where ppl can push their bytes around w/o the hassle that we currently have.

I think it would be much better to use the OSS instance provided by sonar source (http://nemo.sonarsource.org/) as I'm sure it is easy to get a project setup.
Comment 3 Mickael Istria CLA 2011-10-14 09:21:32 EDT
I agree for using the "nemo" instance at SonarSource.
However, I am not sure about the policy to use it. Should the EF oversee the usage of nemo.sonarsource.org or are committers or project lead free to use it without control from the EF ? The second solution would be fine in a first time I guess.
Comment 4 David Williams CLA 2011-11-01 00:07:31 EDT
From the discussion, it sounds like this should be closed as "won't fix". 

Mickael, Are you "waiting" or expecting any more here? 

I'm not exactly sure what you are asking in comment 3, but normally the EF doesn't control anything about what tools projects use ... that's not to say projects could use it ... just saying its not really up to EF, as far as I know. If you want to advocate projects use it, I suspect bloging about it or something is the best approach (as well as discussing with any projects you are a committer on). 

I'll assign to you, Mickael, so it can stay open if you think there's more to do or discuss, but if not, I suggest closing as "won't fix". 

Thanks,
Comment 5 Mickael Istria CLA 2011-11-02 05:53:44 EDT
I'm investigating to move SWTBot build to nemo, but there are some more work to do before that step.
As advised, I'll report my conclusions as a blog post when I get some things to share. Then we'll see if some other projects are interested, and whether there are need for some more help from community and foundation.
Comment 6 Mickael Istria CLA 2011-11-08 04:25:30 EST
Finally, I made some progress on this topic. SWTBot has some reports hosted on nemo.sonarsource.org. But it appears that Sonar guys have to build the project on their side to set up reports on nemo. This is about security and performance concerns.
So currently, it cannot be the same build that is run on hudson,eclipse.org, and which has reports on nemo.sonarsource.org.

The conclusion is that: using Nemo is not satisfying enough. We do need a Sonar instance for projects at Eclipse.org.
I understand the lack of resources to set it up, but it is not an immediate requiremetn. Hence I reopen this bug because I'm pretty sure it will be fixed one day, and I think it will be a very hot topic in a few monthes ;)
Comment 7 Mickael Istria CLA 2011-11-10 07:42:35 EST
Nemo (and the related Jenkins instance at Cloudbees that hosts builds) has the limitation that it cannot run UI tests.
Comment 8 Evgeny Mandrikov CLA 2011-11-11 14:23:18 EST
Hi,

I didn't tried yet, but according to the page https://cloudbees.zendesk.com/entries/382175-xvnc-plugin should be possible to run UI tests in CloudBees.
Comment 9 Mickael Istria CLA 2011-11-23 12:00:22 EST
(In reply to comment #2)
> As much as I would like to see this happening, given the limited resources,
> both human and machine at the EF, I would rather see the resources be invested
> on getting a workable and stable hudson environment where ppl can push their
> bytes around w/o the hassle that we currently have.

I think that one bug #364605 is resolved, it is just about a very few steps for a sysadmin to set up a basic Sonar instance: Download, install, set up SSO (http://docs.codehaus.org/display/SONAR/LDAP+Plugin).
Comment 10 Anthony Dahanne CLA 2011-11-23 13:17:08 EST
(In reply to comment #9)
> I think that one bug #364605 is resolved,
> it is just about a very few steps for
> a sysadmin to set up a basic Sonar instance: Download, install, set up SSO
> (http://docs.codehaus.org/display/SONAR/LDAP+Plugin).

I don't think it is necessary to wait for SSO at Eclipse; a typical usage (hum.. at least mine ;-) ) of Sonar is the admin role + everybody can push analysis (I mean we could limit one machine, the one running hudson for example, to trigger sonar analysis; configuring mysql it should be possible I guess; ah that's right configuring Sonar for Eclipse would mean having a MySQL server too, I don't think the embedded default database would fit...)

(In reply to comment #6)
> So currently, it cannot be the same build that is run on hudson,eclipse.org,
> and which has reports on nemo.sonarsource.org.

makes sense, I imagine it is because they don't want to open their mysql db instance to the whole internet; maybe if we can give them just one ip from hudson.eclipse.org they would accept.
Evgeny, do you think it is posssible ?

All in all I don't think duplicating the CI build on another machine is acceptable... whether Eclipse has its own Sonar instance (with MySQL) whether we find a public sonar instance that accepts to host Sonar analysis from hudson.eclipse.org

Great initiative by the way !
Comment 11 Mickael Istria CLA 2012-01-05 10:54:00 EST
So, will 2012 be the year of static code analysis and easy coverage reports at Eclipse.org ? I hope so!
Would it be possible for webmasters to set uo a basic sonar.eclipse.org ?
Comment 12 Xavier Seignard CLA 2012-01-06 05:12:07 EST
If any help is needed, I'll be more than pleased to help.

Regards,

Xavier
Comment 13 Mickael Istria CLA 2012-01-06 07:41:00 EST
This would also reduce the set of responsabilities of the CBI parent pom: http://wiki.eclipse.org/Maven/Parent_POM
The parent pom aims to provide analysis, but this would be almost free thanks to "mvn sonar:sonar" and the Jenkins Sonar plugin.
Comment 14 Mickael Istria CLA 2012-04-02 04:28:58 EDT
I was told during EclipseCon that it could be possible to get a VM installed at Eclipse.org where someone (let's say me) could set up a basic instance of Sonar in order to share feedback with the rest of the community.

IMHO, Sonar is now a must-have, especially if in the context of a Long Term Support initative, and should be part of the CBI things.

So, is it possible to get a test sandbox? Wayne, Denis?
Comment 15 Mike Milinkovich CLA 2012-04-05 09:49:59 EDT
From what I can tell, Sonar is open sourced under the GPLv3. Can you please confirm that you would only be using the pure open source versions of Sonar? 

In the interests of vendor neutrality, we cannot have any proprietary extensions running on our servers, even if they are offered at no charge by the vendor.
Comment 16 Mickael Istria CLA 2012-04-05 09:57:39 EDT
> Can you please confirm that you would only be using the pure open source versions of Sonar?

I've never used any proprietary extension of Sonar, and so far I have always been happy with it.
But maybe that would require further investigation to see whether usage of Sonar@Eclipse does not require proprietary extensions (for example about SSO and so on).

That may be something to discuss with SonarSource guys.
Comment 17 Evgeny Mandrikov CLA 2012-04-05 10:21:46 EDT
Note from me with SonarSource hat: Which kind of SSO do you need? If LDAP, then Sonar LDAP Plugin is also under LGPLv3 - http://docs.codehaus.org/display/SONAR/LDAP+Plugin

P.S. feel free to contact us, if you need more details or help...
Comment 18 Denis Roy CLA 2012-04-05 11:18:06 EDT
> I was told during EclipseCon that it could be possible to get a VM installed at
> Eclipse.org where someone (let's say me) could set up a basic instance of Sonar
> in order to share feedback with the rest of the community.

If your installation doesn't need to handle authentication or use shared filesystems, then I can see if we have available slots.  Can you please confirm?
Comment 19 Mickael Istria CLA 2012-04-05 11:20:44 EDT
(In reply to comment #18)
> If your installation doesn't need to handle authentication or use shared
> filesystems, then I can see if we have available slots.  Can you please
> confirm?

I think it is ok for this POC.
Comment 20 Mickael Istria CLA 2012-04-18 11:57:10 EDT
(In reply to comment #18)
> If your installation doesn't need to handle authentication or use shared
> filesystems, then I can see if we have available slots.

What are the next step for this?
Comment 21 Alex Lagarde CLA 2012-04-19 07:33:02 EDT
+1 For having a Sonar Instance, thanks to Xavier Intent is ready for it https://hudson.eclipse.org/hudson/job/mylyn-docs-intent-0.7-nightly/ .
Comment 22 Mickael Istria CLA 2012-06-12 07:18:20 EDT
(In reply to comment #18)
> > I was told during EclipseCon that it could be possible to get a VM installed at
> > Eclipse.org where someone (let's say me) could set up a basic instance of Sonar
> > in order to share feedback with the rest of the community.
> 
> If your installation doesn't need to handle authentication or use shared
> filesystems, then I can see if we have available slots.  Can you please
> confirm?

Is it still possible to get a VM where we could start some experiments?
Comment 23 Denis Roy CLA 2012-06-12 10:55:59 EDT
Hi Mickael,

I've checked, and we currently don't have any available slots for virtual servers.  Even existing servers are struggling with the limited RAM allocations.

We have some older hardware that is scheduled to be retired soon (within a couple of months).  That hardware will be repurposed for experimental vservers such as is required here.

We'll keep you informed.
Comment 24 Mickael Istria CLA 2012-08-21 13:04:48 EDT
Hi Denis,
Any news about those experimental hardware slots?
Comment 25 Mickael Istria CLA 2012-09-06 13:06:41 EDT
All the discussion about Coverage and overall quality of Eclipse 4.2 makes me think that it's probably time to push up code quality analysis...
Comment 26 Denis Roy CLA 2012-09-19 14:02:27 EDT
What are the hardware requirements for this?  If I provide a vserver with 1G RAM and a 20G disk is that sufficient?
Comment 27 Mickael Istria CLA 2012-09-20 02:19:33 EDT
(In reply to comment #26)
> What are the hardware requirements for this?  If I provide a vserver with 1G
> RAM and a 20G disk is that sufficient?

Requirements are listed here: http://docs.codehaus.org/display/SONAR/Requirements
I think what you're proposing is fine. At least, it will be good enough for some experimental integrations. It will probably require some more HW resources later if all Eclipse projects use it, but we're not there yet.
Comment 28 Denis Roy CLA 2012-09-21 10:53:54 EDT
I'm just setting up the server now.  You'll be able to log into the server from your shell account on build.eclipse.org.  The server will be in a non-routable IP address space, but you'll have Internet access via our Proxy server.

We'll then Proxy http[s]://dev.eclipse.org/sonar to the server's port 8080 when sonar is set up.
Comment 29 Denis Roy CLA 2012-09-21 14:31:33 EDT
SSH credentials sent to Mickael.
Comment 30 Wim Jongman CLA 2012-09-22 09:00:40 EDT
(In reply to comment #29)
> SSH credentials sent to Mickael.

Cool. Are you going to include the Nebula builds as well Mickael?
Comment 31 Mickael Istria CLA 2012-09-24 04:08:04 EDT
(In reply to comment #30)
> Cool. Are you going to include the Nebula builds as well Mickael?

I'll use the projects are work on as guineq pigs. That includes Nebula, SWTBot, and GMF-Tooling.
I'll post a blog when there I have things to show. Then we'll be able to discuss about how to use it more widely.
Comment 32 Mickael Istria CLA 2012-09-27 09:13:10 EDT
I could successfully build with command-line SWTBot on build.eclipse.org and get code quality reports in Sonar: https://dev.eclipse.org/sonar/dashboard/index/1

Integration with Jenkins is not yet tested (waiting for bug 390453 ), but I expect no big trouble.
So I think next step is thinking about managing users and permissions. Denis, what do you think is feasilble? Sonar has an LDAP plugin: http://docs.codehaus.org/display/SONAR/LDAP+Plugin
Comment 33 Mickael Istria CLA 2012-10-08 06:18:22 EDT
Sonar is up and running and is integrated with Hudson-Sandbox when using Maven to build. See https://hudson.eclipse.org/sandbox/job/SWTBot-Sonar/ for an example of configuration.
Although Sonar login is still restricted to admin, you can set up a job to push reports to Sonar, you'll be able to browse reports anonymously, but it's already very interesting!

I'm preparing a blog post for some details on how to configure a job to push to Sonar, along with some recommandations on how to use it in order to improve your code.
Comment 34 Mickael Istria CLA 2012-10-08 11:42:27 EDT
I gave some inputs on how to set up your project to use Sonar in this blog post: http://mickaelistria.wordpress.com/2012/10/08/sonar-at-eclipse-org/
Comment 35 Denis Roy CLA 2012-11-15 13:22:03 EST
From bug 393586, there could be a perception, or it could be interpreted that by using the OSS version of Sonar we (the Foundation) are favouring the company SonarSource.  

If that is the case, in the interest of being vendor neutral, we should perhaps consider discontinuing it usage.  So far there are only 2-3 projects using it.
Comment 36 Mike Milinkovich CLA 2012-11-15 13:40:42 EST
(In reply to comment #35)
> From bug 393586, there could be a perception, or it could be interpreted
> that by using the OSS version of Sonar we (the Foundation) are favouring the
> company SonarSource.  
> 
> If that is the case, in the interest of being vendor neutral, we should
> perhaps consider discontinuing it usage.  So far there are only 2-3 projects
> using it.

The rule has always been that we will use free and open source software, even if it originates from a single company. A concrete example is that we use Nexus for maven.eclipse.org, but we will not use Maven Pro because that`s a Sonatype product.

As long as we are using the pure FLOSS version of Sonar, I don`t see an issue.

You may want to read the policy at http://www.eclipse.org/org/documents/Eclipse_Using_Proprietary_Tools_Final.php
Comment 37 Nobody - feel free to take it CLA 2012-11-15 13:54:21 EST
(In reply to comment #35)
> From bug 393586, there could be a perception, or it could be interpreted
> that by using the OSS version of Sonar we (the Foundation) are favouring the
> company SonarSource.  
> 
> If that is the case, in the interest of being vendor neutral, we should
> perhaps consider discontinuing it usage.  So far there are only 2-3 projects
> using it.

We should not discontinue usages based on perceptions but based on rules.
Comment 38 Mickael Istria CLA 2012-12-08 09:41:59 EST
Sonar is FOSS, and is then legally available there for Eclipse projects: https://dev.eclipse.org/sonar
Comment 39 Mickael Istria CLA 2013-08-19 07:01:48 EDT
For the record, here is the entry-point for documentation about Sonar @eclipse.org: http://wiki.eclipse.org/Sonar
Comment 40 Denis Roy CLA 2013-09-26 09:22:55 EDT
*** Bug 154159 has been marked as a duplicate of this bug. ***