Community
Participate
Working Groups
We should measure code coverage as part of running tests during continuous build. Instructions can be found on bug 154159 and on the Eclipse wiki: http://wiki.eclipse.org/index.php/Code_Coverage_with_Emma
Turns out that measuring of test coverage is fairly straight forward to integrate with the current build using cobertura (http://cobertura.sourceforge.net/). Is there any interest in generating code coverage report as part of test execution? I am in the process of setting up a hudson instance at http://mylyn.eclipse.org/hudson/ which would make the results accessible to everyone. If the server has sufficient resources tests would run on every commit or nightly otherwise.
I've done this with cobertura. The classes under test must be instrumented, and the instrumented classes must have their classpath augmented with the cobertura runtime. To do this I had the ant build alter the bundles under test: extract and instrument their classes, and alter the manifest to include a cobertura bundle. I hope that there's a better way, since what I did was non-trivial and adds significantly to the build time.
I simply instrument jars after extracting them to the dropins folder and then add cobertura to the boot class path of the JVM that launches the Eclipse instance running tests. The overhead seems minimal, 10% at most.
See sample "coverage report from Textile-J":https://textile-j.dev.java.net/builds/net.java.textilej/2.2.864/coverage/frame-summary.html
+1, it's nice to have code coverage. That said, see this related article "Code coverage tools can be misleading":http://greensopinion.blogspot.com/2008/05/code-coverage-tools-can-be-misleading.html
That's a great post. I very much agree with the point that measuring code coverage can be used as a tool to find code that wasn't executed but it's difficult to read other results out of the reports. As discussed on the call I'll go ahead and setup build and test plans on the mylyn.eclipse.org hudson in the next couple of weeks. Coverage reports will be accessible from the build server and may either run nightly or more regularly depending on the additional overhead imposed by measuring test coverage.
+1 for doing this Some comparisons: * http://raibledesigns.com/rd/entry/emma_vs_cobertura_for_code * http://rwatsh.blogspot.com/2008/03/emma-vs-cobertura.html
https://docs.sonatype.org/display/TYCHO/How+to+integrate+EMMA+%28code+coverage%29+with+Tycho
We could also think about using JaCoCo (follow up of EclEmma), Platform is already doing this with JaCoCo and it's IP approved. * http://eclemma.org/jacoco/index.html * http://wiki.eclipse.org/JaCoCo/Proposal * http://relengofthenerds.blogspot.com/2011/03/sdk-code-coverage-with-jacoco.html
I added support for JaCoCo to the parent pom. To meassure test coverage -Dcoverage.skip=false needs to be specified. Coverage and other quality reports are published to http://mylyn.org/sonar/. This is currently configured for Mylyn Builds and Mylyn Docs but I plan on adding additional builds in the next few days.
