Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 540419

Summary: Fail Gerrit build if new Java warnings are introduced
Product: [Eclipse Project] Platform Reporter: Lars Vogel <Lars.Vogel>
Component: UIAssignee: Alexander Kurtakov <akurtakov>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: akurtakov, daniel_megert, Lars.Vogel, mat.booth, michael.keppler, rolf.theunissen, sravankumarl
Version: 4.10   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
See Also: https://git.eclipse.org/r/141950
Whiteboard:
Bug Depends on: 506778, 547470    
Bug Blocks:    

Description Lars Vogel CLA 2018-10-24 04:53:52 EDT
Just discussed that with Michael Keppler on EclipseCon that is would be nice to fail the Gerrit verification build, if the number of warnings increase with a change.

Possible solutions I can think of:

1.) Find a Jenkins plug-ins which supports this
2.) Write a test which extracs the number of Java warnings from a project and compare it to a predefined value
Comment 1 Michael Keppler CLA 2018-10-28 09:28:56 EDT
Generally that is possible, but there is a twist to it. You can have all kinds of warnings be reported and collected in Jenkins using the Warnings plugin (previously available as a whole bunch of plugins for Checkstyle, Findbugs, PMD, ...). See https://github.com/jenkinsci/warnings-plugin/blob/master/doc/Documentation.md for the documentation.

And that plugin (as well as many others) allows to configure thresholds to set a build to failed or unstable. Those thresholds can be absolute numbers of issues or new issues. See https://github.com/jenkinsci/warnings-plugin/blob/master/doc/Documentation.md#quality-gate-configuration

That is almost what you need. However, in my experience with gerrit based builds in my company, new issues doesn't mean "new issues in this commit", but "new issues since last build of this project". Therefore this delta is not really useful for gerrit builds, as it just tells the difference since another arbitrary gerrit build.

Still, we could use this "new issues" reporting on the master build, instead of on the gerrit build, to at least have some more reporting. This would work by marking an otherwise green build as yellow (unstable), if new issues have been found since last build of master. Since the plugin supports git blame, developers can also be identified easily.

One last word of warning: I have used the predecessor plugins of the warning plugin for several years and they worked fine, but the new plugin is currently only in beta (and goes through a major rewrite), so there might be some more hickups than normal.
Comment 2 Sravan Kumar Lakkimsetti CLA 2019-05-05 23:12:30 EDT
@Mat,

Can you take look in to this?
Comment 3 Dani Megert CLA 2019-05-06 03:42:44 EDT
I think it will be hard to achieve (see comment 1).
Comment 4 Dani Megert CLA 2019-05-10 06:08:00 EDT
This will not work at all for the following repositories
- eclipse.platform.runtime
- eclipse.platform.ui
because they specify -nowarn in their root pom.xml.
Comment 5 Lars Vogel CLA 2019-05-10 06:14:37 EDT
(In reply to Dani Megert from comment #4)
> This will not work at all for the following repositories
> - eclipse.platform.runtime
> - eclipse.platform.ui
> because they specify -nowarn in their root pom.xml.

Lets try to remove it and see what happens.
Comment 6 Eclipse Genie CLA 2019-05-10 06:16:55 EDT
New Gerrit change created: https://git.eclipse.org/r/141950
Comment 7 Dani Megert CLA 2019-05-10 06:24:30 EDT
(In reply to Lars Vogel from comment #5)
> (In reply to Dani Megert from comment #4)
> > This will not work at all for the following repositories
> > - eclipse.platform.runtime
> > - eclipse.platform.ui
> > because they specify -nowarn in their root pom.xml.
> 
> Lets try to remove it and see what happens.
Well, the proposed change will work, but you insisted in bug 506778 to use project settings. I was always against that. And current status is that it causes bug 546976.
Comment 8 Dani Megert CLA 2019-05-10 08:03:23 EDT
(In reply to Dani Megert from comment #7)
> (In reply to Lars Vogel from comment #5)
> > (In reply to Dani Megert from comment #4)
> > > This will not work at all for the following repositories
> > > - eclipse.platform.runtime
> > > - eclipse.platform.ui
> > > because they specify -nowarn in their root pom.xml.
> > 
> > Lets try to remove it and see what happens.
> Well, the proposed change will work, but you insisted in bug 506778 to use
> project settings. I was always against that. And current status is that it
> causes bug 546976.
Lars, your proposed change also fixes the comparator bug 546976. I would really like to revert bug 546976. Besides clearing the road for this bug here  and fixing the other bug, I am still convinced that it is better that Gerrit reports what also the official build reports, e.g. same warnings and errors. If you are OK with that, I will handle the revert.
Comment 9 Lars Vogel CLA 2019-05-10 08:16:39 EDT
(In reply to Dani Megert from comment #8)
> Lars, your proposed change also fixes the comparator bug 546976. I would
> really like to revert bug 546976. Besides clearing the road for this bug
> here  and fixing the other bug, I am still convinced that it is better that
> Gerrit reports what also the official build reports, e.g. same warnings and
> errors. If you are OK with that, I will handle the revert.

+1
Comment 10 Dani Megert CLA 2019-05-10 09:18:06 EDT
(In reply to Lars Vogel from comment #9)
> (In reply to Dani Megert from comment #8)
> > Lars, your proposed change also fixes the comparator bug 546976. I would
> > really like to revert bug 546976. Besides clearing the road for this bug
> > here  and fixing the other bug, I am still convinced that it is better that
> > Gerrit reports what also the official build reports, e.g. same warnings and
> > errors. If you are OK with that, I will handle the revert.
> 
> +1
Thanks Lars, much appreciated!
Comment 11 Dani Megert CLA 2019-05-10 09:21:47 EDT
(In reply to Michael Keppler from comment #1)
> Generally that is possible, but there is a twist to it. You can have all
> kinds of warnings be reported and collected in Jenkins using the Warnings
> plugin (previously available as a whole bunch of plugins for Checkstyle,
> Findbugs, PMD, ...). See
> https://github.com/jenkinsci/warnings-plugin/blob/master/doc/Documentation.md
> for the documentation.
> 
> And that plugin (as well as many others) allows to configure thresholds to
> set a build to failed or unstable. Those thresholds can be absolute numbers
> of issues or new issues. See
> https://github.com/jenkinsci/warnings-plugin/blob/master/doc/Documentation.md#quality-gate-configuration
> 
> 
> That is almost what you need. However, in my experience with gerrit based
> builds in my company, new issues doesn't mean "new issues in this commit",
> but "new issues since last build of this project". Therefore this delta is
> not really useful for gerrit builds, as it just tells the difference since
> another arbitrary gerrit build.
For us this would work because our policy is to be COMPILE warning free.

Could another option be to fail the build based on some console output, i.e. simply fail when we see a compile warning in the console output?
Comment 12 Mat Booth CLA 2019-05-31 11:46:04 EDT
(In reply to Dani Megert from comment #11)
> Could another option be to fail the build based on some console output, i.e.
> simply fail when we see a compile warning in the console output?

Doesn't the compiler have a "treat warnings as errors" flag?
Comment 13 Alexander Kurtakov CLA 2019-05-31 11:56:58 EDT
(In reply to Mat Booth from comment #12)
> (In reply to Dani Megert from comment #11)
> > Could another option be to fail the build based on some console output, i.e.
> > simply fail when we see a compile warning in the console output?
> 
> Doesn't the compiler have a "treat warnings as errors" flag?

It's in the works Bug 547470  and Bug 547591 after that we should be able to pass -Dmaven.compiler.failOnWarning for that.
Comment 14 Alexander Kurtakov CLA 2019-09-20 12:37:09 EDT
SWT repo now has -Dmaven.compiler.failOnWarning=true passed to maven and warning causes build failure as in https://ci.eclipse.org/platform/job/eclipse.platform.swt-Gerrit/167/console. 
I'm going to resolve this bug as it's up to each repo to now get to clean build and enable the setting.
Comment 15 Lars Vogel CLA 2019-09-20 12:48:33 EDT
Kudos to the SWT team for having warnings