Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 368722

Summary: m2e Skip Tests does not skip when maven-surefire-plugin configuration has skip element
Product: z_Archived Reporter: David Biesack <David.Biesack>
Component: m2eAssignee: Project Inbox <m2e.core-inbox>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: David.Biesack, fbricon, igor
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
Eclipse project demonstrating problem none

Description David Biesack CLA 2012-01-16 10:36:01 EST
Build Identifier: Version: Indigo Service Release 1 Build id: 20110916-0149

Running a m2e launch configuration for 'package' still runs
tests even if Skip Tests is set.

Reproducible: Always

Steps to Reproduce:
1. create a new Java project
2. Add Maven nature
3. create this pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>bare.maven</groupId>
  <artifactId>bare.maven</artifactId>
  <version>0.0.1-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <skip>${continuous.integration.build}</skip>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

4. create a source folder src/main/java and add a dummy A.java source file
5. create a soruce folder src/test/java and add a Test.java source file

import static org.junit.Assert.fail;
import org.junit.Test;
public class Test {
    @Test public void failure() {
        fail("force failure.");
    }
}

6. Create a Maven launch configuration for the project to build the 'package' goal.
7. Check the 'Skip Tests' option in the launch configuration.
8. Run the launch configuration.

The tests run even though the option is set to not run them.

(Note that the tests do not run if you add 
continuous.integration.build to the environment with a value of true,
but this should not be necessary.)
Comment 1 David Biesack CLA 2012-01-16 10:52:41 EST
Update platform to all, since this happens for me on Linux and for others on Windows.
Comment 2 Igor Fedorenko CLA 2012-01-16 10:54:35 EST
I am unable to reproduce the problem. Please attach complete standalone maven project that works on command line with "mvn clean package -Dmaven.test.skip=true" but fails when running "clean package" goals with "Skip Tests" from m2e.
Comment 3 David Biesack CLA 2012-01-16 12:08:36 EST
Created attachment 209566 [details]
Eclipse project demonstrating problem

extract this with 'jar -xf m2e-368722.jar' and add to eclipse.

 mvn -Dmaven.test.skip=true clean package

does not run tests.

Add m2e launch config to run 'package' but without 'Skip Tests' set;
run and the tests run (but fail as expected).

Change the launch config and enable 'Skip Tests' and run it.
It still runs the tests; see the log below

Although the tests do not run when using 'clean package',
that is because with Skip Tests, m2e does not even compile
the tests. However, Eclipse will compile the tests when I edit/save them,
and with just the 'package' goal, then m2e will
run them. I expect 'Skip Tests' to skip running tests,
even if they are compiled. (For a large project, we don't
want to always do a 'clean'; we just want to build the jar.)

log:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Unnamed - m2e.368722:368722:jar:0.0.1-SNAPSHOT
[INFO]    task-segment: [package]
[INFO] ------------------------------------------------------------------------
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (ISO-8859-1 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /r/sanyo.unx.sas.com/vol/vol220/u22/sasdjb/eclipse-workspace/m2e-368722/src/main/resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (ISO-8859-1 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /r/sanyo.unx.sas.com/vol/vol220/u22/sasdjb/eclipse-workspace/m2e-368722/src/test/resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Not compiling test sources
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /r/sanyo.unx.sas.com/vol/vol220/u22/sasdjb/eclipse-workspace/m2e-368722/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running bug368722.test.TestFail
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec <<< FAILURE!

Results :

Failed tests: 
  failure(bug368722.test.TestFail)

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to /r/sanyo.unx.sas.com/vol/vol220/u22/sasdjb/eclipse-workspace/m2e-368722/target/surefire-reports for the individual test results.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 second
[INFO] Finished at: Mon Jan 16 11:57:12 EST 2012
[INFO] Final Memory: 10M/148M
[INFO] ------------------------------------------------------------------------
Comment 4 Igor Fedorenko CLA 2012-01-16 12:31:02 EST
The observed behaviour matches Maven command line, i.e. the following sequence of commands executed from command line results in the same failure

   mvn clean test-compile
   mvn -Dmaven.test.skip=true package
Comment 5 David Biesack CLA 2012-01-16 12:35:46 EST
reopen to try use case
Comment 6 David Biesack CLA 2012-01-16 13:24:02 EST
But

 mvn clean test-compile
 mvn -Dmaven.test.skip=true -DskipTests package

does not run the tests

as per 

http://maven.apache.org/plugins/maven-surefire-plugin/examples/skipping-test.html

You can also skip the tests via command line by executing the following command:

mvn install -DskipTests
If you absolutely must, you can also use the maven.test.skip property to skip compiling the tests. maven.test.skip is honored by Surefire, Failsafe and the Compiler Plugin.

mvn install -Dmaven.test.skip=true

i.e. maven.test.skip skips compiling the tests, skipTests skips running them.

Keep in mind that Eclipse is going to be compiling the tests when it updates the project. The "Skip Tests" option should do what it says, skip tests (not just skip compiling the tests), because with Eclipse also compiling the tests, I don't want to have to do 'clean' operations before 'package'.
Comment 7 Igor Fedorenko CLA 2012-01-16 14:17:48 EST
Sorry, I meant

 mvn clean test-compile
 mvn -Dmaven.test.skip=true package

"Skip Tests" is a shortcut for "-Dmaven.test.skip=true" command line option, and as far as I know, maven-surefire-plugin does not provide another way to skip test execution for the attached example project (see [1]).

[1] http://maven.apache.org/plugins/maven-surefire-plugin/examples/skipping-test.html
Comment 8 David Biesack CLA 2012-01-16 14:49:58 EST
I guess what I'm asking for is to make "Skip Tests" be shorthand for

-Dmaven.test.skip=true -DskipTests

as documented on the surefire-plugin page. "Skip Tests" implies
(to me) that tests are skipped. If not, then this should be labeled
"Skip compiling tests" (but that is much less useful, since Eclipse
has probably already compiled them)

The following runs tests via mvn:

  mvn clean test-compile && mvn -Dmaven.test.skip=true package

This makes little sense because test-compile has already compiled
the tests, so it does not save much or offer real benefit.

This compiles but does not run tests:

  mvn clean test-compile && mvn -DskipTests package

and this does not compile or run them:

  mvn -Dmaven.test.skip=true -DskipTests clean package

imho,  it is more important in m2e to be able to skip test execution
than to just skip compiling tests.

(I don't see this bug as INVALID.)
Comment 9 Igor Fedorenko CLA 2012-01-16 15:44:31 EST
I believe you already verified that "mvn -Dmaven.test.skip=true -DskipTests package", i.e. without "clean" does not suppress execution of the tests for the attached project. There is not much we can do about it from m2e end.
Comment 10 David Biesack CLA 2012-01-16 16:13:25 EST
No, to the contrary, I verified that

 mvn -Dmaven.test.skip=true -DskipTests       package
 mvn -Dmaven.test.skip=true -DskipTests clean package
 mvn                        -DskipTests       package

all do what I expect: create the jar without running the tests;
the only difference is whether the tests are compiled or not.

That is, using -DskipTests suppresses test execution as documented
on the plugin page when using mvn command, but not using m2e

I did experiment and found that adding a JRE option

-DskipTests=true

to the m2e launch configuration does what I expect.

This reinforces (in my mind) that m2e should be setting
skipTests when using "Skip Tests" and not just maven.test.skip
Comment 11 Igor Fedorenko CLA 2012-01-16 16:16:49 EST
fair enough.
Comment 12 Igor Fedorenko CLA 2013-09-29 09:58:47 EDT
closing old/stale bugreports
Comment 13 David Biesack CLA 2013-09-30 10:27:43 EDT
Why is this "old/stale"? Why WONTFIX?
This bug is still valid; running m2e 'package' with this option checked
still runs tests if Eclipse has previously compiled them.
The fix is pretty easy: just define skipTests 
(If I define skipTests in the Parameters section
of the main tab of the launch configuration,
the 'package' build correctly skips running the tests
But I should not have to do that, the 'Skip Tests' checkbox should
do that)
Comment 16 David Biesack CLA 2013-11-27 10:23:21 EST
thanks for the fix!
Comment 17 Denis Roy CLA 2021-04-19 13:25:22 EDT
Moved to https://github.com/eclipse-m2e/m2e-core/issues/