Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 349164 - Mockito cannot mock the classes under development
Summary: Mockito cannot mock the classes under development
Status: NEW
Alias: None
Product: Orbit
Classification: Tools
Component: bundles (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Ketan Padegaonkar CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 506247
Blocks:
  Show dependency tree
 
Reported: 2011-06-13 02:44 EDT by Frank Appel CLA
Modified: 2018-04-30 12:38 EDT (History)
15 users (show)

See Also:


Attachments
target for reproducing problem (1.63 MB, application/octet-stream)
2011-06-13 02:44 EDT, Frank Appel CLA
no flags Details
Workspace for reproducing the problem (1.48 MB, application/octet-stream)
2011-06-13 03:00 EDT, Frank Appel CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frank Appel CLA 2011-06-13 02:44:16 EDT
Created attachment 197871 [details]
target for reproducing problem

Build version: 1.8.4 (org.mockito_1.8.4.v201102171835.jar).
Steps to reproduce (see attached workspace and target for easy reproduction):
  - create an empty plugin project
  - create test fragment for this project
  - use a target that provides mockito, junit, hamcrest and objenesis
  - set the bundle dependencies in the fragment (junit, mockito, objenesis)
  - create a public non final class in the project
  - mock this class in a junit test in the fragment
  - launching the test throws the following Exception:

org.mockito.exceptions.base.MockitoException: 
Mockito cannot mock this class: class org.mockito.problem.MyClassToMock
Mockito can only mock visible & non-final classes.

You can easily reproduce this problem by using the attachments to this bug. Extract the workspace.zip and open Eclipse using this workspache. Create a new target definition (File -> New -> Other, Target Definition) in the org.mockito.problem test to use the attached target.zip. To do so extract the target.zip and open the target definition file with the target definition editor. Add a directory location pointing to the extracted target directory. Don't forget to push "Set as Target Platform" in the right upper corner of the editor once your target content is selected. Refresh your Workspace (F5 on package explorer). Now run MyClassToMock_Test in the org.mockito.problem.test fragment.

The workspace contains a CVS-checkout of v1_8_4 branch as closed project. Opening this project the JUnit test runs without problem.

After debugging the problem for a while, it seems that the signing process during eclipse packaging exposes the problem. Looking at org.mockito.cglib.core.ReflectUtils line 41 you'll see how the class variable PROTECTION_DOMAIN is set using ReflectionUtils.class.getProtectionDomain(). Later in line 383 (defineClass) this protection domain is used to actually define the byte array containing the cglib mock as class.

As far as I understand this, the eclipse bundle packaging process defines signing information for the ReflectionUtils.class. These are represented in the protection domain used by the code in question. The package signers of the original class do not fit with those and cause the defining process to fail.
Comment 1 Frank Appel CLA 2011-06-13 03:00:20 EDT
Created attachment 197872 [details]
Workspace for reproducing the problem

Unfortunately it turns out that the workspace zip containing the plugin infos is to big for upload. Therefore the zip only contains the projects. You'll have to import those projects into an empty workspace using File -> Import, Existing Project into Workspace. After that follow the description in the first comment.
Comment 2 Holger Staudacher CLA 2011-06-13 04:44:35 EDT
I'm facing the same problems.
Comment 3 David Williams CLA 2011-06-13 16:44:11 EDT
Assigning to Ketan, as the listed "contact" for this Orbit bundle. 

It would take some time (and study) for me to understand the technical issues, but a quick internet search found at least one related mention. At 

http://stackoverflow.com/questions/2981476/does-jmockit-have-any-drawbacks-at-all

it is mentioned that "You can't use the signed junit.jar file shipped with Eclipse" ... so ... not sure if that's the exact same issue you are seeing ... or, if you are already using an unsigned junit.jar and we'd also have to not sign mockito for it to work? 

Before we'd decide not to sign mockito.jar, I'd like to understand better exactly why that is. Is it a permanent design limitation? Is there some fix (besides not signing)?
Comment 4 Ketan Padegaonkar CLA 2011-06-13 18:32:35 EDT
I'm a bit confused in that I do not understand much of the signing process and how this signing validation works and how it fits with the classloading mechanism (I only have a rough idea about things). Are you suggesting that this is an issue with the orbit bundle OR the signing authority on various bundles OR hamcrest itself?

Having said that:

Mocks aren't stubs<http://martinfowler.com/articles/mocksArentStubs.html>. Mocking is for interfaces, stubbing is for concrete classes. AFAIK, mockito works by using asm to bytecode manipulate classes which doesn't fly with signed classes.

Pretty much every developer of mock frameworks have been my colleagues at some point and everyone recommends not mocking concrete classes.

This is an issue we've seen working with mockito and some other mocking frameworks with mocking concrete classes that are signed. Since then we've moved away to hiding the eclipse classes behind interfaces to hide behind an interface to be able to unit test without loading up eclipse and try to mock concrete classes since its a smell anyways.
Comment 5 Frank Appel CLA 2011-06-14 00:55:52 EDT
(In reply to comment #4)
> Mocks aren't stubs<http://martinfowler.com/articles/mocksArentStubs.html>.
> Mocking is for interfaces, stubbing is for concrete classes. AFAIK, mockito
> works by using asm to bytecode manipulate classes which doesn't fly with signed
> classes.
I'm aware of that article, but I understand it that way that mocks are used for  behaviour verification and stubs are used for state verification "[...]Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification". So maybe I'm getting this wrong, but I don't see the point of distinguishing here between interface and class types.

Anyway, I only used a class type for simplification of the example. The same problem occurs with interfaces. 

> This is an issue we've seen working with mockito and some other mocking
> frameworks with mocking concrete classes that are signed.

It is important to point out that neither the interfaces nor the classes that I want to mock are signed, since they are types that exists in my workspace projects.   

(In reply to comment #3)
> it is mentioned that "You can't use the signed junit.jar file shipped with
> Eclipse" ... so ... not sure if that's the exact same issue you are seeing ...
> or, if you are already using an unsigned junit.jar and we'd also have to not
> sign mockito for it to work? 
I'm not an expert with signing stuff, but after debugging the code of mockito I think it is a different problem that is located in the library itself.
 
> Before we'd decide not to sign mockito.jar, I'd like to understand better
> exactly why that is. Is it a permanent design limitation? Is there some fix
> (besides not signing)?

This is how mockito determines the projection domain used later for class definition.


static {
  PROTECTION_DOMAIN = (ProtectionDomain)AccessController
    .doPrivileged(new PrivilegedAction() {
       public Object run() {
         return ReflectUtils.class.getProtectionDomain();
       }
     });


The class definition takes place here:


public static Class defineClass(String className, 
                                byte[] b,
                                ClassLoader loader)
  throws Exception
{
  Object[] args = new Object[]{
    className,
    b,
    new Integer(0),
    new Integer(b.length),
    PROTECTION_DOMAIN
  };
  return (Class)DEFINE_CLASS.invoke(loader, args);
}

DEFINE_CLASS is a Method object that points to the according java.lang.ClassLoader.defineClass(...). The latter method calls ClassLoader.checkCerts(). A comment in that code states

// first class in this package gets to define which
// certificates must be the same for all other classes
// in this package

If you debug the example above you will see that the first class loaded in the package is the class or the interface that you want to mock. As the certificates of the mock are provided by the PROTECTION_DOMAIN.getCodeSource().getCertificates() they do not match. This throws a SecurityException, which is swallowed. But it leads to the misleading exception message mentioned in my description above.

Not signing mockito would solve the problem described above, since then the certificates of both classes would be the same (that is to say none). But once you want to mock a class that actually has certificates, I would expect that the same problem occurs, since the certificates again would not match.

Maybe I'm getting this all wrong, since it isn't my domain at all. But I think a solution would be to retrieve the protection domain of the class I want to mock instead using the static approach.

So having said all this and if I'm not mistaken it's maybe a bug in mockito itself and should be reported to the mockito project - what do you think?
Comment 6 Ketan Padegaonkar CLA 2011-06-14 02:13:51 EDT
(In reply to comment #5)
> So having said all this and if I'm not mistaken it's maybe a bug in mockito
> itself and should be reported to the mockito project - what do you think?

This is difficult to say since orbit is just an aggregation of upstream libs that are maintained elsewhere. Mockito in the form as it is bundled at orbit is an exact binary as downloaded from the mockito website.

If you're able to reproduce this with upstream mockito, it's likely a bug with mockito. This could also be a case of the environment in which mockito runs within the OSGi runtime and the classloader magic that happens in there. I'm not qualified to comment on either as I'm merely a consumer of mockito as you are. I'm happy to connect you to the upstream mockito author since he's a good friend :)
Comment 7 Daniel Schneller CLA 2012-04-20 11:23:16 EDT
Has there been any progress on this issue? I am facing the same problem.
Comment 8 Holger Staudacher CLA 2012-04-20 11:25:31 EDT
Maybe it's noteworthy that the new Mockito version ship as OSGi bundles.
Comment 9 Markus Knauer CLA 2012-11-07 10:52:17 EST
(In reply to comment #5)
> Not signing mockito would solve the problem described above, since then the
> certificates of both classes would be the same (that is to say none). But
> once you want to mock a class that actually has certificates, I would expect
> that the same problem occurs, since the certificates again would not match.

Yep, that seems to be true. I am running into this very issue where the bundle under test is signed, and the fragment with the test classes is not signed, e.g. this snippet from a stack trace:

Caused by: org.apache.maven.surefire.testset.TestSetFailedException: org.eclipse.rap.rwt.osgi.internal.ApplicationLauncherImpl_Test; nested exception is java.lang.SecurityException: class "org.eclipse.rap.rwt.osgi.internal.ApplicationReferenceImpl"'s signer information does not match signer information of other classes in the same package

(In reply to comment #8)
> Maybe it's noteworthy that the new Mockito version ship as OSGi bundles.

I tried it with an unmodified version (i.e. org.mockito.mockito-all_1.9.0.jar) and faced the same problems. Note that this bundle jar is not signed.
Comment 10 Markus Knauer CLA 2012-11-07 11:06:50 EST
Cross-referencing to a signing issue reported against Mockito:

Issue 393:
SecurityException when using spy() on class from signed JAR file
http://code.google.com/p/mockito/issues/detail?id=393
Comment 11 Gunnar Wagenknecht CLA 2014-04-15 07:05:33 EDT
(In reply to Frank Appel from comment #5)
> Not signing mockito would solve the problem described above, since then the
> certificates of both classes would be the same (that is to say none). But
> once you want to mock a class that actually has certificates, I would expect
> that the same problem occurs, since the certificates again would not match.

I think the analysis is correct. Just run into the same issue again when introducing Mockito to a new project. The Orbit bundle is signed, which causes the problems.

And I keep forgetting the workaround I have in place:
File -> Import -> Plug-ins -> From Repository -> org.mockito

This checks out Mockito from Orbit CVS as a project (which is unsigned) and suddenly it works.
Comment 12 David Williams CLA 2014-04-17 10:44:14 EDT
Since there is a work around, perhaps we just just add a note do ipLog.xml file about "how to use" (or, point to a wiki) and count that as the resolution of this bug? 

While technically we could leave it unsigned, if absolutely necessary, it sounds like that would "mess up" people who where using it in "final builds" for production, where their plugins or fragments were signed? Then they'd still get the "not matching signatures" problem?
Comment 13 Gunnar Wagenknecht CLA 2014-07-17 16:45:53 EDT
FYI:

I submitted a pull request to CGLIB in order to allow Mockito (and other users of CGLIB) to use the proper ProtectionDomain when generating classes.

https://github.com/cglib/cglib/pull/15

Feedback is welcome!
Comment 14 Alexander Nyßen CLA 2016-10-19 23:45:40 EDT
It seems Mockito has moved from CGLIB to ByteBuddy with its 2.0 release (see https://github.com/mockito/mockito/issues/248). As such, this could probably be removed by updating Mockito.
Comment 15 Lorenzo Bettini CLA 2018-01-25 03:12:01 EST
(In reply to Alexander Nyßen from comment #14)
> It seems Mockito has moved from CGLIB to ByteBuddy with its 2.0 release (see
> https://github.com/mockito/mockito/issues/248). As such, this could probably
> be removed by updating Mockito.

But Mockito from Orbit is still 1.9...
Comment 16 Roland Grunberg CLA 2018-04-30 12:38:10 EDT
For testing purposes, 
http://download.eclipse.org/tools/orbit/N-builds/N20180427095450/
http://download.eclipse.org/tools/orbit/N-builds/N20180427095450/repository

Latest mockito is now 2.13.0, which should be part of our eventual Photon M7 contribution.