Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 245251 - Thread safety issues in using Equinox PackageAdmin
Summary: Thread safety issues in using Equinox PackageAdmin
Status: RESOLVED DUPLICATE of bug 247522
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: 3.4   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 3.5   Edit
Assignee: equinox.framework-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 247636
  Show dependency tree
 
Reported: 2008-08-26 10:13 EDT by Glyn Normington CLA
Modified: 2009-02-04 15:42 EST (History)
2 users (show)

See Also:


Attachments
Testcase (6.64 KB, application/octet-stream)
2008-08-28 05:48 EDT, Glyn Normington CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Glyn Normington CLA 2008-08-26 10:13:04 EDT
Build ID: Equinox 3.4.0

Steps To Reproduce:
Intermittent. It is possible to reproduce the symptom below by installing bundles concurrently on multiple threads via the bundle context API.

The most recent symptom was:
Caused by: java.lang.ArrayIndexOutOfBoundsException: 77
at org.eclipse.osgi.framework.util.KeyedHashSet.elements(KeyedHashSet.java:173)
at org.eclipse.osgi.internal.resolver.StateImpl.getBundles(StateImpl.java:203)
at org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl.createState(StateObjectFactoryImpl.java:313)
at org.eclipse.osgi.internal.baseadaptor.StateManager.getState(StateManager.java:251) 

Unfortunately, the testcase which case reproducing predictably is now passing.

The purpose of raising this bug report was to enable us to wrap Equinox in a thread-safety wrapper which ensures only one thread uses Equinox at once. Details follow.

More information:

To coordinate the use of the Equinox framework with our code, we have wrapped PackageAdminImpl in a layer that, for each public method, synchronizes on an object before delegating each call. Unfortunately, there is a hole in this scheme because refreshPackages runs a thread which calls  the synchronized method doResolveBundles. doResolveBundles results in a calling sequence which calls a public method of PackageAdmin. The net is that this deadlocks ([1]) since the wrapper lock and the default synchronization lock are obtained in different orders on different threads.

We could override refreshPackages to solve this, but we'd have to reproduce its logic pretty much verbatim to ensure compatibility with Equinox and we then get into legal issues since we'd be copying EPL code into a GPL codebase (or we'd have to manage a separate EPL jar just for this wrapper class, which is pretty awful). Also, copying code isn't a great technical solution.

A much cleaner solution would be if we could override and wrap doResolveBundles, but we can't do this since doResolveBundles has package visibility. (We can't add our wrapper to the same package as PackageAdminImpl as this breaks the JAR signing.)

So please would you consider making doResolveBundles protected, rather than having package visibility? If you feel it is too sensitive, another solution is for the Thread.run method in refreshPackages to call a new protected method, say runDoResolveBundles, which simply calls the existing doResolveBundles. Then we could overload runDoResolveBundles and delegate to it in the normal way.

[1] https://issuetracker.springsource.com/browse/PLATFORM-146
Comment 1 Thomas Watson CLA 2008-08-26 12:06:57 EDT
I opened bug 245271 to change the visibility of the internal method PackageAdminImpl.doResolveBundles to protected for the 3.4.x stream to allow you to make progress.  

I am leaving this bug for 3.5 to address the thread safety issues when accessing the resolver state.  Glyn, any testcases you can provide to reproduce would be appreciated (even if they are ones that used to fail but are passing with luck now).
Comment 2 Glyn Normington CLA 2008-08-28 05:48:22 EDT
Created attachment 111180 [details]
Testcase

I have attached a stand-alone testcase which may reproduce this bug or at least other thread-safety issues in Equinox.

Here's the install test, packaged as an Eclipse project. I've bundled Equinox in the zip as it's part of the project but you may want to delete it...

The test should be fairly self-explanatory. There are three variables:

- bundleCount: controls the number of bundles to be created and installed per thread
- threadCount: controls the number of threads that will perform bundle installation
- addDataToBundles: controls whether or not the created bundles will contain any data beyond a manifest. This was added as we found we could get different errors depending on the size of the bundles, presumably because it alters a timing window as Equinox copies and extracts the bundle to its storage area.

Unfortunately the test doesn't always produce the array index out of bounds exception but it usually produces an error of some sort with threadCount >= 2 and a reasonable number of bundles.
Comment 3 Glyn Normington CLA 2008-08-28 05:50:49 EDT
(In reply to comment #2)
> Here's the install test, packaged as an Eclipse project. I've bundled Equinox
> in the zip as it's part of the project but you may want to delete it...

This text should not have been in the comment. Someone else wrote the testcase and I deleted Equinox from the zip file to keep the attachment small. Sorry for any confusion!
Comment 4 Thomas Watson CLA 2009-02-04 15:42:22 EST
I'm marking this as a dup of bug 247522.  The resolver/state implementation did not have proper thread safety which surfaced in many unpredictable ways when accessing, resolving, modifying the resolver state from multiple threads.

The State implementation has been made thread safe in 3.5 (M4) which will fix the original PackageAdmin issue from this bug report.

*** This bug has been marked as a duplicate of bug 247522 ***