Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 362046

Summary: Virgo Tomcat Server dumps after being stopped by Ctrl-C when running with Equinox 3.8
Product: [RT] Virgo Reporter: Glyn Normington <glyn.normington>
Component: runtimeAssignee: Glyn Normington <glyn.normington>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: eclipse, hsiliev, tjwatson
Version: 3.0.1.RELEASE   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
serviceability/logs/log.log extract
none
Equinox 3.8 JAR for reproducing the problem none

Description Glyn Normington CLA 2011-10-26 06:39:48 EDT
When Virgo Tomcat Server is built at HEAD, started normally, and then stopped using Ctrl-C, several similar dumps are produced - see the attached log extract and the example below.

Note this does not happen with Virgo Kernel built at HEAD.

[2011-10-26 11:14:49.848] ERROR iLogServiceListener@788390b0 org.osgi.service.log.LogService                                   Bundle org.eclipse.osgi_3.8.0.N20111022-2000, An unexpected runtime error has occurred. java.lang.IllegalStateException: BundleContext is no longer valid
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.checkValid(BundleContextImpl.java:931)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.getServiceReferences(BundleContextImpl.java:498)
	at org.eclipse.virgo.kernel.osgi.framework.OsgiFrameworkUtils.getServices(OsgiFrameworkUtils.java:114)
	at org.eclipse.virgo.kernel.install.artifact.internal.StandardArtifactStateMonitor.getListenerHolders(StandardArtifactStateMonitor.java:368)
	at org.eclipse.virgo.kernel.install.artifact.internal.StandardArtifactStateMonitor.onStopping(StandardArtifactStateMonitor.java:260)
	at org.eclipse.virgo.kernel.install.artifact.internal.bundle.BundleDriverBundleListener.bundleChanged(BundleDriverBundleListener.java:90)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:847)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
	at org.eclipse.osgi.framework.internal.core.Framework.publishBundleEventPrivileged(Framework.java:1569)
	at org.eclipse.osgi.framework.internal.core.Framework.publishBundleEvent(Framework.java:1505)
	at org.eclipse.osgi.framework.internal.core.Framework.publishBundleEvent(Framework.java:1500)
	at org.eclipse.osgi.framework.internal.core.BundleHost.stopWorker(BundleHost.java:506)
	at org.eclipse.osgi.framework.internal.core.AbstractBundle.suspend(AbstractBundle.java:566)
	at org.eclipse.osgi.framework.internal.core.Framework.suspendBundle(Framework.java:1207)
	at org.eclipse.osgi.framework.internal.core.StartLevelManager.decFWSL(StartLevelManager.java:592)
	at org.eclipse.osgi.framework.internal.core.StartLevelManager.doSetStartLevel(StartLevelManager.java:257)
	at org.eclipse.osgi.framework.internal.core.StartLevelManager.shutdown(StartLevelManager.java:215)
	at org.eclipse.osgi.framework.internal.core.InternalSystemBundle.suspend(InternalSystemBundle.java:284)
	at org.eclipse.osgi.framework.internal.core.Framework.shutdown(Framework.java:693)
	at org.eclipse.osgi.framework.internal.core.Framework.close(Framework.java:600)
	at org.eclipse.osgi.framework.internal.core.InternalSystemBundle$1.run(InternalSystemBundle.java:261)
	at java.lang.Thread.run(Thread.java:680)
Comment 1 Glyn Normington CLA 2011-10-26 06:40:33 EDT
Created attachment 205973 [details]
serviceability/logs/log.log extract
Comment 2 Glyn Normington CLA 2011-10-26 06:43:10 EDT
I checked the behaviour of VTS 3.0.1.RELEASE and this bug is not present there.
Comment 3 Hristo Iliev CLA 2011-10-26 07:18:24 EDT
This is most probably caused by the recent changes introduced with Bug 358129
Comment 4 Hristo Iliev CLA 2011-10-29 09:07:59 EDT
virgo-tomcat-server-3.1.0.CI-2011-10-28_22-55-28 seems to work fine on Windows
Comment 5 Hristo Iliev CLA 2011-10-29 09:26:33 EDT
VTS build at HEAD also works fine on Windows with SUN JDK 1.6
Comment 6 Hristo Iliev CLA 2011-10-31 03:53:02 EDT
virgo-tomcat-server-3.1.0.CI-2011-10-30_22-55-29.zip on Linux does not produce the error
Comment 7 Chris Frost CLA 2011-10-31 06:50:54 EDT
A local build from the 25th of October works fine on my Mac, tried a few clean and normal starts from a fresh unzip.
Comment 8 Glyn Normington CLA 2011-10-31 07:27:41 EDT
Yes, confirming that a fresh build runs fine. Apologies, but I now realise that I had zapped the version of Equinox I was running with to a 3.8 build in order to verify an Equinox fix.

Overlaying a fresh build with this version of Equino enables the problem to reproduce.

I overlaid it as follows:

find virgo-tomcat-server-3.0.1.RELEASE/ -name "org.eclipse.osgi-*jar" -exec cp org.eclipse.osgi_3.8.0.N20111022-2000.jar {} \;

i.e. I kept the old filename so Virgo startup wouldn't complain.

I will attach that version of Equinox to this bug for completeness.
Comment 9 Glyn Normington CLA 2011-10-31 07:28:23 EDT
Created attachment 206203 [details]
Equinox 3.8 JAR for reproducing the problem
Comment 10 Glyn Normington CLA 2011-11-22 10:20:57 EST
Interestingly, this problem does not seem to reproduce with the Virgo kernel.
Comment 11 Hristo Iliev CLA 2011-11-22 10:22:53 EST
Usually after CTRL-C the VT stops tomcat. I guess this functionality actually causes the problem.
Comment 12 Glyn Normington CLA 2011-11-22 10:26:56 EST
(In reply to comment #11)
> Usually after CTRL-C the VT stops tomcat. I guess this functionality actually
> causes the problem.

That may be the case, but it's far from obvious. The stack traces (in the description of this bug and in the attachment) seem far more general. I should know when I can pin down which bundle context is invalid
Comment 13 Glyn Normington CLA 2011-11-22 10:30:36 EST
On a recent VTS build with Equinox replaced with the attached version, the problem recreates. As Hristo observes, the dump occurs after Tomcat has been shut down:

^C[2011-11-22 15:29:18.609] System Bundle Shutdown       <KE0010I> Shutdown initiated. 
[2011-11-22 15:29:18.862] System Bundle Shutdown       <TC0002I> Stopping Tomcat. 
[2011-11-22 15:29:21.909] System Bundle Shutdown       <TC0003I> Stopped Tomcat. 
[2011-11-22 15:29:23.273] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-918' generated 
[2011-11-22 15:29:24.600] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-283' generated 
[2011-11-22 15:29:26.455] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-607' generated 
[2011-11-22 15:29:28.373] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-460' generated 
[2011-11-22 15:29:30.071] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-374' generated 
[2011-11-22 15:29:31.517] System Bundle Shutdown       <ME0003I> Dump 'serviceability/dump/2011-11-22-15-29-073' generated
Comment 14 Glyn Normington CLA 2011-11-22 10:34:04 EST
The invalid bundle context belongs to the org.eclipse.virgo.kernel.deployer bundle.
Comment 15 Glyn Normington CLA 2011-11-22 11:00:52 EST
Note that the same issue arises if VTS is shut down using vsh:shutdown.
Comment 16 Glyn Normington CLA 2011-11-22 12:01:53 EST
It seems that stopping the framework is perhaps behaving differently on Equinox 3.8 to how it did on 3.7.0.v20110613. Copying Tom in case he is aware of any relevant Equinox changes.

Tom: I wonder if anything has changed which means bundles are stopped in a radically different order. In this scenario we have a kernel bundle stopping before on in the user region, which didn't used to happen.
Comment 17 Glyn Normington CLA 2011-11-23 07:00:18 EST
Kernel commit 4fda1b0241ce4b960014a21bb3c6a14136ff9842 added code to tolerate the deployer bundle being stopped prior to application bundles being stopped. I view this as a tactical fix and am still somewhat concerned that, if there was a basic change in Equinox, other issues may arise.

However, this fix enables Ctrl-C to shut VTS down without dumping, so closing.