Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 154894

Summary: OutOfMemoryError while xpath analysis for 11K rules using IBMJDK 1.5
Product: z_Archived Reporter: Praful Rajawat <prafulr>
Component: TPTPAssignee: Eugene Chan <ewchan>
Status: CLOSED FIXED QA Contact:
Severity: blocker    
Priority: P3 CC: apnan, labadie, zung
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: Windows XP   
Whiteboard:
Bug Depends on:    
Bug Blocks: 141647    
Attachments:
Description Flags
Patch
none
updated patch none

Description Praful Rajawat CLA 2006-08-23 13:43:45 EDT 
I have used only TPTP from http://www.eclipse.org/tptp/home/downloads/drops/TPTP-4.2.0.2.html
and imported old 0.1 format was symptom catalog and access.log.

Set SUN JDK 1.5 and run the TPTP xpath analysis on access.log
Works fine.
But if I apply TPTP xpath patch on top of above driver from http://download.torolab.ibm.com/download/TPTP/4.2.0.2/patch/bug141647.zip,
I get OutOfMemoryError.
AC consumers are on SUN JDK 1.5 and that is why this is very important to get it fix

!SESSION 2006-08-23 13:15:34.640 -----------------------------------------------Without TPTP XPATH Patch
eclipse.buildId=M20060629-1905
java.fullversion=J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20060504 (JIT enabled)
J9VM - 20060501_06428_lHdSMR
JIT  - 20060428_1800_r8
GC   - 20060501_AA
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86 -clean

!ENTRY org.eclipse.update.configurator 2006-08-23 13:15:35.651
!MESSAGE Duplicate feature found on site platform:/base/ : org.apache.commons_logging.feature

!ENTRY org.eclipse.update.configurator 2006-08-23 13:15:37.454
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.aix.ppc_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:15:37.464
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.hpux.PA_RISC_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:15:37.464
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.linux.x86_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:15:37.464
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.solaris.sparc_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.tptp.monitoring.logui 1 1 2006-08-23 13:16:57.499
!MESSAGE LogImportJob.run C:\TSL\resources\perf\accessTest_20k.log: deltaTime=6460 - usedMemoryDelta=115317384 - freeMemoryDelta=2305912 - totalMemoryDelta=117623296 - usedMemoryAfter=175722552 - freeMemoryAfter=6561736 - totalMemoryAfter=182284288

!ENTRY org.eclipse.osgi 2 1 2006-08-23 13:17:02.246
!MESSAGE NLS missing message: _88 in: org.eclipse.hyades.sdb.internal.editors.messages

!ENTRY org.eclipse.osgi 2 1 2006-08-23 13:17:02.256
!MESSAGE NLS missing message: _89 in: org.eclipse.hyades.sdb.internal.editors.messages

!ENTRY org.eclipse.hyades.sdb 1 1 2006-08-23 13:17:15.074
!MESSAGE ImportSymptomDBPage2.run : deltaTime=6279 - usedMemoryDelta=129123600 - freeMemoryDelta=54281456 - totalMemoryDelta=183405056 - usedMemoryAfter=282802912 - freeMemoryAfter=107727136 - totalMemoryAfter=390530048

!ENTRY org.eclipse.tptp.platform.log.views 1 1 2006-08-23 13:18:44.863
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=63281 - usedMemoryDelta=75672432 - freeMemoryDelta=-75672432 - totalMemoryDelta=0 - usedMemoryAfter=374419888 - freeMemoryAfter=16110160 - totalMemoryAfter=390530048
!SESSION 2006-08-23 13:25:05.300 -----------------------------------------------With TPTP XPATH Patch
eclipse.buildId=M20060629-1905
java.fullversion=J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20060504 (JIT enabled)
J9VM - 20060501_06428_lHdSMR
JIT  - 20060428_1800_r8
GC   - 20060501_AA
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86 -clean

!ENTRY org.eclipse.update.configurator 2006-08-23 13:25:08.815
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.aix.ppc_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:25:08.835
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.hpux.PA_RISC_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:25:08.835
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.linux.x86_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.update.configurator 2006-08-23 13:25:08.835
!MESSAGE Could not install bundle plugins/org.eclipse.hyades.execution.solaris.sparc_4.2.0.v200606131519/   No Bundle-NativeCode match

!ENTRY org.eclipse.osgi 2 1 2006-08-23 13:25:20.853
!MESSAGE NLS missing message: _88 in: org.eclipse.hyades.sdb.internal.editors.messages

!ENTRY org.eclipse.osgi 2 1 2006-08-23 13:25:20.853
!MESSAGE NLS missing message: _89 in: org.eclipse.hyades.sdb.internal.editors.messages

!ENTRY org.eclipse.ui 4 4 2006-08-23 13:27:40.834
!MESSAGE Unhandled event loop exception

!ENTRY org.eclipse.ui 4 0 2006-08-23 13:27:40.834
!MESSAGE java.lang.OutOfMemoryError
!STACK 0
java.lang.OutOfMemoryError
	at com.ibm.oti.vm.VM.initializeClassLoader(Native Method)
	at java.lang.ClassLoader.<init>(ClassLoader.java:114)
	at org.eclipse.tptp.platform.provisional.fastxpath.compiler.EclipseCompiler$MyClassLoader.<init>(EclipseCompiler.java:316)
	at org.eclipse.tptp.platform.provisional.fastxpath.compiler.EclipseCompiler.generateClass(EclipseCompiler.java:78)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.compiler.CompilerHelper.buildCompiledExpression(CompilerHelper.java:150)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.compiler.CompilerHelper.getCompiledExpressionEvaluator(CompilerHelper.java:116)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.FastXPathEngine.compile(FastXPathEngine.java:160)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.FastXPathEngine.compile(FastXPathEngine.java:114)
	at org.eclipse.tptp.platform.provisional.fastxpath.FastXPathAPI.getCompiledExpressionEvaluator(FastXPathAPI.java:253)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.processCBECommonBaseEvents(XPathCorrelationEngine.java:274)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.buildResults(XPathCorrelationEngine.java:216)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.correlate(XPathCorrelationEngine.java:171)
	at org.eclipse.hyades.sdb.internal.util.LogAnalyzerDelegator.correlate(LogAnalyzerDelegator.java:76)
	at org.eclipse.tptp.platform.log.views.internal.actions.ExtendedAnalyzeAction$1.run(ExtendedAnalyzeAction.java:133)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

!ENTRY org.eclipse.tptp.platform.log.views 1 1 2006-08-23 13:27:40.934
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=124149 - usedMemoryDelta=-46776376 - freeMemoryDelta=654651448 - totalMemoryDelta=607875072 - usedMemoryAfter=233868472 - freeMemoryAfter=715352392 - totalMemoryAfter=949220864

!ENTRY org.eclipse.core.jobs 4 2 2006-08-23 13:27:40.934
!MESSAGE An internal error occurred during: "Analyzing...".
!STACK 0
java.lang.OutOfMemoryError
	at com.ibm.oti.vm.VM.initializeClassLoader(Native Method)
	at java.lang.ClassLoader.<init>(ClassLoader.java:114)
	at org.eclipse.tptp.platform.provisional.fastxpath.compiler.EclipseCompiler$MyClassLoader.<init>(EclipseCompiler.java:316)
	at org.eclipse.tptp.platform.provisional.fastxpath.compiler.EclipseCompiler.generateClass(EclipseCompiler.java:78)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.compiler.CompilerHelper.buildCompiledExpression(CompilerHelper.java:150)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.compiler.CompilerHelper.getCompiledExpressionEvaluator(CompilerHelper.java:116)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.FastXPathEngine.compile(FastXPathEngine.java:160)
	at org.eclipse.tptp.platform.provisional.jre14.fastxpath.FastXPathEngine.compile(FastXPathEngine.java:114)
	at org.eclipse.tptp.platform.provisional.fastxpath.FastXPathAPI.getCompiledExpressionEvaluator(FastXPathAPI.java:253)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.processCBECommonBaseEvents(XPathCorrelationEngine.java:274)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.buildResults(XPathCorrelationEngine.java:216)
	at org.eclipse.tptp.symptom.analysis.XPathCorrelationEngine.correlate(XPathCorrelationEngine.java:171)
	at org.eclipse.hyades.sdb.internal.util.LogAnalyzerDelegator.correlate(LogAnalyzerDelegator.java:76)
	at org.eclipse.tptp.platform.log.views.internal.actions.ExtendedAnalyzeAction$1.run(ExtendedAnalyzeAction.java:133)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)
Comment 1 Praful Rajawat CLA 2006-08-23 13:46:05 EDT 
cc
Comment 2 Praful Rajawat CLA 2006-08-23 13:47:34 EDT 
Sorry, I meen IBM JDK 1.5 :-)
Comment 3 Eugene Chan CLA 2006-08-23 13:53:29 EDT 
Praful, according to the trace stack you provided. 'jre14' is detected and used.
Comment 4 Eugene Chan CLA 2006-08-23 13:55:48 EDT 
Would you also provide the access.log and symptom catalog to reproduce the problem.
Comment 5 Praful Rajawat CLA 2006-08-23 14:06:18 EDT 
This problem also in tptp.sdk-TPTP-4.2.1-200608041327.zip
Comment 6 Praful Rajawat CLA 2006-08-23 14:13:07 EDT 
From the trace you can see I have used IBM JDK 1.5
java.fullversion=J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20060504
(JIT enabled)
May be JRE detection is not correct.
Comment 7 Alex Nan CLA 2006-08-23 15:10:05 EDT 
Eugene, it's OK to use the org.eclipse.tptp.platform.provisional.jre14.fastxpath.FastXPathEngine for IBM JRE 5 because both IBM JREs 1.4.2 and 5 are using the same org.apache.XPathAPI. Sun is using in JDK 5 a different XPathAPI and not the Apache one.
Comment 8 Eugene Chan CLA 2006-08-23 15:21:54 EDT 
Thanks Alex for your information. 
Praful, does the problem happen in a particual JVM or it's also happening in JVM other than IBM JDK 1.5? IBM/SUN/1.4.2/1.5?

Thanks.
Comment 9 Praful Rajawat CLA 2006-08-23 17:07:52 EDT 
It works in IBM JDK 1.4.2.
I have not tested this in SUN JVN
Comment 10 Alex Nan CLA 2006-08-25 09:36:04 EDT 
Created attachment 48713 [details]
Patch

Please test the attached patch that seems to solve the OutOfMemoryError.
Comment 11 Eugene Chan CLA 2006-08-25 11:34:15 EDT 
Thanks Alex, the patch you provided is reviewed and smoketested. It solves the memory leak problem in the code.

I have tested the patch on IBM 1.4.2, IBM 5, SUN 1.4.2 and SUN 5 and all pass using -Xms256m and -Xmx512m

Here are the performance measurement from the different JVM.
-20000 cbe events, 11600 symotom definitions, analysis one event 3 times

java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build cndev-20040511 (JIT ena
bled: jitc))
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=206627 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=1081 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=641 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=641 

IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows 2000 x86-32 j9vmwi3223-20060
817a (JIT enabled)
J9VM - 20060816_07652_lHdSMR
JIT  - 20060816_2032_r8
GC   - 20060815_AA)
JCL  - 20060817
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=362511 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=831 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=651 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=501

java version "1.4.2_11"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-b06)
Java HotSpot(TM) Client VM (build 1.4.2_11-b06, mixed mode)

!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=164637 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=712 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=620 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=631

java version "1.5.0_08"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
Java HotSpot(TM) Client VM (build 1.5.0_08-b03, mixed mode, sharing)
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=150747 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=731 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=621 
!MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=661
Comment 12 Eugene Chan CLA 2006-08-25 11:43:24 EDT 
Created attachment 48739 [details]
updated patch
Comment 13 Eugene Chan CLA 2006-08-25 12:30:10 EDT 
Update component.
Comment 14 Eugene Chan CLA 2006-08-30 10:28:03 EDT 
fixed in 4.2.1 and HEAD
Comment 15 Eugene Chan CLA 2006-08-31 13:36:44 EDT 
The OOM exception is thrown due to the fact that maximum number of classloader is reached. For historical reasons the J9 JVM imposes an artificial limit on the maximum number of classloaders that an application can create. To workaround the problem without the fix posted here(for earlier drivers), user can adjust the value with JVM argument -Xmxcl. -Xmxcl16384 should also work around the problem reported here.
Comment 16 Praful Rajawat CLA 2006-09-15 16:54:18 EDT 
Closed