Community
Participate
Working Groups
Import the WAS symptom database and analyze a log file. 157 events in the log all four new symptom catalogs. catalog1 : 1279 rules, catalog2: 1482 rules, catalog3: 200 rules, catalog4 : 7812 rules ~3 min 51 sec with xpath on my X40 thinkpad This performance is not acceptable
Marius, please take a look
Moved to 4.2 i4, needs to be fixed in 4.2. I'll keep both this and bug 138213 for now.
Marked as P1 for further investigation and approval.
*** Bug 138213 has been marked as a duplicate of this bug. ***
Moved to 4.2.1 as per discussion with Sri and Valentina.
See also defect TPTP Bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=143454
Eugene attached find the patch for the fast XPath engine that adds support for caching compiled rules and loading them on demand from an indexed byte code cache.
Created attachment 46917 [details] FastXPath engine patch
Alex. Model side patch is committed, except for method selectNodesContext() and evalContext() which is not applicable to this bug.
Fix checked in for the XPathCorrelationEngine.
Model and Correlation Engine code are updated and migrated to compile mode usage with file caching. This is the latest performance measurement: 1999 CBE records, 11603 symptom definitions. fresh workspace analysis all: deltaTime=556172ms analysis all: deltaTime=284719ms analysis all: deltaTime=281235ms analysis 1 event: deltaTime=750ms analysis 1 event: deltaTime=797ms analysis 1 event: deltaTime=828ms analysis all: deltaTime=288391ms fresh workspace, analysis 1 event: deltaTime=14766ms analysis 1 event: deltaTime=813ms analysis 1 event: deltaTime=828ms Bug 138213 Provide a builder for XPath is not yet addressed in this bug. I will close this bug and reopen 138213 post 4.2.1 target.
Thanks for fixing the problem. This fix is urgently required for an IBM component. We will pursue getting this fix into a TPTP 4.2.0.x patch via PMC approval and the normal channels.
Here is a scenario where TPTP 4.2.1 analysis is slower that TPTP 4.2.0.2: 1) Start eclipse with -Xmx500m 2) Import WAS SDB 3) Import Apache access log (attached) 4) Analyze all TPTP 4.2.0.2: 100% complete in 96 seconds (on my machine) TPTP 4.2.1: 35% complete in 96 seconds This is a typical scenario user scenario since it uses an actual SDB from the FTP site.
Created attachment 48524 [details] Apache access log
This is the performance data I collected between TPTP 4202+patch and TPTP421(0804) driver: TPTP 4.2.0.2+patch, Fresh workspace, Analysis all 4 times and anslysis one event 3 times. !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=19177 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9634 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=8503 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=8933 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=120 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=140 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=161 TPTP 4.2.1, Fresh workspace. Analysis all 4 times and analysis one event 3 times. !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=18667 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9453 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=10125 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9203 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=120 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=150 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=180 TPTP 4.2.0.2+patch, Fresh workspace, Analysis one events 3 times and analysis all 4 times. !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=1181 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=81 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=80 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=11276 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9093 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9123 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9033 TPTP 4.2.1, Fresh workspace, Analysis one events 3 times and analysis all 4 times. !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=1382 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=80 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=50 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=12278 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9734 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=9543 !MESSAGE ExtendedAnalyze.run - org.eclipse.tptp.symptom.analysis.XPathAnalysisOperation: deltaTime=10626
This fix now introduces the following problem as described in TPTP defect 154894
Created attachment 48529 [details] CBE file and symptom file used.
Malcom, I run my test on JVM 1.4.2 java version "1.4.2" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2) Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build cndev-20040511 (JIT enabled: jitc))
Eric, the same problem exists in both TPTP4202 and TPTP421 with IBM JVM1.5. Hence, bug 154894 is not introduced by the fix here.
The slowdown I am seeing with the WAS SDB is with IBM JVM 1.5.
*** Bug 155686 has been marked as a duplicate of this bug. ***
From bug 155686 From Abdi Salahshour It appear that there are 2 identical methods in TPTP XpathCorrelationEngine, i.e., processCBECommonBaseEvents and processCommonBaseEvents with 2 different signature type for event -first one CBECommonBaseEvent event and second methods has it as CommonBaseEvent event . In JD we use processCommonBaseEvents and we now how to create events of type CommonBaseEvent and we do not understand what CBECommonBaseEvent type is? So, why are there 2 identical methods with 2 different signature and why not the optimization code implemented in both? This looks like too late in the cycle to make consumer make this kind of changes to take advantage of a performance bug in the original code. We need this same fix in processCommonBaseEvents api too for JD to gain the same performance.
Optimization code added to CBE producer handler.
close bug
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since this enhancement/defect has been resolved and unverified for more than 1 year and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.
