Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 112371

Summary: Performance and memory improvements for the three producer Common Base Event implementations.
Product: z_Archived Reporter: Eric Labadie <labadie>
Component: TPTPAssignee: Paul Slauenwhite <paulslau>
Status: CLOSED FIXED QA Contact:
Severity: enhancement    
Priority: P1 CC: knight, zung
Version: unspecifiedKeywords: plan
Target Milestone: ---   
Hardware: All   
OS: All   
URL: http://www.eclipse.org/tptp/groups/Architecture/documents/features/hf_112371.html
Whiteboard: closed460
Bug Depends on: 119461, 119464, 119467, 119474    
Bug Blocks:    
Deadline: 2006-03-29   
Attachments:
Description Flags
EventFormatterPerformancePatch.txt
none
EventHelpersPerformancePatch.txt
none
XmlUtilityPerformancePatch.txt none

Description Eric Labadie CLA 2005-10-12 13:39:57 EDT
it appears that we are spending 44% of our instructions during a CBE XML log 
import in org/apache/xerces/parsers/SAXParser.<init> and that we are init a 
SAXParser per event.   Is there a way that we can avoid this by reusing the 
parser?     If so, we could save upwards of 60% of the total time. I ran the 
analysis on GLA 4.0. Let me know if you have questions

A trace shows a hiearchical  view of the call flow during the import.  I've 
highlighted EventFormatter.parse, which is 88.5% of the instructions.   
XML11Configuration.parser which is 26% and XMLReaderFactory.createXMLReader 
which is 61% and ends up call SAXPArser.<init>
Comment 1 Marius Slavescu CLA 2005-10-12 13:58:27 EDT
Did you use Import Log wizard to import the CBE XML log?

Try to import the same log with Import Profile File (make sure you add
<CommonBaseEvents> as root element) and compare the results.
Comment 2 Paul Slauenwhite CLA 2005-10-13 22:14:57 EDT
This defect is not critical since there is no functional breakage nor 
regression of function.  As such, decreasing the severity to normal 

P2 Highly desirable and planned for this release, but not stop ship 
normal A problem making a function difficult to use but no special work around 
is required
Comment 3 Paul Slauenwhite CLA 2005-10-13 22:15:50 EDT
Created attachment 28271 [details]
EventFormatterPerformancePatch.txt
Comment 4 Paul Slauenwhite CLA 2005-10-13 22:32:54 EDT
Although in your test environment you are using a JDK (e.g. IBM JDK) that ships 
the Xerces parser, some JDKs (e.g. Sun JDK v1.4.x) ships a proprietary XML 
Parser implementation (e.g. Crimson) which often deviates from the XML Parser 
specification.   For example, although the XML Parser specification specifies 
that "once a parse is complete, an application may reuse the same XMLReader 
object, possibly with a different input source", the TPTP and EMF Teams have 
found this not to be entirely correct for all XML Parser implementations.  As 
such, we have opted to create a new instances of the org.xml.sax.XMLReader and 
org.eclipse.hyades.logging.events.cbe.util.SAXEventHandler on each invocation 
of the 
org.eclipse.hyades.logging.events.cbe.util.associationEnginesFromCanonicalXMLDoc
() and org.eclipse.hyades.logging.events.cbe.util.eventsFromCanonicalXMLDoc() 
methods.  Also, the proposed fix (see attach) contains the overhead of 
synchronizing each call to the above static methods as well as the memory 
consumption of a static lock object.  Ultimately, more investigation is needed 
to determine the residual side affects of the proposed fix (see attach) and 
possible the design and implementation of an alternative fix to achive similar 
performance benefits without the high risk of a functional breakage in certain 
run-times.  As such, re-targeting to TPTP v4.2.
Comment 5 Paul Slauenwhite CLA 2005-10-18 15:49:55 EDT
Sizing (PW):
 
Design:        2 
Code:          1 
Test:          2 
Documentation: 0.5
------------------
Total:         5.5
Comment 6 Paul Slauenwhite CLA 2005-11-18 10:45:44 EST
Performance/memory improvements for the native Common Base Event implementation.

Based on discussions with an internal IBM group on using native Common Base 
Event implementation on constrained devices (e.g. Palm, PocketPC, etc.), the 
code should be thoroughly reviewed for possible performance/memory improvements.

For example, all setters persist the old value before setting the new value for 
no reason.  For instance,

char* oldLocalInstanceId = commonBaseEvent->_localInstanceId;

This work should extend to both Java (EMF and non-EMF) Common Base Event 
implementations.  As such, changing summary accordingly.

One potential performance improvement (analysis required) is to store the 
template Common Base Event as an XML fragment/document in the 
org.eclipse.hyades.logging.events.cbe.impl.TemplateContentHandlerImpl.  When 
the completeEvent(CommonBaseEvent) method is invoked, the Common Base Event XML 
fragment/document would be de-serialized into the parameter Common Base Event.  
However, the org.eclipse.hyades.logging.events.cbe.util.SAXEventHandler would 
need to be changed to allow callers to provide a parameter Common Base Event 
and the SAXEventHandler would need to respect the existing properties in the 
parameter Common Base Event.

Comment 7 Paul Slauenwhite CLA 2005-11-18 10:46:08 EST
*** Bug 100601 has been marked as a duplicate of this bug. ***
Comment 8 Paul Slauenwhite CLA 2005-11-18 10:47:07 EST
Created attachment 30229 [details]
EventHelpersPerformancePatch.txt
Comment 9 Eric Labadie CLA 2005-11-18 10:48:24 EST
Please raise this as a P1
Comment 10 Paul Slauenwhite CLA 2005-11-18 10:49:53 EST
These performance patches are NOT product quality.  As such, they require 
reworking, code review and exhausting testing.
Comment 11 Paul Slauenwhite CLA 2005-11-18 10:50:59 EST
(In reply to comment #9)
> Please raise this as a P1

This is not a P1 since TPTP can ship with this defect (see 
http://www.eclipse.org/tptp/home/documents/process/development/bugzilla.html).
Comment 12 Paul Slauenwhite CLA 2005-11-18 11:51:10 EST
Created attachment 30238 [details]
XmlUtilityPerformancePatch.txt
Comment 13 Christine Knight CLA 2005-11-28 14:17:57 EST
Please change this to "enh" and raise it to P1. We are asking for it to be a feature in 4.2
Comment 14 Paul Slauenwhite CLA 2005-12-05 12:29:19 EST
Regarding back porting the fixes proposed and potential for this defect to a previously released version of TPTP (e.g. TPTP v3.3) with the same function, there is no technical limitation to complete this work.  Since all previously released versions of TPTP are considered stable, all performance improvements require extensive testing to decrease the risk of a functional breakage in certain run-time(s).  Once performance improvements have been designed, implemented and testing on the target release/run time(s), the only cost to port to any previous release of TPTP with the same function will be testing for all supported run time(s).

The sizing (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=112371#c5) remains accurate for this enhancement since an exhaustive performance evaluation of this code as well as more investigation is required to determine the residual side affects of the proposed fixes possibly requiring the design and implementation of alternative fixes to be completed.
Comment 15 Paul Slauenwhite CLA 2005-12-06 14:26:20 EST
Since these performance and memory improvements are not trivial and require design and review as well as weeks of implementation and testing, this defect is being be converted to an enhancement with child defects for individual work items for consideration in the TPTP v4.2 plan.

P1 Cannot ship without this enhancement 
Comment 16 Paul Slauenwhite CLA 2005-12-06 14:38:15 EST
Description document:

http://www.eclipse.org/tptp/groups/Architecture/documents/features/hf_112371.html
Comment 17 Paul Slauenwhite CLA 2006-03-30 10:55:05 EST
Completed the following sub-defects/work items:

119461 
119464 
119467  

Since https://bugs.eclipse.org/bugs/show_bug.cgi?id=119474 is a general  performance and memory defect for all three producer Common Base Event implementations, we will keep this defect open for the remainder of TPTP V4.2.0 to address any further performance and memory issues.

Marking this enhancement as completed/fixed.
Comment 18 Paul Slauenwhite CLA 2006-05-05 12:42:09 EDT
16.76% improvement for event creation, serialization and de-serialization for small events.   
Comment 19 Paul Slauenwhite CLA 2009-06-30 12:16:42 EDT
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since this enhancement/defect has been resolved and unverified for more than 1 year and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.