Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 209342 - Binary Data Transfer Format for Profiling (Profiler-side)
Summary: Binary Data Transfer Format for Profiling (Profiler-side)
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: TPTP (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P1 normal (vote)
Target Milestone: ---   Edit
Assignee: Stanislav Polevic CLA
QA Contact:
URL:
Whiteboard: closed460
Keywords:
Depends on:
Blocks: 196713 209343
  Show dependency tree
 
Reported: 2007-11-09 09:55 EST by Stanislav Polevic CLA
Modified: 2016-05-05 10:43 EDT (History)
13 users (show)

See Also:


Attachments
Initial implementation of the Binary Format (123.97 KB, patch)
2008-01-29 09:54 EST, Stanislav Polevic CLA
no flags Details | Diff
BF emitter with new format header support (120.61 KB, patch)
2008-02-19 10:29 EST, Stanislav Polevic CLA
no flags Details | Diff
Minor optimizations and two new system messages (128.85 KB, patch)
2008-02-26 12:00 EST, Stanislav Polevic CLA
no flags Details | Diff
Handshaking protocol introduced (150.66 KB, patch)
2008-03-21 09:40 EDT, Stanislav Polevic CLA
no flags Details | Diff
Profiler-side support for BF (151.58 KB, patch)
2008-03-24 11:19 EDT, Stanislav Polevic CLA
no flags Details | Diff
Fix broken build for Windows (17.99 KB, patch)
2008-03-25 13:07 EDT, Stanislav Polevic CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stanislav Polevic CLA 2007-11-09 09:55:31 EST
Implement new binary format to replace existing XML format.
- Standalone mode:
Allow the possibility to switch between XML and binary format by entering new command line parameter: format=binary|xml (default binary)
- enabled/controlled mode (via AC)
Introduce negotiation protocol allowing client to determine the possibility to use binary format (compatibility with older clients/profilers)
Comment 1 Stanislav Polevic CLA 2008-01-29 09:54:33 EST
Created attachment 88134 [details]
Initial implementation of the Binary Format

This patch incorporates all changes required for producing binary trace file in standalone mode.
To do that use new option 'format=binary'.
The implementation is not complete though it covers basic design and structure of the new format.
Comment 2 Marius Slavescu CLA 2008-01-29 10:18:50 EST
Wouldn't be better to have the XML format default (to ensure that old clients will work) and if you connect with a new client it will set the binary format (this will be default on the client side) or whatever format the user had chosen.
Comment 3 Stanislav Polevic CLA 2008-01-29 11:07:09 EST
Thanks Marius!
Making XML format default for standalone mode does make sense. I shall think on it and provide explanations within the next patch.

Format negotiation protocol is not implemented yet, but it would work similar to your suggestion.
Comment 4 Harm Sluiman CLA 2008-01-30 13:57:07 EST
(In reply to comment #3)
> Thanks Marius!
> Making XML format default for standalone mode does make sense. I shall think on
> it and provide explanations within the next patch.
> Format negotiation protocol is not implemented yet, but it would work similar
> to your suggestion.

I think the suggestion is to make the default XML in all cases, and the client (workbench) or command line user can specifically ask for binary. 
I am assuming that we will have a new file extension like trcbin or something, and we will add support for streaming to file on the workbench side as well as a new import hook.
Comment 5 Mikhail Voronin CLA 2008-01-31 07:23:23 EST
IMO, It's better to configure for XML format when working with old clients.

Default mode should better provide best available format by default.

Users who we want to attract by new better profiler in TPTP 4.5 should get good first experience with default options.

Comment 6 Harm Sluiman CLA 2008-01-31 08:42:37 EST
(In reply to comment #5)
> IMO, It's better to configure for XML format when working with old clients.
> Default mode should better provide best available format by default.
> Users who we want to attract by new better profiler in TPTP 4.5 should get good
> first experience with default options.

Agreed, so when a 4.5 workbench launches or attaches to the TI agent it can request binary data and the user gets the benifits. 

What we have to deal with is if a pre 4.5 workbench connects with a 4.5 TI agent then the agent needs to produce XML without being asked to.

The same holds true for standalone mode. Mixed environments are normal for TPTP.
Comment 7 Chris Elford CLA 2008-01-31 16:56:13 EST
This is a more general question than one related to binary xfer...

I understand that we support 4.5 workbench connecting to pre-4.5 agents.  

I was not aware that we were supposed to support pre 4.5 workbenches connecting to 4.5 agents.  That seems a strange combination to support.

Thx,
Chris
Comment 8 Harm Sluiman CLA 2008-02-10 22:27:16 EST
(In reply to comment #7)
> This is a more general question than one related to binary xfer...
> 
> I understand that we support 4.5 workbench connecting to pre-4.5 agents.  
> 
> I was not aware that we were supposed to support pre 4.5 workbenches connecting
> to 4.5 agents.  That seems a strange combination to support.
> 
> Thx,
> Chris
> 

The old workbench naturally doesn't have to support the new agent but the newer agent has to maintain backward compatability. We see these configurations in the real world all the time. Particularly when customers are doing staged release updates. You can often have 100 users spread across 3 release levels of the workbench sharing 10 dept. servers that are in transition over a few weeks.

Either way, it doesn't seem to difficult to maintain this backward compatible behavior.
Comment 9 Marius Slavescu CLA 2008-02-14 11:19:48 EST
(In reply to comment #7)
> This is a more general question than one related to binary xfer...
> 
> I understand that we support 4.5 workbench connecting to pre-4.5 agents.  
> 
> I was not aware that we were supposed to support pre 4.5 workbenches connecting
> to 4.5 agents.  That seems a strange combination to support.
> 
> Thx,
> Chris
> 

The backward compatibility, newer workbench should work on previous workspaces and with previous agents, should be ensured at all levels (model, transport protocol, persistence format, preferences etc).

The way I understand binary profiling support it shouldn't be any problem in leaving the old behavior in place.

The new profiling format/behavior should be build as a vertical component beside the old one, and just set the old one by default when working with old AC (pre TPTP 4.5) and new one default for AC 4.5 (including IAC 4.5). 
AC 4.5 profiling agent should be able to handle requests from a pre TPTP 4.5 workbench.

To ensure these rules there should be a negotiation protocol between workbench and agent(s), which will identify the best workbench/agent behavior path for each profiling session.





Comment 10 Stanislav Polevic CLA 2008-02-19 10:29:27 EST
Created attachment 90073 [details]
BF emitter with new format header support

Changes:
- introduce new format header (stabilized)
- remove system message #1 (ex-format header)
Comment 11 jkubasta CLA 2008-02-20 08:30:08 EST
Stanislav, you have this enhancement listed in the weekly schedules and yet this feature is not in plan for 4.5.  Are you planning to resolve this in 4.5? If so, please create a FDD and request arch board review
Comment 12 Stanislav Polevic CLA 2008-02-20 08:54:52 EST
Actually, this is a part of bug 196713.
W/o this emitting part BF format loader is meaningless.
So, yes, I am going to resolve this in 4.5i6
Comment 13 Stanislav Polevic CLA 2008-02-21 10:30:36 EST
Joanna, parent bug 196713 has description document - is it enough or I should add more details?
Alexander has scheduled AG review on next week.
Comment 14 Paul Slauenwhite CLA 2008-02-21 16:04:11 EST
(In reply to comment #13)
> Alexander has scheduled AG review on next week.

Alexander has not yet requested an AG review.
Comment 15 Alexander N. Alexeev CLA 2008-02-21 16:10:52 EST
> Alexander has not yet requested an AG review.
> 
Well, I do it right now. :)
Paul, could you arrange AG review for this enhancement on Thursday or Friday next week. 
Thanks.
Comment 16 Paul Slauenwhite CLA 2008-02-22 07:34:16 EST
(In reply to comment #15)
> > Alexander has not yet requested an AG review.
> > 
> Well, I do it right now. :)
> Paul, could you arrange AG review for this enhancement on Thursday or Friday
> next week. 
> Thanks.
> 

Done.  Fri Feb 29 @ 10 AM ET.
Comment 17 Paul Slauenwhite CLA 2008-02-22 11:19:27 EST
This should be a child defect of the root-level enhancement.
Comment 18 Stanislav Polevic CLA 2008-02-26 11:58:17 EST
(In reply to comment #17)
> This should be a child defect of the root-level enhancement.
> 

Yes, it depends on bug 196713 which is root-level enhancement.
Comment 19 Stanislav Polevic CLA 2008-02-26 12:00:43 EST
Created attachment 90763 [details]
Minor optimizations and two new system messages

Changes:
- new system message defining character encoding 
- new system message defining max qualified CPU frequency
- CPrintBinary code dealing with output buffer is optimized
Comment 20 Paul Slauenwhite CLA 2008-02-27 10:11:13 EST
(In reply to comment #18)
> (In reply to comment #17)
> > This should be a child defect of the root-level enhancement.
> > 
> 
> Yes, it depends on bug 196713 which is root-level enhancement.
> 

Usually child defects block parent enhancements (e.g. work item required for the enhancement).
Comment 21 Stanislav Polevic CLA 2008-02-27 11:21:10 EST
Thanks Paul. I changed dependencies.
Comment 22 Stanislav Polevic CLA 2008-03-21 09:40:23 EDT
Created attachment 93127 [details]
Handshaking protocol introduced

Changes:
- handshaking protocol (binary/xml format enabling) is introduced
Comment 23 Stanislav Polevic CLA 2008-03-21 09:41:16 EDT
Igor, could you, please, verify this patch and check it into repository.
Comment 24 Stanislav Polevic CLA 2008-03-24 11:19:28 EDT
Created attachment 93278 [details]
Profiler-side support for BF

Final patch for commitment.
Comment 25 jkubasta CLA 2008-03-24 11:32:41 EDT
Resolving defect as fixed. Please reopen if problem found during test pass.
Comment 26 Alexander N. Alexeev CLA 2008-03-24 11:39:07 EDT
patch is committed to the HEAD
Comment 27 Stanislav Polevic CLA 2008-03-25 13:07:41 EDT
Created attachment 93439 [details]
Fix broken build for Windows

Fix broken build for Windows EM64T and IPF
Comment 28 Alexander N. Alexeev CLA 2008-03-25 17:00:40 EDT
Comment on attachment 93439 [details]
Fix broken build for Windows

patch was moved to appropriate bug 223830
Comment 29 Paul Slauenwhite CLA 2009-06-30 10:37:43 EDT
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since the originator of this enhancement/defect has an inactive Bugzilla account and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.