Bug 335027

Comment 1 Nobody - feel free to take it 2011-01-25 12:22:55 EST

Link to GEF http://www.eclipse.org/gef/

Comment 2 Marc Khouzam 2011-01-27 15:33:32 EST

Maybe this could also be interesting
http://www.eclipse.org/proposals/tools.windowbuilder/
I believe it is a tool from Instantiations (now Google) which is free and is in the process of becoming open-source.

Comment 3 Marc Khouzam 2011-01-31 11:59:37 EST

Thanks Bill for the details proposal!  I think we can really bring a great feature to the CDT with this.  Here are my first impressions/comments.  At the end, I've also added some suggested requirements.

- About the different Presentations and their implementation.   What kind of 'configuration' of the presentation do we want to support?
    - Should a target be able to provide some information about how the Presentation should look (how many cores, type of I/O and memory elements, etc).  I assume we will need this for the Visualizer view to update when switching between launches.
    - Should a Presentation be configurable by a 'sysadmin'?  Meaning that someone in a group would take care of setting up the Presentation for her/his group to re-use.
    - Should the end-user be able to customize a Presentation?  Show a subset of cores, choose what will be shown in the cores (threads, a particular register value, etc)
    - Would we use GEF, plugin extension-points, XML?

- Hardware type:
  - We are interested in the following use-case: large number of cores (maybe 64), but the layout is not very important.  What is important is that the layout be efficient and that data be shown and controlled well.

- "Note that the launch does not have to be invoked in Debug mode -- the Visualizer should work just as well for non-debugged applications."  This will still require some backend entity that will provide information about the target.  I assume that for Tilera it is your 'Shepherd' process.  For open-source, GDB can do such things (list of running processes, info about OS resources), but we still need to start GDB, which is done in Debug mode only.  We should eventually have a list of requirements that the Visualizer will need from a backend (although these requirements may be per Presentation type)

- the section "Switching between launches", I understand that the Visualizer is limited to showing the information of a single launch.  This made me think that we could apply the Pin&Clone feature to the Visualizer.  This would allow the user to have multiple Visualizer views at the same time and have each one fixed on (pinned) a particular launch or Presentation.

- "the Visualizer View should update to reflect the selected launch's state".  I like that this would be done automatically.  However, we will need to have the information provided by something, e.g, the backend, stored user configuration, or both.

- "When a launch is removed... (Which one to display is somewhat arbitrary...".  Since we are synchronizing the DebugView (DV) and the Visualizer, it probably makes sense for the Visualizer to start showing the launch that is select in the DV.  If none are selected (which seems to be the case after the user removes a launch), the the Visualizer should still match and show nothing.

- "Selection in the Visualizer View". "Where more than one such item is selected, the item selected in the Debug View is arbitrary".  If the Visualizer allows to select multiple elements then we could also have the DV select those elements.  The other views would show things as for any multi-selection as handled today (which seems to be that the top-most selection is the chosen one)

- "Actions on selections".  Pawel and Patrick had revived the proposal of having a top-level debug toolbar for these actions.  We should keep this in mind so we can poke them about it :-) http://bugs.eclipse.org/258767

- In the "Requirements" section, the first parapgraph ends with "between the available views."  I'm assuming you meant "available Presentations".

- In the "Requirements" section you mention "we are proposing a single view, multiple-presentation model here".  The Pin&Clone idea could enhance this pretty well I think by allowing to have different visualizer views, which a different presentation each, with the same or a different target as well.

- "periodic updating of the view ("polling" model)".  I think this should also allow for user-triggered manual update.

- "An action framework for performing operations on the model and its elements".  I'm hoping we can use Eclipse's Command framework and also re-use the already defined actions/commands such as 'resume', 'stepOver', etc

- In the Implementation section, "Question: do we actually need this, or can we assume the user will create their own view (i.e. the "Grid View")".  I think the framework could provide some abstract classes of the functionality we feel is standard to any Presentation, while the extended classes would be provided by the Presentations plugin(s).

- I think it makes sense to start with a "Grid view" as an example since you have experience with that.  Having a concreate example will be a good way to make progress.  Maybe this "Grid view" can be a little more generic that Tilera's so it does not have to be a square, for example.

Suggested other requirements:
============================
- The Visualizer view should support Grouping and Hiding, like the Debug view soon will.  Such a feature would probably redefine the current layout based on hidden items and on grouped items.

- Depending on what kind of configuration the Visualizer will support, we may need support persitence of user-selected changes to the view.  We may also want an export/import feature which would allow a user to easily save/restore/share a Presentation that she customized (like can be done for Eclipse perspectives).

- The Visualizer view should support Synchronized operations like the Debug View will.  The concept of a "leader" for a synchronized operation should also be supported.  Defining such a leader and showing it using the Visualizer view will require some support.

- Support for a concept of zoom-level.  As the zoom-level increases, the number of e.g., cores would be less, while the content show for each e.g., core, would increase, as more real-estate was now available. 

- Support for 'locked'/'protected' elements.  The user could select elements in the Visualizer that would be marked as protected.  The protected elements would never be included in operations (e.g., suspend, resume) even if they are selected.  This has at least two advantages:
    1- We could do a graphical selection and not need to remove elements that are 'protected'
    2- Special elements (e.g., a core running a watchdog or a heartbeat) could be protected from being accidently stopped or started.


Questions/Suggestions for the proposed implementation
- We could have a concept of "building blocks" which would be like widgets that the user could use and configure (size, content, etc).  Does this correspond to "VisualizerViewerElement"?

- A set of predefined building blocks (widgets) could be provided.  New ones could be defined by the user.

- We would need "fundamental building blocks" which would correspond to concepts understood by the backend/target such as threads, cores, chips, etc.

Comment 4 William Swanson 2011-01-31 15:49:40 EST

Thanks for the detailed feedback, Marc -- responses inlined below:

   - About the different Presentations and their implementation.   What kind of
   'configuration' of the presentation do we want to support?
       - Should a target be able to provide some information about how the
   Presentation should look (how many cores, type of I/O and memory elements,
   etc).  I assume we will need this for the Visualizer view to update when
   switching between launches.

The target is the model, not the view, hence the target should not prescribe
the presentation, since it should know nothing about the presentation.
(Or in fact which of many presentation(s) is in use.)

That said, if there is important layout information about the target that would
be useful to a presentation, then the application "model" should provide this
layout info somehow, in a presentation-neutral format, so that it can be used
by a presentation that needs it (as Tilera's Grid View does).

In our case, one of the messages reported from the backend (the monitor/shepherd)
is an XML description of the target's hardware layout. This is used to update
the Grid View's presentation whenever we switch views. This is just one example,
however.

In the Visualizer View world, we'd want such layout-related information
to come from the model. I can add to the Requirements/Implementation the need
for some kind of target description information -- format and structure TBD.
It might be coarse (here's some XML, you figure it out) or fine-grained
(here's a TargetLayoutModel with appropriate substructure, etc.).
This would depend a lot on what's common across platforms. (This is similar
to the generalization that every debug target necessarily has processes,
threads, and stack frames.)


       - Should a Presentation be configurable by a 'sysadmin'?  Meaning that
   someone in a group would take care of setting up the Presentation for her/his
   group to re-use.
       - Should the end-user be able to customize a Presentation?  Show a subset
   of cores, choose what will be shown in the cores (threads, a particular
   register value, etc)

I've assumed that one would want the target to be self-describing, so there
would be little need for configuration from the user/admin. (Anything we can
avoid here is good, since having to configure something is a great reason
never to use it.) However, if there _are_ general cases where a user might
need or want to configure some aspect of the presentation, we should capture
that as a requirement. Otherwise, I'd say one is free to add a presentation
that requires some additional configuration info, but it's up to the presentation
to get that information and make use of it. We don't have to make it part
of the base framework.


       - Would we use GEF, plugin extension-points, XML?

This is asking about two different things:

- For the model-description data, that comes from the model, as noted above.
 
- For presentation implementation, yes, if there was some standard
  package we can settle on to provide base classes for the Viewer
  and ViewerElement types, that would be good. However, if possible we
  should leave the framework open enough to allow someone to come along
  and create a bare-bones "canvas with some draw commands" presentation,
  if all they want is something really plain and simple.

I'll add a discussion of this in the Implementation section.


   - Hardware type:
     - We are interested in the following use-case: large number of cores (maybe
   64), but the layout is not very important.  What is important is that the
   layout be efficient and that data be shown and controlled well.

Good point -- the Grid View does have a specific hardware layout in mind,
but one might also want to visualize a generic parallel platform.
I've added this as an example hardware type.


   - "Note that the launch does not have to be invoked in Debug mode -- the
   Visualizer should work just as well for non-debugged applications." This will
   still require some backend entity that will provide information about the
   target.  I assume that for Tilera it is your 'Shepherd' process.  For
   open-source, GDB can do such things (list of running processes, info about OS
   resources), but we still need to start GDB, which is done in Debug mode only. 
   We should eventually have a list of requirements that the Visualizer will
   need from a backend (although these requirements may be per Presentation 
   type)

I'll add the need for a "backend" framework to the implementation section.


   - the section "Switching between launches", I understand that the Visualizer is
   limited to showing the information of a single launch.  This made me think that
   we could apply the Pin&Clone feature to the Visualizer.  This would allow the
   user to have multiple Visualizer views at the same time and have each one fixed
   on (pinned) a particular launch or Presentation.

I'll add this as a Requirement on the visualizer view.


   - "the Visualizer View should update to reflect the selected launch's state". 
   I like that this would be done automatically.  However, we will need to have
   the information provided by something, e.g, the backend, stored user
   configuration, or both.

Yep, hence the need for some kind of "backend framework". In our case this is
the monitor/shepherd, though the information provided by them is mediated 
through an internal state model, so the Grid View does not talk directly to
the backend.


   - "When a launch is removed... (Which one to display is somewhat arbitrary...".
    Since we are synchronizing the DebugView (DV) and the Visualizer, it probably
   makes sense for the Visualizer to start showing the launch that is select in
   the DV.  If none are selected (which seems to be the case after the user
   removes a launch), the the Visualizer should still match and show nothing.

We may want to go a little further -- in the Grid View, I actually created
a selection-tracking class that kept track of the most recently touched launches,
so when the current one went away, I could always switch the view back to the
next-most-recently viewed one. I found this was helpful when a user was monitoring
two or three launches, since it let the IDE be "smart" about what they were most
likely to want to see next.

In any case, I've added the suggestion about using the Debug View selection
to the doc.


   - "Selection in the Visualizer View". "Where more than one such item is
   selected, the item selected in the Debug View is arbitrary".  If the Visualizer
   allows to select multiple elements then we could also have the DV select those
   elements.  The other views would show things as for any multi-selection as
   handled today (which seems to be that the top-most selection is the chosen one)

I'll treak the requirement to say we default to however the DV, etc. handle
multiple selections.


   - "Actions on selections".  Pawel and Patrick had revived the proposal of
   having a top-level debug toolbar for these actions.  We should keep this in
   mind so we can poke them about it :-) http://bugs.eclipse.org/258767

To some extent, where the actions live is outside the scope of this design,
except in specifying whether the Visualization View has its own toolbar.
In any case, I don't think we want to completely remove the toolbar
on the Debug and/or Visualization views, though one might pare it down
to just the most "useful" actions for the current context.


   - In the "Requirements" section, the first parapgraph ends with "between the
   available views."  I'm assuming you meant "available Presentations".

Correct, and fixed.


   - In the "Requirements" section you mention "we are proposing a single view,
   multiple-presentation model here".  The Pin&Clone idea could enhance this
   pretty well I think by allowing to have different visualizer views, which a
   different presentation each, with the same or a different target as well.

I'll edit the requirement to suggest using multiple views with pin-and-clone.


   - "periodic updating of the view ("polling" model)".  I think this should also
   allow for user-triggered manual update.

Absolutely, though we should above all else avoid _requiring_ manual
update from the user, since this (a) feels cheap and (b) loses any
ability for the view to report "late breaking" events.


   - "An action framework for performing operations on the model and its
   elements".  I'm hoping we can use Eclipse's Command framework and also re-use
   the already defined actions/commands such as 'resume', 'stepOver', etc

This is the general idea. I'm not proposing creating a new mechanism here.


   - In the Implementation section, "Question: do we actually need this, or can we
   assume the user will create their own view (i.e. the "Grid View")".  I think
   the framework could provide some abstract classes of the functionality we feel
   is standard to any Presentation, while the extended classes would be provided
   by the Presentations plugin(s).

I've modified that item to replace the Question with your suggestions.


   - I think it makes sense to start with a "Grid view" as an example since you
   have experience with that.  Having a concreate example will be a good way to
   make progress.  Maybe this "Grid view" can be a little more generic that
   Tilera's so it does not have to be a square, for example.

No, Tilera's Grid View (at least in our IDE) would still be what it is,
a platform-centric 2D view of the Tilera chip's hardware. I don't say there
would not be a "Parallel Core View" example, etc. that generalizes things,
however.


   Suggested other requirements:
   ============================
   - The Visualizer view should support Grouping and Hiding, like the Debug view
   soon will.  Such a feature would probably redefine the current layout based on
   hidden items and on grouped items.

Added.


   - Depending on what kind of configuration the Visualizer will support, we may
   need support persitence of user-selected changes to the view.  We may also want
   an export/import feature which would allow a user to easily save/restore/share
   a Presentation that she customized (like can be done for Eclipse perspectives).

Unless we have a strong requirement, I'm not assuming any user editing features.
A presentation may want to persist some state, i.e. to remember a user's selections.
It's not clear whether this wants to be part of the framework or just be something
the presentation itself does.


   - The Visualizer view should support Synchronized operations like the Debug
   View will.  The concept of a "leader" for a synchronized operation should also
   be supported.  Defining such a leader and showing it using the Visualizer view
   will require some support.

Hmmm -- "leader"-ness could just be a model property like visibility
or grouping, which it's up to a presentation to decide how to present.

Actions/commands are not specified by the visualization framework;
we just assume there's a mechanism for invoking an action, and that it
can somehow get the current selection from the view in order to 
invoke the appropriate backend operation.

It's still not clear to me whether "synchronized" is even meaningful to the
view, since as I understand it this is a feature of the backend: you tell
the debugger "hey, resume X,Y,Z simultaneously", and somehow it goes and
does it, and the events come back and the view updates itself. If there needs
to be some "Okay, X,Y,Z resumed at the same time" display, that's an event
from the backend, which the presentation needs to field and display somehow.


   - Support for a concept of zoom-level.  As the zoom-level increases, the number
   of e.g., cores would be less, while the content show for each e.g., core, would
   increase, as more real-estate was now available. 

I'll add a requirement for scalability, though how it's accomplished may be
merely a presentation detail. For example, the Grid View handles scaling by
making the visualization fit the available space, so one can "zoom" by resizing
the view bigger. However, I can definitely see the value in a presentation
wanting to allow explicit zooming/scrolling of a visualization with lots
of context or detail.


   - Support for 'locked'/'protected' elements.  The user could select elements in
   the Visualizer that would be marked as protected.  The protected elements would
   never be included in operations (e.g., suspend, resume) even if they are
   selected.  This has at least two advantages:
       1- We could do a graphical selection and not need to remove elements that
   are 'protected'
       2- Special elements (e.g., a core running a watchdog or a heartbeat) could
   be protected from being accidently stopped or started.

This feels like an aspect of the selection action, rather than the framework,
since there's nothing in the framework that prevents the selection commands
from having the concept of "locked" items, and a presentation can treat
"locked" like any other displayable property. However, I'll add it as a
suggestion of "new properties" we'd want the framework to support.


   Questions/Suggestions for the proposed implementation

   - We could have a concept of "building blocks" which would be like widgets that
   the user could use and configure (size, content, etc).  Does this correspond to
   "VisualizerViewerElement"?

Maybe, though VisualizerViewerElement could equally be an interface supported
by any graphic element that wants to be presented on a viewer.
I'll add a note to that effect.


   - A set of predefined building blocks (widgets) could be provided.  New ones
   could be defined by the user.

As noted above, I'm not thinking of a view editor -- from the user's point
of view it's a fixed display. The implementor of a presentation might want
some useful base classes, etc. to make their life simpler.


   - We would need "fundamental building blocks" which would correspond to
   concepts understood by the backend/target such as threads, cores, chips, etc.

Hmmm... I agree we might have standard *model* concepts for threads, cores, chips,
and so forth. It's less clear that there would be standard graphic elements
for such things, because one might want to represent them in vastly different
ways. In a symmetric chip like Tilera's, for example, the cores can be shown
as a grid of tiles. In some other environment, they might be separate icons,
or a row of "blinky lights", or who knows what.

More likely, there would be base classes for types of presentation items,
like SelectableViewerElement, or LabelViewerElement, or ViewerElementDecoration,
or such like, and perhaps aggregators like RectangularGridViewerElementContainer, etc.

I know what I needed for the Grid View, but it would be good to have some
other use cases or presentation examples to draw some common patterns from.


Okay, I've updated the Wiki page to reflect the above changes.

Thanks again for the feedback, Marc!
--- WRS

Comment 5 Eugene Ostroukhov 2011-08-30 15:12:28 EDT

I would like to share some requirements that we have in our project. I am not sure if they are related to Visualizer View per se - but I do believe that they may become common to CDT-based IDEs in the future.

1. Rich filtering support. Debug view only shows the threads the user cares about - IDE provides ways to lookup threads by different attribute (HW coordinates, variable value, current stack frame). Basically, this is a search field and some query language.
2. Drill-down support. Imagine cluster debugger that should let the the user drill-down the hierarchy Cluster (see node occupancies and state)->Node (see CPUs occupancies)->CPU (see threads as in the visualizer matrix view).
3. Different views on the process space (i.e. threads can have logical organization (i.e. in groups) but the user may also want to view threads based on some HW coordinates).

Comment 6 Marc Khouzam 2011-09-12 09:28:21 EDT

Bill, can you post the info about the GitHub repo you created?  Thanks

Comment 7 Marc Khouzam 2011-09-13 15:51:40 EDT

(In reply to comment #5)
> I would like to share some requirements that we have in our project. I am not
> sure if they are related to Visualizer View per se - but I do believe that they
> may become common to CDT-based IDEs in the future.
> 
> 1. Rich filtering support. Debug view only shows the threads the user cares
> about - IDE provides ways to lookup threads by different attribute (HW
> coordinates, variable value, current stack frame). Basically, this is a search
> field and some query language.
> 2. Drill-down support. Imagine cluster debugger that should let the the user
> drill-down the hierarchy Cluster (see node occupancies and state)->Node (see
> CPUs occupancies)->CPU (see threads as in the visualizer matrix view).
> 3. Different views on the process space (i.e. threads can have logical
> organization (i.e. in groups) but the user may also want to view threads based
> on some HW coordinates).

Those are good suggestions.  We have a list of features of interest on the Multicore group wiki: http://wiki.eclipse.org/CDT/MultiCoreDebugWorkingGroup#Features_of_current_interest
Each one has a description and maybe use cases.  Feel free to add your suggestions to the relevant parts so that we can keep track of them.  Thanks.

Comment 8 William Swanson 2011-09-13 15:54:35 EDT

The GitHub CDT clone that's been set up for visualizer framework
development work is here:

  https://github.com/Traveller7/cdt/tree/master

The Visualizer framework in particular can be found here:

  https://github.com/Traveller7/cdt/tree/master/visualizer

Comment 9 Marc Khouzam 2011-09-13 16:08:59 EDT

Now that Bill has put the first version of the Visualizer framework plugins on
GitHub, I think the next step is to implement a example Visualizer for CDT.

So the big question is: what do we want to have in CDT as a Debug Visualizer?

I suggest we start with some basic features, and keep building on it to get a
nice and useful view.  Here is what we could start with:

1- show a big square for each core of the target
2- core number displayed in the corner or in a hover of each square
3- visual display of threads on the core they were last known to be running
4- coloring of threads based on state (running/suspended/crashed)
5- coloring of cores based on state of its threads
6- a refresh to update the display (threads have moved, etc)
7- ability to perform run-control operations from the visualizer view

This would give users a graphical representation of the Debug view, which would work on any machine (single core, double core, quad-core, etc)

The implementation will use DSF-GDB and its DSF services and events to obtain
the relevant information.

Comments are very welcome.

Using DSF-GDB is my strong point; doing the graphical parts is not.  Bill, do
you have code that we could re-use for the graphical parts?  I was thinking of
base class methods to draw cores, draw threads on cores, etc.

Comment 10 Marc Khouzam 2011-09-14 13:31:07 EDT

(In reply to comment #9)

For the first feature set, here is what I have in mind.  First, we can mimic the TraceControlView class for the infrastructure to access the DSF session and services.  As for what we need from the DSF services:

1- show a big square for each core of the target

We need the number of cores of the target.  There is no service
providing this yet.  I suggest a new service that deals with the
hardware.  I'll need to think about the details of the API, but
it would be something like (pseudo code, not using DSF's async model
for simplicity):

int IHardware.getCoresCount(ITargetContext)

2- core number displayed in the corner or in a hover of each square

ICoreContext[] IHardware.getCores(ITargetContext)
ICoreData IHardware.getExecutionData(ICoreContext)
ICoreData {
  String getName();
 ...
}

3- visual display of threads on the core they were last known to be running

The core a thread is running on.  This exists already:

IGdbThreadDMData IProcesses.getExecutionData(IThreadDMContext)
IGdbThreadDMData {
   String[] getCores();
}

Note that his works for processes as well, where getCores() returns the core where all the threads of that process run.

4- coloring of threads based on state (running/suspended/crashed)

The running state of threads.  This exists:

boolean IRunControl.isSuspended(IExecutionContext)
IExecutionDMData getExecutionData(IExecutionDMContext)
IExecutionDMData {
   StateChangeReason getStateChangeReason();
}

5- coloring of cores based on state of its threads

Super-set of point 4.

6- a refresh to update the display (threads have moved, etc)

Auto refresh can be achieved for the state of threads using
DSF events:
IRunControl.ISuspendedDMEvent and IRunControl.IResumedDMEvent.

As running threads can change cores very often, only a manual refresh will be supported (only available functionality in GDB anyway).  Pressing the Refresh button would re-trigger all the calls to the services described above.

7- ability to perform run-control operations from the visualizer view

Same as debug view.  We can probably re-use the UI actions directly.

==

I can work on setting all that up, but after that, we need to converts that information into a nice graphical display...

Comment 11 William Swanson 2011-09-14 14:00:03 EDT

(In reply to comment #10)

This all looks good, just a couple thoughts on the last two points:

> 6- a refresh to update the display (threads have moved, etc)
> 
> Auto refresh can be achieved for the state of threads using
> DSF events:
> IRunControl.ISuspendedDMEvent and IRunControl.IResumedDMEvent.
> 
> As running threads can change cores very often, only a manual refresh will be
> supported (only available functionality in GDB anyway).  Pressing the Refresh
> button would re-trigger all the calls to the services described above.

The refesh issue came up with the Tilera Grid View as well.
It's really desirable to auto-update the view with new thread locations
while the application is running. This doesn't have to be super-precise.
It's understood that things may be moving around between "snapshots".
But having the view automatically update with some periodicity
gives a much stronger sense of the program's activity.

At the very least we can have the visualizer periodically poll
the target to request the current location of its thread(s).
There may be some skew in thread positions while the application
is running, but his is okay, and when we hit a breakpoint we can
do one more poll to get the final, accurate positions.

We _could_ also have a manual update button, but if we can figure out
a reasonable polling interval it should ideally not be necessary.


> 7- ability to perform run-control operations from the visualizer view
>
> Same as debug view.  We can probably re-use the UI actions directly.

We may need Visualizer actions that map from the visualizer's current
selection to something recognizable to the Debug view, but yes once we
have the Debug view selection set appropriately, we can use the corresponding debug actions.

Comment 12 Marc Khouzam 2011-09-14 14:04:26 EDT

(In reply to comment #11)

> The refesh issue came up with the Tilera Grid View as well.
> It's really desirable to auto-update the view with new thread locations
> while the application is running. This doesn't have to be super-precise.
> It's understood that things may be moving around between "snapshots".
> But having the view automatically update with some periodicity
> gives a much stronger sense of the program's activity.
> 
> At the very least we can have the visualizer periodically poll
> the target to request the current location of its thread(s).
> There may be some skew in thread positions while the application
> is running, but his is okay, and when we hit a breakpoint we can
> do one more poll to get the final, accurate positions.
> 
> We _could_ also have a manual update button, but if we can figure out
> a reasonable polling interval it should ideally not be necessary.

Ok, since you have gone through this before, let's go with that recommendation.  Let's have a view property to allow the user to set a polling interval.

> > 7- ability to perform run-control operations from the visualizer view
> >
> > Same as debug view.  We can probably re-use the UI actions directly.
> 
> We may need Visualizer actions that map from the visualizer's current
> selection to something recognizable to the Debug view, but yes once we
> have the Debug view selection set appropriately, we can use the corresponding
> debug actions.

The synchronization between the two views is still blurry to me, implementation-wise.  But I figured we'll address that once we have a prototype that displays.

Comment 13 Eugene Ostroukhov 2011-09-14 14:26:02 EDT

> 4- coloring of threads based on state (running/suspended/crashed)

Please note that possible thread states may change between runtimes and architectures (i.e. it is not limited to 3 values). Some debugger frontends may want to visualize deadlocks, thread starvation, different crash reasons, waiting on barriers/mutexes.

Comment 14 Marc Khouzam 2011-09-14 14:33:22 EDT

(In reply to comment #13)
> > 4- coloring of threads based on state (running/suspended/crashed)
> 
> Please note that possible thread states may change between runtimes and
> architectures (i.e. it is not limited to 3 values). Some debugger frontends may
> want to visualize deadlocks, thread starvation, different crash reasons,
> waiting on barriers/mutexes.

Good point.  The framework should have an extendable type to be used by actual Visualizers.

Comment 15 Marc Khouzam 2011-09-15 13:55:21 EDT

I wrote the skeleton for a new DSF-GDB visualizer using the Visualizer framework.  It simply displays a test showing the string of the running process and all its threads.  You need a DSF session to be selected for it to display.

I have pushed the new plugin to the GitHub repo.

Below are the steps I did, which is what I believe is the recipe to create a new Visualizer using the framework.

Does this make sense?

1- Create a new plugin or use an existing one
2- add dependencies to org.eclipse.cdt.visualizer and org.eclipse.cdt.visualizer.ui
3- create an extension of org.eclipse.ui.views and use org.eclipse.cdt.visualizer.ui.VisualizerView.  This is to allow to open the new Visualizer view.  This view can display many different Visualizers.
4- create an extension of org.eclipse.ui.perspectiveExtensions, again for the Visualizer view.
5- create an extension of org.eclipse.cdt.visualizer.ui.visualizer and point to a new class implementing IVisualizer (can use as base class GraphicCanvasVisualizer or Visualizer).  This is the new Visualizer itself which will display some new graphical representation of the data.
6- implement the new subclass of Visualizer to display things

Comment 16 Marc Khouzam 2011-09-22 11:45:56 EDT

I was talking with a colleague that has access to a 24 core machine.  He mentioned that the machine is divided into 4 CPUs running 6 cores each.  He also said that memory can be locked by a CPU, which makes it more efficient to run threads sharing memory on the came CPU.

So, with a Visualizer, it would be nice to show the grouping of cores inside a CPU, so the user can see if threads are co-located on a CPU or not.

GDB does not provide this information, but we could have our DSF service get that info directly from Linux.

Something to consider once we have our first prototype.

Comment 17 William Swanson 2011-09-22 11:53:26 EDT

Agreed, and this is a good example of the kind of use case we should
keep in mind when specing out the "layout" service we've talked about,
which would collect this kind of info and provide it in a standard form.

Comment 18 Marc Khouzam 2011-09-22 13:08:32 EDT

(In reply to comment #17)
> Agreed, and this is a good example of the kind of use case we should
> keep in mind when specing out the "layout" service we've talked about,
> which would collect this kind of info and provide it in a standard form.

I'm about to start working on that service now.
I'm gonna start with IGdbHardware which will have a method to return the number of cores and some info about them.  I'll make the service Experimental so that we can keep adding to it, even once it is officially part of CDT.

Bill, any progress on the graphical side of things?

Comment 19 Marc Khouzam 2011-09-22 14:42:51 EDT

If the info is available, would it be relevant to optionally include CPU (core) utilization in the Multicore Visualizer?  Do you think that fits?

Comment 20 William Swanson 2011-09-22 16:32:23 EDT

Maybe, but that's leading over into profile/trace information, and we
might want to make that a separate service, keyed by cpu number,
process/thread id, etc. rather than having everything on one service.

Comment 21 Marc Khouzam 2011-09-23 16:04:50 EDT

(In reply to comment #20)
> Maybe, but that's leading over into profile/trace information, and we
> might want to make that a separate service, keyed by cpu number,
> process/thread id, etc. rather than having everything on one service.

Sure, that makes sense.  I'm more wondering if that is something that belongs in the Multicore Visualizer, or are we crossing the line of what should be shown in there?

Comment 22 Marc Khouzam 2011-09-23 16:13:16 EDT

Created attachment 203940 [details]
First version of IGDBHardware service interface

The Multicore Visualizer requires information that is not currently provided by the DSF-GDB services.

I suggest creating a new IGDBHardware service that will provide that information.  Attached is a patch for the first version I propose.  In summary:

- define a context for the hardware target.  This will be GDBControlDMContext at this time
- define a context for a CPU.  A CPU is a physical entity that can have one or more cores.  It is sometimes referred to as a 'socket'.
- define a context for core.  This context represents a logical core; I say 'logical' to include cores that are created because of Simultaneous Multi-Threading, where a physical core is actually considered two logical cores.
- define data for CPU and for core
- defined methods to getCPUs(), getCores(), getExecutionData().

The plan is that a CPUDmc will be the parent of one or more CoreDmcs.

I'm very interested in comments or suggestions.

Comment 23 William Swanson 2011-09-23 16:37:08 EDT

(In reply to comment #21)
> (In reply to comment #20)
> > Maybe, but that's leading over into profile/trace information, and we
> > might want to make that a separate service, keyed by cpu number,
> > process/thread id, etc. rather than having everything on one service.
> 
> Sure, that makes sense.  I'm more wondering if that is something that belongs
> in the Multicore Visualizer, or are we crossing the line of what should be
> shown in there?

It feels like a different "category" of data, if you see what I mean, which
kind of suggests some kind of layering or view selection. For example we might
have a basic "just the facts ma'am" view of cores/processes/threads, and then by
some kind of toggle or selection add one or more kinds of performance/trace data
(E.g. a "CPU utilization" view, or a "disk utilization" view, etc.), overlaid on
or replacing parts of the base view.

I'm kinda thinking we should shoot for the minimal view first, then look into
ways to expand upon it by adding these extra data overlays. But I'm not suggesting
we should not explore the kinds of extra state information (like CPU utilization) we can get our hands on, because these will help us define what visual overlay extensions we would want to include.

And bear in mind that this wants to be a useful example and/or base for other
folks to build on, so we should strive to keep it clean and simple.

Comment 24 Marc Khouzam 2011-09-26 09:02:02 EDT

(In reply to comment #23)

> It feels like a different "category" of data, if you see what I mean, which
> kind of suggests some kind of layering or view selection. For example we might
> have a basic "just the facts ma'am" view of cores/processes/threads, and then
> by
> some kind of toggle or selection add one or more kinds of performance/trace
> data
> (E.g. a "CPU utilization" view, or a "disk utilization" view, etc.), overlaid
> on
> or replacing parts of the base view.

I really like the concept of optional layers!  Once we get to that point, we may consider making it a new extension point, for people to add their own layer to an existing visualizer.

> I'm kinda thinking we should shoot for the minimal view first, then look into
> ways to expand upon it by adding these extra data overlays. But I'm not
> suggesting
> we should not explore the kinds of extra state information (like CPU
> utilization) we can get our hands on, because these will help us define what
> visual overlay extensions we would want to include.

Agreed.  Let's get something running before going further :-)

Comment 25 Marc Khouzam 2011-09-27 23:31:40 EDT

When trying to update GitHub with the latest eclipse changes, Bill and I realized that we cannot use rebase towards public repositories because rebase re-writes the history which would break any clones people might have made.

So, we needed to have the GitHub master branch unchanged, to allow to cleanly merge new commits from eclipse.org.  All our Visualizer work would need to go into a new visualizer branch, instead of master.

To achieve this, we had to do a reset on master, which re-writes history.  Therefore, anyone fetching or pulling will get errors.  If anyone cloned this repo, they need to get a new clone (same GitHub repo) to properly get the changes.  After that, everything should remain clean.  Sorry for any inconvenience, if anyone did have a clone.

Comment 26 Marc Khouzam 2011-09-29 15:19:38 EDT

Created attachment 204322 [details]
Data available for MulticoreVisualizer

I've just push a couple of commits to GitHub.  
- The service implementation now provides the list of cores for a local and for a remote session on Linux.
- The Multicore Visualizer now shows a text printout of CPUs (if available), of cores, and of the threads running on those cores.

The attached screenshot shows the printout to show the available data.

All we need now is to go graphical instead of textual.  We have the information to allow us to draw cores and their threads.

Comment 27 Marc Khouzam 2011-10-13 11:42:25 EDT

One aspect that is still unclear, is if/how to introduce the new IGDBHardware contexts (ICPU and ICore) into the DSF-GDB context hierarchy.

The complexity is two-fold:

1- although threads are associated with cores and cores with CPUs, a thread can move around.  I believe this is the first time we have such a case.  We may be able to update the thread context with its new parent once we reflect this change, but I'm not sure yet.

2- a Process is a conceptual entity (container for threads) and has an N-to-N mapping wit cores.  One process can 'run' on multiple cores, and many processes can 'run' on a single core.  So we're not dealing with a clear parent child relationship as we have for threads.

Comment 28 William Swanson 2011-11-15 11:31:59 EST

Hi, Marc --

I've pushed some new stuff to the visualizer GitHub repository, on the branch
"visualizer_review_new_ui".

High-level summary:
- a proposed package restructuring for the Multicore Visualizer example,
   adding "model" and "view" packages, plus a "utils" package for some
   helper classes that make it simpler to deal with the DSF calls
- a first cut at a graphic visualization for cpus/cores/threads,
   following the pattern in your text-based DSF service example
- an example of the kinds of UI "model" objects we were talking about
   for cpus/cores/threads, to store visualizer state
- an example of a "listener-like" layer/API that isolates the visualizer code
   from the hairier DSF access stuff (more on this below)
- a few supporting changes in the visualizer framework

More detail:
- In the "internal.ui.view" package, I've added my MulticoreVisualizer
  implementation, along with the canvas class and graphic objects it uses.
  The canvas class in particular follows the style I use in the Grid View,
  where gathering/caching of model data is decoupled from refreshing the
  visualizer display.

  I haven't done anything yet with visualizer selection, toolbar, menubar, etc.
  This is just a first cut at the visual look of the visualization.

  I'm thinking the current internal.ui.MulticoreVisualizer class either goes
  away or becomes a "debug" view for development use only.

- In the "internal.ui.model" package, I've added an example of a
  "VisualizerModel" object and nested classes for cpus, cores, and threads,
  which is used to capture a "snapshot" of the state we retrieve via DSF and
  hand it to the visualizer for later display.  (The Todo class is used when
  constructing these snapshot models. More on this below.)

- In the "internal.utils" package, there are a number of helper/facade classes
  that arose from me trying to wrap my brain around the DSF framework.

  A few of these are just renamings to make things more obvious (like
  OnNthDoneMonitor for CountingRequestMonitor). I also added a "DSFSessionState"
  to bundle up a few separate DSF session/context references that seem to be
  used together a lot.

  One part of this I'd really like you to review and comment on is the
  DSFDebugModel and DSFDebugModelListener classes. Here I tried to organize
  the DSF lookup code into a more "listener-like" callback pattern.
  Basically, I define a DSFDebugModelListener interface, which an object that
  wants to make DSF calls can implement. The object can then hand itself (or a
  proxy listener) into the functions on DSFDebugModel. All the anonymous class
  and RequestMonitor stuff happens within DSFDebugModel, and when a given
  DSFDebugModel method completes, it calls back to the appropriate method on the
  listener, which can then handle the results as usual.

  We can discuss the appropriate method namings and signatures for the
  DSFDebugModel methods (and for the class/interface names as well), this is
  just an example of how this might look. There's an example of how this works
  in the internal.ui.view.MulticoreVisualizer class, in the getVisualizerModel()
  method and subsequent methods.

  I find this pattern to be much cleaner and more maintainable than coding
  directly on top of DSF (maybe because I'm so used to the listener/event
  paradigm from UI coding).  What do you think? Would it make sense to an API
  convenience layer like this to DSF?

  As a side note, I decided that the function of CountingRequestMonitor is
  better handled as an aspect of the model being constructed: that is, the
  DSFDebugModel call just returns a list of cores for a CPU, or a list of
  threads for a core. It doesn't care that a given call happens to be the one
  that "finishes" the model being constructed; instead, that should be tracked
  and handled by the listener method(s) that are actually constructing the
  model.  (You can see an example of this in the getVisualizerModel() methods,
  where I use the Todo helper class to track the remaining work.) Again, what do
  you think? I'm new to DSF so I'm not sure whether the CountingRequestMonitor
  class has uses aside from model construction.

- I've made a few supporting changes in the visualizer framework:
  - I added the concept of "decorations" to GraphicObjects, so CPUs can
    draw their IDs on top of the cores that get placed on top of them
  - I added a Colors resource manager for SWT Color objects
  - I added a Timer convenience wrapper for the Swing Timer class
    (maybe I should use the SWT timer interface instead? I didn't bother
     rewriting this class since I already had it)
  Some of this stuff feels like it wants to be in a general utility library
  somwhere; I just added it to the Visualizer framework's utils package for now.

- I also made one tweak to the CoreList.getCoreList() method in
  cdt.dsf.gdb.internal, so that it returns core IDs that are unique across all
  cores. This may not be the right way to do this (it would be best if the core
  IDs matched the Linux cpu IDs assigned to them), but it at least allows me to
  show the proper layout of threads in the visualizer.

A few "new user" comments on the DSF API:

  - I'm still not quite clear on the DSF hierarchy for processes/threads; the
    extra "container" and "executable" types make it confusing how DSF really
    represents processes vs. threads. (In particular, I'm not sure whether the
    current DSFDebugModel.getThreads() method can -- or should -- be broken out
    into separate getProcesses() and getThreadsForProcess() calls, for example.)

  - The reliance on anonymous classes and closure variables to pass state
    around really makes code hard to understand and maintain. I'm no stranger
    to these concepts, but it took me the better part of a morning just to
    understand and reimplement the DSF example you wrote for looking up
    processes and threads, to tie it into the visualizer, and I didn't even
    want to _think_ about trying to change or extend it once I was done.
    That's a big reason why I took a stab at the listener-like version of
    things. For simple lookups like cores for cpus, etc. there wants to be
    a simpler, cleaner API.

  - Might just be my opinion, but it really feels wrong to be using annotations
    to define the methods to be called on service event handler instances.
    This is kinda the canonical example of what interfaces are used for,
    and using annotations means one can't use Eclipse's own class hierarchy
    inspection tools to discover the methods on an event handler interface
    and verify that an object properly implements the methods it should.
    (Yes, using annotations makes it easier to simply "flag" methods
    to receive events, but if one doesn't want to have to implement every
    method on an interface, the way to do it is extend, or delegate to,
    an adapter class instead.)

Otherwise, the DSF framework works really well, and I'm able to display an
application running on a quad-core Linux box here. I've even worked out the
visualizer display code for arbitrary numbers of cores/cpus, if we have the
chance to run on anything larger.  (That was a struggle, let me tell you.
It's not as trivial as it looks getting the cpus and cores to efficiently
use the available screen space!)

Anyway, have a look at the checked-in code, and let me know if you have any
comments/suggestions. If this all looks fine we can merge the review branch
to the visualizer branch. Actually, there are no pending changes on the
visualizer branch, so we should be able to just do an inlining merge, right?

Comment 29 Marc Khouzam 2011-12-02 15:31:12 EST

(In reply to comment #28)
> Hi, Marc --
> 
> I've pushed some new stuff to the visualizer GitHub repository, on the branch
> "visualizer_review_new_ui".

Thanks Bill!  I have started reviewing things and will have comments by early next week.

After the review, I would like to start helping with the code for parts you would like me to handle.  Can you point me to something you feel would be good for me to work on?

Comment 30 William Swanson 2011-12-02 15:52:51 EST

There's a couple things we can focus on:
(a) remaining required stuff for the multicore visualizer
(b) anything "extra" we should consider adding to
     add depth and interest when presenting it

For (a), here's a stab at a short list of items.
Feel free to remind me of any stuff I've left out:
- using actual cpu numbers, process/thread ids, etc. [you?]
- displaying current execution state via colors [you/me?]
- dynamic (event-based and/or periodic) updating [you/me?]
- drag selection of processes/threads [me]
- acceptance of selection from debug view, etc. [me]
- visualizer toolbar/menu commands [me]

For (b) I've been thinking that we'll want to be able
to show that the multicore visualizer, and the framework
in general, aren't just a one-trick pony.

One thing we've talked about is having some kind of layering
of information. Even if we didn't have any additional
DSF-derived debug information to display, perhaps we could
make the execution state (running/suspended/crashed) be
an "overlay" that one could turn on/off, just as an example.

Something else that came to mind is that it would be good
to have a simple example of a completely different kind
of visualizer, to get across that this really is a pluggable
architecture.

A simple idea that came to mind is a visualizer that takes
a text selection (like some random source code in a text
editor) and produces a simple bar-graph of it, like most
common characters, etc. Something like:
   ________________
e [________________] 120
t [____________]     90
a [__________]       80
... etc...

Just something we can toss up on the screen (just by
selecting some text) in order to say "Oh, by the way, the
framework isn't just for multicore visualization, you could
use it to support other kinds of development views."

Suggestions are welcome for something more "real world",
but the point of this is to be something quick to do
yet immediately "obvious" as a visualization.

I'm thinking it might be good for you to add this, Marc,
just so we can show that someone other than me can
write a graphic visualizer :-) With the existing code for
the multicore visualizer code as an example, this should
not be a huge task.

Anything else we should consider?

Comment 31 Marc Khouzam 2011-12-05 10:30:15 EST

(In reply to comment #30)
> There's a couple things we can focus on:
> (a) remaining required stuff for the multicore visualizer
> (b) anything "extra" we should consider adding to
>      add depth and interest when presenting it

I'll (c) contributing the code to CDT.  To avoid a last minute rush, and to make sure we have something in CDT for EclipseCon, I suggest we start the contribution process in January.  I will take care of the details.  However, we should look at:

1- what is missing to make it functional enough to be contributed
2- all code must have proper copyright headers
3- Does Bill have the right to contribute the code with respect to Tilera

Here is the IP Poster that describes what steps we will follow:
http://eclipse.org/legal/EclipseLegalProcessPoster.pdf

> For (a), here's a stab at a short list of items.
> Feel free to remind me of any stuff I've left out:
> - using actual cpu numbers, process/thread ids, etc. [you?]

Could this sometimes crowd the display too much and there
for should be optional?  If so, I could be another layer.
Or we can use a hover.  Or both.

I can take a stab at that.  Although if we want it to be
a layer, we should define the layering code first.

> - displaying current execution state via colors [you/me?]

Of the processes/threads.  But not the cores.  I don't think 
I have that information available for cores yet.
I can try that one too, as it is related to the dynamic updating.

> - dynamic (event-based and/or periodic) updating [you/me?]

Ah yes, that is something I can do easily with DSF events.
I'll take care of that.

> - drag selection of processes/threads [me]

That will look great in a demo.

> - acceptance of selection from debug view, etc. [me]

Are you referring to the synchronization with Debug view?

> - visualizer toolbar/menu commands [me]

I suggest you try things with 3.7M3 since the global
debug toolbar is enabled in that build.
Will this item allow the user to do run control operations
(step, resume, interrupt), from the Visualizer?  I think
that would be really good for the demo.

> For (b) I've been thinking that we'll want to be able
> to show that the multicore visualizer, and the framework
> in general, aren't just a one-trick pony.
> 
> One thing we've talked about is having some kind of layering
> of information. Even if we didn't have any additional
> DSF-derived debug information to display, perhaps we could
> make the execution state (running/suspended/crashed) be
> an "overlay" that one could turn on/off, just as an example.

This is for processes/threads or were you thinking of cores.

Sure, that could be an overlay.  The id (as I wrote above),
could be an overlay as well.  In fact, maybe almost
everything could be an overlay.  That way we keep things
isolated and following the same pattern, and we give
the user total flexibility.

I suggest we define the overlay code and then work on feature
that can be overlays.

BTW, whenever I talk to people of the Multicore Visualizer,
they respond very well to the idea of overlay; it seems to
be an approach that will receive good acceptance.

> Something else that came to mind is that it would be good
> to have a simple example of a completely different kind
> of visualizer, to get across that this really is a pluggable
> architecture.
> 
> A simple idea that came to mind is a visualizer that takes
> a text selection (like some random source code in a text
> editor) and produces a simple bar-graph of it, like most
> common characters, etc. Something like:
>    ________________
> e [________________] 120
> t [____________]     90
> a [__________]       80
> ... etc...
> 
> Just something we can toss up on the screen (just by
> selecting some text) in order to say "Oh, by the way, the
> framework isn't just for multicore visualization, you could
> use it to support other kinds of development views."
> 
> Suggestions are welcome for something more "real world",
> but the point of this is to be something quick to do
> yet immediately "obvious" as a visualization.
> 
> I'm thinking it might be good for you to add this, Marc,
> just so we can show that someone other than me can
> write a graphic visualizer :-) With the existing code for
> the multicore visualizer code as an example, this should
> not be a huge task.

That would be nice.  Let's try to get the main features
ready and the contribution underway, then I/we can create
an example plugin that uses the visualizer framework
to do something like what you suggest.

> Anything else we should consider?

I think for this plan sounds good, including the contribution.

Depending on how well (or badly :-O) we progress during the next 
three months we'll adapt to make sure we are ready for
EclipseCon.

Comment 32 William Swanson 2011-12-05 12:57:52 EST

(In reply to comment #31)
> 
> > - using actual cpu numbers, process/thread ids, etc. [you?]
> 
> Could this sometimes crowd the display too much and there
> for should be optional?  If so, I could be another layer.
> Or we can use a hover.  Or both.

We shouldn't go overboard on layers, especially for simple details
that would generally want to be there. In writing the Grid View
(and the Visualizer) I've designed the drawing code to selectively
omit detail (like labels) when the view is resized too small
to clearly display them. The current visualizer does this for
core and cpu numbers, and we can use a similar pattern for other
kinds of labels.

> > - displaying current execution state via colors [you/me?]
> 
> Of the processes/threads.  But not the cores.  I don't think 
> I have that information available for cores yet.
> I can try that one too, as it is related to the dynamic updating.

The background color of the cores can reflect the general state of
the processes/threads on that core (we do something similar with
the Grid View).

> > - dynamic (event-based and/or periodic) updating [you/me?]
> 
> Ah yes, that is something I can do easily with DSF events.
> I'll take care of that.

Cool, thanks!

> > - drag selection of processes/threads [me]
> 
> That will look great in a demo.
> 
> > - acceptance of selection from debug view, etc. [me]
> 
> Are you referring to the synchronization with Debug view?

Yes, this goes with the item above it, I was just making
it clear that selection handling wants to be bidirectional.

> > - visualizer toolbar/menu commands [me]
> 
> I suggest you try things with 3.7M3 since the global
> debug toolbar is enabled in that build.

Hmmm -- what's that in Git tags? Do we need to rebase the
visualizer branch to pick up new stuff (or alternatively,
do another merge from main?)

BTW, we still need to merge the current pending visualizer
changes into the visualizer branch. I'm waiting on your feedback
to do that (or if it's generally okay with you, I can go ahead
and merge it in, and we can deal with review changes after.)


> Will this item allow the user to do run control operations
> (step, resume, interrupt), from the Visualizer?  I think
> that would be really good for the demo.

Yes, assuming I don't have to jump through too many hoops
to invoke the corresponding actions from the Debug View.

> > For (b) I've been thinking that we'll want to be able
> > to show that the multicore visualizer, and the framework
> > in general, aren't just a one-trick pony.
> > 
> > One thing we've talked about is having some kind of layering
> > of information. Even if we didn't have any additional
> > DSF-derived debug information to display, perhaps we could
> > make the execution state (running/suspended/crashed) be
> > an "overlay" that one could turn on/off, just as an example.
> 
> This is for processes/threads or were you thinking of cores.

Processes/threads, yes, though it could include cores -- that is,
an "overlay" might just select a different way of drawing things.
So instead of cores/cpus being flat green, their color might reflect
state, etc.

> Sure, that could be an overlay.  The id (as I wrote above),
> could be an overlay as well.  In fact, maybe almost
> everything could be an overlay.  That way we keep things
> isolated and following the same pattern, and we give
> the user total flexibility.
> 
> I suggest we define the overlay code and then work on feature
> that can be overlays.

It's not clear that overlays are necessarily part of the framework,
since they're basically a technique for a given visualization to
provide "alternate" views of its content. So this may just be a
feature of a given visualizer, like the Multicore visualizer,
though we might add some supporting classes, etc. to the framework
if there's enough reusable stuff.

> > Something else that came to mind is that it would be good
> > to have a simple example of a completely different kind
> > of visualizer, to get across that this really is a pluggable
> > architecture.
> >
> > ...
> > 
> > I'm thinking it might be good for you to add this, Marc,
> > just so we can show that someone other than me can
> > write a graphic visualizer :-) With the existing code for
> > the multicore visualizer code as an example, this should
> > not be a huge task.
> 
> That would be nice.  Let's try to get the main features
> ready and the contribution underway, then I/we can create
> an example plugin that uses the visualizer framework
> to do something like what you suggest.

Cool, thanks again.

> > Anything else we should consider?
> 
> I think for this plan sounds good, including the contribution.
> 
> Depending on how well (or badly :-O) we progress during the next 
> three months we'll adapt to make sure we are ready for
> EclipseCon.

Great -- let me know if there's anything I need to run down on
this end to help facilitate things.

Comment 33 Marc Khouzam 2011-12-06 10:51:53 EST

(In reply to comment #32)
> We shouldn't go overboard on layers, especially for simple details
> that would generally want to be there. In writing the Grid View
> (and the Visualizer) I've designed the drawing code to selectively
> omit detail (like labels) when the view is resized too small
> to clearly display them. The current visualizer does this for
> core and cpu numbers, and we can use a similar pattern for other
> kinds of labels.

That is nice!

> > Are you referring to the synchronization with Debug view?
> 
> Yes, this goes with the item above it, I was just making
> it clear that selection handling wants to be bidirectional.

Do we need clarification on the approach for this?  We had talked about using the DebugContextProvider API provided by the platform.  We should ask Pawel to join the multicore conf call next week to give us input on that.  I will contact him.

> > > - visualizer toolbar/menu commands [me]
> > 
> > I suggest you try things with 3.7M3 since the global
> > debug toolbar is enabled in that build.
> 
> Hmmm -- what's that in Git tags? Do we need to rebase the
> visualizer branch to pick up new stuff (or alternatively,
> do another merge from main?)

:-)
You have to install a new Eclipse version.  Doing this is a good thing anyway to make sure our code works with the platform that will become the Juno release.  Pretty soon we will even need to move to the 4.x platform, but let's wait a bit on that, I'll bring it up at the CDT meeting today.
You can get the M3 platform build at:
eclipse.org -> Download Eclipse -> 
Other Downloads (under Eclipse Classic)->
Eclipse 3.x downloads are here-> 3.8M3

(sorry I had written 3.7, it is 3.8)


> BTW, we still need to merge the current pending visualizer
> changes into the visualizer branch. I'm waiting on your feedback
> to do that (or if it's generally okay with you, I can go ahead
> and merge it in, and we can deal with review changes after.)

I will post my comments today.  Sorry about the delay.

> It's not clear that overlays are necessarily part of the framework,
> since they're basically a technique for a given visualization to
> provide "alternate" views of its content. So this may just be a
> feature of a given visualizer, like the Multicore visualizer,
> though we might add some supporting classes, etc. to the framework
> if there's enough reusable stuff.

I was imagining extension points and some infrastructure to add overlays to a perticular visualizer.  The actual visualizer could be added to the extension point definition.  I'll think about it some more and make you a proposal.

Comment 34 Marc Khouzam 2011-12-06 14:30:53 EST

> Hi, Marc --
Hi Bill,

sorry the long delay, but thank you for this update.

(In reply to comment #28)

> I've pushed some new stuff to the visualizer repository, on the branch "visualizer_review_new_ui".
> (I'm thinking I'll use "visualizer_review_X" as a pattern for naming review branches.)

That sounds ok, although I'm thinking that I will probably focus reviews on DSF-related aspects.  The graphical/visualizing parts of things are really your domain, and I don't my opinions would bring much value.  That is not to say that I won't test, debug and have a look at that part of the code, but really, what you suggest pretty much goes in that area.

So, for code changes that are not DSF-related, you can probably go straight to the main branch.

> There's a bunch of stuff here. To summarize:

A general comment.  It may be wise to replace all hard-code drawing values with constants.  We could have a single file for all constants which would allow to very quickly change the look and feel of the Multicore Visualizer by easily changing e.g., colors, thinkness of lines, etc.  Eventually, we will probably need to make such things preferences.  For example, MulticoreVisualizerCPU.paintDecorations() uses hard-code values, along
with other such methods.

> - a proposed package restructuring, adding "model" and "view" packages,
>     plus a "utils" package for some helper classes that make it simpler to deal with the DSF calls

Makes sense.

> - a first cut at a graphic visualization for cpus/cores/threads, following the
>     pattern in your text-based DSF service example
> - an example of the kinds of "model" objects I was talking about for cpus/cores/threads
> - an example of a "listener-like" layer/API that isolates the visualizer code from the
>    hairier DSF access stuff (more on this below)

I see your reasoning behind this part.  I think with DSF-experience, you may not see as much value in this change though.  I reviewed it in detail below and wrote comments.

> - a few supporting changes in the visualizer framework
> 
> More detail:
> - In the "internal.ui.view" package, I've added my MulticoreVisualizer implementation,
>    along with the canvas class and graphic objects it uses. The canvas class in particular
>    follows the style I use in the Grid View, where gathering/caching of model data
>    is decoupled from refreshing the visualizer display.
>
>    I haven't done anything yet with visualizer selection, toolbar, menubar, etc.
>    This is just a first cut at the visual look of the visualization.

Nice!
And these other features will make everything even better.

>    I'm thinking the current internal.ui.MulticoreVisualizer class either goes away or
>    becomes a "debug" view for development use only.

I've removed it and pushed it to the visualizer_review_new_ui branch.

> - In the "internal.ui.model" package, I've added an example of a "VisualizerModel" object
>    and nested classes for cpus, cores, and threads, which is used to capture a "snapshot"
>    of the state we retrieve via DSF and hand it to the visualizer for later display.
>    (The Todo class is used when constructing these snapshot models. More on this below.)

I will need to get comfortable with the code, and I think the best way to do that is to add a feature.  So I'm thinking of starting with the dynamic updating we talked about, which should give me reasons to dive into that code.

> - In the "internal.utils" package, there are a number of helper/facade classes that arose
>   from me trying to wrap my brain around the DSF framework.

I understand that some aspects of DSF are heavy and not at all obvious.  However, I believe that with experience, they become relatively easy to use.  So, I am not against adding helper classes, but those methods really would need to make things clearer for a new user.  Let's go over each one to discuss their value.

1- OnDoneMonitor.java and OnDataRequestDoneMonitor.java
This class is meant to avoid a new cycle of the DSF executor by using the ImmediateExecutor. I actually use the ImmediateExecutor in most of my RMs so I definitly see your point. But the name 'OnDoneMonitor' does not feel very clear to me; I had to read the code to figure out what it was meant for. 
How about 'ImmediateRequestMonitor' and 'ImmediateDataRequestMonitor'?
If these classes are worth implementing, then they belong in DSF directly.
I'm not sure Pawel will see the value, but we can try to propose a patch if you feel it is worth the effort.  I can probably make a case for it, as we have had troubles with deeply nested calls to the executor through RMs.  One thing though, OnDoneMonitor.handleCompleted() must either be removed or must
call super.handleCompleted().

2- OnNthDoneMonitor.java
This class has two aspects: the Immediate aspect, and calling setDoneCount from 
the constructor.  For the Immediate aspect, if we suggest such a concept for DSF as discussed in the point above, we should probably apply it to all subclasses of RM (RequestMonitor, DataRequestMonitor, CountingRM, MultiRM, and RequestMonitorWithProgress). As for calling setDoneCount from the constructor, we should add a constructor to CountingRequestMonitor directly, if we feel it would be useful.

3- DSFSessionState.java
I also see the value of this class.  It makes the code much simpler for classes
that use it.  I'm not a big fan of the name though...
Also, the fact that it mixes automatically using the executor for some calls
(e.g., addServiceEventListener) but not others (e.g., getService()) seems
risky to me. 
Also, it probably shouldn't have specific methods like
getHardwareContext() and getHardwareService(), but have generic ones
since we may later need a different service or context.
I think it is ok for a helper class for the MulticoreVisualizer specifically,
since it helps.

>   A few of these are just renamings to make things more obvious (like OnNthDoneMonitor
>   for CountingRequestMonitor). I also added a "DSFSessionState" to bundle up a few
>   separate DSF session/context references that seem to be used together a lot.
> 
>   The part of this I'd really like you to review and comment on is the DSFDebugModel
>   and DSFDebugModelListener classes. Here I tried to "rationalize" the DSF lookup code
>   into a more "listener-like" callback pattern. Basically, I define a DSFDebugModelListener
>   interface, which an object that wants to make DSF calls can implement. The object can
>   then hand itself (or a proxy listener) into the functions on DSFDebugModel. All the
>   anonymous class and RequestMonitor stuff happens within DSFDebugModel,
>   and when a given DSFDebugModel method completes, it calls back to the appropriate
>   method on the listener, which can then handle the results as usual.
> 
>   We can discuss the appropriate method namings and signatures for the DSFDebugModel
>   methods (and for the class/interface names as well), this is just an example
>   of how this might look. There's an example of how this works in the
>   internal.ui.view.MulticoreVisualizer class, in the getVisualizerModel() method
>   and subsequent methods.
> 
>   I find this pattern to be much cleaner and more maintainable than coding directly
>   on top of DSF (maybe because I'm so used to the listener/event paradigm from UI coding).
>   What do you think? Would it make sense to an API convenience layer like this to DSF?

I like how you can use the listener pattern to handle DSF calls with this.  However, being used to DSF, I actually found it harder to follow.  The mapping from, for example, DSFDebugModel.getCPUs() to getCPUsDone() is done in another class (DSFDebugModel), which requires the reader of the code to go see to confirm which method will be called once getCPUs() is called.  In fact, nothing would make a reader realize that getCPUs() is actually an asynchronous call, since it does not require an RM.

Another problem is that it may happen that in some cases, when calling getCPUs() you want to do one thing, when done, while you'd prefer to do something else in another case; defining the RM directly inside the call to getCPUs() allows you to tailor the callback per situation.

Also, by using a listener instead of an RM, it may make debugging more difficult
if we forget to call the listener's callback method.  This is not new, because we can forget to call RM.done(), but now we'd have a second pattern to pay attention to for forgotten calls.

I think you could achieve a very similar effect by using this pattern:

DSFDebugModel.getCPUs(new ImmediateRequestMonitor(null) {
    @Override
    protected void handleCompleted() {
        getCPUsDone();
    }
});

>   As a side note, I decided that the function of CountingRequestMonitor is better handled
>   as an aspect of the model being constructed: that is, the DSFDebugModel call just returns
>   a list of cores for a CPU, or a list of threads for a core. It doesn't care that a given call
>   happens to be the one that "finishes" the model being constructed; instead, that should be
>   tracked and handled by the listener method(s) that are actually constructing the model.
>   (You can see an example of this in the getVisualizerModel() methods, where I use the
>   Todo helper class to track the remaining work.) Again, what do you think? I'm new to
>   DSF so I'm not sure whether the CountingRequestMonitor class has uses aside from
>   model construction.

It makes sense to me.

> - I've made a few supporting changes in the visualizer framework:
>   - I added the concept of "decorations" to GraphicObjects, so CPUs can draw their IDs
>      on top of the cores that get placed on top of them

GraphicCanvas.java
- in paintCanvas(), should the decoration be drawn right after
the graphic object is drawn, instead of drawing all decorations at the end?
So as to keep the decorations at the same z-ordering as their mathing graphics object?
Or do we specifically want all decorations to be shown above everything else (as your last above sentence makes me think)?

- in getGraphicObject(Class<?> type, int x, int y), we could use a ListIterator to traverse in reverse (not a big deal, I just thought it may be more efficient, although I' not sure)
- in getGraphicObjectFor(Object value), "if (gobj.getData() == value)" should use equals() instead of ==
- getDataFor() does not seem to add a lot of value unless it is for inheritance?
  It could be made static as well, although that might not be a good idea for inheritance.

>   - I added a Colors resource manager for SWT Color objects
>   - I added a Timer convenience wrapper for the Swing Timer class
>     (maybe I should use the SWT timer interface instead? I didn't bother rewriting this class
>      since I already had it)
>   Some of this stuff feels like it wants to be in a general utility library somwhere;
>   I just added it to the Visualizer framework's utils package for now.

I have not experience here, but it does look like there would be some supporting
methods in SWT or Eclipse for these things.  But maybe not.  Let's go with what
you feel is best.

> - I also made one tweak to the CoreList.getCoreList() method in cdt.dsf.gdb.internal,
>   so that it returns core IDs that are unique across all cores. This may not be the
>   right way to do this (it would be best if the core IDs matched the Linux cpu IDs
>   assigned to them), but it at least allows me to show the proper layout of threads
>   in the visualizer.

I'm not sure the algorithm you used is correct if you have a different number of 
cores on the different CPUs.  For example if you have 4 cores on CPU 0 and 2 cores on CPU 1, the 6 cores will have absoluteCoreID of: 0, 1, 2, 3, 2, 3.  As you can see the numbers overlap.

Could we simply use a global counter that increments by one every time we find
a new core?

> A few "new user" comments on the DSF API:
>   - I'm still not quite clear on the DSF hierarchy for processes/threads;
>     the extra "container" and "executable" types make it confusing how
>     DSF really represents processes vs. threads. 

This aspect has been a thorn since the start, and it only in the DSF-GDB implementation.
DSF simply defines the different contexts, and it is up the implementation to choose how to organize the hierarchy.

In the current implementation, 'containerDMC' is a process being debugged by GDB; it contains information that makes sense only for GDB (like 'thread-group id').  'processDMC' is any process running on the system, being debugged or not; it holds information that makes sense at the OS level (like 'OS pid').  The example I like is that when you connect to a target, you can list all running processes, those are only 'processDMC'; but once you choose one of them for GDB to debug, that process gets a new 'containerDMC' which has 'processDMC' as parent.

Note that I said "in the current implementation" because with the grouping feature, the mapping of containers, may change to become a more generic container.  But let's not worry about that for now.

'executionDMC' represents an entity of execution that is being debugged.  A debugged process is an 'exectionDMC' (as well as a 'containerDMC'); a debugged thread is an 'executionDMC' (but not a 'containerDMC').  Similarly to 'processDMC', a 'threadDMC' is the OS representation of a thread, even if that thread is not under control of the debugger.

You can see an ascii drawing of the hierarchy in GDBProcesses_7_0.
The thing to note is that if you have a 'containerDMC', you can get its 'processDMC' parent, but not vice versa.

>     (In particular, I'm not sure whether
>     the current DSFDebugModel.getThreads() method can -- or should -- be broken
>     out into separate getProcesses() and getThreadsForProcess() calls, for example.)

Probably would be easier to read and maintain.  In fact, we probably will need a way to get the list of processes in the future, so might as well have it available.

>   - The reliance on anonymous classes and closure variables to pass state around
>     really makes code hard to understand and maintain. I'm no stranger to this stuff,
>     but it took me the better part of a morning just to understand and reimplement
>     the DSF example for looking up processes and threads, and I didn't even want to
>     think about trying to make changes it in once I was done. That's a big reason why
>     I took a stab at the listener-like version of things. For simple lookups like
>     cores for cpus, etc. there wants to be a simpler, cleaner API.

I agree that some methods with nested calls get much too complicated.  But I believe that is because I don't bother coding it nicely.  I think we can just devide the work into asynchronous methods and make it much easier to read.

>   - It really feels wrong to be using annotations to define the methods to be called on
>     service event handler instances. This is kinda the canonical example of what interfaces
>     are used for, and using annotations means one can't use Eclipse's own class hierarchy
>     inspection tools to discover the methods on an interface and/or verify that an object
>     properly implements the methods it should.

Yes, I guess there are drawbacks to that approach.  However, it is quite similar to the listener approach.

> Otherwise, the DSF framework works really well, and I'm able to display an application
> running on a quad-core Linux box here. I've even worked out the visualizer display code
> for arbitrary numbers of cores/cpus, if we have the chance to run on anything larger.
> (That was a struggle, let me tell you. It's not as trivial as it looks getting the cpus and
>  cores to efficiently use the available screen space!)

I think that effort has a lot of value.  Being able to tell the community and present at EclipseCon a Multicore Visualizer that can work with an arbitrary numbers of cores/cpus, will make people pay attention.
Thanks for taking the time to get it working like that!

> Anyway, have a look at the checked-in code, and let me know if you have any
> comments/suggestions. If this all looks fine we can merge the review branch
> to the visualizer branch. Actually, there are no pending changes on the visualizer branch,
> so we should be able to just do an inlining merge, right?

When you want to push those change to the Visualizer branch, you can set the
Visualizer branch, then rebase the review_ui branch onto it.

So, thanks again for all this work.  I will now get more technically involved so I hope to have much faster turnaround on reviews.

One concern that I have about the code is that it does not follow the same coding style as the other two dsf.gdb plugins.  And that is a tricky thing since you have your coding style and I have mine.  What I suggest is that we try to have dsf.gdb.multicorevisualizer follow the pattern of the other two dsf.gdb plugins, but let the two cdt.visualizer plugins follow whichever style your are more comfortable with.

I'm not asking that you change any code for this.  But I would like your persmission for me to modify the dsf.gdb.multicorevisualizer plugin ss I start coding in it, so that it matches the other two.  One example is using fMemberVariable (f for 'field', which was how DSF call such variables, and we just continued) instead of m_memberVariable.

What do you think?

I'm going to start looking at the dynamic update feature right away and get my hands dirty again.

Comment 35 Marc Khouzam 2011-12-07 09:13:14 EST

Created attachment 208050 [details]
Poor display of 1000 threads

Hi Bill,

I have a dumb program that creates 1000 threads.  The current display of the multicore visualizer does not handle that very well.  See attachment.

I wonder if you have dealt with this for the Grid view or if this is a new use case? Should we spread out the threads in 2D instead of a single line?
Should we group the threads into sets?  For examples, large arrays
are divided in groups of 100 in the variables view (this does not work for DSF yet, but it work for the java Debugger); the user can then open that sub-group to actually see the array content.  That is an approach we could use.

Comment 36 Marc Khouzam 2011-12-07 09:44:53 EST

I was thinking of our approach of adding the Visualizer view from the multicore visualizer plugin.  This may not be a good idea.  If someone else defines a plugin with a new visualizer, should they add the visualizer view themselves?

If they do and the user has the multicore visualizer also, we'll have two visualizer views; if they don't, and the user does not have the multicore visualizer plugin, they won't have the view at all.

I think we should define the view in plugin.xml of the cdt.visualizer.ui plugin.  If there are no visualizers defined (multicore visualizer plugin not present), there is an informative string in the view saying: "(No visualizers defined.)"

If you agree, I have a patch I can push.

Comment 37 William Swanson 2011-12-07 11:02:07 EST

(In reply to comment #35)
> Poor display of 1000 threads

As I mentioned in the conf call a while back, we don't normally
have to worry about that case, because having a gazillion
threads on the same core is usually not an effective use
of processor resources. One should be using a master/worker
pattern, or thread pooling, etc. so that ideally one or at most
a few threads are running on each core.

This hasn't come up as an issue for the Grid View because
developers tend to be savvy about multicore design, but
you're right that people running generic code in the visualizer
are more likely to try "let see if it goes to eleven"
kinds of programs, which would raise these kinds of issues.

We can modify the visualizer so that if more than
a displayable number of threads are on a given core, we only draw
a few, then a "..." indication, or otherwise indicate there are
"An Awful Lot of Threads Here". (In a tooltip, or other mechanism,
we can show the individual thread IDs, if the list isn't
ridiculously long, and if it is, we use "..." again.)

I wouldn't go overboard trying to squeeze in lots of threads,
because as noted above, this case should be the exception
rather than the rule. (And in fact, having a lot of threads
crowded on one tile is a good visual indicator of imbalanced
code, so you don't _want_ it to look too neat and tidy! :-)

Comment 38 William Swanson 2011-12-07 11:12:04 EST

(In reply to comment #36)
> 
> I think we should define the [Visualizer] view in plugin.xml of the
> cdt.visualizer.ui plugin.  If there are no visualizers defined
> (multicore visualizer plugin not present), there is an informative string
> in the view saying: "(No visualizers defined.)"
> 
> If you agree, I have a patch I can push.

Makes sense -- the visualizer view is intended to be shared by all
the defined visualizers, ours and anyone else's. One could even argue
that at some point it might have to move to an Eclipse platform plugin,
because visualizations aren't necessarily restricted to CDT or debugging.
But we're assuming the boundary is CDT for now, so cdt.visualizer.ui
sounds okay.

Comment 39 William Swanson 2011-12-07 11:55:58 EST

(In reply to comment #34)

  > I've pushed some new stuff to the visualizer repository,
  > on the branch "visualizer_review_new_ui".
  > (I'm thinking I'll use "visualizer_review_X" as a pattern for naming review branches.)

  That sounds ok, although I'm thinking that I will probably focus reviews on
  DSF-related aspects.  The graphical/visualizing parts of things are really your
  domain, and I don't my opinions would bring much value. That is not to say
  that I won't test, debug and have a look at that part of the code, but really,
  what you suggest pretty much goes in that area.

Having a second set of eyes look over my Java code is always helpful.
And comments on appearance/behavior are always welcome.

  A general comment.  It may be wise to replace all hard-code drawing values with
  constants.  We could have a single file for all constants which would allow to
  very quickly change the look and feel of the Multicore Visualizer by easily
  changing e.g., colors, thinkness of lines, etc.  Eventually, we will probably
  need to make such things preferences.  For example,
  MulticoreVisualizerCPU.paintDecorations() uses hard-code values, along
  with other such methods.

I generally do have constants at least partly pulled out as named locals
or class constants, for self-documentation, but certainly we can extract
them further into a "constants" class or the like.

I'd be cautious about trying to overparameterize the drawing code.
Tweaking colors/fonts, etc. is one thing, but trying to make it fully
customizable is probably out of scope here. And you know someone will always
demand the One More Thing. ("Hey, why can't I make this line here
exactly 3.2 picas, dot-dot-dashed, and... and... MAUVE, HUH?" :-)

If someone really wants to do a complete visual overhaul of the visualizer,
they might be better served if we make things pluggable enough that
it's easy for them to roll their own graphic presentation to replace ours.

  > - In the "internal.ui.model" package, I've added an example of a "VisualizerModel" object
  >    and nested classes for cpus, cores, and threads, which is used to capture a "snapshot"
  >    of the state we retrieve via DSF and hand it to the visualizer for later display.
  >    (The Todo class is used when constructing these snapshot models. More on this below.)

  I will need to get comfortable with the code, and I think the best way to do
  that is to add a feature.  So I'm thinking of starting with the dynamic
  updating we talked about, which should give me reasons to dive into that code.

Cool, and do let me know if the state object structure makes sense to you,
or if we need to tweak it.

  1- OnDoneMonitor.java and OnDataRequestDoneMonitor.java
  This class is meant to avoid a new cycle of the DSF executor by using the
  ImmediateExecutor. I actually use the ImmediateExecutor in most of my RMs so I
  definitly see your point. But the name 'OnDoneMonitor' does not feel very clear
  to me; I had to read the code to figure out what it was meant for. 
  How about 'ImmediateRequestMonitor' and 'ImmediateDataRequestMonitor'?

That makes sense. One aspect of these classes was wrapping up the very common
usage of ImmediateExecutor, which we should do since it cleans up the code.

Another issue I was having, though, was that the names of these Monitor claaes
(RequestMonitor, etc.) are somewhat long, and also biased towards what the
class _is_ in the DSF architecture rather than what it's used _for_ in code.
(It feels a little like calling an ArrayList a DataProcessingImpedenceAdapter.)
I wound up calling the helper class "OnDoneMonitor", etc. because that helped
me remember the intent (e.g. when the asynch operation or data retrieval
is done, here's what to do next).

  If these classes are worth implementing, then they belong in DSF directly.
  I'm not sure Pawel will see the value, but we can try to propose a patch if you
  feel it is worth the effort.  I can probably make a case for it, as we have had
  troubles with deeply nested calls to the executor through RMs.

I'm not wedded to these helper classes as written or as named; I put them in
to (a) help me learn DSF and (b) prompt us to think about how we might
name/factor the actual DSF classes better, as we're doing here.

  One thing though, OnDoneMonitor.handleCompleted() must either be removed or must
  call super.handleCompleted().

For now I'll add super.handleCompleted(), just so the method is still visible
in the source for that class.


  2- OnNthDoneMonitor.java
  This class has two aspects: the Immediate aspect, and calling setDoneCount from 
  the constructor.  For the Immediate aspect, if we suggest such a concept for
  DSF as discussed in the point above, we should probably apply it to all
  subclasses of RM (RequestMonitor, DataRequestMonitor, CountingRM, MultiRM, and
  RequestMonitorWithProgress).

Agreed -- as noted above I think this will help make the code cleaner
(assuming immediate usage like this is a common case).

  As for calling setDoneCount from the constructor, we should add a constructor
  to CountingRequestMonitor directly, if we feel it would be useful.

Also good, again as a means of shortening the code.


  3- DSFSessionState.java
  I also see the value of this class.  It makes the code much simpler for classes
  that use it.  I'm not a big fan of the name though...

It's a placeholder name, so I'm not attached to it. I wasn't sure of the scope
of the pieces it wraps up, so I just stuck in "State". I'm very cool with giving
it a better name if you have suggestions.

  Also, the fact that it mixes automatically using the executor for some calls
  (e.g., addServiceEventListener) but not others (e.g., getService()) seems
  risky to me.

I was mainly encapsulating common actions as I encountered them in the
multicore visualizer example. If we should consistently use the executor
for everything, that's fine by me.

  Also, it probably shouldn't have specific methods like
  getHardwareContext() and getHardwareService(), but have generic ones
  since we may later need a different service or context.

HardwareContext() seemed like the "outermost" context in this case, so there
needed to be some way to get that, but yes, if there's an appropriate way to
generalize this, we should.

  I think it is ok for a helper class for the MulticoreVisualizer specifically,
  since it helps.

Perhaps we need a base "DSFSessionMumble" class, and then the multicore session
class derives from it and overrides a "getTopLevelContext()" method to return
the hardware context.


  >   The part of this I'd really like you to review and comment on is the DSFDebugModel
  >   and DSFDebugModelListener classes. Here I tried to "rationalize" the DSF lookup code
  >   into a more "listener-like" callback pattern...

  I like how you can use the listener pattern to handle DSF calls with this. 
  However, being used to DSF, I actually found it harder to follow.  The mapping
  from, for example, DSFDebugModel.getCPUs() to getCPUsDone() is done in another
  class (DSFDebugModel), which requires the reader of the code to go see to
  confirm which method will be called once getCPUs() is called.

Did you mean to say the mapping is defined in the DSFDebugModelListener interface?
This is kinda the intent: the interface defines the set of methods that the
implementing class must implement, and documents when and why they are called;
though the names of the interface methods alone should make it somewhat obvious.

This is one reason I was troubled by the use of annotations to define the handlers
for DSF service events, because that meant I didn't have an inspectable interface
that captures the API behavior, so I couldn't be sure what I could/should implement.

  In fact, nothing would make a reader realize that getCPUs() is actually
  an asynchronous call, since it does not require an RM.

We could add "Async" to the method names, to call this out, and javadoc
that they perform an asynchronous operation. But listeners are in general
asychronous critters to begin with (at least in UIs), so this should be
well understood.


  Another problem is that it may happen that in some cases, when calling
  getCPUs() you want to do one thing, when done, while you'd prefer to do
  something else in another case; defining the RM directly inside the call to
  getCPUs() allows you to tailor the callback per situation.

Do you have an example in mind? We could provide alternative methods
that take RM classes to support this case. It still seems likely that
there's a common case where all the caller wants is the list of CPU
contexts and they'll then figure out what they want to do with them.
That's the usage the current methods are intended to cover.

  Also, by using a listener instead of an RM, it may make debugging more
  difficult
  if we forget to call the listener's callback method.  This is not new, because
  we can forget to call RM.done(), but now we'd have a second pattern to pay
  attention to for forgotten calls.

By "we" here, you mean us as in the implemeters of the DSFDebugModel?
By definition, we should have gotten it right, and I'd argue that it's
much easier for us to get it right once than it is for each and every consumer
of the DSF cpu/core/thread data to write the correct DSF code themselves.

  I think you could achieve a very similar effect by using this pattern:

  DSFDebugModel.getCPUs(new ImmediateRequestMonitor(null) {
      @Override
      protected void handleCompleted() {
	  getCPUsDone();
      }
  });

That does make sense, and that's likely the model to follow for the
alternative forms of these methods. This does tend to lead back to
the "mixed" form of anonymous classes and closure variables, though.
I'd argue the "common case" code should avoid that if possible.


  >   As a side note, I decided that the function of CountingRequestMonitor is better handled
  >   as an aspect of the model being constructed: that is, the DSFDebugModel call just returns
  >   a list of cores for a CPU, or a list of threads for a core. It doesn't care that a given call
  >   happens to be the one that "finishes" the model being constructed; instead, that should be
  >   tracked and handled by the listener method(s) that are actually constructing the model.
  >   (You can see an example of this in the getVisualizerModel() methods, where I use the
  >   Todo helper class to track the remaining work.) Again, what do you think? I'm new to
  >   DSF so I'm not sure whether the CountingRequestMonitor class has uses aside from
  >   model construction.

  It makes sense to me.

Wow, really? (Just kidding. :-) I'm glad I wasn't missing something important here.
Does the Todo helper class seem useful, or is there an existing mechanism for this?

  > - I've made a few supporting changes in the visualizer framework:
  >   - I added the concept of "decorations" to GraphicObjects, so CPUs can draw their IDs
  >      on top of the cores that get placed on top of them

  GraphicCanvas.java
  - in paintCanvas(), should the decoration be drawn right after
  the graphic object is drawn, instead of drawing all decorations at the end?
  So as to keep the decorations at the same z-ordering as their mathing graphics
  object?
  Or do we specifically want all decorations to be shown above everything else
  (as your last above sentence makes me think)?

Yes, the idea is that a CPU is drawn first, then its cores, and then we add
the CPU's decorations, which draws the cpu id in the lower right -- this insures
the cpu ID is drawn "on top of" the core(s), if they happen to overlap.
(Ideally they shouldn't, I'm just coding defensively against the case where
the window is small and/or the margins are changed so overlapping is more likely.)


  - in getGraphicObject(Class<?> type, int x, int y), we could use a ListIterator
  to traverse in reverse (not a big deal, I just thought it may be more
  efficient, although I' not sure)

The issue is that Iterators require allocation/GCing of an iterator instance,
so it can actually be more efficient to use indexes and get().

  - in getGraphicObjectFor(Object value), "if (gobj.getData() == value)" should
  use equals() instead of ==

Good point, though in this case I'm intentionally matching graphic objects
with specific "value" objects. If I use equals() then (a) I have to check
for NULL before calling equals() and (b) the value type needs to define
a unique equals().

  - getDataFor() does not seem to add a lot of value unless it is for
  inheritance?
    It could be made static as well, although that might not be a good idea for
  inheritance.

My habit is to always include the "opposite" operation, even if it's not immediately
used, unless there's a reason to omit it (setters for getters, except for "constant"
properties), because it's usually not long until it suddenly becomes necessary. :-)

  >   - I added a Colors resource manager for SWT Color objects
  >   - I added a Timer convenience wrapper for the Swing Timer class
  >     (maybe I should use the SWT timer interface instead? I didn't bother rewriting this class
  >      since I already had it)
  >   Some of this stuff feels like it wants to be in a general utility library somwhere;
  >   I just added it to the Visualizer framework's utils package for now.

  I have not experience here, but it does look like there would be some
  supporting methods in SWT or Eclipse for these things. But maybe not.
  Let's go with what you feel is best.

From doing the Tilera IDE I know we need a Color manager of some kind.
For the rest, if you're not "oh yeah, _obviously_ you should do X"
it's probably okay as is for now.


  > - I also made one tweak to the CoreList.getCoreList() method in cdt.dsf.gdb.internal,
  >   so that it returns core IDs that are unique across all cores. This may not be the
  >   right way to do this (it would be best if the core IDs matched the Linux cpu IDs
  >   assigned to them), but it at least allows me to show the proper layout of threads
  >   in the visualizer.

  I'm not sure the algorithm you used is correct if you have a different number of 
  cores on the different CPUs.  For example if you have 4 cores on CPU 0 and 2
  cores on CPU 1, the 6 cores will have absoluteCoreID of: 0, 1, 2, 3, 2, 3.  As
  you can see the numbers overlap.

Good point, I _was_ presuming homogenous CPUs; they might not be.

  Could we simply use a global counter that increments by one every time we find
  a new core?

Weeeelll, the question here is, is there an "official" Linux ID for cpus,
the same as there is for cores? If so, we should use that, since that's what users
will be expecting. Same with core IDs, and process/thread IDs; as much as possible
we should reflect reality. Only if there isn't such a concept should we resort
to counter-based id's.

  > A few "new user" comments on the DSF API:
  >   - I'm still not quite clear on the DSF hierarchy for processes/threads;
  >     the extra "container" and "executable" types make it confusing how
  >     DSF really represents processes vs. threads. 

  This aspect has been a thorn since the start, and it only in the DSF-GDB
  implementation. DSF simply defines the different contexts, and it is up
  the implementation to choose how to organize the hierarchy.

  ...

This description helps a bit, in particular working from the specific hierarchy
of debugger/process/thread to the DMC objects, rather than vice-versa.
It's understood that there may be exceptions, but it's good to start with
"here's how you represent this well-known case" to make things clear.


  >     (In particular, I'm not sure whether
  >     the current DSFDebugModel.getThreads() method can -- or should -- be broken
  >     out into separate getProcesses() and getThreadsForProcess() calls, for example.)

  Probably would be easier to read and maintain.  In fact, we probably will need
  a way to get the list of processes in the future, so might as well have it
  available.

Do you want to write that, or should I?

  >   - It really feels wrong to be using annotations to define the methods to be called on
  >     service event handler instances. This is kinda the canonical example of what interfaces
  >     are used for, and using annotations means one can't use Eclipse's own class hierarchy
  >     inspection tools to discover the methods on an interface and/or verify that an object
  >     properly implements the methods it should.

  Yes, I guess there are drawbacks to that approach.  However, it is quite
  similar to the listener approach.

Annotations are really meant for things like automated code generation (like J2EE),
where one is generating boilerplate code alongside the hard-coded stuff. Using them
to express code logic carries some of the same risks as using macros to generate
complex code in C/C++: if something breaks, it's harder to understand why, because
the "normal" compilation and checking mechanisms have been subverted.

  > Otherwise, the DSF framework works really well, and I'm able to display an application
  > running on a quad-core Linux box here. I've even worked out the visualizer display code
  > for arbitrary numbers of cores/cpus, if we have the chance to run on anything larger.
  > (That was a struggle, let me tell you. It's not as trivial as it looks getting the cpus and
  >  cores to efficiently use the available screen space!)

  I think that effort has a lot of value.  Being able to tell the community and
  present at EclipseCon a Multicore Visualizer that can work with an arbitrary
  numbers of cores/cpus, will make people pay attention.
  Thanks for taking the time to get it working like that!

I suppose working for a company with a chip with dozens of cores on it
had something to do with it... :-)

  > Anyway, have a look at the checked-in code, and let me know if you have any
  > comments/suggestions. If this all looks fine we can merge the review branch
  > to the visualizer branch. Actually, there are no pending changes on the visualizer branch,
  > so we should be able to just do an inlining merge, right?

  When you want to push those change to the Visualizer branch, you can set the
  Visualizer branch, then rebase the review_ui branch onto it.

Cool -- I'll make a few tweaks as indicated by your comments, push that to
the new_ui branch, and then rebase onto the visualizer branch. We can deal
with other comments here as we go.

(I'll let you know before I do the rebase, just in case.)

  One concern that I have about the code is that it does not follow the same
  coding style as the other two dsf.gdb plugins.  And that is a tricky thing
  since you have your coding style and I have mine.  What I suggest is that we
  try to have dsf.gdb.multicorevisualizer follow the pattern of the other two
  dsf.gdb plugins, but let the two cdt.visualizer plugins follow whichever style
  your are more comfortable with.

Question: is there a standard CDT naming convention I should be following here?
It'll save time if I don't have to go back and rename stuff later.

  I'm not asking that you change any code for this.  But I would like your
  persmission for me to modify the dsf.gdb.multicorevisualizer plugin ss I start
  coding in it, so that it matches the other two.  One example is using
  fMemberVariable (f for 'field', which was how DSF call such variables, and we
  just continued) instead of m_memberVariable.

  What do you think?

Sure, we should follow the "standard", if one exists.

  I'm going to start looking at the dynamic update feature right away and get my
  hands dirty again.

Cool, welcome back to the pool! (Mind the sharks at the deep end... ;-)

Comment 40 Marc Khouzam 2011-12-07 13:18:38 EST

(In reply to comment #37)
> (In reply to comment #35)
> > Poor display of 1000 threads
> 
> As I mentioned in the conf call a while back, we don't normally
> have to worry about that case, because having a gazillion
> threads on the same core is usually not an effective use
> of processor resources. One should be using a master/worker
> pattern, or thread pooling, etc. so that ideally one or at most
> a few threads are running on each core.
> 
> This hasn't come up as an issue for the Grid View because
> developers tend to be savvy about multicore design, but
> you're right that people running generic code in the visualizer
> are more likely to try "let see if it goes to eleven"
> kinds of programs, which would raise these kinds of issues.
> 
> We can modify the visualizer so that if more than
> a displayable number of threads are on a given core, we only draw
> a few, then a "..." indication, or otherwise indicate there are
> "An Awful Lot of Threads Here". (In a tooltip, or other mechanism,
> we can show the individual thread IDs, if the list isn't
> ridiculously long, and if it is, we use "..." again.)
> 
> I wouldn't go overboard trying to squeeze in lots of threads,
> because as noted above, this case should be the exception
> rather than the rule. (And in fact, having a lot of threads
> crowded on one tile is a good visual indicator of imbalanced
> code, so you don't _want_ it to look too neat and tidy! :-)

Agreed.

This bring up an administrative point.  How do we track such changes/decisions?  We won't always be able to do them right away, but leaving them deep in a comment of this bug pretty much ensures we won't remember to do it after a while.

I suggest we start opening sub-bugs for smaller Visualizer tasks.  It will help with the tracking.  So, I have opened Bug 365942 to track this issue.

Comment 41 Marc Khouzam 2011-12-07 13:50:33 EST

(In reply to comment #38)
> (In reply to comment #36)
> > 
> > I think we should define the [Visualizer] view in plugin.xml of the
> > cdt.visualizer.ui plugin.
>
> Makes sense -- the visualizer view is intended to be shared by all
> the defined visualizers, ours and anyone else's. One could even argue
> that at some point it might have to move to an Eclipse platform plugin,
> because visualizations aren't necessarily restricted to CDT or debugging.
> But we're assuming the boundary is CDT for now, so cdt.visualizer.ui
> sounds okay.

I pushed the update to the Visualizer branch.

Comment 42 Marc Khouzam 2011-12-07 15:08:45 EST

(In reply to comment #39)

> I generally do have constants at least partly pulled out as named locals
> or class constants, for self-documentation, but certainly we can extract
> them further into a "constants" class or the like.
> 
> I'd be cautious about trying to overparameterize the drawing code.
> Tweaking colors/fonts, etc. is one thing, but trying to make it fully
> customizable is probably out of scope here. And you know someone will always
> demand the One More Thing. ("Hey, why can't I make this line here
> exactly 3.2 picas, dot-dot-dashed, and... and... MAUVE, HUH?" :-)


Good point.  I think it would be safer for us to have the ability to
easily tweek the graphics though.  It is going to be annoying for us
if we want to try make a line thicker and have to go through the code
to find the right places to change it.  So, let's try to keep all
contant globals for ourselves.

> If someone really wants to do a complete visual overhaul of the visualizer,
> they might be better served if we make things pluggable enough that
> it's easy for them to roll their own graphic presentation to replace ours.

Actually, for a total overhaul, they can just copy the whole code and change
it themselves.  Easier for us :-)

>   1- OnDoneMonitor.java and OnDataRequestDoneMonitor.java
>   This class is meant to avoid a new cycle of the DSF executor by using the
>   ImmediateExecutor. I actually use the ImmediateExecutor in most of my RMs so
> I
>   definitly see your point. But the name 'OnDoneMonitor' does not feel very
> clear
>   to me; I had to read the code to figure out what it was meant for. 
>   How about 'ImmediateRequestMonitor' and 'ImmediateDataRequestMonitor'?
> 
> That makes sense. One aspect of these classes was wrapping up the very common
> usage of ImmediateExecutor, which we should do since it cleans up the code.

I'll write a new class for DSF and propose it.

> Another issue I was having, though, was that the names of these Monitor claaes
> (RequestMonitor, etc.) are somewhat long, and also biased towards what the
> class _is_ in the DSF architecture rather than what it's used _for_ in code.
> (It feels a little like calling an ArrayList a DataProcessingImpedenceAdapter.)
> I wound up calling the helper class "OnDoneMonitor", etc. because that helped
> me remember the intent (e.g. when the asynch operation or data retrieval
> is done, here's what to do next).

The 'RequestMonitor' class used to be called 'Done' :-) so I see your point.
Let's stick to the current naming scheme since it's there; it won't be long
before you are totally comfortable with them.

>   2- OnNthDoneMonitor.java
>   This class has two aspects: the Immediate aspect, and calling setDoneCount
> from 
>   the constructor.  For the Immediate aspect, if we suggest such a concept for
>   DSF as discussed in the point above, we should probably apply it to all
>   subclasses of RM (RequestMonitor, DataRequestMonitor, CountingRM, MultiRM,
> and
>   RequestMonitorWithProgress).
> 
> Agreed -- as noted above I think this will help make the code cleaner
> (assuming immediate usage like this is a common case).
> 
>   As for calling setDoneCount from the constructor, we should add a constructor
>   to CountingRequestMonitor directly, if we feel it would be useful.
> 
> Also good, again as a means of shortening the code.

I'll take care of that in DSF.

>   >   The part of this I'd really like you to review and comment on is the
> DSFDebugModel
>   >   and DSFDebugModelListener classes. Here I tried to "rationalize" the DSF
> lookup code
>   >   into a more "listener-like" callback pattern...
> 
>   I like how you can use the listener pattern to handle DSF calls with this. 
>   However, being used to DSF, I actually found it harder to follow.  The
> mapping
>   from, for example, DSFDebugModel.getCPUs() to getCPUsDone() is done in
> another
>   class (DSFDebugModel), which requires the reader of the code to go see to
>   confirm which method will be called once getCPUs() is called.
> 
> Did you mean to say the mapping is defined in the DSFDebugModelListener
> interface?

No, I really meant DSFDebugModel :-)
I guess the problem I'm having is that MulticoreVisualizer passes itself ('this')
when calling e.g., DSFDebugModel.getCPUs(), it is not obvious that
MulticoreVisualizer.getCPUsDone() is the callback.  Maybe simply having a
dedicated class MutlicoreVisualizerListener would make things more readable.

> This is one reason I was troubled by the use of annotations to define the
> handlers
> for DSF service events, because that meant I didn't have an inspectable
> interface
> that captures the API behavior, so I couldn't be sure what I could/should
> implement.

In this case the API, althouth not obvious at first sight, is relatively
simply, which is probably why it was never a problem.  You just create
a method to receive DSF events and you put the type of events as a method
parameter; finally, you add the annotation and register with the session.
I think using an interface would be difficult because DSF events are
extended by others and an interface couldn't be prepared for all those
types.  

I believe Pawel said the annotation solution was the one that gave
the most flexibility.  But we'd have to ask him to be sure.

>   In fact, nothing would make a reader realize that getCPUs() is actually
>   an asynchronous call, since it does not require an RM.
> 
> We could add "Async" to the method names, to call this out, and javadoc
> that they perform an asynchronous operation. But listeners are in general
> asychronous critters to begin with (at least in UIs), so this should be
> well understood.

I haven't used listeners all that much.  I do see they are async.  However
are they usually used to listen for the result of an async method?
I've seen them used to listen for events.  I see the two things as different.
When one calls a method, we normally expect a sync return value.  With DSF
we don't have that, but we have an async result in one of the params.
The platform uses that pattern as well.   It uses the done() pattern
as well.
Registering a listener for an event that can happen sometime in the future
is similar but still different.

>   Another problem is that it may happen that in some cases, when calling
>   getCPUs() you want to do one thing, when done, while you'd prefer to do
>   something else in another case; defining the RM directly inside the call to
>   getCPUs() allows you to tailor the callback per situation.
> 
> Do you have an example in mind? We could provide alternative methods
> that take RM classes to support this case. It still seems likely that
> there's a common case where all the caller wants is the list of CPU
> contexts and they'll then figure out what they want to do with them.
> That's the usage the current methods are intended to cover.

In this case, my comment may not apply directly, but I've seen on multiple
occasions in DSF that some async methods are called at different points
of the code and the RM they are passed does something different.  So I was
making a more general comment.

>   Also, by using a listener instead of an RM, it may make debugging more
>   difficult
>   if we forget to call the listener's callback method.  This is not new,
> because
>   we can forget to call RM.done(), but now we'd have a second pattern to pay
>   attention to for forgotten calls.
> 
> By "we" here, you mean us as in the implemeters of the DSFDebugModel?
> By definition, we should have gotten it right, and I'd argue that it's
> much easier for us to get it right once than it is for each and every consumer
> of the DSF cpu/core/thread data to write the correct DSF code themselves.

We may not be as comfortable in a couple of years, when a bug come in :-)
As for consumers, they will need to learn the DSF way of coding (which, yes,
is complex), so I think we can't do much here to save them :-(

>   I think you could achieve a very similar effect by using this pattern:
> 
>   DSFDebugModel.getCPUs(new ImmediateRequestMonitor(null) {
>       @Override
>       protected void handleCompleted() {
>       getCPUsDone();
>       }
>   });
> 
> That does make sense, and that's likely the model to follow for the
> alternative forms of these methods. This does tend to lead back to
> the "mixed" form of anonymous classes and closure variables, though.
> I'd argue the "common case" code should avoid that if possible.

I guess I'm so used to it that it does not bother me.
Let's leave things as you have them for now if you are more comfortable
like that.

>   >   As a side note, I decided that the function of CountingRequestMonitor is
> better handled
>   >   as an aspect of the model being constructed: that is, the DSFDebugModel
> call just returns
>   >   a list of cores for a CPU, or a list of threads for a core. It doesn't
> care that a given call
>   >   happens to be the one that "finishes" the model being constructed;
> instead, that should be
>   >   tracked and handled by the listener method(s) that are actually
> constructing the model.
>   >   (You can see an example of this in the getVisualizerModel() methods,
> where I use the
>   >   Todo helper class to track the remaining work.) Again, what do you think?
> I'm new to
>   >   DSF so I'm not sure whether the CountingRequestMonitor class has uses
> aside from
>   >   model construction.
> 
>   It makes sense to me.
> 
> Wow, really? (Just kidding. :-) I'm glad I wasn't missing something important
> here.
> Does the Todo helper class seem useful, or is there an existing mechanism for
> this?

Ok, you caught me :-)  The explanation you wrote above made sense to me
but I didn't dwell on the code part too much :-O
I'll have a closer look as I get deeper into the dynamic update feature.

>   Could we simply use a global counter that increments by one every time we
> find
>   a new core?
> 
> Weeeelll, the question here is, is there an "official" Linux ID for cpus,
> the same as there is for cores? If so, we should use that, since that's what
> users
> will be expecting. Same with core IDs, and process/thread IDs; as much as
> possible
> we should reflect reality. Only if there isn't such a concept should we resort
> to counter-based id's.

Looking at /proc/cpuinfo, I don't think there is a global numbering scheme
on Linux.  But you are right that a global counter is not very safe and could
be hard to keep consitent.  We can do the math using /proc/cpuinfo to unikely
number the cores.

>   >     (In particular, I'm not sure whether
>   >     the current DSFDebugModel.getThreads() method can -- or should -- be
> broken
>   >     out into separate getProcesses() and getThreadsForProcess() calls, for
> example.)
> 
>   Probably would be easier to read and maintain.  In fact, we probably will
> need
>   a way to get the list of processes in the future, so might as well have it
>   available.
> 
> Do you want to write that, or should I?

I'll deal with the proposed DSF Immediate classes mentione above, so feel
free to deal with this change if that is ok with you.

> Question: is there a standard CDT naming convention I should be following here?
> It'll save time if I don't have to go back and rename stuff later.

I found this:
http://wiki.eclipse.org/CDT/policy#Code_Formatting
http://wiki.eclipse.org/Naming_Conventions

but it does not mentions a rule for fields/members.  I usually keep
the format of the existing plugin.  For new plugins, I think you are 
free to do what you prefer.

Comment 43 Marc Khouzam 2011-12-08 11:18:13 EST

> > - dynamic (event-based and/or periodic) updating [you/me?]
> 
> Ah yes, that is something I can do easily with DSF events.
> I'll take care of that.

When a thread gets created, do we want to update the entire Model, or do we want to support a specific update of the display for that one thread?
A full update is very easy to implement, but is that too expensive?

Comment 44 William Swanson 2011-12-08 11:32:57 EST

(In reply to comment #43)

> When a thread gets created, do we want to update the entire Model, or do we
> want to support a specific update of the display for that one thread?
> A full update is very easy to implement, but is that too expensive?

Just to be clear, we're talking about updating the model objects rather
than updating the Visualizer display of them?  The latter happens periodically
and relatively infrequently so it's okay for the view to rewalk the model.

I'd say update the model dynamically, unless it proves monumentally unweildy,
because we're also going to want to dynamically capture things like changes
to thread location/state, etc. and we don't want to regenerate
the entire model whenever those happen.

Comment 45 Marc Khouzam 2011-12-08 11:43:18 EST

(In reply to comment #44)
> (In reply to comment #43)
> 
> > When a thread gets created, do we want to update the entire Model, or do we
> > want to support a specific update of the display for that one thread?
> > A full update is very easy to implement, but is that too expensive?
> 
> Just to be clear, we're talking about updating the model objects rather
> than updating the Visualizer display of them?  The latter happens periodically
> and relatively infrequently so it's okay for the view to rewalk the model.
> 
> I'd say update the model dynamically, unless it proves monumentally unweildy,
> because we're also going to want to dynamically capture things like changes
> to thread location/state, etc. and we don't want to regenerate
> the entire model whenever those happen.

Ok, I'll try to update only the part of the model that changes.

Comment 46 Marc Khouzam 2011-12-08 16:04:28 EST

(In reply to comment #45)
> > Just to be clear, we're talking about updating the model objects rather
> > than updating the Visualizer display of them?  The latter happens periodically
> > and relatively infrequently so it's okay for the view to rewalk the model.

I'm trying to figure out when the visualizer display currently refreshes?  Is there a periodic refresh or is it only when the selection changes?

I have code that update the existing model each time a thread is created, but I don't see it reflected in the visualizer unless I cause a selection change in the debug view.

If there is not periodic refresh (which we may not need, but you had mentioned something like that once I believe), then we need a way to kick the visualizer when the model changes.

Thanks for any clarification.

Comment 47 William Swanson 2011-12-09 11:44:37 EST

(In reply to comment #46)
> 
> I'm trying to figure out when the visualizer display currently refreshes?  Is
> there a periodic refresh or is it only when the selection changes?

> I have code that update the existing model each time a thread is created,
> but I don't see it reflected in the visualizer unless I cause a selection
> change in the debug view.

There is a requestUpdate() methdod on MulticoreVisualizerCanvas,
which sets a timer (unless it's already been set), which then updates
the canvas from it's currently set model (i.e. the last thing passed
into the setModel() method, after a delay of about a 10th of a second.
This allows multiple repaint requests to be "batched" to cut down on
redraw costs. (In other words, when updating the model in response to
dynamic events, you should not call update() directly; you should call
requestUpdate() to get this batching behavior.)

So if you're just tweaking the content of the VisualizerModel that was
last handed to the canvas, you may just need to do a requestUpdate()
call on the canvas.

Note that when the selection changes in the Debug View, if the selected
object is mappable to a VisualizerModel by MulticoreVisualizer,
then setModel() will get called on the canvas, which amongst other things
causes a requestUpdate() -- this is why a selection change in the
Debug View causes an update of the visualizer.

As a side note, something else that's needed is to ensure that the
initial selection in the Debug View when a debugged launch starts
is mappable to a displayable model object. (In Tilera's Grid View,
for example, I'm able to map from our custom launch object, etc.
to the model to display, so basically anything selectable in the
Debug View is also displayable in the Visualizer.)

Comment 48 Marc Khouzam 2011-12-12 11:56:01 EST

(In reply to comment #47)

> So if you're just tweaking the content of the VisualizerModel that was
> last handed to the canvas, you may just need to do a requestUpdate()
> call on the canvas.

Perfect, works great!

> Note that when the selection changes in the Debug View, if the selected
> object is mappable to a VisualizerModel by MulticoreVisualizer,
> then setModel() will get called on the canvas, which amongst other things
> causes a requestUpdate() -- this is why a selection change in the
> Debug View causes an update of the visualizer.

Ah yes, I see it now.

> As a side note, something else that's needed is to ensure that the
> initial selection in the Debug View when a debugged launch starts
> is mappable to a displayable model object. (In Tilera's Grid View,
> for example, I'm able to map from our custom launch object, etc.
> to the model to display, so basically anything selectable in the
> Debug View is also displayable in the Visualizer.)

The entries that won't work right now are the GdbLaunch node and the gdb node.  I'll look into it.

Comment 49 Marc Khouzam 2011-12-12 16:24:07 EST

Someone suggested to me that we support visualizing Linux cgroups:
http://en.wikipedia.org/wiki/Cgroups

What do you think?  I haven't dug into the details, but I could open a new bugzilla if it makes sense.

Comment 50 William Swanson 2011-12-12 16:42:31 EST

(In reply to comment #49)
> Someone suggested to me that we support visualizing Linux cgroups:
> http://en.wikipedia.org/wiki/Cgroups
> 
> What do you think?  I haven't dug into the details, but I could open a new
> bugzilla if it makes sense.

Hmmm... this kinda feels more like overlay territory, in that it's not
an integral part of the app itself, but rather a resource management mechanism.
One might similarly argue for visualizing other kinds of system-level associations like thread groups, etc.

I don't have a hard-and-fast metric for what belongs in the "base" multicore visualization, but as a rough rule it should just be the processes/threads,
their execution state, and other immediate stuff that's useful for an
"at a glance" view of the base state.

We could then have overlays for such things as:
- resource boundaries, like Cgroups, imposed on the app from the "outside"
- literal boundaries, like the process/thread tree making up each application
- semantic boundaries, like the set of worker processes for each master, etc.

Not saying we shouldn't do things like this, just thinking we should put
together a list of possibilities and figure out which want to become overlays,
in useful categories like those above.

Comment 51 Marc Khouzam 2011-12-13 04:56:24 EST

(In reply to comment #50)
> > http://en.wikipedia.org/wiki/Cgroups
> 
> Hmmm... this kinda feels more like overlay territory,

That is fine with me.  I love the overlay concept :-)
I'll open a bug to track the idea, but it does not mean it is something that needs to be done soon; it is just so that we publicize the idea.

Comment 52 Marc Khouzam 2011-12-13 09:12:24 EST

(In reply to comment #51)
> (In reply to comment #50)
> > > http://en.wikipedia.org/wiki/Cgroups
> > 
> > Hmmm... this kinda feels more like overlay territory,
> 
> That is fine with me.  I love the overlay concept :-)
> I'll open a bug to track the idea, but it does not mean it is something that
> needs to be done soon; it is just so that we publicize the idea.

I opened Bug 366537

Comment 53 Marc Khouzam 2011-12-13 10:55:25 EST

I've just pushed a new branch to GitHub: marc_changes_visualizer.
It is meant to contain proposed changes to the visualizer branch.

Currently there is a commit to enable dynamic updating of the Multicore Visualizer for thread creation and thread exits.

The main point is that I factored the @DsfServiceEventHandler methods out of MulticoreVisualizer and added them into a new class MulticoreVisualizerEventListener.  That new class is meant to keep all the event logic contained (although there isn't so much code right now).

Are there other events we want to listen for right now?  I think that Running/Suspended events will be interesting once we show thread state, but for now, not so much.

Bill, if it looks good to you, I can merge it into the Visualizer branch.

Comment 54 William Swanson 2011-12-13 14:13:48 EST

(In reply to comment #53)
> I've just pushed a new branch to GitHub: marc_changes_visualizer.
> It is meant to contain proposed changes to the visualizer branch.
>
> ...
> 
> Bill, if it looks good to you, I can merge it into the Visualizer branch.

This looks okay to merge.

Comment 55 Marc Khouzam 2011-12-13 15:20:30 EST

(In reply to comment #54)

> This looks okay to merge.

Thanks
I pushed to the visualizer branch

Comment 56 Marc Khouzam 2011-12-13 16:40:15 EST

(In reply to comment #48)

> > As a side note, something else that's needed is to ensure that the
> > initial selection in the Debug View when a debugged launch starts
> > is mappable to a displayable model object. (In Tilera's Grid View,
> > for example, I'm able to map from our custom launch object, etc.
> > to the model to display, so basically anything selectable in the
> > Debug View is also displayable in the Visualizer.)
> 
> The entries that won't work right now are the GdbLaunch node and the gdb node. 
> I'll look into it.

I pushed a changed to marc_changes_visualizer which allows to display the MulticoreV content even when the launch or gdb nodes are selected.

I had to make some changes to DSFSessionState because it use to require a DMContext, while the launch and gdb nodes don't have one.

Next, I'll try to address:
- displaying current execution state via colors

I'm thinking of:
- running thread => green
- stopped thread => yellow
- crash thread => red

or should we go for green/red and keep the crashing case, which is more rare for some other color, maybe black?

You mentioned that you color the actual cores.  What is the rule you use?  Something like:
- all threads running on core => core running (green?)
- one or more threads running on core => core suspended (yellow?)
- thread crash on core  => core crash (red?)

Comment 57 William Swanson 2011-12-13 16:53:47 EST

(In reply to comment #56)
> I pushed a changed to marc_changes_visualizer which allows to display the
> MulticoreV content even when the launch or gdb nodes are selected.

Cool -- thanks, Marc!

> I had to make some changes to DSFSessionState because it use to require a
> DMContext, while the launch and gdb nodes don't have one.
> 
> Next, I'll try to address:
> - displaying current execution state via colors
> 
> I'm thinking of:
> - running thread => green
> - stopped thread => yellow
> - crash thread => red

This is what we do in the Grid View, and it still makes sense.
After all, being suspended isn't "bad", it just means action is needed
to continue. Crashing, on the other hand, is "bad".

We should also be good about accessibility, and make sure the green/red
hues have different brightnesses, etc. (In the Grid View, I use a
bright green with dark text, and a dark red with light text,
so it's clear there's a difference beyond the color.)

> You mentioned that you color the actual cores.  What is the rule you use? 
> Something like:
> - all threads running on core => core running (green?)
> - one or more threads running on core => core suspended (yellow?)
                        ^^^^^^^ suspended
> - thread crash on core  => core crash (red?)

Basically, yes. We use an "all stop" debugging model where a
single process suspended suspends everything, and resuming resumes
everything, but if we're on a platform where some processes
can be suspended and others running, we should reflect that.

Comment 58 Marc Khouzam 2011-12-14 10:09:10 EST

(In reply to comment #57)
> (In reply to comment #56)
> > I pushed a changed to marc_changes_visualizer which allows to display the
> > MulticoreV content even when the launch or gdb nodes are selected.
> 
> Cool -- thanks, Marc!

I've pushed the change to the Visualizer view.

> > I'm thinking of:
> > - running thread => green
> > - stopped thread => yellow
> > - crash thread => red
> 
> This is what we do in the Grid View, and it still makes sense.
> After all, being suspended isn't "bad", it just means action is needed
> to continue. Crashing, on the other hand, is "bad".

Do you have code to re-use for this?  Or do you expect that only minor tweaks will be necessary?

> We should also be good about accessibility, and make sure the green/red
> hues have different brightnesses, etc. (In the Grid View, I use a
> bright green with dark text, and a dark red with light text,
> so it's clear there's a difference beyond the color.)

You mean for color blind people, for example?  That is a good point.

> > You mentioned that you color the actual cores.  What is the rule you use? 
> > Something like:
> > - all threads running on core => core running (green?)
> > - one or more threads running on core => core suspended (yellow?)
>                         ^^^^^^^ suspended

Woops :-)

> > - thread crash on core  => core crash (red?)
> 
> Basically, yes. We use an "all stop" debugging model where a
> single process suspended suspends everything, and resuming resumes
> everything, but if we're on a platform where some processes
> can be suspended and others running, we should reflect that.

With GDB we have both all-stop and non-stop debugging.  I always tend to favor the non-stop mode, which is what Ericsson uses.  But we do keep supporting all-stop.

Comment 59 Marc Khouzam 2011-12-14 10:09:39 EST

(In reply to comment #58)
> I've pushed the change to the Visualizer view.

I meant 'the Visualizer branch'.  sorry

Comment 60 William Swanson 2011-12-14 12:07:45 EST

(In reply to comment #58)
 
> > > I'm thinking of:
> > > - running thread => green
> > > - stopped thread => yellow
> > > - crash thread => red
> > 
> > This is what we do in the Grid View, and it still makes sense.
> > After all, being suspended isn't "bad", it just means action is needed
> > to continue. Crashing, on the other hand, is "bad".
> 
> Do you have code to re-use for this?  Or do you expect that only minor tweaks
> will be necessary?

It's not a big deal, we just add a "state" attribute to the Visualizer's
model objects for processes/threads, set it when we get new state from the
DSF framework, and add drawing code to the corresponding canvas objects
that set the color accordingly.

> > We should also be good about accessibility, and make sure the green/red
> > hues have different brightnesses, etc. (In the Grid View, I use a
> > bright green with dark text, and a dark red with light text,
> > so it's clear there's a difference beyond the color.)
> 
> You mean for color blind people, for example?  That is a good point.

Correct, the general rule is that information conveyed by color should
also be visible without color. Color-blindness is one reason why.

Comment 61 Marc Khouzam 2011-12-14 12:11:05 EST

(In reply to comment #60)

> It's not a big deal, we just add a "state" attribute to the Visualizer's
> model objects for processes/threads, set it when we get new state from the
> DSF framework, and add drawing code to the corresponding canvas objects
> that set the color accordingly.

That is exactly what I was doing currently.  I'm glad it is what you had in mind.

> > > We should also be good about accessibility, and make sure the green/red
> > > hues have different brightnesses, etc. (In the Grid View, I use a
> > > bright green with dark text, and a dark red with light text,
> > > so it's clear there's a difference beyond the color.)
> > 
> > You mean for color blind people, for example?  That is a good point.
> 
> Correct, the general rule is that information conveyed by color should
> also be visible without color. Color-blindness is one reason why.

I know the platform group pays a lot of attention to that.

Comment 62 Marc Khouzam 2011-12-14 13:54:38 EST

(In reply to comment #61)
> (In reply to comment #60)
> 
> > It's not a big deal, we just add a "state" attribute to the Visualizer's
> > model objects for processes/threads, set it when we get new state from the
> > DSF framework, and add drawing code to the corresponding canvas objects
> > that set the color accordingly.

I pushed an update to marc_changes_visualizer that shows thread state using green for running and yellow for suspended.  It also adds the DSF events to update the color when the state changes.  I'll try the same thing for the entire core, to see if it looks good or not.

Comment 63 Marc Khouzam 2011-12-14 15:36:08 EST

Created attachment 208404 [details]
Screenshot showing state for threads and cores

I added some coloring for state for cores.  I'm not sure if it is desirable or not.  Attached is a screenshot.  The color of the core is based on the state of its threads.

I pushed it as a new commit to marc_visualizer_changes.

I think the next thing I will look at is:
- using actual cpu numbers, process/thread ids, etc.
- maybe a proposal of how to handle layers.

I was thinking that with the proposed feature from Bill:
- selection
- run control operations
- sync with Debug view

we will be in a pretty good position, feature-wise, for the EclipseCon Demo.  We may want to spend some time polishing things, testing and examining the user experience.  Thoughts?

Comment 64 William Swanson 2011-12-15 10:34:34 EST

(In reply to comment #63)
> I added some coloring for state for cores.  I'm not sure if it is desirable or
> not.  Attached is a screenshot.  The color of the core is based on the state of
> its threads.

That looks really good -- I have no problems with it.

Comment 65 Marc Khouzam 2011-12-16 14:57:29 EST

(In reply to comment #64)
> (In reply to comment #63)
> > I added some coloring for state for cores.  I'm not sure if it is desirable or
> > not.  Attached is a screenshot.  The color of the core is based on the state of
> > its threads.
> 
> That looks really good -- I have no problems with it.

I just pushed a bunch of things to the visualizer branch.  There is now state coloring for the threads, including crashed state.  I also did a couple of bug fixes on the branch as well.

I know there are some obscure bugs, but we'll have time to address those before EclipseCon.

Comment 66 William Swanson 2012-01-05 12:53:47 EST

I've checked in support for click/drag selection of threads
in the Multicore visualizer, along with a few other tweaks.

Note that the visualizer selection is not yet connected
to the Debug View selection. That's the next step.

Comment 67 William Swanson 2012-01-09 16:44:08 EST

I've checked in support for synchronizing the Visualizer canvas selection
with the Debug View selection. Selecting threads in the canvas now selects
the corresponding threads in the Debug View (or the topmost stack frame,
if stack frames are currently visible for the thread).

One potential issue, not with selection but with the debug actions, is that
the run/suspend/step, etc. actions are currently enabled only when a single
debuggable thing (process, thread, etc.) is selected. If one selects multiple
items in the visualizer, multiple items are selected in the Debug view, and
the debug actions are disabled. It would be better if when multiple debuggable
items are selected, the action clicked (run, step, etc.) could be applied to
each of them in turn (in an arbitrary order determined by the implementation).

Comment 68 Marc Khouzam 2012-01-10 09:41:21 EST

(In reply to comment #67)
> I've checked in support for synchronizing the Visualizer canvas selection
> with the Debug View selection. Selecting threads in the canvas now selects
> the corresponding threads in the Debug View (or the topmost stack frame,
> if stack frames are currently visible for the thread).

Awesome!
I also noticed that when selecting the process context, all threads of that process get selected.  Nice!

I have to say, this takes the Multicore-Visualizer to a whole new level!

> One potential issue, not with selection but with the debug actions, is that
> the run/suspend/step, etc. actions are currently enabled only when a single
> debuggable thing (process, thread, etc.) is selected. If one selects multiple
> items in the visualizer, multiple items are selected in the Debug view, and
> the debug actions are disabled. It would be better if when multiple debuggable
> items are selected, the action clicked (run, step, etc.) could be applied to
> each of them in turn (in an arbitrary order determined by the implementation).

Yes, only being able to perform actions on a single thread feels very limiting when using the Multicore visualizer. We have Bug 330974 opened about this, and Dobrin has mentioned some interest in it.  I believe he even started working on it.  We should ask him about it.  I don't believe this will be too hard.

Using the new feature quickly, the one thing I felt was missing was the context menu.  For some reason, I felt like it was more natural to right-click on my multicore visualizer selection to perform the Resume/Suspend.  If I remember correctly, this is part of the planned features, right?

Nice work Bill!

Comment 69 William Swanson 2012-01-12 13:23:17 EST

I've checked in a first cut of toolbar/context-menu support for the visualizer.

I was able to re-use the action classes for the Debug View, and since they're
initialized from the Debug View, they update/enable appropriately when the
Debug View's context changes. I also added "Select All" and "Refresh" commands
to the visualizer.

Comment 70 William Swanson 2012-01-17 12:37:28 EST

I've checked in a change to the multicore visualizer that modifies
the painting of the thread "dots". For any process(es) that have one
or more threads selected in the visualizer, we now draw all their threads
with a slightly brighter hue. Threads belonging to processes which are
not part of any selection are subtly darker. The intent is to subtly
emphasize threads that are part of the "currently selected process(es)".

(We'll give this a try, and see if it works, or if it's too subtle. :-)

Comment 71 Marc Khouzam 2012-01-17 13:58:25 EST

(In reply to comment #70)
> I've checked in a change to the multicore visualizer that modifies
> the painting of the thread "dots". For any process(es) that have one
> or more threads selected in the visualizer, we now draw all their threads
> with a slightly brighter hue. Threads belonging to processes which are
> not part of any selection are subtly darker. The intent is to subtly
> emphasize threads that are part of the "currently selected process(es)".
> 
> (We'll give this a try, and see if it works, or if it's too subtle. :-)

That was fast!

I tried it out but it wasn't very clear in my scenario.  I ran two processes with say 10 threads each.  By fluke, the first process had all 10 threads running, while the other had all 10 suspended.  

When I selected a thread, all the running threads got brighter (or all the suspended ones for the other process).  This made it so that I couldn't compare the brightness of colors with the normal brightness, so I didn't noticed a difference.

When I have some suspended threads brighter and others not, it is noticeable (although I have to really focus :).

Maybe using an highlight as we do for the process thread would be easier to see.  Instead of white we could use another color for it.

Comment 72 Marc Khouzam 2012-01-19 12:01:19 EST

To make sure we have enough time to get the code contributed, I have opened the Contribution Questionnaire:
https://dev.eclipse.org/ipzilla/show_bug.cgi?id=5977
It is only visible to committers.

I will soon be attaching the code to it so the review can start.

Comment 73 William Swanson 2012-01-20 11:08:12 EST

I did a comment/cleanup pass over the code, and I've also added a section
to the visualizer feature's ABOUT.txt document that summarizes the steps
needed to create a visualizer.

We're good to go for the review at this point.

Comment 74 William Swanson 2012-01-23 15:59:16 EST

Created attachment 209939 [details]
Proposed initial visualizer code merge to CDT (patch file)


Here is a patch containing the initial code contribution for:
- the Visualizer Framework feature
- the Multicore Visualizer feature built on top of it

This code was written by me, except for roughly 10% contributed
by Marc Khouzam (CDT committer).

I have the right to contribute this code to Eclipse.

I have an Eclipse Committer Employer Consent Form filed with
the Eclipse Committer Records department, granting permission
from my employer for me to participate in Eclipse projects.

I have verified that the source files have the appropriate
Eclipse license header.

Comment 75 Marc Khouzam 2012-02-09 21:35:58 EST

I got an update on the CQ today.  It has been picked up for review.  Instead of a patch file, they asked for a tar file of the files, to make the review faster.  I have attached such a tar file, and the review should now take place shortly.

Comment 76 Marc Khouzam 2012-02-10 11:44:34 EST

Drum roll please.

The CQ has been approved!  I can now commit the Visualizer contribution!

Please not the commit contains:
- one new directory in the CDT structure: "git/visualizer" (like "git/dsf-gdb" or "git/core")
- 5 new plugins, 2 of which are feature plugins:
  git/dsf-gdb/org.eclipse.cdt.dsf.gdb.multicorevisualizer.ui
  git/dsf-gdb/org.eclipse.cdt.gnu.multicorevisualizer-feature
  git/visualizer/org.eclipse.cdt.visualizer.core
  git/visualizer/org.eclipse.cdt.visualizer.ui
  git/visualizer/org.eclipse.cdt.visualizer-feature


Doug, 
is there anything I should do for releng?
I believe I have to update the root pom.xml file to include those 5 plugins.
What about making the two features available on the update site? anything to do for that?
Anything else?  Hudson maybe?  Or is the pom.xml enough?

Thanks

And congrats Bill!

Comment 77 Marc Khouzam 2012-02-10 11:45:36 EST

Before committing it was suggested that:

> Prior to checkin you may want to rephrase a couple of comments that contain a
> reference to "hack*".  I include the applicable file names should you wish to
> do that:
> 
> MulticoreVisualizerCanvas.java 
> MulticoreVisualizerSelectionFinder.java 
> ExtensionElement.java
> MulticoreVisualizer.java

I will do that.

Comment 78 Marc-André Laperle 2012-02-10 13:10:43 EST

(In reply to comment #76)
> What about making the two features available on the update site? anything to do
> for that?
> Anything else?  Hudson maybe?  Or is the pom.xml enough?

I think you need to update 
org.eclipse.cdt.master.git/category.xml

Comment 79 Marc Khouzam 2012-02-10 14:55:01 EST

(In reply to comment #78)

> I think you need to update 
> org.eclipse.cdt.master.git/category.xml

Thanks!

What about:
./releng/org.eclipse.cdt.releng/maps/cdt.map
./releng/org.eclipse.cdt.releng/buildsite.xml
./releng/org.eclipse.cdt.releng/build.xml
./releng/org.eclipse.cdt.releng/visualizer/customTargets.xml

Looks like these files have entries for our different plugin/features...

Comment 80 Marc Khouzam 2012-02-10 15:45:28 EST

The patch "Proposed initial visualizer code merge to CDT (patch file)" has now been committed to master!

To get the code just do a fetch of master.
Then import the three projects:
  dsf-gdb/org.eclipse.cdt.dsf.gdb.multicorevisualizer.ui
  visualizer/org.eclipse.cdt.visualizer.core
  visualizer/org.eclipse.cdt.visualizer.ui

I still have to see if there are other releng things to fix (see comment 79).

We need a N&N entry at http://wiki.eclipse.org/CDT/User/NewIn90
which will help people try out the feature.

Thanks Bill for the contribution!

I suggest to track all future issues using other bugs.

Bill, any reason to keep this bug open, or can we have the pleasure of marking it FIXED?

Comment 81 William Swanson 2012-02-10 15:51:23 EST

(In reply to comment #80)
> 
> I suggest to track all future issues using other bugs.
> 
> Bill, any reason to keep this bug open, or can we have the pleasure of marking
> it FIXED?

We might use it to track remaining feature/fix work for the EclipseCon demo,
but we could do that anyway, or use a separate bug, so yes, we can mark it FIXED.

Comment 82 Marc Khouzam 2012-02-14 13:52:14 EST

(In reply to comment #79)

> What about:
> ./releng/org.eclipse.cdt.releng/maps/cdt.map
> ./releng/org.eclipse.cdt.releng/buildsite.xml
> ./releng/org.eclipse.cdt.releng/build.xml
> ./releng/org.eclipse.cdt.releng/visualizer/customTargets.xml
> 
> Looks like these files have entries for our different plugin/features...

Confirmed this is not needed:

-----Original Message-----
From: cdt-dev-bounces@eclipse.org [mailto:cdt-dev-bounces@eclipse.org] On Behalf Of Doug Schaefer
Sent: Tuesday, February 14, 2012 1:45 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] What releng changes are needed when adding new plugins

org.eclipse.cdt.releng is the old build system. It no longer applies.
The pom.xml and category.xml is all that's needed.

Doug.

Comment 83 Marc Khouzam 2012-02-14 14:46:57 EST

(In reply to comment #78)
> (In reply to comment #76)
> > What about making the two features available on the update site? anything to do
> > for that?
> > Anything else?  Hudson maybe?  Or is the pom.xml enough?
> 
> I think you need to update 
> org.eclipse.cdt.master.git/category.xml

I got this one wrong.
I updated 
  releng/org.eclipse.cdt.repo/category.xml
which needed to be updated, but I didn't update
   org.eclipse.cdt.master.git/category.xml
which is in another git repo.

I will do this now.

Comment 84 Marc-André Laperle 2012-02-15 20:35:19 EST

I tried the visualizer with MinGW's GDB 7.4 and CreateThread and it showed a single empty green square (and a 0 at the bottom). What are the requirements for GDB's implementation for the visualizer? Perhaps it should be added to the N&N?

Comment 85 Marc Khouzam 2012-02-16 09:17:22 EST

(In reply to comment #84)
> I tried the visualizer with MinGW's GDB 7.4 and CreateThread and it showed a
> single empty green square (and a 0 at the bottom). What are the requirements
> for GDB's implementation for the visualizer? Perhaps it should be added to the
> N&N?

Apologies about that.  I completely forgot to mention this.

On Windows, GDB does not report information about cores, so the Multicore Visualizer cannot work.

However, from Windows debugging a Linux target using gdbserver, things may work, although I never tried it.  I would be interested to know the result of such a test, if someone tries it.

I will update the N&N

Comment 86 William Swanson 2012-02-16 11:54:27 EST

We should also add a reminder about enabling the appropriate debugging options
in the launch when debugging with the visualizer. I've added the following text to
the New and Noteworthy entry:

  Also, when creating a debugging launch to use with the Visualizer, remember to enable the following GDB options on the Debugger tab in the launch editor. (These are not specifically required by the visualizer itself; they enable the kind of fine-grained process/thread debugging control that the visualizer is intended for.)

   . Non-stop mode
   . Automatically debug forked processes

Comment 87 CDT Genie 2012-02-29 13:24:54 EST

*** cdt git genie on behalf of Marc Khouzam ***

    Bug 335027: Visualizer View feature.  Releng updates.

[*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=7ab76fca2c473589af53b5ba1bc82feb22a5cad8

Comment 88 CDT Genie 2012-02-29 13:24:54 EST

*** cdt git genie on behalf of Marc Khouzam ***

    Bug 335027: Visualizer View feature.  Releng updates.

[*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=7ab76fca2c473589af53b5ba1bc82feb22a5cad8
*** cdt git genie on behalf of Marc Khouzam ***

    Bug 335027: Visualizer View feature.  Releng updates.

[*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=7ab76fca2c473589af53b5ba1bc82feb22a5cad8

Comment 89 CDT Genie 2012-02-29 13:25:14 EST

*** cdt git genie on behalf of William R. Swanson ***

    Bug 335027: Visualizer View feature.  Initial contribution.

[*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=093697a0167b5b9bb0287baa0dd38547a976f1e5