Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 315677 - [variables] Add "hasSubExpressions" method to IExpressions
Summary: [variables] Add "hasSubExpressions" method to IExpressions
Status: RESOLVED FIXED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-debug-dsf (show other bugs)
Version: 7.0   Edit
Hardware: PC Windows 7
: P3 enhancement (vote)
Target Milestone: 8.0   Edit
Assignee: Nobody - feel free to take it CLA
QA Contact: Pawel Piech CLA
URL:
Whiteboard:
Keywords:
Depends on: 317324 317325
Blocks:
  Show dependency tree
 
Reported: 2010-06-03 17:04 EDT by Nobody - feel free to take it CLA
Modified: 2010-06-25 17:29 EDT (History)
3 users (show)

See Also:
marc.khouzam: review+
pawel.1.piech: review+
nobody: review+


Attachments
Proposed solution. (7.91 KB, patch)
2010-06-04 12:45 EDT, Nobody - feel free to take it CLA
nobody: iplog-
Details | Diff
Updated patch. (11.14 KB, patch)
2010-06-07 19:17 EDT, Nobody - feel free to take it CLA
nobody: iplog-
Details | Diff
Patch based on the extension of IExpressionDMData interface (8.63 KB, patch)
2010-06-08 13:57 EDT, Nobody - feel free to take it CLA
nobody: iplog-
Details | Diff
Patch based on Pawel's proposal (11.68 KB, patch)
2010-06-17 18:28 EDT, Nobody - feel free to take it CLA
nobody: iplog-
Details | Diff
Added missing since tag. (11.72 KB, patch)
2010-06-18 14:29 EDT, Nobody - feel free to take it CLA
nobody: iplog-
Details | Diff
Deprecate method and fix JUnit tests (10.96 KB, patch)
2010-06-25 11:45 EDT, Marc Khouzam CLA
marc.khouzam: iplog-
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nobody - feel free to take it CLA 2010-06-03 17:04:12 EDT
Currently "getSubExpressionCount" of the "IExpressions" service is used to handle both IHasChildrenUpdate and IChildrenCountUpdate requests. Adding "hasSubExpressions" will allow models to distinguish these cases and handle it differently.
Comment 1 Pawel Piech CLA 2010-06-03 17:21:12 EDT
You may also be able to implement has children just by looking at the element type.  Having the model return a false positive for has-children is not a big deal because the view handles it pretty gracefully.
Comment 2 Nobody - feel free to take it CLA 2010-06-03 17:29:37 EDT
(In reply to comment #1)
> You may also be able to implement has children just by looking at the element
> type.  Having the model return a false positive for has-children is not a big
> deal because the view handles it pretty gracefully.
Yes, you're right. I have been busy implementing the IRegisters service based on variable objects that's why I'm so focused on it.
Comment 3 Nobody - feel free to take it CLA 2010-06-04 12:45:55 EDT
Created attachment 171126 [details]
Proposed solution.

This patch adds a new interface that extends IExpressions2 and modifies the exisiting implementation to take advantage of the new API.
Comment 4 Nobody - feel free to take it CLA 2010-06-04 12:49:44 EDT
Marc, please review this patch. It introduces a new interface and I don't know if it can be accepted for the maintenance branch.
Comment 5 Marc Khouzam CLA 2010-06-04 13:42:33 EDT
1- the IExpressions3 should be officially approved by Pawel, as it is in DSF

2- Can you remove the implements of IExpressions2 from MIExpressions

3- Why the change to MIVariableManager?

Assuming #3 is a mistake, then I don't think this is as efficient as what you would like.  As proposed (without the change to MIVariableManager), I believe the call to hasSubExpressions() will cause as many -var-list-children as getSubExpressionCount().

I was thinking that we have two options:

a- we change ExpressionDMData to no longer know the number of children, but only hasChildren().  No one actually uses ExpressionDMData.getNumChildren since it is not part of IExpressionDMData.  I don't know why we have that method.  This means also changing ExprMetaGetVarInfo to not know the number of children, but only hasChildren().  This will avoid all attempts to list the children for IExpressions.getExpressionData() and the new IExpressions.hasSubExpressions().

Of course, the problem is that is a non-backwards-compatible change.  We could actually keep this change backwards-compatible, but ExpressionDMData.getNumChildren() would not work and we should indicate that clearly.

b- We could not reuse ExprMetaGetVar but define a new ExprMetaGetHasChildren, which would simply check if there are children or not.  This can be done without using -var-list-children
Comment 6 Nobody - feel free to take it CLA 2010-06-04 14:34:20 EDT
(In reply to comment #5)
Thanks Marc.

> 2- Can you remove the implements of IExpressions2 from MIExpressions

Sure.
 
> 3- Why the change to MIVariableManager?
> 
> Assuming #3 is a mistake, then I don't think this is as efficient as what you
> would like.  As proposed (without the change to MIVariableManager), I believe
> the call to hasSubExpressions() will cause as many -var-list-children as
> getSubExpressionCount().
>

Not sure I understand you. The changes in MIVariableManager are essential. You are absolutely right: without it hasSubExpressions() will cause as many -var-list-children as getSubExpressionCount().
The differences between two calls are following:

- hasSubExpression() always uses the "numchild" value returned by "var-create", so no "var-list-children" calls

- getSubExpressionsCount() depends on isNumChildrenHintTrustworthy() to decide whether to make "var-list-children" call.

This problem is specific to variable objects that's why I changed MIVariableManager.

> I was thinking that we have two options:
> 
> a- we change ExpressionDMData to no longer know the number of children, but
> only hasChildren().  No one actually uses ExpressionDMData.getNumChildren since
> it is not part of IExpressionDMData.  I don't know why we have that method. 

Yes, it is confusing. I thought I had to use a higher level classes.
 
> This means also changing ExprMetaGetVarInfo to not know the number of children,
> but only hasChildren().  This will avoid all attempts to list the children for
> IExpressions.getExpressionData() and the new IExpressions.hasSubExpressions().
> 
> Of course, the problem is that is a non-backwards-compatible change.  We could
> actually keep this change backwards-compatible, but
> ExpressionDMData.getNumChildren() would not work and we should indicate that
> clearly.
>

As far as I understand ExprMetaGetVarInfo is associated with the data returned by var-create. If we go with this option I think that we should keep getNumChildren(). The way GDB manages the members of C++ classes should be modified instead. It should be possible to switch off "public/protected/private" children in the var-create command.
 
> b- We could not reuse ExprMetaGetVar but define a new ExprMetaGetHasChildren,
> which would simply check if there are children or not.  This can be done
> without using -var-list-children

I'll take a look at this option but first I would like to clarify the above issues.
Comment 7 Pawel Piech CLA 2010-06-04 14:37:05 EDT
(In reply to comment #5)
> 1- the IExpressions3 should be officially approved by Pawel, as it is in DSF

After reading Marc's comments it seems to me that it may be more appropriate to add a IExpressionDMData.hasChildren(), because it would reduce the need for another async call by the client.  In the javadoc indicate that this method could give false positives.  I.e. it may be more efficient for the debugger to make a best guess at whether an expression has children, and to find out for sure clients should call getSubExpressionCount.
I would prefer not to add a new interface in the DSF maintenance release (@since 2.1.1?).  Maybe you could work around it by accessing an internal method in DSF-GDB UI plugin.

> Of course, the problem is that is a non-backwards-compatible change.  We could
> actually keep this change backwards-compatible, but
> ExpressionDMData.getNumChildren() would not work and we should indicate that
> clearly.
Could you use getNumChildren() to return the "numchild value"
Comment 8 Marc Khouzam CLA 2010-06-04 14:45:19 EDT
(In reply to comment #7)

> > Of course, the problem is that is a non-backwards-compatible change.  We could
> > actually keep this change backwards-compatible, but
> > ExpressionDMData.getNumChildren() would not work and we should indicate that
> > clearly.
> Could you use getNumChildren() to return the "numchild value"

I don't understand...
I thought you meant that when getNumChildren() is called (if ever) it would make a call to the service and actually fetch the children only then.  I think that would work.  Is that what you meant?
Comment 9 Marc Khouzam CLA 2010-06-04 14:57:09 EDT
(In reply to comment #6)

> Not sure I understand you. The changes in MIVariableManager are essential. You
> are absolutely right: without it hasSubExpressions() will cause as many
> -var-list-children as getSubExpressionCount().
> The differences between two calls are following:
> 
> - hasSubExpression() always uses the "numchild" value returned by "var-create",
> so no "var-list-children" calls
> 
> - getSubExpressionsCount() depends on isNumChildrenHintTrustworthy() to decide
> whether to make "var-list-children" call.
> 
> This problem is specific to variable objects that's why I changed
> MIVariableManager.

All true.
The problem is that ExprMetaGetVar is also used by IExpressions.getExpressionData() so the change to MIVariableManager will affect that call and will make the returned IExpressionDMData not have the right number of children.  Of course, the number of children is not actually used, so I think you are effectively implementing suggestion (a), but not completely since it will break ExpressionDMData.getNumChildren()

> As far as I understand ExprMetaGetVarInfo is associated with the data returned
> by var-create. If we go with this option I think that we should keep
> getNumChildren(). The way GDB manages the members of C++ classes should be
> modified instead. It should be possible to switch off
> "public/protected/private" children in the var-create command.

That would be much nicer.
But we'll have to keep dealing with older GDBs for a while, so I think we still need a proper handling in CDT.  Replacing ExpressionDMData.getNumChildren with ExpressionDMData.hasChildren is sufficient I think.
Comment 10 Marc Khouzam CLA 2010-06-04 15:02:51 EDT
(In reply to comment #7)
> (In reply to comment #5)
> > 1- the IExpressions3 should be officially approved by Pawel, as it is in DSF
> 
> After reading Marc's comments it seems to me that it may be more appropriate to
> add a IExpressionDMData.hasChildren(), because it would reduce the need for
> another async call by the client.  In the javadoc indicate that this method
> could give false positives.  I.e. it may be more efficient for the debugger to
> make a best guess at whether an expression has children, and to find out for
> sure clients should call getSubExpressionCount.
> I would prefer not to add a new interface in the DSF maintenance release
> (@since 2.1.1?).  Maybe you could work around it by accessing an internal
> method in DSF-GDB UI plugin.

BTW, I like this idea.  But why the false positives?  For GDB at least, we can always know if there are children or not (we can't know the exact number without -var-list-children sometimes, but we can know != 0)
Comment 11 Marc Khouzam CLA 2010-06-04 15:18:02 EDT
(In reply to comment #10)
> > I.e. it may be more efficient for the debugger to
> > make a best guess at whether an expression has children, and to find out for
> > sure clients should call getSubExpressionCount.
>
> BTW, I like this idea.  But why the false positives?  For GDB at least, we can
> always know if there are children or not (we can't know the exact number
> without -var-list-children sometimes, but we can know != 0)

Sorry, I get it now.  Some debuggers may choose to avoid properly answering such a IExpressions.hasChildren() for efficiency but to always return true instead.
Comment 12 Nobody - feel free to take it CLA 2010-06-04 15:59:06 EDT
(In reply to comment #7)
> (In reply to comment #5)
> > 1- the IExpressions3 should be officially approved by Pawel, as it is in DSF
> 
> After reading Marc's comments it seems to me that it may be more appropriate to
> add a IExpressionDMData.hasChildren(), because it would reduce the need for
> another async call by the client.  In the javadoc indicate that this method
> could give false positives.  I.e. it may be more efficient for the debugger to
> make a best guess at whether an expression has children, and to find out for
> sure clients should call getSubExpressionCount.
> I would prefer not to add a new interface in the DSF maintenance release
> (@since 2.1.1?).  Maybe you could work around it by accessing an internal
> method in DSF-GDB UI plugin.
 
I don't think we need to mention it. False positives are used in many situations.

> Could you use getNumChildren() to return the "numchild value"

That's what I am proposing in this patch.
Comment 13 Nobody - feel free to take it CLA 2010-06-04 16:06:05 EDT
(In reply to comment #9)
> I think you are effectively implementing suggestion (a), but not completely
> since it will break ExpressionDMData.getNumChildren()

My point is that ExpressionDMData.getNumChildren() is an MI extension, so it should return the "numchild" value. See comment 7.
Comment 14 Nobody - feel free to take it CLA 2010-06-04 16:08:41 EDT
Anyway, I'll try to implement it without API changes but we should consider to make the appropriate API changes later.
Comment 15 Marc Khouzam CLA 2010-06-04 16:20:07 EDT
(In reply to comment #13)
> (In reply to comment #9)
> > I think you are effectively implementing suggestion (a), but not completely
> > since it will break ExpressionDMData.getNumChildren()
> 
> My point is that ExpressionDMData.getNumChildren() is an MI extension, so it
> should return the "numchild" value. See comment 7.

I see what you mean now.  When you guys say "numchild" you mean what GDB returns in -var-create.

The iffy thing here is that when GDB says "numchild" is 2 for private/public, then -var-list-children will actually create two children: private and public.  But in our case, we ignore that level of children, so when we will say that there are 2 children based on "numchild", a call to getSubExpressions() could return something different.  This seems bad to me.

But again ExpressionDMData.getNumChildren() is not even used so it feels like a lot of energy spend to support something we don't use :-)  Maybe we can simply javadoc that method to say that it should only be used to know if children exists and not as an exact number, getSubExpressionsCount should be used instead for an exact number.  Basically, making it behave like hasChildren() without renaming it to avoid breaking the API
Comment 16 Nobody - feel free to take it CLA 2010-06-04 17:06:03 EDT
(In reply to comment #15)
> But again ExpressionDMData.getNumChildren() is not even used so it feels like a
> lot of energy spend to support something we don't use :-)  Maybe we can simply
> javadoc that method to say that it should only be used to know if children
> exists and not as an exact number, getSubExpressionsCount should be used
> instead for an exact number.  Basically, making it behave like hasChildren()
> without renaming it to avoid breaking the API

Sounds good to me. We can deprecate getNumChildren() to avoid confusion. But that's an API change.
Comment 17 Nobody - feel free to take it CLA 2010-06-04 18:05:27 EDT
(In reply to comment #7)
> I would prefer not to add a new interface in the DSF maintenance release
> (@since 2.1.1?).  Maybe you could work around it by accessing an internal
> method in DSF-GDB UI plugin.

I can add a new internal DSF/GDB service (say GdbMIExpressions) that extends MIExpressions and implements hasSubExpressions() and overwrite updateHasElementsInSessionThread() in GdbVariableVMNode.
The only problem is there is no package for internal services in DSF/GDB. Is it OK to add an internal package?
Comment 18 Marc Khouzam CLA 2010-06-04 21:08:11 EDT
(In reply to comment #17)
> (In reply to comment #7)
> > I would prefer not to add a new interface in the DSF maintenance release
> > (@since 2.1.1?).  Maybe you could work around it by accessing an internal
> > method in DSF-GDB UI plugin.
> 
> I can add a new internal DSF/GDB service (say GdbMIExpressions) that extends
> MIExpressions and implements hasSubExpressions() and overwrite
> updateHasElementsInSessionThread() in GdbVariableVMNode.
> The only problem is there is no package for internal services in DSF/GDB. Is it
> OK to add an internal package?

I don't know if that is ok for a maintenance branch, but I don't see why it wouldn't be.  Besides that point, I'm fine with that.

I'm a little confused about what you will do in the end though :-)
I guess I'll wait for patch.
Comment 19 Nobody - feel free to take it CLA 2010-06-06 15:41:24 EDT
(In reply to comment #18)
> (In reply to comment #17)
> > (In reply to comment #7)
> > > I would prefer not to add a new interface in the DSF maintenance release
> > > (@since 2.1.1?).  Maybe you could work around it by accessing an internal
> > > method in DSF-GDB UI plugin.
> > 
> > I can add a new internal DSF/GDB service (say GdbMIExpressions) that extends
> > MIExpressions and implements hasSubExpressions() and overwrite
> > updateHasElementsInSessionThread() in GdbVariableVMNode.
> > The only problem is there is no package for internal services in DSF/GDB. Is it
> > OK to add an internal package?
> 
> I don't know if that is ok for a maintenance branch, but I don't see why it
> wouldn't be.  Besides that point, I'm fine with that.
> 
> I'm a little confused about what you will do in the end though :-)
> I guess I'll wait for patch.

I thought (wrongly) that I could extend MIExpressions but I need access to some of its private members :( 
As there are other issues that need to be addressed to improve performance let's wait with this patch for an appropriate realease.
Comment 20 Nobody - feel free to take it CLA 2010-06-07 19:17:36 EDT
Created attachment 171357 [details]
Updated patch.
Comment 21 Nobody - feel free to take it CLA 2010-06-07 19:20:39 EDT
The important part of the original patch is missing, no wander Marc was confused. The updated patch still contains a new interface, so it can only be applied to a major version.
Comment 22 Pawel Piech CLA 2010-06-07 19:25:37 EDT
(In reply to comment #21)
> The important part of the original patch is missing, no wander Marc was
> confused. The updated patch still contains a new interface, so it can only be
> applied to a major version.

Instead of IExpressions3.hasSubExpressions(), could we add an IExpressionDMData3.hasSubExpressions()?  Can you see any reason that would be a bad idea?

(In reply to comment #12)
> I don't think we need to mention it. False positives are used in many
> situations.
They may be used in many situations, but it's an important fact about the API :-)
Comment 23 Nobody - feel free to take it CLA 2010-06-07 20:09:08 EDT
(In reply to comment #22)
> Instead of IExpressions3.hasSubExpressions(), could we add an
> IExpressionDMData3.hasSubExpressions()?  Can you see any reason that would be a
> bad idea?

I can't come up with any reason against it. It just seems to me that "hasSubExpressions" and "getSubExpressionCount" should belong to the same interface. IExpressionDMData is probably the better place because it contains the type information which (as you suggested earlier) can be used to implement these two methods.
Comment 24 Pawel Piech CLA 2010-06-07 23:35:08 EDT
(In reply to comment #23)
> I can't come up with any reason against it. It just seems to me that
> "hasSubExpressions" and "getSubExpressionCount" should belong to the same
> interface. IExpressionDMData is probably the better place because it contains
> the type information which (as you suggested earlier) can be used to implement
> these two methods.

It's just a matter of chunking information in such a way that it is convenient for the client, but not so much so as to degrade performance.  I'm guessing that in the worst case, the service can calculate the has-children property based on expression type, which is already in IExpressionDMData, so as not to force the service to retrieve a much larger chunk of data.  A separate hasSubExpressions() call seems like an overkill to me...
Comment 25 Nobody - feel free to take it CLA 2010-06-08 13:57:16 EDT
Created attachment 171436 [details]
Patch based on the extension of IExpressionDMData interface

According to Pawel's request this patch uses an extension of IExpressionDMData interface.
Comment 26 Nobody - feel free to take it CLA 2010-06-08 14:07:18 EDT
(In reply to comment #22)
> Instead of IExpressions3.hasSubExpressions(), could we add an
> IExpressionDMData3.hasSubExpressions()?  Can you see any reason that would be a
> bad idea?
See https://bugs.eclipse.org/bugs/attachment.cgi?id=171436. The disadvantage is that we have to get data to check whether it is an instance of the new interface. If it is not we have to make another async call from the previos call to get the children count.
Comment 27 Pawel Piech CLA 2010-06-08 16:55:18 EDT
(In reply to comment #26)
> See https://bugs.eclipse.org/bugs/attachment.cgi?id=171436. The disadvantage is
> that we have to get data to check whether it is an instance of the new
> interface. If it is not we have to make another async call from the previos
> call to get the children count.

Oh wow, you're right, that is ugly.  In that case, what do you think of adding:

interface IExpressions3 extends IExpressions {
    public interface IExpressionDMDataExtension extends IExpressionDMData {
        
        boolean hasChildren();
    }


    void getExpressionDataExtension(
        IExpressionDMContext dmc, 
        DataRequestMonitor<IExpressionDMDataExtension> rm);
}

The only thing I find annoying is the lack of symmetry between IExpression3 and IExpressionDMDataExtension names.
Comment 28 Nobody - feel free to take it CLA 2010-06-08 18:56:17 EDT
(In reply to comment #27)
> Oh wow, you're right, that is ugly.  In that case, what do you think of adding:
> 
> interface IExpressions3 extends IExpressions {
>     public interface IExpressionDMDataExtension extends IExpressionDMData {
> 
>         boolean hasChildren();
>     }
> 
> 
>     void getExpressionDataExtension(
>         IExpressionDMContext dmc, 
>         DataRequestMonitor<IExpressionDMDataExtension> rm);
> }
> 
> The only thing I find annoying is the lack of symmetry between IExpression3 and
> IExpressionDMDataExtension names.

Yes, this will work and I too don't like the names. I am afraid this would follow by IExpressions<N> and IExpressionDMDataExtension<N> :(
Comment 29 Marc Khouzam CLA 2010-06-14 16:06:13 EDT
From going through this bug again, I believe the ball is in Mikhail's court now.

If I'm wrong and I'm supposed to do something, just let me know :-)
Comment 30 Nobody - feel free to take it CLA 2010-06-14 16:09:31 EDT
(In reply to comment #29)
> From going through this bug again, I believe the ball is in Mikhail's court
> now.
> 
> If I'm wrong and I'm supposed to do something, just let me know :-)

Your right, it's in my court now :)
Comment 31 Nobody - feel free to take it CLA 2010-06-17 18:28:43 EDT
Created attachment 172169 [details]
Patch based on Pawel's proposal
Comment 32 Nobody - feel free to take it CLA 2010-06-17 18:30:04 EDT
Now ball is in Marc's or Pawel's court:)
Comment 33 Nobody - feel free to take it CLA 2010-06-18 14:29:34 EDT
Created attachment 172242 [details]
Added missing since tag.

This patch extends the existing API so the upversioning of dsf and dsf.gdb plug-ins is required. It is assumed the next version of dsf plug-ins is 2.2 and 3.1 for dsf.gdb plug-ins.
Comment 34 Pawel Piech CLA 2010-06-24 12:48:24 EDT
The interface extensions are OK by me.
Comment 35 Nobody - feel free to take it CLA 2010-06-24 13:29:31 EDT
Thanks Pawel. Applied to HEAD.
Comment 37 Marc Khouzam CLA 2010-06-25 11:45:43 EDT
Created attachment 172772 [details]
Deprecate method and fix JUnit tests

I've deprecated the old getNumChildren() method as we talked about.

I've also fixed the JUnit tests to work with the changes from this bug.

Committed to HEAD.
Comment 38 Marc Khouzam CLA 2010-06-25 11:46:10 EDT
Mikhail's changes look good to me.
Comment 39 Marc Khouzam CLA 2010-06-25 11:47:08 EDT
Mikhail, can you review the last patch I committed to make sure it is in line with what you had in mind.

Also, let me know if you have any issues with getting the JUnit tests to run on your machine :-)
Comment 40 Nobody - feel free to take it CLA 2010-06-25 12:16:56 EDT
(In reply to comment #39)
> Also, let me know if you have any issues with getting the JUnit tests to run on
> your machine :-)
Marc, please excuse my ignorance, but what exactly I need to install to get org.eclipse.test.performance plug-in? I really don't want to install the entire TPTP in my workspace - have had enough troubles with it. That's the main reason I didn't install the dsf tests.
Comment 42 Pawel Piech CLA 2010-06-25 12:25:56 EDT
(In reply to comment #40)
org.eclipse.test.performance is part of Eclipse core project, it has no outside dependencies.  You can get it from /cvsroot/eclipse.
Comment 43 Marc Khouzam CLA 2010-06-25 14:18:21 EDT
(In reply to comment #40)
> (In reply to comment #39)
> > Also, let me know if you have any issues with getting the JUnit tests to run on
> > your machine :-)
> Marc, please excuse my ignorance, but what exactly I need to install to get
> org.eclipse.test.performance plug-in? I really don't want to install the entire
> TPTP in my workspace - have had enough troubles with it. That's the main reason
> I didn't install the dsf tests.

The DSF-GDB JUnit tests are under
org.eclipse.cdt.tests.dsf.gdb
which you should check-out in your workspace.

You can then refer to 
http://wiki.eclipse.org/CDT/cdt-debug-dsf-gdb#Automated_testing
on how to run them.

I run them on Linux and John is running them on Windows
Comment 44 Nobody - feel free to take it CLA 2010-06-25 17:29:43 EDT
(In reply to comment #39)
> Mikhail, can you review the last patch I committed to make sure it is in line
> with what you had in mind.

The patch looks good to me.