Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 121127

Summary: [DataBinding] JFace Binding of Value Holder Relationships should be more dynamic
Product: [Eclipse Project] Platform Reporter: Scott Delap <scott>
Component: UIAssignee: Boris Bokowski <bokowski>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: P3 CC: bradleyjames, djo, gmendel, peter
Version: 3.2Keywords: helpwanted
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:

Description Scott Delap CLA 2005-12-15 16:34:23 EST
Binding controls and other properties to specific object instances is the average case.  In my experience however you often run into the case of binding object instances that aren't available until runtime and change during runtime.  As a result you have numerous master/detail type relationships between views/guis that are "glued" together by value holders.  In the case of JFace Binding this seems to be done by having an IUpdatableValue be a selection holder for instance.  This part is fine.  I'm currently finding the verbose type specific code needed to bind to specific values of the object which the IUpdatableValue contains a reference to frustracting however.    Case in point I have an object that has been selected which has a "name" property.  Currently I have to specify a property descriptor as follows:

new Property(iupdateableValueObject, "name", String.class, false)

I would instead like to be able to simply use:

new Property(iupdateableValueObject), "name")

Since I've specified an IUpdatableValue object and a property, it should be assumed that I want to bind to a property of the object it holds a reference to not the IUpdatableValue itself.  Since type checking can't be done until runtime anyway, I'd argue against doing it during binding unless specifically specified.  You could keep the more specific case and just provide a looser case.  I would also argue that if the initial value of the IUpdatableValue is null you can't check the name type of the object to be a String anyway.
Comment 1 Dave Orme CLA 2005-12-15 17:43:21 EST
Scott, or whoever triages this bug, please add [DataBinding] to the beginning of the summary.
Comment 2 Boris Bokowski CLA 2005-12-16 11:53:29 EST
The problem is that the updatable created using

dbc.createUpdatable(new Property(iupdateableValueObject, "name", String.class, false))

has to implement getValueType() so that default converters/validators can be looked up. Since the current value of the iupdateableValueObject can be null at the time you bind (i.e. the value type cannot be determined at that time), I don't see how we can remove the requirement of specifying the value type (and the boolean for specifying whether it is a collection).

Closing as WONTFIX. If you have an idea how this can be solved in a better way, please reopen.
Comment 3 Scott Delap CLA 2005-12-16 14:34:17 EST
I can see the point with your case.  The main piece of it is associating converters at bind time.  I can see doing such and along with that type checking at bind time.  I can also see an alternative implementation where type converters are lazily looked up during data synchronization.  So if no type is specified then no converter is wired in.  When synchronization occurs between target and model, both types would be dynamically examined.  If a valid converter is found in a map of cached singleton converter instances for the two types it would then be used.  This setup would keep the benefits you currently have but in essence allow a duck typing like flexibility.  Say I have an object classed an Expression which has a getValue() method which returns type Object.  Some expressions may hold Integers, others Strings etc.  I'm selecting them from a list which then changes the IUpdatableValue holder.  I've then bound the "value" property to the selected object in a typeless manner.  As long as some mechanism in binding knows how to converter from the getValue() object type to the target (probably String), everything will work.
Comment 4 Dave Orme CLA 2005-12-16 15:21:58 EST
I think that Scott has a point here.

Suggest we resolve this as "later" since we can add the behavior Scott is describing later without affecting API, which is our primary concern right now.
Comment 5 Peter Centgraf CLA 2006-06-15 18:38:02 EDT
There are three independent issues here:

A. Should the framework resolve BindSpec members at bind-time or dynamically?
B. Do I want to supply my own BindSpec members?
C. Do I want the framework to create IObservables for me?


I see several cases:

1. Client code wants to fully resolve the BindSpec at bind-time, but wants the framework to lookup the appropriate validators, etc. from factories based on explicit static types.  This allows the framework to do error checking at bind-time and provide fail-fast behavior.  

a. [bind-time, no, no]  Client code passes in IObservable instances, and the framework gets the types via getValueType() or similar.  getValueType() always returns the same result, so the result can be cached/reused.

bind(IObservable,IObservable, boolean [static=true])


b. [bind-time, no, yes]  The IObservable instances are not available at bind-time, so the client code passes in target descriptions with explicit types as Class or EMF EClass instances.

bind(IObservable,       StaticDescription)
bind(StaticDescription, IObservable)
bind(StaticDescription, StaticDescription)


2. Client code wants the framework to lookup new validators, etc. from factories based on the dynamic types of specific value instances, whenever the values change.  No explicit typing is necessary, and binding behavior is dynamically polymorphic.  The result of getValueType() is not stable, so it cannot be cached.  BindSpecs must be rebuilt whenever the IObservables post a change event.

a. [dynamic, no, no]  The client passes in fully-formed IObservable instances.

bind(IObservable, IObservable)


b. [dynamic, no, yes]  For convenience, the client passes in target descriptions (without type information).

bind(IObservable,        DynamicDescription)
bind(DynamicDescription, IObservable)
bind(DynamicDescription, DynamicDescription)


3. [dynamic, yes, maybe]  Client wants the behavior to be mostly dynamic, but needs to override specific parts of the BindSpec.  Client builds a BindSpec and leaves some properties null.  Pieces that are missing are loaded dynamically based on the type of the current value instance.  The client may pass in target descriptions or fully-initialized IObservables.

bind(DynamicDescription, DynamicDescription, BindSpec)
bind(IObservable,        DynamicDescription, BindSpec)
bind(DynamicDescription, IObservable,        BindSpec)
bind(IObservable,        IObservable,        BindSpec)


4. [bind-time, yes, maybe]  Client wants the BindSpec to be fully-resolved at bind-time.  Parts of the BindSpec are custom, but the rest should have default values looked up based on explicit static types.

bind(StaticDescription, StaticDescription, BindSpec)
bind(IObservable,       StaticDescription, BindSpec)
bind(StaticDescription, IObservable,       BindSpec)
bind(IObservable,       IObservable,       BindSpec)
Comment 6 Peter Centgraf CLA 2006-06-16 12:36:16 EDT
Cases 1+4 and cases 2+3 could be merged together (simplifying the API) if the description objects can act in both static and dynamic modes.  I was going to suggest using a marker interface for Descriptions, but perhaps we could require a single method, instead.

/**
 * Common supertype for all descriptions of IObservable targets.
 * @see org.eclipse.jface.databindings.beans.Property
 */
public interface Description {
    /**
     * Accesses the static type of the target of this description.
     * This type will be used to configure a BindSpec at bind-time.
     * @return a <Class|EClass> describing the static type of the target,
     *         or null iff BindSpec members should be resolved dynamically
     */
    public Object getTargetType();
}

Similarly, IObservable should be self-describing, with the following methods.  Furthermore, IObservable can extend Description, like so:

public interface IObservable extends Description {

    // current IObservable members...

    /**
     * @return true iff getTargetType() returns a constant value, regardless
     *         of the current target instance
     */ 
    public boolean isStaticType();

    /**
     * @return a <Class|EClass> describing the type of the target observed here,
     *         for purposes of resolving BindSpec members -- not necessarily
     *         equal to target.getClass() for any particular target instance,
     *         but getTargetType().isAssignableFrom(target.getClass())
     */
    public Object getTargetType();
}

The API for all possible behaviors resolves to these two signatures:

/**
 * Create a new Binding, building BindSpec members via the currently-installed 
 * BindSupportFactory's.  If A or B are not instanceof IObservable, they will
 * be used to create new IObservables via the currently-installed
 * BindingFactory's.
 * @param A a Description (possibly an IObservable) for one side of the binding
 * @param B a Description (possibly an IObservable) for the other side
 */
public Binding bind(Description A, Description B);

/**
 * ...
 * @param spec a BindSpec containing pre-configured Binding support objects
 *             -- null members will be replaced via BindSupportFactory's
 */
public Binding bind(Description A, Description B, BindSpec spec);

Comment 7 Boris Bokowski CLA 2006-06-16 12:53:44 EDT
> The API for all possible behaviors resolves to these two signatures:
...
> public Binding bind(Description A, Description B);
...
> public Binding bind(Description A, Description B, BindSpec spec);
...

I have thought about this, too - except that we would like to be able to pass SWT objects as descriptions (e.g. bind(text, observableStringValue)), and SWT widgets don't implement Description. We could handle that by falling back to just

public Binding bind(Object target, Object model);
public Binding bind(Object target, Object model, BindSpec bindSpec);

but this would require a very long explanation in the Javadoc as to what these methods actually do.
Comment 8 Boris Bokowski CLA 2006-06-16 12:55:08 EDT
I should add that we are also getting feedback from users who only use the IObservable part of the framework that they would like to see IObservables without run-time type information like getValueType(), getElementType(), etc.
Comment 9 Peter Centgraf CLA 2006-06-16 14:31:32 EDT
I can see the benefit of specifying SWT objects directly, if you want to bind to default properties.  (The default properties should be well-documented, of course.)  However, you would need to provide additional overloads to specify static or dynamic BindSpec behavior.

If the type accessors are removed from IObservable and/or the description objects, BindSpec factory behavior is ambiguous.  TableModelDescription is typical.  With no type accessors, what is the expected behavior?

1. Create BindSpecs at bind-time based on the dynamic types of the column properties.

What if the model is empty at bind-time?
What if the table is SWT.VIRTUAL?
What if the column has a polymorphic type?  Which row's types should be used?

2. Create BindSpecs on the fly based on the dynamic types of each row and column.

What if the column doesn't have a polymorphic type, and the BindSpec could have  been safely cached?
What if the UI designer wants all rows to conform to the behavior of a common supertype?

If you leave the type accessors intact, you can still provide clever default behavior or overloads for convenience.  If you remove the accessors, customization is difficult or impossible.
Comment 10 Boris Bokowski CLA 2006-06-16 15:50:02 EDT
(In reply to comment #8)

I just filed bug 147515 for the request to separate run-time type information from the IObservable hierarchy.
Comment 11 Dave Orme CLA 2006-06-16 22:45:52 EDT
In the case where type information is not specified, getValueType can return the most specific type information it has.  If the value holder has not been assigned yet (ie: there is no selection), the type is Object.class.  As soon as it is assigned, the type becomes value.getClass().

If we went this way we would essentially be implementing Smalltalk type system semantics when the client hasn't specified type information.  It's less efficient and can't catch very many errors at bind time compared with static typing but it does work.  :-)

But this is maybe an argument for centralized event propogation engine.  Such an engine would know about the chains of bindings and might be able to apply type inference rules similar to Haskall and more modern languages to gain the convenience of dynamic languages but the safety of statically-typed languages.
Comment 12 Peter Centgraf CLA 2006-06-19 15:21:07 EDT
The current framework implements a Binding creation pipeline:

                    value(x2)
                       |
                   target(x2)
                 /            
    description(x2)     ----> type(x2)
        |             /          | 
IObservableFactory*  /        BindSupportFactory*
        |           /            |
    IObservable(x2)           BindSpec
                \             /
                IBindingFactory*
                       |
                    Binding

I use "target" to mean the actual slot that is being observed.  For example, the target might be the "text" property of a particular SWT Text widget.  The description indirectly contains the target.  For example, the description might be the SWT Text widget itself, or an object that contains the Text widget and additional information.  The type might ultimately come from the value, the target, or the description.  The framework always accesses it through the IObservable.

The problem for dynamic clients is that this pipeline assumes that the types are fixed at bind-time.  Therefore, it also expects the BindSpec to be fully-resolved and immutable for the life of the Binding.  This assumption ignores the polymorphic nature of Java objects (i.e. types can vary from value to value in the same target), but it enables caching of the BindSpec for better performance.  Some clients prefer polymorphism over speed and vice versa.

So... supporting dynamic behavior is a significant change from the current design.  You cannot satisfy this by simply moving the type information around, because it changes the definition of what BindSpec represents, the behavior of Binding, and the pipeline that creates both.  Dave is correct that this introduces radically different semantics.

The dynamic pipeline should look like this:

                    value(x2)
                       |      \
                   target(x2)  \
                 /              \
    description(x2)           type(x2)
        |                        | 
IObservableFactory*           BindSupportFactory*
        |                        |
    IObservable(x2)           BindSpec
                \             /
                IBindingFactory*
                       |
                    Binding

The problem for static clients is that the target description and the target type information often come from the same place.  It is awkward to specify these things separately.  Also, the type information can sometimes be inferred from the target automatically.  For example, a JavaBeans property has an explicit type == the return type of the get method.  Note that this inference can often be performed even if the target does not contain a value instance.  (You can reflect for the static return type of a method even if the return value is null.)

So... the current static mode of operation can provide a simplier API for some circumstances.  This can be implemented with overloads for the description constructors without a major change to the framework.

Both static and dynamic clients will sometimes want to bypass the normal pipeline at various points, providing explicit values rather than letting the factories generate them.  No matter where the client code patches in, the framework should be able to distinguish between static and dynamic behavior.
Comment 13 Dave Orme CLA 2006-06-19 23:16:38 EDT
(In reply to comment #12)
> Both static and dynamic clients will sometimes want to bypass the normal
> pipeline at various points, providing explicit values rather than letting the
> factories generate them.  No matter where the client code patches in, the
> framework should be able to distinguish between static and dynamic behavior.

These echo my thoughts.  My previous data binding frameworks were purely dynamic.  I've come to apprciate the static way.  However, the current framework does not maintain enough metadata in order to efficiently infer types through multiple levels of bindings.

Consider the classic:

Customers -->> Orders --> Line items

Assuming a table supporting in-place editing, this can be modelled with 3 tables: one for customers, another for orders, and the last for line items.

Yet, the current framework has no way to infer the type of the Orders object from the binding of the Customers table.  Similarly, we can't currently infer the data type of the LineItem object from the binding information of the Orders table.  We just have no (current) way to walk the binding chain up to the parent binding.

If we're going to have bindings or observables register themselves using setData() on SWT controls, maybe we can recycle this information to be able to do better type inference up binding chains?

Being able to perform type inference in this way would provide nearly all the API convenience of the dynamically typed approach, yet with the bind-time safety afforded by a statically-typed binding mechanism.
Comment 14 Peter Centgraf CLA 2006-06-20 09:06:33 EDT
I think we're talking about different problems.  What you describe is dynamic in the sense that the types used for creating BindSpecs are looked up at runtime (specifically, bind-time), but it is still static in the sense that the RTTI of target values do not affect the behavior of the system.  Extending your example, imagine a type hierarchy like this:

         Line Item
        /         \
Product Item     Freight Item

The SWT Table that contains Line Items actually holds objects with polymorphic types.  Product Items and Freight Items might have different validators or converters.  The current framework would have to bind to the common supertype, which may not expose all of the desired properties.  A bind-time BindSpec would have to act on the whole hierarchy using instanceof tests, casts, and the big-ugly-switch approach.  The Java dispatch mechanism would be preferable.

I understand from comments elsewhere that this polymorphic form of dynamic behavior was consciously excluded from the design.  What was the rationale?  Perhaps I'm simply overlooking a compensating advantage.  This decision will definitely cause problems for some types of applications, though.  (Personally, I think I can make do with the BUS solution for now.)

To address Dave's concerns... the current framework is (almost) sufficient to resolve the types properly at bind-time.  Description objects usually identify a slot of some kind, which has its own static type (e.g. the return type of a JavaBean accessor, the data type for SWT selections, etc.)  The descriptions should be smart enough to look up the type of the slot, regardless of whether it contains data.  No type inference would be necessary -- just reflection.  Custom IObservables would have to provide type information explicitly or bypass the normal Binding pipeline.

In fact, you absolutely do not want to infer the type from the current value.  For example, you wouldn't want to infer Freight Item for a JavaBean property that might contain arbitrary Line Items.  Any such inference mechanism would plant virtual land mines that future data model changes might trip inadvertently.  The behavior is either fully static, fully dynamic, or broken.  :-P
Comment 15 Peter Centgraf CLA 2006-06-20 09:20:33 EDT
I just realized that the Java type system wrecks the idea of reflecting on JavaBean property types.  Even with generics, the compiler erases type variables to their upper bound.  In the case of collections, the upper bound is always Object, which is useless.  There's no way to avoid some kind of explicit type specifier if you want to have type-safe static/bind-time behavior for collections.  Your choices appear to be:

1. Create BindSpecs using explicitly specified types.
2. Create different BindSpecs using the type of each target value, dynamically.
3. Create BindSpecs using the type of the target value at bind-time, if one exists, and hope/pray that all values conform to that type.
Comment 16 Peter Centgraf CLA 2006-06-20 09:28:52 EDT
There's another possibility:

4. Lookup the type automatically for IObservables/descriptions that are backed with useful static types, and fall back on option 1 or 3 for everything else.

The problem with 4 is that the framework can't know until bind-time which IObservables have useful static types, so there's no way to statically enforce that they are provided when necessary.
Comment 17 Boris Bokowski CLA 2006-06-20 11:08:31 EDT
To complicate matters even further, here is another detail of the current design: getValueType() etc. returns a java.lang.Object, not a java.lang.Class. The reason for this is that we wanted to allow for clients who have their own (richer than Java) type system.  This also enables model-driven binding, i.e. the lookup of converters and validators based on annotations of some sort on the model side.
Comment 18 Peter Centgraf CLA 2006-06-20 14:18:41 EDT
Comment #17 shouldn't be a problem.  The IObservables and the factories have to use matching metadata.  If client code wants to supply factories that work with different metadata, it also needs to supply IObservables (or descriptions and IObservableFactories) that produce the metadata, and vice versa.  This is true regardless of when the framework creates the BindSpec.

I was mistaken in my comment above about generic type erasure.  Type parameters for declarations are still available at runtime, so you can extract reliable static types for collection members at bind-time.  Ironically, the type parameters for object instances are lost, however.  Regardless, this will still be a problem for 1.4 JVMs and for client code that does not use generics in 5+.
Comment 19 Dave Orme CLA 2006-06-20 23:18:29 EDT
(In reply to comment #16)
> There's another possibility:
> 
> 4. Lookup the type automatically for IObservables/descriptions that are backed
> with useful static types, and fall back on option 1 or 3 for everything else.
> 
> The problem with 4 is that the framework can't know until bind-time which
> IObservables have useful static types, so there's no way to statically enforce
> that they are provided when necessary.

This is the option I was proposing.

You can't always use slot data types for more reasons than the ones you listed: 

1) Nested Observables don't have a parent slot data type more specific than java.lang.Object since they are normally bound to the Selection of some SWT control, which itself does not have a data type.

2) Complicating matters further, sometimes it is useful to instantiate a WritableValue or a WritableList as a plumbing placeholder.  In an ideal world, you shouldn't have to specify the data type of the WritableValue or WritableList unless the object needs polymorphic behavior like you described; it should infer its own type when it is bound to its parent in the data flow graph (or when its parent is eventually bound to something with a static type).

The result would be kind of a lazy binding where the actual binding doesn't occur until enough type information can be inferred that everybody knows what his (or her?) data type is.
Comment 20 Dave Orme CLA 2006-06-20 23:50:00 EDT
Actually, looking at the previous comments, I'm pretty sure I still won't have communicated.  :-(  Let's try again.  :-)  Let's look the Customers --> Orders --> LineItems bindings again, with nearly everything in the data flow graph spelled out.

The following represents all bindings that are created starting from the model side and progressing through the full data flow pipeline to the target side (working from memory, but hopefully it's exact enough to make my point).

<<Model>>________________________--->>>___________________________________________________________<<Target>>

[List<Customer>]-[ObservableMapping]-[ListBinding]-[ObservableList]-[CustomerTable.content]

[CustomerTable.selection]-[ObservableValue]-[ValueBinding]-[NestedObservableValue]-[ListBinding<orders>]-[OrdersTable.content]

[OrdersTable.selection]-[ObservableValue]-[ValueBinding]-[NestedObservableValue]-[ListBinding<lineItems>]-[LineItemsTable.content]


Please notice the following:

- There are three data binding chains here, but each chain is completely independent of the others.  To say the same thing another way, the binding to a table's selection at the beginning of a row does not know about the existence of, much less the data type of the table's content binding at the end of the previous row.

- These same selection attributes are untyped.

- As a result, the ObservableValue that gets bound to a widget's selection has no idea of its type unless it's told in the Description object.  (This is the current API.)

- But (and this was my observation): If these three data binding chains were *not* independent, the selection binding could continue traversing backwards to the content binding of the same table (the end of the previous row) to find out the table's data type.  It could then infer its own data type from this information.


This implies two requirements.

1) That we can lazily propogate data types through the data flow graph as it is being constructed.

2) That the programmer is only required to specify data types in cases where there are polymorphic data types.

Peter, you are right that this might lead to switch/case statements in converters or validators.  (I think one could probably use a strategy pattern in a parent converter to delegate to a correct child converter for the actual data type.)  In any case, you are right that this complicates converters and validators a bit.

But you also asked what the benefit of this architecture is.  The primary one is that it keeps you from creating bindings that cannot possibly work since the framework can do static type checking.  (Boris and I are both closet Haskell fans. :-)  The secondary benefit is that when the framework knows the data types of everything, it can supply better error messages when things go wrong.

In theory, this should make it possible to create an API that is more accessible to newbies because the framework could do more to help them when things go wrong.

The problem so far in practice has been that this has resulted in an API that is harder to use than we would prefer, especially for newbies, because we haven't gotten the type inference part right yet.  :-)

Comment 21 Peter Centgraf CLA 2006-06-21 09:16:33 EDT
Comment #20 is very helpful in clarifying your points -- thanks.

Propagating type information from one Binding to another would be a nice feature, indeed.  That is an orthogonal problem to what I described, though.  It's important to note that pushing types from one Binding to another does not imply any form of type inference.  In your example, the framework is responsible for pushing data from List<Customer> to CustomerTable.content.  Since it knows that there is a (true, compiler-checked) static guarantee that all content instanceof Customer, it also has a static guarantee that the Viewer selection will be instanceof Customer.  (If this is a Binding directly to an SWT Table, all bets are off -- the TableItems will reduce everything to String.)

I wish (again) that the JFace API could use type parameters.  With a generics-enabled API, the framework could have genuine static guarantees.  What you describe as "static checking" is really just best-effort, fail-fast runtime checking.  That is a critical distinction, because the Java 5 generics implementation erases instance type parameters at runtime, so it's impossible to be as safe as with true static checking.

My point from #14 is that the Java type system, even when enforced by the compiler, would not guarantee that Customer.class.equals(CustomerTable.selection.getClass()).  It only guarantees that Customer.class.isAssignableFrom(CustomerTable.selection.getClass()).  All Java types are polymorphic by nature, so all Bindings would fit into Dave's requirement 2.

Consider my example to see the harm of type inference based on instances, rather than slot declarations.  If LineItemsTable.content initially contains only Product Items, the framework would create a BindSpec based on ProductItem.  At some later point, a FreightItem is added to the list.  BANG.

Worse yet, what if some future extension adds a new ServiceChargeItem (extends LineItem) to the mix?  How would such an extension integrate with the existing converters and validators?  The BUS approach does not scale well.

What if there is one BindSupportFactory that responds to all LineItems (to handle a table containing mixed subclasses) and another that responds specifically to ProductItems (to handle a properties pane)?  Client code would need to use different factory lists or combine the two factories somehow.  Ugly.

I'm facing these problems already in my own app, so they're not just a theoretical exercise.  Maintaining multiple factory lists is a real pain, and it breaks encapsulation in a bad way.  I expect future extensions of this nature (perhaps even this precise example), which will be out of my control.  I currently have to invent my own protocol (which re-implements parts of the Java type system) to get graceful behavior.

IMHO, a typed functional language would not try to fight the type system as you are currently doing.  In Lisp, for example, you could implement the conversion and validation routines as typed methods and let the runtime call the appropriate overloads based on the RTTI.  If a conversion routine is not available for a specific type, behavior would gracefully degrade to supertype behavior.  If a more-specific conversion routine is added by an extension, it would override the supertype behavior automatically.  In a weakly typed language like SmallTalk, Objective-C, or Python, anything but the fully-dynamic approach would be ludicrous.  The current BindSpec pipeline is implementing type semantics different from anything I've encountered before.

P.S.  I'm also a big fan of functional style languages.  You should check out Scala, if you haven't already.  It runs on the JVM.  I <3 type-safe mixins.  :-)
Comment 22 Dave Orme CLA 2006-06-22 01:02:08 EDT
(In reply to comment #21)
> Comment #20 is very helpful in clarifying your points -- thanks.

And this has been a very useful conversation. :-)

> Propagating type information from one Binding to another would be a nice
> feature, indeed.  That is an orthogonal problem to what I described, though. 
> It's important to note that pushing types from one Binding to another does not
> imply any form of type inference.  In your example, the framework is
> responsible for pushing data from List<Customer> to CustomerTable.content. 
> Since it knows that there is a (true, compiler-checked) static guarantee that
> all content instanceof Customer, it also has a static guarantee that the Viewer
> selection will be instanceof Customer.

Yes.

>  (If this is a Binding directly to an
> SWT Table, all bets are off -- the TableItems will reduce everything to
> String.)

Maybe I wasn't clear.  If the selection binding knows about the parent binding (the content binding), it can get the Customer type from the content binding since that binding knows this information.  You're right that the SWT control loses this information.

> I wish (again) that the JFace API could use type parameters.  With a
> generics-enabled API, the framework could have genuine static guarantees.  

I've wished for this too, but I believe that Platform has to be 1.4 compatible.  Is this correct Boris?

> What
> you describe as "static checking" is really just best-effort, fail-fast 
> runtime
> checking.  That is a critical distinction, because the Java 5 generics
> implementation erases instance type parameters at runtime, so it's impossible
> to be as safe as with true static checking.

Right.  You still have to manually specify collection element data types no matter what.  Using my recommendation, you just wouldn't have to specify them twice: once for the collection binding itself and *again* when binding to the selection.  :-)

> My point from #14 is that the Java type system, even when enforced by the
> compiler, would not guarantee that
> Customer.class.equals(CustomerTable.selection.getClass()).  It only guarantees
> that Customer.class.isAssignableFrom(CustomerTable.selection.getClass()).  All
> Java types are polymorphic by nature, so all Bindings would fit into Dave's
> requirement 2.

In theory yes.  In practice, I have not seen this happen very often.  In my experience, maybe 20% of table bindings need polymorphic behavior.

> Consider my example to see the harm of type inference based on instances,
> rather than slot declarations.  If LineItemsTable.content initially contains
> only Product Items, the framework would create a BindSpec based on 
> ProductItem.
>  At some later point, a FreightItem is added to the list.  BANG.

I'm not sure I follow this.  You may or may not get an error.  If you are only editing fields that are common to ProductItem and FreightItem, then everything should continue to work exactly as before.  The fields don't change their type just because some subclass was added.

If you have converters that convert an entire ProductItem into something else and back, yes, this would need to be handled.  But you would need to handle this anyway, right?

Am I missing something here?

> Worse yet, what if some future extension adds a new ServiceChargeItem (extends
> LineItem) to the mix?  How would such an extension integrate with the existing
> converters and validators?  The BUS approach does not scale well.

Again, this assumes that the ServiceChargeItem object itself needs to be round-trip marshalled and unmarshalled to some other neutral data type.

> What if there is one BindSupportFactory that responds to all LineItems (to
> handle a table containing mixed subclasses) and another that responds
> specifically to ProductItems (to handle a properties pane)?  Client code would
> need to use different factory lists or combine the two factories somehow. 
> Ugly.

I'd use nested factories.

Have the standard data binding factory look up converters for the abstract class.  The converters for the abstract class then looks up specific converters for whatever concrete type it happens to receive.  It then delegates the actual type conversion to the inner converter that it looked up.

> I'm facing these problems already in my own app, so they're not just a
> theoretical exercise.  Maintaining multiple factory lists is a real pain, and
> it breaks encapsulation in a bad way.  I expect future extensions of this
> nature (perhaps even this precise example), which will be out of my control.  I
> currently have to invent my own protocol (which re-implements parts of the Java
> type system) to get graceful behavior.
> 
> IMHO, a typed functional language would not try to fight the type system as you
> are currently doing.  In Lisp, for example, you could implement the conversion
> and validation routines as typed methods and let the runtime call the
> appropriate overloads based on the RTTI.  If a conversion routine is not
> available for a specific type, behavior would gracefully degrade to supertype
> behavior.  If a more-specific conversion routine is added by an extension, it
> would override the supertype behavior automatically.  In a weakly typed
> language like SmallTalk, Objective-C, or Python, anything but the fully-dynamic
> approach would be ludicrous.  The current BindSpec pipeline is implementing
> type semantics different from anything I've encountered before.

Now you've got me curious. :-)

It seems to me that there are two choices:

- Match types at bind time.  (Like what a Haskell RTL does.)

- Don't worry about types at all, just match method signatures when you've got actual objects in hand.  (The Smalltalk type system.)

Is there a third option we've overlooked (assuming Java 1.4)?

> P.S.  I'm also a big fan of functional style languages.  You should check out
> Scala, if you haven't already.  It runs on the JVM.  I <3 type-safe mixins. 
> :-)

I really like Scala from what I've read of it.  My current work needs something that is interpreted dynamically, not compiled statically to bytecode, so I've been working with pnuts, which is pretty nice too.


Thanks again for your thoughts!
Comment 23 Dave Orme CLA 2006-06-22 01:07:43 EDT
>- Match types at bind time.  (Like what a Haskell RTL does.)

(Technically not the RTL, but the compiler or interpreter as in the case of HUGS.)
Comment 24 Peter Centgraf CLA 2006-06-22 11:39:13 EDT
(In reply to comment #22)
> Maybe I wasn't clear.  If the selection binding knows about the parent binding
> (the content binding), it can get the Customer type from the content binding
> since that binding knows this information.  You're right that the SWT control
> loses this information.

My point is that an SWT Table binding acts as a filter on the data content, not just the type information.  It does no good to propagate the type information if there's no path for the actual data.  :-)  To get behavior like you describe, you'd have to ensure that the binding pipeline preserves the complete data from both endpoints at each link.  For example, UIs don't often display database keys, but this data is required to recover the endpoint object.  Since JFace Viewers maintain references in the endpoint type, this isn't a problem.  SWT widgets know only about the Strings being displayed.

> Right.  You still have to manually specify collection element data types no
> matter what.  Using my recommendation, you just wouldn't have to specify them
> twice: once for the collection binding itself and *again* when binding to the
> selection.  :-)

If generic types are specified, you could do better.  This is just icing on the cake, though.

> In theory yes.  In practice, I have not seen this happen very often.  In my
> experience, maybe 20% of table bindings need polymorphic behavior.

I don't think you should dismiss this requirement so easily.  You'd be ruling out a whole class of UIs that make use of dynamic behavior and/or plug-in extensibility.  Any application with a context-sensitive properties pane would suffer, including mine.  ;-)

> >  At some later point, a FreightItem is added to the list.  BANG.
> 
> I'm not sure I follow this.  You may or may not get an error.  If you are only
> editing fields that are common to ProductItem and FreightItem, then everything
> should continue to work exactly as before.  The fields don't change their type
> just because some subclass was added.

Keep in mind, we're talking about two different questions related to types: 1. Does the framework infer types at bind-time at all, or use the RTTI of specific values as they change?  Bind-time types cannot be truly polymorphic, because obviously there's just one type, no matter what it is.  2. If the framework fixes the types at bind-time, does the framework use the slot's type or the current value's type?  IMHO, it is a mistake to use the current value type.  The slot type determines the real type constraint, so using a different constraint would violate type safety.

E.g. if the framework infers ProductItem, it will make calls like this:  createDomainValidator(ProductItem.class).  If the DBC contains a factory that responds to ProductItem specifically, e.g. to support a subtype-specific property editor pane, it might produce an IConverter that assumes it will only recieve ProductItems.  Although the compiler doesn't guarantee this, the framework is essentially making a runtime promise.  If that IConverter is passed a FrieghtItem, it won't know what to do with it.  One way or another, it will fail, perhaps with a ClassCastException or an IllegalArgumentException.

If the framework calls createDomainValidator(LineItem.class) -- with the true type constraint -- the ProductItem-specific BindSupportFactory would not respond.  A lower-priority factory that acts on LineItems would respond instead.  All is well.

> If you have converters that convert an entire ProductItem into something else
> and back, yes, this would need to be handled.  But you would need to handle
> this anyway, right?

Isn't that the whole point of a converter?  As I said in #20, you can work around the problem by maintaining different lists of factories -- one list that has the specific subclass-only factories and another for the general case.  This is awkward in practice, because it forces the client to maintain these separate lists and keep track of which one should be used in which circumstance.  The declarative, centralized nature of the bindings framework is lost.

Actually, I'm not sure about this workaround, since you'd have two bindings against the same JFace Viewer in different DBCs (one for content and one for selection).  Obviously, this goes against the idea of a centralized binding-network manager.  I might need to bypass the whole factory mechanism.

> > Worse yet, what if some future extension adds a new ServiceChargeItem (extends
> > LineItem) to the mix?  How would such an extension integrate with the existing
> > converters and validators?  The BUS approach does not scale well.
> 
> Again, this assumes that the ServiceChargeItem object itself needs to be
> round-trip marshalled and unmarshalled to some other neutral data type.

Not necessarily.  It just assumes that ServiceChargeItems need both subclass/specific and superclass/general behavior anywhere within the purview of the DBC.

> I'd use nested factories. ...

This is exactly what I meant by "re-implements parts of the Java
type system."  If I have to create my own factory system, I've just bypassed a big chunk of the framework.  I'd rather that the framework handled this, even if I have to write it myself and contribute it.  :-)

> It seems to me that there are two choices:
> 
> - Match types at bind time.  (Like what a Haskell RTL does.)

This is reasonable, as long as the matching algorithm has similar semantics to the Java dispatch mechanism.  It should be possible to override the binding behavior for specific subclasses without removing the more general behavior for superclasses.  Preferably, client code should not need to manage the priority list explicitly, since the type system contains sufficient information.

This option interacts with the usage of arbitrary static metadata, which implies an arbitrary priority system.  I don't know how the framework could possibly "match types" if the metadata defines a non-standard type system.

> - Don't worry about types at all, just match method signatures when you've got
> actual objects in hand.  (The Smalltalk type system.)

The Smalltalk analogy breaks down when the behavior does not actually belong to the object being called, but to a third object (e.g. a converter).  I think you're referring more to the Smalltalk style conventions, not the type system.  Smalltalk (and Obj-C, Python, etc.) encourage you to make calls on statically-untyped objects and let the runtime do dynamic dispatch based on RTTI.  In this case, we're talking about dispatch to third-party converters, etc, but it's a very similar process.  I think this is also a very reasonable solution for bindings.  Again, I would want the dispatch mechanism to mirror the Java type system.

> Is there a third option we've overlooked (assuming Java 1.4)?

Of course.  :-)  This is an example of the general mixin problem.  The software engineering and computer science literature is full of interesting options.

I think an apt analogy is the Lisp method-dispatch mechanism.  Lisp methods are first-class objects that are called by name, without reference to a callee type or instance.  The runtime does pattern matching based on the argument types, similar to Java method overload matching.  Java semantics can be simulated by placing the callee as the first argument of the method.  We would need a simplified version of this mechanism that acts on 1 or 2 arguments.  It would occupy the space in the framework where factory lists are now.  Option 1 above would accept type constraints as arguments at bind-time.  Option 2 would accept value instances and use their RTTI.
Comment 25 Peter Centgraf CLA 2006-06-22 12:00:10 EDT
Oh, I should say explicitly, I don't think that option 1 and 2 are mutually-exclusive.  I see no reason why both kinds of bindings couldn't co-exist in the same DBC.  The API would need only minimal change to support this (e.g. allow null return values from getValueType() etc., maintain a "dynamic" flag on Binding).

If you must choose only one, option 2 would be preferable.  Option 1 does provide some nice fail-fast error checking and a small performance improvement, but it prevents basic functionality for a whole class of applications.  That doesn't seem like a good trade-off.
Comment 26 Dave Orme CLA 2006-06-23 00:28:16 EDT
(In reply to comment #24)
> (In reply to comment #22)
> My point is that an SWT Table binding acts as a filter on the data content, not
> just the type information.  It does no good to propagate the type information
> if there's no path for the actual data.  :-)  To get behavior like you
> describe, you'd have to ensure that the binding pipeline preserves the complete
> data from both endpoints at each link.  For example, UIs don't often display
> database keys, but this data is required to recover the endpoint object.  Since
> JFace Viewers maintain references in the endpoint type, this isn't a problem. 
> SWT widgets know only about the Strings being displayed.

Yes.

> > Right.  You still have to manually specify collection element data types no
> > matter what.  Using my recommendation, you just wouldn't have to specify them
> > twice: once for the collection binding itself and *again* when binding to the
> > selection.  :-)
> 
> If generic types are specified, you could do better.  This is just icing on the
> cake, though.

Point taken.

> > In theory yes.  In practice, I have not seen this happen very often.  In my
> > experience, maybe 20% of table bindings need polymorphic behavior.
> 
> I don't think you should dismiss this requirement so easily.  You'd be ruling
> out a whole class of UIs that make use of dynamic behavior and/or plug-in
> extensibility.  Any application with a context-sensitive properties pane would
> suffer, including mine.  ;-)

Point taken again.  I'm actually intending to use data binding in exactly this way on another app I'm working on right now, so the point is timely as well. :-)

> > >  At some later point, a FreightItem is added to the list.  BANG.
> > 
> > I'm not sure I follow this.  You may or may not get an error.  If you are only
> > editing fields that are common to ProductItem and FreightItem, then everything
> > should continue to work exactly as before.  The fields don't change their type
> > just because some subclass was added.
> 
> Keep in mind, we're talking about two different questions related to types: 1.
> Does the framework infer types at bind-time at all, or use the RTTI of specific
> values as they change?  Bind-time types cannot be truly polymorphic, because
> obviously there's just one type, no matter what it is.  2. If the framework
> fixes the types at bind-time, does the framework use the slot's type or the
> current value's type?  IMHO, it is a mistake to use the current value type. 
> The slot type determines the real type constraint, so using a different
> constraint would violate type safety.
> 
> E.g. if the framework infers ProductItem, it will make calls like this: 
> createDomainValidator(ProductItem.class).  If the DBC contains a factory that
> responds to ProductItem specifically, e.g. to support a subtype-specific
> property editor pane, it might produce an IConverter that assumes it will only
> recieve ProductItems.  Although the compiler doesn't guarantee this, the
> framework is essentially making a runtime promise.  If that IConverter is
> passed a FrieghtItem, it won't know what to do with it.  One way or another, it
> will fail, perhaps with a ClassCastException or an IllegalArgumentException.
> 
> If the framework calls createDomainValidator(LineItem.class) -- with the true
> type constraint -- the ProductItem-specific BindSupportFactory would not
> respond.  A lower-priority factory that acts on LineItems would respond
> instead.  All is well.

Right.  Point taken.

My point was although you're right that our current bindings aren't polymorphic, there's no reason why the converters and validators themselves couldn't be.

All the same, I think you've made a strong case for implementing a fully dynamic type system a la Smalltalk's.

> > If you have converters that convert an entire ProductItem into something else
> > and back, yes, this would need to be handled.  But you would need to handle
> > this anyway, right?
> 
> Isn't that the whole point of a converter?  As I said in #20, you can work
> around the problem by maintaining different lists of factories -- one list that
> has the specific subclass-only factories and another for the general case. 
> This is awkward in practice, because it forces the client to maintain these
> separate lists and keep track of which one should be used in which
> circumstance.  The declarative, centralized nature of the bindings framework is
> lost.
> 
> Actually, I'm not sure about this workaround, since you'd have two bindings
> against the same JFace Viewer in different DBCs (one for content and one for
> selection).  Obviously, this goes against the idea of a centralized
> binding-network manager.  I might need to bypass the whole factory mechanism.

I agree that this isn't ideal.  I still like the idea of making the converters themselves polymorphic.  ie: have a LineItemToStringConverter that maintains a registry of converters for each subclass of LineItem that might need converting and uses the right one based on the instanceof the current object.

Actually PolymorphicConverter could easily be written generically, take as a constructor parameter the abstract type whose hierarchy it's supposed to manage, and perform the instanceof lookup itself.

Hmmmm....  Maybe this is the right way to implement dynamic behavior in the current system to begin with--have a DynamicConverter and a DynamicValidator.  They advertise themselves as being able to handle anything related to java.lang.Object.  If they are specified, they provide the dynamic converter/validator lookup behavior.

I'll have to look into this way of implementing what you're asking for.

> > > Worse yet, what if some future extension adds a new ServiceChargeItem (extends
> > > LineItem) to the mix?  How would such an extension integrate with the existing
> > > converters and validators?  The BUS approach does not scale well.
> > 
> > Again, this assumes that the ServiceChargeItem object itself needs to be
> > round-trip marshalled and unmarshalled to some other neutral data type.
> 
> Not necessarily.  It just assumes that ServiceChargeItems need both
> subclass/specific and superclass/general behavior anywhere within the purview
> of the DBC.

If you're thinking of how the same object may need to represent itself in a toString() fashion and a slot-by-slot fashion in a property editor, I think I can see where you're coming from.

Is that what you're driving at?

> > I'd use nested factories. ...
> 
> This is exactly what I meant by "re-implements parts of the Java
> type system."  If I have to create my own factory system, I've just bypassed a
> big chunk of the framework.  I'd rather that the framework handled this, even
> if I have to write it myself and contribute it.  :-)

I'm thinking this wouldn't be such a big deal to plug into the framework the way it's written right now.  You can either supply a PolymorphicConverter / PolymorphicValidator pair in the BindSpec if you need it piecemeal or you can write a different BindSupportFactory that always uses the Polymorphic[Converter|Validator].

I'd love to see this included in the framework.

> > It seems to me that there are two choices:
> > 
> > - Match types at bind time.  (Like what a Haskell RTL does.)
> 
> This is reasonable, as long as the matching algorithm has similar semantics to
> the Java dispatch mechanism.  It should be possible to override the binding
> behavior for specific subclasses without removing the more general behavior for
> superclasses.  Preferably, client code should not need to manage the priority
> list explicitly, since the type system contains sufficient information.
> 
> This option interacts with the usage of arbitrary static metadata, which
> implies an arbitrary priority system.  I don't know how the framework could
> possibly "match types" if the metadata defines a non-standard type system.

Good point.

> > - Don't worry about types at all, just match method signatures when you've got
> > actual objects in hand.  (The Smalltalk type system.)
> 
> The Smalltalk analogy breaks down when the behavior does not actually belong to
> the object being called, but to a third object (e.g. a converter).  I think
> you're referring more to the Smalltalk style conventions, not the type system. 
> Smalltalk (and Obj-C, Python, etc.) encourage you to make calls on
> statically-untyped objects and let the runtime do dynamic dispatch based on
> RTTI.  In this case, we're talking about dispatch to third-party converters,
> etc, but it's a very similar process.  I think this is also a very reasonable
> solution for bindings.  Again, I would want the dispatch mechanism to mirror
> the Java type system.

Yes of course. :-)

> > Is there a third option we've overlooked (assuming Java 1.4)?
> 
> Of course.  :-)  This is an example of the general mixin problem.  The software
> engineering and computer science literature is full of interesting options.
> 
> I think an apt analogy is the Lisp method-dispatch mechanism.  Lisp methods are
> first-class objects that are called by name, without reference to a callee type
> or instance.  The runtime does pattern matching based on the argument types,
> similar to Java method overload matching.  Java semantics can be simulated by
> placing the callee as the first argument of the method.  We would need a
> simplified version of this mechanism that acts on 1 or 2 arguments.  It would
> occupy the space in the framework where factory lists are now.  Option 1 above
> would accept type constraints as arguments at bind-time.  Option 2 would accept
> value instances and use their RTTI.

So basically you're saying we should have both options.

I think that Boris and I agree on this in theory.  The catch has always been to find the simplest possible implementation of this idea that could work. :-)

But you've got me thinking and I'm going to pursue this idea of having an alternate BindSupportFactory that generates polymorphic converters and validators all the time.  I'm interested to see if this approach leads to a simple implementation of the dynamic type system using our existing infrastructure.

Or to say the same thing another way, I'm pretty sure that this approach is on the right track: starting with a stricter type system and weakening it rather than starting with a relaxed type system and trying to strengthen it.

(And you've helped a lot through this discussion. :-)
Comment 27 Dave Orme CLA 2006-06-23 16:53:08 EDT
*** Bug 116349 has been marked as a duplicate of this bug. ***
Comment 28 Boris Bokowski CLA 2007-06-26 14:07:53 EDT
What should we do with this bug?
Comment 29 Peter Centgraf CLA 2007-06-29 14:54:55 EDT
It depends on what happens to the automatic converters and validators.  In essence, the whole factory mechanism was removed from scope for 3.3 API, so this discussion isn't relevant.  All Bindings must have their converters and validators specified explicitly in client code, except for a small handful of primitives and final classes (e.g. String).  I've been generally happy with the change, since it greatly simplified the API, and I reworked most of this functionality myself anyway.  (I've even created custom converters for some of the primitive types.)

However, if 3.4 re-introduces a kind of central repository/factory for converters and validators, this will be relevant again.  Personally, I'm not eager to see more plumbing for this.  Like undo support, it's fraught with application-level design tradeoffs.  If I can instantiate the existing converters and validators with public API, and if those implementations are made a bit more configurable, I'll be happy.  (I've submitted requests for these changes already.)  I can configure my UI behavior better than you can, frankly.

Sooo.... I guess I'm voting for CLOSED/WONTFIX here.
Comment 30 Dave Orme CLA 2007-06-29 16:55:09 EDT
+1.
Comment 31 Boris Bokowski CLA 2007-06-29 22:21:50 EDT
There you are. :)