Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 324701

Summary: Support Systems Dynamics style models.
Product: [Modeling] AMP Reporter: Miles Parker <milesparker>
Component: AMFAssignee: Jonas Ruttimann <jonas.ruettimann>
Status: CLOSED FIXED QA Contact: Jonas Ruttimann <jonas.ruettimann>
Severity: enhancement    
Priority: P3 CC: lukas.schmid
Version: 0.8.0Flags: milesparker: iplog+
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
Prerequisites for system dynamics
none
SD Plugins
none
Prerequisites for model extendability and SD Plugins
none
Prerequisites for model extendability and SD Plugins none

Description Miles Parker CLA 2010-09-07 16:56:29 EDT
This is an "epic-lite" feature to support Systems Dynamics style models in a seamless well integrated way. Le't discuss the overall effort here and then break out bite sized chunks of functionality as appropriate. In general, we'll need:

a) Some kind of meta-model AMF support.
b) Basic editor support for existing EMF tree style editor.
c) Basic Runtime support that from which we can replicate any arbitrary Systems Dynamics style model.

then, either as a follow-on feature, or as part of this main effort, we'll want:

d) A classic Systems Dynamics style editor, just as IMS-FHS has created for Global Simulate.
e) Support for faster runtime execution as has always been envisioned for the MetaABM / AMF project by inferring what can be calculated in O(1) vs. O(t).

This project will be the first contribution of the IMS-FHS team based in part on their SD efforts.

All interested parties are encouraged to provide comments on functionality and implementation.
Comment 1 Miles Parker CLA 2011-02-10 12:27:49 EST
Jonas,

I'm giving this one to you :)
Comment 2 Jonas Ruttimann CLA 2011-02-14 08:11:33 EST
Created attachment 188897 [details]
Prerequisites for system dynamics
Comment 3 Jonas Ruttimann CLA 2011-02-14 08:18:31 EST
Thanks, Miles :)

Support for system dynamic models has reached a point where I think, most of the changes in the interface between AMP and the SD-plugins have been done. I've just uploaded a patch for the master branch on Git. Please take a look. Shall I push these changes to Git?

BTW: SD-Plugins are not ready to be committed yet. This patch is only about the extendability of AMP.

(In reply to comment #1)
> Jonas,
> 
> I'm giving this one to you :)
Comment 4 Miles Parker CLA 2011-02-14 13:37:46 EST
Hi Jonas,

Wow. That's a lot of cool stuff there -- including some good general bits. Especially the dynamic variable looks cool. Let me check on what we need to do with IP -- I don't think we can check in yet but I should hear back on that soon. Clarifying question -- was this all work done after you became a committer or was some of it before hand?

One sort of annoying thing about the Eclipse patches is that they don't ignore whitespace. Since we have the code formatted differently -- I think that's my fault as I hadn't been using pure Eclipse formatter -- the patch is seeing a lot of things changed that really haven't (I think). I guess it would be a good deal of trouble to go and reconstruct them ignoring whitespace and I'm not sure that is even possible, but if you'd like I could do a code cleanup first so that we have exactly the same formatting.

I think we should use the following rule "built-in Eclipse" + 120 character max line width. Eike suggested putting this into all projects so that it will be workspace independent and I think that is a good idea along with making the compiler compatibility changes that you suggested.

cheers,

Miles
Comment 5 Jonas Ruttimann CLA 2011-02-15 12:33:05 EST
(In reply to comment #4)
> Hi Jonas,
> 
> Wow. That's a lot of cool stuff there -- including some good general bits.
> Especially the dynamic variable looks cool. Let me check on what we need to do
> with IP -- I don't think we can check in yet but I should hear back on that
> soon. Clarifying question -- was this all work done after you became a
> committer or was some of it before hand?

Yes, there are quite some bits that should make AMP generally better and more extensible. 

Please note that Dynamic Variables can *not* be added in the metaabm-editor. I've only integrated an Interface for children of Agents so that an SD-plugin can extend the metaabm tree.

All of the work has been done *after* me becoming a committer. No third party code or library has been used. So with IP we should have no problem at all.

> 
> One sort of annoying thing about the Eclipse patches is that they don't ignore
> whitespace. Since we have the code formatted differently -- I think that's my
> fault as I hadn't been using pure Eclipse formatter -- the patch is seeing a
> lot of things changed that really haven't (I think). I guess it would be a good
> deal of trouble to go and reconstruct them ignoring whitespace and I'm not sure
> that is even possible, but if you'd like I could do a code cleanup first so
> that we have exactly the same formatting.
> 
> I think we should use the following rule "built-in Eclipse" + 120 character max
> line width. Eike suggested putting this into all projects so that it will be
> workspace independent and I think that is a good idea along with making the
> compiler compatibility changes that you suggested.

Right. We should definitely use the same formatter and compiler settings. File encoding would also be a nice thing to have configured. That will make collaborative work a lot easier!
Comment 6 lukschmi CLA 2011-02-16 08:05:26 EST
Hi Miles,
what do you think about the following questions and remarks concerning the integration of system dynamics (sd) functionality into amp ?

It is out of question, that we want to contribute to amp. However it is not yet clear what we actually should contribute and how this could work for a most efficient way. There are at least the following two options :

• We add suitable extension points to the amp framework so that an external sd-plugin can be used (this has already been done by Jonas). The sd-plugin itself can be provided as an open-source plugin for amp i.e. via sourceforge. 

• We add the whole sd functionality (i.e. dynamic variables) directly into the amp framework. As consequence every thing will be part of the eclipse project.

The first option may be a little bit easier for us because we do not have to deal with IP problems or coderefactoring (i.e. name conventions). The latter option has the potential that others could be encouraged to contribute to the sd part as well. The question does also have a strategic aspect: does it make sense to extend the agent modeling plattform with extensive sd functionality or would such untertaken demand for a new project orientation in the long-run (i.e. towards a multi paradigm modeling plattform).

Because of limited ressources it is essential for us to contribute as much as possible to the modeling crowd in a direct, efficient way.

cheers,
Lukas
Comment 7 Miles Parker CLA 2011-02-16 13:58:55 EST
Thanks Lukas,

Here are my thoughts about this..

(In reply to comment #6)


>• We add suitable extension points to the amp framework so that an external
sd-plugin can be used (this has already been done by Jonas). The sd-plugin
itself can be provided as an open-source plugin for amp i.e. via sourceforge. 

I think this would work fine and perhaps could be a good first step. Such extensions would need to be general enough that arbitrary frameworks could plug-in. OTOH, it would miss a lot of opportunity to have the different pieces inform and interact with each other. OTOHOH, there is benefit in that it might push toward lower coupling. It is hard to say without diving into specifics whether it would be good or not, but then we get into cross-project management issues which would be a concern. Questions like "are we building this because AMP needs it or because an outside project needs it?" It is ok to answer the latter in some cases, but it is a slippery slope.

>• We add the whole sd functionality (i.e. dynamic variables) directly into the
amp framework. As consequence every thing will be part of the eclipse project.

Well, that's the one that I like. But I don't want to be pushy. :)

> The first option may be a little bit easier for us because we do not have to
> deal with IP problems or coderefactoring (i.e. name conventions). The latter

Let's try to get a handle on IP problems, as that will drive everything else. Are your concerns to do with your own code base -- i.e. getting organizational approval, getting sign-off from all of the committers, etc.. -- or with third party libraries? For the former, we can prob. pretty easily determine whether there will be any issues with that. For the latter, if you can give me a list of 3rd party components I can check that against Orbit -- the set of third-party libs already blessed by Eclipse. If everything is on that list then we should be in good shape.

> option has the potential that others could be encouraged to contribute to the
> sd part as well. The question does also have a strategic aspect: does it make
> sense to extend the agent modeling plattform with extensive sd functionality or
> would such untertaken demand for a new project orientation in the long-run
> (i.e. towards a multi paradigm modeling plattform).

Well, I think that the AMP project has always been a multi-paradigm platform, in the sense that I think the methodology is naturally suited to ABM, SD, Discrete event and perhaps other approaches. And actually I think getting this right is what would make AMP a really home run project. As we've discussed, there are technical and business issues here. In the case where AMP included much more significant SD functionality would you see the strategic hurdles being mostly technical and compatibility issues or branding and governance issues, or both?

> Because of limited ressources it is essential for us to contribute as much as
> possible to the modeling crowd in a direct, efficient way.

Yes, I agree completely. That's why I made the decision to get involved in Eclipse community a few years ago rather than trying to develop a fully independent project. I think that the open governance model, transparency, common licensing, etc.. overcome the issues that prevent people from different organizations to work together toward a common goal or that create so many annoyances that nothing gets done. The thing that is a little difficult to get I think looking form the outside in is that while it *seems* that there are a lot of barriers -- IP issues, project management rules, etc.. -- after becoming involved in Eclipse projects, you see most of those perceived barriers as actually extremely helpful to getting things out efficiently. Basically, using biology analogy, Eclipse is like a semi-permeable barrier that allows ideas and benefit to flow in and out, but keeps internal cohesion and organization.

To me the second way seems a lot neater. You might know that I am already in this situation, because Ascape cannot be made EPL. That means that I need to coordinate between a Sourceforge project and an Eclipse project, and it is a pain that slows down or even dissuades me form doing certain things -- and that is for a project that up to recently (Oliver Mannion is now a contributor as well) I have been solely responsible for.

Anyway, my suggestion is to dig into where the specific concerns are re: Eclipse and try to address them. If we can't address all of them, then we'll figure out a second best solution.
Comment 8 Jonas Ruttimann CLA 2011-02-18 11:48:15 EST
Created attachment 189295 [details]
SD Plugins

These plugins extend AMP with SD functionality. In the metaabm tree editor you'll find an additional element.

Todo: Plugins need to be called "org.eclipse..."
Comment 9 Miles Parker CLA 2011-02-18 12:16:27 EST
Awesome! Now, we certainly will need a CQ for this one, correct? i.e. it contains code that was created before you became committer and/or that was written by other members of your team?
Comment 10 lukschmi CLA 2011-02-19 04:31:32 EST
Hi Miles,

thanks for your answer - these are very good news!

In this case we will head for an extension of amp with fundamental sd functionality. At the moment we see neither any IP nor technical problems. The only a little bit annoying thing is the code refactoring to the "org.eclipse..." notation.

The code so far provided by Jonas has been programmed by himself after he got committer status. However it will probably be suitable that Urs (for code contribution) as well as Corinne and myself (for strategic and modelling contribution) will get this status as well.

The branding of "amp" as an agent modelling toolkit is certainly justified for the moment. If the futur will bring more and more functionality belonging to other paradigms (like sd or discrete event...) we can still discuss about the label of the tool. 

I think the only point we can have a look is the naming of general things (i.e. "agent model" or "agent modeling perspective" or "agent execution perspective"). I think it could be less confusing for sd-modellers if this names are more general regarding the multi-paradigm orientation. What do you think about leaving the word "agent" away? What would be the effort of doing this?

cheers, lukas
Comment 11 Miles Parker CLA 2011-02-21 16:15:02 EST
(In reply to comment #10)
> In this case we will head for an extension of amp with fundamental sd
> functionality. At the moment we see neither any IP nor technical problems. The
> only a little bit annoying thing is the code refactoring to the
> "org.eclipse..." notation.

Yes, I've been through that. The best I can suggest is to do it in the following order.

1) Disconnect from version control if you are using SVN because it makes a huge mess.
2) Rename and move all projects using refactoring tools.
3) Do refactoring for package names. (That will take care of manifest dependencies.)

It really isn't that hard, but let's agree on package names first. My thought is to use org.eclipse.amp.sd hierarchy for everything that is directly plugin related but not EMF based, and org.eclipse.amp.amf.sd for everything that is model based or dependent. Things that appear more general could be in org.eclipse.amp.* with whatever sub project (agf,axf,..) makes the most sense.

If you'd like we can create an interim Eclipse Labs project so that we can make sure integration is working before committing everything to main project.

> The code so far provided by Jonas has been programmed by himself after he got
> committer status. However it will probably be suitable that Urs (for code
> contribution) as well as Corinne and myself (for strategic and modelling
> contribution) will get this status as well.

re: Urs, perhaps he can start with making bug reports and patches and so on and then we'll move ahead as his role develops. As discussed with Jonas coming on to project, it's important to follow a well defined consistent path of committer involvement so that we have an established pattern for future committers within Eclipse community that is even-handed and that fits in with the way other projects do things. Not to discourage of course! re: you and Corrine, it sounds like you're referring to a different kind of involvement than directly committing code - pls correct me if I'm wrong. For Eclipse there are many ways for people to become involved in a project -- because Eclipse projects themselves are very much software driven the "committer" status is setup for people that expect to contribute code on a day to day, week to week basis. Basically the philosophy is that ultimately we vote with our feet, so that the more code development resources an organization puts in, naturally the more project direction they will have since all of us do what we do based on how well it fits in with our organizations broader goals.

There are many other ways to have a high level of support and input into projects. First of course, there are bugs, mailing list, forums, etc.. That's actually very important because it is how the project agenda overall is set. As a contributor you can also submit modeling example projects, documentation, etc.. (And if there is a significant amount of those sort of artifacts then committer status might be appropriate as well.)

Another very important option way to be involved at an organizational level is through an Industry or interest working group. We've done some work last year toward setting up something like that for Social Science Modeling.  Another is an organizational member of Eclipse Foundation. That involves a financial contribution to the foundation itself, but it might be worth considering. Let me know if you want more info on any of this but my thought is that this can evolve.

> The branding of "amp" as an agent modelling toolkit is certainly justified for
> the moment. If the futur will bring more and more functionality belonging to
> other paradigms (like sd or discrete event...) we can still discuss about the
> label of the tool. 

Yes, and I have been thinking a lot about this recently. Of course a big part of effort is simply refactoring, name recognition, project configuration logos, etc.. etc.. One of the major issues in the Eclipse community is that the word "Modeling" itself is so overloaded as to be meaningless. And as you know, the word "Agent" in the software world is also hugely overloaded. Especially in social and computer science it can mean anything from a single agent that acts as an avatar for a user to very light-weight models and includes agents representing real systems, agents managing software systems, software using agents to interact with other pieces of software, etc..

But one thought I have had partly because this is philosophically where I've been thinking is to consider a name change to "Active Modeling Project". This fits in with having a broader set of tools of interest to people who aren't even doing Modeling and Simulation. For example business process or defining higher-level software behavior. People often refer to the latter as "Model Execution". I also like "Artificial Modeling Project". ;) The big advantage of these is that there is no need to change most of the project stuff. :)

> I think the only point we can have a look is the naming of general things (i.e.
> "agent model" or "agent modeling perspective" or "agent execution
> perspective"). I think it could be less confusing for sd-modellers if this
> names are more general regarding the multi-paradigm orientation. What do you
> think about leaving the word "agent" away? What would be the effort of doing
> this?

See above. Some thoughts. First, on the modeling side, it might make sense to actually have a "Systems Dynamics" perspective or something. That's because perspectives are really just ways to organize windows and so on, and someone who is primarily focussed on SD might want things set up in a different way. For example, there will be different editors and so on. Second there are some cases where "Agent Modeling" makes sense, and of course one of the primary targets of the tools is still "Agent-Based" Modelers. And again Modeling is far too over loaded. If we make change above then it could be "Active Modeling" but that might be confusing. OTOH, I can see value in integrating the tools completely. I think that on the execution side it is a little simpler, perhaps we could have a "Model Execution" but again I find that a bit confusing. Let's give some more thought to this one.

More to follow..
Comment 12 Miles Parker CLA 2011-02-21 16:22:43 EST
I wanted to do a separate thread for IP issues to make things clearer for Eclipse legal team.

(In reply to comment #10)
> The code so far provided by Jonas has been programmed by himself after he got
> committer status. However it will probably be suitable that Urs (for code
> contribution) as well as Corinne and myself (for strategic and modelling
> contribution) will get this status as well.

OK, does that include the second attachment in comment 8? What about the SD diagram editor, execution engine and so on? I'm hoping that you would be able to contribute that of course and I understand that that is based on a code base that has already been significantly developed?
Comment 13 Miles Parker CLA 2011-02-24 14:02:39 EST
I don't want to be a pest, but I'm pinging on this as I'd like to get the CQ packaged up for Eclipse IP asap.

cheers,

Miles
Comment 14 Jonas Ruttimann CLA 2011-02-25 01:50:45 EST
Sorry for the delay. I'm on it!

(In reply to comment #13)
> I don't want to be a pest, but I'm pinging on this as I'd like to get the CQ
> packaged up for Eclipse IP asap.
> 
> cheers,
> 
> Miles
Comment 15 Miles Parker CLA 2011-02-25 02:01:20 EST
Sorry Jonas, my fault. I thought that I hadn't received an answer on the second attachment, but I had. So I'll check with EMO to be certain, but my sense is that we will not need a CQ for this.

I was thinking that the SD Plugins contained some of the stuff from the global simulate project, but I am realizing now that this is not the case. So to be totally clear, do you anticipate any of that earlier code, i.e. runtime, graphical editing tools, etc.. making their way into AMP?
Comment 16 lukschmi CLA 2011-02-28 10:43:37 EST
> So to be
> totally clear, do you anticipate any of that earlier code, i.e. runtime,
> graphical editing tools, etc.. making their way into AMP?

Hi Miles,

as far as we can say now, there will be no code contribution, which originally comes from a previous project. Certainly most of the ideas have been developed and thought about a long time but the corresponding code has and will be written especially for the amp project.
Comment 17 Miles Parker CLA 2011-02-28 16:24:58 EST
OK, I think we're good to go. Reading http://www.eclipse.org/legal/EclipseLegalProcessPoster.pdf it is quit clear that we meet the criteria for Figure 1, Figure 7, Figure 8.

I can only make one suggestion. Consider changing the names to Eclipse namespace *before* submitting -- and then attach code here again of course -- as it will be much more difficult to do project level refactoring once it get's into git.
Comment 18 Jonas Ruttimann CLA 2011-03-16 04:46:05 EDT
Created attachment 191282 [details]
Prerequisites for model extendability and SD Plugins

Please review this patch. If there are no objections, I will be ready to push these changes to the repository.

First this patch contains all prerequisites for AMP model extendability. Secondly, it contains 5 new plugins that extend the AMP model with very fundamental SD features. If these plugins are loaded, a MetaABM tree will have the ability to add a container for "Dynamic Variables". Elements of this container are "Aux", "Flow" and "Stock". Each of these "Dynamic Variables" can have a connection to another "Dynamic variable". When creating a new Escape project, an additional builder is being added to the projct to generate the necessary Java code.
Comment 19 Miles Parker CLA 2011-03-16 14:03:35 EDT
Hi Jonas,

I'll look at them right away. This is coincidental timing as we today is the Indigo +3 M6 build, Eclipse talk for "last chance to make major API changes". We could always wait for M7 and try to make an exception but it would be really nice to get this in under the wire. So that's what I'm going to try to do. I hope you don't mind, but if all looks good, I may go ahead and push from this end in order to test the build and get everything in before the close of business here, but why don't you send me or the list an email when you get in in the morning -- if history serves I'll still be frantically trying to get the build up and going.

cheers,

Miles
Comment 20 Miles Parker CLA 2011-03-17 00:09:31 EDT
For other annoying build related reasons that you'll see on the list, I wasn't able to get any of this out for Indigo M6. Now I'm dealing with trying to get those issues sorted out. So for everyone's sanity -- I don't want to pick now to figure out how git merge works :) -- please wait until I get that build stuff worked out before uploading these changes. I'll let you know just as soon as I do but it might not be until tmrw.
Comment 21 Jonas Ruttimann CLA 2011-04-12 12:06:12 EDT
Created attachment 193063 [details]
Prerequisites for model extendability and SD Plugins

This time the patch builds on the current Git version, pulled by Buckminster. So you should be able to apply the patch to your version. I'm ready to commit.
Comment 22 Miles Parker CLA 2011-04-12 12:10:19 EDT
Please do! :)
Comment 23 Jonas Ruttimann CLA 2011-04-13 10:02:40 EDT
Committed!

(In reply to comment #22)
> Please do! :)