Bug 366706

Comment 1 Joseph Vincens 2011-12-28 08:06:52 EST

Created attachment 208821 [details]
IBMi program call specification

Comment 2 Dan Darnell 2011-12-28 17:09:12 EST

Hi Joe V. Really intriguing work. I love the way that RBD works today (it does ALL the work, basically) for making program calls but in a world where structured records don't exist and a fixed-length character type doesn't exist (etc.) then I see where an annotation-based approach makes sense. I am much happier with this approach than with the notion of using PCML/XPCML (except that a wizard might someday (optionally) use PCML to generate the annotations).

I have a few questions:

1. I don't see much support for structures. These are very common as parameters to ILE programs. Please take a look at the example in my article:

http://www.systeminetwork.com/article/rpg-programming/the-egl-has-landed-calling-rpg-programs-from-egl-62294

Figure 4 shows an RPG program with a parameter that is a structure which, in turn, contains an array of structures which also contains an array of structures. In short, structures, nested structures, and arrays of structures are extremely common as program parameters. I did a review here of our EGL-to-RPG program calls and 97% of them use parameters that are structures, arrays of structures, or both.

There is mention on page 18 of the document of the AS400Structure type but I can't tell from what is written whether an example such as the one in my article will work or how one would code for it. Thoughts?

2. Do I understand correctly that the use of "validElementCountVariable" is optional? The other option being to specify an "elementCount" on the array definition?

3. Do stateful and stateless calls still come into play somehow? It is less important to me today since, in the rich UI world, everything we code uses stateless calls. But I know that there are ongoing discussions on the forum about stateful calls to service programs and such even from Rich UI. Just wondering what the thinking is about this.

4. Today we use a build descriptor linkage option like this:

<callLink >
<remoteCall pgmName="*" conversionTable="CSOE037" library="*LIBL" location="PROGRAMCONTROLLED" luwControl="SERVER" remoteBind="GENERATION" remoteComType="JAVA400" remotePgmType="STATELESS"/>
</callLink>

Clearly, there is no longer a pgmName="*" equivalent. Each program will require a function defined with an @IBMiProgram annotation.

In order to accomplish the library="*LIBL" equivalent I am assuming that the following is kosher under this EDT proposal:

libraryName="/QSYS.LIB/*LIBL.LIB/"

Correct?

With that same build descriptor we determine program location at runtime with the following:

sysvar.remoteSystemId = system;	

Is there some way to do this under this EDT proposal or is the program location purely a function of the DD setup?


My primary concern though is with the structures and array of structures so if you can shed any light on that topic I would appreciate it very much.

Regards,

Dan

Comment 3 Joseph Vincens 2012-01-03 10:44:34 EST

Thanks for the comments and questions. I've embedded my answers with the questions.
(In reply to comment #2)
> Hi Joe V. Really intriguing work. I love the way that RBD works today (it does
> ALL the work, basically) for making program calls but in a world where
> structured records don't exist and a fixed-length character type doesn't exist
> (etc.) then I see where an annotation-based approach makes sense. I am much
> happier with this approach than with the notion of using PCML/XPCML (except
> that a wizard might someday (optionally) use PCML to generate the annotations).
> 
> I have a few questions:
> 
> 1. I don't see much support for structures. These are very common as parameters
> to ILE programs. Please take a look at the example in my article:
> 
> http://www.systeminetwork.com/article/rpg-programming/the-egl-has-landed-calling-rpg-programs-from-egl-62294
> 
> Figure 4 shows an RPG program with a parameter that is a structure which, in
> turn, contains an array of structures which also contains an array of
> structures. In short, structures, nested structures, and arrays of structures
> are extremely common as program parameters. I did a review here of our
> EGL-to-RPG program calls and 97% of them use parameters that are structures,
> arrays of structures, or both.
> 
> There is mention on page 18 of the document of the AS400Structure type but I
> can't tell from what is written whether an example such as the one in my
> article will work or how one would code for it. Thoughts?
> 

Structures will be modeled as records and sub structures will be broken out as individual records. I'll use your example to illustrate. My RPG is weak at best, I'm assuming numbers (num(2, 0)) are packed, if they are not an AS400DecimalXXX annotation would need to be added to each number. It looks like figure 5 is the RBD EGL equivalent of the RPG data structure:
record rootStruct type BasicRecord
      05 rootName char(20);
      05 branches [50]; 
         10 branchNbr num(2, 0); 
         10 branchName char(20);
         10 items [25]; 
            15 itemNbr num(2, 0); 
            15 itemName char(20);
end

Here is the equivalent EDT EGL:
record rootStruct 
      rootName string(20);//05 rootName char(20);
//create a record for the substructure
      branches Branch[] = new Branch[50];//05 branches [50]; create a record for the substructure
end

record rootStruct 
      rootName string{@AS400Text{length = 20}};//05 rootName char(20);
//create a record for the substructure
      branches Branch[]{@AS400Array{elementCount = 50}};//05 branches [50]; 
end

record rootStruct 
      rootName string{@AS400Text{length = 20}};//05 rootName char(20);
//create a record for the substructure
      branches Branch[];//05 branches [50]; 
end

record Branch 
    branchNbr decimal(2,0);//10 branchNbr num(2, 0); 
    branchName string(20);//10 branchName char(20);
//create a record for the substructure
    items Item[] = new Item[25];//10 items [25]; 
end

record Item 
    itemNbr decimal(2,0);//15 itemNbr num(2, 0); 
    itemName string(20);//15 itemName char(20);
end

The same records are used in JavaScript and Java generation. 
I've shown 3 different implementations of rootStruct.branches (the same could to Branch.items) the first initializes the array to a size of 50 the second uses the @AS400Array annotation to initialize the array size, and the 3rd leaves the array size to the application. 
All are correct and they really depend on the application. I'll explain the differences so you can choose what's best for you. 
The first will create an array that is always xx elements whether they are populated or not. So when ever a rootStuct is instantiated (even in JavaScript gen) you will get an array of 50. 
The second will only create an array of 50 during the conversion to AS400 data. The 3rd is the weakest because there is no indication that the array actually has a specific size and it's up to the application to create the correct number of elements.
If the array is always 50 I would recommend the first, if the array may contain empty data or is a large structure , I would suggest the second as the application will create the number of elements and the conversion to AS400 data will resize it. The 3rd option is the most risky as it requires the application programmer to always create the correct array. 



I also showed that parameterized strings (string(20)) or non parameterized string{@AS400Text{length = 20}} can be used. Again either is correct and is a matter of preference. 


> 2. Do I understand correctly that the use of "validElementCountVariable" is
> optional? The other option being to specify an "elementCount" on the array
> definition?
> 
validElementCountVariable and elementCount have 2 different purposes.

validElementCountVariable is optional and only used when the host program returns data back to Java. validElementCountVariable is used for "variable size arrays". I've seen some applications that have an array parameter and a number parameter. The number defines the number of valid elements in the returned array. The EDT runtime uses the variable specified by validElementCountVariable to generate code that will resize the returned array. So if elementCount = 10 and the RPG program retrieves only 6 valid rows, the validElementCountVariable would be 6 and the runtime would resize the array to 6.

elementCount is also optional (I need to add ? to the spec). This is used to size/resize the array before it is converted to AS400 data. This would be used if an array is only a return and the application did not create an array with the correct number of elements. The AS400 toolkit requires the proper size buffer be created and sent to the host. elementCount will create the correct number of empty elements. If you don't include an elementCount the application is required to create the correct size array.


> 3. Do stateful and stateless calls still come into play somehow? It is less
> important to me today since, in the rich UI world, everything we code uses
> stateless calls. But I know that there are ongoing discussions on the forum
> about stateful calls to service programs and such even from Rich UI. Just
> wondering what the thinking is about this.
> 

Yes. When you use a AS400Connection Resource type connection you can specify getConnectionFromSessionPool. This will get a connection based on the session ID. I had thought of calling the flag persistentConnection, but I wanted a  more descriptive name (should I change this?). If you are operating in a non J2EE environment there is no session, so an exception would be thrown.

> 4. Today we use a build descriptor linkage option like this:
> 
> <callLink >
> <remoteCall pgmName="*" conversionTable="CSOE037" library="*LIBL"
> location="PROGRAMCONTROLLED" luwControl="SERVER" remoteBind="GENERATION"
> remoteComType="JAVA400" remotePgmType="STATELESS"/>
> </callLink>
> 
> Clearly, there is no longer a pgmName="*" equivalent. Each program will require
> a function defined with an @IBMiProgram annotation.
> 
> In order to accomplish the library="*LIBL" equivalent I am assuming that the
> following is kosher under this EDT proposal:
> 
> libraryName="/QSYS.LIB/*LIBL.LIB/"
> 
> Correct?
> 
> With that same build descriptor we determine program location at runtime with
> the following:
> 
> sysvar.remoteSystemId = system;    
> 
> Is there some way to do this under this EDT proposal or is the program location
> purely a function of the DD setup?
> 
> 

> libraryName="/QSYS.LIB/*LIBL.LIB/"
> 
> Correct?
Yes, but this is an area that should be discussed. See below.

> Is there some way to do this under this EDT proposal or is the program location
> purely a function of the DD setup?
maybe

The AS400Connection Resource egldd definition is similar to the linkage in RBD. There are differences:
1. There is no link between a program name and it's linkage (you manage the connections by specifying connectionFunction or ConnectionResource).
2. The conversion table is controlled either by the connection or by specifying a specific encoding on a string field.
3. Location can be controlled problematically by specifying a connectionFunction. The function must return an AS400 object. This gives you more control on how connections are made not only where they are located. 
4. Since there is no EGL host runtime LUW must be controlled by your code.
5. There is currently no library on the AS400Connection Resource, library is on the program definition. Someone has already made a verbal comment that this may be wrong because during development the program is from 1 library, during test it's from a 2nd, and in production it's a 3rd. So I'm thinking of moving library to the AS400Connection Resource definition and removing it from the IBMiProgram annotation. 

A connection does not have to be a function of the DD. By providing a connectionFunction you can write a function that creates a new AS400 object, or gets a connection from the EDT EGL JTOpen connection pool, or gets a connection from the EDT EGL session connection pool, or does a SysLib.getResource to get a connection. If you have different resources for different environments (ie development, test, production) you can't use connectionResource you must use a connectionFunction. 

BTW there is an error in the IBMiAnnotation definition it has connectionMethod but the text uses connectionFunction. The annotation will be changed to connectionFunction.

> My primary concern though is with the structures and array of structures so if
> you can shed any light on that topic I would appreciate it very much.
> 
> Regards,
> 
> Dan


thanks,
Joe

Comment 4 Dan Darnell 2012-01-03 15:16:04 EST

Hi Joe,

Thanks for the detailed answers. I like the approach on the arrays and structures. It is a significant improvement over the way we work in RBD that the same records can be used for Java and JavaScript generation in EDT! Nice too that there are options for indicating array sizes and such. A great deal of flexibility there.

[Joe]"This will get a connection based on the session
ID. I had thought of calling the flag persistentConnection, but I wanted a 
more descriptive name (should I change this?)."

No strong opinion one way or the other about what you end up calling it. I like "getConnectionFromSessionPool" and I'm sure the purpose of it will be documented.

[Joe]"So I'm thinking of moving library to the AS400Connection Resource definition and removing it from the IBMiProgram annotation."

There are situations where under one connection it will be desirable to call programs in different libraries. Library at the IBMiProgram annotation level is therefore essential. I wouldn't move it. If you add a library to the resource definition then it should act as a default with the IBMiProgram annotation acting as an override.

Now, in practice, it is typical to use the library list and not hardcode program locations (hence the importance of libraryName="/QSYS.LIB/*LIBL.LIB/") but I still say that having the Toolbox-like ability to specify a library name (or *LIBL) at the program level is going to be important.

As for "connectionFunction" -- it sounds a lot more versatile than I originally thought when reading the document. I need to wrap my head around it a bit.

--Dan

Comment 5 Joseph Vincens 2012-02-07 13:59:52 EST

Changes that need to be included in the spec.
- a Call statement will be required for Java generated applications to access host programs.
- Annotation AS400Array.validElementCountVariable changes to AS400Array.returnCountVariable.
- Annotation AS400Date.ibmiSeparator changes to ibmiSeparatorChar.
- Annotation AS400Time.ibmiSeparator changes to ibmiSeparatorChar.
- Annotation IBMiProgram: connectionFunction is removed, connectionResource is replaced with connectionResourceBindingURI string.
- Add Annotation AS400Float4 only valid on smallfloat.
- Add Annotation AS400Float8 only valid on float.

- JNDI will not be supported in 0.8.0.
- EGLDD getConnectionFromSessionPool is removed library string is added.

Comment 6 Joseph Vincens 2012-02-14 15:23:44 EST

Created attachment 211009 [details]
IBMi program call specification 0.8.0

Updated with various annotation changes
Added call statement to support passing the connection.

Comment 7 Joseph Vincens 2012-02-23 11:22:58 EST

Created attachment 211509 [details]
IBMi program call specification 0.8.1

Comment 8 Joseph Vincens 2012-02-23 11:24:24 EST

Created attachment 211510 [details]
IBMi program call specification 0.8.1

The page numbers on the index were not correct.

Comment 9 Joseph Vincens 2012-03-29 09:03:24 EDT

implemented

Comment 10 Lisa Lasher 2012-04-03 23:51:22 EDT

Set IPLog flag for the entire bug, to record Dan Darnell's contribution to the design.