Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 367261

Summary: Extend New Service and Library wizards for database access
Product: z_Archived Reporter: Will Smythe <smythew>
Component: EDTAssignee: Song Fan <songfan>
Status: CLOSED FIXED QA Contact:
Severity: enhancement    
Priority: P1 CC: chenzhh, jinfahua, mayunf, opepermans, pharmon, songfan, svihovec, tdramsey, tww
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on: 370388, 370672, 370910, 367259, 367260, 370395, 370673    
Bug Blocks: 363081    
Attachments:
Description Flags
Prototype code showing possible service and records
none
The Slides and meeting minutes of the requirement & deisgn meeting.
none
The Slides and meeting minutes of the requirement & deisgn meeting.
none
patch lasher: iplog+

Description Will Smythe CLA 2011-12-20 21:38:47 EST
See Bug 363081 for more details. The following was extracted (and updated) from 363081, and can provide a basis for the design:

I think this could be implemented as an extension to the New EGL Service (and Library) wizard, much like the New EGL Record wizard has a set of 'templates' you can choose from. The flow might look like this:

1) Developer launches the New EGL Service wizard
2) Developer supplies an EGL part name (MyNewService) and selects a package.
3) Developer chooses the 'Database access' template from the list of templates,
and clicks Next.
4) Developer selects a database connection and then selects 1 or more database
tables (this wizard panel would resemble the wizard panel that already exists
in the New EGL Record Wizard) and clicks Next.
5) The source for the new service (and new Entity record) is displayed in the
Preview page. The developer clicks Finish to create the new service file, which
includes the service part and entity record(s).

The source for the service would look *similar* to the service code
generated by the RBD Data Access Application wizard today (except there would
be 'repeated' functions if more than 1 table is selected). For example: if the
developer had selected the ORDER and CUSTOMER tables, 1 service part would be
created (and it would be named whatever it was named in the first page of the
wizard), but it would contain functions like getAllOrders() and
getAllCustomers() and deleteOrder() and deleteCustomer().

The service should support a StatusRec-like record (to provide status of a
functional call back to the caller) and should also provide a function to fetch
x number of rows starting at a specific index (in order to support paging
through a long table).
Comment 1 Brian Svihovec CLA 2012-01-05 15:23:34 EST
Due to the fact that this line item is intended to cover a high level topic, I believe it should only be targeted at the .8 milestone (assuming we think all of the work will be finished in the .8 time frame).  As we design this feature, new work items can be added and targeted at a specific iteration.  

Technically, I would prefer it if we did not target these high level features at any milestone, since it is possible that extending the Service and Library wizards might cross multiple releases.  

If you agree with this change, please target this item at the .8 milestone.  If you do not agree, please post your comments here.
Comment 2 Tony Chen CLA 2012-01-09 01:12:47 EST
I
Comment 3 Tony Chen CLA 2012-01-09 01:13:45 EST
Brian, I agree this should be set to 0.8 milestone, and separate bugs might need to be created for more detailed tasks.
Comment 4 Tony Chen CLA 2012-01-09 10:34:03 EST
By looking at the description, it seems we are going an approach to add the Database Application support into each wizard. We already have Record wizard which can created Record from database, now service and library wizard are going to be enhanced the same way. There might be some usability issues since the functions are separated. 

Scenario 1. When creating a service from a DB table, where does the Record come from? Will the service wizard duplicate some of the Record wizard function or actually launches the Record wizard? What if user has already created Record somewhere? 

Scenario 2. We already have a database app created using the Record Wizard, Service Wizard & Library Wizard. Now a new column is added to the table. How do we update our app to use the new column? Should user run each Wizard again?
Comment 5 Tony Chen CLA 2012-01-09 10:36:53 EST
Tim has talked about the concept of modeling an "Application Design Pattern" with EGL in one of his emails, and said the Data Access Wizard is the perfect place to show how Application models are defined and fit in with data models and how these separate patterns. 

I'm adding Tim to this bug
Comment 6 Brian Svihovec CLA 2012-01-27 14:25:09 EST
This work is considered one of the main themes for the .8 release.
Comment 7 Will Smythe CLA 2012-01-31 08:04:04 EST
In response to Comment 5 - regardless of whether we come up with an application type that allows a developer to model a data access app, developers still need the ability to create a conventional EGL service and library to access data in a database. So, I think this application pattern requirement needs to be broken out from this bug and discussed independently.
Comment 8 Will Smythe CLA 2012-01-31 08:07:21 EST
Created attachment 210309 [details]
Prototype code showing possible service and records
Comment 9 Will Smythe CLA 2012-01-31 08:07:58 EST
I started prototyping what I thought the service might look like (see attached). It's based on the service produced in the RBD Data Access (DAA) wizard, but I tried to come up with a solution for the way we return 'status'. I don't like how the current DAA-produced service forces the developer into passing either an empty object/array or Statusrec on the call statement - this is not logical to me (and will confuse new users). If the developer wants all Employee records (for example), the API should be 'getEmployeeListAll() returns (Employee[])'. Of course, we should also provide an API that returns back status information, since this is important in some situations/apps.

The attached code is nowhere near complete, but a starting point. It demonstrates an approach where there are duplicate APIs, one that returns a concrete type (e.g. Employee), another that returns a Results, that includes status and the data being requested. 

(Fan - this is the same code I sent to you back in December via email)
Comment 10 Song Fan CLA 2012-02-02 01:26:06 EST
Created attachment 210424 [details]
The Slides and meeting minutes of the requirement & deisgn meeting.
Comment 11 Song Fan CLA 2012-02-02 01:34:16 EST
Created attachment 210425 [details]
The Slides and meeting minutes of the requirement & deisgn meeting.
Comment 12 Song Fan CLA 2012-02-02 01:36:07 EST
Hi Will and Brian,

According to the discussion yesterday, I update the slide and add the Solution/TODO information (in BLUE box) in the document. Would you please review it?

Appreciate for all of your help very much!
Comment 13 Brian Svihovec CLA 2012-02-02 15:28:29 EST
The latest design looks good to me.  I agree with trying to re-use as much code as possible from the existing SQL Record Wizard.
Comment 14 fahua jin CLA 2012-02-03 02:36:20 EST
(In reply to comment #13)
> The latest design looks good to me.  I agree with trying to re-use as much code
> as possible from the existing SQL Record Wizard.

Today when we review the design for how to generate the data access code after user selects the table, Tony proposed a good approach to generate the service code.

Our original design is that generate the service code from Java object (so-called the Java to EGL code generator). 

Tony's approach is to use the EGL to EGL generator, it has following steps.
1) Re-use existing code of record wizard to generate the record from table.
2) The EGL record code is compiled to IR model.
3) Read the IR model & generate to the EGL service code.

The advantage for the 2nd proposal is that, the code for generating the record to EGL service (the 3rd step) can be re-used if we want to add a wizard template for 'creating service from SQL records'. Also, maybe it will have some benefits when we want to add other record to EGL service wizard (E,g, generate the service for the record generated from XML/JSON etc).

Do you have any comments?
Comment 15 fahua jin CLA 2012-02-03 03:14:00 EST
(In reply to comment #14)
> (In reply to comment #13)
> > The latest design looks good to me.  I agree with trying to re-use as much code
> > as possible from the existing SQL Record Wizard.
> 
> Today when we review the design for how to generate the data access code after
> user selects the table, Tony proposed a good approach to generate the service
> code.
> 
> Our original design is that generate the service code from Java object
> (so-called the Java to EGL code generator). 
> 
> Tony's approach is to use the EGL to EGL generator, it has following steps.
> 1) Re-use existing code of record wizard to generate the record from table.
> 2) The EGL record code is compiled to IR model.
> 3) Read the IR model & generate to the EGL service code.
> 
> The advantage for the 2nd proposal is that, the code for generating the record
> to EGL service (the 3rd step) can be re-used if we want to add a wizard
> template for 'creating service from SQL records'. Also, maybe it will have some
> benefits when we want to add other record to EGL service wizard (E,g, generate
> the service for the record generated from XML/JSON etc).
> 
> Do you have any comments?

One another problem is that we don't have the EGL record files generated in the project before finishing the service creation wizard. However, we want to display the code of service in the last page of the wizard. So it will have problem for the 2nd step - we don't have the IR model for EGL records yet, so the EGL service code cannot be generated as well. We need to get the EGL record IR model in the memory, and then generate the service code.

So one question for Paul, do we have any methods for passing the string value of EGL file contents, and the returned value is IR model?
Comment 16 Will Smythe CLA 2012-02-03 09:19:01 EST
What details of the record does the SQL service/library code actually need? I would hope the service/library could be as generic as possible (other than function names that match the name of the record, e.g. getEmployee). Is knowledge about the fields of the record needed during creation of the service/library code? 

(To say it another way, I would think the fully-qualified record part name and the  names of the key fields are the only input to the service creation code)
Comment 17 Tony Chen CLA 2012-02-03 09:36:14 EST
The only one place that required information besides record name and id field is 

function getEmployeeListWithStatus(listSpec ListSpecification) returns (EmployeeListWithStatus)
   ...
   prepare selectStatement from dataSource
                        with "SELECT EMPNO, FIRSTNME, MIDINIT, LASTNAME, WORKDEPT, PHONENO, HIREDATE, JOB, EDLEVEL, SEX, BIRTHDATE, SALARY, BONUS, COMM " +
                                listSpec.fromTablesWhere;


In this sample code, all fields (or table column) is needed. 


(In reply to comment #16)
> What details of the record does the SQL service/library code actually need? I
> would hope the service/library could be as generic as possible (other than
> function names that match the name of the record, e.g. getEmployee). Is
> knowledge about the fields of the record needed during creation of the
> service/library code? 
> 
> (To say it another way, I would think the fully-qualified record part name and
> the  names of the key fields are the only input to the service creation code)
Comment 18 Will Smythe CLA 2012-02-03 09:39:49 EST
Does "select * " + listSpec.fromTablesWhere not work?
Comment 19 Song Fan CLA 2012-02-09 07:49:53 EST
Hi Will,

Another question about the design:
If there are 2 tables with the same name in different schema, the generator will generate 2 EGL record with the same record name, then a compile error will be reported.

Should we add some restriction to do not allow developer select tables from different schema, or check if there are 2 tables with the same name before going to the preview page?

Thanks very much!
Comment 20 Will Smythe CLA 2012-02-09 08:45:01 EST
I believe we decided to put all the functions for get, add, update, etc for multiple tables in the same service/library. If so, I think we should prevent the developer from selecting multiple tables/views with the same name in the wizard (by showing an error on the wizard page and preventing next/finish). I think this scenario is unlikely, so I don't think this approach is overly aggressive.
Comment 21 Song Fan CLA 2012-03-05 03:20:59 EST
Created attachment 212049 [details]
patch
Comment 22 Song Fan CLA 2012-03-05 10:15:02 EST
fixed
Comment 23 Lisa Lasher CLA 2012-03-30 18:31:26 EDT
close