Bugzilla – Full Text Bug Listing
| Summary: | Fetch method of Scripted dataset is not called each time a chart is using it | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | z_Archived | Reporter: | Julien B <julien.baillagou> | ||||||||||||
| Component: | BIRT | Assignee: | Zhiqiang Qian <zqian> | ||||||||||||
| Status: | VERIFIED WORKSFORME | QA Contact: | Xin Zhao <XZhao> | ||||||||||||
| Severity: | normal | ||||||||||||||
| Priority: | P3 | CC: | bluesoldier, chmray, foxm, hao.zhou, lzhu, paul, vladk.dev, wyan, zqian | ||||||||||||
| Version: | 3.7.1 | ||||||||||||||
| Target Milestone: | 4.4.1 | ||||||||||||||
| Hardware: | PC | ||||||||||||||
| OS: | Windows XP | ||||||||||||||
| Whiteboard: | |||||||||||||||
| Attachments: |
|
||||||||||||||
I can confirm the problem. It seems that fetched data of scripted dataset is being cached. I have no idea how to disable the cache. I guess this behavior has been caused by the recent changes in the org.eclipse.birt.data.engine.impl.EngineExecutionHints.populateCachedDataSets(DataEngineImpl, IDataQueryDefinition[]) method. It worked in the version org.eclipse.birt.runtime_3.7.1.v20110816-1621. I can't reproduce this issue with 3.7.1-20110905 and 3.7.2 Created attachment 208818 [details]
Test case
Comment on attachment 208818 [details] Test case (In reply to comment #2) > I can't reproduce this issue with 3.7.1-20110905 and 3.7.2 Hi every one, I quickly created a sample test case with eclipse birt 20110916-0149. I created a list of users (User #0, User #1...) For each user, I use a scripted dataset to display "User data". You shall notice that <Username> will remain on User #0, even if the user changes. Regards, This bug is reproduced by BIRT team. We will look into it and get back. Please be noted that we usually only expect the one data set query to be executed only once cross the report execution. So it is not suggest to write script that based on the assumption that the data set query being execute multiple times. As this multiple time execution might be optimized by BIRT. For this specific case, please add a dummy data set parameter to your data set in nested table, this will lead BIRT to treat the data set as "dynamically changing" so that do not optimize the execution. (In reply to comment #5) > This bug is reproduced by BIRT team. We will look into it and get back. > > Please be noted that we usually only expect the one data set query to be > executed only once cross the report execution. So it is not suggest to write > script that based on the assumption that the data set query being execute > multiple times. As this multiple time execution might be optimized by BIRT. > > For this specific case, please add a dummy data set parameter to your data set > in nested table, this will lead BIRT to treat the data set as "dynamically > changing" so that do not optimize the execution. Hi Lin, Thanks for your answer and the tip! I think that it would be great then to have a flag in scripted dataset configuration that can disable caching... or force fetching after each "open" event.... Julien, Sure BIRT team is investigating the feasibility to add the flag, maybe not only for script data set but also for all kind of data sets.. Thanks. Lin We will do following enhancement in 3.7.2: 1. Enable a per data set flag that governs the data set cache behavior 2. By default we will cache all the data set Please be noted that the existing report design needs to be updated to disable the cache. You can access the flag thru: Property Editor->Properties->Advanced->Needs Cache for Data Engine. Thanks. Lin Hi, That's great! Thanks to the birt team for the job. I'll look into it in the next release. Regards, Jul The flag is a reasonable solution, but could you make the default to NOT cache the data set thereby preserving the previous behavior. The problem is that we have several hundred reports affected by this change and do not have time in our schedule to made the change and repeat the QA cycle. I understand what you're doing here, but please make the default to preserve the old behavior. If this had been done in a major functional release with some notice to the community, that might have been different. However, changing basic behavior doesn't seem like an appropriate change for a fix pack release. Mike, By default BIRT need to automatically decide whether to cache a data set or not -- this is our behavior that prevailed for 3 or 4 years, and change that sure will lead to the complaint from other BIRT users, as most of them actually expect one DB query being issued instead a couple per report execution. For your case, is that feasible that we just provide a patch jar for you so that in your release all the cache will be disabled? And after that you will have plenty of time to update your report designs to pick up next BIRT release? If that works for you, please provide the version of BIRT you are using. We will build a patch upon it. Thanks. Lin Mike, Some additional notes: when I say the feature exists for 3 to 4 years, I mean that the "data set automatic cache" feature. For this specific bug, it uses nested table bound to one data set. It is true that prior to 3.7 we do re-execute the data set script -- this is however consider a bug and we fixed it in 3.7.2. As this flag on data set is consumed by not only the nested table but also the top level tables, we cannot change its default value to false. Thanks. Lin I'll check with the product team directly affected by this and get back to you. I understand that the Scripted Data Set did behave differently from others. Whether that was right or wrong, its how the system worked for several years and we coded to that. I was hoping you could change the default for just the Scripted Data Set so the others wouldn't be affected. Mike, For the query caching, actually script data set should be same as other data sets. And the flag is defined in general data set element, which is inherited by other data sets. Is the case you confront exactly like the one proposed by Julien? To my understanding, Julien's complaint is actually not for script data set, but the caching of data set in nested query. Thanks. Lin hi Lin In comment 11, you offer to build a patch for us. That would really help. Please make it for BIRT V3.7.1. Thanks a lot! Mike Created attachment 210699 [details]
The patch for Mike
Hey Mike, Please see patch attached. Just replace it with the file of same name (org.eclipse.birt.data_3.7.1.v20110905.jar) under Eclipse/plugins directory. Should there be any further problem please let us know. Thanks. Lin hi Lin Thanks so much for creating the patch so quickly. The product team will try it today. I'll let you know when I hear back from them. Mike hi That patch looks like it resolves our problem. We need to do some more testing, but so far everything seems to be working. Would it be possible for you to supply us the source code for the fix? Our product is Eclipse based so we need to be able to provide source if we are asked. thanks again!! Mike Created attachment 210748 [details]
The changed source file.
Mike, Please use the attached file EngineExecutionHints.java to replace the one with same name under folder src\org\eclipse\birt\data\engine\impl\ Thanks. Lin Hi Mike and Lin, To precise my "complaint" just like you said, I'd say that it is just for scripted data set. Considering the following : - A first JDBC dataset (1) for a table - An other JDBC dataset (2) for a nested second table The second JDBC dataset (2) will surely have a parameter that comes from a row of the first dataset (1) and, here, everything is OK because the second dataset will be computed dynamically. However, the scripted dataset is, for me, rather specific because the data is managed "by hand" and for this, my preference goes to "disable the cache by default" for the Scripted data set only. However, it is just my point of view... Thanks, Julien Julien, Thank you for the update. Actually we have received user's complaint that their script data set, while used in subquery, being executed multiple times.. (they use filter rather than data set parameter binding to dynamically change the data loaded by script data set). If we change back the behavior, the user will complain again. Anyway it is a policy that we by default only execute the same query 1 time thru out the report generation. User will need to change the cache setting in case they do not want that behavior.We by now do not want to distinguish script data set with others. For you specific case, I am not sure the reason that you want to handle the dynamic change of data all by hand. To me it is more natural to use the data set parameter feature of the script data set in BIRT,please see attached example -- script data set can handle data set parameters exactly like other kind of data sets, i.e. JDBC. Thanks. Lin Created attachment 210829 [details]
The script data set parameter example
Lin, It is not a problem to me to disable the cache for my specific case. I just wanted to be clear on my need! Thanks for the example. I'll look into it ! Julien hi Lin Thanks again for the quick patch and source file. It resolves our reports issue and solves a big release problem for us. Mike (In reply to comment #5) The situation at my institution is such that for every account number in the query results I need to display the account information in a separate table. The way that I was doing this was by placing the table with in a list. The list is getting the different account numbers and printing them appropriately, but the table within the list is not getting updated. Is there an example of adding a dummy data set parameter available? Is this the same issue? Thank you for any help in this matter. Christine Updating BIRT or even just the .jar file may not be an option at our institution, so I was looking for an example of a dummy data set parameter. > This bug is reproduced by BIRT team. We will look into it and get back. > Please be noted that we usually only expect the one data set query to be > executed only once cross the report execution. So it is not suggest to write > script that based on the assumption that the data set query being execute > multiple times. As this multiple time execution might be optimized by BIRT. > For this specific case, please add a dummy data set parameter to your data set > in nested table, this will lead BIRT to treat the data set as "dynamically > changing" so that do not optimize the execution. Created attachment 213263 [details]
The dummy dset parameter example
Hey,
Please see attached example, based on Julien's originally attached rptdesign.
Thanks.
Lin
(In reply to comment #28) > Created attachment 213263 [details] > The dummy dset parameter example > Hey, > Please see attached example, based on Julien's originally attached rptdesign. > Thanks. > Lin Thank you for the quick reply. When I try to open the report design I get an error about version 3.2.23 not being supported. Can you save this in a previous version? I am using 3.7.1. As I mentioned in my post I cannot upgrade at this time. Also my question about using a list with a table that is not being refreshed was not addressed. Is this the same issue or is this something else and should be tracked as a new bug? (In reply to comment #29) > (In reply to comment #28) > > Created attachment 213263 [details] > > The dummy dset parameter example > > Hey, > > Please see attached example, based on Julien's originally attached rptdesign. > > Thanks. > > Lin > Thank you for the quick reply. When I try to open the report design I get an > error about version 3.2.23 not being supported. Can you save this in a > previous version? I am using 3.7.1. As I mentioned in my post I cannot > upgrade at this time. > Also my question about using a list with a table that is not being refreshed > was not addressed. Is this the same issue or is this something else and should > be tracked as a new bug? I also wanted to mention that I am not using a scripted data source and all my data sets currently have report parameters associated with them. Introduce "needsCache" flag, and by default this flag is set to "true" which indicates the data set will be cached automatically at DtE's side. User needs to change this flag if he or she does not want to cache the data set. In this case, if the flag is set to "false" on UI. The result will be as expected. See commit 22f60dcc378bfe64c19594d637c672a267427edc. |
Build Identifier: Eclipse BIRT 3.7.1 On a report, I have to display a chart for each row of a table. A scripted data source is used to aggregate data. This dataset is used as datasource for the chart. On Birt 3.7.0 : the open,fetch,close methods is called for each chart On Birt 3.7.1 : the open,close methods is called for each chart but the fetch method is just called for the first chart The consequence is that the dataset is not reinitialised for each chart and the first one of the list is repeated for each row (the following chart are basing on the data.) Reproducible: Always Steps to Reproduce: 1. Create a table (example: a table of users). Create a scripted datasource and override the methods: - "open" : sets a initial index of 0 - "fetch" : use the following code: var tabUserData = reportContext.getGlobalVariable("tabUserData"); if(count < tabResultatsGraphique.length){ row["PROJECT_NAME"] = tabUserData[count]["PROJECT_NAME"]; row["TIME"] = tabUserData[count]["TIME"]; count++; return true; } return false; 2.Create a "Line chart" that refers this scripted data set as data source in a way that one chart is created by displayed user. 3. If I place a debug log on the data set events, On Birt 3.7.0 : the open,fetch,close methods is called for each chart On Birt 3.7.1 : the open,close methods is called for each chart but the fetch method is just called for the first chart