Community
Participate
Working Groups
Eugene pointed out on bug#194140 that multi task data retrieval could potentially use up a lot of memory if many task data are requested from server. Investigate changing api to use ITaskCollector interface and push data direct to offline storage bypassing caching etc.
Tood bad it will stay in 2.0. I am not sure how we would migrate api into new implementation...
No need to migrate API, this can happen with a catching policy change.
Mik, I am not sure I understand. The point of this call is to retrieve data for multiple tasks, and current API forces it to store all of them in memory and return that collection as a result parameter. How would you tweak caching policy to avoid getting those hits and still have single request for multiple tasks?
Yes, without changing this method to use a collector capable of persisting each task data as it is received, we are stuck loading all into memory.
So, what is the plan for API migration?
What I was referring to is that if we need to we can create something like an OfflineBufferredSet that encapsulates the write/cache policy. I am not saying that this will necessarily be a better solution than deprecating that method and providing a new one, but it is possible. For the typical case the actual impact of putting things in memory should be minimal thanks to our safety threshold (5K hits) and the reasons Rob mentions on bug 194140.
In addition, when we do get to this I do not want us to implement a naiive implementation that causes a large number bunch of file writes just to free up allocation of a few MB of memory allocation because that could reduce performance on a typical machine in the common case. So we should do some profile of the cache before settling on any solution.
(In reply to comment #6) > What I was referring to is that if we need to we can create something like an > OfflineBufferredSet that encapsulates the write/cache policy. That won't work, because data set is created by connector. Using collector API would allow to manage buggering and caching on the framework side. > For the typical case the actual impact of putting things in memory should be > minimal thanks to our safety threshold (5K hits) and the reasons Rob mentions on bug 194140. Hmm. I unzipped all zips from by bugzilla offline folder (which is ~20Mb zipped now) and got ~120Mb xml file. I guess there is a lots of formatting in there, but it is still quite lot of data (if loaded in memory). (In reply to comment #7) > In addition, when we do get to this I do not want us to implement a naiive > implementation that causes a large number bunch of file writes just to free up > allocation of a few MB of memory allocation because that could reduce > performance on a typical machine in the common case. So we should do some > profile of the cache before settling on any solution. Sure thing. That is another reason to use collector API instead returning collections. So, stuff can be tuned on the framework.
*** Bug 200408 has been marked as a duplicate of this bug. ***
Done. Added collector parameter to AbstractTaskDataHandler.getMultiTaskData(). Rob, you might want to create a new bug for Bugzilla to incrementally "send" back TaskData objects as they are received from the repository instead of queuing them up in the connector.
Created attachment 95603 [details] mylyn/context/zip
Created bug#226853 to address this in the Bugzilla connector.
