Community
Participate
Working Groups
I have two queries against a repository, one of them having ~4000 tasks, the other ~500. Those queries cover all tasks of the repository I have access to. Ocasionally I open a task (which had no incoming decoration) and suddently get the editor header message "incoming changes". Clicking it shows for example a comment added a while ago (and in the meantime I got plenty of incoming notifications on other tasks of the same query). I have seen this a couple of times, it always appears random as I do see incomings in the task list for other tasks, and I can't make out why this exact task is different. Shawn mentioned that the big query could be a problem there, but could it delay those incomings for that long, and shouldn't I see a warning next to the query if it somehow failed?
I just did a "synchronize all tasks" on the repository in question... On the almost 5300 tasks in the 2 queries I had 16 tasks having comments (from up to about 3 weeks ago) which did not show up before. Those tasks were in different queries. Overall 103 tasks changed in in the last 3 weeks. I do recall some error I had a while ago (might be about 3 weeks, not sure) about mylyn and something about workspace (really don't remember much, error log cleared). Shawn suggest that an issue might be that the last-synced date is set before the incoming state is successfully written to disc and thus doesn't show up. Don't know about the internals there, so I have no idea.
Steps to reproduce this issue would be very helpful.
(In reply to comment #2) > Steps to reproduce this issue would be very helpful. I totally agree. Can't reproduce it, and ever since I set up a whole new workspace (and tasklist), that problem has not shown (it's been over a month)
Actually Steffen, I did a "Synchronize all tasks" on a repository (from the context menu in the task repositories view) where I have over 6000 tasks in total in the task list of, and got 3 missing incomings (one 6 weeks ago, one about 3, one 1 week ago). The incomings had comments/attachments/resolution changed (not all had all those changes - but they all were pretty common changes I see incomings for all the time).
I see it also more or less regularly. I have a query showing changed tasks (i.e. tasks "changed in 1 days"). Sometimes new tasks appear in the list but not marked as incoming. When I double-click on it it shows that there are changes and then updates the contents. I believe in a while those would show as incoming even if I don't touch them. Not sure how to reproduce that either.
I'm also seeing the same problem with a query of only 90 tasks from the Eclipse.org bugzilla.
Yeah, I am still seeing this regularly. Steps to reproduce is simply "sync all tasks" periodically and suddently seeing incomings that are old.
This bug is cutting me off from the community: I comment somewhere and never get notified that there are responses&follow-up questions. This reneders Mylyn close to useless for me! Raising to critical.
I have also noticed this. Frank, when we last investigated this we identified two areas that could be problematic: * BugzillaRepositoryConnector.preSynchronization() assumes that the maximum request size is 7 kb. I think in the past we have had servers silently cut off requests that were over the limit. Not sure if 7 kb assumption is still valid. * BugzillaRepositoryConnector.postSynchronization() uses the modification timestamp of any task in the task list. If a task is opened while a synchronization is running and it has had a subsequent change the modification time stamp that is used in postSynchronization() doesn't match the query result any more and the next query would use a time stamp that is too recent missing incomings in between. I think Bugzilla returns the modification time stamp as part of the query results so we might be able to make the time stamp logic much more robust using the extra data that we get back.
(In reply to comment #9) > I have also noticed this. Frank, when we last investigated this we identified > two areas that could be problematic: > > * BugzillaRepositoryConnector.preSynchronization() assumes that the maximum > request size is 7 kb. I think in the past we have had servers silently cut off > requests that were over the limit. Not sure if 7 kb assumption is still valid. > > * BugzillaRepositoryConnector.postSynchronization() uses the modification > timestamp of any task in the task list. If a task is opened while a > synchronization is running and it has had a subsequent change the modification > time stamp that is used in postSynchronization() doesn't match the query result > any more and the next query would use a time stamp that is too recent missing > incomings in between. The modification time stamp is only used when we not have a time stamp in event.getData(). But event.setData() is called with the query_timestamp from the rdf result and can not be null. Maybe we should remove the else part in getSynchronizationTimestamp because it can not be executed.What is about TaskDataManager.putUpdatedTaskData() here we set an TaskAttribute ITasksCoreConstants.ATTRIBUTE_TASK_SUPPRESS_INCOMING. Thoughts?
Created attachment 223695 [details] mylyn/context/zip
I looked at the code that handles the query time stamp and it has severe bugs: 1. There is code to set the query timestamp in queryForChanged() and performQuery(). The former is never executed since the timestamp is set in performQuery(). 2. Every query execution updates the timestamp in the session while only the first query for changed tasks should ever do that. In consequence the timestamp of the query last executed for a repository is taken into account. That means the Bugzilla connector misses most incomings that happen between execution of the first changes query and execution of the last query for the repository. The probability to missing incomings increases with the number of queries and tasks and slowness of the connection. This matches my experience. I have pushed a proposed fix here: https://git.eclipse.org/r/#/c/9105/. The change is still missing a test case.
Nice find Steffen, this makes perfect sense. I have lots of queries, too, and sometimes the server is really slow, increasing the chances of this to happen.
Here is an new patch set with a junit test.
Created attachment 224495 [details] mylyn/context/zip
Frank, I also created a test and have commented on the review with further details. I have submitted the fix to master and the 3.8.x branch and uploaded a new weekly build. It'd be great if some of you could try the following steps: 1. Update to the latest master or weekly build 2. Synchronize all tasks for Bugzilla repositories from the context menu of the Task Repositories view 3. Run for a few days/weeks 4. Repeat 2 and check if there are any incomings that should have been reported as part of regular synchronizations
Frank discovered that there is another problem with the way the timestamp is formatted. It appears that SqlifyDate in Bugzilla's Search.pm ignores the timezone and interprets the timestamp as server local time. Frank, we should file a bug against Bugzilla to allow specification of a timezone. For the connector we'll need to remove the date formatter step and use the date stamp as returned by the server (which should be in server local time).
I take that back, on second look, the code invokes str2time which does parse timezone properly so this should work as expected: http://search.cpan.org/~gbarr/TimeDate-1.20/lib/Date/Parse.pm.
Frank, iit looks to me like this indeed a bug in Bugzilla 4.2 (and later): http://mylyn.org/bugs40/buglist.cgi?chfieldfrom=2012-12-09%2022:40%20CET&bug_id=1 -> 1 bug http://mylyn.org/bugs40/buglist.cgi?chfieldfrom=2012-12-09%2022:40%20PDT&bug_id=1 -> 0 bugs http://mylyn.org/bugs42/buglist.cgi?chfieldfrom=2012-12-13%2000:29%20CET&bug_id=1 -> 1 bug http://mylyn.org/bugs42/buglist.cgi?chfieldfrom=2012-12-13%2000:29%20PDT&bug_id=1 -> 1 bug It looks like Bugzilla is not properly parsing the timestamp in _timestamp_translate(). It works as expected again if I comment out this line in Search.pm to force parsing of the timestamp: return if $value !~ /^(?:[\+\-]?\d+[hdwmy]s?|now)$/i We'll have to file a bug against Bugzilla and implement a work around since 4.2 will always interpret the timestamp as local time regardless of the time zone specified and add a test case on our end for this specific problem. Great that you found that!
Here is a test case that demonstrates the problem. It passes on 4.0 but fails on 4.2: https://git.eclipse.org/r/9214 .
Here is a fix that should work for all Bugzilla versions: https://git.eclipse.org/r/9216.
I filed a bug against Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=821997.
Steffen I have a new junit test and make sure that this work for all versions and not only for > 4.2
(In reply to comment #23) > Steffen I have a new junit test and make sure that this work for all versions > and not only for > 4.2 Unfortunately we can't make the test pass on 4.2 since querying with a timezone is broken on Bugzilla 4.2/4.4. Frank, we should push to get https://bugzilla.mozilla.org/show_bug.cgi?id=674503 fixed for 4.2.4 and 4.4.0. In the meantime I would like to use the fix that I already submitted. I tested it on 3.6 and 4.0 with the vagrant configurations and that worked as expected. Is that still failing for you? If we can avoid it I would like to keep the current simple logic and not add version checks unless there is a need for that (which I would like to understand first).
Note that I have been running on this for a week or so now and I haven't seen any missing incoming tasks from my BZ repositories.
Thanks Shawn, I can confirm the same on my end. Bugzilla issues that we discovered as part of working on this bug are tracked here: https://bugzilla.mozilla.org/show_bug.cgi?id=674503 https://bugzilla.mozilla.org/show_bug.cgi?id=822629
