Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 267508 - fix and speed up JiraFilterTest
Summary: fix and speed up JiraFilterTest
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P2 trivial (vote)
Target Milestone: 3.3   Edit
Assignee: Thomas Ehrnhoefer CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-07 14:14 EST by Steffen Pingel CLA
Modified: 2009-09-30 17:43 EDT (History)
1 user (show)

See Also:


Attachments
statistics of JiraFilterTest..testJiraFilterRefresh (4.90 KB, image/png)
2009-03-26 16:43 EDT, Thomas Ehrnhoefer CLA
no flags Details
patch v1 (1.15 KB, patch)
2009-09-16 14:49 EDT, Thomas Ehrnhoefer CLA
thomas.ehrnhoefer: review?
Details | Diff
mylyn/context/zip (15.57 KB, application/octet-stream)
2009-09-16 14:49 EDT, Thomas Ehrnhoefer CLA
no flags Details
(additional) patch (864 bytes, patch)
2009-09-21 16:51 EDT, Thomas Ehrnhoefer CLA
thomas.ehrnhoefer: review?
Details | Diff
mylyn/context/zip (9.78 KB, application/octet-stream)
2009-09-21 16:51 EDT, Thomas Ehrnhoefer CLA
no flags Details
patch v3 (3.08 KB, patch)
2009-09-30 14:25 EDT, Thomas Ehrnhoefer CLA
no flags Details | Diff
mylyn/context/zip (17.93 KB, application/octet-stream)
2009-09-30 14:25 EDT, Thomas Ehrnhoefer CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2009-03-07 14:14:18 EST
The test runs for 21 minutes on average stalling the test suite.
Comment 1 Thomas Ehrnhoefer CLA 2009-03-26 16:43:51 EDT
Created attachment 130015 [details]
statistics of JiraFilterTest..testJiraFilterRefresh
Comment 2 Thomas Ehrnhoefer CLA 2009-03-26 17:08:35 EDT
Bamboo suggests that testJiraFilterRefresh takes the longest. ALthough (as the screenshot shows) the test did run a lot faster until about build 80, and from about build 115 it is way faster again. I cannot see any code changes that would related to that, so it is most probably either the build server or the JIRA test server (rather latter I guess). I wonder if the current time is still an issue. It is about 2min for the test and 3min for the Suite. So testJiraFilterRefresh is still the longest test in the whole mylyn tests, but it is not sticking out by a whole lot.

The hotspot there is definitely 
<code>TasksUiInternal.synchronizeQuery(connector, query, null, false);</code>
taking rougly 3/4 of the whole duration of the test.

When running that locally, I get a whole bunch of exception like this one (not sure if that is related to the very long time the call takes, but exceptionhandling surely does take some time):

[2009-03-26T1-49-59] Status ERROR: org.eclipse.mylyn.tasks.core code=0 Notification failed for: org.eclipse.mylyn.internal.tasks.core.externalization.TaskListExternalizationParticipant@1fcbc19 java.lang.IllegalArgumentException: Attempted to beginRule: org.eclipse.mylyn.internal.tasks.core.ITasksCoreConstants$MutexSchedulingRule@5d7e06, does not match outer scope rule: ObjectSchedulingRule [http://mylyn.eclipse.org/jira-enterprise-3.13.1], Exception:
java.lang.IllegalArgumentException: Attempted to beginRule: org.eclipse.mylyn.internal.tasks.core.ITasksCoreConstants$MutexSchedulingRule@5d7e06, does not match outer scope rule: ObjectSchedulingRule [http://mylyn.eclipse.org/jira-enterprise-3.13.1]
	at org.eclipse.core.runtime.Assert.isLegal(Assert.java:64)
	at org.eclipse.core.internal.jobs.ThreadJob.illegalPush(ThreadJob.java:122)
	at org.eclipse.core.internal.jobs.ThreadJob.push(ThreadJob.java:232)
	at org.eclipse.core.internal.jobs.ImplicitJobs.begin(ImplicitJobs.java:58)
	at org.eclipse.core.internal.jobs.JobManager.beginRule(JobManager.java:230)
	at org.eclipse.mylyn.internal.tasks.core.externalization.ExternalizationManager$ExternalizationJob.run(ExternalizationManager.java:199)
	at org.eclipse.mylyn.internal.tasks.core.externalization.ExternalizationManager.requestSave(ExternalizationManager.java:138)
	at org.eclipse.mylyn.internal.tasks.core.externalization.TaskListExternalizationParticipant.containersChanged(TaskListExternalizationParticipant.java:152)
	at org.eclipse.mylyn.internal.tasks.core.TaskList.fireDelta(TaskList.java:260)
	at org.eclipse.mylyn.internal.tasks.core.TaskList.notifyElementsChanged(TaskList.java:482)
	at org.eclipse.mylyn.internal.tasks.core.TaskList.notifyElementChanged(TaskList.java:503)
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataManager.putUpdatedTaskData(TaskDataManager.java:218)
	at org.eclipse.mylyn.internal.tasks.core.sync.SynchronizeQueriesJob$1.putTaskData(SynchronizeQueriesJob.java:159)
	at org.eclipse.mylyn.internal.tasks.core.sync.SynchronizeQueriesJob$TaskCollector.accept(SynchronizeQueriesJob.java:89)
	at org.eclipse.mylyn.internal.jira.core.JiraRepositoryConnector$QueryHitCollector.collectIssue(JiraRepositoryConnector.java:569)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssHandler.endElement(JiraRssHandler.java:651)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssReader.readRssFeed(JiraRssReader.java:47)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssSessionCallback.parseResult(JiraRssSessionCallback.java:112)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssSessionCallback.run(JiraRssSessionCallback.java:94)
	at org.eclipse.mylyn.internal.jira.core.service.web.JiraWebSession.doInSession(JiraWebSession.java:94)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssClient.doInSession(JiraRssClient.java:46)
	at org.eclipse.mylyn.internal.jira.core.service.web.rss.JiraRssClient.executeNamedFilter(JiraRssClient.java:51)
	at org.eclipse.mylyn.internal.jira.core.service.JiraClient.executeNamedFilter(JiraClient.java:217)
	at org.eclipse.mylyn.internal.jira.core.service.JiraClient.search(JiraClient.java:455)
	at org.eclipse.mylyn.internal.jira.core.JiraRepositoryConnector.performQuery(JiraRepositoryConnector.java:133)
	at org.eclipse.mylyn.internal.tasks.core.sync.SynchronizeQueriesJob.synchronizeQuery(SynchronizeQueriesJob.java:290)
	at org.eclipse.mylyn.internal.tasks.core.sync.SynchronizeQueriesJob.synchronizeQueries(SynchronizeQueriesJob.java:247)
	at org.eclipse.mylyn.internal.tasks.core.sync.SynchronizeQueriesJob.run(SynchronizeQueriesJob.java:184)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Comment 3 Steffen Pingel CLA 2009-09-08 14:08:25 EDT
Thanks Pawel. I have applied the patch.

Most test failures were caused by outdated JIRA indices causing wrong results to be returned for searches. I have fixed that by reseting the indexes in JIRA. 

One test in JiraFilterTest is still failing on the build server. I have pasted the stack trace on bug 267508 to track fixing the failure there.
Comment 4 Steffen Pingel CLA 2009-09-08 16:01:56 EDT
Comment #3 should have been posted to bug 287736. 

This is the stack trace of the test currently failing:

null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.filterRefresh(JiraFilterTest.java:84)
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.testJiraFilterRefresh(JiraFilterTest.java:72)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3470)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3117)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2405)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2369)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2221)
	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:493)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:113)
	at org.eclipse.test.UITestApplication.runApplication(UITestApplication.java:138)
	at org.eclipse.test.UITestApplication.run(UITestApplication.java:60)
	at org.eclipse.test.UITestApplication.start(UITestApplication.java:210)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:194)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
	at org.eclipse.core.launcher.Main.main(Main.java:34)
Comment 5 Steffen Pingel CLA 2009-09-08 16:02:34 EDT
Reopening.
Comment 6 Thomas Ehrnhoefer CLA 2009-09-16 14:48:13 EDT
I suspect it took that long because it actually uses the predefined filter "my issues", which can potentially return a lot of results on a long running and not cleaned DB instance.

I suggest adding a "my recent" filter to the server which only returns issues created by the current user in the last 10min.
Comment 7 Thomas Ehrnhoefer CLA 2009-09-16 14:49:47 EDT
Created attachment 147362 [details]
patch v1

This patch looks for a "My Recent" filter first, and uses that if it is available.
I added this filter (as described in my previous comment) to the 3.13.1 server instance.
Comment 8 Thomas Ehrnhoefer CLA 2009-09-16 14:49:49 EDT
Created attachment 147363 [details]
mylyn/context/zip
Comment 9 Steffen Pingel CLA 2009-09-16 19:02:02 EDT
Great stuff. I have applied the patch.
Comment 10 Steffen Pingel CLA 2009-09-19 15:54:47 EDT
Test is still failing on build server but passes on mylyn.eclipse.org. Can you check that the filter exists on the local test server?

null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.filterRefresh(JiraFilterTest.java:89)
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.testJiraFilterRefresh(JiraFilterTest.java:77)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
Comment 11 Thomas Ehrnhoefer CLA 2009-09-21 11:27:15 EDT
Sure, I will do that.
Not sure if it is better to have a failing or slow test though...we could get the default "all mine" filter if the other one is not available.
Comment 12 Thomas Ehrnhoefer CLA 2009-09-21 16:51:35 EDT
Created attachment 147735 [details]
(additional) patch

This patch should take care of that issue. I assumed more than 1 filter, which is wrong and also unnecessary, so I removed that statement.
Comment 13 Thomas Ehrnhoefer CLA 2009-09-21 16:51:37 EDT
Created attachment 147736 [details]
mylyn/context/zip
Comment 14 Steffen Pingel CLA 2009-09-21 23:45:51 EDT
Thanks!
Comment 15 Steffen Pingel CLA 2009-09-24 21:16:57 EDT
Sorry, the test is failing again:

Custom query wrong date pattern@ j i r a 4.0.0- beta2/ enterprise

expected:<Status OK: org.eclipse.core.runtime code=0 OK null> but was:<Status ERROR: org.eclipse.mylyn.internal.jira.core code=1 Unexpected result code 400 while running query: http://dev2.tasktop.com/jira-enterprise-4.0.0/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?decorator=none&reset=true&tempMax=5000&&created:after=01%2FOkt%2F08&created:before=01%2FOkt%2F08 null>
junit.framework.AssertionFailedError: expected:<Status OK: org.eclipse.core.runtime code=0 OK null> but was:<Status ERROR: org.eclipse.mylyn.internal.jira.core code=1 Unexpected result code 400 while running query: http://dev2.tasktop.com/jira-enterprise-4.0.0/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?decorator=none&reset=true&tempMax=5000&&created:after=01%2FOkt%2F08&created:before=01%2FOkt%2F08 null>
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.testCustomQueryWrongDatePattern(JiraFilterTest.java:178)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)

Custom query wrong date pattern@ j i r a 3.13.1/ enterprise

expected:<Status OK: org.eclipse.core.runtime code=0 OK null> but was:<Status ERROR: org.eclipse.mylyn.internal.jira.core code=1 Unexpected result code 500 while running query: http://dev2.tasktop.com/jira-enterprise-3.13.1/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?decorator=none&reset=true&tempMax=5000&&created:after=01%2FOkt%2F08&created:before=01%2FOkt%2F08 null>
junit.framework.AssertionFailedError: expected:<Status OK: org.eclipse.core.runtime code=0 OK null> but was:<Status ERROR: org.eclipse.mylyn.internal.jira.core code=1 Unexpected result code 500 while running query: http://dev2.tasktop.com/jira-enterprise-3.13.1/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?decorator=none&reset=true&tempMax=5000&&created:after=01%2FOkt%2F08&created:before=01%2FOkt%2F08 null>
	at org.eclipse.mylyn.jira.tests.core.JiraFilterTest.testCustomQueryWrongDatePattern(JiraFilterTest.java:178)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
Comment 16 Steffen Pingel CLA 2009-09-24 21:17:19 EDT
Reopening.
Comment 17 Thomas Ehrnhoefer CLA 2009-09-30 14:25:31 EDT
Created attachment 148457 [details]
patch v3

Jira4 was missing the saved filter as well.

Also, since the Fixture now keeps a hold of the repository, the test has to undo all changes to the properties, so I added some code to test to do that.

Seems like the JiraFileterTest runs fine on 3.13.5 and 4.0rc1 now.
Comment 18 Thomas Ehrnhoefer CLA 2009-09-30 14:25:38 EDT
Created attachment 148458 [details]
mylyn/context/zip
Comment 19 Steffen Pingel CLA 2009-09-30 14:50:26 EDT
I have applied the patch but I don't understand why this is necessary. JiraFixture does not maintain a reference to the repository and each test should be creating a new instance. It seems that there would be something else going wrong if resetting the property fixed it.
Comment 20 Thomas Ehrnhoefer CLA 2009-09-30 16:41:20 EDT
You are right Steffen, the problem was caused due to bug 184806. I added a patch there which purges the clients in jiraclientfactory
Comment 21 Steffen Pingel CLA 2009-09-30 17:43:20 EDT
Thanks for looking into that. I have added reset method to JiraFixture and reverted the changes for the last patch since they shouldn't be necessary any longer.