Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 290465 - fix Bugzilla test suite
Summary: fix Bugzilla test suite
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P1 normal (vote)
Target Milestone: 3.4   Edit
Assignee: Robert Elves CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 300704
Blocks: 290614 296894
  Show dependency tree
 
Reported: 2009-09-24 19:23 EDT by Steffen Pingel CLA
Modified: 2010-04-09 18:28 EDT (History)
1 user (show)

See Also:


Attachments
fix connection leaks (6.42 KB, patch)
2009-09-24 19:24 EDT, Steffen Pingel CLA
no flags Details | Diff
current state (44.82 KB, image/png)
2009-12-09 17:23 EST, Robert Elves CLA
no flags Details
fix BugzillaTaskHyperlinkDetectorTest (2.18 KB, patch)
2009-12-12 16:52 EST, Frank Becker CLA
no flags Details | Diff
mylyn/context/zip (2.89 KB, application/octet-stream)
2009-12-12 16:52 EST, Frank Becker CLA
no flags Details
date time fix / refactoring (17.80 KB, patch)
2009-12-12 18:16 EST, Robert Elves CLA
no flags Details | Diff
mylyn/context/zip (67.85 KB, application/octet-stream)
2009-12-12 18:16 EST, Robert Elves CLA
no flags Details
rev (17.87 KB, patch)
2009-12-12 18:23 EST, Robert Elves CLA
no flags Details | Diff
commited patch (15.27 KB, patch)
2009-12-13 17:06 EST, Frank Becker CLA
no flags Details | Diff
mylyn/context/zip (12.28 KB, application/octet-stream)
2009-12-13 17:06 EST, Frank Becker CLA
no flags Details
patch to use fixture (2.94 KB, patch)
2010-01-21 20:47 EST, Robert Elves CLA
no flags Details | Diff
screenshot (78.03 KB, image/png)
2010-01-25 16:21 EST, Steffen Pingel CLA
no flags Details
inprogress (15.63 KB, patch)
2010-02-16 12:32 EST, Robert Elves CLA
no flags Details | Diff
inprogress (18.07 KB, patch)
2010-02-16 14:01 EST, Robert Elves CLA
no flags Details | Diff
update (15.99 KB, patch)
2010-02-16 15:14 EST, Robert Elves CLA
no flags Details | Diff
commited patch (990 bytes, patch)
2010-02-25 13:39 EST, Frank Becker CLA
no flags Details | Diff
mylyn/context/zip (1.96 KB, application/octet-stream)
2010-02-25 13:39 EST, Frank Becker CLA
no flags Details
more changes (1.98 KB, patch)
2010-02-25 18:01 EST, Steffen Pingel CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2009-09-24 19:23:24 EDT
Some Bugzilla tests are still failing due to changes to the test infrastructure. A initial set of headless tests has been fixed as part of bug 287226 but the remaining tests still need to be ported.
Comment 1 Steffen Pingel CLA 2009-09-24 19:24:23 EDT
Created attachment 148066 [details]
fix connection leaks
Comment 2 Robert Elves CLA 2009-10-08 21:19:24 EDT
Status update: Tests are in better shape now. Time tracking needs to be enabled to fix some tests.  BugzillaRepository32Test and one other. Should have them fixed before release date.
Comment 3 Steffen Pingel CLA 2009-12-03 22:30:43 EST
Rob, what is the current status of this?
Comment 4 Robert Elves CLA 2009-12-09 16:57:08 EST
BugzillaRepository32Test had failures. Looking at this now...
Comment 5 Robert Elves CLA 2009-12-09 17:23:12 EST
Created attachment 154168 [details]
current state

first failure was a network timeout that passed after rerun
Comment 6 Steffen Pingel CLA 2009-12-09 17:42:56 EST
Do you think it's feasible to aim for having this fixed within in the next weeks so we can get a green test run for all of Mylyn before contributing to Helios M4?
Comment 7 Robert Elves CLA 2009-12-09 19:22:04 EST
Yes, lets consider the Bugzilla component locked until we have these tests passing.

BugzillaTaskHyperlinkDetectorTest - Frank could you investigates since you've been working on link detection improvements.
   
BugzillaRepositoryConnectorTest - I'll look into getting time tracking enabled in the configuration for the test servers.

BugzillaRepository32Test - Frank, another one for you. I'll commit what porting I've done then if you could look at the failing tests that would be great.

BugzillaDateTimeTests - I'll take a look at this.
Comment 8 Frank Becker CLA 2009-12-12 16:52:08 EST
Created attachment 154373 [details]
fix BugzillaTaskHyperlinkDetectorTest

here the commited patch
Comment 9 Frank Becker CLA 2009-12-12 16:52:12 EST
Created attachment 154374 [details]
mylyn/context/zip
Comment 10 Robert Elves CLA 2009-12-12 18:16:37 EST
Created attachment 154375 [details]
date time fix / refactoring

Great Frank. Here's the timing test patch as committed.
Comment 11 Robert Elves CLA 2009-12-12 18:16:41 EST
Created attachment 154376 [details]
mylyn/context/zip
Comment 12 Robert Elves CLA 2009-12-12 18:23:28 EST
Created attachment 154377 [details]
rev
Comment 13 Frank Becker CLA 2009-12-13 03:44:34 EST
Rob,

can you please correct the Flag definition in teh test repositories.

What is need for testFlags
					requestable	multiplicable	specifically_requestable
1) BugFlag1       no				no						no
2) BugFlag2       yes				no						no
3) BugFlag3       yes				yes					no
4) BugFlag4       yes				no						yes
Comment 14 Frank Becker CLA 2009-12-13 17:06:04 EST
Created attachment 154384 [details]
commited patch

Here is what I have commited.

I did some changes in the test bugzilla repositories (3.2 and 3.4) for the Flags
Comment 15 Frank Becker CLA 2009-12-13 17:06:10 EST
Created attachment 154385 [details]
mylyn/context/zip
Comment 16 Robert Elves CLA 2009-12-14 17:13:31 EST
Great. Time test now passing as well. Marking resolved!
Comment 17 Steffen Pingel CLA 2010-01-16 16:27:09 EST
Great stuff! I have added the Bugzilla tests back to the AllConnectorTests test suite.
Comment 18 Steffen Pingel CLA 2010-01-16 17:13:39 EST
The last build failed with the error which could be related to our local test server setup being outdated. Frank, Rob, please let me know if I need to make any changes to the test server to fix any of these failures.

It also looks like AllBugzillaTests does not actually run tests against multiple repository configurations like AllBugzillaHeadlessStandaloneTests does. Can you change AllBugzillaTests to follow the same pattern of creating nested test suites to run relevant tests against all fixtures?



BugzillaRepositoryConnectorTest : Time tracker (testTimeTracker) History
java.lang.NullPointerException
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testTimeTracker(BugzillaRepositoryConnectorTest.java:787)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
(24 more lines...)
BugzillaRepositoryConnectorTest : Mid air collision (testMidAirCollision) History
Mid-air collision expected
junit.framework.AssertionFailedError: Mid-air collision expected
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMidAirCollision(BugzillaRepositoryConnectorTest.java:577)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
(25 more lines...)
BugzillaRepositoryConnectorTest : Std workflow32 (testStdWorkflow32) History
org.eclipse.core.runtime.CoreException: An unkown repository error has occurred.
	at org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.parseHtmlError(BugzillaClient.java:1691)
	at org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.postTaskData(BugzillaClient.java:1093)
	at org.eclipse.mylyn.bugzilla.tests.support.BugzillaFixture.submitTask(BugzillaFixture.java:228)
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.doStdWorkflow(BugzillaRepositoryConnectorTest.java:415)
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testStdWorkflow32(BugzillaRepositoryConnectorTest.java:347)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
(28 more lines...)
BugzillaAttachmentHandlerTest : Attach to existing report (testAttachToExistingReport) History
null expected:<A[] repository error ha...> but was:<A[n unkown] repository error ha...>
junit.framework.ComparisonFailure: null expected:<A[] repository error ha...> but was:<A[n unkown] repository error ha...>
	at org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachToExistingReport(BugzillaAttachmentHandlerTest.java:96)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
(25 more lines...)
BugzillaAttachmentHandlerTest : Attachment token (testAttachmentToken) History
null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachmentToken(BugzillaAttachmentHandlerTest.java:178)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
(25 more lines...)
BugzillaRepository32Test : Custom attributes (testCustomAttributes) History
no colors set
junit.framework.AssertionFailedError: no colors set
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepository32Test.testCustomAttributes(BugzillaRepository32Test.java:379)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
(25 more lines...)
BugzillaRepository32Test : Flags (testFlags) History
null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepository32Test.testFlags(BugzillaRepository32Test.java:188)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
	at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
(25 more lines...)
Comment 19 Robert Elves CLA 2010-01-21 20:24:41 EST
(In reply to comment #18)

> It also looks like AllBugzillaTests does not actually run tests against multiple
> repository configurations like AllBugzillaHeadlessStandaloneTests does. Can you
> change AllBugzillaTests to follow the same pattern of creating nested test
> suites to run relevant tests against all fixtures?

I'll look into that now.


> BugzillaRepositoryConnectorTest : Time tracker (testTimeTracker) History
> java.lang.NullPointerException
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testTimeTracker(BugzillaRepositoryConnectorTest.java:787)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)


Under Group Security option in the server preferences the timetrackinggroup needs to be editbugs for this to pass I believe.

> (24 more lines...)
> BugzillaRepositoryConnectorTest : Mid air collision (testMidAirCollision)
> History
> Mid-air collision expected
> junit.framework.AssertionFailedError: Mid-air collision expected
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMidAirCollision(BugzillaRepositoryConnectorTest.java:577)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> (25 more lines...)

> BugzillaRepositoryConnectorTest : Std workflow32 (testStdWorkflow32) History
> org.eclipse.core.runtime.CoreException: An unkown repository error has occurred.
> at
> org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.parseHtmlError(BugzillaClient.java:1691)
> at
> org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.postTaskData(BugzillaClient.java:1093)
> at
> org.eclipse.mylyn.bugzilla.tests.support.BugzillaFixture.submitTask(BugzillaFixture.java:228)
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.doStdWorkflow(BugzillaRepositoryConnectorTest.java:415)
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testStdWorkflow32(BugzillaRepositoryConnectorTest.java:347)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> (28 more lines...)
 
Not sure about this one, Frank?

> BugzillaAttachmentHandlerTest : Attach to existing report
> (testAttachToExistingReport) History
> null expected:<A[] repository error ha...> but was:<A[n unkown] repository error
> ha...>
> junit.framework.ComparisonFailure: null expected:<A[] repository error ha...>
> but was:<A[n unkown] repository error ha...>
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachToExistingReport(BugzillaAttachmentHandlerTest.java:96)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> (25 more lines...)

Fixed in head.

> BugzillaAttachmentHandlerTest : Attachment token (testAttachmentToken) History
> null
> junit.framework.AssertionFailedError: null
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachmentToken(BugzillaAttachmentHandlerTest.java:178)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> (25 more lines...)

Lets see if this fails again.


> BugzillaRepository32Test : Custom attributes (testCustomAttributes) History
> no colors set
> junit.framework.AssertionFailedError: no colors set
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepository32Test.testCustomAttributes(BugzillaRepository32Test.java:379)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> (25 more lines...)

Custom attributes are missing from the test instance. Name: cf_multiselect with values Red, Green, Yellow, Blue

> BugzillaRepository32Test : Flags (testFlags) History
> null
> junit.framework.AssertionFailedError: null
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepository32Test.testFlags(BugzillaRepository32Test.java:188)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
> at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:206)
> at org.eclipse.test.UITestApplication$3.run(UITestApplication.java:195)
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> (25 more lines...)

There are some flags that need to be enabled on the test repository. Frank?
Comment 20 Robert Elves CLA 2010-01-21 20:47:11 EST
Created attachment 156891 [details]
patch to use fixture

Looks like we have a number of failures against 3.4.1 mostly due to suite configuration so I'll need to look into cleaning this up.
Comment 21 Robert Elves CLA 2010-01-21 21:22:49 EST
Steffen, after the next run, could you post a revised failure dump. Thanks.
Comment 22 Steffen Pingel CLA 2010-01-25 16:14:33 EST
I have exported the databases from mylyn.eclipse.org and imported them on the test server. Rob, please check the results of the last test run, there are 33 failures.

We should also do the following:
* Go through all of the test classes and figure out which need to run against different repository configuration (e.g. it's sufficient to run BugzillaSearchPageTest once outside of the fixture suites since it doesn't do anything repository specific).
* Add version check or modify the fixture suite setup to exclude tests that are not compatible with older Bugzilla versions (e.g. when testing custom fields).
* Remove all init..() methods in AbstractBugzillaTest and replace them by usage of the fixture.
* Remove Bugzilla 3.2.2 and 3.2.3 specific tests as 3.2.4 and later are the only supported versions in the 3.2.x stream now.
Comment 23 Steffen Pingel CLA 2010-01-25 16:21:55 EST
Created attachment 157178 [details]
screenshot
Comment 24 Steffen Pingel CLA 2010-01-31 14:31:26 EST
Thanks for fixing the tests, Rob. Looks like we are down to 4 failures:

org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest	  
Attachment token@3.2.4
null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachmentToken(BugzillaAttachmentHandlerTest.java:346)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	
	
Attachment token@3.4.4
null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachmentToken(BugzillaAttachmentHandlerTest.java:346)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	
	
org.eclipse.mylyn.bugzilla.tests.EncodingTest	  
Proper encoding upon post@3.4.4
org.apache.commons.httpclient.ProtocolException: The server localhost failed to respond with a valid HTTP response
	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1987)
	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
	at org.eclipse.mylyn.internal.bugzilla.core.GzipPostMethod.execute(GzipPostMethod.java:55)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.eclipse.mylyn.commons.net.WebUtil$2.execute(WebUtil.java:319)
	at org.eclipse.mylyn.commons.net.WebUtil$2.execute(WebUtil.java:1)
	at org.eclipse.mylyn.internal.commons.net.MonitoredRequest.call(MonitoredRequest.java:51)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
	at java.util.concurrent.FutureTask.run(FutureTask.java:123)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
	at java.lang.Thread.run(Thread.java:595)
	
	
org.eclipse.mylyn.bugzilla.tests.core.BugzillaCustomFieldsTest		  
Custom attributes new task@3.2.4
null
junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.core.BugzillaCustomFieldsTest.testCustomAttributesNewTask(BugzillaCustomFieldsTest.java:172)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
Comment 25 Steffen Pingel CLA 2010-01-31 14:33:19 EST
And this test also fails intermittently:

org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest		  
Mid air collision@3.4.4
Mid-air collision expected
junit.framework.AssertionFailedError: Mid-air collision expected
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMidAirCollision(BugzillaRepositoryConnectorTest.java:578)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
	at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:354)
Comment 26 Steffen Pingel CLA 2010-02-04 21:11:16 EST
I saw that you have disabled all tests that were previously running against Bugzilla < 3.0. While we don't officially support those versions anymore wouldn't it make sense to run a very basic set of tests against them? We already have the setup in place and know that most tests were passing. Don't bother with it if you think the cost of maintaining these tests is too high in the end.
Comment 27 Robert Elves CLA 2010-02-11 12:52:45 EST
Added 222 back to the list. The others are going to be too much overhead (and are relatively old). Still a couple spurious failures but otherwise passing green locally. Hopefully I'll get some more time to look into these odd failures today.
Comment 28 Robert Elves CLA 2010-02-16 12:32:37 EST
Created attachment 159218 [details]
inprogress
Comment 29 Robert Elves CLA 2010-02-16 14:01:08 EST
Created attachment 159228 [details]
inprogress
Comment 30 Robert Elves CLA 2010-02-16 15:14:07 EST
Created attachment 159235 [details]
update
Comment 31 Robert Elves CLA 2010-02-16 17:02:02 EST
Fixture now passing.
Comment 32 Steffen Pingel CLA 2010-02-16 17:23:00 EST
Thanks Rob. Can you summarize what was causing the failures? Have the concerns that I raised in comment 22 been addressed?
Comment 33 Steffen Pingel CLA 2010-02-16 18:30:35 EST
The 3.5 test run failed again:

org.eclipse.core.runtime.CoreException: An unknown repository error has occurred: ada
[...]
	at org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.parseRepositoryResponse(BugzillaClient.java:1673)
	at org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.parseHtmlError(BugzillaClient.java:1515)
	at org.eclipse.mylyn.internal.bugzilla.core.BugzillaClient.getTaskData(BugzillaClient.java:1804)
	at org.eclipse.mylyn.bugzilla.tests.support.BugzillaFixture.getTask(BugzillaFixture.java:233)
	at org.eclipse.mylyn.bugzilla.tests.BugzillaAttachmentHandlerTest.testAttachmentToken(BugzillaAttachmentHandlerTest.java:320
Comment 34 Frank Becker CLA 2010-02-17 15:14:04 EST
(In reply to comment #33)
> The 3.5 test run failed again:
> 
> org.eclipse.core.runtime.CoreException: An unknown repository error has
> occurred: ada

Sorry I can not reproduce this. Test creates the http://mylyn.eclipse.org/bugshead/show_bug.cgi?id=15 without an error.
Comment 35 Steffen Pingel CLA 2010-02-17 17:14:54 EST
Looks like it's all good now. Rob ran the tests a couple more times and the all seem to have been passed. Rob, it would be great if you could still summarize the cause of failures and clarify if it's feasible to backport the fix to the 3.3.x maintenance stream?
Comment 36 Robert Elves CLA 2010-02-17 19:33:25 EST
(In reply to comment #32)
> Thanks Rob. Can you summarize what was causing the failures? Have the concerns
> that I raised in comment 22 been addressed?
Yes thanks for that summary of actions Steffen, they should all be addressed now.

(In reply to comment #35)
> Looks like it's all good now. Rob ran the tests a couple more times and the all
> seem to have been passed. Rob, it would be great if you could still summarize
> the cause of failures and clarify if it's feasible to backport the fix to the
> 3.3.x maintenance stream?

To summarize the fix, there was a bug in how the html responses were parsed in the client. When parsing responses the title element is checked for success/failure. In a few places, the code looked for the title, then if it didn't find a successful indicator, the stream was reset and passed along to another function to be rescanned for error cases.  Some of the response pages now appear to have more content in the header resulting in the marker being overrun and the reset subsequently failing. This was leading to the second parse missing the <title> element completely.  The response parsing has now been consolidated and now only performs a single parse without the need of markers. So there was some refactoring involved but given some mileage backporting should be feasible.
Comment 37 Steffen Pingel CLA 2010-02-17 19:51:30 EST
Thanks Rob. Great that you applied that refactoring and fixed the bug. It sounds like the changes were significant so I think we are better off disabling the failing tests in 3.3.x branch. There is only one week to the release and backporting the amount of changes would add some overhead and introduce too high of a risk of regressions.
Comment 38 Steffen Pingel CLA 2010-02-18 16:16:53 EST
This failure occurred last night on Eclipse 3.6. Logging it here in case the test case needs improvement:

java.lang.NullPointerException
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMultiRetrievalFailure(BugzillaRepositoryConnectorTest.java:94)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
Comment 39 Steffen Pingel CLA 2010-02-20 16:51:09 EST
The failure reoccured on 3.5:

BugzillaRepositoryConnectorTest : Multi retrieval failure@3.2.4 (testMultiRetrievalFailure@3.2.4) History
java.lang.NullPointerException
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMultiRetrievalFailure(BugzillaRepositoryConnectorTest.java
	
Rob, can you take a look?
Comment 40 Steffen Pingel CLA 2010-02-22 01:51:41 EST
Failure on Bugzilla 3.5.2+/Eclipse 3.6:

junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.RepositoryReportFactoryTest.testInvalidCredentials222(RepositoryReportFactoryTest.java:59)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)

The test needs to be improved to provide more meaningful output in case of a failure.
Comment 41 Robert Elves CLA 2010-02-24 11:28:33 EST
(In reply to comment #39)
> The failure reoccured on 3.5:
> 
> BugzillaRepositoryConnectorTest : Multi retrieval failure@3.2.4
> (testMultiRetrievalFailure@3.2.4) History
> java.lang.NullPointerException
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMultiRetrievalFailure(BugzillaRepositoryConnectorTest.java
> 
> Rob, can you take a look?

Yes, thanks Steffen, I'll take a look at these failures tomorrow.
Comment 42 Frank Becker CLA 2010-02-25 13:39:38 EST
Created attachment 160219 [details]
commited patch

add fail to make sure that we get the expected error
Comment 43 Frank Becker CLA 2010-02-25 13:39:48 EST
Created attachment 160220 [details]
mylyn/context/zip
Comment 44 Steffen Pingel CLA 2010-02-25 18:01:53 EST
Created attachment 160258 [details]
more changes
Comment 45 Steffen Pingel CLA 2010-02-25 18:03:23 EST
Good catch Frank. In addition it's important that we record the actual failure message in case it doesn't match the expected message (see patch).
Comment 46 Steffen Pingel CLA 2010-03-12 04:10:17 EST
Expected 6583, got: [summary]
junit.framework.AssertionFailedError: Expected 6583, got: [summary]
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testSynchChangedReports(BugzillaRepositoryConnectorTest.java:895)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)

I guess the error handling has to be improved a little further :)...
Comment 47 Frank Becker CLA 2010-03-26 15:28:04 EDT
It looks like we have a problem with all bugzilla test repositories. 

Here the error:

Can't connect to the database.
Error: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
  Is your database installed and up and running?
  Do you have the correct username and password selected in localconfig?
Comment 48 Steffen Pingel CLA 2010-03-26 21:57:21 EDT
I think the server was upgraded and has more memory now. I'll take a look why tests are failing now.
Comment 49 Steffen Pingel CLA 2010-03-30 17:12:27 EDT
I have restarted the mysql server. Can you retry?
Comment 50 Frank Becker CLA 2010-04-02 08:02:35 EDT
(In reply to comment #49)
> I have restarted the mysql server. Can you retry?

Now the tests are OK!
Comment 51 Robert Elves CLA 2010-04-06 13:44:06 EDT
Great! Looking good at this end as well.
Comment 52 Steffen Pingel CLA 2010-04-07 16:57:52 EDT
There are 2 new failures:

junit.framework.AssertionFailedError: null
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMultiRetrievalFailure(BugzillaRepositoryConnectorTest.java:91)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)

Invalid id error should have resulted
junit.framework.AssertionFailedError: Invalid id error should have resulted
	at org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testSingleRetrievalFailure(BugzillaRepositoryConnectorTest.java:64)
	at org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
Comment 53 Robert Elves CLA 2010-04-09 17:59:46 EDT
(In reply to comment #52)
> There are 2 new failures:
> 
> junit.framework.AssertionFailedError: null
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testMultiRetrievalFailure(BugzillaRepositoryConnectorTest.java:91)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)
> 
> Invalid id error should have resulted
> junit.framework.AssertionFailedError: Invalid id error should have resulted
> at
> org.eclipse.mylyn.bugzilla.tests.BugzillaRepositoryConnectorTest.testSingleRetrievalFailure(BugzillaRepositoryConnectorTest.java:64)
> at
> org.eclipse.mylyn.commons.tests.support.ManagedTestSuite.run(ManagedTestSuite.java:127)


All good now.
Comment 54 Robert Elves CLA 2010-04-09 18:28:01 EDT
Marking resolved. Lets create new bugs to address future failures.