Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 399803 - Bugzilla puppet configuration should be reproducible
Summary: Bugzilla puppet configuration should be reproducible
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: 3.9   Edit
Assignee: Frank Becker CLA
QA Contact: Frank Becker CLA
URL:
Whiteboard:
Keywords:
Depends on: 393640
Blocks:
  Show dependency tree
 
Reported: 2013-02-03 07:06 EST by Steffen Pingel CLA
Modified: 2013-04-27 09:15 EDT (History)
0 users

See Also:


Attachments
mylyn/context/zip (1.90 KB, application/octet-stream)
2013-02-17 10:50 EST, Frank Becker CLA
no flags Details
mylyn/context/zip (6.52 KB, application/octet-stream)
2013-04-27 07:51 EDT, Frank Becker CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2013-02-03 07:06:57 EST
The Bugzilla instances provisioned by puppet should correspond to an exact version, e.g. 3.6.12 (instead of the latest from the 3.6 branch). This will allow us to reproduce the exact configuration later on and explicitly control which versions are tested.

defaultsites.pp should also specify this, e.g. and use a unique database and url location so multiple service releases of the same release branch can be run in parallel.

  bugzilla::site { "bugs-3.6.12":
    major   => "3",
    minor   => "6",
    service   => "12",
  }
Comment 1 Frank Becker CLA 2013-02-03 08:13:18 EST
This is already possible.


  bugzilla::site { "bugs36":
    major   => "3",
    minor   => "6",
  }

use the default value bugzilla-stable an the tag name. So we always get the latest version of this branch.

With 

  bugzilla::site { "bugs36":
    major   => "3",
    minor   => "6",
    branchTag => "bugzilla-3.6.12",
  }

we get the 3.6.12 version.

Actual bugzilla-3.6.12 and bugzilla-stable are tags for the same revision (7307)

Only for bugshead where we use trunk we are not reproducible!
Comment 2 Steffen Pingel CLA 2013-02-03 09:10:31 EST
(In reply to comment #1)
> With
> 
> bugzilla::site { "bugs36":
> major   => "3",
> minor   => "6",
> branchTag => "bugzilla-3.6.12",
> }
> 
> we get the 3.6.12 version.

Great! Then we just need to update the configuration to use "bugzilla-3.6.12" as the title and set the branchTag accordingly.

> Actual bugzilla-3.6.12 and bugzilla-stable are tags for the same revision (7307)

True, but this could change later.
 
> Only for bugshead where we use trunk we are not reproducible!

Ok, that's fine for trunk.
Comment 3 Frank Becker CLA 2013-02-17 10:50:46 EST
Should we change defaultsite.pp to

a) add the correct branchTag to every site
b) add  options so we can enable/disable bzr update and reset the database when run puppet

Thoughts?
Comment 4 Frank Becker CLA 2013-02-17 10:50:49 EST
Created attachment 227166 [details]
mylyn/context/zip
Comment 5 Steffen Pingel CLA 2013-02-18 04:38:09 EST
(In reply to comment #3)
> a) add the correct branchTag to every site

Yes.

> b) add  options so we can enable/disable bzr update and reset the database when
> run puppet

No. We'll just provision all versions that are specified and worry about deleting older version later. Each version has to be provisioned into a unique directory so that they don't conflict.
Comment 6 Frank Becker CLA 2013-02-18 23:46:40 EST
patchs set 2 of review https://git.eclipse.org/r/#/c/10449/ use cleanDB to drop and recreate the database for every instance. I did it his way to have only a short time where the database is not initialized (until update bugzilla_checksetup $version is done).
Comment 7 Frank Becker CLA 2013-02-20 11:25:26 EST
Patch set 3 is the update from 4.0.9 to 4.0.10 and 4.2.4 to 4.2.5 as released today.
Comment 8 Frank Becker CLA 2013-02-25 14:36:51 EST
Sorry by mistake a create a new review (https://git.eclipse.org/r/#/c/10632/) which depends on https://git.eclipse.org/r/#/c/10449
Comment 9 Frank Becker CLA 2013-03-09 14:48:05 EST
Fond an error in the service.json file.

Please review https://git.eclipse.org/r/#/c/11023/
Comment 10 Steffen Pingel CLA 2013-04-16 11:13:36 EDT
Frank, looks like we are almost done here. I pushed a change here that uses the new framework support for discovering fixtures: https://git.eclipse.org/r/#/c/11944/ . Can you take a look?
Comment 11 Steffen Pingel CLA 2013-04-16 20:02:28 EDT
Frank, I think we should also make the following changes:

* Only include properties if they diverge from the default, i.e. the listing on http://mylyn.org/ should only show default = "1" but exclude the property if it is "0" to make the listing shorter. Same for the other properties.
* Existing configurations should be disabled prior to provisioning so that we only end up with the most recent configurations in the services list. Otherwise the list of configurations will grow very large over time and slow down tests: https://git.eclipse.org/r/#/c/11961/.
Comment 12 Steffen Pingel CLA 2013-04-19 17:23:02 EDT
Thanks for reviewing. We have two items left here from my point of view:

* Simplify consumption of services and streamline Bugzilla fixture and harness to ensure tests run reliably. A draft implementation is here but it's not complete: https://git.eclipse.org/r/#/c/11944/
* Only include properties if they diverge from the default, i.e. the listing on http://mylyn.org/ should only show default = "1" but exclude the property if it is "0" to make the listing shorter. Same for the other properties.
Comment 13 Steffen Pingel CLA 2013-04-21 22:32:33 EDT
Thanks for updating the changes. I have merged them. I pushed one more review with a few minor changes: https://git.eclipse.org/r/#/c/12088/.
Comment 14 Steffen Pingel CLA 2013-04-24 13:59:52 EDT
Tests are passing again and are running against all Bugzilla fixtures! Anything left to do here?
Comment 15 Frank Becker CLA 2013-04-24 15:39:02 EDT
(In reply to comment #14)
> Tests are passing again and are running against all Bugzilla fixtures!
> Anything left to do here?

Maybe we should change the following to 3.6.13
* bugzilla-3.6.12-custom-wf
* bugzilla-3.6.12-custom-wf-and-status
* bugzilla-3.6.12-xml-rpc-disabled


Only include properties if they diverge from the default, i.e. the listing on http://mylyn.org/

The value of property 'xmlrpc_enabled' is not really but we have no place where we put this on by default.

Should we track this on a new bug?
Comment 16 Steffen Pingel CLA 2013-04-24 16:02:41 EDT
(In reply to comment #15)
> Maybe we should change the following to 3.6.13
> * bugzilla-3.6.12-custom-wf
> * bugzilla-3.6.12-custom-wf-and-status
> * bugzilla-3.6.12-xml-rpc-disabled

Makes sense. That should be straight forward. Can you push a review?

> Only include properties if they diverge from the default, i.e. the listing on
> http://mylyn.org/
> 
> The value of property 'xmlrpc_enabled' is not really but we have no place where
> we put this on by default.

I consider that to be the default since it's set for all configurations except one and site.pp defines it as "$xmlrpc_enabled = true" so I think we are good with that.
Comment 17 Steffen Pingel CLA 2013-04-26 14:52:19 EDT
Not sure if it's related to recent changes but noticed an exception that got logged in this build:

https://hudson.eclipse.org/sandbox/job/mylyn-tasks-gerrit/471/console

[2013-04-26T2-47-40] Status ERROR: org.eclipse.mylyn.tasks.ui code=0 Error opening task org.eclipse.core.runtime.CoreException: Error reading task data, Exception:
org.eclipse.core.runtime.CoreException: Error reading task data
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataStore.readState(TaskDataStore.java:153)
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataStore.getTaskDataState(TaskDataStore.java:57)
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataManager$1.execute(TaskDataManager.java:122)
	at org.eclipse.mylyn.internal.tasks.core.TaskList.run(TaskList.java:673)
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataManager.getWorkingCopy(TaskDataManager.java:119)
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataManager.getWorkingCopy(TaskDataManager.java:110)
	at org.eclipse.mylyn.tasks.ui.editors.AbstractTaskEditorPage.createModel(AbstractTaskEditorPage.java:733)
	at org.eclipse.mylyn.internal.bugzilla.ui.editor.BugzillaTaskEditorPage.createModel(BugzillaTaskEditorPage.java:397)
	at org.eclipse.mylyn.tasks.ui.editors.AbstractTaskEditorPage.initModel(AbstractTaskEditorPage.java:1280)
	at org.eclipse.mylyn.tasks.ui.editors.AbstractTaskEditorPage.init(AbstractTaskEditorPage.java:1272)
	at org.eclipse.ui.forms.editor.FormEditor.registerPage(FormEditor.java:657)
	at org.eclipse.ui.forms.editor.FormEditor.configurePage(FormEditor.java:348)
	at org.eclipse.ui.forms.editor.FormEditor.addPage(FormEditor.java:190)
	at org.eclipse.mylyn.tasks.ui.editors.TaskEditor.addPages(TaskEditor.java:410)
	at org.eclipse.ui.forms.editor.FormEditor.createPages(FormEditor.java:138)
	at org.eclipse.ui.forms.editor.SharedHeaderFormEditor.createPages(SharedHeaderFormEditor.java:98)
	at org.eclipse.mylyn.tasks.ui.editors.TaskEditor.createPages(TaskEditor.java:196)
	at org.eclipse.ui.part.MultiPageEditorPart.createPartControl(MultiPageEditorPart.java:348)
	at org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:670)
	at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:465)
	at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
	at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:313)
	at org.eclipse.ui.internal.presentations.PresentablePart.setVisible(PresentablePart.java:180)
	at org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(PresentablePartFolder.java:270)
	at org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(LeftToRightTabOrder.java:65)
	at org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectPart(TabbedStackPresentation.java:473)
	at org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.java:1245)
	at org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1198)
	at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1597)
	at org.eclipse.ui.internal.PartStack.add(PartStack.java:493)
	at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:103)
	at org.eclipse.ui.internal.PartStack.add(PartStack.java:479)
	at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:112)
	at org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.java:63)
	at org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java:225)
	at org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:213)
	at org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:808)
	at org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManager.java:707)
	at org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:666)
	at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.java:2942)
	at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:2850)
	at org.eclipse.ui.internal.WorkbenchPage.access$11(WorkbenchPage.java:2842)
	at org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2793)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2789)
	at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2773)
	at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2756)
	at org.eclipse.mylyn.tasks.ui.TasksUiUtil.openEditor(TasksUiUtil.java:182)
	at org.eclipse.mylyn.internal.tasks.ui.util.TasksUiInternal.createAndOpenNewTask(TasksUiInternal.java:866)
	at org.eclipse.mylyn.bugzilla.tests.ui.BugzillaTaskEditorTest.testOpenNewEditor(BugzillaTaskEditorTest.java:72)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at junit.framework.TestCase.runTest(TestCase.java:168)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at org.eclipse.mylyn.commons.sdk.util.ManagedTestSuite.run(ManagedTestSuite.java:152)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
	at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
	at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
	at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:123)
	at org.eclipse.tycho.surefire.osgibooter.OsgiSurefireBooter.run(OsgiSurefireBooter.java:85)
	at org.eclipse.tycho.surefire.osgibooter.AbstractUITestApplication$1.run(AbstractUITestApplication.java:35)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3563)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3212)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2701)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2665)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2499)
	at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:679)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:668)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:123)
	at org.eclipse.tycho.surefire.osgibooter.UITestApplication.runApplication(UITestApplication.java:31)
	at org.eclipse.tycho.surefire.osgibooter.AbstractUITestApplication.run(AbstractUITestApplication.java:114)
	at org.eclipse.tycho.surefire.osgibooter.UITestApplication.start(UITestApplication.java:37)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1386)
Caused by: java.io.IOException: Error parsing task data: Repository "http://mylyn.org/bugzilla-4.2.5" not found for kind "bugzilla"
	at org.eclipse.mylyn.internal.tasks.core.data.TaskDataStore.readState(TaskDataStore.java:148)
	... 106 more
Comment 18 Steffen Pingel CLA 2013-04-26 14:52:44 EDT
I proposed a fix here: https://git.eclipse.org/r/#/c/12248/.
Comment 19 Frank Becker CLA 2013-04-27 07:51:26 EDT

(In reply to comment #16)
> (In reply to comment #15)
> > Maybe we should change the following to 3.6.13
> > * bugzilla-3.6.12-custom-wf
> > * bugzilla-3.6.12-custom-wf-and-status
> > * bugzilla-3.6.12-xml-rpc-disabled
> 
> Makes sense. That should be straight forward. Can you push a review?
> 
> > Only include properties if they diverge from the default, i.e. the listing on
> > http://mylyn.org/
> >
> > The value of property 'xmlrpc_enabled' is not really but we have no place
> where
> > we put this on by default.
> 
> I consider that to be the default since it's set for all configurations except
> one and site.pp defines it as "$xmlrpc_enabled = true" so I think we are good
> with that.
here the review https://git.eclipse.org/r/12270
Comment 20 Frank Becker CLA 2013-04-27 07:51:30 EDT
Created attachment 230209 [details]
mylyn/context/zip
Comment 21 Steffen Pingel CLA 2013-04-27 09:15:40 EDT
Thanks! I submitted the review and will re-provision the Bugzilla repositories on the server shortly.