Community
Participate
Working Groups
In order to run automated UI tests for EPP with Jubula we need a database that is accessible from Hudson slaves.
Felix, can you provide more background what databases are supported? I am not sure that we have the option to run an Oracle database.
*** Bug 364554 has been marked as a duplicate of this bug. ***
Well, technically we do have access to an Oracle database. It just happens to be MySQL is all ;) So do you just need access to a dbserver of some kind or is there something more specific? Is network access ok or does it need to be 'local'? -M.
(In reply to comment #3) > Well, technically we do have access to an Oracle database. It just happens to > be MySQL is all ;) > > So do you just need access to a dbserver of some kind or is there something > more specific? Is network access ok or does it need to be 'local'? > > -M. The database contains the Jubula test definitions and test results. Therefore we need network access for every user to maintain and extend the tests and for the Hudson jobs to run the tests. I would prefer Oracle because this is the supported and tested database for Jubula. (See https://s3.amazonaws.com/jubula/install.pdf)
Well, it's unlikely we will ever run an Oracle db. The best I can do is provide access to a mysql db on build. You won't be able to access it 'remotely'(from your office) but from build.eclipse.org will be fine. -M.
> but from build.eclipse.org will be fine. .. and from the Hudson master & non-test slaves too.
We also can use an H2 db locally that we will use for the test runs. We can add tests by exporting/importing them via xml. then we don´t need an oracle db. Then we need a server for the db and the jubula installation. And as a start a Win32 VM for the eclipse packages. Then we can start tests via Hudson.
(In reply to comment #7) > Then we need a server for the db and the jubula installation. > And as a start a Win32 VM for the eclipse packages. I'm confused. It sounded like you were going to run the server 'locally'? Where does the Windows VM fit into this? Perhaps I should be asking: Given our current hudson setup(with unix,OSX and Windows slaves) what do you need that isn't available? -M.
When we can use given slaves and put the current jubula agent on it, we don´t need any further VMs. The agents will start the eclipse packages on every target. (Lin, Win Mac) But we still need the jubula client for connecting to the agents and start the test runs. This client should use the H2 db locally. The client test runs itself will be started via Hudson. The content of the db (test libs and projects) can be exchange via a xml export by the package maintainers if they want to add tests. Writing new tests can be done in the package maintainers environment locally. The new tests can be imported back into the db.
(In reply to comment #9) > When we can use given slaves and put the current jubula agent on it, we don´t > need any further VMs. The agents will start the eclipse packages on every > target. > (Lin, Win Mac) Would we (need to) keep the agents running similar to how the Hudson agents are already running on the slaves? > But we still need the jubula client for connecting to the agents and start the > test runs. This client should use the H2 db locally. The client test runs itself > will be started via Hudson. This would only be needed on the slave where the initial build runs, right? Would it make sense to limit this to one of slaves, e.g. "master" to simplify setup? > The content of the db (test libs and projects) can be exchange via a xml export > by the package maintainers if they want to add tests. Writing new tests can be > done in the package maintainers environment locally. The new tests can be > imported back into the db. That sounds like a good solution to me to avoid the hassle of setting up a database.
(In reply to comment #10) > Would we (need to) keep the agents running similar to how the Hudson agents are > already running on the slaves? I was wondering about that. Can you start the agents via an ant task(or some other script)? I think that's how the platform team starts Eclipse to run their UI tests. -M.
> I was wondering about that. Can you start the agents via an ant task(or some > other script)? Yes, we can start that via hudson. 1. Start autagent on the target 2. Run test suite via Jubula client 3. Stop autagent If we just have one jubula client for this then we have to run this one after the other (target by target, package by package). But I think it isn´t critical right now, we can add more clients if we really need them.
> Perhaps I should be asking: Given our current hudson setup(with unix,OSX and > Windows slaves) what do you need that isn't available? I have some other questions: 1. Do the slaves have a GUI frontend where jubula can work on? 2. It is necessary that we have a clean test environment to avoid side effects. Are there any other jobs (or user interaction) on the servers that can disturb the jubula test run? 3. How can we get access to add everything we need?
(In reply to comment #13) > 1. Do the slaves have a GUI frontend where jubula can work on? The Linux slaves use the Hudson Xvnc plug-in to start Xvnc as part of running the job. Not sure about Windows but I would assume that we have the default Windows UI available. > 2. It is necessary that we have a clean test environment to avoid side effects. > Are there any other jobs (or user interaction) on the servers that can disturb > the jubula test run? Each Hudson job gets an isolated Xvnc. > 3. How can we get access to add everything we need? Once a job has been created committers can configure it as needed.
> Each Hudson job gets an isolated Xvnc. Is it possible to get a virtual machine for each job? We used Xvnc about two years ago and we had some timing problems with that solution. At bredex we use VMWare but I think solutions like VirtualBox should also work.
(In reply to comment #15) > Is it possible to get a virtual machine for each job? No. We are presently out of room for new virtual servers, and those are linux only(without guis so you'd still need to consider xvnc). -M.
Felix, what if we scoped the initial deployment to the following: * Use database serialized from XML that is checked into source repository * Run on Hudson Linux slave only in an exclusive Xvnc instance (we can add tests on Windows later) What would we need in terms of infrastructure to be present on the slave? I think ideally we would be able to provision the Jubula agent as part of the build, e.g. through a Maven plug-in but I assume that's not (yet) supported?
After discussing this terms internally here at Bredex we decided that we should involve you in testing your EPPs first and thinking about the technical details later. Therefore we want to set up a git for the test projects and the test environment here at bredex. Steffen if you like to be the guinea pig, you are welcome ;-). I will inform you here when everything is ready. Once the smoke tests are ready, we still can decide what should be moved to build.eclipse.org.
Thanks Felix. That sounds reasonable as an interim solution. In the interest of vendor neutrality we should figure out what is missing on Eclipse.org in the long term or how we can integrate with the existing infrastructure.
Moving to Hudson. Steffen, what's left to do here?
Good question. I haven't looked at this in a while. I'll chat with the Jubula team at EclipseCon to check what the next steps would be to move this forward.
Please keep me in the loop -- I'll be at ECon as well and can participate if needed.
We had a quick discussion and this is what we came up with: * Download and extract the EPP build * Create a job that imports the DB from a Git repository using dbtool * Run the aut using runaut * Execute tests using testrun From a quick look it seems that Jubula command line tools are all OSGi applications. Markus, Is that correct? It could be most simple to use Tycho's tycho-eclipserun-plugin [1] to run the applications in that case assuming they are published to a p2 repository. Has anyone tried this before? Is there any ready to use automation for downloading and extracting an OSGi application to prepare the testbed? [1] ycho-eclipserun-plugin
Based on what I have come up with on bug 364551 comment 1 we don't necessarily need to install anything to do some basic Jubula testing for EPP. At least initially it seems that we are best of starting with the most basic setup that just runs everything from Maven and shares the database through git. Felix, if you are still interested in providing some basic tests we should look at the stuff on bug 364551 and see if we can build on that. I'll close this bug for now and move further discussion to bug 364551.
