Bugzilla – Full Text Bug Listing
| Summary: | Random test failures | ||
|---|---|---|---|
| Product: | [ECD] Orion | Reporter: | Michael Rennie <Michael_Rennie> |
| Component: | JS Tools | Assignee: | Curtis Windatt <curtis.windatt.public> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | mamacdon |
| Version: | 10.0 | ||
| Target Milestone: | 10.0 | ||
| Hardware: | PC | ||
| OS: | Mac OS X | ||
| Whiteboard: | |||
|
Description
Michael Rennie
Curtis, can you have a look please? Mark can explain how to run the tests locally using Sauce. (In reply to Michael Rennie from comment #1) > Curtis, can you have a look please? > > Mark can explain how to run the tests locally using Sauce. This script will run the tests: https://github.com/eclipse/orion.client/blob/master/releng/org.eclipse.orion.client.releng/run-tests-locally.sh You'll need to have Node, npm, and grunt (just do 'npm install -g grunt-cli') installed. @Curtis contact me privately and I will give you the values for SAUCE_USERNAME and SAUCE_ACCESS_KEY. Thanks for helping me to get set up Mark. Looks like the cause of the failure in the last build is the crossFileTests. I get the same 120sec timeout from JSMochaSuite running locally, and the suite passes if CrossFileTests is removed. I have seen this happen while self hosting, where the crossfile tests sit until they time out, but it is trivial to restart them. When they run, the cross file tests run quickly. So perhaps we just need the tests to fail sooner and not hit the 120sec timeout. This also does not explain the random test failures from the ESLint suite in the builds prior to the timeout. http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=9f4d5573277fe1f9df5c433b9af1e833531b0b27 Sends an error callback if the xhr request to get a file fails. If the tests are run from the test directory, rather than server root, /js-tests/javascript won't be appended to the url. http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=3c4419d440ecc7251235bd594d6fa68acd3b5c27 After sending the error callback to the test, it also sends an error response back to the Tern server. If it doesn't receive a response, Tern would stop responding to 'implementation' requests. This would cause the tests to time out. I didn't add any checks for unknown worker requests for now. We don't respect the delFile request which we call during each test setup. I don't think a check here would be helpful. There is definitely some flakiness when the read request fails. Maybe we just need to send back a specific error response. Currently if I just a send an error response to Tern, the tests will simply get back 'no implementations found'. Or maybe in the openImplementationPlugin or in our handler we are ignoring some error information. http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=95af7c8e98e721ada5f8b04d03f4ba0aded16cb8 Adds code to handle absolute paths and relative paths whether running at the root or the test folder. I am adding a new test that ends up using orion/object which the current worker code tries to look up relative to /javascript/js-tests even though it isn't relative. Throwing an exception in resolver when the XHR file lookup failed doesn't bubble up to the tests as expected. Probably because it is running an asynch operation the error is uncaught, noted on the console and the test just times out. http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=5d8c8eca4358ee9fa033696d174eeead02c66f71 Content assist requests default to a 5 sec timeout. If this timeout is reached we simply return an empty array. This commit adds in an optional return object so the tests can be aware that a timeout happened. No idea if a timeout is the issue, considering some of the requests finish in under 50ms, but that is my best explanation for the spurious failures. Note that we can't always return an error message as the editor interprets a string return as a single valid proposal. http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=342b71221248879558d7de0f0982bd18a6b42935 Mike's fix to ensure the deferred isn't sitting around appears to have solved the remaining random test failures. The tests are passing in the builds so I am marking this as FIXED. |