Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 481281 - A ui.tests.performance test "bombs out" ?
Summary: A ui.tests.performance test "bombs out" ?
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.6   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-UI-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks: 454921 480076
  Show dependency tree
 
Reported: 2015-11-02 21:29 EST by David Williams CLA
Modified: 2020-05-24 16:49 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2015-11-02 21:29:06 EST
Or, maybe related to Memory, but in either case, the culprit appears to be 

org.eclipse.ui.tests.performance.UIPerformanceTestSuite


The initial part of the log (on my local system) is as follows. 

I'll check if production machine has similar. 

----- Generate 10000 identifiers
Generate 10000 identifiers: setUp...
Trying to connect over network with // jdbc protocol;  org.apache.derby.jdbc.ClientDriver to //192.168.1.10:1527/perfDB ...
connect succeeded!
Generate 10000 identifiers: tearDown...

!SESSION 2015-11-02 22:34:18.766 -----------------------------------------------
eclipse.buildId=4.5.0.I20150603-2000
java.version=1.8.0_60
java.vendor=Oracle Corporation
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments:  -application org.eclipse.test.uitestapplication formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,/data/shared/hudson/hudsonhome/jobs/ep46I-perf-lin64-baseline/workspace/workarea/I20151102-2015/eclipse-testing/test-eclipse/eclipse/org.eclipse.ui.tests.performance.UIPerformanceTestSuite.xml -testPluginName org.eclipse.ui.tests.performance -className org.eclipse.ui.tests.performance.UIPerformanceTestSuite -timeout 7200000 -junitReportOutput /data/shared/hudson/hudsonhome/jobs/ep46I-perf-lin64-baseline/workspace/workarea/I20151102-2015/eclipse-testing/results/linux.gtk.x86_64_8.0
Command-line arguments:  -application org.eclipse.test.uitestapplication -data /data/shared/hudson/hudsonhome/jobs/ep46I-perf-lin64-baseline/workspace/workarea/I20151102-2015/eclipse-testing/test-eclipse/eclipse/performance-workspace-platform-ui formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,/data/shared/hudson/hudsonhome/jobs/ep46I-perf-lin64-baseline/workspace/workarea/I20151102-2015/eclipse-testing/test-eclipse/eclipse/org.eclipse.ui.tests.performance.UIPerformanceTestSuite.xml -testPluginName org.eclipse.ui.tests.performance -className org.eclipse.ui.tests.performance.UIPerformanceTestSuite -os linux -ws gtk -arch x86_64 -consolelog -timeout 7200000 -junitReportOutput /data/shared/hudson/hudsonhome/jobs/ep46I-perf-lin64-baseline/workspace/workarea/I20151102-2015/eclipse-testing/results/linux.gtk.x86_64_8.0

!ENTRY org.eclipse.equinox.util 4 0 2015-11-02 22:34:49.581
!MESSAGE Unable to create more threads!
Active Thread Pool tasks: 0
!STACK 0
java.lang.RuntimeException: buffer fill failed: java.lang.OutOfMemoryError: unable to create new native thread
	at org.eclipse.equinox.internal.util.pool.ObjectPool.getObject(ObjectPool.java:158)
	at org.eclipse.equinox.internal.util.impl.tpt.threadpool.ThreadPoolManagerImpl.getObject(ThreadPoolManagerImpl.java:97)
	at org.eclipse.equinox.internal.util.impl.tpt.threadpool.ThreadPoolManagerImpl.execute(ThreadPoolManagerImpl.java:203)
	at org.eclipse.equinox.internal.util.impl.tpt.threadpool.ThreadPoolFactoryImpl.execute0(ThreadPoolFactoryImpl.java:112)
	at org.eclipse.equinox.internal.util.impl.tpt.timer.TimerImpl.run(TimerImpl.java:110)
	at java.lang.Thread.run(Thread.java:745)
----- Massively Recursive TrimLayout computeSize
Massively Recursive TrimLayout computeSize: setUp...
Comment 1 David Williams CLA 2015-11-03 16:03:03 EST
So far, I do not see this same error pattern on the production machine. But, there are other exceptions. Such for recent I-build, see
https://hudson.eclipse.org/perftests/job/ep46I-perf-lin64-baseline/ws/workarea/I20151103-0800/eclipse-testing/results/linux.gtk.x86_64_8.0/org.eclipse.ui.tests.performance.UIPerformanceTestSuite.txt



So, the 'thread issue' might depend on some machine (OS) specific setting? 
(BTW, that is an important problem, to me, since when I try to run locally, 
the excessive threads ends up hosing up my whole machine! See bug 480076.)

BUT, even on the production machine, I do not think the test is working correctly. 

all "text log files" I've looked at, for performance tests, have messages such as 

Trying to connect over network with // jdbc protocol;  org.apache.derby.jdbc.ClientDriver to //172.25.25.57:1527/perfDB ...
connect succeeded!

[... and then eventually conclude with ... ]

stored 7 new datapoints in DB
disconnecting from DB

This test does have the "connect succeeded", but never a message about stored datapoints, nor disconnection -- and, if in fact it does not "disconnect", that might lead to other subsequent problems?
Comment 2 David Williams CLA 2015-11-03 21:49:00 EST
Removed "creating too many threads" from title. 

I do think it may create excessive threads, but that's probably not a problem on production machine. 

I'll add more detail about some findings to bug  bug 480076.
Comment 3 David Williams CLA 2015-11-03 22:39:23 EST
(In reply to David Williams from comment #1)

> This test does have the "connect succeeded", but never a message about
> stored datapoints, nor disconnection -- and, if in fact it does not
> "disconnect", that might lead to other subsequent problems?

Note: when ran on "current build" there are is some of the same errors (I think), but, perhaps not as many. Such as see 

https://hudson.eclipse.org/perftests/view/Eclipse%20and%20Equinox/job/ep46I-perf-lin64/ws/workarea/I20151103-0800/eclipse-testing/results/linux.gtk.x86_64_8.0/org.eclipse.ui.tests.performance.UIPerformanceTestSuite.txt

And, in the end, does write 

stored 50 new datapoints in DB
disconnecting from DB

= = = = = 

This is significant if it runs on current build, but not on baseline (4.5). 

The reason for significance is that we actually run exactly the same tests -- from current build. So ... this implies there may be a "breaking change" from 4.6 to 4.5. 

That might be "ok", if using "highly internal behaviour", but, if using API then this might be problematic for the 4.6 release. That is, the "broken API" does not cause compile errors, but causes a change in behaviour? 

I suggest a close look be taken, and if the solution is not obvious, (where, solution means a test that works on 4.6 and 4.5) then I suggest the test be disabled, for now. It won't do much good to have a "regression test" without any reference data to compare it to. (It might still do some good ... but ... just "not very much good").
Comment 4 David Williams CLA 2015-11-03 23:07:33 EST
According to the "unit test summaries" -- for the performance tests -- it appears there are two (other?) tests that always fail on both current, and baseline, looking for "ECommandService". 

See 

http://download.eclipse.org/eclipse/downloads/drops4/I20151103-0800/baseline/html/org.eclipse.ui.tests.performance_linux.gtk.x86_64_8.0.html

http://download.eclipse.org/eclipse/downloads/drops4/I20151103-0800/performance/html/org.eclipse.ui.tests.performance_linux.gtk.x86_64_8.0.html

Please "comment out" these tests too, unless a  safe, quick fix is obvious.
Comment 5 Eclipse Genie CLA 2020-05-24 16:49:47 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.