Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 347402

Summary:

The build takes too long

Product:

[RT] Jetty

Reporter:

Greg Wilkins <gregw>

Component:

build

Assignee:

Joakim Erdfelt <joakim.erdfelt>

Status:

RESOLVED FIXED

QA Contact:

Severity:

enhancement

Priority:

CC:

jetty-inbox, joakim.erdfelt

Version:

7.4.1

Target Milestone:

7.2.x

Hardware:

All

OS:

All

See Also:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=352991
https://bugs.eclipse.org/bugs/show_bug.cgi?id=352999
https://bugs.eclipse.org/bugs/show_bug.cgi?id=353509

Whiteboard:

Attachments:

Description	Flags
modules which take more than a second to unit test	none

Description Greg Wilkins

2011-05-27 03:12:20 EDT

The build is taking > 13 minutes on a 8 CPU machine and is hardly using any CPU.  There are too many tests that do too much waiting.

We need to reduce the number of tests that do stuff like Thread.sleep(2000) and that is a) unreliable b) a waste of time.

Code should be able to be unit tested without having to do waits.

Comment 1 Thomas Becker

2011-05-27 12:15:02 EDT

Created attachment 196782 [details]
modules which take more than a second to unit test

Added a screenshot of all modules which take more than a second to execute it's unit tests.

Comment 2 Joakim Erdfelt

2011-09-12 13:02:09 EDT

Here's the current timing information for 7.5.2-SNAPSHOT

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Jetty :: Project .................................. SUCCESS [8.838s]
[INFO] Jetty :: Utilities ................................ SUCCESS [41.455s]
[INFO] Jetty :: JMX Management ........................... SUCCESS [9.927s]
[INFO] Jetty :: IO Utility ............................... SUCCESS [6.678s]
[INFO] Jetty :: Http Utility ............................. SUCCESS [5.030s]
[INFO] Jetty :: Continuation ............................. SUCCESS [1.998s]
[INFO] Jetty :: Server Core .............................. SUCCESS [3:33.950s]
[INFO] Jetty :: Websocket ................................ SUCCESS [37.929s]
[INFO] Jetty :: Security ................................. SUCCESS [12.746s]
[INFO] Jetty :: Servlet Handling ......................... SUCCESS [5.584s]
[INFO] Jetty :: Asynchronous HTTP Client ................. SUCCESS [1:30.419s]
[INFO] Jetty :: XML utilities ............................ SUCCESS [3.229s]
[INFO] Jetty :: JASPI Security ........................... SUCCESS [2.533s]
[INFO] Jetty :: Webapp Application Support ............... SUCCESS [5.006s]
[INFO] Test :: Jetty Servlet Tester ...................... SUCCESS [2.509s]
[INFO] Jetty :: Utility Servlets and Filters ............. SUCCESS [56.856s]
[INFO] Jetty :: Jetty JSP Additions ...................... SUCCESS [1.602s]
[INFO] Jetty :: Deployers ................................ SUCCESS [21.800s]
[INFO] Jetty :: AJP ...................................... SUCCESS [6.521s]
[INFO] Jetty :: JNDI Naming .............................. SUCCESS [2.620s]
[INFO] Jetty :: Plus ..................................... SUCCESS [3.548s]
[INFO] Jetty :: Servlet Annotations ...................... SUCCESS [2.865s]
[INFO] Jetty :: Rewrite Handler .......................... SUCCESS [5.000s]
[INFO] Jetty :: Policy Tool .............................. SUCCESS [20.977s]
[INFO] Jetty :: Monitoring ............................... SUCCESS [2:11.860s]
[INFO] Jetty :: Start .................................... SUCCESS [3.204s]
[INFO] Jetty :: Nested ................................... SUCCESS [1.600s]
[INFO] Jetty :: Overlay Deployer ......................... SUCCESS [7.999s]
[INFO] Jetty Tests :: Parent ............................. SUCCESS [0.078s]
[INFO] Jetty Tests :: Sessions :: Parent ................. SUCCESS [0.042s]
[INFO] Jetty Tests :: Sessions :: Common ................. SUCCESS [2.233s]
[INFO] Jetty :: NoSQL Session Managers ................... SUCCESS [2.942s]
[INFO] Test :: Jetty Test Webapp ......................... SUCCESS [4.637s]
[INFO] Jetty :: Distribution Assemblies .................. SUCCESS [3.737s]
[INFO] Test :: Continuation .............................. SUCCESS [6.696s]
[INFO] Test :: Continuation - (Jetty 6) .................. SUCCESS [11.122s]
[INFO] Jetty :: Nested Test .............................. SUCCESS [2.323s]
[INFO] Example :: Jetty Embedded ......................... SUCCESS [2.737s]
[INFO] Jetty Tests :: WebApps :: Parent .................. SUCCESS [0.095s]
[INFO] Jetty Tests :: WebApp :: RFC2616 .................. SUCCESS [1.482s]
[INFO] Jetty Tests :: Integrations ....................... SUCCESS [1:26.124s]
[INFO] Jetty Tests :: Sessions :: Hash ................... SUCCESS [19.094s]
[INFO] Jetty Tests :: Sessions :: JDBC ................... SUCCESS [1:01.200s]
[INFO] Jetty Tests :: Login Service ...................... SUCCESS [4.666s]
[INFO] Jetty :: Aggregate Project ........................ SUCCESS [0.082s]
[INFO] Jetty :: Aggregate :: HTTP Server ................. SUCCESS [2.779s]
[INFO] Jetty :: Aggregate :: HTTP Client ................. SUCCESS [2.383s]
[INFO] Jetty :: Aggregate :: Servlet Server .............. SUCCESS [2.886s]
[INFO] Jetty :: Aggregate :: WebApp Server ............... SUCCESS [3.324s]
[INFO] Jetty :: Aggregate :: Plus Server ................. SUCCESS [3.248s]
[INFO] Jetty :: Aggregate :: All Server .................. SUCCESS [4.346s]
[INFO] Jetty :: Aggregate :: All core Jetty .............. SUCCESS [4.427s]
[INFO] Jetty :: OSGi ..................................... SUCCESS [0.102s]
[INFO] Jetty :: OSGi :: Boot ............................. SUCCESS [4.134s]
[INFO] Jetty :: OSGi :: Boot JSP ......................... SUCCESS [2.653s]
[INFO] Jetty :: OSGi :: Boot Logback ..................... SUCCESS [4.853s]
[INFO] Jetty :: OSGi :: Boot :: Warurl ................... SUCCESS [2.450s]
[INFO] Jetty :: OSGi :: HttpService ...................... SUCCESS [2.403s]
[INFO] Jetty :: OSGi :: Example Equinox Tools ............ SUCCESS [2.742s]
[INFO] Jetty :: OSGi :: Test ............................. SUCCESS [10.396s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:20.312s
[INFO] Finished at: Mon Sep 12 09:59:41 MST 2011
[INFO] Final Memory: 188M/400M
[INFO] ------------------------------------------------------------------------

Comment 3 Joakim Erdfelt

2011-09-12 14:49:03 EDT

Thread.sleep in test cases is pretty prevalent.


[joakim@lapetus org.eclipse.jetty.project][master]$ grep -RHn Thread.sleep * | grep -v aggregate | grep "src/test" | sed -e "s/^.*:[0-9]*:.*\(Thread.sleep[^)]*)\).*/\1/g"  | sort | uniq -c | sort -nr
     23 Thread.sleep(100)
     22 Thread.sleep(200)
     21 Thread.sleep(10)
     20 Thread.sleep(1000)
     14 Thread.sleep(500)
     11 Thread.sleep(50)
      7 Thread.sleep(2000)
      6 Thread.sleep(PAUSE)
      5 Thread.sleep(25)
      4 Thread.sleep(1100)
      3 Thread.sleep(5)
      3 Thread.sleep(300)
      3 Thread.sleep(1)
      2 Thread.sleep(wait)
      2 Thread.sleep(_timeout * 2)
      2 Thread.sleep(sleep)
      2 Thread.sleep(interval)
      2 Thread.sleep(500L)
      2 Thread.sleep(5000)
      2 Thread.sleep(4000)
      2 Thread.sleep(250)
      2 Thread.sleep(1+random.nextInt(10)
      2 Thread.sleep(1500)
      2 Thread.sleep(1000L)
      2 Thread.sleep(10000)
      1 Thread.sleep(TIMEOUT*4)
      1 Thread.sleep(timeout * 2)
      1 Thread.sleep(sleep.get()
      1 Thread.sleep(sleep_for)
      1 Thread.sleep(scavenge * 3000)
      1 Thread.sleep( runTestLength )
      1 Thread.sleep(_resumeAfter)
      1 Thread.sleep(_requestMaxTime * 2)
      1 Thread.sleep(requestInterval)
      1 Thread.sleep(_random.nextInt(290)
      1 Thread.sleep(random.nextInt(1000)
      1 Thread.sleep(_random.nextInt(100)
      1 Thread.sleep(pause1)
      1 Thread.sleep(pause0)
      1 Thread.sleep(Long.parseLong(request.getParameter("sleep")
      1 Thread(sleeper)
      1 Thread.sleep(DURATION)
      1 Thread.sleep(_completeAfter)
      1 Thread.sleep(320000)
      1 Thread.sleep(3000)
      1 Thread.sleep(30)
      1 Thread.sleep(2 * JdbcTestServer.SAVE_INTERVAL * 1000)
      1 Thread.sleep(20)
      1 Thread.sleep(1+_random.nextInt(20)
      1 Thread.sleep(1001)

NOTE: Some of these are likely commented out in the test code.

Comment 4 Joakim Erdfelt

2011-12-08 13:08:20 EST

Closing this as a bug.
The basic improvements have been performed.
Further improvements will occur in an iterative fashion.