Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 347402 - The build takes too long
Summary: The build takes too long
Status: RESOLVED FIXED
Alias: None
Product: Jetty
Classification: RT
Component: build (show other bugs)
Version: 7.4.1   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 7.2.x   Edit
Assignee: Joakim Erdfelt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-27 03:12 EDT by Greg Wilkins CLA
Modified: 2011-12-08 13:08 EST (History)
2 users (show)

See Also:


Attachments
modules which take more than a second to unit test (106.70 KB, image/png)
2011-05-27 12:15 EDT, Thomas Becker CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Greg Wilkins CLA 2011-05-27 03:12:20 EDT
The build is taking > 13 minutes on a 8 CPU machine and is hardly using any CPU.  There are too many tests that do too much waiting.

We need to reduce the number of tests that do stuff like Thread.sleep(2000) and that is a) unreliable b) a waste of time.

Code should be able to be unit tested without having to do waits.
Comment 1 Thomas Becker CLA 2011-05-27 12:15:02 EDT
Created attachment 196782 [details]
modules which take more than a second to unit test

Added a screenshot of all modules which take more than a second to execute it's unit tests.
Comment 2 Joakim Erdfelt CLA 2011-09-12 13:02:09 EDT
Here's the current timing information for 7.5.2-SNAPSHOT

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Jetty :: Project .................................. SUCCESS [8.838s]
[INFO] Jetty :: Utilities ................................ SUCCESS [41.455s]
[INFO] Jetty :: JMX Management ........................... SUCCESS [9.927s]
[INFO] Jetty :: IO Utility ............................... SUCCESS [6.678s]
[INFO] Jetty :: Http Utility ............................. SUCCESS [5.030s]
[INFO] Jetty :: Continuation ............................. SUCCESS [1.998s]
[INFO] Jetty :: Server Core .............................. SUCCESS [3:33.950s]
[INFO] Jetty :: Websocket ................................ SUCCESS [37.929s]
[INFO] Jetty :: Security ................................. SUCCESS [12.746s]
[INFO] Jetty :: Servlet Handling ......................... SUCCESS [5.584s]
[INFO] Jetty :: Asynchronous HTTP Client ................. SUCCESS [1:30.419s]
[INFO] Jetty :: XML utilities ............................ SUCCESS [3.229s]
[INFO] Jetty :: JASPI Security ........................... SUCCESS [2.533s]
[INFO] Jetty :: Webapp Application Support ............... SUCCESS [5.006s]
[INFO] Test :: Jetty Servlet Tester ...................... SUCCESS [2.509s]
[INFO] Jetty :: Utility Servlets and Filters ............. SUCCESS [56.856s]
[INFO] Jetty :: Jetty JSP Additions ...................... SUCCESS [1.602s]
[INFO] Jetty :: Deployers ................................ SUCCESS [21.800s]
[INFO] Jetty :: AJP ...................................... SUCCESS [6.521s]
[INFO] Jetty :: JNDI Naming .............................. SUCCESS [2.620s]
[INFO] Jetty :: Plus ..................................... SUCCESS [3.548s]
[INFO] Jetty :: Servlet Annotations ...................... SUCCESS [2.865s]
[INFO] Jetty :: Rewrite Handler .......................... SUCCESS [5.000s]
[INFO] Jetty :: Policy Tool .............................. SUCCESS [20.977s]
[INFO] Jetty :: Monitoring ............................... SUCCESS [2:11.860s]
[INFO] Jetty :: Start .................................... SUCCESS [3.204s]
[INFO] Jetty :: Nested ................................... SUCCESS [1.600s]
[INFO] Jetty :: Overlay Deployer ......................... SUCCESS [7.999s]
[INFO] Jetty Tests :: Parent ............................. SUCCESS [0.078s]
[INFO] Jetty Tests :: Sessions :: Parent ................. SUCCESS [0.042s]
[INFO] Jetty Tests :: Sessions :: Common ................. SUCCESS [2.233s]
[INFO] Jetty :: NoSQL Session Managers ................... SUCCESS [2.942s]
[INFO] Test :: Jetty Test Webapp ......................... SUCCESS [4.637s]
[INFO] Jetty :: Distribution Assemblies .................. SUCCESS [3.737s]
[INFO] Test :: Continuation .............................. SUCCESS [6.696s]
[INFO] Test :: Continuation - (Jetty 6) .................. SUCCESS [11.122s]
[INFO] Jetty :: Nested Test .............................. SUCCESS [2.323s]
[INFO] Example :: Jetty Embedded ......................... SUCCESS [2.737s]
[INFO] Jetty Tests :: WebApps :: Parent .................. SUCCESS [0.095s]
[INFO] Jetty Tests :: WebApp :: RFC2616 .................. SUCCESS [1.482s]
[INFO] Jetty Tests :: Integrations ....................... SUCCESS [1:26.124s]
[INFO] Jetty Tests :: Sessions :: Hash ................... SUCCESS [19.094s]
[INFO] Jetty Tests :: Sessions :: JDBC ................... SUCCESS [1:01.200s]
[INFO] Jetty Tests :: Login Service ...................... SUCCESS [4.666s]
[INFO] Jetty :: Aggregate Project ........................ SUCCESS [0.082s]
[INFO] Jetty :: Aggregate :: HTTP Server ................. SUCCESS [2.779s]
[INFO] Jetty :: Aggregate :: HTTP Client ................. SUCCESS [2.383s]
[INFO] Jetty :: Aggregate :: Servlet Server .............. SUCCESS [2.886s]
[INFO] Jetty :: Aggregate :: WebApp Server ............... SUCCESS [3.324s]
[INFO] Jetty :: Aggregate :: Plus Server ................. SUCCESS [3.248s]
[INFO] Jetty :: Aggregate :: All Server .................. SUCCESS [4.346s]
[INFO] Jetty :: Aggregate :: All core Jetty .............. SUCCESS [4.427s]
[INFO] Jetty :: OSGi ..................................... SUCCESS [0.102s]
[INFO] Jetty :: OSGi :: Boot ............................. SUCCESS [4.134s]
[INFO] Jetty :: OSGi :: Boot JSP ......................... SUCCESS [2.653s]
[INFO] Jetty :: OSGi :: Boot Logback ..................... SUCCESS [4.853s]
[INFO] Jetty :: OSGi :: Boot :: Warurl ................... SUCCESS [2.450s]
[INFO] Jetty :: OSGi :: HttpService ...................... SUCCESS [2.403s]
[INFO] Jetty :: OSGi :: Example Equinox Tools ............ SUCCESS [2.742s]
[INFO] Jetty :: OSGi :: Test ............................. SUCCESS [10.396s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:20.312s
[INFO] Finished at: Mon Sep 12 09:59:41 MST 2011
[INFO] Final Memory: 188M/400M
[INFO] ------------------------------------------------------------------------
Comment 3 Joakim Erdfelt CLA 2011-09-12 14:49:03 EDT
Thread.sleep in test cases is pretty prevalent.


[joakim@lapetus org.eclipse.jetty.project][master]$ grep -RHn Thread.sleep * | grep -v aggregate | grep "src/test" | sed -e "s/^.*:[0-9]*:.*\(Thread.sleep[^)]*)\).*/\1/g"  | sort | uniq -c | sort -nr
     23 Thread.sleep(100)
     22 Thread.sleep(200)
     21 Thread.sleep(10)
     20 Thread.sleep(1000)
     14 Thread.sleep(500)
     11 Thread.sleep(50)
      7 Thread.sleep(2000)
      6 Thread.sleep(PAUSE)
      5 Thread.sleep(25)
      4 Thread.sleep(1100)
      3 Thread.sleep(5)
      3 Thread.sleep(300)
      3 Thread.sleep(1)
      2 Thread.sleep(wait)
      2 Thread.sleep(_timeout * 2)
      2 Thread.sleep(sleep)
      2 Thread.sleep(interval)
      2 Thread.sleep(500L)
      2 Thread.sleep(5000)
      2 Thread.sleep(4000)
      2 Thread.sleep(250)
      2 Thread.sleep(1+random.nextInt(10)
      2 Thread.sleep(1500)
      2 Thread.sleep(1000L)
      2 Thread.sleep(10000)
      1 Thread.sleep(TIMEOUT*4)
      1 Thread.sleep(timeout * 2)
      1 Thread.sleep(sleep.get()
      1 Thread.sleep(sleep_for)
      1 Thread.sleep(scavenge * 3000)
      1 Thread.sleep( runTestLength )
      1 Thread.sleep(_resumeAfter)
      1 Thread.sleep(_requestMaxTime * 2)
      1 Thread.sleep(requestInterval)
      1 Thread.sleep(_random.nextInt(290)
      1 Thread.sleep(random.nextInt(1000)
      1 Thread.sleep(_random.nextInt(100)
      1 Thread.sleep(pause1)
      1 Thread.sleep(pause0)
      1 Thread.sleep(Long.parseLong(request.getParameter("sleep")
      1 Thread(sleeper)
      1 Thread.sleep(DURATION)
      1 Thread.sleep(_completeAfter)
      1 Thread.sleep(320000)
      1 Thread.sleep(3000)
      1 Thread.sleep(30)
      1 Thread.sleep(2 * JdbcTestServer.SAVE_INTERVAL * 1000)
      1 Thread.sleep(20)
      1 Thread.sleep(1+_random.nextInt(20)
      1 Thread.sleep(1001)

NOTE: Some of these are likely commented out in the test code.
Comment 4 Joakim Erdfelt CLA 2011-12-08 13:08:20 EST
Closing this as a bug.
The basic improvements have been performed.
Further improvements will occur in an iterative fashion.