Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 512604

Summary:	[newindex] Reduce the time spent in Database.flush()
Product:	[Eclipse Project] JDT	Reporter:	Stefan Xenos <sxenos>
Component:	Core	Assignee:	Stefan Xenos <sxenos>
Status:	VERIFIED FIXED	QA Contact:
Severity:	normal
Priority:	P3	CC:	eclipse.sprigogin, kevintownsend4219, manoj.palat
Version:	4.6
Target Milestone:	4.7 M6
Hardware:	PC
OS:	Linux
See Also:	https://git.eclipse.org/r/91673 https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=34b0667e460867b2a8c881af3ff53f5fd2aafaaa
Whiteboard:
Bug Depends on:	512538
Bug Blocks:

Description Stefan Xenos

2017-02-22 23:55:05 EST

Users have reported that indexing is running slower than it should be. Most of the runtime during indexing is going into Database.flush(). Currently, we flush every time a write lock is released. We could speed this up a lot if we performed larger chunks less frequently.

Comment 1 Eclipse Genie

2017-02-22 23:56:08 EST

New Gerrit change created: https://git.eclipse.org/r/91673

Comment 2 Stefan Xenos

2017-02-23 00:14:10 EST

Pre-patch:

!ENTRY org.eclipse.jdt.core 1 0 2017-02-22 21:10:05.043
!MESSAGE Indexing done.
  Located 185 indexables in 11ms
  Collected garbage from 0 files in 0ms, average time = 0.00ms
  Tested 185 fingerprints in 2987ms, average time = 16.1ms
  Indexed 216326 classes in 322187ms, average time = 1.49ms
  Updated 0 paths in 49ms, average time = 0.00ms

Allocated size: 929132544 bytes


Post-patch:

!ENTRY org.eclipse.jdt.core 1 0 2017-02-22 21:00:52.056
!MESSAGE Indexing done.
  Located 185 indexables in 14ms
  Collected garbage from 0 files in 0ms, average time = 0.00ms
  Tested 185 fingerprints in 2857ms, average time = 15.4ms
  Indexed 216326 classes in 200503ms, average time = 0.927ms
  Updated 0 paths in 106ms, average time = 0.00ms

Allocated size: 929132544 bytes

So the current patch-in-progress improves the indexing time by 38% in the case where there are no deletions or garbage.

Comment 3 Eclipse Genie

2017-02-24 15:54:03 EST

Gerrit change https://git.eclipse.org/r/91673 was merged to [master].
Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=34b0667e460867b2a8c881af3ff53f5fd2aafaaa

Comment 4 Stefan Xenos

2017-02-24 21:16:02 EST

I suspect the slow flush times are due to index corruption during deletion -- probably deletion of large blocks.

Comment 5 Stefan Xenos

2017-02-27 00:04:02 EST

Indexing performance is still not where I want it, but I'll mark this as fixed since the originally-requested enhancement has been implemented.

Comment 6 Manoj N Palat

2017-03-07 10:10:25 EST

Verified for Eclipse Oxygen 4.7 M6 Build id: I20170305-2000
Note: direct verification not done - verification by code inspection and the figure mentioned in comment 2.