Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 332875

Summary:	AbstractNode.getStartLine too slow
Product:	[Modeling] TMF	Reporter:	Mark Christiaens <mark.g.j.christiaens>
Component:	Xtext	Assignee:	Project Inbox <tmf.xtext-inbox>
Status:	CLOSED FIXED	QA Contact:
Severity:	normal
Priority:	P3	CC:	mark.g.j.christiaens, sebastian.zarnekow, sven.efftinge
Version:	2.0.0	Flags:	sebastian.zarnekow: indigo+
Target Milestone:	M7
Hardware:	PC
OS:	Linux
Whiteboard:

Description Mark Christiaens

2010-12-17 17:46:43 EST

Build Identifier: I20101028-1441

Xtext 2.0 does some kind of validation that iterates over all the Diagnostics associated with a resource.  When you have quite a few of those (a couple of thousand in my big file) then the code will try to associate an actual line number with them as illustrated in this profilation report:

Name	Time (ms)

org.eclipse.xtext.builder.clustering.ClusteringBuilderState.doUpdate(BuildData, Map, IProgressMonitor)	5417
org.eclipse.xtext.builder.builderState.AbstractBuilderState.updateMarkers(ResourceSet, ImmutableList, IProgressMonitor)	5417
org.eclipse.xtext.builder.builderState.MarkerUpdaterImpl.updateMarker(ResourceSet, ImmutableList, IProgressMonitor)	5417
org.eclipse.xtext.builder.builderState.MarkerUpdaterImpl.addMarkers(IFile, Resource, IProgressMonitor)	5407
org.eclipse.xtext.validation.ResourceValidatorImpl.validate(Resource, CheckMode, CancelIndicator)	5034
org.eclipse.xtext.validation.ResourceValidatorImpl.markerFromXtextResourceDiagnostic(Resource$Diagnostic, Severity, IDiagnosticConverter$Acceptor)	3356
org.eclipse.xtext.validation.DiagnosticConverterImpl.convertResourceDiagnostic(Resource$Diagnostic, Severity, IDiagnosticConverter$Acceptor)	3356
org.eclipse.xtext.diagnostics.AbstractDiagnostic.getLine()	3336
org.eclipse.xtext.nodemodel.impl.SyntheticCompositeNode.getStartLine()	3336
org.eclipse.xtext.nodemodel.impl.AbstractNode.getStartLine()	3336
org.eclipse.xtext.util.Strings.countLines(String)	3312
org.eclipse.xtext.util.Strings.countLines(String, char[])	1542
java.lang.String.toCharArray()	1012

As you can see, doing AbstractDiagnostic.getLine is quite costly.  It copies a range of the original document to a substring (that's probably cheap since the same backing array is used) and then that string is transformed to a CharArray.  Then the number of newlines are counted in that CharArray.  That ends up to be a lot of copying going on for a big file with a lot of Diagnostic messages.  I had the impression that my editor was doing this for more than a minute.  

Maybe an AbstractNode should contain the number of lines it covers so that the total line count can quickly be established by iterating from that node up to the root of the tree?  

Reproducible: Always

Comment 1 Sebastian Zarnekow

2011-01-30 09:21:05 EST

The IDiagnosticConverter should get the chance to convert diagnostics to issues in batch mode. This would allow to use a precomputed, sorted list of line numbers / offset data.

Comment 2 Sebastian Zarnekow

2011-04-19 08:45:24 EDT

It was indeed very slow.

Each of the mentioned operations were performed for all nodes in the given domain-model input:

Input size: 300kb

iterate nodes as no-op: 17ms
INode.getTotalOffset: 21ms
INode.getOffset: 33ms
INode.getTotalEndOffset: 20ms
INode.getTotalLength: 20ms
INode.getLength: 42ms
INode.getTotalStartLine: 58725ms
INode.getStartLine: 58500ms
INode.getTotalEndLine: 58784ms
<Abort>

New times are:
--------------

Input size: 300kb

iterate nodes as no-op: 16ms
INode.getTotalOffset: 22ms
INode.getOffset: 37ms
INode.getTotalEndOffset: 20ms
INode.getTotalLength: 21ms
INode.getLength: 39ms
INode.getTotalStartLine: 32ms
INode.getStartLine: 46ms
INode.getTotalEndLine: 31ms
INode.getEndLine: 74ms

Input size: 2MB

iterate nodes as no-op: 86ms
INode.getTotalOffset: 97ms
INode.getOffset: 269ms
INode.getTotalEndOffset: 100ms
INode.getTotalLength: 96ms
INode.getLength: 242ms
INode.getTotalStartLine: 175ms
INode.getStartLine: 350ms
INode.getTotalEndLine: 191ms
INode.getEndLine: 449ms

Comment 3 Karsten Thoms

2017-09-19 17:17:31 EDT

Closing all bugs that were set to RESOLVED before Neon.0

Comment 4 Karsten Thoms

2017-09-19 17:28:51 EDT

Closing all bugs that were set to RESOLVED before Neon.0