Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 365424

Summary: Multiple js file parsing on every js file modification
Product: [WebTools] JSDT Reporter: Natalia Bartol <natalia.bartol>
Component: GeneralAssignee: Project Inbox <jsdt.javascript-inbox>
Status: RESOLVED WONTFIX QA Contact: Chris Jaun <cmjaun>
Severity: normal    
Priority: P3 CC: jacek.pospychala, natalia.bartol, zhaozhongwei
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Attachments:
Description Flags
Example JavaScript project
none
this patch will improve this bug none

Description Natalia Bartol CLA 2011-12-02 08:06:03 EST
Build Identifier: org.eclipse.jsdt.core from HEAD

Each modification of standalone js file triggers reconciling and building. During build all js files (sources as well as external libraries) as parsed, more than one time. Is it necessary?

E.g. 
.options file:
org.eclipse.wst.jsdt.core/debug=true
org.eclipse.wst.jsdt.core/debug/builder=true

Debug output after entering just one character 'a' in an empty myscript.js:

[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/xhr.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/xhr.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]
[parsing    C:/workspace/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]


This seems to be causing serious performance issues for large projects with few thousands of js files. 



Reproducible: Always
Comment 1 Natalia Bartol CLA 2011-12-02 08:12:19 EST
Created attachment 207837 [details]
Example JavaScript project

To see described issue add .options file to your eclipse with content:

org.eclipse.wst.jsdt.core/debug=true
org.eclipse.wst.jsdt.core/debug/builder=true
org.eclipse.wst.jsdt.core/debug/compiler=true

BTW default .options file could be attached to org.eclipse.wst.jsdt.core plugin. 

Start your eclipse with -debug -console parameters.

Import attached JavaScript project.
Open myscript.js.
Type one character. 
You should see output as in previous comment.
Comment 2 Zhongwei Zhao CLA 2011-12-02 08:31:28 EST
Created attachment 207838 [details]
this patch will improve this bug

this patch is for bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=362024
I did nothing,just uncomment several lines,I think there must be some reason to comment these lines,can anyone tell me?

Thanks
Comment 3 Natalia Bartol CLA 2011-12-02 08:39:02 EST
Debug output after applying Zhao's patch:

[parsing    C:/to-remove-test-workspace-3011/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/xhr.js - #1/1]
[parsing    C:/to-remove-test-workspace-3011/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/browserWindow.js - #1/1]
[parsing    C:/to-remove-test-workspace-3011/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/system.js - #1/1]
[parsing    C:/to-remove-test-workspace-3011/.metadata/.plugins/org.eclipse.wst.jsdt.core/libraries/baseBrowserLibrary.js - #1/1]


Seems to be better. Each files is parsed only once. 

Is it necessary to parse ALL js files in a project in case of a single file modification? Is there a way to exclude external libraries?
Comment 4 Nitin Dahyabhai CLA 2011-12-02 11:53:25 EST
I'm guessing that was the only file in the project.  The parsing is done at the behest of the as-you-type validation, which essentially compiles the file, creating type bindings.  Bindings remain in memory during the entire operation, and automatically building them for every type in every file encountered when we were looking for just one type causes memory to spike and cause OOMEs readily when working with larger projects (bug 345797).  The trade-off made was to only build them for the type we were actually looking for, and while this increases the I/O if we need to parse a file again, it scales better memory-wise.  You're seeing these files parsed regardless because your project designates Window as the common supertype--so we have to build that, as well as the types for its member properties.  There may be ways to reduce that redundancy as well, but for now those lines being commented out is entirely intentional.
Comment 5 Chris Jaun CLA 2013-10-23 14:16:52 EDT
For the reasons Nitin mentioned, this is working as designed.