Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 546641

Summary: [console] IOConsoleInputStream.available() should not return -1
Product: [Eclipse Project] Platform Reporter: Paul Pazderski <paul-eclipse>
Component: DebugAssignee: Paul Pazderski <paul-eclipse>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: loskutov, sarika.sinha
Version: 4.12   
Target Milestone: 4.12 M3   
Hardware: All   
OS: All   
See Also: https://bugs.eclipse.org/bugs/show_bug.cgi?id=307309
https://git.eclipse.org/r/141011
https://git.eclipse.org/r/141013
https://git.eclipse.org/c/platform/eclipse.platform.debug.git/commit/?id=e464979869b087a1af7c21fa4b54136db563bda4
https://git.eclipse.org/c/platform/eclipse.platform.debug.git/commit/?id=6fd317d247b94e2e4184ff79f665c7eb848ceb6e
Whiteboard:
Bug Depends on:    
Bug Blocks: 545769, 552030    

Description Paul Pazderski CLA 2019-04-23 03:40:03 EDT
(In reply to Paul Pazderski from comment #14)
> Current implementation of IOConsoleInputStream#available() does not adhere to
> InputStream#available() JavaDoc (as mentioned in comment 3). Available
> should either return estimated number of bytes (may be 0 if none available)
> or 0 if end of stream and may throw an exception if already closed. I see no
> option here to return -1.
> 
> Second, available() after close() returns -1 sometimes, sometimes not but
> what's worse is that either way the next available() throws an exception.
> Either all available() after close() should throw or none.
> 
> Third, read() after close can succeed so I must assume it is valid to read
> remaining content from stream after close. But the following sequence fails:
> call available() on open but empty stream, call read() on same empty stream
> (read will block), call close() from other thread, now the previous read()
> throws exception. The same situation does not throw exception without the
> available().
> Due to the third problem the IOConsoleInputStream is problematic if wrapped
> in an InputStreamReader.

The first attempt to fix it caused a regression. (see bug 307309 comment 19)
Comment 1 Paul Pazderski CLA 2019-04-23 15:49:12 EDT
(In reply to Andrey Loskutov from bug 307309 comment #19)
> Regression is: I see that some ant build jobs never terminate now, because
> of not closed input streams, so they are logged as:
> 
> 3eclipse.buildId=4.12.0.I20190421-1800
> java.version=11.0.2
> java.vendor=Oracle Corporation
> BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
> Command-line arguments:  -os win32 -ws win32 -arch x86_64 -data
> C:\workspaces\platform
> 
> org.eclipse.core.jobs
> Warning
> Mon Apr 22 22:55:58 CEST 2019
> Job found still running after platform shutdown.  Jobs should be canceled by
> the plugin that scheduled them during shutdown:
> org.eclipse.debug.internal.ui.views.console.ProcessConsole$InputReadJob

Unfortunately I could not reproduce this warning in a normal way. Neither with running an ant target in eclipse. I already considered bug 307309 comment #10 in my initial patch and also saw the problems in git history.

The warning says the job is still running. The job does little more than reading from input stream in a loop until the read returns -1 (or the stream is null but this should never happen before the other condition). The ProcessConsole either reads from a FileInputStream (irrelevant since I changed nothing here) or from IOConsoleInputStream. The read() either returns the available data (the loop continues) or it blocks to wait to wait for more data until it is closed and no more data is available, than read() returns -1 and the loop ends. I see no flaws at this point. Neither with or without my patch.

So the only situation the job is 'still running' I can imagine is when the stream is not closed and the read never returns. The ProcessConsole close the stream if the process already terminates before console initialization or if the terminate event is received or if the console is disposed.

The warning is emitted by the JobManager shutdown (I found this message nowhere else). On shutdown, my eclipse for testing first stops the DebugUIPlugin which disposes the process consoles which again stops any pending InputReadJobs. The JobManager is stopped afterwards and all InputReadJobs already terminated.

My only success to reproduce this warning was to trigger a race condition in ProcessConsole init so that the process can terminate but the console miss the termination (open a new bug for this?) and after that manually shutdown the JobManager (which is discouraged but possible).

Actually the JobManager tries to stop the job and produce the warning after this attempt failed. So maybe the problem is that InputReadJob is unstoppable. The warning says the plugin should cancel the job but DebugUIPlugin does cancel the job (or better dispose the consoles) and I found no problems there nor how my change could interfere it.
Comment 2 Eclipse Genie CLA 2019-04-23 15:53:10 EDT
New Gerrit change created: https://git.eclipse.org/r/141011
Comment 3 Eclipse Genie CLA 2019-04-23 15:53:36 EDT
New Gerrit change created: https://git.eclipse.org/r/141013
Comment 4 Paul Pazderski CLA 2019-04-23 15:55:21 EDT
Long story short I could not reproduce this problem and you may retry my change
https://git.eclipse.org/r/#/c/141011/
in combination with this new change
https://git.eclipse.org/r/141013
which makes the InputReadJob cancelable.
Comment 5 Andrey Loskutov CLA 2019-04-24 05:19:00 EDT
(In reply to Paul Pazderski from comment #4)
> Long story short I could not reproduce this problem and you may retry my
> change
> https://git.eclipse.org/r/#/c/141011/
> in combination with this new change
> https://git.eclipse.org/r/141013
> which makes the InputReadJob cancelable.

I will check this later. I have lot of SDK projects in my workspace, some of them are using external ant builders, and the effect was that after a workspace rebuild (I update the platform often) some of those builders output left "hanging" in the Console view and I was not able to "remove" them and saw those warnings on shutdown. I had no chance to verify yet if the revert fixed this or not, but looking on the recent platform changes only the available() change was related.
Comment 7 Andrey Loskutov CLA 2019-04-24 11:28:34 EDT
This is the script that sometimes produces the "hanging" input job: rt.equinox.p2/bundles/org.eclipse.equinox.p2.publisher.eclipse/scripts/buildExtraJAR.xml

The script is configured to run on project build, interestingly I do not always see the console opening, probably because it is terminating faster as we can show the console.
Comment 8 Paul Pazderski CLA 2019-04-24 14:24:48 EDT
The many short processes may in fact had triggered bug 546710 which was, for my test, a precondition to produce the warning.

I'll try to reproduce it again with the new information.
Comment 10 Paul Pazderski CLA 2019-05-14 03:55:38 EDT
The termination problems came out to be unrelated with this fix.
Comment 11 Sarika Sinha CLA 2019-05-21 04:53:11 EDT
@Paul,
Please verify this bug with the new I build.
Comment 12 Paul Pazderski CLA 2019-05-21 14:05:33 EDT
Verified with Build id: I20190520-1805