Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 385738 - Launching command line exceeds the process creation command limit on *nix
Summary: Launching command line exceeds the process creation command limit on *nix
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Debug (show other bugs)
Version: 3.7   Edit
Hardware: PC Linux
: P3 normal with 3 votes (vote)
Target Milestone: 4.9 M2   Edit
Assignee: Cédric Chabanois CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed, noteworthy
: 534732 (view as bug list)
Depends on: 547874 327193 393675 552037
Blocks:
  Show dependency tree
 
Reported: 2012-07-23 09:52 EDT by Michael Rennie CLA
Modified: 2019-10-15 06:07 EDT (History)
27 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Rennie CLA 2012-07-23 09:52:17 EDT
+++ This bug was initially created as a clone of Bug #327193 +++

eclipse.buildId=I20120710-0800
java.version=1.7.0-ea
java.vendor=Oracle Corporation
BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_CA
Framework arguments:  -product org.eclipse.sdk.ide -showlocation
Command-line arguments:  -product org.eclipse.sdk.ide -data
/home/cwindatt/WS/Workspace Target -dev
file:/home/cwindatt/WS/Workspace/.metadata/.plugins/org.eclipse.pde.core/Target/dev.properties
-os linux -ws gtk -arch x86 -consoleLog -console -showlocation

Error
Wed Jul 18 11:16:10 CDT 2012
Exception occurred executing command line.

java.io.IOException: Cannot run program
"/home/cwindatt/Java/jdk7/jdk1.7.0/bin/java" (in directory
"/home/cwindatt/WS/Workspace Target/testJava"): error=7, Argument list too long
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    at java.lang.Runtime.exec(Runtime.java:615)
    at org.eclipse.debug.core.DebugPlugin.exec(DebugPlugin.java:861)
    at
org.eclipse.jdt.launching.AbstractVMRunner.exec(AbstractVMRunner.java:87)
    at
org.eclipse.jdt.internal.launching.StandardVMRunner.run(StandardVMRunner.java:332)
    at
org.eclipse.jdt.launching.JavaLaunchDelegate.launch(JavaLaunchDelegate.java:101)
    at
org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:855)
    at
org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:704)
    at
org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:1018)
    at
org.eclipse.debug.internal.ui.DebugUIPlugin$8.run(DebugUIPlugin.java:1222)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Caused by: java.io.IOException: error=7, Argument list too long
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
    at java.lang.ProcessImpl.start(ProcessImpl.java:130)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1021)
    ... 10 more

http://www.in-ulm.de/~mascheck/various/argmax/
Comment 1 Eclipse Genie CLA 2017-06-03 09:33:42 EDT
New Gerrit change created: https://git.eclipse.org/r/98586
Comment 2 Terence Miller CLA 2017-11-22 03:58:46 EST
I have the same issue on windows 7 using spring sts, maven spring-boot:run INFO] --- spring-boot-maven-plugin:1.5.7.RELEASE:run (default-cli) @ admin-server ---
[INFO] Attaching agents: []
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.206 s
[INFO] Finished at: 2017-11-22T09:54:29+01:00
[INFO] Final Memory: 77M/451M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:1.5.7.RELEASE:run (default-cli) on project admin-server: Could not exec java: Cannot run program "C:\Program Files\Java\jdk1.8.0_131\jre\bin\java.exe": CreateProcess error=206, Der Dateiname oder die Erweiterung ist zu lang -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Comment 3 Igor Fedorenko CLA 2017-11-23 16:29:26 EST
Can jdt.debug developers please comment on this issue? I can personally confirm the provided patch solves the problem on OSX and I believe other members of my team use it successfully on Linux. Thank you in advance.
Comment 4 Sarika Sinha CLA 2017-11-23 22:50:15 EST
@Andrey,
 Will you be able to review this?
Comment 5 Andrey Loskutov CLA 2017-11-24 04:10:40 EST
(In reply to Igor Fedorenko from comment #3)
> Can jdt.debug developers please comment on this issue? I can personally
> confirm the provided patch solves the problem on OSX and I believe other
> members of my team use it successfully on Linux. Thank you in advance.

I haven't tried this, but looking at the code I doubt this will work for every application.

The problems I see with this approach:

# All application code expecting and doing something via plain File IO API with the files from classpath will fail, in worst case without really explaining why (file not found??? It IS there!)
# All application code expecting and doing something via extra classloading will most likely fail to load classes from inside jars in the "debug" jar.
# The application will have different behavior running native vs running in debugger. Because the changing the shape of the deployed code happens under the hood, the real reason will be not obvious, and such errors like above will drive people crazy.
# So while for simple application this patch will work, doing something more advanced will most likely fail. I don't think we should do this.

This is clearly a Java limitation, may be we just should catch this and explain in a more "nice" way, before Java crashes?

Another way around this would be to investigate if we can change the classpath directory for the debugging application to /tmp/whatever and link (or copy) all the libraries to that directory. Linking is nicer, but again here the code could fail if it tries to resolve file paths and they are not in the "expected" hierarchy anymore.
Comment 6 Till Brychcy CLA 2017-11-24 04:54:47 EST
As alternative for java 9 and later, @argfiles can be used with the "java" command, see
https://bugs.openjdk.java.net/browse/JDK-8027634
https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-4964-AE84-121F5F6CF111
Comment 7 Andrey Loskutov CLA 2017-11-24 08:10:21 EST
(In reply to Till Brychcy from comment #6)
> As alternative for java 9 and later, @argfiles can be used with the "java"
> command, see
> https://bugs.openjdk.java.net/browse/JDK-8027634
> https://docs.oracle.com/javase/9/tools/java.htm#JSWOR-GUID-4856361B-8BFD-
> 4964-AE84-121F5F6CF111

Yep. So if we detect Java 9 as target VM, we can use that as the solution.
Comment 8 Igor Fedorenko CLA 2017-11-24 08:20:10 EST
(In reply to Andrey Loskutov from comment #5)
> (In reply to Igor Fedorenko from comment #3)
> > Can jdt.debug developers please comment on this issue? I can personally
> > confirm the provided patch solves the problem on OSX and I believe other
> > members of my team use it successfully on Linux. Thank you in advance.
> 
> I haven't tried this, but looking at the code I doubt this will work for
> every application.
> 
> The problems I see with this approach:
> 
> # All application code expecting and doing something via plain File IO API
> with the files from classpath will fail, in worst case without really
> explaining why (file not found??? It IS there!)
> # All application code expecting and doing something via extra classloading
> will most likely fail to load classes from inside jars in the "debug" jar.
> # The application will have different behavior running native vs running in
> debugger. Because the changing the shape of the deployed code happens under
> the hood, the real reason will be not obvious, and such errors like above
> will drive people crazy.

This is not true. In-memory classloading shape remains exactly the same when using classpath-only jar file. Only applications that directly inspect -cp/-classpath command line parameters will be able to tell the difference.

> # So while for simple application this patch will work, doing something more
> advanced will most likely fail. I don't think we should do this.
> 
> This is clearly a Java limitation, may be we just should catch this and
> explain in a more "nice" way, before Java crashes?

The thing is, currently it is not possible to use Eclipse to run/debug applications after they grow beyond certain size. And while I agree the proposed solution has certain limitations, I still believe it is far better than forcing users away from Eclipse. 

I wonder if we should make the classpath-only-jar logic explicit user choice, for example

* Classpath-only-jar logic will be disabled by default but there will be launch configuration toggle to enable it.
* If application classpath gets longer than current operating system limits, the launch will fail with explicit error popup "The application/test cannot be launched because classpath is too long" or something along those lines. The user will have a choice to either cancel the launch or enable classpath-only-jar logic.

What do you think?

> Another way around this would be to investigate if we can change the
> classpath directory for the debugging application to /tmp/whatever and link
> (or copy) all the libraries to that directory. Linking is nicer, but again
> here the code could fail if it tries to resolve file paths and they are not
> in the "expected" hierarchy anymore.

This is not a viable option. First, it only buys relatively small extra classpath length. More importantly, though, it does change classpath shape, which will break applications that locate filesystem resources relative to classpath URL (our application happens to do that).
Comment 9 Sarika Sinha CLA 2017-11-26 22:35:40 EST
(In reply to Igor Fedorenko from comment #8)
> 
> I wonder if we should make the classpath-only-jar logic explicit user
> choice, for example
> 
> * Classpath-only-jar logic will be disabled by default but there will be
> launch configuration toggle to enable it.
> * If application classpath gets longer than current operating system limits,
> the launch will fail with explicit error popup "The application/test cannot
> be launched because classpath is too long" or something along those lines.
> The user will have a choice to either cancel the launch or enable
> classpath-only-jar logic.
> 
> What do you think?

I am ok with this approach!!
Comment 10 Sarika Sinha CLA 2017-12-01 07:27:06 EST
Looks like it needs more discussion.
Comment 11 Cédric Chabanois CLA 2018-02-12 04:43:11 EST
I can work on another fix, based on the proposal from Igor Fedorenko

The approach would be to :

- display an error popup "The [application|test] cannot be launched because classpath is too long" if we detect that classpath is longer than the operating system limits with 2 choices : cancel / enable [classpath-only-jar|@argfiles]

Do we try to start the app and then check classpath length on error or do we check classpath length before starting the app ? 

- have an option in the launch configuration (in the arguments tab ?) to enable classpath only jar or @argfiles

- if jdk >= 9 , use @argfiles instead of classpath only jar

Are you ok with this proposal ?
Comment 12 Andrey Loskutov CLA 2018-02-12 04:50:33 EST
(In reply to Cédric  Chabanois from comment #11)
> I can work on another fix, based on the proposal from Igor Fedorenko
> 
> The approach would be to :
> 
> - display an error popup "The [application|test] cannot be launched because
> classpath is too long" if we detect that classpath is longer than the
> operating system limits with 2 choices : cancel / enable
> [classpath-only-jar|@argfiles]
> 
> Do we try to start the app and then check classpath length on error or do we
> check classpath length before starting the app ? 
> 
> - have an option in the launch configuration (in the arguments tab ?) to
> enable classpath only jar or @argfiles
> 
> - if jdk >= 9 , use @argfiles instead of classpath only jar
> 
> Are you ok with this proposal ?

There is no need to ask user in case a solution without side effects exist (@argfiles).

So basically I'm fine with the proposal from comment 9 + use @argfiles automatically for JRE>=9.
Comment 13 Sarika Sinha CLA 2018-02-12 05:03:15 EST
(In reply to Andrey Loskutov from comment #12)
Also see https://bugs.openjdk.java.net/browse/JDK-8160859
JEP 293: Guidelines for JDK Command-Line Tool Options
Comment 14 Sarika Sinha CLA 2018-04-24 04:47:06 EDT
Moving out of 4.8. If still planned - please move it back.
Comment 15 Sarika Sinha CLA 2018-05-31 06:09:16 EDT
*** Bug 534732 has been marked as a duplicate of this bug. ***
Comment 16 Cédric Chabanois CLA 2018-05-31 17:14:03 EDT
I am working on a new fix that should be ready soon.
In the meantime, I created a p2 repository with a patch feature with the fix I am working on : https://github.com/cchabanois/eclipse.jdt.debug
Comment 17 Eclipse Genie CLA 2018-07-01 09:34:48 EDT
New Gerrit change created: https://git.eclipse.org/r/125302
Comment 18 Sarika Sinha CLA 2018-07-02 01:14:48 EDT
(In reply to Eclipse Genie from comment #17)
> New Gerrit change created: https://git.eclipse.org/r/125302

@Andrey, Can you please review this?
Comment 19 Andrey Loskutov CLA 2018-07-02 05:25:13 EDT
(In reply to Sarika Sinha from comment #18)
> (In reply to Eclipse Genie from comment #17)
> > New Gerrit change created: https://git.eclipse.org/r/125302
> 
> @Andrey, Can you please review this?

I'm looking into this right now.
Comment 21 Andrey Loskutov CLA 2018-07-13 10:16:19 EDT
Merged in master.

Cedric, please provide N&N entry, you can create a Gerrit patch for this repo: https://git.eclipse.org/r/www.eclipse.org/eclipse/news.git
Comment 22 Eclipse Genie CLA 2018-07-17 09:50:56 EDT
New Gerrit change created: https://git.eclipse.org/r/126182
Comment 23 Andrey Loskutov CLA 2018-07-17 09:55:11 EDT
(In reply to Eclipse Genie from comment #22)
> New Gerrit change created: https://git.eclipse.org/r/126182

NPE like 

!ENTRY org.eclipse.debug.core 4 2 2018-07-16 23:19:54.240
!MESSAGE Problems occurred when invoking code from plug-in: "org.eclipse.debug.core".
!STACK 0
java.lang.NullPointerException
	at org.eclipse.jdt.internal.launching.LaunchingPlugin.deleteProcessTempFiles(LaunchingPlugin.java:1206)
	at org.eclipse.jdt.internal.launching.LaunchingPlugin.handleDebugEvents(LaunchingPlugin.java:1199)
	at org.eclipse.debug.core.DebugPlugin$EventNotifier.run(DebugPlugin.java:1134)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.debug.core.DebugPlugin$EventNotifier.dispatch(DebugPlugin.java:1168)
	at org.eclipse.debug.core.DebugPlugin$EventDispatchJob.run(DebugPlugin.java:441)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:60)

I guess this was socket launch which failed to connect.
Comment 25 Sarika Sinha CLA 2018-07-25 04:54:44 EDT
Reminder for N&N!!
Comment 26 Sarika Sinha CLA 2018-07-29 23:17:59 EDT
http://download.eclipse.org/eclipse/downloads/drops4/I20180729-0800/testresults/html/org.eclipse.jdt.debug.tests_ep49I-unit-cen64-gtk3-java10_linux.gtk.x86_64_10.html

2 tests failed on linux.

testVeryLongClasspathWithClasspathOnlyJar	Failure	N/A

junit.framework.AssertionFailedError
at org.eclipse.jdt.debug.tests.launching.LongClassPathTests.testVeryLongClasspathWithClasspathOnlyJar(LongClassPathTests.java:113)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.eclipse.jdt.debug.tests.AbstractDebugTest.runBare(AbstractDebugTest.java:2554)
at org.eclipse.jdt.debug.tests.DebugSuite$1.run(DebugSuite.java:57)
at java.base/java.lang.Thread.run(Thread.java:844)
	36.256
testVeryLongClasspathWithArgumentFile	Failure	N/A

junit.framework.AssertionFailedError
at org.eclipse.jdt.debug.tests.launching.LongClassPathTests.testVeryLongClasspathWithArgumentFile(LongClassPathTests.java:149)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.eclipse.jdt.debug.tests.AbstractDebugTest.runBare(AbstractDebugTest.java:2554)
at org.eclipse.jdt.debug.tests.DebugSuite$1.run(DebugSuite.java:57)
at java.base/java.lang.Thread.run(Thread.java:844)
Comment 28 Sarika Sinha CLA 2018-07-31 01:34:19 EDT
(In reply to Sarika Sinha from comment #27)
> Very frequent on java 10 linux.
> http://download.eclipse.org/eclipse/downloads/drops4/I20180730-0800/
> testresults/html/org.eclipse.jdt.debug.tests_ep49I-unit-cen64-gtk3-
> java10_linux.gtk.x86_64_10.html

Created Bug 537520 to handle that.

N&N should be added by Wednesday 1st August.
Comment 29 Sarika Sinha CLA 2018-07-31 06:03:40 EDT
@Andrey, can you verify this with the I build on linux.
I am having Windows.
Comment 30 Cédric Chabanois CLA 2018-07-31 07:50:46 EDT
I could reproduce the issue on Linux with java 10. Looks like we got these test failures because the jars are now both on classpath and modulepath (at least in the tests). I will try to figure out why.

I am also working on the N&N. Should be ready in a few hours.
Comment 31 Andrey Loskutov CLA 2018-07-31 08:33:46 EDT
Build I20180729-2000 looks good on RHEL 7.4 Linux (I tested jar creation for the launch config), but I've failed to construct a manual example with the length of > 2097152 to test the warning.
Comment 32 Eclipse Genie CLA 2018-07-31 09:55:14 EDT
New Gerrit change created: https://git.eclipse.org/r/126827
Comment 34 Sarika Sinha CLA 2018-07-31 22:58:35 EDT
Thanks Cedric!!!
Comment 35 Sarika Sinha CLA 2018-08-21 00:18:05 EDT
The new option "Use temporary JAR to specify classpath" is limited to Classpath tab? It is not seen in dependencies tab for Java 9 and above ?
Comment 36 Andrey Loskutov CLA 2018-08-24 05:09:21 EDT
(In reply to Sarika Sinha from comment #35)
> The new option "Use temporary JAR to specify classpath" is limited to
> Classpath tab? It is not seen in dependencies tab for Java 9 and above ?

Yes, Java 9 offers a better way to specify classpath.
Comment 37 Sarika Sinha CLA 2018-08-24 05:56:32 EDT
(In reply to Eclipse Genie from comment #32)
> New Gerrit change created: https://git.eclipse.org/r/126827

@Cedric, can you please create a new changeset with the resized images for N&N? 
Instructions.html states that 
"The image should be no more than 520 pixels wide "
Comment 38 Sarika Sinha CLA 2019-04-22 01:35:13 EDT
https://www.eclipse.org/forums/index.php/m/1805696/#msg_1805696

Looks like a regression from this bug.
Comment 39 Sarika Sinha CLA 2019-10-15 06:06:46 EDT
(In reply to Andrey Loskutov from comment #36)
> (In reply to Sarika Sinha from comment #35)
> > The new option "Use temporary JAR to specify classpath" is limited to
> > Classpath tab? It is not seen in dependencies tab for Java 9 and above ?
> 
> Yes, Java 9 offers a better way to specify classpath.

Regression For Projects > 9 Bug 552037