Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 276763 - [launcher] launcher fails to use OpenJDK-based VM when -vm is specified
Summary: [launcher] launcher fails to use OpenJDK-based VM when -vm is specified
Status: VERIFIED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: 3.5   Edit
Hardware: Macintosh Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: 3.6 M5   Edit
Assignee: Andrew Niefer CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-05-18 15:07 EDT by David Green CLA
Modified: 2010-01-18 19:21 EST (History)
8 users (show)

See Also:


Attachments
patch for a hacked findVMLibrary (10.04 KB, patch)
2009-05-19 12:28 EDT, David Green CLA
no flags Details | Diff
the shell script that I use to build OpenJDK (426 bytes, text/plain)
2009-05-19 12:31 EDT, David Green CLA
no flags Details
hack to launch VM via exec rather than JNI (10.87 KB, patch)
2009-05-19 16:18 EDT, David Green CLA
no flags Details | Diff
patch finds OpenJDK libjvm (19.40 KB, patch)
2009-05-20 15:28 EDT, David Green CLA
no flags Details | Diff
working patch (19.77 KB, patch)
2009-05-22 01:31 EDT, David Green CLA
no flags Details | Diff
updated without whitespace (6.35 KB, patch)
2009-05-22 11:04 EDT, Andrew Niefer CLA
no flags Details | Diff
updated (7.71 KB, patch)
2010-01-06 13:45 EST, Andrew Niefer CLA
no flags Details | Diff
patch (6.49 KB, patch)
2010-01-06 15:23 EST, Kevin Barnes CLA
no flags Details | Diff
carbon binary (115.15 KB, application/octet-stream)
2010-01-06 15:27 EST, Kevin Barnes CLA
no flags Details
cocoa binary (104.61 KB, application/octet-stream)
2010-01-06 15:27 EST, Kevin Barnes CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Green CLA 2009-05-18 15:07:41 EDT
If '/Versions/' string is not found in -vm argument then the -vm argument is ignored.  This prevents the use of non-Apple VMs such as OpenJDK.

The culprit appears to be findVMLibrary(char*) at eclipseCarbon.c:290, where in my case the command is passed as '/Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/bin/java'.  The code looks for '/Versions' in the passed command and if not found returns JAVA_FRAMEWORK.  Thus '/Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/bin/java' becomes '/System/Library/Frameworks/JavaVM.framework' and the platform VM is used instead of the specified one, effectively making it impossible to specify a VM that follows a standard (non-Apple) directory layout.
Comment 1 Mike Wilson CLA 2009-05-19 10:07:49 EDT
Obviously, this isn't the solution, but as a workaround (i.e. hack), does it work if you install under that directory?
Comment 2 Andrew Niefer CLA 2009-05-19 10:57:01 EDT
The mac launcher is currently only able to use jvms that are properly installed into the JavaVM.Framework.

Workaround would be to launch using java:
java -jar org.eclipse.equinox.launcher_1.0.200.v20090513.jar

The apple VMs require a vm argument -XstartOnFirstThread, I don't know if this will be needed here.
Comment 3 David Green CLA 2009-05-19 12:28:52 EDT
Created attachment 136331 [details]
patch for a hacked findVMLibrary

(In reply to comment #1)
> Obviously, this isn't the solution, but as a workaround (i.e. hack), does it work if you install under that directory?

No, it doesn't.

I also tried running with the attached patch, but it didn't work.  My guess is that either my patch is doing something bad, or the OpenJDK VM doesn't launch with JNI.

(In reply to comment #2)
> The mac launcher is currently only able to use jvms that are properly installed
> into the JavaVM.Framework.

It would be nice to fix that.

> Workaround would be to launch using java:
> java -jar org.eclipse.equinox.launcher_1.0.200.v20090513.jar

Thanks for the workaround.

> The apple VMs require a vm argument -XstartOnFirstThread, I don't know if this
> will be needed here.

OpenJDK accepts -XstartOnFirstThread no problem.  I don't know if it's required -- but it does work.
Comment 4 David Green CLA 2009-05-19 12:31:08 EDT
Created attachment 136332 [details]
the shell script that I use to build OpenJDK

The attached shell script is what I use to build the bsd-port of jdk7 on my mac.

Other relevant information that may help can be found here: http://greensopinion.blogspot.com/2009/05/eclipse-35-galileo-on-jdk-7.html
Comment 5 Andrew Niefer CLA 2009-05-19 13:28:26 EDT
With your attached change, you will need to pass a -vm argument which points to a shared library that we can do dlopen and dlsym on.  On windows this is usually jvm.dll, on *nix it is libjvm.so.  I'm not sure what it is on the mac.

See eclipseCarbonCommon.c loadLibrary, findSymbol
Comment 6 David Green CLA 2009-05-19 16:18:38 EDT
Created attachment 136382 [details]
hack to launch VM via exec rather than JNI

In the spirit of hacking to better understand the problem I've created this patch which causes VM to be launched via exec() rather than JNI.  After removing -Xdock:icon from my ecilpse.ini the VM is launched without any issues (except of course bug 276564).

As far as JNI, I'm not exactly sure what is involved to get it to work.  The OpenJDK (bsd-port) VM that I'm working with has *.dylib file extensions on relevant libraries:

pre. 
lt035:jdk7-32bit-2009-05-08 dgreen$ find . -name 'libjvm*' -print
./hotspot/import/jre/lib/i386/client/libjvm.dylib
./hotspot/import/jre/lib/i386/server/libjvm.dylib
./hotspot/outputdir/bsd_i486_compiler1/product/libjvm.dylib
./hotspot/outputdir/bsd_i486_compiler1/product/libjvm.dylib.1
./hotspot/outputdir/bsd_i486_compiler2/product/libjvm.dylib
./hotspot/outputdir/bsd_i486_compiler2/product/libjvm.dylib.1
./j2re-image/lib/i386/client/libjvm.dylib
./j2re-image/lib/i386/server/libjvm.dylib
./j2sdk-image/jre/lib/i386/client/libjvm.dylib
./j2sdk-image/jre/lib/i386/server/libjvm.dylib
./lib/i386/client/libjvm.dylib
./lib/i386/server/libjvm.dylib

I hope that helps.
Comment 7 David Green CLA 2009-05-20 15:28:15 EDT
Created attachment 136567 [details]
patch finds OpenJDK libjvm

Attaching  a patch that causes the launcher to correctly find OpenJDK libjvm.  The patch seems to work well when stepping through but fails to launch the vm.  I get the following message on the console:

pre. 
Error occurred during initialization of VM
Unable to load native library: dlopen(/Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/jre/lib/i386/libjava.dylib, 1): Library not loaded: libjvm.dylib
  Referenced from: /Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/jre/lib/i386/libjava.dylib
  Reason: image not found

The patch seems to find libjvm.dylib and sets the LD_LIBRARY_PATH to @"/Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/jre/lib/i386/client:/Users/dgreen/Documents/packages/jdk7-32bit-2009-05-08/jre/lib/i386"@ which looks right to me.

At this point I'm a little stuck and could use some help.
Comment 8 David Green CLA 2009-05-20 15:55:17 EDT
the error is caused at eclipseJNI.c:341

bc. 
	if( createJavaVM(&jvm, &env, &init_args) == 0 ) {
	
It seems that when the function is called some of the DSO libraries are not loaded.
Comment 9 Andrew Niefer CLA 2009-05-20 18:52:37 EDT
I wonder if there are other directories that need to go on the LD_LIBRARY_PATH.

If you compiled the OpenJDK vm yourself, there must be source for the java launcher.  You could see if it does anything else.  The fundamental steps for JNI are just: load the library, find "JNI_CreateJavaVM", call that method.
Comment 10 David Green CLA 2009-05-22 01:31:33 EDT
Created attachment 136759 [details]
working patch

the attached patch successfully launches jdk7 bsd port using JNI

the secret seems to be to set DYLD_FALLBACK_LIBRARY_PATH as suggested by Kurt Miller on the bsd-port-dev list
Comment 11 Andrew Niefer CLA 2009-05-22 11:04:32 EDT
Created attachment 136812 [details]
updated without whitespace

Thanks David.  This is the patch without the whitespace changes so that it is easier to see what is going on.

I removed ifdefs for Solaris and AIX.  We will need to see what the constants for x86_64 look like, patch currently has amd64.

restartLauncher used to fork, now it doesn't.  I don't think this matters, but we'll want to confirm that.  (*nix doesn't fork).

We will need to ask SWT about the path adjustments for mozilla/netscape.  Those may not apply on mac.

We should also find out if Apache Harmony runs on the Mac, see also bug 196006.
Comment 12 David Green CLA 2009-05-25 16:55:33 EDT
(In reply to comment #11)
> This is the patch without the whitespace changes so that it is
> easier to see what is going on.

Thanks.  CDT was reformatting the file every time I saved it.

> I removed ifdefs for Solaris and AIX.  We will need to see what the constants
> for x86_64 look like, patch currently has amd64.

The bsd port currently uses amd64 for the 64-bit build on OS X.  I don't know why.

> restartLauncher used to fork, now it doesn't.  I don't think this matters, but
> we'll want to confirm that.  (*nix doesn't fork).

How can we conform that?  (I pulled the code from *nix)

> We will need to ask SWT about the path adjustments for mozilla/netscape.  Those
> may not apply on mac.

Agreed.  Who's the best person to contact here?

> We should also find out if Apache Harmony runs on the Mac, see also bug 196006.

The Harmony project doesn't provide any binaries for Mac: http://harmony.apache.org/download.cgi
Comment 13 Kevin Barnes CLA 2009-05-25 17:00:29 EDT
CC Grant and SSQ for the browser question.
Comment 14 Grant Gayed CLA 2009-05-26 10:07:22 EDT
The path does not need to be changed to help the Browser on OSX.
Comment 15 David Green CLA 2009-07-03 12:12:27 EDT
Any chance of this one being scheduled for 3.5.1?  If so, what's left to do here (what would it take to have this work committed?)
Comment 16 Andrew Niefer CLA 2009-07-06 13:27:21 EDT
This is too big a feature to do in a maintenance release, so it won't be in 3.5.1.

Other than my initial review, I have not looked more closely at this yet because I am currently unable to get an OpenJDK based vm due to licensing/legal issues.
Comment 17 Mike Wilson CLA 2009-07-07 16:08:13 EDT
(In reply to comment #16)
> This is too big a feature to do in a maintenance release, so it won't be in
> 3.5.1.
> 
> Other than my initial review, I have not looked more closely at this yet
> because I am currently unable to get an OpenJDK based vm due to licensing/legal
> issues.
> 
Just so we're all on the same page, this says that for R3.6 we could put in a community contributed patch that enables OpenJDK if and only if:
1) We can prove that it has no impact on the existing uses
2) We get lots of community support for testing.
Comment 18 Thomas Watson CLA 2009-07-08 09:28:04 EDT
I agree with Mike's statement.  We will need lots of community testing with this patch.  Out of curiosity do we know of other non-Apple VMs?  If so do we expect this patch to enable them to work also?
Comment 19 David Green CLA 2009-07-08 11:39:12 EDT
(In reply to comment #16)
> This is too big a feature to do in a maintenance release, so it won't be in
> 3.5.1.

That's fine.  I was hopeful, but I understand.

> Other than my initial review, I have not looked more closely at this yet because
> I am currently unable to get an OpenJDK based vm due to licensing/legal issues.

It seems unlikely that OpenJDK licensing will change much in time for 3.6.  Is it possible to move this one forward with community-based testing?

(In reply to comment #17)
> Just so we're all on the same page, this says that for R3.6 we could put in a
> community contributed patch that enables OpenJDK if and only if:
> 1) We can prove that it has no impact on the existing uses
> 2) We get lots of community support for testing.

Sounds good.  I'll provide as much testing as I can to help move this one forward.  If we can show that I can launch using OpenJDK, and others can launch using Apple VMs will that be enough?  If not, what would be an acceptable level of testing?

Also I'm interested in improving the quality of this patch until it's commit-ready.  If someone could review the patch and let me know if there are any issues (wrt code quality, design etc) that need to be resolved before it will be considered that would be great.

(In reply to comment #18)
> Out of curiosity do we know of other non-Apple VMs?  If so do we expect
> this patch to enable them to work also?

The only non-Apple VMs that I'm aware of are:

* "IBM JDK":http://www.ibm.com/developerworks/java/jdk/
* "Oracle JRockit":http://www.oracle.com/technology/products/jrockit/index.html
* "Harmony":http://harmony.apache.org/

As far as I am aware none of them provide an Apple download.  In theory if they _did_ provide Apple downloads this patch should help, however until such time as they start providing Apple binaries it's anyone's guess.
Comment 20 David Green CLA 2009-08-10 15:56:26 EDT
related: it seems that OpenJDK is to be the default java and jdk on ubuntu: https://lists.ubuntu.com/archives/ubuntu-devel-announce/2008-July/000460.html
Comment 21 Mike Wilson CLA 2009-08-12 09:30:34 EDT
Every Mac already has a "Sun + Apple tweaks" VM on it, and almost all Mac users will automatically take every software update they are given so effectively every mac has exactly what it needs to run Eclipse with no other work required. This bug is about enabling an alternative VM to also work, for those that find that compelling.

I think it's awesome that Ubuntu is going with OpenJDK.

Comment 22 David Green CLA 2009-08-12 20:48:34 EDT
(In reply to comment #21)
> This bug is about enabling an alternative VM to also work, for those that find that compelling.

Agreed.

I'm still looking for feedback on how to move this bug forward.  See comment #19 for details
Comment 23 David Green CLA 2009-08-20 13:33:19 EDT
OpenJDK 6 is available for Mac via MacPorts: http://landonf.bikemonkey.org/code/java/OpenJDK6_MacPorts.20090516.html
this should help people who wish to test changes related to this bug.

Looks like this is a candidate for 3.6, but is currently unassigned.  It would be good to get this one in early in the 3.6 game to provide as much time as possible for community exposure.  What can I do to help move this one forward?
Comment 24 David Green CLA 2009-10-16 12:06:00 EDT
It would be a shame to leave this one hanging after we've moved it so far along.  What will it take to move this issue forward?
Comment 25 Andrew Niefer CLA 2009-10-20 13:44:54 EDT
I'll try to look at this in M4
Comment 26 David Green CLA 2009-10-20 13:48:25 EDT
(In reply to comment #25)
> I'll try to look at this in M4

Fantastic.  Let me know if there's anything that I can do to help.
Comment 27 Andrew Niefer CLA 2009-10-20 14:23:02 EDT
This is also somewhat related to bug 288408
Comment 28 Thomas Watson CLA 2009-12-02 11:32:39 EST
Is this something that can make M4?  We are getting pretty late for M4.
Comment 29 Thomas Watson CLA 2009-12-02 16:08:56 EST
I briefly discussed this with Andrew.  It is unfortunately getting too late to do this for M4 (warm-up build is this Friday).  We must look at this early in M5.
Comment 30 Andrew Niefer CLA 2010-01-06 13:45:32 EST
Created attachment 155435 [details]
updated
Comment 31 Kevin Barnes CLA 2010-01-06 15:23:37 EST
Created attachment 155448 [details]
patch

This is the version to be released
Comment 32 Kevin Barnes CLA 2010-01-06 15:27:17 EST
Created attachment 155449 [details]
carbon binary
Comment 33 Kevin Barnes CLA 2010-01-06 15:27:59 EST
Created attachment 155450 [details]
cocoa binary
Comment 34 Andrew Niefer CLA 2010-01-06 15:34:55 EST
The above three comments are by me working from Kevin's computer.  David can you please try the attached binaries.
Comment 35 David Green CLA 2010-01-07 19:45:16 EST
Wow, that's great!!! I'm able to lauch using OpenJDK 7.  Some details from the Eclipse about dialog:

pre. 
java.runtime.name=OpenJDK Runtime Environment
java.runtime.version=1.7.0-internal-dgreen_2009_05_08_12_49-b00
java.specification.name=Java Platform API Specification
java.specification.vendor=Sun Microsystems Inc.
java.specification.version=1.7
java.vendor=Sun Microsystems Inc.
java.vendor.url=http://java.sun.com/
java.vendor.url.bug=http://java.sun.com/cgi-bin/bugreport.cgi
java.version=1.7.0-internal
java.vm.info=mixed mode
java.vm.name=OpenJDK Client VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Sun Microsystems Inc.
java.vm.specification.version=1.0
java.vm.vendor=Sun Microsystems Inc.
java.vm.version=15.0-b02
Comment 36 David Green CLA 2010-01-07 19:46:25 EST
I forgot to mention: I only tested the cocoa binary.
Comment 37 Andrew Niefer CLA 2010-01-08 17:30:19 EST
I put this in for next IBuild
Comment 38 David Green CLA 2010-01-11 18:58:22 EST
Using *File -> Switch Workspace* doesn't work with this patch: the Eclipse instance doesn't restart properly.

Version: 3.6.0
Build id: N20100110-2000
Comment 39 Andrew Niefer CLA 2010-01-14 13:59:23 EST
I released changes to the mac restartLauncher function to revert back to the fork + execv pattern.
Comment 40 David Green CLA 2010-01-18 19:21:07 EST
Verified that it works on N20100117-2000, including switching workspaces.  Also verified that restarts after installing new bundles works correctly.

Thanks Andrew!