Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 572799

Summary: Replace arm64 with aarch64 for Mac
Product: [Eclipse Project] Platform Reporter: Lakshmi P Shanmugam <lshanmug>
Component: SWTAssignee: Lakshmi P Shanmugam <lshanmug>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: akurtakov, sravankumarl, tjwatson
Version: 4.18   
Target Milestone: 4.20 M2   
Hardware: PC   
OS: Mac OS X   
See Also: https://git.eclipse.org/r/c/equinox/rt.equinox.framework/+/179607
https://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=fe7c590145edab0f74a395ac26b3ee36fa5ac174
https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/179657
https://git.eclipse.org/r/c/equinox/rt.equinox.bundles/+/179661
https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/179662
https://git.eclipse.org/r/c/platform/eclipse.platform.swt.binaries/+/179682
https://git.eclipse.org/r/c/equinox/rt.equinox.p2/+/179751
https://git.eclipse.org/r/c/platform/eclipse.platform.ui/+/179758
https://git.eclipse.org/r/c/platform/eclipse.platform.releng.aggregator/+/179836
https://git.eclipse.org/r/c/equinox/rt.equinox.binaries/+/179844
https://git.eclipse.org/r/c/equinox/rt.equinox.framework/+/179845
https://git.eclipse.org/c/platform/eclipse.platform.ui.git/commit/?id=d023d8bbc621b7f172140e8f72686ad20d1f7e57
https://git.eclipse.org/c/platform/eclipse.platform.resources.git/commit/?id=44486007982366360ebadcee8557cac15dc91b04
https://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/?id=3abba70cd6a7dcfe63d490e73c484aaf2d382dd1
https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=f9fca3b5feb92b269fdfc8e15e37bedf9267b99e
https://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=d3184098f2920d7f40e54b8c617d910c6db3b307
https://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/?id=5c0c7d05365d26c55a3f6ae8989b1ca305437c0e
https://git.eclipse.org/c/equinox/rt.equinox.binaries.git/commit/?id=eaefde0adac1b28c5c4347f8a1ab36cda393a349
https://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=49f17862d789ecaf98b712fd509f1a157d4fdb58
https://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=a11873a75262d7c9db7ffcc1d6b303536601e0cb
https://bugs.eclipse.org/bugs/show_bug.cgi?id=573631
Whiteboard:
Bug Depends on:    
Bug Blocks: 565690    

Description Lakshmi P Shanmugam CLA 2021-04-13 05:32:37 EDT
Platform.isLoadable() returns false with SWT Mac arm64.
SWT manifest file sets SWT-Arch=arm64 but Library.os() returns aarch64 causing mismatch.
Comment 1 Lakshmi P Shanmugam CLA 2021-04-14 08:45:01 EDT
(In reply to Lakshmi P Shanmugam from comment #0)
> Platform.isLoadable() returns false with SWT Mac arm64.
> SWT manifest file sets SWT-Arch=arm64 but Library.os() returns aarch64
> causing mismatch.

Correction, Library.arch() return aarch64.
The SWT_LIB_DIR also uses this arch() and creates directory with aarch64. This can be fixed in Library.arch().
Need to check if System.getProperty("os.arch") is used in other places as well.
Comment 2 Lakshmi P Shanmugam CLA 2021-04-20 17:34:45 EDT
Did a Quick Search for "os.arch" in the releng aggregator repo and looks like the JVM property is used in a few places in equinox as well.
May be we should just use "aarch64" instead on "arm64" in osgi.arch, to be consistent with Java and avoid problems in the future. This will also fix this SWT issue, without performing any mapping.

The reason we started using arm64 was Apple uses it in all places, even uname -m prints arm64.

@Tom, what do you suggest?
Comment 3 Thomas Watson CLA 2021-04-20 22:53:50 EDT
(In reply to Lakshmi P Shanmugam from comment #2)
> Did a Quick Search for "os.arch" in the releng aggregator repo and looks
> like the JVM property is used in a few places in equinox as well.
> May be we should just use "aarch64" instead on "arm64" in osgi.arch, to be
> consistent with Java and avoid problems in the future. This will also fix
> this SWT issue, without performing any mapping.
> 
> The reason we started using arm64 was Apple uses it in all places, even
> uname -m prints arm64.
> 
> @Tom, what do you suggest?

According to https://docs.osgi.org/reference/osnames.html

arm64 should be an alieas to aarch64 and ultimately this should result in osgi.arch and org.osgi.framework.processor being set to aarch64

Unfortunately the file alias file does not have this alias in bundles/org.eclipse.osgi/container/src/org/eclipse/osgi/internal/framework/processor.aliases

org.eclipse.osgi.internal.framework.EquinoxConfiguration.initializeProperties() as well as the equinox launcher that passes the -arch will need to be updated to handling this mapping also.  I think PDE would also need to be updated to use the correct value when configuring a launch configuration.
Comment 4 Lakshmi P Shanmugam CLA 2021-04-20 23:37:26 EDT
(In reply to Thomas Watson from comment #3)
> (In reply to Lakshmi P Shanmugam from comment #2)
> > Did a Quick Search for "os.arch" in the releng aggregator repo and looks
> > like the JVM property is used in a few places in equinox as well.
> > May be we should just use "aarch64" instead on "arm64" in osgi.arch, to be
> > consistent with Java and avoid problems in the future. This will also fix
> > this SWT issue, without performing any mapping.
> > 
> > The reason we started using arm64 was Apple uses it in all places, even
> > uname -m prints arm64.
> > 
> > @Tom, what do you suggest?
> 
> According to https://docs.osgi.org/reference/osnames.html
> 
> arm64 should be an alieas to aarch64 and ultimately this should result in
> osgi.arch and org.osgi.framework.processor being set to aarch64
> 
> Unfortunately the file alias file does not have this alias in
> bundles/org.eclipse.osgi/container/src/org/eclipse/osgi/internal/framework/
> processor.aliases
> 
> org.eclipse.osgi.internal.framework.EquinoxConfiguration.
> initializeProperties() as well as the equinox launcher that passes the -arch
> will need to be updated to handling this mapping also.  I think PDE would
> also need to be updated to use the correct value when configuring a launch
> configuration.

Thanks Tom! So, would it be better to use aarch64 instead of arm64 in SDK bundles?
Comment 5 Thomas Watson CLA 2021-04-21 11:29:36 EDT
(In reply to Lakshmi P Shanmugam from comment #4)
> (In reply to Thomas Watson from comment #3)
> > (In reply to Lakshmi P Shanmugam from comment #2)
> > > Did a Quick Search for "os.arch" in the releng aggregator repo and looks
> > > like the JVM property is used in a few places in equinox as well.
> > > May be we should just use "aarch64" instead on "arm64" in osgi.arch, to be
> > > consistent with Java and avoid problems in the future. This will also fix
> > > this SWT issue, without performing any mapping.
> > > 
> > > The reason we started using arm64 was Apple uses it in all places, even
> > > uname -m prints arm64.
> > > 
> > > @Tom, what do you suggest?
> > 
> > According to https://docs.osgi.org/reference/osnames.html
> > 
> > arm64 should be an alieas to aarch64 and ultimately this should result in
> > osgi.arch and org.osgi.framework.processor being set to aarch64
> > 
> > Unfortunately the file alias file does not have this alias in
> > bundles/org.eclipse.osgi/container/src/org/eclipse/osgi/internal/framework/
> > processor.aliases
> > 
> > org.eclipse.osgi.internal.framework.EquinoxConfiguration.
> > initializeProperties() as well as the equinox launcher that passes the -arch
> > will need to be updated to handling this mapping also.  I think PDE would
> > also need to be updated to use the correct value when configuring a launch
> > configuration.
> 
> Thanks Tom! So, would it be better to use aarch64 instead of arm64 in SDK
> bundles?

haha, I read my comment and realized I did not make that clear at all.  Yes!  I think it would be better if we used aarch64 all around.  I was just trying to outline where I thought we would need changes to do that.
Comment 6 Alexander Kurtakov CLA 2021-04-21 11:34:03 EDT
It's quite confusing right now that Linux uses aarch64 and MacOS uses arm64 so I would even suggest changing the macos fragments to match the preexisting linux fragments which use the canonical architecture name.
Comment 7 Eclipse Genie CLA 2021-04-21 11:37:28 EDT
New Gerrit change created: https://git.eclipse.org/r/c/equinox/rt.equinox.framework/+/179607
Comment 8 Thomas Watson CLA 2021-04-21 11:52:12 EDT
(In reply to Alexander Kurtakov from comment #6)
> It's quite confusing right now that Linux uses aarch64 and MacOS uses arm64
> so I would even suggest changing the macos fragments to match the
> preexisting linux fragments which use the canonical architecture name.

Yes, we should be consistent on the arch value regardless of the OS in my opinion.
Comment 10 Lakshmi P Shanmugam CLA 2021-04-21 12:53:54 EDT
Thanks for clarifying, Tom! Will make the changes.
Comment 11 Eclipse Genie CLA 2021-04-22 04:12:26 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/179657
Comment 12 Eclipse Genie CLA 2021-04-22 05:25:58 EDT
New Gerrit change created: https://git.eclipse.org/r/c/equinox/rt.equinox.bundles/+/179661
Comment 13 Eclipse Genie CLA 2021-04-22 05:32:19 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/179662
Comment 14 Thomas Watson CLA 2021-04-22 09:19:32 EDT
Why do we have these three mac launcher fragments:

bundles/org.eclipse.equinox.launcher.cocoa.macosx
bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64
bundles/org.eclipse.equinox.launcher.cocoa.macosx.x86_64

https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/bundles


The following both claim they are the arm64 one:
bundles/org.eclipse.equinox.launcher.cocoa.macosx
bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64

Eclipse-PlatformFilter: (& (osgi.ws=cocoa) (osgi.os=macosx) (osgi.arch=arm64) )

I assume the org.eclipse.equinox.launcher.cocoa.macosx one can be deleted and the org.eclipse.equinox.launcher.cocoa.macosx.arm64 one should be renamed to aarch64?
Comment 15 Sravan Kumar Lakkimsetti CLA 2021-04-22 09:39:39 EDT
(In reply to Thomas Watson from comment #14)
> Why do we have these three mac launcher fragments:
> 
> bundles/org.eclipse.equinox.launcher.cocoa.macosx
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.x86_64
> 
> https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/bundles
> 
> 
> The following both claim they are the arm64 one:
> bundles/org.eclipse.equinox.launcher.cocoa.macosx
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64
> 
> Eclipse-PlatformFilter: (& (osgi.ws=cocoa) (osgi.os=macosx)
> (osgi.arch=arm64) )
> 
> I assume the org.eclipse.equinox.launcher.cocoa.macosx one can be deleted
> and the org.eclipse.equinox.launcher.cocoa.macosx.arm64 one should be
> renamed to aarch64?

bundles/org.eclipse.equinox.launcher.cocoa.macosx is a symbolic link to bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64.

This is workaround for Bug 570960 in p2. Without this arm64 build won't succeed. I tried to fix Bug 570960 but failed. Once the p2 bug or we move to universal binary we can get rid of symbolic link.
Comment 16 Lakshmi P Shanmugam CLA 2021-04-22 09:54:05 EDT
(In reply to Thomas Watson from comment #14)
> Why do we have these three mac launcher fragments:
> 
> bundles/org.eclipse.equinox.launcher.cocoa.macosx
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.x86_64
> 
> https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/bundles
> 
> 
> The following both claim they are the arm64 one:
> bundles/org.eclipse.equinox.launcher.cocoa.macosx
> bundles/org.eclipse.equinox.launcher.cocoa.macosx.arm64
> 
> Eclipse-PlatformFilter: (& (osgi.ws=cocoa) (osgi.os=macosx)
> (osgi.arch=arm64) )
> 
> I assume the org.eclipse.equinox.launcher.cocoa.macosx one can be deleted
> and the org.eclipse.equinox.launcher.cocoa.macosx.arm64 one should be
> renamed to aarch64?

This also causes multiple launcher folders in the Eclipse SDK build - Bug 572817
Comment 17 Thomas Watson CLA 2021-04-22 10:06:55 EDT
Unfortunately I have very little knowledge of the native code in the launcher. I tried to locate the code that would do this but my understanding is that the native launcher is what sends in the -arch value.  I assume today it is passing arm64 today and that needs to change to aarch46 for the -arch argument to the java launcher?
Comment 18 Sravan Kumar Lakkimsetti CLA 2021-04-22 10:48:37 EDT
(In reply to Thomas Watson from comment #17)
> Unfortunately I have very little knowledge of the native code in the
> launcher. I tried to locate the code that would do this but my understanding
> is that the native launcher is what sends in the -arch value.  I assume
> today it is passing arm64 today and that needs to change to aarch46 for the
> -arch argument to the java launcher?

I am bit confused with this. Actually we don't pass -arch argument to java launcher in the native code. 

In the launcher we identify java native library appropriate for the architecture and load that library.

This is based on the DEFAULT_OS_ARCH attribute passed from build.sh to Makefile.mak in turn to eclipseCocoa.c.
Comment 19 Thomas Watson CLA 2021-04-22 11:22:22 EDT
(In reply to Sravan Kumar Lakkimsetti from comment #18) 
> I am bit confused with this. Actually we don't pass -arch argument to java
> launcher in the native code. 


I base this off the value I see for eclipse.commands property which can be viewed by Eclipse->About->Installation Details->Configuration tab

eclipse.commands=-os
macosx
-ws
cocoa
-arch
x86_64
-showsplash
-launcher
/Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS/eclipse
-name
Eclipse
--launcher.library
/Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.2.200.v20210406-1409/eclipse_11403.so
-startup
/Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../Eclipse/plugins/org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar
--launcher.appendVmargs
-keyring
/Users/tjwatson/.eclipse_keyring
-vm
/Library/Java/JavaVirtualMachines/adoptopenjdk-11-openj9.jdk/Contents/Home/bin/../lib/server/libjvm.dylib

That property gets set based on the String[] args passed to the launcher Main class.  Also confirmed by setting a breakpoint in Main.run(String[]) and looking at what the String[] args are based by the native launcher.
Comment 20 Sravan Kumar Lakkimsetti CLA 2021-04-22 11:41:56 EDT
(In reply to Thomas Watson from comment #19)
> (In reply to Sravan Kumar Lakkimsetti from comment #18) 
> > I am bit confused with this. Actually we don't pass -arch argument to java
> > launcher in the native code. 
> 
> 
> I base this off the value I see for eclipse.commands property which can be
> viewed by Eclipse->About->Installation Details->Configuration tab
> 
> eclipse.commands=-os
> macosx
> -ws
> cocoa
> -arch
> x86_64
> -showsplash
> -launcher
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS/eclipse
> -name
> Eclipse
> --launcher.library
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../
> Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.2.200.
> v20210406-1409/eclipse_11403.so
> -startup
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../
> Eclipse/plugins/org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar
> --launcher.appendVmargs
> -keyring
> /Users/tjwatson/.eclipse_keyring
> -vm
> /Library/Java/JavaVirtualMachines/adoptopenjdk-11-openj9.jdk/Contents/Home/
> bin/../lib/server/libjvm.dylib
> 
> That property gets set based on the String[] args passed to the launcher
> Main class.  Also confirmed by setting a breakpoint in Main.run(String[])
> and looking at what the String[] args are based by the native launcher.

Thanks Tom. Let me investigate.
Comment 21 Eclipse Genie CLA 2021-04-22 12:24:19 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.swt.binaries/+/179682
Comment 22 Sravan Kumar Lakkimsetti CLA 2021-04-22 13:13:56 EDT
(In reply to Thomas Watson from comment #19)
> (In reply to Sravan Kumar Lakkimsetti from comment #18) 
> > I am bit confused with this. Actually we don't pass -arch argument to java
> > launcher in the native code. 
> 
> 
> I base this off the value I see for eclipse.commands property which can be
> viewed by Eclipse->About->Installation Details->Configuration tab
> 
> eclipse.commands=-os
> macosx
> -ws
> cocoa
> -arch
> x86_64
> -showsplash
> -launcher
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS/eclipse
> -name
> Eclipse
> --launcher.library
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../
> Eclipse/plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.2.200.
> v20210406-1409/eclipse_11403.so
> -startup
> /Users/tjwatson/dev/tools/eclipse/latest/Eclipse.app/Contents/MacOS//../
> Eclipse/plugins/org.eclipse.equinox.launcher_1.6.100.v20201223-0822.jar
> --launcher.appendVmargs
> -keyring
> /Users/tjwatson/.eclipse_keyring
> -vm
> /Library/Java/JavaVirtualMachines/adoptopenjdk-11-openj9.jdk/Contents/Home/
> bin/../lib/server/libjvm.dylib
> 
> That property gets set based on the String[] args passed to the launcher
> Main class.  Also confirmed by setting a breakpoint in Main.run(String[])
> and looking at what the String[] args are based by the native launcher.

Here is the place where we add commandline argument -arch https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/features/org.eclipse.equinox.executable.feature/library/eclipse.c#n1179

This is also coming from DEFAULT_OS_ARCH provided by build.sh.
Comment 23 Eclipse Genie CLA 2021-04-23 13:37:56 EDT
New Gerrit change created: https://git.eclipse.org/r/c/equinox/rt.equinox.p2/+/179751
Comment 24 Eclipse Genie CLA 2021-04-23 14:07:18 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.ui/+/179758
Comment 25 Eclipse Genie CLA 2021-04-26 09:48:37 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.releng.aggregator/+/179836
Comment 26 Eclipse Genie CLA 2021-04-26 12:59:44 EDT
New Gerrit change created: https://git.eclipse.org/r/c/equinox/rt.equinox.binaries/+/179844
Comment 27 Eclipse Genie CLA 2021-04-26 13:01:59 EDT
New Gerrit change created: https://git.eclipse.org/r/c/equinox/rt.equinox.framework/+/179845
Comment 28 Lakshmi P Shanmugam CLA 2021-04-26 17:27:03 EDT
Note on the patches: The name of fragments, osgi.arch and arch has been changed to aarch64. But, all the native libraries will continue to be built with -arch arm64. aarch64 is not recognized and gives invalid arch name error.
Comment 34 Lakshmi P Shanmugam CLA 2021-04-27 15:47:49 EDT
Merged the platform changes.
@Tom, can you please merge the equinox changes?
Comment 39 Lakshmi P Shanmugam CLA 2021-04-27 17:16:19 EDT
Thanks Tom for merging the patches.
Comment 40 Lakshmi P Shanmugam CLA 2021-04-29 07:39:30 EDT
(In reply to Lakshmi P Shanmugam from comment #0)
> Platform.isLoadable() returns false with SWT Mac arm64.
> SWT manifest file sets SWT-Arch=arm64 but Library.os() returns aarch64
> causing mismatch.

Tested on M1 machine today.
This is now fixed with I20210427-1800. Platform.isLoadable() returns true as expected.
Comment 41 Lakshmi P Shanmugam CLA 2021-04-30 06:37:06 EDT
Tested the Eclipse Mac aarch64 build I20210429-1800 on Mac M1.
The Eclipse SDK Installation details page shows aarch64 as arch and osgi.arch.

Also, updated the bug title to reflect the changes done.