Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 549836

Summary: Enhance Mac signing service to sign Mac libraries
Product: [Technology] CBI Reporter: Lakshmi P Shanmugam <lshanmug>
Component: signing-serviceAssignee: CBI Inbox <cbi-inbox>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: daniel_megert, hubert+eclipseorg, mikael.barbero, mknauer, sravankumarl
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X   
Whiteboard:
Bug Depends on:    
Bug Blocks: 547159, 550135, 550545, 550674, 550677    

Description Lakshmi P Shanmugam CLA 2019-08-07 06:40:49 EDT
From https://bugs.eclipse.org/bugs/show_bug.cgi?id=547159#c3, the macOS signing service needs to be enhanced to allow lib signing. 

All the libraries need to be signed to allow notarisation of the Eclipse app.
Comment 1 Lakshmi P Shanmugam CLA 2019-08-09 06:56:53 EDT
Using the service, we also need a way to specify the hardened runtime while signing the app.

From [1] and [2], the command for app signing will be like this and java.entitlements file will contain the entitlements.

codesign --entitlements java.entitlements --options runtime --deep -vvv -f --sign "Developer ID Application: Bla Bla (XXXX)" YourApp.app

[1] - https://bugs.openjdk.java.net/browse/JDK-8223671?focusedCommentId=14282346&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14282346
[2] - https://github.com/AdoptOpenJDK/openjdk-build/issues/1130
Comment 2 Lakshmi P Shanmugam CLA 2019-08-22 02:12:22 EDT
Hi Mikael,
Any updates on the enhanced signing service?
Comment 3 Mikaël Barbero CLA 2019-08-22 07:52:14 EDT
No formal update yet. FYI, the maven plugin will need to be enhanced to pass the entitlements so it's likely it won't transparent for you and will require some pom changes.
Comment 4 Lakshmi P Shanmugam CLA 2019-08-27 05:11:12 EDT
(In reply to Mikaël Barbero from comment #3)
> No formal update yet. FYI, the maven plugin will need to be enhanced to pass
> the entitlements so it's likely it won't transparent for you and will
> require some pom changes.

Thanks for the update, Mikael!
Can we please get the service to sign the libraries first so that we can start working on signing the libraries?

We will then try the service to sign with hardened runtime when it's available and make the necessary pom changes.
Comment 5 Mikaël Barbero CLA 2019-08-29 06:25:01 EDT
(In reply to Lakshmi Shanmugam from comment #4)
> (In reply to Mikaël Barbero from comment #3)
> > No formal update yet. FYI, the maven plugin will need to be enhanced to pass
> > the entitlements so it's likely it won't transparent for you and will
> > require some pom changes.
> 
> Thanks for the update, Mikael!
> Can we please get the service to sign the libraries first so that we can
> start working on signing the libraries?

A new version of the service is running at the (temporary) URL http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT

You can call it like described on https://wiki.eclipse.org/IT_Infrastructure_Doc#Web_service, e.g.: 

curl -o libswt-pi-cocoa-4926r21.signed.jnilib -F file=@libswt-pi-cocoa-4926r21.jnilib http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT

Note that the service accept any kind of file as input, as well as zip archives of one or more .app (like the current service in production). 

> We will then try the service to sign with hardened runtime when it's
> available and make the necessary pom changes.

I'm on it.
Comment 6 Mikaël Barbero CLA 2019-08-29 12:54:58 EDT
(In reply to Mikaël Barbero from comment #5)
> > We will then try the service to sign with hardened runtime when it's
> > available and make the necessary pom changes.
> 
> I'm on it.

The service http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT has been updated to enable the hardebed runtime requirement. Any signing request now use "--options runtime". Also when signing an zipped application, --deep is passed to codesign.

Finally, you have the ability to add entitlements (if necessary) to the signing process. To do that:

curl -o org.eclipse.sdk.ide-macosx.cocoa.x86_64.signed.zip -F file=@org.eclipse.sdk.ide-macosx.cocoa.x86_64.zip -F entitlements=@eclipse.entitlements http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT

The maven plugin with the support for adding entitlements is not ready yet. I'll publish a snapshot tomorrow morning.
Comment 7 Sravan Kumar Lakkimsetti CLA 2019-08-30 04:35:11 EDT
(In reply to Mikaël Barbero from comment #5)
> (In reply to Lakshmi Shanmugam from comment #4)
> > (In reply to Mikaël Barbero from comment #3)
> > > No formal update yet. FYI, the maven plugin will need to be enhanced to pass
> > > the entitlements so it's likely it won't transparent for you and will
> > > require some pom changes.
> > 
> > Thanks for the update, Mikael!
> > Can we please get the service to sign the libraries first so that we can
> > start working on signing the libraries?
> 
> A new version of the service is running at the (temporary) URL
> http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT
> 
> You can call it like described on
> https://wiki.eclipse.org/IT_Infrastructure_Doc#Web_service, e.g.: 
> 
> curl -o libswt-pi-cocoa-4926r21.signed.jnilib -F
> file=@libswt-pi-cocoa-4926r21.jnilib
> http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT
> 
> Note that the service accept any kind of file as input, as well as zip
> archives of one or more .app (like the current service in production). 
> 
> > We will then try the service to sign with hardened runtime when it's
> > available and make the necessary pom changes.
> 
> I'm on it.

I am getting No route to host when I run from releng jipp

command used curl -o eclipse -F filedata=@unsigned-eclipse http://172.30.206.146:8282/macosx-signing-service/1.0.1-SNAPSHOT

job: https://ci.eclipse.org/releng/job/Mac-native-lib-signer/
Comment 8 Mikaël Barbero CLA 2019-08-30 04:36:46 EDT
I've just noticed that as well. The machine seems to be down. Unfortunately, I've no remote hardware control on this test machine and we will have to wait for Ottawa to wake up to restart the machine.
Comment 9 Mikaël Barbero CLA 2019-08-30 04:38:52 EDT
As a side note in your job https://ci.eclipse.org/releng/job/Mac-native-lib-signer/

you use

-F filedata=

when you should use

-F file=

(see comment #5 and https://wiki.eclipse.org/IT_Infrastructure_Doc#Web_service)
Comment 10 Mikaël Barbero CLA 2019-08-30 15:36:11 EDT
(In reply to Mikaël Barbero from comment #6)
> The maven plugin with the support for adding entitlements is not ready yet.
> I'll publish a snapshot tomorrow morning.

I've published a new snapshot version (1.1.8-SNAPSHOT) of the eclipse-macsigner maven plugin. You can now add a reference to an entitlements file, e.g.

<plugin>
  <groupId></gropuId>
  <artifactId></artifactId>
  <version></version>
  <configuration>
    <entitlements>${project.build.directory}/products/product.entitlement</entitlements>
  <configuration>
</plugin>


Also, the machine is back online.
Comment 11 Sravan Kumar Lakkimsetti CLA 2019-09-03 01:39:04 EDT
(In reply to Mikaël Barbero from comment #10)
> (In reply to Mikaël Barbero from comment #6)
> > The maven plugin with the support for adding entitlements is not ready yet.
> > I'll publish a snapshot tomorrow morning.
> 
> I've published a new snapshot version (1.1.8-SNAPSHOT) of the
> eclipse-macsigner maven plugin. You can now add a reference to an
> entitlements file, e.g.
> 
> <plugin>
>   <groupId></gropuId>
>   <artifactId></artifactId>
>   <version></version>
>   <configuration>
>    
> <entitlements>${project.build.directory}/products/product.entitlement</
> entitlements>
>   <configuration>
> </plugin>
> 
> 
> Also, the machine is back online.

I tried https://ci.eclipse.org/releng/job/Mac-native-lib-signer/ job today, but the input and output are showing same size. and codesign is reporting as not signed at all. 

Can you please check?
Comment 12 Mikaël Barbero CLA 2019-09-03 04:01:24 EDT
This one looks good to me

https://ci.eclipse.org/releng/job/Mac-native-lib-signer/10/

and the result:

$ codesign --verbose=4 --display -r- libswt-awt-cocoa-4928r15.jnilib
Executable=/Users/mbarbero/Downloads/libswt-awt-cocoa-4928r15.jnilib
Identifier=SigningServlet-7775035483327888744-unsigned-libswt-awt-cocoa-4928r15
Format=Mach-O thin (x86_64)
CodeDirectory v=20500 size=336 flags=0x10000(runtime) hashes=3+2 location=embedded
VersionPlatform=1
VersionMin=657920
VersionSDK=658432
Hash type=sha256 size=32
CandidateCDHash sha1=e8b68e2ce8bf113c3c698a00929077027b3c7da9
CandidateCDHash sha256=73180575fefca549e363ccfcdf8e290be0db561d
Hash choices=sha1,sha256
Page size=4096
CDHash=73180575fefca549e363ccfcdf8e290be0db561d
Signature size=9078
Authority=Developer ID Application: Eclipse Foundation, Inc. (JCDTMS22B4)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=3 Sep 2019 at 09:58:59
Info.plist=not bound
TeamIdentifier=JCDTMS22B4
Runtime Version=10.12.0
Sealed Resources=none
designated => identifier "SigningServlet-7775035483327888744-unsigned-libswt-awt-cocoa-4928r15" and anchor apple generic and certificate 1[field.1.2.840.113635.100.6.2.6] /* exists */ and certificate leaf[field.1.2.840.113635.100.6.1.13] /* exists */ and certificate leaf[subject.OU] = JCDTMS22B4
Comment 13 Sravan Kumar Lakkimsetti CLA 2019-09-03 04:17:31 EDT
Sorry my source url was wrong that caused the failure.
Comment 14 Lakshmi P Shanmugam CLA 2019-09-09 06:26:37 EDT
@Mikael,
Does EPP create their own Eclipse.app? Then, they should use this enhanced service too to sign the app? Or are they already using it?
Comment 15 Mikaël Barbero CLA 2019-09-09 06:28:07 EDT
You're right, they do create their own .app/.dmg. They should  use this new version of the service. 

Adding Markus.
Comment 16 Lakshmi P Shanmugam CLA 2019-09-09 07:04:20 EDT
For completeness sake, making note of the changes made in platform build to use the enhanced service to sign Eclipse.app.

1. Update the cbi version to 1.1.8-SNAPSHOT in the parent pom
https://git.eclipse.org/r/#/c/149014/1/eclipse-platform-parent/pom.xml

2. Create an entitlements file and add a reference to the entitlements file while mac app signing.
https://git.eclipse.org/r/#/c/148750/

3. Add signerUrl to point to the new url.
https://git.eclipse.org/r/#/c/149025/

The configuration looks like this after the changes - https://bugs.eclipse.org/bugs/show_bug.cgi?id=550135#c26

Commands to verify if the app is signed correctly
https://bugs.eclipse.org/bugs/show_bug.cgi?id=550135#c21
Comment 17 Lakshmi P Shanmugam CLA 2019-09-12 05:12:56 EDT
(In reply to Mikaël Barbero from comment #15)
> You're right, they do create their own .app/.dmg. They should  use this new
> version of the service. 
> 
> Adding Markus.

Hi Mikael/Markus,
Do you know if the change has been made in EPP build? Can you please provide the link to their downloads page, I can give the RC2 candidate a try on 10.15 beta?
Comment 18 Mikaël Barbero CLA 2019-09-12 05:14:20 EDT
AFAICT, there has been no change. I submitted bug 550997 and patch https://git.eclipse.org/r/#/c/149396/ to move forward with this topic.
Comment 19 Lakshmi P Shanmugam CLA 2019-09-12 05:28:23 EDT
(In reply to Mikaël Barbero from comment #18)
> AFAICT, there has been no change. I submitted bug 550997 and patch
> https://git.eclipse.org/r/#/c/149396/ to move forward with this topic.

Thanks!
Comment 20 Markus Knauer CLA 2019-09-12 05:31:36 EDT
Thank you, I've had a quick look at your change, added a +2 and already merged it to speed up the build process for the release.
Comment 21 Mikaël Barbero CLA 2019-09-30 10:48:32 EDT
We're done here.