Bugzilla – Full Text Bug Listing
| Summary: | Product-Export for MacOS does not set executable bit on native launcher (when building on Windows) | ||||||
|---|---|---|---|---|---|---|---|
| Product: | z_Archived | Reporter: | Steffen Kriese <steffen.kriese> | ||||
| Component: | Tycho | Assignee: | Project Inbox <tycho-inbox> | ||||
| Status: | CLOSED WORKSFORME | QA Contact: | |||||
| Severity: | enhancement | ||||||
| Priority: | P3 | CC: | caniszczyk, christian.campo, gregory.amerson, igor, jan.sievers, kim.moir, Lars.Vogel, peter, philippn, pierre.gaufillet, pierre.gaufillet, rolf.theunissen, sbouchet, stephan.druskat, t-oberlies, tanis, thanh.ha | ||||
| Version: | unspecified | ||||||
| Target Milestone: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Windows All | ||||||
| Whiteboard: | |||||||
| Attachments: |
|
||||||
|
Description
Steffen Kriese
Created attachment 201912 [details]
example parent.pom to reproduce the bug
I don't think that there is anything that Tycho can do in this case. There is no executable flag in windows, so if you pack a build result from a Windows machine and unpack it on a different machine, there will be no executable flag set. The other way round works, so in order to fix your problem, you should move your build to a *nix machine. Sounds reasonable, but it works with pde-build with the same scenario. So what kind of magic does pde-build use to get this running? What is the output format of the PDE build? The result is a zip-file. FWIW, the ant zip task (which I suppose is used by PDE headless build) seems to have some support for unix permissions http://ant.apache.org/manual/Tasks/zip.html I doubt that the Ant zip task will can set permissions for linux when the zip is created on windows. I would say that it is kind of normal that it is not possible to produce archives with executable flags on a windows machine. You could say that this is a bug, but none that I would fix. Nevertheless, it would be theoretically possible to do. This would require to change - the p2 chmod touchpoint so that it writes the permissions to extra files (on windows); - the plexus archiver, to take the permissions from these files and include them in the archive. If you feel that it is worth this effort, and you want to drive the necessary changes, feel free to reopen this issue as enhancement. @Chris, @Kim: Can you shed some light on how that is done in the PDE build ? What I am seeing here in this bug in comment 7 that it seems to be impossible for Tycho to deliver executables for Mac (that execute) when they are built on Windows computers. That is especially bad, when you build for multiple target OSes that you would not have your build computer on each of these OSes but rather choose an arbitrary OS for your build computer. It seems I am doing that with PDE build already for ages. Is that done with an Ant task or what is it ? thanks christian Is anybody actually using Windows as their canonical build-and-release systems? (In reply to comment #9) > Is anybody actually using Windows as their canonical build-and-release systems? We are using a Windows7 computer to build Riena with Jenkins….Otherwise we wouldn't have this problem. And we use it for all kind of other framework builds BTW. (that are not yet at eclipse) Do you want to exclude Windows as a supported platform for Tycho ? Or do you want to exclude the usage of Tycho as a cross platform build tool ? Once you like to support that, you get these kind of problems. And PDE got them sorted out (somehow). Is it that we need to know how the magic is done then I am sure Kim or Chris can help ? Or is it that you believe that this functionality does not belong into Tycho and thats why it is a WONTFIX ? (In reply to comment #10) > Do you want to exclude Windows as a supported platform for Tycho ? Or do you > want to exclude the usage of Tycho as a cross platform build tool ? Cross-platform builds work with Tycho, as long as you don't use Windows as build machine. Tycho will run on Windows in a typical local development scenario, i.e. you build something locally and play with the results on the same machine. > Once you like to support that, you get these kind of problems. And PDE got them > sorted out (somehow). An IIRC, Athena [1], which provides PDE build templates, doesn't even run on anything but Linux. So in that comparison, Tycho is already much better than the PDE. [1] http://wiki.eclipse.org/Common_Build_Infrastructure > Or is it that you believe that this functionality does not belong into > Tycho and thats why it is a WONTFIX ? I don't believe that this is a bug. It can be considered as a missing feature, and you are welcome to reopen the ticket as such, e.g. if you want to advertise it or provide patches. But to be honest: I don't believe that it is an important feature, so at least I won't do more than reviewing patches for it. Ok so what I do is I reopen the bug and set it to enhancement. That is ok for me to consider it an enhancement. If you think that it is not important thats ok. I am pretty sure we can not move any framework that we support to Tycho until it is fixed though and we have to see how that could be achieved. So we can leave that enhancement bug lingering for future reference for other projects that have the same problem. Is that ok for you guys ? (In reply to comment #12) > I am pretty sure we can not move any framework that we support to Tycho until it > is fixed though and we have to see how that could be achieved. Just out of curiosity: I didn't quite understand how you manage to build product archives with permissions on Windows today. Do you have a link to your build, or could you provide the steps to do this? (In reply to comment #13) > (In reply to comment #12) > > I am pretty sure we can not move any framework that we support to Tycho until it > > is fixed though and we have to see how that could be achieved. > Just out of curiosity: I didn't quite understand how you manage to build > product archives with permissions on Windows today. Do you have a link to your > build, or could you provide the steps to do this? I believe zip and certainly tar file formats support executable bit, so it's just a matter of coding tycho setting this bit inside the archive directly, i.e. not relying on filesystem support for this bit. Can't be that hard... and I will be happy to review and apply a quality patch ;-) (In reply to comment #13) > (In reply to comment #12) > > I am pretty sure we can not move any framework that we support to Tycho until it > > is fixed though and we have to see how that could be achieved. > Just out of curiosity: I didn't quite understand how you manage to build > product archives with permissions on Windows today. Do you have a link to your > build, or could you provide the steps to do this? This product build is planned to be at eclipse.org but it is not yet there. We have submitted a talk for ECE where we present it for the first time. That thing was maintained in CVS previously and we have just moved it to Git and changed the build from PDE build (actually pluginbuilder that creates a PDE build) to tycho maven build. We believe that the problem is the branding in the product build. While in PDE it takes the eclipse launcher i.e. Eclipse.app and just patches the icon or something into that archive, the tycho build actually unpacks, changes the content and packs it again. In the process of doing that, the attributes from within the archive that are not supported by the OS get lost…like the excutable flag on windows. My assumption is that the branding in PDE actually uses a zip task or something to overwrite the existing branding and does not actually unzip the archive. Does that sound possible ? (In reply to comment #15) > Does that sound possible ? This is all just guesswork - I would rather like to see this working. How to you build product archives on a Windows machine for Mac with correctly set executable bits in the archive? (In reply to comment #16) > (In reply to comment #15) > > Does that sound possible ? > This is all just guesswork - I would rather like to see this working. How to > you build product archives on a Windows machine for Mac with correctly set > executable bits in the archive? We just use a product build in both cases….. so we have something like scrum box.product in which we specify the .ico or .icns or .xpm files for the icon for the various platforms in PDE we specify that scrum box.product file in the build.properties as product= In Tycho we also havve <configuration><products><product><id>scrumbox.product</.…. I think only for Mac is the Scrumbox.app file in fact a zip archive. For Windows and Linux that is different. So on Mac you need to get the .icns somewhere into the structure of the .zip archive. PDE does that with an Ant task and Tycho unpacks, replaces the file and packs again. All properties that are not supported by the OS are getting lost…. I took some time to investigate this problem or better the reason why this works in PDE Build.
PDE Build works that it first generates an .xml ant script and then it executes it. There is one ant script for mac called assemble.org.eclipse.pde.build.container.feature.macosx.cocoa.x86_64.xml.
In there is a target for our product that looks like this:
<target name="assemble.archive">
<antcall target="customAssembly">
<param name="rootFolder" value="${eclipse.base}/macosx.cocoa.x86_64/${collectingFolder}"/>
<param name="customTarget" value="pre.archive"/>
</antcall>
<zip destfile="${archiveFullPath}" filesonly="false" whenempty="skip" update="true">
<zipfileset dir="${eclipse.base}/plugins/org.jfreechart_1.0.13" prefix="${pluginArchivePrefix}/org.jfreechart_1.0.13"/>
<!--....more zipfilesets of various jars....-->
<zipfileset dir="${eclipse.base}/macosx.cocoa.x86_64/${collectingFolder}" includes="**/**" excludes="${launcherName},libcairo-swt.so,${launcherName},libcairo-swt.so,${launcherName},${launcherName}.app/Contents/MacOS/${launcherName},${launcherName}" prefix="${archivePrefix}" />
<zipfileset file="${eclipse.base}/macosx.cocoa.x86_64/${collectingFolder}/${launcherName}.app/Contents/MacOS/${launcherName}" fullpath="${archivePrefix}/${launcherName}.app/Contents/MacOS/${launcherName}" filemode="755" />
<zipfileset file="${eclipse.base}/macosx.cocoa.x86_64/${collectingFolder}/${launcherName}" fullpath="${archivePrefix}/${launcherName}" filemode="755" />
</zip>
</target>
So the filemode="755" sets the flag for the executable to read+execute.
Not sure if Tycho uses Ant Tasks or Java API to create the ZIP. And if Java does it use the regular Zip API or the one from Apache ?
But thats how it is done in the PDE Build.
HI, just my 0.02 cents, but does zip really takes care about file permissions ? For linux and macOS eclipse downloads, the packages are not using .zip, but .tar.gz ( taht is handling the execution permission case ) http://download.eclipse.org/eclipse/downloads/drops4/S-4.2M4-201112092100/index.php So for eclipse applications, the packaging ( and more especailly the file extension ) should be .zip for windows, and .tar.gz for *nix cheers, it is possible to configure Tycho 0.14 to create tar.gz product archives for linux and macosx, see bug 368079. (In reply to comment #20) > it is possible to configure Tycho 0.14 to create tar.gz product archives for > linux and macosx, see bug 368079. thanks igor. (In reply to comment #19) > just my 0.02 cents, but does zip really takes care about file permissions ? yes the zip created by the maven archiver component has file permission info retained, but this doesn't help when creating the zip from content on a file system without file permissions. (In reply to comment #22) > (In reply to comment #19) > > just my 0.02 cents, but does zip really takes care about file permissions ? > > yes the zip created by the maven archiver component has file permission info > retained, but this doesn't help when creating the zip from content on a file > system without file permissions. Thats true. I think a helpful solution "only" needs to supply a possibility in Tycho (say a property or marker) somewhere where I can specify which file needs to have the exe-flag set when I include it in a zip. The fact that the flag got lost in the filesystem would then become irrelevant. I just add it no matter whether the native filesystem had it or not. (In reply to comment #23) > Thats true. I think a helpful solution "only" needs to supply a possibility in > Tycho (say a property or marker) somewhere where I can specify which file needs > to have the exe-flag set when I include it in a zip. The fact that the flag got > lost in the filesystem would then become irrelevant. I just add it no matter > whether the native filesystem had it or not. IMHO this is not a solution for a portable build. The permission info is stored in the p2 metadata, and having to specify them somewhere else again is just a hack. Getting the permissions out of the p2 metadata shouldn't be hard either: The chmod touchpoint (in org.eclipse.equinox.p2.touchpoint.natives) just needs to write the permissions to some file when running on Windows instead of ignoring the chmod altogether. Tycho could then pick up the information from these (e.g.) ".permissions" files and pass them to the archiver (given that this is possible). (In reply to comment #18) > I took some time to investigate this problem or better the reason why this works > in PDE Build. Good effort, but I don't believe that hard-coding known permissions is a solution either. Tycho should not need to know about Eclipse installation format produced by the director. It took ages to teach p2 to do the same product-related tricks that the PDE does (see bug 331974), I just don't want to do this again for Tycho. I was able to reproduce this issue on macosx as my build system. In my case I was building the CBI Platform and noticed when building on macosx the executable bit on the native launcher was missing. This affected 2 files: -rw-r--r-- 1 user staff 25072 5 Jul 15:49 eclipse -rw-r--r-- 1 user staff 25072 5 Jul 15:49 Eclipse.app/Contents/MacOS/eclipse When building on linux the result was different for the macosx tar.gz producing a working launcher: -rw-r--r-- 1 user staff 25072 5 Jul 13:16 eclipse -rwxr-xr-x 1 user staff 25072 5 Jul 13:16 Eclipse.app/Contents/MacOS/eclipse I'm able to launch by double clicking on Eclipse.app using the linux version of the build. However the version built by macosx is missing the permissions. Something else noticed on the macosx build though is our build leaves a directory /target/products/org.eclipse.sdk.epp/macosx/cocoa/x86_64/eclipse which I assume is the directory before the tar.gz file is created. It leaves a very different result: lrwxr-xr-x 1 user staff 34 5 Jul 15:52 eclipse -> Eclipse.app/Contents/MacOS/eclipse -rwxr-xr-x 1 user staff 25072 5 Jul 15:49 Eclipse.app/Contents/MacOS/eclipse Not only is the shortcut is actually a symbolic link, the native launcher has the correct permissions leading me to believe the issue is caused when the tar.gz file is created. I thought I'd try making my own tar of this directory using the following command. tar -pcvzf macosx.tar.gz eclipse When I extracted it, the correct permissions were produced: lrwxr-xr-x 1 user staff 34 5 Jul 15:52 eclipse -> Eclipse.app/Contents/MacOS/eclipse -rwxr-xr-x 1 user staff 25072 5 Jul 15:49 eclipse (In reply to comment #25) > I was able to reproduce this issue on macosx as my build system. In my case I > was building the CBI Platform and noticed when building on macosx the > executable bit on the native launcher was missing. > When building on linux the result was different for the macosx tar.gz producing > a working launcher: this is a different bug. See plexus-io tar archiver bug 368089 and workaround described there. (In reply to comment #25) > Not only is the shortcut is actually a symbolic link https://jira.codehaus.org/browse/PLXCOMP-117 Restoring OS flag - this bug is explicitly a request to support executable bits if the build machine is running Windows (In reply to comment #26) > (In reply to comment #25) > > I was able to reproduce this issue on macosx as my build system. In my case I > > was building the CBI Platform and noticed when building on macosx the > > executable bit on the native launcher was missing. > > When building on linux the result was different for the macosx tar.gz producing > > a working launcher: > > this is a different bug. See plexus-io tar archiver bug 368089 and > workaround described there. The workaround in bug 368089 does not work at all. I just tried that on a Mac build environment and it does not set the executable flags properly. Is there any solution planned or any true workaround? (In reply to comment #29) > I just tried that on a Mac build environment Then, this is the wrong bug for you (see comment #28). There should be some support now on windows to store Linux file permissions into extra metadata of a file, see https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/ It should be feasible, if p2 would add this extra metadata in its chmod touchpoint and tycho would set the permissions in the zip/tar.gz file if this metadata exists. Eclipse Tycho is moving away from this bugs.eclipse.org issue tracker to https://github.com/eclipse/tycho/issues/ instead. If this issue is relevant to you, your action is required. 0. Verify this issue is still happening with latest Tycho 2.4.0-SNAPSHOT if issue has disappeared, please change status of this issue to "CLOSED WORKFORME" with some details about your testing environment and how you did verify the issue; and you're done if issue is still present when latest release: * Create a new issue at https://github.com/eclipse/tycho/issues/ ** Use as title in GitHub the title of this Bugzilla ticket (may include the bug number or not, at your own convenience) ** In the GitHub description, start with a link to this bugzilla ticket ** Optionally add new content to the description if it can helps towards resolution ** Submit GitHub issue * Update bugzilla ticket ** Add to "See also" property (up right column) the link to the newly created GitHub issue ** Add a comment "Migrated to <link-to-newly-created-GitHub-issue>" ** Set status as CLOSED MOVED ** Submit All issues that remain open will be automatically closed next week or so. Then the Bugzilla component for Tycho will be archived and made read-only. Tycho issue have been moved to Github. Please reopen at https://github.com/eclipse/tycho/issues if this issue is still relevant. |