Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 243163

Summary: [jar exporter] export directory entries in "Runnable JAR File"
Product: [Eclipse Project] JDT Reporter: Tom Blough <tom>
Component: UIAssignee: Dani Megert <daniel_megert>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: P3 CC: daniel_megert, ferenc.hechler, pembo13, tom
Version: 3.4.2Keywords: helpwanted
Target Milestone: 3.5 M2   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
patch to export directories into runnable JARs daniel_megert: iplog+, daniel_megert: review+

Description Tom Blough CLA 2008-08-05 09:49:00 EDT
Please add an option to "add directory entries" to the new "Runnable JAR File" export.  This option already exists in the older "JAR File" export.

I ran into a problem where Jar-ing some existing projects with Ganymede caused getClass().getClassLoader().getResource( path ) to return null due to the fact that the empty directory entries had not been exported to the Jar.
Comment 1 Ferenc Hechler CLA 2008-08-05 10:53:00 EDT
We try to keep the UI as simple as possible. 
It should be easy to create a runnable jar without confusing the user.

So we should decide whether it is generally good to include the directory entries or not.

I see no problem to always include the diretcory entries.

Are you sure that the problem was caused by missing directory entries?
I had no problem till now. I am using jdk 1.6 under windows.

Best regards,

     feri
Comment 2 Tom Blough CLA 2008-08-05 19:26:00 EDT
Ferenc,

I must say that I agree with your proposal.  Always adding directory entries to the archive has a negligible effect on the archive size, and no other downside that I can think of.  I'm curious as to why the developers on the original JAR export thought it necessary to provide the option to NOT export the directories.
Perhaps one of them is still around and can shed some light on why they chose this path?

I am sure the problem was caused by not having the directory entries in the JAR.  getClassLoader.getResource will return the URL version of a directory on a filesystem given a String directory path.  It returns null on a JAR unless the empty directory entry is included in the JAR.  A quick web search for directory entries in JAR files turns up a few instances referring to directory entries in a JAR:

<http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4761949> Same problem reported as a bug to Sun Developer Network.  It appears that Sun did not consider the inconsistent operation of getResource to be a bug.

<http://dev.eclipse.org/newslists/news.eclipse.webtools/msg11077.html> Eclipse Webtools posting concerning Facelets needing directory entries in a Jar.

<http://www.velocityreviews.com/forums/t133491-repacking-jar-files.html> seems to suggest that including directory entries in a JAR will cause problems extracting the files.  A quick test with both Winzip 9.0SR1, and Jar.exe 1.6.0_01 shows that this is not a problem with these two archivers.

<http://www.kaffe.org/pipermail/kaffe/2000-February/125731.html> posting from Kaffe forums noting change to Kaffe's jar util to create directory entries to make it compatible with Sun's Jar util behavior of including directory entries.

Regards,

Tom


Comment 3 Ferenc Hechler CLA 2008-08-06 02:42:50 EDT
A short analyse after a quick look into the code:


jarpackagerfat.JarWriter4:
[--- Line 72 ---]
public void write(File file, IPath destinationPath) throws CoreException {
  try {
    Assert.isLegal(!fJarPackage.areDirectoryEntriesIncluded());//Fat JAR writer does not include directory entries
[------]

ok, that has to be changed...

JarWriter4 is inherited from class jarpackager.JarWriter which has an addDirectories() method. This method creates missing directory entries including its parents. 
A list of already created directory entries is managed, so that each dir is added only once.
So the change would be to replace the assertion with the following line of code:

[---]
if (fJarPackage.areDirectoryEntriesIncluded())
  addDirectories(destinationPath, file);
[---]

The initial value in JarPackageData is setIncludeDirectoryEntries(false).
This has to be overwritten in the init() or performFinish() method of class FatJarPackageWizard.

That should be not much to do, but I am affraid a lot of JUnit tests checking the JAR-Generation will fail and have to be adjusted.   :(
Comment 4 Ferenc Hechler CLA 2008-08-06 02:50:45 EDT
correction:

* jarpackagerfat.JarWriter4 inherits from jarpackager.JarWriter3

* the method JarWriter3.addDirectories() only takes one argument: addDirectories(destinationPath);

the rest of the analysis should be ok.  ;)
Comment 5 Ferenc Hechler CLA 2008-08-06 04:08:13 EDT
I would like to move the add-directory-check in JarWriter3 from the public methode write() to the protected method addEntry() which is called at end:
jw3.write() -> jw3.addFile() -> jw3.addEntry()

So all callers of addEntry have their directories added and have not to add the add-directory-check too. Theese are:

JarWriter3.addFile()
JarWriter4.addFile()
JarWriter4.addZipEntry()
JarWriter4.addZipEntryStream()

As a result we have not to add any checks in JarWriter4 any more.

The behaviour of the public JarWriter3 remains unchanged and JarWriter4 is internal.


Changes in FatJarExportTests:
[--- 2 times ---]
  JarPackageData data= createJarPackageData(project, testName, false);
  data.setIncludeDirectoryEntries(true);
[---------------]

The good news: all Tests are still green, only positive checks for Jar-Entries are done.  :)


Comment 6 Dani Megert CLA 2008-08-06 04:24:07 EDT
>I'm curious as to why the developers on the original JAR
>export thought it necessary to provide the option to NOT export the
>directories.
The problem is that it is not specified what the Java jar command exactly does. Sometimes it adds empty directories and sometimes it does not. Fact is that standard Java (not speaking about other third party code) does work without those directories and hence adding them by default just seemed wrong. Basically, if a tool fails by the absence of empty directories that tool made wrong assumptions.

In order to be consistent the Runnable JAR file exporter should also add that option and not pollute the JAR with empty directories out of the box.
Comment 7 Ferenc Hechler CLA 2008-08-06 04:53:42 EDT
We have stripped off many options from the Runnable-Jar-File exporter.
As a result the exporter is not usable for all purposes.
It is meant as a simple tool which works for most of the cases (the more, the better).

Examples for options which are not available to be changed in the UI:

* Exclude filter for files, e.g. "*.properties" (we include all)
* Additional Class-Path entries (we always use ".")
* Subfolder-name for Export as "small" runnable Jar with lib-folder containing required libraries (we use "<jarname>_lib")

So I would not like to add an option for this rather technical issue.
If is is best to allways include the directories, we can do so.

From my point of view it should be uncritical to blow up the jar. 
We are talking about the FatJarExportWizard.   ;)

Perhaps it would help to see if there are real problems (not theoretically) with missing directory entries. I do not think, that Facelets will be exported as Runnable JAR.


Best regards,

   feri
Comment 8 Dani Megert CLA 2008-08-06 05:00:17 EDT
>So I would not like to add an option for this rather technical issue.
>If is is best to allways include the directories, we can do so.
Well, by adding all empty directories which is not required by Java you seem to react to some rather technical issue outside Eclipse ;-)

But given that it produces a fat JAR we can probably live with the empty directories being added. Just don't go and make it the default for the JAR writer.
Comment 9 Ferenc Hechler CLA 2008-08-06 05:50:04 EDT
Created attachment 109283 [details]
patch to export directories into runnable JARs

* Added support for the JarPackageData.fIncludeDirectoryEntries to the JarWriter4.

* Refactored method addDirectories(IPath) to call new method addDirectories(String) which is needed by JarWriter4. Behaviour of JarWriter3 remained unchanged.

* Set include-directories Flag to TRUE for Runnable-Jar-Exporter.

* Set include-directories Flag to TRUE in createJarPackageData()

No testcases have been added to check for directory-entries in export

REMARK: If this patch will be accepted, it has to be worked into Bug 219530
Comment 10 Dani Megert CLA 2008-08-06 05:57:52 EDT
I suggest we first commit this one and then you can adjust the patches in bug 219530.
Comment 11 Ferenc Hechler CLA 2008-08-19 14:26:20 EDT
I have looked for the JARs generated via the exportet ANT build-files.
The "jar"-Task has an attribute "filesonly" which determines whether directory entries are created or not. 
No changes have to be done, because the default value is "false", so directory entries are always created.
Comment 12 Dani Megert CLA 2008-08-20 02:50:20 EDT
But as you can see the option to not include directories is something people want/need otherwise it wouldn't be there.
Comment 13 Ferenc Hechler CLA 2008-08-20 03:38:07 EDT
But the default is to include the directories.   
:-)
Comment 14 Dani Megert CLA 2008-09-15 05:42:31 EDT
Thanks for the patch. Committed to HEAD with two minimal changes:
- removed the following comment from JarWriter3.addDirectories(IPath):
     "Calls addDirectories(String)."
  as we should not write this in stone
- add missing @since 3.5 tag

Available in builds >= I20080914-0800.
Comment 15 Ferenc Hechler CLA 2008-09-15 15:10:20 EDT
Should I adjust the patch for bug 219530 which is in legal review or will this be done later?


Comment 16 Dani Megert CLA 2008-09-16 02:25:48 EDT
>Should I adjust the patch for bug 219530 which is in legal review or will this
>be done later?
Please adjust it now. It makes it easier to review against latest code.
Comment 17 Ferenc Hechler CLA 2008-09-16 18:01:10 EDT
the merging into bug 219530 against the latest code is not that simple, everything conflicts...
"Removed trailing whitespace and organized the imports"
;)
Comment 18 Arthur Pemberton CLA 2010-04-16 15:08:09 EDT
I seem to be running into this very problem in 2010, Eclipse 3.5.2, Java 1.6.0.

I am making use of .getResource() and I am getting nulls after exporting to "Runnable jar file".
Comment 19 Arthur Pemberton CLA 2010-04-16 15:19:29 EDT
Please disregard my last comment: apparently, things like '//' in the resource path work outside the JAR but not inside it.