Bug 74664

Comment 1 Jeff McAffer 2004-10-05 09:37:51 EDT

This is unfriendly since it will effectively provide the META-INF "package".  
As a result, any local META-INF will be overridden which the META-INF from your 
required plugins.

I don't see how htis would work in regular Java since any given classpath might 
contain these META-INF files and the jars on the classpath could be in any 
order.  So it is anyone's guess as to which META-INF would actually be used.

The better approach is to load the required files directly using Bundle.getEntry
().

Comment 2 Igor Fedorenko 2004-10-05 12:13:05 EDT

Thank you for your response, Jeff.

Are you suggesting use of Bundle.getEntry internally by Eclipse ClassLoader
implementation or by external code that I run in Eclipse? The latter is not
possible for at least two reasons. First, that's not my code that uses resources
from META-INF directory, so I am not able to  change it (in my case, this is
WebSphere Application Client). Second, this code has to run both inside and
outside of Eclipse.

At the same time I agree that my "patch" has few undesirable side effects and
should not be applied as is. I just hacked something quick to keep me going ;-)
Let me try to explain what I think a correct implementation should do

1. ClassLoader.getResource and ClassLoader.getResources should use the same
search order as ClassLoader.getClass. They should not stop the search at first
directory match (i.e. if I have META-INF directory in multiple classpath entries
all of the should be considered during the search).
2. ClassLoader.getResource() should return the first matching resource.
3. ClassLoader.getResources should return all matching resources.
4. Resource names should not be restricted by java package and class name rules.

Speaking of implementation, I think resource lookup should be decoupled from 
class (now they both use the same package to classpath elements map). This will
provide support for arbitrary resource names (there is nothing wrong with
"META-INF/some /wierd directory/name" resource name) without breaking more
restrictive java package/class naming rules. If you are interested, I might be
able to provide better patch that solves these problems.

Comment 3 Jeff McAffer 2004-10-07 12:36:03 EDT

I see the problem but am still curious how code written in such a way should 
expect to work. In normal Java they have no idea what jars wil lbe on the 
classpath ahead of them.  Loading resources from the META-INF "package" will 
give random results.  The only interesting case is getResources() where you get 
them all.  

so, some investigation in this area would be useful and a more detailed patch 
would clarify the expected behaviour

Comment 4 Igor Fedorenko 2004-10-07 23:25:23 EDT

Here are few examples of how different libraries/products load files from
META-INF directory. 

First, below is a snippet from javax.xml.transform.FactoryFinder. Looks like SUN
expects to find only one service description, and if there are many -- first one
wins.

<quote>
        String serviceId = "META-INF/services/" + factoryId;
        // try to find services in CLASSPATH
        InputStream is=null;
        if (classLoader == null) {
            is=ClassLoader.getSystemResourceAsStream( serviceId );
        } else {
            is=classLoader.getResourceAsStream( serviceId );
        }
</quote>

Here is another example -- JBoss looks for some WS4EE deployment descriptors.
Again, they expect to find only one deployment descriptor.

<quote>
         String[] infDirs = new String[]{"META-INF", "WEB-INF"};
         for (int i = 0; resourceURL == null && i < infDirs.length; i++)
         {
            String resName = infDirs[i] + "/ws4ee-deployment.xml";
            resourceURL = resourceCL.findResource(resName);
         }
</quote>


Another example is Jakarta Commons Discovery Component (which is used for
example by Apache Axis and by WebSphere WebServices implementation) -- it uses
ClassLoader.getResources(String) to discover service providers, again, under
META-INF/services directory. There is interesting comment in
org.apache.commons.discovery.jdk.JDK12Hooks

<quote>
    public Enumeration getResources(ClassLoader loader,
                                    String resourceName)
        throws IOException
    {
        /**
         * The simple answer is/was:
         *    return loader.getResources(resourceName);
         * 
         * However, some classloaders overload the behavior of getResource
         * (loadClass, etc) such that the order of returned results changes
         * from normally expected behavior.
         * 
         * Example: locate classes/resources from child ClassLoaders first,
         *          parents last (in some J2EE environs).
         * 
         * The resource returned by getResource() should be the same as the
         * first resource returned by getResources().  Unfortunately, this
         * is not, and cannot be: getResources() is 'final' in the current
         * JDK's (1.2, 1.3, 1.4).
         * 
         * To address this, the implementation of this method will
         * return an Enumeration such that the first element is the
         * results of getResource, and all trailing elements are
         * from getResources.  On each iteration, we check so see
         * if the resource (from getResources) matches the first resource,
         * and eliminate the redundent element.
         */
        
        final URL first = (URL)loader.getResource(resourceName);
        final Enumeration rest = loader.getResources(resourceName);
</quote>

And I am sure that there are very many other libraries and products that use
META-INF. Also, META-INF is only one of the two problems with the current
Eclipse implementation, spaces should be allowed in resource names too. The
following three calls are all legitimate and all three should return requested
resources even if these resources happen to belong to required plugins.


ClassLoader cl = getClass().getClassLoader();
cl.getResource("com/ibm/tivoli/orchestrator/oerr.properties");
cl.getResource("META-INF/other.properties");
cl.getResource("this is a name too/more.properties");

PS: this was "investigation" part, "more detailed patch" will likely take some
time as I do not have too much spare time at the moment.

Comment 5 Pascal Rapicault 2004-10-14 22:55:47 EDT

I've done some investigation and I can't really see a way that would make everybody happy when using 
a generated manifest.mf. Indeed, the current code filters out meta-inf and all packages that do not 
have a .class file, and this is fine for all eclipse plugins. Loosening that rule could break people code.

It seems to me that the solution is to have manually handcrafted manifest (by opposition to the 
generated ones) listing precisely things that must be present on the classpath, and of course they could 
include the meta-inf folder if they want.

Regarding the behavior for the lookup of resource, it does match the one of classes when things are 
exported.

Comment 6 Jeff McAffer 2004-10-15 15:13:03 EDT

I agree with comment 5.  The auto transformation of plugin.xml should be as 
safe as possible.  if we go around doing Provide-Package: META-INF this could 
be disruptive.  People how have this special need should craft specific 
manifest.mf files which detail the behaivour and ensure that their code does 
the right thing.

Comment 7 Igor Fedorenko 2004-10-15 16:11:16 EDT

I guess I am convienced, but instead of manual creation of manifests (for the
record, WAS application client requires some 200+ jar files) I'll write little
script to do that.

Comment 8 Jeff McAffer 2004-10-19 21:29:58 EDT

Igor, for context, do all 200+ of these need to have this behaviour?  It seems 
that only plugins which might have services (or some other thing discovered via 
this mechanism) would need such manifests.

In any event, do you think there is anything for us to do here or can we close 
the issue?

Comment 9 Igor Fedorenko 2004-10-20 09:04:25 EDT

These 200+ jar files are "thirdparty" to my application, I do not have access to
their sources and do not know anything about their implementation. I wrap these
jar files in a plugin (similarly to what you do to ANT or JUnit) and use them as
I would use thirdparty library in a standalone application.

Any thirdparty library that uses thread context classloader to load resources
from META-INF or directory with spaces will have this problem. Note that the
resources can belong to thirdparty library or they can belong to my plugins.

Simple script that generates manifest looks like a reasonable
workaround/solution. You can close this issue.

Comment 10 Jeff McAffer 2004-10-22 15:30:55 EDT

Closing as suggested.