Bugzilla – Full Text Bug Listing
| Summary: | Need a way to calculate the "uses" directive on exports | ||
|---|---|---|---|
| Product: | [Eclipse Project] PDE | Reporter: | Thomas Watson <tjwatson> |
| Component: | UI | Assignee: | Brian Bauman <baumanbr> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | hargrave, jeffmcaffer, john.arthorne, ob1.eclipse, pascal, Peter.Kriens, wassim.melhem |
| Version: | 3.3 | ||
| Target Milestone: | 3.3 M5 | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Whiteboard: | |||
|
Description
Thomas Watson
I think some of the work Oleg is doing in the PDE incubator for API tools could help here. The code for "Usage Discovery" may be able to be used to calculate the "uses" directive on Export-Package statements? Thoughts Oleg? Otherwise this bug report is pretty quite. Any plans to address this for 3.3? Yes, the reference extractor from PDE-incubator/api-tooling might help. On a more general note does it mean that you'd like all Export-Package directives to have a "uses" clause? It would be a bit of a pain to maintain it. Yes, every API package should define the "uses" directive. That should be consider part of defining your API. Yes I agree it is a pain to maintain, but my hope is tooling will relieve us from that pain. As for what will actually get into 3.3, Jeff and I had an oral agreement that we will leave all package management work (import/export) to Oleg and we will incorporate what is appropriate for development time into PDE and will leave what is appropriate for build time for Releng. Note that oral agreements are binding in the province of Ontario. Tooling and support for "uses" makes sense, but I don't think we can reasonably expect development teams to add and maintain "uses" for every single API package. It just seems like too much work and prone to mistakes/staleness. In the Eclipse team context, does the problem only arise with packages that are imported using Import-Package (i.e., service packages)? That would greatly narrow the scope of the required effort, since most bundles use Require-Bundle. This does not only apply to service packages or Import-Package constraints. See bug 169593 and bug 155996 for more details. We can see many issues if multiple versions of the same bundle are installed and resolved at the same time. If we want to support such scenarios then we must start defining the "uses" directive. Assigning to Bauman. Brian, we envision this functionality to have two UI hooks: 1. a button in the Export-Package section of the plug-in manifest editor 2. an option in the Exported Packages section of the Organize Manifests wizard that would be checked by default. Jeff may have some working code that can be used as starting point. The ideal solution, from a PDE/UI standpoint, would have to use JDT/Core APIs when calculating the uses directive. Should we include the package of a super class or interface of a class? Ie. A public class in a public package is extending an internal class, should the internal class's package be included in the 'uses' directive? CCing Peter Kriens who have some experience on this. I have been calculating the uses directive for about 2 years now in btool and now also in bnd (http://www.aqute.biz/Code/Bnd). I fully agree that the developer should NEVER add the uses themselves, too error prone. Same is true for Import-Package btw. We get a bad rap for being complicated because we expose these to the developer unnecessary (all information is in the class files, so why on earth should we repeat it??). PDE should be able to calculate the uses directive for plug-ins for 3.3M5. As per comment #7, it can be done from the Organize Manifest Wizard or a button on the Export Package Section of the Manifest Editor. Also included some formatting for the directive so Manifest don't become completely unreadable. I just tested the function. Nice job, particularly on the formatting. Let me start by saying that, in my opinion, the solution offered in this bug is highly counterproductive and not ready for use. It won't solve the problem but rather introduce another hard-to-remember step for all Eclipse developers. Anybody really thinks this change will make an average Eclipse developer happy? If not, why we are doing it? While the OSGi wiring problem is real, the solution, to me, is done in a wrong dimension. To me, the real problem is that OSGi needs some data that is rather expensive to calculate on every startup. So, what if we cache it - either during initial Eclipse run or during the bundle build process. Moreover, after attempting to use it I was left with the impression that either I am missing something or there are bugs in its implementation. Details: - Including "uses" directive in a day-to-day software development ties manifest.mf files to PDE tools. It will be impractical (or impossible) to maintain manifest.mf tools outside of PDE as calculating "uses" directives by hands will be very time consuming and error-prone. - Usability: in the current implementation "uses" directives are guaranteed to either not to be used by consumers or to become stale and outdated. The action of calculating uses should be automatic, not a separate button. Of course, this is rather hard to implement without impacting performance - but that's one other reason why I believe that this solution goes in a wrong direction. - Results: It seems that there are some issues (or bugs?) in the "uses" directives generated. Here are some examples: a) What packages are included in the "uses" directive? Do we include "x-friends"? ExtensionTracker from o.e.equinox.registry has "import org.eclipse.core.internal.runtime.RuntimeLog;" but no "uses" is generated for the "org.eclipse.core.internal.runtime". b) Used packages used not shown RegistryStrategy from o.e.equinox.registry imports org.eclipse.osgi.util.NLS but it's not in the "uses" clause BaseExtensionHandle from the same bundle has "import org.eclipse.core.runtime.*;" but no uses are generated c) Java packages I noticed that there is uses directive generated for the "javax.xml.parsers". While I can understand the pragmatic reason for this, it begs a question which Java packages are included in the "uses" directive and which aren't. To summarize: - The current solution ties manifest.mf files to PDE tools - Due to the manual nature of the solution proposed, "uses" directives are guaranteed to go out of synch - There are some issues with results it currently generates I do believe that we need to consider solving the OSGi wiring problem in some other way, perhaps by calculating information it needs on the first startup or during the build process. Oleg, just because a java class imports a package does not mean its API "uses" that package. See comment 0 for more information on what it means for API to use another package. The OSGi R4 specification also gives more details. I think you have a point about usability but can we not work around that by flagging the manifest as stale? But I will let Wassim deal with the workflows. Other options are to generate the uses directives in the build process. We could look into generating the uses clauses at bundle install time in the runtime but this would not really comply with the OSGi specification. BTW, I am not in favor of pushing this on anyone at this point in 3.3. I think it is much to late to recommend developers update all their manifests to include uses clauses at this point. There are too many questions left at this point. I am not sure if it is fine to limit "uses" directive to the classes present in the signatures of APIs.
Consider the following scenario:
Bundle A has an API method that declared as:
java.lang.Object #getFoo().
It returns java.lang.Object, but has a Javadoc saying that it really returns Class1 or Class2 (in this case, Class1 and Class2 are coming from a bundle VeryBasicBundle).
Plugin developer makes a Bundle B uses an API from the Bundle A and does:
if (#getFoo() instanceof Class1) {
...
}
In this case to me it looks like even as Class1 is not present in the API, OSGi still needs to both BundleA and BundleB to wire to the same VeryBasicBundle.
Is this correct?
(In reply to comment #15) These are semantics not expressed in the API signature and no amount of runtime or tooling can help with that. This is clearly a pathological case where the developer must hand code the uses clause. BJ, what is "pathological" with that example? I intentionally simplified it so that it was easy to read. However, even in the simplified form it is not uncommon. Think adaptors. Think listeners. Your answer highlights another point. OK, let's say I have some "pathological" code that needed "usage" directives to be adjusted. Now what? Next time me (or somebody else) runs an automated tools it overwrites my hand-coded "uses" directives? (In reply to comment #17) The pathological part is that the API says it returns Object but the javadoc says it returns some other type. If the user of the API is then expected to cast the value to some other type, this is not captured in the API signature and automated tooling cannot handle this. You would then need some other tooling help like an annotation or doclet tag which described the actual return types vs. the one specified in the API signature. Then the tools could take this additional metadata into account to calculate the proper uses clause. (If the user is not expected to cast the object to some other type, then that type does not need to be in a uses clause since the code does not directly reference that class.) Yes, the tooling need to handle the use hand coding the uses clauses which means it is not always possible to completely calculate the uses clauses (without an annotation or doclet tag as I described above). I don't think this is any different from automatically calculating the import-package statement. If the code does a Class.forName on some class, the programmer may need to manually add a package to the import-package statement. A trivial case where downcasting is necessary: a method that returns a Collection. (Rememeber that runtime is supposed to run on Java 1.4.2 so no new nice Java5 features.) In this case if you need an element of a Collection, you have to downcast. Let's say you getting a List from BundleA and elements of the list are of type Class1 which is in some other BundleBasic. The automated tool can catch this is you create "uses" for all classes it is using, not just for what's in the API signatures. My point here is that there are some really simple scenarios which looks like they need "uses" directives even if the used classes are not present in the API signatures. This is why developers need to craft the uses clauses and it cannot be left to some automatic generation tool all the time. We can do markup etc to help in the automation and checking. In the end though there will be cases where developers have to provide input. Hmmm... How would developer know when "uses" statement has to be adjusted? Even if I only modify "implementation details" due to the ["pathological" :-)] patterns shown in the comment 15 and comment 19, I need to check if my modifications ever trickle into APIs. And it will be quite non-trivial to figure that out - that's for a person who understands the reasons behind the "uses" clause. (Note to myself: money-making opportunity. Consulting company "Solving your "uses" problem in 7 days or your money back" :-).) Oleg, your points are valid. But to me this whole topic about "pathological" API is exposing a real deficiency in how Eclipse (and to a greater extend Java) defines API in a pre 1.5 JRE. Someone has to write the javadoc to make it clear to clients what the pathological case is because it is not clear from return types or the param types. Is the "uses" clause harder to maintain than the language you have to remember to maintain in your javadoc? Any time you change your pathological (I just do not have a better term ;-) cases then you have to remember to update your javadoc. It seems like we have been playing pretty loose in our API definitions WRT the pathological case. If we want tooling to automatically generate uses clauses for these cases then we need an annotation or doclet tag like BJ suggests in comment 18. But unfortunately developers still must remember to update the annotation or doclet tag anytime they make a change to the API. One thing I'd like to point out is that downcasting is used often. Every time results returned by another bundle are downcasted, you'll have a problem with "uses". (As long as you limit calculation of "uses" to APIs.) Moreover, typically, the person doing downcasting will be a different person from the person who generated "uses" statement. Now consider if 3 bundles are involved. Bundle1 provides some classes used by the Bundle2. Bundle2 provides some classes to be used by Bundle3. The person writing Bundle2 might not even know what are the actual object types provided by the Bundle1 so he *can't* provide a correct "uses" statement. Bottom line is that unless you are going to outlaw downcasting in Eclipse world, "uses" statements should be calculated based on all the contents of the bundle, not just APIs. (And I have a funny feeling that similar problem will be in up-casting but that is a bit too complicated to figure out at this early morning hour :-).) Oleg, please provide actual bundles that demostrate you usecase. I think code speaks louder than words in this case. I'm having a hard time following the scenario. I am re-closing the bug report because it tracks the completion of this feature. Oleg, if you would like enhancements to the existing function or you find bugs in the current implementation, please open a separate bug report and cc the usual suspects. Thanks. |