Community
Participate
Working Groups
Created attachment 148103 [details] Team Project Set I20090922-0800 The PDE Classpath container should consider required Execution Environments when resolving dependent plug-ins. Example: I have the following plug-ins in my workspace (among others), see attached psf: - org.junit v3_8_2 - org.junit JUnit4_incubator_bug153429 (project name "org.junit-4.5") - org.eclipse.jdt.debug.tests HEAD Now, the "Plug-in Dependencies" classpath container of org.eclipse.jdt.debug.tests contains org.junit-4.5, which is wrong since org.eclipse.jdt.debug.tests compiles to 1.4 class files, but the referenced org.junit-4.5 contains 1.5 class files. => org.eclipse.jdt.debug.tests cannot run with a 1.4 VM due to the class files from org.junit-4.5. org.eclipse.jdt.debug.tests's manifest.mf does not yet contain a "Bundle-RequiredExecutionEnvironment: J2SE-1.4" entry, but even when I add that, the PDE classpath still contains org.junit-4.5. Expected: The org.junit v3_8_2 with EE J2SE-1.3.
A consequence of this is that org.eclipse.jdt.internal.junit.launcher.TestKindRegistry.getContainerTestKindId(IJavaElement) wrongly resolves to the JUnit4 launcher when I create a new JUnit Plug-in Test launch configuration (since JUnit4 classes are on the classpath). Note: When I actually launch a this with a PDE launch configuration and a 1.4 VM, then p2 does not resolve org.junit-4.5 but correctly resolves org.junit (v3_8_2).
PDE uses a single resolution model that handles all versions of known Execution Environments. This is so that we can compile multiple projects in the workspace that use different Execution Environments but still use the same resolution model. In order to keep org.eclipse.jdt.debug.tests from resolving in the case above we would need to setup a separate resolution model for each execution environment as specified by each bundle in the workspace an target. Then we would only add bundles to each resolution model which has an execution environment which satisfies the required EE of the bundle. I believe this would vastly complicate the PDE model and would impact performance. Another option is to have PDE check the EE of each bundle added to the class path container and make sure its EE is "compatible" with the bundle project. If not we could have some kind of error/warning I guess.
Quickly tried to reproduce, bug after importing the psf file the debug test plug-in has org.junit in the classpath container (not org.junit-4.5). Tried cleaning and building, still no change. I wonder if the order the plug-ins are imported/built could be affecting it. Will try some more.
Sorry, I missed that org.eclipse.test.performance also needs to be checked out. > Quickly tried to reproduce, bug after importing the psf file the debug test > plug-in has org.junit in the classpath container (not org.junit-4.5). Tried > cleaning and building, still no change. The PDE container does not seem to refresh on clean. It only refreshes when it's forced to because a required plug-in got closed. The bug is that it doesn't refresh when a plug-in is added/opened. When you close both JUnit plug-ins, you choose between having org.junit and org.junit-4.5 in the container: Whichever plug-in is opened first lands in the container and stays.
(In reply to comment #2) Hmm, sounds complicated. My main intent was actually not to get the EE story completely straight, but to make sure old test projects continue to work as before when we add org.junit version 4.5 to the build. The example of the org.eclipse.jdt.debug.tests plug-in shows that we need even more than the EE checks to accomplish that (the test plug-in does not specify an EE at all). We should probably just add a special tweak in the PDE container so that it always takes the latest org.junit < 4.0.0 when the Require-Bundle does not specify a version. Otherwise we would also have to start "guessing" the EE for plug-ins that don't declare it, e.g. based on compiler settings and by looking at the class file version.
I think the correct fix may be to have test plug-ins update their dependencies. An unspecified required version is now ambiguous, so PDE does not know what to do... Generally, this only effects test plug-ins, so I'm not too concerned about the compatibility issues.
> The PDE Classpath container should consider required Execution Environments > when resolving dependent plug-ins. Tom correct me if I'm wrong, but this wouldn't model what actually happens at runtime. At runtime, the framework doesn't consider the declared EE of dependencies when choosing which bundle to resolve references to. If you are running with a Java 5 VM, then it will happily wire jdt.debug.tests to Junit4 regardless of the EE of the jdt.debug.tests bundle. However, if the actual runtime JRE was 1.4 then junit4 would not be available and it would correctly resolve to Junit3. Although from comments it sounds like there's a bug in the PDE container unrelated to this general request.
(In reply to comment #7) > > The PDE Classpath container should consider required Execution Environments > > when resolving dependent plug-ins. > > Tom correct me if I'm wrong, but this wouldn't model what actually happens at > runtime. At runtime, the framework doesn't consider the declared EE of > dependencies when choosing which bundle to resolve references to. If you are > running with a Java 5 VM, then it will happily wire jdt.debug.tests to Junit4 > regardless of the EE of the jdt.debug.tests bundle. However, if the actual > runtime JRE was 1.4 then junit4 would not be available and it would correctly > resolve to Junit3. > > Although from comments it sounds like there's a bug in the PDE container > unrelated to this general request. John is correct. If the tests cannot run with junit 4 then they should put an upper bound on their version range for junit. Otherwise they could get wired to junit 4 if the running EE is appropriate for junit 4. I agree with Darin's comment 6.
>Generally, this only effects test plug-ins, Right, and therefore they are often not setup as clean as production bundles are and hence don't include a version range, especially when it comes to JUnit where the next breaking version is in a separate bundle so far. Breaking this means breaking many people (I suspect most tests out there are still 3.x ones) and put work on them. Not good! I see that the initially suggested path is not what we want but we should try to find a solution that doesn't hurt people. The tweak from comment 5 looks like a feasible workaround to me.
(In reply to comment #6) > An unspecified required version is now ambiguous, so PDE does not know what to > do... Actually, it should always resolve to the *least* available version, since that's the API the depending plug-in should be compiling against. That would also resolve our issues without any further tweaks.
(In reply to comment #10) > (In reply to comment #6) > > An unspecified required version is now ambiguous, so PDE does not know what to > > do... > > Actually, it should always resolve to the *least* available version, since > that's the API the depending plug-in should be compiling against. That would > also resolve our issues without any further tweaks. PDE uses the Equinox runtime resolver to select the version to compile against. There is a mismatch between the desired version selection at runtime and at build time. At build time we should probably be resolving against the lowest possible provider that matches our version range. At runtime we prefer the highest version for the provider. If we need to go in this direction then PDE will probably need some kind of option to the resolver to reverse the version selection policy. But as John points out in comment 7, if the platform is launched with multiple versions of junit then the runtime will select the highest possible provider that matches the version range. As an aside: OSGi is rethinking what the definition of open ended version ranges mean (i.e. version=1.0). Right now this means 1.0 to infinity. They are considering if it would make more sense to interpret that as version=[1.0,2.0). I personally think that is ripe with disaster but it would have helped in this case.
(In reply to comment #10) > Actually, it should always resolve to the *least* available version, since > that's the API the depending plug-in should be compiling against. That would > also resolve our issues without any further tweaks. The goal of PDE has been to ensure high fidelity between dev and runtime classpaths. This type of change would be unexpected for people and I'm against it. The correct solution as Darin highlighted is to have people update their versions properly in their test plug-ins. I think this is a fine solution if we update our README and migration notes for Eclipse 3.6... since it's test plug-ins and not something people deploy to live applications, I think this is the better route. Maybe people will actually learn about OSGi versioning a bit in the process.
Q: what happens to those that required JUnit 4? I assume they will be broken now, since the bundle name has changed?
(In reply to comment #12) > The goal of PDE has been to ensure high fidelity between dev and runtime > classpaths. This type of change would be unexpected for people and I'm against > it. I agree that HiFi is good. But there *is* a difference between development and run time when there are multiple versions of the same plug-in available. If your version range says you can run with org.junit 3.8 then you should also compile against that version. Silently using the latest version allows code to compile that will never work with the lower version. Isn't using the latest to compile counter the efforts to detect API problems early? (In reply to comment #8) > If the tests cannot run with junit 4 then they should put an > upper bound on their version range for junit. Otherwise they could get wired > to junit 4 if the running EE is appropriate for junit 4. Old tests should actually run fine with JUnit 4, because JUnit 4 is officially compatible with JUnit 3. But the JUnit launcher and the developer should be aware that these tests could be run with only JUnit 3 but not JUnit 4 available. (In reply to comment #13) > Q: what happens to those that required JUnit 4? I assume they will be broken > now, since the bundle name has changed? We will keep the org.junit4 bundle as a wrapper that re-exports org.junit version 4.x, so old clients will be fine.
Discussed with Markus & Tom. There resulting problem is that the JUnit launcher is potentially broken: (1). The JUnit launch configuration delegate currently looks at the build path of the project to see whether JUnit 4 is available. If it is, we default to the JUnit 4 test runner. But if the test project is in the end launched with a 1.4 VM, the JUnit4 bundle is not available at run time and the test runner fails. I could probably look at the compiler compliance etc. by hand to resolve this issue. But when I thought more about it, I realized that there's problem 2: There is also a general problem of having a newer bundle on the build path: (2). When the build path contains the latest available version, then users can write code that only compiles with the latest API, but when you run it with a lower version (for whatever reason), then it will not work. The compiler could detect this, but only if the lowest version is on the build path. In general, it makes sense for the IDE to bind the build path to the lowest available bundle matching required version ranges, to avoid accessing new API. Note: Fixing (2) will fix (1). I suggest that PDE (IDE and build) adopts a policy to bind to the lowest available bundle matching the required version range. Currently OSGi matches to the highest available version, but has API to control this via: org.eclipse.osgi.service.resolver.Resolver.setSelectionPolicy(Comparator) This is a fundamental change. We should have general agreement on this before proceeding. In practive, there are not usually multiple versions of bundles available in target platforms, so I suspect the impact will be minimal (and beneficial).
We should make this configurable potentially.
(In reply to comment #15) > In general, it makes sense for the IDE to bind the build path to the lowest > available bundle matching required version ranges, to avoid accessing new API. > I suggest that PDE (IDE and build) adopts a policy to bind to the lowest > available bundle matching the required version range. Currently OSGi matches to > the highest available version, but has API to control this via: > > org.eclipse.osgi.service.resolver.Resolver.setSelectionPolicy(Comparator) > > This is a fundamental change. We should have general agreement on this before > proceeding. In practive, there are not usually multiple versions of bundles > available in target platforms, so I suspect the impact will be minimal (and > beneficial). I think merely having this feature will make multiple versions more common. Consider that we want to compile against the lower versions to be api clean. So someone places for example 3.5.0 binaries of all their bundles in their target, and also has the latest 3.6 source to compile. PDE/Build currently compiles whatever is resolved, we need to prefer the higher versioned source over the lower version binary, or nothing would get compiled. However, each bundle wants to be compiled against the lowest binaries of its dependencies, not the newest source. I believe this effectively requires us to re-resolve the state with a modified policy for each bundle being compiled. This likely has performance implications for large targets. Also we probably want to restrict the preferring of lower version to things that differ by more than the qualifier. If only the qualifier is different, we should take the source.
(In reply to comment #17) > I think merely having this feature will make multiple versions more common. > > Consider that we want to compile against the lower versions to be api clean. > So someone places for example 3.5.0 binaries of all their bundles in their > target, and also has the latest 3.6 source to compile. > > PDE/Build currently compiles whatever is resolved, we need to prefer the higher > versioned source over the lower version binary, or nothing would get compiled. > However, each bundle wants to be compiled against the lowest binaries of its > dependencies, not the newest source. I'm not sure I understand. How does PDE determine what to compile? If something in the target is binary then you don't need to compile it. Don't you know what is in source form vs. binary form and don't you try to compile all source form bundles? > > I believe this effectively requires us to re-resolve the state with a modified > policy for each bundle being compiled. This likely has performance > implications for large targets. Is the concern here singletons? PDE-UI works around the singleton issue by using a resolver that is in dev mode. We had discussed having PDE-Build use a dev mode resolver but I cannot remember if we ever made that change. This should allow multiple versions of a singleton bundle to exist in the state at the same time. > > Also we probably want to restrict the preferring of lower version to things > that differ by more than the qualifier. If only the qualifier is different, we > should take the source. I'm not sure this is a concern. Is this for cases where you have multiple I-Builds in your target? If so I think that is a degenerate case. Our selection policy could be smart enough to sort the provider versions this way if we think it is important.
(In reply to comment #18) > I'm not sure I understand. How does PDE determine what to compile? PDE/Build makes no effort to ensure all sources are compiled. All bundles binary and source are added to the state and resolved. We then traverse the features and get the resolved bundles, if it happens to be source, we compile it. If the source bundle did not get resolved and the binary did, then the source won't be compiled. PDE/Build has bug 177629 for this idea of compiling against the lower bounds. Pascal and I have talked about this a few times before, unfortunately we never recorded our thoughts in the bug. I think the proper fix for _this_ bug is fixing the version ranges as in comment #6. I believe the idea of compiling against lower versions for api correctness is a much harder problem and discussion on that should perhaps move to bug 177629. I have added more comments there.
(In reply to comment #19) > I think the proper fix for _this_ bug is fixing the version ranges as in > comment #6. I think that is not going to work because it forces these tests to never work with junit 4. Perhaps that is acceptable for a pragmatic solution, but I think we need to have a better story here. We can continue that discussion in bug 177629.
Current sentiment is to have PDE do nothing here and have the JUnit launcher determine that JUnit 4 is on the runtime classpath and do the right thing.
Since it doesn't look like bug 177629 can be tackled soon, could we put some special code into the PDE Classpath Container just for org.junit? Concretely, when the source compliance is <1.5 and the lower bound for org.junit is >= 3.0.0, then put the latest org.junit 3.x on the class path, even if the Require-Bundle clause for org.junit is open-ended. Currently, e.g. Organize Imports offers org.junit.Assert in such a plug-in, although that cannot work with Java <1.5. We once discussed adding a special access rule in this case, but that's not helpful in practice because most test projects have the compiler warning for access rules disabled (since tests often refer to internal types of their target bundle).
>We once discussed adding a special access rule in this case, but that's not >helpful in practice because most test projects have the compiler warning for >access rules disabled We would mark them as 'forbidden' which should work for most cases.
But I agree that fix it along Markus's suggestion would even be better.
*** Bug 434499 has been marked as a duplicate of this bug. ***
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
