Bugzilla – Full Text Bug Listing
| Summary: | Reconciler inconsistent with small sampling | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Equinox | Reporter: | DJ Houghton <dj.houghton> | ||||||||||
| Component: | p2 | Assignee: | DJ Houghton <dj.houghton> | ||||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||||
| Severity: | normal | ||||||||||||
| Priority: | P3 | CC: | john.arthorne, leberre, pascal, tjwatson | ||||||||||
| Version: | 3.8.0 Juno | ||||||||||||
| Target Milestone: | Juno M5 | ||||||||||||
| Hardware: | PC | ||||||||||||
| OS: | Mac OS X - Carbon (unsup.) | ||||||||||||
| Whiteboard: | |||||||||||||
| Bug Depends on: | |||||||||||||
| Bug Blocks: | 363964 | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
DJ Houghton
Do you mean that the solution found by the solver changes at each run or that the plan computed is not installable? It changes with each run. Hum, this is really strange, since the solver is deterministic provided that the input does not change. We sort the IUs before sending them to the resolver, to make sure that the solver answers deterministically. The timeout being based on conflict, not time, the result should also always be the same for a given timeout value. I will take a close look at that problem. Actually hold one for a bit. I've examined the code changes between the version I've been running and the latest, and the fix for Bug 323340 was not applied so I am testing that now. So this may not be a problem. ok. If the IUs are not sorted in the objective function, then the solutions may indeed be different across launches. It is still failing for me in my setup, but unfortunately I can't reproduce it in a stand alone test case yet. My setup is roughly a 3.6.2 with the latest SAT JARs (2.3.0.v20110329) and a patched director bundle with fixes for bug 323340 and bug 336967. I'm not sure what other changes would effect this. I'll investigate further and work on the test case. Then there are still some use cases for which the order of the IUs is not guaranteed in the objective function :( All the IUs to install are optional, right? How many IUs are you considering? I've got about 1650 in the profile and am considering 25 for addition. All (including those already in the profile) are optional. The 25 have dependencies between themselves, and simple ones to the others. (each bundle requires org.eclipse.core.runtime and org.eclipse.ui) I'm still working on a test case. I'll post something tomorrow if I can get one. Here is the potential source of inconsistency:
BigInteger optionalWeight = maxWeight.negate();
long countOptional = 1;
List<IInstallableUnit> requestedPatches = new ArrayList<IInstallableUnit>();
Collection<IRequirement> reqs = metaIu.getRequirements();
for (IRequirement req : reqs) {
if (req.getMin() > 0 || !req.isGreedy())
continue;
IQueryResult<IInstallableUnit> matches = picker.query(QueryUtil.createMatchQuery(req.getMatches()), null);
for (Iterator<IInstallableUnit> iterator = matches.iterator(); iterator.hasNext();) {
IInstallableUnit match = iterator.next();
if (match instanceof IInstallableUnitPatch) {
requestedPatches.add(match);
countOptional = countOptional + 1;
} else {
weightedObjects.add(WeightedObject.newWO(match, optionalWeight));
}
}
}
Is metaIU.getRequirements() deterministic?
If no, we need to sort the IUs there before the loop.
Is picker.query() deterministic?
If no, here again we have to sort the IUs.
Ok, I think I've got a test case. I've released it to master so it is running against all the latest code. I've tried to set it up as close to the reconciler/dropins case as I could. Here is the setup of the IUs:
- there are 5 plug-ins: PluginA, PluginB, PluginC, PluginD, PluginE
- there are 25 plug-ins including all versions
- all versions of all plug-ins have a dependency on org.eclipse.core.runtime and org.eclipse.ui
- version 1.0.0 of all plug-ins have no dependencies on each other
- starting with version 1.1.0, the following dependency tree exists:
A requires B
B requires C
B requires D
C requires D
C requires E
- the version requirement is open-ended so it should match to the highest available
- here are the 25 plug-ins and versions
PluginA 1.0.0, 1.1.0, 1.1.1
PluginB 1.0.0, 1.1.0, 1.1.1, 1.1.2
PluginC 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3
PluginD 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4
PluginE 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5
The test case does the following:
- read the repo containing the 25 IUs to install
- remove the ones which are already installed in the profile
- create a change request with the remaining IUs, including setting up the optional flag
- get the provisioning plan
- validate the highest version of all bundles is either in the plan as an addition, or already in the profile (and not being removed)
There are 3 profile versions in the data folder.
1). 1320249157454.profile.gz - this contains non of the new bundles
2). 1320261982041.profile.gz - this contains the highest versions of all the new bundles
3). 1320270552610.profile.gz - this contains all the bundles, but not the highest versions
Running against profile (1) is successful. The other two fail for me.
I've added a bit of debug info to the console as well. Hopefully I haven't screwed something up in the test case and this can prove helpful.
Thanks for your help, Daniel.
Also Daniel/Pascal, I was going to play around with the new #isOptimal API but wasn't sure where to hook it into the p2 code as there are a couple implementations in sat4j. Any hints? DJ, What is the name of the test case? You might want to use isOptimal() in the method backToIUs is Projector. Oops...one pretty important piece of information I forgot to include. :) The test is called Bug362692. FYI, the optimization function for the first test is constituted of IUs with all the same weight:
-797561089308136884870966149119999996561
Interestingly, the first request (test pass), the solution returned is not optimal, while in the second (that fails), the solution is optimal.
See output given by backToIUs() modified that way:
private void backToIU() {
solution = new ArrayList<IInstallableUnit>();
IVec<Object> sat4jSolution = dependencyHelper.getSolution();
System.out.println("Solution cost:" + dependencyHelper.getSolutionCost());
System.out.println("Solution optimal?" + solver.isOptimal());
for (Iterator<Object> iter = sat4jSolution.iterator(); iter.hasNext();) {
Already installed: PluginB 1.1.2
Already installed: PluginA 1.1.1
Already installed: PluginC 1.1.2
Already installed: PluginD 1.1.4
Already installed: PluginE 1.1.5
Solution cost:-661153478227909469556221859715874815989036637
Solution optimal?false
Solution cost:-1372492797875269911704859442
Solution optimal?true
Plan: [R]PluginB 1.1.2 --> [R]PluginB 1.1.1
Plan: [R]PluginC 1.1.2 --> [R]PluginC 1.1.3
Plan: [R]PluginD 1.1.4 --> [R]PluginD 1.1.3
Plan: [R]PluginE 1.1.5 --> [R]PluginE 1.1.4
org.eclipse.core.runtime.CoreException: PluginD 1.1.4 is in the profile but is being removed.
at org.eclipse.equinox.p2.tests.AbstractProvisioningTest.assertOK(AbstractProvisioningTest.java:123)
at org.eclipse.equinox.p2.tests.planner.Bug362692.validate(Bug362692.java:126)
From the solver point of view, the solution looks fine. There are much less constraints in the second test than in the first one: Dependencies in first test: PluginA 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.1.0 -> PluginB 1.1.0 OR PluginB 1.1.1 OR PluginB 1.1.2 ; PluginA 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.1.0 -> PluginB 1.1.0 OR PluginB 1.1.1 OR PluginB 1.1.2 ; PluginA 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.1.1 -> PluginB 1.1.0 OR PluginB 1.1.1 OR PluginB 1.1.2 ; PluginB 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.0 -> PluginC 1.1.0 OR PluginC 1.1.1 OR PluginC 1.1.3 OR PluginC 1.1.2 ; PluginB 1.1.0 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginB 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.0 -> PluginC 1.1.0 OR PluginC 1.1.1 OR PluginC 1.1.3 OR PluginC 1.1.2 ; PluginB 1.1.0 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginB 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.1 -> PluginC 1.1.0 OR PluginC 1.1.1 OR PluginC 1.1.3 OR PluginC 1.1.2 ; PluginB 1.1.1 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginB 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.1 -> PluginC 1.1.0 OR PluginC 1.1.1 OR PluginC 1.1.3 OR PluginC 1.1.2 ; PluginB 1.1.1 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginB 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.2 -> PluginC 1.1.0 OR PluginC 1.1.1 OR PluginC 1.1.3 OR PluginC 1.1.2 ; PluginB 1.1.2 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.0 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.0 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.0 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.0 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.1 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.1 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.1 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.1 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.2 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.2 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.3 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.3 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginC 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.3 -> PluginD 1.1.2 OR PluginD 1.1.3 OR PluginD 1.1.4 OR PluginD 1.1.1 OR PluginD 1.1.0 ; PluginC 1.1.3 -> PluginE 1.1.1 OR PluginE 1.1.0 OR PluginE 1.1.4 OR PluginE 1.1.5 OR PluginE 1.1.2 OR PluginE 1.1.3 ; PluginD 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginD 1.1.4 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.4 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.0.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.0.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.0 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.0 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.3 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.3 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.4 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.4 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.4 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.4 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.5 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.5 -> org.eclipse.core.runtime 3.6.0.v20100505 ; Dependencies in second test: PluginA 1.1.1 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginA 1.1.1 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginA 1.1.1 -> PluginB 1.1.2 ; PluginB 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginB 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginB 1.1.2 -> PluginC 1.1.2 ; PluginB 1.1.2 -> PluginD 1.1.4 ; PluginC 1.1.2 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginC 1.1.2 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginC 1.1.2 -> PluginD 1.1.4 ; PluginC 1.1.2 -> PluginE 1.1.5 ; PluginD 1.1.4 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginD 1.1.4 -> org.eclipse.core.runtime 3.6.0.v20100505 ; PluginE 1.1.5 -> org.eclipse.ui 3.6.2.M20110203-1100 ; PluginE 1.1.5 -> org.eclipse.core.runtime 3.6.0.v20100505 ; So does that mean we are encoding things incorrectly in the second test case? The dependencies that you've printed for the first case seem correct (although duplicated?) and should apply to all cases. Is it possible for me to hack the code to see those dependencies you printed out, or is that only in sat4j code? (In reply to comment #17) > Is it possible for me to hack the code to see those dependencies you printed > out, or is that only in sat4j code? Actually never mind, just found the flag. Daniel, I'm seeing an inconsistency depending on which mode I'm running in. If I'm using profile 1320249157454.profile.gz (non of the 25 already in the profile) the test case passes if I run in regular mode but it fails if I run with with DEBUG_ENCODING=true. In this case the plan coming back is Empty, we aren't installing any of the new bundles.
It seems the solver we use is different depending on which mode we are in:
if (DEBUG_ENCODING) {
solver = new UserFriendlyPBStringSolver<Object>();
} else {
solver = SolverFactory.newEclipseP2();
}
I'll experiment a bit further but wanted to give you a heads-up.
DJ, what you observe is normal: when in debug mode, the solver used does not solve anything, it just displays on stdout the constraints on a user friendly way. So the very first test will fail with DEBUG=true. (It should be possible to have both the display and the solver running, let me think about it). What I do to see the second encoding, is to change the debug flag at runtime, after setting a breakpoint in the method encode: the second call is the one for the second test. Created attachment 206477 [details]
display if a solution is optimal or not
Created attachment 206485 [details]
Better support for debugging (solves and display the constraints on stdout)
This patch provides a better support for debugging the projector.
currently: in debug mode, the solver is replaced by a fake one that displays the constraints on stdout. There is no resolution being done, so the first trial to resolve something fails.
with the patch: a composite design pattern is used to feed both the fake solver and a real one. As such, the resolution process can proceed, while the constraints are displayed on stdout.
Note however that it requires Sat4j 2.3.1 to work.
So the patch should only be applied after installing Sat4j 2.3.1 from sat4j.ow2.org on the workspace.
People willing to see the trick to display optimal solutions can use partially that patch, so marking the previous huge patch as obsolete.
Created attachment 206877 [details] Sample data Here is the sample data which will reproduce the problem. I have taken both 3.6.2 and the latest Juno I-build and seen the problem. Drop the extracted zip into the drop-ins folder and start with -clean. On the first startup, all the highest versions are installed. On the second startup some of the bundles are reverted to lower versions. I believe this has to do with a combination of 1). the way the dependencies are set up in the sample data set, 2). the way we encode them, and 3). the way the reconciler prepares the ProfileChangeRequest. 1). See comment 10. Note that both B and C require D. 2). Projector#createOptimizationFunction does the encoding and gives priority to things which are new. (and also takes into account things already installed as well as the transitive closure of all incoming bundles) 3). Currently we don't try to re-install things in the drop-ins which are already in the profile. It is believed that if we get the reconciler to re-install everything in the drop-ins all the time then this problem will go away. I will test this theory but based on the number of products which depend on the reconciler as a means of installing, I'm not convinced this is a good change to make in case things go wrong. At a minimum I am guessing that installs from the drop-ins would take longer and we would also hit more "resolver conflict timeout" issues. Created attachment 206883 [details]
Sample data 2
Here is a smaller data set which shows the problem. Same instructions as before, just extract to the dropins folder. Bundle and dependencies are as follows:
B v1, v2
C v1, v2
D v1, v2
Both B and C depend on D [1.0.0, any)
Based on the small data set provided in comment 24 and the fact that this problem can be reproduced in the Eclipse Platform (and not only in a large product), we need to re-visit the logic in the reconciler. Currently (as it has always been) the reconciler reads all the bundles available in all the drop-ins repositories. (dropins, plugins, links, extra sites in the platform.xml, etc) Then it removes from that list, all bundles which are already installed in the profile. After discussions with Pascal, Daniel and John, we've determined this is not the correct behaviour and all bundles in the dropins need to be reconsidered when doing an install. This way all dropins (optional) bundles are given the same weight when determining a solution. If anything, I could imagine a slightly higher weighting for things already installed, to minimize perturbation. It seems the current encoding will maximize perturbation because things that are already installed are not considered. (In reply to comment #26) > If anything, I could imagine a slightly higher weighting for things already > installed, to minimize perturbation. It seems the current encoding will > maximize perturbation because things that are already installed are not > considered. Well, we need to be very careful about that: we cannot change the weights without taking into account the overall problem to avoid compensation effects. If we simply decide to avoid removing from the request the IUs already installed, we should keep the installed packages if there are no new versions available while updating to newer ones if available. This is the default behavior for non optional packages. Note that we have a simpler implementation of optional dependencies in p2cudf that could be integrated in p2 for Juno. The main advantage over the current implementation is to have positive optional weights (i.e. a penalty instead of a reward). That could prevent some compensation effects. I've released a code change to the reconciler in master. It is as we discussed, when we reconcile we need to reconsider everything which was found in the drop-ins. http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=acf42bc70425df51c80f46fbb5d2a0b83d37dd72 Re-opening. As I'm working on some tests I see some behaviour w.r.t. the way we are setting the properties in the reconciler so I need to confirm all is well. Ok, fixed for real this time. And all the tests pass too! The problem was on the first run we would install all the highest versions of the bundles but then when we ran again we included them for consideration when installing, but weren't setting the properties so they were becoming "strict" instead of "optional". The tests were failing because we tried to remove them but they remained in the bundles.info. http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=34a2a34ac248f397e49927cc70c8dbe479c6c8a1 |