Community
Participate
Working Groups
For deploying bundles in your Eclipse workspace to the Virgo test environment, the Virgo tools use a stage directory that is configured as a watched repository for Virgo. If you then create a server instance using Virgo tools and add to the server a number of bundles, in the wrong dependency order, the Virgo tools may deploy the same bundle twice. This happens for example if you have three bundles, A, B, C where C depends on B which in turn depends on A: C -> B -> A. Once you add A,B,C to the server via Virgo tools, they get copied to the stage directory that is a watched repository. Now, if by mistake you configure the bundle order in the server editor as A,C,B instead of A,B,C you will get C duplicated. This is because upon startup A will be resolved, installed, started, then C will be resolved installed and started. Now, since C depends on B amd B is contained in the stage directory which is a watched repository, Virgo will implicitly resolve B and activate it as it is a required dependency of C. This is perfectly fine behavior but the problem is that the fact that B is also listed in the startup order after C will make the tools install another B in Virgo. The OSGi console will then show two B, with same symbolic name and version but different bunlde ID. The system should rather silently ignore the second B or just interrupt deployment with an exception. This subtle bug is difficult to track in large applications. It typically does not result in wrong wirings, but it fools extenders. Workaround: fix the order in which bundles should be activated.
There is a mistake in my previous comment. It's B that gets duplicated, but the first time I uncorrectly wrote C: "Now, if by mistake you configure the bundle order in the server editor as A,C,B instead of A,B,C you will get C duplicated." should have been "Now, if by mistake you configure the bundle order in the server editor as A,C,B instead of A,B,C you will get B duplicated."
*** Bug 407855 has been marked as a duplicate of this bug. ***
Thank you for clarification.
I think that a possible solution to this problem could be implemented in org.eclipse.virgo.ide.runtime.internal.core.DefaultServerDeployer.deploy(IModule...) by checking if the same IModule (version and symbolic name) is already deployed on the server before deploying it. I hope to have some time to investigate a bit more over the next week.
That would be great. I'm looking forward to your contribution.
Florian, could you point me to a how-to for properly setting up a workspace for Virgo-Ide development? I tried as follows: - clone the git repository org.eclipse.virgo.ide - File -> Import Existing Maven Projects and select org.eclipse.virgo.ide/pom.xml - switch to the Luna test drive branch (I am on luna) As this did not compile properly, I tried the "SuggestedTargetPlatform.target" but I still experience compile errors related to SWT.
I experience compile errors in the project: "org.eclipse.virgo.ide.ui.tests" related to SWTBot. Are you refering to these errors? I never used SWTBot on my own and didn't have time to research how to compile and run those tests locally.
I added SWTBot to my Eclipse installation and fixed the suggested target platform to contain the missing org.hamcrest.core. Now my workspace is compile clean on branch "luna_testdrive". HTH, florian
Thanks Florian, no, I am not worried about SWTBot. Looks like Eclipse is not properly setting up the "plugin dependencies" container in the project classpath. The SWT bundle is there, but the O.S. specific part is not (in my case, the GTK 64bit implementation as I am using Ubuntu 12.04), so all references to org.eclipse.swt lead to compiler errors. I'll try again with a fresh Eclipse installation.
Got it, the target platform was specifying Mac OS. Sorry, I did not notice it.
That's most probably my fault. Sorry for any inconvenience caused. I will remove this constraint from the target definition(s).
Sorry for this rather long comment, I think I found two ways to address this problem, and would need some advice about which one is the best/preferred. My workspace now compiles properly, although I cannot manage to have a working launch configuration due to many duplicate singleton bundles in the target platform. In any case this is not an issue, I can build and remotely debug. To recap, the source of the problem is that: - when you add bundles to the Virgo runtime, the Virgo IDE creates a stage directory as a watched repository (the launch configuration passes to the runtime a custom repository configuration file created on the fly) and when you perform the publish operation bundles get copied into the stage directory - if you add bundles to the Virgo runtime in the wrong order, some bundles may be installed, resolved and started twice because (i) they get started the first time as a dependency of some other bundle deployed to the Virgo runtime by the IDE (because they are found in a repository directory) and (ii) they get started a second time by the IDE tools because they have been added to the Virgo runtime in the IDE Apparently, when deployed in this way, Virgo does not check that a bundle is already existing (same symbolic name, same version and same install location) and it gets deployed and started twice, which fools OSGi extenders and leads to issues difficult to track. I guess all of this originates from the fact that the stage directory is configured as a bundle repository, so a wrong artefact order may pass unnoticed and lead to this issue. If instead of doing this the Virgo IDE did configure the stage directory as a repository, the problem would not show up, but I bet there are good reasons that I ignore for configuring the stage folder as a repository folder. So my proposal is that I modify the JmxServerDeployCommand so that before deploying an IModule (which represents a bundle deployed on the Virgo Runtime via the IDE) it checks whether the same bundle is already deployed or not. The check can be done via the Virgo server JMX beans in two ways: a. by enquiring via JMX the user region bean for the list of all deployed bundles and iterating over such list to check whether the bundle which is about to be deployed is already present or not b. by trying to locate and enquiry via JMX a bean named after the bundle, because it seems that Virgo creates a JMX bean for each deployed bundle (under the ArtifactModel "folder") The second options seems cleaner and probably more efficient to me, but I would appreciate some advice from the Virgo committers. Note that the IModel instance that the JmxServerDeployCommand receives for each bundle contains the symbolic name, install location and the IProject but not the bundle version. I believe I should use the symbolic name and install location (which is a path within the stage folder) for checking. I don't think that parsing the IProject MANIFEST for obtaining the version is a good idea, because the IProject in the workspace may contain unpublished changes. Now, whichever the JMX enquiry, I can either 1. modify directly JmxServerDeployCommand to perform the enquiry and simply an skip already deployed IModule 2. create a new dedicated JMX command for that purpose which will be used by the deploy command and modify the ServerBehaviour class (if I remember correctly) to provide a new factory method for it, as for all other commands. This seems more congruent with the current implementation, but is also more invasive. So here are my questions: - which is the most appropriate approach for obtaining the information I need from the runtime (a) or (b). If it's the same I will go for (b). - which ever the selected approach (a) and (b), I will need to create a JMX ObjectName to obtain the bean. Are the constants enumerated somewhere? I could not find them. - In which way would you prefer the change to be implemented (1) or (2)? - the above has been verified with Virgo 3.6. I can imagine that this will work also with 3.5 but I never used earlier versions of Virgo. Are the JMX beans avaialable also in 2.x or should I simply perform the check if the server version is >=3.5, also considering the 2.x is pretty old? - are there coding conventions, formatting preferencese etc I should comply with? Thanks.
Given the fact that you are more familiar with this code than I am I'd say it is up to you to decide which option to choose. Please find my 2 cent inline. - which is the most appropriate approach for obtaining the information I need from the runtime (a) or (b). If it's the same I will go for (b). I'd prefer (b) - the option without looping. - which ever the selected approach (a) and (b), I will need to create a JMX ObjectName to obtain the bean. Are the constants enumerated somewhere? I could not find them. Which JMX ObjectName are you refering to? There are some constants in the package: org.osgi.jmx.framework. public interface BundleStateMBean { /** * The Object Name prefix for this mbean. The full object name also contains * the framework name and uuid as properties. */ String OBJECTNAME = JmxConstants.OSGI_CORE + ":type=bundleState,version=1.7"; Is this what you are looking for? - In which way would you prefer the change to be implemented (1) or (2)? As stated above I'd leave it up to you to choose. - the above has been verified with Virgo 3.6. I can imagine that this will work also with 3.5 but I never used earlier versions of Virgo. Are the JMX beans avaialable also in 2.x or should I simply perform the check if the server version is >=3.5, also considering the 2.x is pretty old? Implementing this for 3.5+ only seems pragmatically to me. - are there coding conventions, formatting preferencese etc I should comply with? Yes. Please have a look at our Wiki: https://wiki.eclipse.org/Virgo/Committers/Coding
In relation to JMX, via the Eclipse debugger I found out that to obtain the JMX bean created by Virgo for a deployed bundle named "org.example.bundle" I must perform the following JMX call: ObjecName name = ObjectName.getInstance("org.eclipse.virgo.kernel:type=ArtifactModel,artifacttype=bundle,version=*,region=org.eclipse.virgo.region.user,name=org.example.bundle"); Set<ObjectInstance> objects = jmxConnection.queryMBeans(name); So I was wondering whether some one from the server runtime team could confirm that I can trust the "org.eclipse.virgo.kernel" JMX domain to always contain a bean for an already deployed bundle and if there are constants declared somewhere for the key/value pairs (folders) I must specify in the object name, e.g. "type=ArtifactModel" and "artifacttype=bundle". Thanks. GianMaria.
No, not to my knowledge. A widespread "pattern" in the runtime code is something like: private static final String OBJECT_NAME_PATTERN = "%s:type=Repository,name=%s"; /** * Creates a uniform object name based on the name of the repository * * @param mBeanDomain the domain name to use * @param repositoryName The name of the repository * @return The uniform object name * @throws NullPointerException but this should never happen * @throws MalformedObjectNameException if the generated object name would be badly formed */ public static ObjectName createObjectName(String mBeanDomain, String repositoryName) throws MalformedObjectNameException, NullPointerException { return new ObjectName(String.format(OBJECT_NAME_PATTERN, mBeanDomain, repositoryName)); We might consider introducing something like "org.eclipse.virgo.util.jmx.ObjectNameBuilder". Perhaps in oev.util.jmx? What do you think?
In principle I don't like constant classes for "specification" constants, e.g. "HTTP" "GET" "POST" etc., because those strings are not going to change in the future, and listing them in a constants class often has the side effect of leading to a code base where many other classes refer to the constants class, sometimes to the point that unnecessary dependencies between packages (or even worse bundles) may be introduced. I was asking for constants worried that the JMX internals of the Virgo Server may be changed and the IDE may remain out of sync. However, after more careful thinking, I can imagine that the MBean names used by Virgo are a sort of "SPI" and I believe that they are not subject to frequent changes from on Virgo version to another. Plus, a quick search for ObjectName in the code base reveals that all occurrences are located in the org.eclipse.virgo.ide.runtime.internal.core.command package within the Jmx* classes, so I guess there is very low risk that something is forgotten or left out if a refactoring is needed due to changes in a future runtime version. Given that the Mars release is due this week, I believe it should give more value to end users to have a working Virgo IDE out as soon as possible, so I would suggest to postpone refactorings to a later time, maybe to the SR1.
Created attachment 254598 [details] Work in progress patch for review
I attached a work in progress patch for review. The patch is quite simple: - JmxServerDeployCommand is changed so that before deploying an IModule (bundle) it invokes another command to check whether the same bundle is already deployed or not - JmxServerCheckBundleDeployedCommand is the command used to check whether a given IModule is already deployed or not As I don't think JmxServerCheckBundleDeployedCommand is that reusable, I propose to initially make it a class with default visibility, that will be used by JmxServerDeployCommand only. Therefore I will not to modify IServerRuntimeProvider to add a factory method for creating JmxServerCheckBundleDeployedCommand instances. Comments: - The MBean created by Virgo for each deployed bundle under the ArtifactModel domain contains the bundle symbolic name and version but not the install location. To the contrary, the IModule contains the symbolic name and the IProject and can be used to derive the install location but the version is missing. - To verify that the bundle to be deployed is already deployed or not, I am parsing the MANIFEST.MF of the IModule given its install location, and enquiring the ArtifactModel with that version. Parsing the MANIFEST.MF and calling the JMX bean has no noticeable performance effect on my 3 years old Lenovo thinkpad running Ubuntu 12.04. Still, I'll try to postpone the MANIFEST.MF parsing and execute if and only if the ArtifactModel contains at least one bundle with the given symbolic name. - As already written in other comments, I am parsing the MANIFEST in the install location (stage folder) and not the one in the IProject as the IProject may contain unpublished changes. - There is a farily simple error management: should the check fail in any way, the result of the check will be "bundle not already deployed" and Virgo IDE will therefore behave like it did without this patch. - This patch solves the problem described by this bugzilla entry but has a consequence. If the user workspace contains bundle A and B, where B depends on A but the user configures the artifact order so that B appears before A, when starting B Virgo will also start A. In this case, it will log in the server console that B is installed and started while there will be no log statement about A which will be silently resolved and started as it happens for any other bundle in the bundle repository. So the user may be confused by the fact that the server is started but there is not log entry about bundle A, despite being it added to the server instance. A check via the OSGi console will however reveal that A has been resolved/started. Without this patch, the log line will be there but the container will contain two instances of A. As I cannot control from the IDE the server log, I could only open a popup at the and of the deploy to warn the user about the misconfiguration, but this looks clumsy to me. So I would rather do nothing. Still to do: - apply the correct formatting preferences and coding conventions - use the same approach for the ObjectName constant as used in the runtime and suggested by Florian - use a differnt ObjectName for Virgo 2.x - evaluate the feasibility of postponing MANIFEST.MF parsing - change the source code comment with something more appropriate (I copied the header from another source and it now contains SpringSource as the copyright owner, but what is the correct header now that the project organization has changed?
Created attachment 254640 [details] Work in progress patch for review - version 2
I attached a new revision of the patch Done: - use an ObjectName constant that mimics the suggestion from Florian - postponed MANIFEST.MF parsing Still to do: - apply the correct formatting preferences and coding conventions - change the source code comment with something more appropriate (I copied the header from another source and it now contains SpringSource as the copyright owner, but what is the correct header now that the project organization has changed? - use a different ObjectName for Virgo 2.x, if needed. I added a new constructor the JmxServerDeployCommand which accepts a boolean to enable the "already deployed check" and override the factory method in the Virgo 3.5 provider to use such constructor so that the check is activated.
Created attachment 254667 [details] Work in progress patch for review - version 3
Here is an additional revision of the patch. The main improvement is that now, besides checking that a bundle is already deployed, the code also verifies that it is in ACTIVE state and if not it starts the bundle. In all my tests, implicitly deployed bundles were always ACTIVE, but a colleague of mine pointed out that a bundle in an OSGi repository may remain in resolved state, and that would be inconsistent with the standard behaviour of the Virgo IDE which always starts the bundles it deploys.
Hi, any feedback? Meanwhile I have been using the patch on my workstation without experiencing any side effect.
Setting target milestone to 1.0.1.RELEASE.
Hi Florian, I see you are pretty busy, do you want me to make the patch a pull request on github? Maybe that will save you some time. GianMaria.
No, that's not necessary. Thanks for the offer. I already applied the patch locally for review. I was a bit puzzled by the name "Work in progress...". Do you intend to add something?
I was able to reproduce the issue with latest Virgo IDE: 128 ACTIVE a_1.0.0 129 ACTIVE c_1.0.0 130 ACTIVE b_1.0.0 131 ACTIVE b_1.0.0
With the patch applied I can see that only three bundles are deployed: 125 ACTIVE a_1.0.0 ... 130 ACTIVE c_1.0.0 131 ACTIVE b_1.0.0
Pushed with [fbea140] to master. * Fixed copyright header of new file. * Fixed a typo Thanks for your contribution.
Hi Florian, sorry for not being able to help you out yesterday, I was on leave, but I am glad you were able to reproduce the issue by yourself. With respect to WIP, I was waiting for your review and the correct copyright header, so again thank you for your fix. GianMaria.
Changed Target Milstone to 1.0.2.RELEASE. @GianMaria Thanks for spotting this.
