Bugzilla – Full Text Bug Listing
| Summary: | Provide CDO builds | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Modeling] EMF | Reporter: | Eike Stepper <stepper> | ||||
| Component: | cdo.core | Assignee: | Eike Stepper <stepper> | ||||
| Status: | CLOSED FIXED | QA Contact: | |||||
| Severity: | enhancement | ||||||
| Priority: | P2 | CC: | Ed.Merks, nboldt | ||||
| Version: | 1.0 | ||||||
| Target Milestone: | --- | ||||||
| Hardware: | PC | ||||||
| OS: | Windows XP | ||||||
| Whiteboard: | |||||||
| Bug Depends on: | 194409, 201839 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
|
Description
Eike Stepper
Eike: I notice you have .class and .jar files in CVS in your org.eclipse.emf.cdo.weaver plugin. This is generally taboo, esp. without going through IPzilla or reusing Orbit stuff. I don't see license details in that plugin either, nor in emf.cdo-feature. Are these classfiles approved, or are they another example of build-time-only stuff? If they're not approved (or they can be downloaded elsewhere), they should be removed from CVS. I also found two copies of EcoreEList.class in both emf.cdo.weaver and emf.cdo.weaver.ajdt. Why are they there? The story is rather complicated. I'll try to make it short and start with the goal. Oh, no the beginning is so funny that I must mention it again ;-) Everything started with my EMF feature request in http://dev.eclipse.org/newslists/news.eclipse.tools.emf/msg05660.html Ed gave his legendary answer: "I don't like this kind of design as a general pattern because it allows and encourages the communal fountain to be poisoned at its source. [...] I see the factory that creates objects like a common water fountain from which everyone gets their water. If we allow someone to inject cola flavor into the fountain because they really like that flavor and they really badly need the water to have a certain level of sugar, that's great for them, but the thirsty person with diabetes will be very unhappy that the water is poisoned. So adding the cola flavor to the water is something that you should do after you've take the water out of the fountain, not something you do by changing the fountain." Just a moment, I must pour the tears out of my eyes, ROTFL ;-) 3 years and 2 CDO versions later I decided to provide my own slightly modified copy of the communal fountain for the majority of users without diabetes. In order to keep my fountain up to date with the communal one I decided not to modify the fountain sources but rather use AspectJ to always add the cola flavor to the latest communal binaries.</rotfl> Technically that means there are some compiled aspects (classes) in the weaver plugin. When a user installs the weaver plugin his installation gets analyzed at startup time. The weaver plugin detects generated models that are not converted (including Ecore itself) and asks the user if he wants to apply the aspects to them ("Are you sure you don't have diabetes?"). Of course there's a trick called load time weaving which would be completely transparent for the user but I conciously decided that I want to give the user explicit control over the weaving decision. Also, AFAIK, load time weaving requires the user to add some external config switches which I can't control programmatically. So I puzzled another trick: The weaver plugin unpacks the installed plugin, weaves in my aspects, jars the result and writes the new jar with a higher version number ("-CDO" appended) to the original folder. The next restart will load the weaved model plugin and provide the cola flavor. Restoring the diabetes aware state means simply removing the generated model plugin. This also means that other developers who like to make use of my hooks can't do it because the new capabilities are not visible to them when they install only original Ecore. The checked-in binaries in the weaver plugin are my compiled aspects plus the AspectJ runtime+weaver jars needed to get the aspects into the models at runtime. Then there's the weaver.ajdt project (it's only a plugin so that I can use PDE classpath configuration). Its only task is to compile the aspects within my local workspace and copy the resulting binaries into the weaver plugin. This could be part of the headless build as well but first I had to play with it to learn and the headless build wouldn't make it easier. The weaver.ajdt plugin is not expected to be part of any distribution! In order to be able to compile the aspects I need the classes they are expected to be applied to. That's the reason why I have a copy of the Ecore sources in my weaver.ajdt project. There's one special case: EcoreEList.class can't be modified by AspectJ (in the way I need it) because the methods I need to intercept are not overridden in EcoreEList but only inherited from a class in org.eclipse.emf.common. It's certainly an issue of the AspectJ compiler that it can't add overrides but I can't solve it otherwise. So I maintain a copy of the EcoreEList source, add NOOP overrides and then apply my aspects. Ed, you will remember that I asked about the API stability of EcoreEList? To sum up: - The whole story is quite complicated - But it works like a charm - All binaries (AspectJ jars, modified EcoreEList.class and own apsects) are subject to EPL - org.eclipse.emf.cdo.weaver.ajdt is only needed here at my home (but I need in in CVS of course) Nedd more explanation? Your CDO build [1] is progressing nicely, assuming that you start with Eclipse, EMF, OCL (still required for compiling tests?), and Net4j. It's not 100% there, but the errors [2] look like more inter-feature/inter-plugin issues, not build issues. [1]http://emft.eclipse.org/modeling/emft/cdo/build/ [2]http://emft.eclipse.org/modeling/emft/downloads/?project=cdo&sortBy=date&hlbuild=0#latest Note that as with Net4j you'll need a test.xml file in your test plugin. I don't need OCL at the moment. The build stops with:
/home/www-data/build/modeling/emft/cdo/downloads/drops/0.8.0/N200709210629/org.eclipse.modeling.common.releng/scripts/getDependenciesHelper.xml:43: java.net.MalformedURLException: no protocol: ${oclURL}
I rebuilt with OCL URL specified and that problem is gone.
--> Please remove OCL from my deps!
The new error message is:
/home/www-data/build/modeling/emft/cdo/downloads/drops/0.8.0/N200709210638/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build/scripts/genericTargets.xml:85: Unable to find plug-in: org.eclipse.emf.cdo.ui_0.0.0. Please check the error log for more details.
I don't know what causes this. The map seems ok and the features as well...
The previous error did not happen again (don't know why). Now a tricky one: http://emft.eclipse.org/modeling/emft/cdo/downloads/drops/0.8.0/N200709210652/buildlog.txt I've already answered the question here: http://www.eclipse.org/newsportal/article.php?id=3554&group=eclipse.technology.emft#3554 Summary: We *must* ignore these errors somehow. The only alternative I could imagine of is to apply my aspects (see comment #2) to the checkedout EMF Ecore plugin. I'd prefer not to bother with AspectJ headless weaving if possible! Created attachment 78975 [details] files to tweak to add/remove OCL dependency (In reply to comment #4) > I don't need OCL at the moment. The build stops with: > I rebuilt with OCL URL specified and that problem is gone. > --> Please remove OCL from my deps! (In reply to comment #5) > We *must* ignore these errors somehow. > The only alternative I could imagine of is to apply my aspects (see comment #2) > to the checkedout EMF Ecore plugin. I'd prefer not to bother with AspectJ > headless weaving if possible! Ignore a compiler error? In MDT.OCL, Christian sets per-plugin build.properties values so that he can control warning suppression. Try adding this to the plugins you don't care about compiling correctly: javacFailOnError=false javacFailOnError=false does not make a difference: build fails ;-( In order to get a bit further I comented out the code that causes the compile errors (this can't become permanent of course). Now the build fails with missing getDependencies.xml: http://emft.eclipse.org/modeling/emft/cdo/downloads/drops/0.8.0/N200709221018/buildlog.txt No idea what's that... (In reply to comment #9) > Now the build fails with missing getDependencies.xml: > No idea what's that... That's an artefact of the old EMFT-style build, which I evidently missed when updating you to the Modeling-style build. In builder/tests/customTargets.xml, line 69, I replaced "org.eclipse.emft.common.releng" with "org.eclipse.modeling.common.releng". You should be good to go now. New problem: /home/www-data/build/modeling/emft/cdo/downloads/drops/0.8.0/N200709230422/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build/scripts/genericTargets.xml:85: Unable to find plug-in: org.eclipse.emf.cdo.tests_0.0.0. Please check the error log for more details. --> http://emft.eclipse.org/modeling/emft/cdo/downloads/drops/0.8.0/N200709230422/buildlog.txt Is it related to the version number "_0.0.0"? Reason: o.e.e.cdo.tests has a dependency on o.e.net4j.tests which is not part of the Net4j SDK. I have temporarily removed the dependency and copied the related classes to cdo.tests. I fear we need an additional build dep on CDO ;-( Nick: Could you do that please? New problem: The CDO tests need my derby *fragment* which seems to be missing. Since I can't reference fragments from my manifest.mf I added the jar directly to the .classpath of the JDT project. Nick, any ideas? A general question with respect to testing: The automatic tests seem to be run in Eclipse or OSGi. Is there a possibility to run tests standalone? (In reply to comment #12) > Reason: o.e.e.cdo.tests has a dependency on o.e.net4j.tests which is not part > of the Net4j SDK. > Nick: Could you do that please? If you don't want to have your tests be part of your SDK, we'll just need to add another dependency when building. This is mostly done via customTargets.xml, but will require some lower-level (cross-project) script changes too. (In reply to comment #13) > New problem: > The CDO tests need my derby *fragment* which seems to be missing. Since I can't > reference fragments from my manifest.mf I added the jar directly to the > .classpath of the JDT project. You mean the fragment isn't being bundled into the Net4j SDK? (In reply to comment #14) > A general question with respect to testing: > The automatic tests seem to be run in Eclipse or OSGi. Is there a possibility > to run tests standalone? Yes, they run in a headless Eclipse using AntRunner. If you want to run them w/o Eclipse, that is possible too, as a secondary test process AFTER the normal build-integrated JUnits. Would that work? Or are you saying you want ALL your tests to run w/o Eclipse? > If you don't want to have your tests be part of your SDK, we'll just need to > add another dependency when building. This is mostly done via > customTargets.xml, but will require some lower-level (cross-project) script > changes too. I've looked into customTargets.xml already but didn't figure out how to provide the file and url thing. Yes, please do that ;-) > You mean the fragment isn't being bundled into the Net4j SDK? It is! But the cdo build even does not work if I do it w/o tests. No idea other than it seems to relate to the fragment... > Yes, they run in a headless Eclipse using AntRunner. If you want to run them > w/o Eclipse, that is possible too, as a secondary test process AFTER the normal > build-integrated JUnits. Would that work? Or are you saying you want ALL your > tests to run w/o Eclipse? Currently all tests run standalone here at home but that will certainly change in the future. (In reply to comment #16) > > If you don't want to have your tests be part of your SDK, we'll just need to > > add another dependency when building. This is mostly done via > > customTargets.xml, but will require some lower-level (cross-project) script > > changes too. > I've looked into customTargets.xml already but didn't figure out how to provide > the file and url thing. Yes, please do that ;-) Will do, tomorrow. > > You mean the fragment isn't being bundled into the Net4j SDK? > It is! But the cdo build even does not work if I do it w/o tests. No idea other > than it seems to relate to the fragment... I'm unexperienced w/ fragments, so this is a new area of PDE build goodness. FWIW, the "without tests" checkbox on the build page will not *run* the tests, but will still attempt to *build* them. To disable building them you'll have to edit buildAll.xml, but the build page will show that the build failed because it's expecting 4 zips, not 3. You could replace the part of buildAll.xml that builds the tests with something that creates an empty zip... but that's only a temporary solution, since as a good Eclipse Citizen you should be providing tests for your component (even if no one ever downloads 'em). > > Yes, they run in a headless Eclipse using AntRunner. If you want to run them > > w/o Eclipse, that is possible too, as a secondary test process AFTER the normal > > build-integrated JUnits. Would that work? Or are you saying you want ALL your > > tests to run w/o Eclipse? > Currently all tests run standalone here at home but that will certainly change > in the future. OK, then I won't worry about this one. Marking this bug ASSIGNED since the build basically works now. For the remaining issues I've filed separate bugs: bug #204408 "javacFailOnError=false does not work" bug #204232 "Add or verify branding plugins for Net4+CDO" bug #204409 "Stdout not shown in tests log" bug #204364 "Separate features folder in CVS" bug #204410 "CDO test needs Net4j test dependency" When will you be providing your first I or S build for 0.8.0? Ganymede M2 is this week -- it would be good to start at least publishing I builds now and drop your first stable for M3 (mid-November), if you're not comfortable doing your 0.8.0M2 build this week. I can't close this bug until we know the promote process works, and I would like this off my todo list. ;-) (In reply to comment #19) http://www.eclipse.org/modeling/emft/news/relnotes.php?project=cdo&version=HEAD http://www.eclipse.org/modeling/emft/downloads/?project=cdo Fixed. Move to verified as per bug 206558. Reversioned due to graduation CLOSING |