Bugzilla – Full Text Bug Listing
| Summary: | Provide Net4j builds | ||
|---|---|---|---|
| Product: | [Modeling] EMF | Reporter: | Eike Stepper <stepper> |
| Component: | cdo.net4j | Assignee: | Eike Stepper <stepper> |
| Status: | CLOSED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P1 | CC: | Ed.Merks, nboldt |
| Version: | 1.0 | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Whiteboard: | |||
| Bug Depends on: | 194409, 201839 | ||
| Bug Blocks: | |||
|
Description
Eike Stepper
Eike: I've hit a problem running your new 0.8.0 build, with Eclipse 3.3 / EMF 2.3 / JDK 5.0 [1]. [1]http://emft.eclipse.org/modeling/emft/downloads/?showAll=1&hlbuild=N200709182348&project=net4j#N200709182348 Project 'org.eclipse.net4j.jms.api' is missing required library: 'lib/geronimo-jms_1.1_spec-1.1.jar' Is this something you plan to redistribute? Or just something required at build-time (in order to allow the examples to compile)? If the former, please submit it to IPzilla. If the latter, can you download a copy and put it in /home/www-data/build/3rdPartyjars/ on emft.eclipse.org? You can then rerun the build from the new build page [2], but you'll have to edit your examples' customTargets.xml so that the build knows where to find this jar. See Teneo [3] for an example of how to do this. [2]http://emft.eclipse.org/modeling/emft/net4j/build/ [3]/cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.teneo.releng/builder/sdk/customTargets.xml#postFetch Then, you'll have to remove them from your zips so they're not redistributed (unless you get approved by IPzilla). Again, see Teneo for example code -- search for the string "remove_thirdpartyjars" [4]. [4]/cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.teneo.releng/buildAll.xml Also, if there's anything listed in your mapfile [5] that's deprecated or that you do not plan to release as part of your builds, please update the file and commit the changes to CVS. [5]/cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.net4j.releng/maps/net4j.map I can move things into a deprecated folder so that the mapfile generator will ignore them; or, you can just always build from a static mapfile that you manually maintain. (In reply to comment #1) > Project 'org.eclipse.net4j.jms.api' is missing required > library: 'lib/geronimo-jms_1.1_spec-1.1.jar' It's currently not expected to be redistributed. It's only needed when users decide to install the Net4j based JMS system that I provide as an example. And of course we need it at build time. I send you a private email with the jar. BTW. how can I easily transfer files to Eclipse.org? I'll go through the other steps as soon as you signal me that you've copied the jar to the server... (In reply to comment #2) > (In reply to comment #1) > > Project 'org.eclipse.net4j.jms.api' is missing required > > library: 'lib/geronimo-jms_1.1_spec-1.1.jar' > It's currently not expected to be redistributed. > As soon as you signal me that you've copied the jar to the server... Ping! > BTW. how can I easily transfer files to Eclipse.org? Generally I use WinSCP on windows or Konqueror's fish:// ioslave on KDE-based linux. For commandline (cygwin/linux): scp or rsync Still having classpath problems [1] despite checking out the geronimo jar into the correct lib/ folder during the build. [1]http://emft.eclipse.org/modeling/emft/build/log-viewer.php?project=net4j&build=0.8.0/N200709192016/&offset=1599&step=50 Do you think adding this to the net4j.jms.api plugin's build.properties would help, per the feature build docs [2]? jars.compile.order = . extra.. = lib/geronimo-jms_1.1_spec-1.1.jar [2]http://help.eclipse.org/help33/index.jsp?topic=/org.eclipse.pde.doc.user/reference/pde_feature_generating_build.htm Frankly, I'm at a loss here with the addition of that jar, everything builds fine in my workspace, but not during a headless build. Could it be an export issue in one or more manifest files? Ed, do you have any ideas? Maybe I'm wrong because it was long ago but this looks a bit like the (headless) build problem that caused me to stop providing regular builds for 0.7.x. Unfortunately I've no clue where the old logs with these errors are. Nick can you emember these issues? Some missing exception class was reported that was nevertheless present in the net4j.util plugin. All the org.eclipse.net4j.jms plugins are "only" examples. What happens if you ignore them completely in the build? The org.eclipse.net4j.db plugin will be used by org.eclipse.emf.cdo.server.db again. I looked into the build folders and noticed a strange thing: All plugins properly compiled have "@dot" folder with the compiled classes. But the suspicious db plugin doesn't have one. Shouldn't it be compiled before its classes are used in the compilation of the org.eclipse.net4j.jms.server.jdbc plugin (where the problem manifests itself)? One more thing. I've added empty dummy plugins: - org.eclipse.ant.optional.junit - org.eclipse.test This is just to get rid of some warnings in my workspace. They are not needed anywhere and don't have to be checked out for the headless build. Should they be removed from net4j.map? Two more things. 1) org.eclipse.ecf.provider.net4j is a planned initial contribution to ECF. I will delete the CVS folders completely when I'm able to hand the code over to Scott. It should not be part of my build. 2) I've noticed that the net4j.map conatins 2 entries for org.eclipse.net4j.db.derby, one with plugin@ and one with fragment@. It is a fragment at the moment. It could be come a plugin in the future but that has no priority. Ok, I've played a bit with the map file. The order of entries seems to be unimportant. I've removed (commented out) several entries (see previous comments). The point is, when removing org.eclipse.net4j.jms.server.jdbc (the one with compile errors) from the map file *and* the examples feature.xml the build goes on - surprise!! ;-) I'd like to see if everything else builds fine. The omitted plugin is not used by others. If everything else is ok, we could try to build CDO which makes also use of the org.eclipse.net4j.db plugin (the one which is not found although it is there). If that works as it should, we can be sure that the root cause is not in the exporting plugin. If it causes the same symptoms chances are high that the root cause is in the exporting plugin rather than in two importing plugins. But I don't come to that point due to a new problem: http://emft.eclipse.org/modeling/emft/net4j/downloads/drops/0.8.0/N200709200640/buildlog.txt Apart from the above I believe that the root cause is a bug in the build process itself! If you look at any of the available build logs, you'll notice that the plugin that is reported to be unknown (well, classes in it) is really not compiled up to that time. Who/what determines the build order??? With my above approach we'd also be able to see if the org.eclipse.net4j.db plugin would be compiled at all or if it's "simply" a wrong build order... (In reply to comment #9) > Ok, I've played a bit with the map file. The order of entries seems to be > unimportant. Yes, order is not important in the map file, but everything that's needed must be there, including ant/junit and o.e.test. > But I don't come to that point due to a new problem: >http://emft.eclipse.org/modeling/emft/net4j/downloads/drops/0.8.0/N200709200640/buildlog.txt No, that's because you probably don't have o.e.test mentioned in your test plugins/features. See the procedures doc for more info [1]. [1]http://wiki.eclipse.org/Modeling_Project_Releng/Building_Zips_And_Jars#Build_Problems_.26_Solutions So the real issue here is to figure out how to force PDE to do things in the correct order. And that's where I'm stuck. In your latest map file, I notice you have three entries for org.eclipse.test. Granted, two are commented out... but why do you have a copy within your cvs tree? You should only need the official one, from the official tag, in order for your tests to build and run. (In reply to comment #10) > Yes, order is not important in the map file, but everything that's needed must > be there, including ant/junit and o.e.test. Yes, they are there, but before each was there twice. > No, that's because you probably don't have o.e.test mentioned in your test > plugins/features. See the procedures doc for more info [1]. They are mentioned in the tests feature and appear in the map file with correct version tag (AFAIK). Any other hints? > So the real issue here is to figure out how to force PDE to do things in the > correct order. And that's where I'm stuck. Are we already able to build without the o.e.n.jms.server.jdbc plugin so that we can see whether o.e.n.db gets built at all? (In reply to comment #11) > In your latest map file, I notice you have three entries for org.eclipse.test. > Granted, two are commented out... but why do you have a copy within your cvs > tree? You should only need the official one, from the official tag, in order > for your tests to build and run. --> comment #7 Eike:
I've found a number of problems which should be easy to fix:
a) there's no org.eclipse.net4j-feature, yet you reference it in other features.
<requires>
<import feature="org.eclipse.net4j" version="0.8.0" match="compatible"/>
</requires>
You need this feature. It should contain all the other runtime features, like .db and .ui, UNLESS you want these features to live ONLY in the SDK, but not the runtime. However you structure things, you need to have the SDK feature containing everything except tests (examples are optional - they can live in the SDK or as a separate feature/zip). Tradionally, SDK = runtime feature + source + doc, but as I said, some people also include examples in there too.
b) The order that you list features IS IMPORTANT (unlike the order in the map file). I'd totally forgotten this, but it's crucial that your SDK feature, which is the first entry point in the build cascade, lists the features in the order you want them built. Because .examples depends on .db, .db must be listed first. Also, because .doc contains javadoc derived from .source, .source must be done before .doc.
c) To disable building certain parts of your code base, simply remove references to those plugins/features sin your feature cascade. If the SDK feature does not contain the examples feature, and the examples feature is not built (see buildAll.xml in your .releng project) then those features/plugins will never be created. Or, if you want to build part of the examples but not all, edit the examples feature.xml to omit the pieces you don't want to build.
d) You don't have an examples branding plugin. I don't know if this is a problem, but I'm making a note of it in case we hit a new brick wall further along.
(In reply to comment #14) > a) there's no org.eclipse.net4j-feature Correction. I'm blind. ;-) First good builds out the door today. Woohoo! [1] [1]http://emft.eclipse.org/modeling/emft/downloads/?showAll=1&hlbuild=N200709201949&sortBy=date&project=net4j#N200709201949 Things to verify: * SDK zip contains doc * doc plugin contains javadoc + sources * features contain the new modeling32.png icon instead of the old eclipse32.png icon (see bug 177595) * features can be installed from update site (see bug 201839 comment 15) Also, I notice you have a .source-feature, instead of this being generated. Is this another 'make my workspace pretty / suppress PDE warnings' hack? ;-) Other components just generate their entire source feature & plugin from the template files in the SDK, eg in SDK feature's build.properties: generate.feature@org.eclipse.net4j.source=org.eclipse.net4j (In reply to comment #16) > Things to verify: > * SDK zip contains doc Verified. > * doc plugin contains javadoc + sources JavaDoc contained. Sources missing (related to the below?) > * features contain the new modeling32.png icon (see bug 177595) Done. > * features can be installed from update site (see bug 201839 comment 15) buildUpdate.sh does not work (see bug #201839 comment #16 comment #17) > Also, I notice you have a .source-feature, instead of this being generated. Is > this another 'make my workspace pretty / suppress PDE warnings' hack? ;-) YES! > Other components just generate their entire source feature & plugin from the template > files in the SDK, eg in SDK feature's build.properties: > generate.feature@org.eclipse.net4j.source=org.eclipse.net4j I have that. I have removed the dummy entry from net4j.map The SDK is missing the some sources. These should come through o.e.net4.db-feature: org.eclipse.net4j.db org.eclipse.net4j.db.derby These should come through o.e.net4.examples-feature: org.eclipse.net4j.jms.admin org.eclipse.net4j.jms.server.jdbc org.eclipse.net4j.jms.server org.eclipse.net4j.jms Forgotten: These should come through o.e.net4.ui-feature: org.eclipse.net4j.ui org.eclipse.net4j.util.ui I've installed the SDK feature into a fresh Eclipse.
Some *Feature* Details in the About dialog are not ok:
org.eclipse.net4j - OK
org.eclipse.net4j.doc - OK
org.eclipse.net4j.db - UNRESOLVED %featureName and %providerName
org.eclipse.net4j.source - UNRESOLVED %featureName and %providerName
org.eclipse.net4j.ui - UNRESOLVED %featureName and %providerName
org.eclipse.net4j.examples - MISSING
org.eclipse.net4j.sdk - MISSING
Some *Plugin* Details in the About dialog are not ok:
- All 15 plugins are listed - OK
- org.eclipse.net4j.jms had a type in Bundle-Vendor - FIXED
- All but the following plugins have UNRESOLVED %featureName and %providerName
org.eclipse.net4j
org.eclipse.net4j.doc
I have no idea how to fix the remaining issues!
I was able to fix most of the UNRESOLVED %featureName and %providerName issues. Now only the source feature+plugin still have this problem. From what I can see the files are correct - they seem to be ignored?! (In reply to comment #21) > I was able to fix most of the UNRESOLVED %featureName and %providerName issues. > > Now only the source feature+plugin still have this problem. > From what I can see the files are correct - they seem to be ignored?! > AFAIK, these values will be pulled from what's in the SDK's source*Template folders. (In reply to comment #18) > The SDK is missing the some sources. > > These should come through o.e.net4.db-feature: > org.eclipse.net4j.db > org.eclipse.net4j.db.derby > > These should come through o.e.net4.examples-feature: > org.eclipse.net4j.jms.admin > org.eclipse.net4j.jms.server.jdbc > org.eclipse.net4j.jms.server > org.eclipse.net4j.jms > Add more source generation directives to your SDK: /org.eclipse.emf.net4j/plugins/org.eclipse.net4j.sdk-feature/build.properties (In reply to comment #22) > AFAIK, these values will be pulled from what's in the SDK's source*Template > folders. Apart from the crap in the build.properties: bin.includes = plugin.xml,\ plugin.properties,\ ... feature.properties,\ feature.xml The whole file seems to be ignored. When I unpack the sdk zip, the source feature contains only feature.xml and the source plugin is missing all the legal stuff and the plugin.properties. Do we need this mysterious "rootfiles" folder? I've googled this one: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf/deprecated/org.eclipse.emf-feature/org.eclipse.emf.sdk/build.properties?root=Modeling_Project&view=co Should I add "root=rootfiles"? What are the semantics of "generate.feature@ABC=XYZ"? I know now semantics of "generate.feature@ABC=XYZ" ;-) Yes, you need a root folder w/ license stuff in it. Compare with other EMFT or EMF projects for examples. Note that the URL you mentioned is in a *deprecated* folder so don't use that as gospel. The EMF SDK feature is now in the features/ folder. For an example of a component with multiple namespaces (and therefore multiple source feature generation directives) see EMF.Transaction. I have neither a "rootfiles" folder in my SDK project nor do I have "root=rootfiles" in its build.properties. Nevertheless there does not appear to be a problem with that! All the files of the project folder are also present in the feature folder of the zip. Is it somewhat redundant? Regarding the missing files in source feature+plugin, I have already compared with Teneo but I can't see a significant difference in our sources. Only that Martins zip is ok and mine is not. (In reply to comment #28) > I have neither a "rootfiles" folder in my SDK project nor do I have > "root=rootfiles" in its build.properties. Nevertheless there does not appear to > be a problem with that! All the files of the project folder are also present in > the feature folder of the zip. Is it somewhat redundant? Completely. But required for good project health. It's not required from a build POV, it's required from a legal POV. But IANAL, TINLA. I just know what the IBM-authored projects and components have had to do. Perhaps it's not as strict an enforcement when you don't have the Legal folks breathing on you all the time. ;-) > Regarding the missing files in source feature+plugin, I have already compared > with Teneo but I can't see a significant difference in our sources. Only that > Martins zip is ok and mine is not. Check your feature.xml files -- see if things are in the same order. Or diff with an older project, like MDT-OCL instead. Marking this bug ASSIGNED because the build is actually there. Follow-up bugs are filed seperately, for example bug 204350, bug 204232, ...) Now I understand what root=rootfiles means. First I thought it refers to files that are to be additionally copied to the feature root. But no, these files/folders go to the eclipse installation root when unzipping the feature!! Cool, this could be very useful to distribute initial config folders and the like... See bug 202418 comment 19. Need a first promoted build before we can close this bug. (In reply to comment #32) http://www.eclipse.org/modeling/emft/news/relnotes.php?project=net4j&version=HEAD http://www.eclipse.org/modeling/emft/downloads/?project=net4j Move to verified as per bug 206558. Reversioned due to graduation CLOSING |