Community
Participate
Working Groups
We should add it carefully :) Here is the CQ https://dev.eclipse.org/ipzilla/show_bug.cgi?id=1910
It's yours Dave :)
Thanks...I'll work on getting this done before the end of the year. David Williams mentioned that the Xerces 2.9.0 serializer.jar file has some conflicts with the version that comes with the IBM 1.5 JDK. I'll try to avoid this conflict when packaging everything. I've CC'd David on this so that he can provide any additional feedback in what he found with Xerces 2.9
Question I have with Xalan as it was approved for other projects. Xalan includes the ability to compile XSLTs to java byte code using XSLTC. However, the source for this requires the org.apache.bcel packages which aren't currently in orbit. BCEL was approved for use in CQ 88, so I think we need another CQ to get BCEL into orbit as well. However, going to be sligth delay while I get my committer password reset, I seemed to have forgotten it. :) Anyways, I'll get the CQ submitted or if another person wants to submit it, that would be great as well.
Okay got my password going again, so I'll see what I can do to get the BCEL items through IP review. I did a search and didn't see it already. Also, I will try and get Xalan 2.7.1 started as well, since that is more directly compatible with the Xerces 2.9.0. Xalan 2.7.0 includes it's own serializer which isn't api or package compatible with the existing Xerces serializer. Regardless we'll need the BCEL approved in order to get this added correctly.
Alright, it looks like I have a clean compile with the dependcies set correctly...need to do a small unit test to make sure that Xalan executes a transformation correctly. A couple of additional reuse CQs will need to be created in order to complete this: 1. Java_Cup will need to have a bundle created, this already has been approved for a couple of different Eclipse projects. 2. BCEL - has already been approved, so we just need a re-use. I have created bundles already for the above, and have them stored locally. So will have them ready to go when IP approval is given.
Added bcel dependency.
Added dependency on java_cup, bug214428
added org.apache.xalan version 2.7.0 to orbit. Chris or David if you could double check this I would appreciate it. I followed the instructions that Chris pointed me to on the Orbit Wiki page: http://wiki.eclipse.org/Adding_Bundles_to_Orbit Let me know if you see anything wrong. Only thing I don't think was done, was the necessary map files. Also, this does require the org.apache.bcel v5.2 and java_cup.runtime v0.10 bundles as well, as xerces 2.8.0 or greater. Xalan includes it's own serializer in 2.7.0, in version 2.7.1 it uses the xerces 2.9.0 serializer. Doug, if you could check these bundles out and make sure they work with the XSLT launching and debugging it would be appreciated.
Excellent. I'll test them when I get a chance. One question - how do dependencies like this work? If we make a build that depends on the new Orbit bundles, will these bundles be available when the user downloads our build? Are they immediately available, or do they have to specifically go somewhere to fetch them? Is it better to not change our code to depend on these bundles for the meanwhile?
I've handled this before by making sure that the orbit bundles are added to the feature that I'm working on, that way it is included, and when update manager is run it isn't included unless the version specified in the feature is newer than the version already installed. As for building, that is probably a question that David Williams can answer better.
(In reply to comment #10) > > As for building, that is probably a question that David Williams can answer > better. > Yep. The bundle is "included" in the feature. It'll be automatically be "packaged up" with the feature during the build. That's the way 'common logging', etc., are currently working. The one thing that has to be coordinated is the map file. As it is, in WTP (normal and incubator), we just include a copy of the orbit map file (the "get" version, produced after the Orbit build) in the (appropriate) releng directory ... so, we have to be careful about picking the one we want (eventually the one that will Xalan in it) and then making sure we update it if we are using 'preliminary' Orbit builds. We are currently using the "recommended" one for Ganymede M4.
(In reply to comment #8) > > Chris or David if you could double check this I would appreciate it. I > followed the instructions that Chris pointed me to on the Orbit Wiki page: > > http://wiki.eclipse.org/Adding_Bundles_to_Orbit > > Let me know if you see anything wrong. Dave, I pulled this in from CVS and it appears you added the source, as source? I've never actually added source to any of the ones I do, yet, but think it's added as a zip file? Also, I didn't really see the classes as they normally are, in orbit bundles. You do know we don't "rebuild" from source, right? (Or, at least, haven't so far). Not sure why. But, I typically create the binary bundle using New, Project, Plugin Project from Existing jar ... then select the actual jar that comes with the binary distribution of Xalan. Let me know if you don't want to re-try and want some more direct help getting this in Orbit ... I'd be glad to help. Thanks,
(In reply to comment #11) > Dave, I pulled this in from CVS and it appears you added the source, as source? > > I've never actually added source to any of the ones I do, yet, but think it's > added as a zip file? Yeah, typically it is added as a Zip, but one thing I found doing it this way was some dependencies that weren't known by the original CQ that this was done with. In particular, the original CQ did not mention (that I'm aware) that BCEL and Java_Cup.Runtimes were required in order for Xalan to be compiled. Now I could change it so that I check the .class files and bin directory into CVS as well, but not sure if this really should be a best practice. Another one in which I discovered dependency issues is with XMLBeans CQ. It has dependencies on Saxon for XQuery and XPath support but Saxon hasn't passed IP so just including the straight binaries with out recompiling would leave a system that might not work entirely. > > Also, I didn't really see the classes as they normally are, in orbit bundles. > > You do know we don't "rebuild" from source, right? (Or, at least, haven't so > far). > Not sure why. I've seen some that have the source and some that don't. You are correct all seem to include the binaries, so I can try to get those added. > > Let me know if you don't want to re-try and want some more direct help getting > this in Orbit ... I'd be glad to help. I'll take another shot later in the week to get the Xalan items done. However, I think from Orbit I would like to see it being compiled nightly, or at least going through the process of making sure the original source compiles with the dependencies added correctly. At least for me, it helped bring to light several additional dependencies that needed CQs to be approved and included.
Alright I uploaded the compiled class files as well. I also did the same for the java_cup dependency. David or Chris could you please check this out to make sure I have everything that is supposed to be there now. I still think we should build these like we do every other plugin within eclipse. Recompiling them does help to reveal some hidden dependencies that may not be necessarily noted in the source or documentation.
(In reply to comment #14) > Alright I uploaded the compiled class files as well. I also did the same for > the java_cup dependency. > > David or Chris could you please check this out to make sure I have everything > that is supposed to be there now. > > I still think we should build these like we do every other plugin within > eclipse. Recompiling them does help to reveal some hidden dependencies that > may not be necessarily noted in the source or documentation. > Hmmm, still no good, IMHO. I think we need a non-re-compiled version, for sure! You mention the hidden dependencies, and that's fine, if, as a developer, you want to explore and understand the code better ... but, for an Orbit bundle there are draw backs. (And, by the way, there are other tools to explore dependencies in binary code ... though for the life of me I can't remember which ones I've used before ... Dependancy Finder? http://depfind.sourceforge.net/ For one thing, I think some of the dependencies should be "optional". If someone is only using this for transforms, for example, they do not need bcel and cup, right? They only need those if they are doing some XSL pre-compiles? Another point of review, I think there's no need for you to re-export any of the bundles ... and you re-export them all. Well, maybe the xerces one ... Nodes, Elements, etc., are part of the interface, right? But ... I think normally best not too, unless there's a compelling reason to, and I think for xerces and xalan, everyone knows they "go together" so would not seem unusual for someone to pre-req them both. Also, I know the xalan guys say, on their website, "has only been tested with xerces 2.9.0". For this reason, I think the xerces constraint should be written as [2.9.0,3.0.0) instead of [2.8.0,2.9.0) unless someone knows otherwise? I suggest we get a "binary" version in the repository, ready to build, so we can start to include this in Orbit builds ... and then add the source as zip a little later on?
(In reply to comment #15) > (In reply to comment #14) > > For one thing, I think some of the dependencies should be "optional". If > someone is only using this for transforms, for example, they do not need bcel > and cup, right? They only need those if they are doing some XSL pre-compiles? They are only needed if the XSLT is going to be compiled down to java byte code. This is controlled a couple of different ways with XALAN. 1. Through the API directly, and the command line utilities (agreed at this point it is optional). 2. Through specifiying a particular XSLT Transformer class to use through the JAXP API. In particular the: org.apache.xalan.xsltc.trax.TransformerFactoryImpl Xalan actually has two transformers, a non-compiling one and the one above. The non-compiling one is: org.apache.xalan.processor.TransformerFactoryImpl So the best answer I can give, is that it depends on which transformer is used if it is optional. > > Another point of review, I think there's no need for you to re-export any of > the bundles ... and you re-export them all. Well, maybe the xerces one ... > Nodes, Elements, etc., are part of the interface, right? But ... I think > normally best not too, unless there's a compelling reason to, and I think for > xerces and xalan, everyone knows they "go together" so would not seem unusual > for someone to pre-req them both. Doesn't it need to be exported if you are using the API's for XALAN debugging as Doug needs to do so for the XSLT Debugger. > > Also, I know the xalan guys say, on their website, "has only been tested with > xerces 2.9.0". For this reason, I think the xerces constraint should be written > as > [2.9.0,3.0.0) instead of [2.8.0,2.9.0) unless someone knows otherwise? Xalan 2.7.0 was released in August 2005, which means it uses at least Xerces 2.8 or greater. Xalan 2.7.1 was released November 2007 and was upgraded for Xerces 2.9.x. Xalan 2.7.0 also has it's own serializer as well. This particular version we are adding now is 2.7.0. http://xml.apache.org/xalan-j/readme.html#notes_270 > > I suggest we get a "binary" version in the repository, ready to build, so we > can start to include this in Orbit builds ... and then add the source as zip a > little later on? > I'll go ahead and get that done this weekend. Again though I have concerns about being in that grey area of IP. I.E. adding something that might have dependencies on something that hasn't passed IP, thus requiring somebody to go outside and add an external dependency. It's one of those grey areas you talk about David.
Just want to let you know what the current Xalan launcher/debugger actually requires for 2.7.0. The only required jar files are xalan.jar and serializer.jar. Xerces is _not_ required (JAXP will make use of any configured parser - the JRE's default parser is perfectly adequate). Currently, the transformer being used for both launching and debugging is the interpretive (non-compiling) transfomer org.apache.xalan.processor.TransformerFactoryImpl. This is absolutely necessary for the debugger, but we could change it to the XSLTC one for the normal launcher if we wanted to. So, for the moment at least, we have no dependency on the XSLTC-dependent packages. And if that helps us clear up any IP issues, we can leave it that way!
(In reply to comment #17) > The only required jar files are xalan.jar and serializer.jar. Xerces is _not_ > required Great. We'll make that optional too. (I know sometimes libraries like this specify xerces just so they can work consistently on older VMs). Speaking of which ... from what I can tell from their website, xalan can run with Jaa 1.3 (technically, not sure who would want too :) so we should specify the Execution Environment as 1.3. (In reply to comment #16) > Xalan 2.7.0 was released in August 2005, which means it uses at least Xerces > 2.8 or greater. Xalan 2.7.1 was released November 2007 and was upgraded for > Xerces 2.9.x. Xalan 2.7.0 also has it's own serializer as well. Thanks, I realize now it must have been 2.7.1 I was reading about. > Again though I have concerns about being in that grey area of IP. I.E. adding > something that might have dependencies on something that hasn't passed IP, thus > requiring somebody to go outside and add an external dependency. > It's one of those grey areas you talk about David. You mean that cup and bcel haven't been approved yet? All the better that they are optional :) But, to clarify, this orbit bundling doesn't really dictate how projects may or may not use them. We, in Orbit, just want to package them in a way that provides reasonable flexibility, when there's a reason for it. Projects can then make the decision that's best for them ... and, believe me, small downloads is often a big concern. So, if someone knew for sure they nor their users would ever want the compile down to byte code, then I'm sure they'd want the slightly smaller download. Where as we in WTP may want the full function eventually, and we'd be free to include all four bundles in our distribution (assuming they pass IP, that is). So, I don't think there's a gray area at this level ... at this level it's just more a technical concern of having correct modularization. > Doesn't it need to be exported if you are using the API's for XALAN debugging > as Doug needs to do so for the XSLT Debugger. Yes, certainly Xalan's public API's need to be exported, but I was talking about the RE-export of whole other bundles. That's normally bad practice. The better practice is just to let clients 'require' all the ones they need. Even though there's some redundancy if "everyone" needs them ... still better!
(In reply to comment #18) > Speaking of which ... from what I can tell from their website, xalan can run > with Jaa 1.3 (technically, not sure who would want too :) so we should specify > the Execution Environment as 1.3. Need to be careful, as I think Xalan does use RegExpressions, and if you make it 1.3, regular expressions can't be used, and we would have to get the apache regular expression library supported. However if you make it 1.4 then it can use the built in Java 1.4 Regular Expression support so no other dependency required. (one of the advantages of compiling from source is you learn these type of things). > > Again though I have concerns about being in that grey area of IP. I.E. adding > > something that might have dependencies on something that hasn't passed IP, thus > > requiring somebody to go outside and add an external dependency. > > It's one of those grey areas you talk about David. > > You mean that cup and bcel haven't been approved yet? All the better that they > are optional :) > Actually, these are approved and in orbit now. Both BCEL and Java_Cup. > But, to clarify, this orbit bundling doesn't really dictate how projects may or > may not use them. We, in Orbit, just want to package them in a way that > provides reasonable flexibility, when there's a reason for it. Projects can > then make the decision that's best for them ... and, believe me, small > downloads is often a big concern. So, if someone knew for sure they nor their > users would ever want the compile down to byte code, then I'm sure they'd want > the slightly smaller download. Where as we in WTP may want the full function > eventually, and we'd be free to include all four bundles in our distribution > (assuming they pass IP, that is). So, I don't think there's a gray area at this > level ... at this level it's just more a technical concern of having correct > modularization. But the problem is in the way we package them. Unless we specify in the documentation of the bundle, which ones are optional and why, then the person using the bundle my expect that all functionality or dependencies are there. I have no problem splitting the functionality out, as long as we are clear to the user community what goes with what. I know that when I used the apache.axis bundles, I had to go hunting for multiple dependecies when I was putting a plugin together that was going to use them. Would have saved me time to know which ones were required. The nasty thing comes when a plugin requires another plugin, and if you download orbit piecemeal, it takes a while get it right. > > > > Doesn't it need to be exported if you are using the API's for XALAN debugging > > as Doug needs to do so for the XSLT Debugger. > > Yes, certainly Xalan's public API's need to be exported, but I was talking > about the RE-export of whole other bundles. That's normally bad practice. The > better practice is just to let clients 'require' all the ones they need. Even > though there's some redundancy if "everyone" needs them ... still better! Agreed. As I said, I'll get this repackaged this weekend.
Repackaged, and created from existing binary jars. modeled after the java_cup.runtime bundle. If somebody could verify this is now correct, I would appreciate it.
(In reply to comment #20) > Repackaged, and created from existing binary jars. modeled after the > java_cup.runtime bundle. > > If somebody could verify this is now correct, I would appreciate it. > Did you remember to commit? :) I don't actually see the .class files. There's a org/apache/xalan directory, but I only see property files in it.
(In reply to comment #21) > (In reply to comment #20) > Did you remember to commit? :) I don't actually see the .class files. > There's a org/apache/xalan directory, but I only see property files in it. Oops...forgot I had to force CVS to add the .class files, because it doesn't by default. Should be in there now, just committed them.
ok, next then is that the manifest.mf file needes Bundle-Localization: plugin And, the build.properties needs some tweaking, it has an unnecessary "java_cup" and you need to include the META-INF directory (and the 'service' files). In fact, I suspect you need to include the service files on the exported classes list ... there's some theoretical issues around this, but we had to for xerces. Last, and minor, I always think it best to set the default charset for the project, to be ISO-8859-1.
(In reply to comment #23) > Last, and minor, I always think it best to set the default charset for the > project, to be ISO-8859-1. I always find this curious...I would think that the default would be UTF-8 as ISO-8859-1 is a subset of UTF-8.
(In reply to comment #24) > (In reply to comment #23) > > Last, and minor, I always think it best to set the default charset for the > > project, to be ISO-8859-1. > > I always find this curious...I would think that the default would be UTF-8 as > ISO-8859-1 is a subset of UTF-8. > The reason to prefer ISO-8859-1 over UTF-8 for a default charset is that it is a single byte charset and therefore survives most forms of "transfer" or "translation" such as to and from cvs or other tool processing.
(In reply to comment #23) > ok, next then is that the manifest.mf file needes > Bundle-Localization: plugin done > And, the build.properties needs some tweaking, it has an unnecessary "java_cup" > and you need to include the META-INF directory (and the 'service' files). done > In fact, I suspect you need to include the service files on the exported > classes list ... there's some theoretical issues around this, but we had to for > xerces. > The project itself is include in the root classpath, so all folders underneath it are included as well. This includes the Manifest folder and all sub folders.
resolving, as I haven't heard anything more that needs to be done.
(In reply to comment #27) > resolving, as I haven't heard anything more that needs to be done. > Well, there is the matter of adding them to the maps, the feature, and releasing them all for a build ... but, I've just done that. So ... _now_ there's nothing more to be done ... well, except to test them after a build!
