Bugzilla – Full Text Bug Listing
| Summary: | [filetransfer][plan] create filetransfer provider from httpclient 4.X | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [RT] ECF | Reporter: | Scott Lewis <slewis> | ||||||
| Component: | ecf.filetransfer | Assignee: | ecf.core-inbox <ecf.core-inbox> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | P3 | CC: | akurtakov, bugs.eclipse.org, caniszczyk, cvgaviao, david_williams, harshana05, henrich.kraemer, hmalphettes, jan.sievers, jesper, john.arthorne, kane.mx, kane.zhu, krzysztof.daniel, mjmeijer, pascal.rapicault, sbouchet, steffen.pingel, thomas.joiner, wim.jongman | ||||||
| Version: | 2.0.0 | Keywords: | helpwanted | ||||||
| Target Milestone: | --- | Flags: | slewis:
iplog+
|
||||||
| Hardware: | PC | ||||||||
| OS: | Windows XP | ||||||||
| Whiteboard: | |||||||||
| Bug Depends on: | |||||||||
| Bug Blocks: | 337449 | ||||||||
| Attachments: |
|
||||||||
|
Description
Scott Lewis
Changed title to refer to httpcomponents This should have originally been marked as an enhancement. Also setting target milestone to 3.1 and adding zx. I'm voting for this, in absence of the proper voting mechanism :P It seems as if this could free the platform from the need to ship org.apache.commons.logging 1.0.4. Votes are great. Realistically (resource wise), contributions are needed to make this a reality. For anyone's info: The incomplete/non-functional skeleton ECF provider based upon httpcore (httpclient 4) is, for the moment, available here (anonymous CVS): :pserver:anonymous@ecf1.osuosl.org:/ecf module: plugins/org.eclipse.ecf.provider.filetransfer.httpcore If there are people identified who are able/willing to contribute substantive work toward completing this provider (i.e. stating via this bug), then I will move this code over to the dev.eclipse.org CVS repo's ECF incubation area. I'm sorry that I can't contribute to this effort. I'm only interested in getting rid of the logging 1.0.4 dependency ;-) (In reply to comment #6) > I'm sorry that I can't contribute to this effort. I'm only interested in > getting rid of the logging 1.0.4 dependency ;-) What I'm essentially saying is...the way to get rid of the dependency is to contribute to the effort ;-). I know. Still, this is too off-topic for me and my time is entirely used up for my own project. The effect of having the duplicate logging bundle in the TP for me is just that I don't seem to get the 1.1.x source bundle installed. Not nice but tolerable ;-) (In reply to comment #8) > I know. Still, this is too off-topic for me and my time is entirely used up for > my own project. I know how you feel Eike. :) Changing target milestone. Changing target milestone. FWIW: Httpclient 4.1 has been added to Orbit. See bug 334937. I'm resetting the target milestone because I (Scott) cannot work on this without some sort of external contribution, support, or assistance. For the record, the amount of work to use Httpclient 4.1 as the ECF filetransfer provider (and therefore with p2 and other filetransfer clients) is small...I won't call it trivial...but I cannot commit to doing this work without some external support for it...i.e. I can't do it on my own personal dime. If there are people that want this to happen (to have ECF filetransfer use httpclient 4.1), then please do something to make it happen. Created attachment 195753 [details]
httpclient 4.1 implementation
I added it as a new plugin with a Bundle-Version of 1.0.0
A couple of questions:
Is httpclient supposed to be included in the "Require-Bundle" or if that is taken care of elsewhere? This is my first time working with OSGi/Eclipse Plugins and that was what worked for me.
Am I correct in assuming that there is only one possible ISSLSocketFactoryModifier for the life of the plugin? If so then it should be possible to rewrite it to where all of the HttpClientRetrieveFileTransfer instances share a single HttpClient instance...I'm not sure whether or not that would be considered best.
Should NTLMProxyDetector be renamed to ProxyDetector now that I have it detecting SPNEGO proxies as well?
I rather belatedly noticed that 4.1 is a 1.5 library, so that means that unless something can be done with OSGi so that it loads this version if > 1.5, it looks like this will have to wait until ECF can move to Java 1.5. Of course even OSGi can do something about it, it doesn't really make sense to do it considering the fact that one of the reasons to switch to 4.1 is to get rid of dependencies rather than add more.
Well...all that said, tell me how it looks.
(In reply to comment #13) > Created attachment 195753 [details] > httpclient 4.1 implementation First, let me say 'thank you'. This is a terrific contribution, and it's highly appreciated. > > I added it as a new plugin with a Bundle-Version of 1.0.0 That's ok with me...although it might be a good idea to give it the same version as the underlying httpclient version (we've done this in the past...just so it's easier to identify). Anyway...for providers this is a judgment call, and I'm ok with either way. > > A couple of questions: > > Is httpclient supposed to be included in the "Require-Bundle" or if that is > taken care of elsewhere? This is my first time working with OSGi/Eclipse > Plugins and that was what worked for me. Typically, we use Import-Package (over Require-Bundle) in most cases...but when I get a chance to get to this I can help with that refactoring. > > Am I correct in assuming that there is only one possible > ISSLSocketFactoryModifier for the life of the plugin? If so then it should be > possible to rewrite it to where all of the HttpClientRetrieveFileTransfer > instances share a single HttpClient instance...I'm not sure whether or not that > would be considered best. I'm not sure if that would be a good idea or not...as doesn't that have some threading implications (as well as other resource implications)? In any event, we can discuss this over the coming few months. I'm not closed to such a thing...but I'm just not sure of the implications. > > Should NTLMProxyDetector be renamed to ProxyDetector now that I have it > detecting SPNEGO proxies as well? That would be fine with me. What are SPNEGO proxies (just for my information)? > > I rather belatedly noticed that 4.1 is a 1.5 library, so that means that unless > something can be done with OSGi so that it loads this version if > 1.5, it > looks like this will have to wait until ECF can move to Java 1.5. Of course > even OSGi can do something about it, it doesn't really make sense to do it > considering the fact that one of the reasons to switch to 4.1 is to get rid of > dependencies rather than add more. I'm personally fine with having it be based on 1.5...but I need to check with the p2/Equinox/Platform team about the 1.5 requirement, as I'm not sure whether that would work for Eclipse (and since ECF's filetransfer is used for the p2 stuff we would need to clear things with them. > > Well...all that said, tell me how it looks. Ok. It's going to take me a little time (days) to get to this...only because I'm very occupied right now with other personal commitments, as well as ECF 3.5.1 (coming out this week), and Indigo simultaneous release. If you don't hear any action from me on this bug after a reasonable amount of time, please feel free to ping me by posting other comments on this bug. Your contribution is most appreciated Thomas! Thanks again. Unfortunately there is not enough time to include this in Indigo, but I'm very hopeful we'll be able to get this into all future ECF releases. Also BTW...Thomas...would you be interested in becoming an ECF committer? One other thing...the attachment 195753 [details] has a type 'patch'...but it seems to be binary. Is it a zip of the new project? Adding Henrich Kramer to this bug. Henrich...I'm adding you because over the coming months I'm hoping that you might be able to provide assist me in reviewing, revising this new contribution from Thomas in order to get it finalized and into the ECF release. >That's ok with me...although it might be a good idea to give it the same >version as the underlying httpclient version (we've done this in the >past...just so it's easier to identify). Anyway...for providers this is a >judgment call, and I'm ok with either way. It doesn't really matter to me either, so I'll leave it up to you. >I'm not sure if that would be a good idea or not...as doesn't that have some >threading implications (as well as other resource implications)? In any event, >we can discuss this over the coming few months. I'm not closed to such a >thing...but I'm just not sure of the implications. As the implementations for the ClientConnectionManager are thread-safe, it shouldn't be a problem on their side. And I think that the only problematic point about sharing the HttpClient between them would be whether or not the SSLSocketFactory changes between requests. However, since I didn't know whether I could count on the SSLSocketFactory staying the same, I didn't investigate fully. However as you said, this is something to discuss in the coming months. >That would be fine with me. What are SPNEGO proxies (just for my information)? "Simple and Protected GSSAPI Negotiation Mechanism"...to be exact it's a general authentication mechanism, however I throw an Exception when trying to authenticate against a proxy that requires it due to the fact that it's not supported in this implementation. HttpClient 4.1 does support SPNEGO, however I believe that it requires Java 1.6 to use it and according to the documentation (http://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html#spnego) it appears that it requires configuration files and registry setup in order to work, so I'm not sure if it is really possible in the scope of ECF. >I'm personally fine with having it be based on 1.5...but I need to check with >the p2/Equinox/Platform team about the 1.5 requirement, as I'm not sure whether >that would work for Eclipse (and since ECF's filetransfer is used for the p2 >stuff we would need to clear things with them. Yeah, unless they decide to go to 1.5, it will have to wait until 4.x >Ok. It's going to take me a little time (days) to get to this...only because >I'm very occupied right now with other personal commitments, as well as ECF >3.5.1 (coming out this week), and Indigo simultaneous release. If you don't >hear any action from me on this bug after a reasonable amount of time, please >feel free to ping me by posting other comments on this bug. Sounds good to me. >Your contribution is most appreciated Thomas! Thanks again. Unfortunately >there is not enough time to include this in Indigo, but I'm very hopeful we'll >be able to get this into all future ECF releases. Yeah, when I went to submit the patch and saw that the release train was in M7 already, I figured that it was too late for this release. > Also BTW...Thomas...would you be interested in becoming an ECF committer? Sure, I'd love to! >One other thing...the attachment 195753 [details] has a type 'patch'...but it >seems to be binary. Is it a zip of the new project? It shouldn't be. When I download it and open it it opens fine as a text file...however if I click on the link it doesn't show properly. Looking at the encoding it looks like it got encoded as Little Endian UCS-2. (I just did a "git diff" and didn't really pay attention to the encoding since I assumed that it would choose properly.) I'll re-encode in ISO 8859-1 and re-upload and see if that fixes the problem. Created attachment 195763 [details]
httpclient 4.1 implementation
Hopefully fixing the encoding problem...
*** Bug 275728 has been marked as a duplicate of this bug. *** (In reply to comment #18) > *** Bug 275728 has been marked as a duplicate of this bug. *** First, thanks Thomas for the contribution. I have made some modifications to what Thomas contributed, and release two new bundles, and a new feature to master: http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/commit/?id=266ffcfeec8b9ea34beb4d78d2e683c15cda6f27 The two new bundles are http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclient4 This is the actual httpclient4-based provider code. http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclient4.ssl This is an ssl fragment for the httpclient4 code...identical in function to the httpclient.ssl fragment for the 3.1-based httpclient provider. the feature project for this new httpclient4 is here http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/releng/features/org.eclipse.ecf.provider.filetransfer.httpclient4.feature All of these bundles are based upon the httpclient 3.1 provider, with changes provided by Thomas as attachment to this bug. There is still a little more work to do on this httpclient4 provider...e.g. 1) Create build metadata (jenkins project, mspec/cspec) to build the new feature. Note that the new feature depends upon 4 bundles from Orbit (httpcomponents and dependencies). 2) Change the areas that were changed in the httpclient 3.1 provider in the last year...for bugs that were fixed...i.e. bug 297742, and bug 370801. 3) Test in various proxy conditions (e.g. NTLM 2, etc). On a related note, version 4.2 of HttpClient has been released: http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt . We'll likely request the new version for Mylyn to leverage the improved (NTLM) authentication support. (In reply to comment #20) > On a related note, version 4.2 of HttpClient has been released: > http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt . We'll > likely request the new version for Mylyn to leverage the improved (NTLM) > authentication support. Hi Steffen. Thanks for the info. A couple of quick questions: 1) Do you have any references to the improved NTLM support (e.g. does it support all NTLM versions...e.g. v2?); 2) Is the 4.2 now version in Orbit? (In reply to comment #21) > 1) Do you have any references to the improved NTLM support (e.g. does it support > all NTLM versions...e.g. v2?); There is some information in the release notes. AFAIK, support for NTLM v2 has significantly improved over HttpClient 3.1 and they made further improvements in 4.2 over 4.1 but there are still limitations. > 2) Is the 4.2 now version in Orbit? Not, yet :). I'll file a CQ and update the Orbit bundles after Juno. Hi Scott, I can't find those two new bundles in repo of ECF daily build[1]. Could you pls build them in ECF daily build? I would like to package them for doing further testing. Thanks. [1] http://build.ecf-project.org/repo/N-HEAD-sdk.feature/lastSuccessful/archive/site.p2/ (In reply to comment #23) > Hi Scott, > > I can't find those two new bundles in repo of ECF daily build[1]. Could you pls > build them in ECF daily build? I would like to package them for doing further > testing. > > Thanks. > > [1] > http://build.ecf-project.org/repo/N-HEAD-sdk.feature/lastSuccessful/archive/site.p2/ I will do this (add to sdk), but it's going to take some time. Although a feature exists for the httpclient4-based stuff, we don't yet have an associated build project, and that sort of has to occur before it goes into the sdk build. In any event...please ask again and I'll get to it asap...I can't do it this week, however. BTW...one of the first things to do is to add in the fixes that have occurred over the past year to the httpclient 3.1-based environment (e.g. the abort fix, the multi-session enhancement, etc). (In reply to comment #24) > ...please ask again and I'll get to it asap...I can't do it this > week, however. Hi Scott, do you have a plan to build them in nightly build of ECF? (In reply to comment #25) > (In reply to comment #24) > > ...please ask again and I'll get to it asap...I can't do it this > > week, however. > Hi Scott, > > do you have a plan to build them in nightly build of ECF? Well, my plan is to do that when I can marshal the resources (mostly Markus K's time) to do so. That isn't right now, however. I'll add Markus to the bug, though. (In reply to comment #26) > Well, my plan is to do that when I can marshal the resources (mostly Markus K's > time) to do so. That isn't right now, however. I'll add Markus to the bug, > though. Hi Scott, Do you still have plan to do it? (In reply to comment #27) Anything I can do? (In reply to comment #27) > (In reply to comment #26) > > Well, my plan is to do that when I can marshal the resources (mostly Markus K's > > time) to do so. That isn't right now, however. I'll add Markus to the bug, > > though. > Hi Scott, > Do you still have plan to do it? Yes, I do, but my time is very limited right now...so I haven't been able to work on it. (In reply to comment #28) > (In reply to comment #27) > Anything I can do? Yes. The first thing is to create a new project in the Jenkins builder to regularly build the (still pre-release) httpclient4 provider. The httpclient4 provider has this feature in the ECF git repo: org.eclipse.ecf.provider.filetransfer.httpclient4.feature It should be possible to model it after the httpclient 3.1 jenkins project...which is here: https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient.feature/ I'm not sure of the Jenkins/Buckminster meta-data changes necessary to build the httpclient4 provider...this is where my knowledge of things about the builder/Buckminster is rusty. I think the thing to do would be to use the httpclient4 feature (above) rather than the httpclient.feature, but essentially copy the httpclient 3.1 Jenkins project metadata. Obviously, some of the Orbit bundles depended upon are different (i.e. the apache httpclient4/httpcomponents bundles...I can't remember right now how that's expressed in the Buckminster meta-data). For consistency, the new jenkins project should be named: C-HEAD-filetransfer.httpclient4.feature I (Scott) **may** get some time this week to work on the releng associated with this bug...as described in comment 29. Has there been any work on this over the past week? If so, please let all know what the state of things are so that I don't reproduce any work that's been done (e.g. creating new jenkins project, modifying httpclient4 meta-data to make it buildable, etc). I will probably need some help/input from Markus K or Wim J about the changes needed to the httpclient metadata (e.g. the inclusion/use of httpclient4 Orbit bundles, etc). Hopefully Markus K and/or Wim J will have some bandwidth for this soon. Thanks. (In reply to comment #30) > I (Scott) **may** get some time this week to work on the releng associated > with this bug...as described in comment 29. Has there been any work on this > over the past week? If so, please let all know what the state of things are > so that I don't reproduce any work that's been done (e.g. creating new > jenkins project, modifying httpclient4 meta-data to make it buildable, etc). I haven't had time to work on this at all. > I will probably need some help/input from Markus K or Wim J about the > changes needed to the httpclient metadata (e.g. the inclusion/use of > httpclient4 Orbit bundles, etc). Hopefully Markus K and/or Wim J will have > some bandwidth for this soon. Just keep asking here and I will try to answer. I have started the creation of the build job but I need to discuss something. If you look at our git repo[1] there seems to be a discrepancy in our features. org.eclipse.ecf.filetransfer.httpclient.feature org.eclipse.ecf.provider.filetransfer.httpclient.feature org.eclipse.ecf.provider.filetransfer.httpclient4.feature The first project is actually build by the regular http file transfer client job in Jenkins. My proposal is to rename the httpclient4 feature to match the naming convention of httpclient feature and remove provider...httpclient feature. http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/releng/features (In reply to comment #32) > I have started the creation of the build job but I need to discuss > something. > > If you look at our git repo[1] there seems to be a discrepancy in our > features. > > org.eclipse.ecf.filetransfer.httpclient.feature > org.eclipse.ecf.provider.filetransfer.httpclient.feature > org.eclipse.ecf.provider.filetransfer.httpclient4.feature > > The first project is actually build by the regular http file transfer client > job in Jenkins. > > My proposal is to rename the httpclient4 feature to match the naming > convention of httpclient feature and remove provider...httpclient feature. That seems OK to me. +1. Ok, I have done that. The new http4 client has been added to the sdk and can be consumed from the daily build at [1] I have also updated the site categories a little bit. I have separated core, remote services, file transfer and file transfer patch into separate categories. Scott please take a look if you want it like this. [1] ECF Target Components Feature - http://download.ecf-project.org/repo/N-HEAD-sdk.feature/lastSuccessful/archive/site.p2/ btw, some filetransfer tests are failing for htppclient4 (In reply to comment #34) > Ok, I have done that. The new http4 client has been added to the sdk and can > be consumed from the daily build at [1] > > I have also updated the site categories a little bit. I have separated core, > remote services, file transfer and file transfer patch into separate > categories. Scott please take a look if you want it like this. > > [1] ECF Target Components Feature - > http://download.ecf-project.org/repo/N-HEAD-sdk.feature/lastSuccessful/ > archive/site.p2/ Thanks very much Wim for the work on this. Just to clarify...for me...would you articulate which new features were added...and where they are in git repo? Also which ones are no longer used. >btw, some filetransfer tests are failing for htppclient4 I'm pretty sure that these are the tests that were failing for the old httpclient provider before the fix to bug 370801. What has to happen now is that the httpclient4 provider has to be fixed in a manner similar to that done to the httpclient 3.1 provider for bug 370801. I've created new bug 389292 to track the work on the needed changes to the httpclient4/httpcomponents provider. This new bug is a clone of 370801 with subject that refers to httpclient4/httpcomponents provider. + org.eclipse.ecf.filetransfer.httpclient4.feature - org.eclipse.ecf.provider.filetransfer.httpclient.feature - org.eclipse.ecf.provider.filetransfer.httpclient4.feature They are located at: http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/releng/features Ok...I needed to make some additions to the httpclient4 build project, and am running into some difficulties with the build/test environment. First, I had to create/add a new test launch config: org.eclipse.ecf.tests.filetransfer/org.eclipse.ecf.tests.filetransfer.httpclient4.launch This launch config differs from the httpclient one in that it depends upon some different orbit bundles. I updated the build project config to use this new launch config rather than the httpclient one. When I run this build project it fails to launch the tests launch config...i.e. see console output here https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/24/ It says this at the bottom: ... Doing full workspace refresh Waiting for jobs to end junit '--launch' 'org.eclipse.ecf.tests.filetransfer/org.eclipse.ecf.tests.filetransfer.httpclient4.launch' '-o' '/opt/hudson/jobs/C-HEAD-filetransfer.httpclient4.feature/workspace/junit_result.xml' '--flatXML' ERROR: The application could not start. Details can be found in the log. WARN: Process /opt/hudson/tools/Java1.4.2_update_18_x86-64/jre/bin/java (Sep 11, 2012 10:48:12 PM) terminated with exit status 13. Doing full workspace refresh Waiting for jobs to end java.lang.IllegalArgumentException: Export has failed because no test session is available. (Maybe the argument -maxTimeAwaitJunitReport will help) Build step 'Use builders from another project' marked build as failure Recording test results Archiving artifacts Sending e-mails to: ecf-build@eclipse.org Sending e-mails to: ecf-build@eclipse.org slewis@composent.com wim.jongman@remainsoftware.com Finished: FAILURE I consulted the .log file in the workspace .metadata area...but it was kind of uninformative...it has this at the bottom: !ENTRY org.eclipse.buckminster.core 0 293 2012-09-11 22:48:08.276 !MESSAGE Waiting for jobs to end !ENTRY org.eclipse.buckminster.runtime 0 293 2012-09-11 22:48:08.479 !MESSAGE junit '--launch' 'org.eclipse.ecf.tests.filetransfer/org.eclipse.ecf.tests.filetransfer.httpclient4.launch' '-o' '/opt/hudson/jobs/C-HEAD-filetransfer.httpclient4.feature/workspace/junit_result.xml' '--flatXML' !ENTRY org.eclipse.pde.launching 4 13 2012-09-11 22:48:15.400 !MESSAGE The application could not start. Details can be found in the log. !ENTRY org.eclipse.buckminster.core 2 293 2012-09-11 22:48:15.762 !MESSAGE Process /opt/hudson/tools/Java1.4.2_update_18_x86-64/jre/bin/java (Sep 11, 2012 10:48:12 PM) terminated with exit status 13. !ENTRY org.eclipse.buckminster.core 0 293 2012-09-11 22:48:25.771 !MESSAGE Doing full workspace refresh !ENTRY org.eclipse.buckminster.core 0 293 2012-09-11 22:48:25.839 !MESSAGE Waiting for jobs to end I think the error is in actually launching the junit framework/process...and I don't know where the log file is for that. The use of jdk 1.4.2 version doesn't seem right either...as I believe that 1.5 is required for use of httpcomponents. Finally, there's possibly a problem with org.apache.commons.logging. The version required by httpclient 3.1 is 1.0.4...and that's what we've been using until now. The version required by httpclient4/httpcomponents is 1.1.1. I see from the created/built p2 repo that the version included in the repo is 1.0.4...which is wrong. Further we have *test* code (a simple test server) that is based upon httpclient 3.1, and so the test code requires httpclient 3.1 and commons logging 1.0.4, while the httpclient4 provider requires commons logging 1.1.1. This may be the cause of the test runtime start failure. I'm not sure how to get past this issue, given that we have a need to run both the httpclient 3.1 and httpclient4 code and logging dependencies in the testing of the httpclient4 code. (In reply to comment #38) > This launch config differs from the httpclient one in that it depends upon > some different orbit bundles. Generally .launch configs should not depend on plugins/features explicitly. Component materialization is handled by Buckminster, meaning it creates a workspace that contains exactly what you want. launch should merely pick up what it is presented with. Otherwise you end up with redundant configuration in a) Buckminster's cspecs and b) the .launch files > I think the error is in actually launching the junit framework/process...and > I don't know where the log file is for that. Relative to the workspace it is in .metadata/.plugins/org.eclipse.pde.core/org.eclipse.ecf.tests.filetransfer.httpclient4/ > The use of jdk 1.4.2 version doesn't seem right either...as I believe that > 1.5 is required for use of httpcomponents. This is a red herring: mkuppe@ecf-builds /opt/hudson/tools $ ls -la lrwxrwxrwx 1 tomcat tomcat 42 Nov 18 2010 Java1.4.2_update_18_x86-64 -> /opt/hudson/tools/Java1.5_update_22_x86-64 drwxr-xr-x 9 tomcat tomcat 4096 Nov 1 2011 Java1.5_update_22_x86-64 drwxr-xr-x 10 tomcat tomcat 4096 Nov 4 2011 Java1.6_update_20_x86-64 > Finally, there's possibly a problem with org.apache.commons.logging. The > version required by httpclient 3.1 is 1.0.4...and that's what we've been > using until now. The version required by httpclient4/httpcomponents is > 1.1.1. I see from the created/built p2 repo that the version included in > the repo is 1.0.4...which is wrong. Further we have *test* code (a simple > test server) that is based upon httpclient 3.1, and so the test code > requires httpclient 3.1 and commons logging 1.0.4, while the httpclient4 > provider requires commons logging 1.1.1. This may be the cause of the test > runtime start failure. I'm not sure how to get past this issue, given that > we have a need to run both the httpclient 3.1 and httpclient4 code and > logging dependencies in the testing of the httpclient4 code. It looks as if 1.0.4 is dragged in by feature "org.eclipse.help" which explicitly depends on 1.0.4.v201101211617. I'm puzzled as to why resolution for 1.1.1 is discarded later on, even though Buckminster finds a match initially: org.apache.commons.logging:osgi.bundle: Trying provider p2(http://ftp.osuosl.org/pub/eclipse/tools/orbit/downloads/drops/R20120526062928/repository/[http://ftp.osuosl.org/pub/eclipse/tools/orbit/downloads/drops/R20120526062928/repository/]) org.apache.commons.logging:osgi.bundle: Found match 1.1.1.v201101211721 (In reply to comment #39) > It looks as if 1.0.4 is dragged in by feature "org.eclipse.help" which > explicitly depends on 1.0.4.v201101211617. I'm puzzled as to why resolution > for 1.1.1 is discarded later on, even though Buckminster finds a match > initially: > > org.apache.commons.logging:osgi.bundle: Trying provider > p2(http://ftp.osuosl.org/pub/eclipse/tools/orbit/downloads/drops/ > R20120526062928/repository/[http://ftp.osuosl.org/pub/eclipse/tools/orbit/ > downloads/drops/R20120526062928/repository/]) > org.apache.commons.logging:osgi.bundle: Found match 1.1.1.v201101211721 With help from Thomas Hallgren, I refresh my memory on the Bucky vs. import-package business. o.a.c.httpclient declares its o.a.c.logging dependency via an import-package statement. Thus, we need to explicitly state the version range somewhere (now in the httpclient4 bundle) to make Bucky aware of the it. logging 1.1.1 is now correctly materialized. (In reply to comment #39) > Generally .launch configs should not depend on plugins/features explicitly. > Component materialization is handled by Buckminster, meaning it creates a > workspace that contains exactly what you want. launch should merely pick up > what it is presented with. > Otherwise you end up with redundant configuration in a) Buckminster's cspecs > and b) the .launch files Tests are now executed (after purging the explicit plugin dependencies from the .launch config) [1]. [1] http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/commit/?id=e093fbfd8af1ef2c236e502adcd225aee571aecd (In reply to comment #41) > (In reply to comment #39) > > > Generally .launch configs should not depend on plugins/features explicitly. > > Component materialization is handled by Buckminster, meaning it creates a > > workspace that contains exactly what you want. launch should merely pick up > > what it is presented with. > > Otherwise you end up with redundant configuration in a) Buckminster's cspecs > > and b) the .launch files > > Tests are now executed (after purging the explicit plugin dependencies from > the .launch config) [1]. > > [1] > http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/commit/ > ?id=e093fbfd8af1ef2c236e502adcd225aee571aecd thanks Markus for the fixes. One more thing that I suspect is related to the launch config...with the latest build/test run: https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/lastCompletedBuild/ I'm getting 64 test errors and they all seem to be because of exceptions like this: org.eclipse.ecf.core.util.ECFRuntimeException: Unable to instantiate schemes for HttpClient. at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransfer.registerSchemes(HttpClientRetrieveFileTransfer.java:199) at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransfer.<init>(HttpClientRetrieveFileTransfer.java:176) at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransferFactory.newInstance(HttpClientRetrieveFileTransferFactory.java:22) ... This appears to be because the SSLSocketFactory can't be created...and I think this is because the httpclient4.ssl fragment is not present/not in launch config? I don't get this or any ssl problem (i.e. the SSLSocketFactory class loaded in the httpclient4 activator is loaded successfully). When I run the tests locally as the .ssl fragment is present and appears to be working/loaded when I run the ECF Filetransfer Tests- Httpclient4.launch config. Any idea why the httpclient4.ssl fragment classes apparently are not getting used/not found in the test run? (In reply to comment #42) > (In reply to comment #41) > > (In reply to comment #39) > > > > > Generally .launch configs should not depend on plugins/features explicitly. > > > Component materialization is handled by Buckminster, meaning it creates a > > > workspace that contains exactly what you want. launch should merely pick up > > > what it is presented with. > > > Otherwise you end up with redundant configuration in a) Buckminster's cspecs > > > and b) the .launch files > > > > Tests are now executed (after purging the explicit plugin dependencies from > > the .launch config) [1]. > > > > [1] > > http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/commit/ > > ?id=e093fbfd8af1ef2c236e502adcd225aee571aecd > > thanks Markus for the fixes. > > One more thing that I suspect is related to the launch config...with the > latest build/test run: > > https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4. > feature/lastCompletedBuild/ > > I'm getting 64 test errors and they all seem to be because of exceptions > like this: > > org.eclipse.ecf.core.util.ECFRuntimeException: Unable to instantiate schemes > for HttpClient. > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransfer. > registerSchemes(HttpClientRetrieveFileTransfer.java:199) > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransfer.<init>(HttpClientRetrieveFileTransfer.java: > 176) > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransferFactory. > newInstance(HttpClientRetrieveFileTransferFactory.java:22) > ... > > This appears to be because the SSLSocketFactory can't be created...and I > think this is because the httpclient4.ssl fragment is not present/not in > launch config? I don't get this or any ssl problem (i.e. the > SSLSocketFactory class loaded in the httpclient4 activator is loaded > successfully). When I run the tests locally as the .ssl fragment is > present and appears to be working/loaded when I run the ECF Filetransfer > Tests- Httpclient4.launch config. > > Any idea why the httpclient4.ssl fragment classes apparently are not getting > used/not found in the test run? The fragment is at least built by Bucky. https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/lastSuccessfulBuild/artifact/site.p2/plugins/ Here's the problem, the build executes the regular filetransfer tests instead of the httpclient4 ones: osgi> ss ecf Framework is launched. id State Bundle 47 <<LAZY>> org.eclipse.ecf_3.1.300.v20111230-0120 Fragments=54 48 <<LAZY>> org.eclipse.ecf.filetransfer_5.0.0.v20111230-0120 49 <<LAZY>> org.eclipse.ecf.identity_3.1.200.v20111230-0120 50 <<LAZY>> org.eclipse.ecf.provider.filetransfer_3.2.0.v20111230-0120 Fragments=53 51 <<LAZY>> org.eclipse.ecf.provider.filetransfer.httpclient_4.0.1.v20111230-0120 Fragments=52 52 RESOLVED org.eclipse.ecf.provider.filetransfer.httpclient.ssl_1.0.0.v20111230-0120 Master=51 53 RESOLVED org.eclipse.ecf.provider.filetransfer.ssl_1.0.0.v20111230-0120 Master=50 54 RESOLVED org.eclipse.ecf.ssl_1.0.100.v20111230-0120 Master=47 81 <<LAZY>> org.eclipse.equinox.p2.transport.ecf_1.0.100.v20120305-0333 Guess we have to find a way to blacklist "httpclient". Btw. remind me why "httpclient4" has been created instead of changing the major version of httpclient? (In reply to comment #44) > Here's the problem, the build executes the regular filetransfer tests > instead of the httpclient4 ones: > > osgi> ss ecf > Framework is launched. > id State Bundle > 47 <<LAZY>> org.eclipse.ecf_3.1.300.v20111230-0120 Fragments=54 > 48 <<LAZY>> org.eclipse.ecf.filetransfer_5.0.0.v20111230-0120 > 49 <<LAZY>> org.eclipse.ecf.identity_3.1.200.v20111230-0120 > 50 <<LAZY>> org.eclipse.ecf.provider.filetransfer_3.2.0.v20111230-0120 > Fragments=53 > 51 <<LAZY>> > org.eclipse.ecf.provider.filetransfer.httpclient_4.0.1.v20111230-0120 > Fragments=52 > 52 RESOLVED > org.eclipse.ecf.provider.filetransfer.httpclient.ssl_1.0.0.v20111230-0120 > Master=51 > 53 RESOLVED > org.eclipse.ecf.provider.filetransfer.ssl_1.0.0.v20111230-0120 Master=50 > 54 RESOLVED org.eclipse.ecf.ssl_1.0.100.v20111230-0120 Master=47 > 81 <<LAZY>> org.eclipse.equinox.p2.transport.ecf_1.0.100.v20120305-0333 > > Guess we have to find a way to blacklist "httpclient". This shouldn't be necessary. The protocol factory extension points (i.e. browseFileTransferProtocolFactory, retrieveFileTransferProtocolFactory) have a priority mechanism to allow multiple providers (of the same protocol) to coexist. Let me look at the prioritization mechanism and see what's going on. Strangely, in my local workspace, all seems well with this (i.e. the httpclient4 provider is preferred over the httpclient 3.1 provider...because both are present and httpclient4 is preferred. Are both/all the provider plugins i.e. httpclient 3.1 and httpclient4...in the test runtime? Or is just the httpclient 3.1? > > Btw. remind me why "httpclient4" has been created instead of changing the > major version of httpclient? Since the httpclient 3.1 provider is depended upon by the platform...and there's going to be lots of testing, etc needed before we and others move to httpclient4, and httpclient4 is very different...I think it will be easier to manage if we and others are able to build, test, run these as separate providers...rather than a new major version of the same provider. (In reply to comment #45) > This shouldn't be necessary. The protocol factory extension points (i.e. > browseFileTransferProtocolFactory, retrieveFileTransferProtocolFactory) have > a priority mechanism to allow multiple providers (of the same protocol) to > coexist. Let me look at the prioritization mechanism and see what's going > on. > > Strangely, in my local workspace, all seems well with this (i.e. the > httpclient4 provider is preferred over the httpclient 3.1 provider...because > both are present and httpclient4 is preferred. > > Are both/all the provider plugins i.e. httpclient 3.1 and httpclient4...in > the test runtime? Or is just the httpclient 3.1? It appears as if the rest (httpclient4 ones) are picked up by dev mode. Don't know if this is what is causing Equinox to not list them. (In reply to comment #46) > (In reply to comment #45) > > This shouldn't be necessary. The protocol factory extension points (i.e. > > browseFileTransferProtocolFactory, retrieveFileTransferProtocolFactory) have > > a priority mechanism to allow multiple providers (of the same protocol) to > > coexist. Let me look at the prioritization mechanism and see what's going > > on. > > > > Strangely, in my local workspace, all seems well with this (i.e. the > > httpclient4 provider is preferred over the httpclient 3.1 provider...because > > both are present and httpclient4 is preferred. > > > > Are both/all the provider plugins i.e. httpclient 3.1 and httpclient4...in > > the test runtime? Or is just the httpclient 3.1? > > It appears as if the rest (httpclient4 ones) are picked up by dev mode. > Don't know if this is what is causing Equinox to not list them. Here is what I have found in the trace output. Looks like httpclient4.ssl is there. java.io.IOException: Cannot get socket factory at org.eclipse.ecf.internal.provider.filetransfer.httpclient4.ssl.SSLSocketFactoryModifier.getSSLSocketFactory(SSLSocketFactoryModifier.java:31) at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransfer.registerSchemes(HttpClientRetrieveFileTransfer.java:195) at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransfer.<init>(HttpClientRetrieveFileTransfer.java:176) at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientRetrieveFileTransferFactory.newInstance(HttpClientRetrieveFileTransferFactory.java:22) at org.eclipse.ecf.internal.provider.filetransfer.Activator.getFileTransfer(Activator.java:608) at org.eclipse.ecf.provider.filetransfer.retrieve.MultiProtocolRetrieveAdapter.sendRetrieveRequest(MultiProtocolRetrieveAdapter.java:92) at org.eclipse.ecf.tests.filetransfer.URLRetrieveTest.testReceive(URLRetrieveTest.java:132) at org.eclipse.ecf.tests.filetransfer.URLRetrieveTest.testReceiveFile1(URLRetrieveTest.java:169) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:592) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at org.eclipse.ecf.tests.filetransfer.AbstractFileTransferTestCase.runBare(AbstractFileTransferTestCase.java:38) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62) at org.eclipse.pde.internal.junit.runtime.CoreTestApplication.run(CoreTestApplication.java:23) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:592) at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:592) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584) at org.eclipse.equinox.launcher.Main.run(Main.java:1438) at org.eclipse.equinox.launcher.Main.main(Main.java:1414) (In reply to comment #47) <stuff deleted> > Here is what I have found in the trace output. Looks like httpclient4.ssl is > there. > > java.io.IOException: Cannot get socket factory > at > org.eclipse.ecf.internal.provider.filetransfer.httpclient4.ssl. > SSLSocketFactoryModifier.getSSLSocketFactory(SSLSocketFactoryModifier.java: > 31) > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransfer. > registerSchemes(HttpClientRetrieveFileTransfer.java:195) ...<rest of stack trace deleted> Hmmm. This is the code here (31 is the line with the throw): public SSLSocketFactory getSSLSocketFactory() throws IOException { final SSLSocketFactory factory = Activator.getDefault().getSSLSocketFactory(); if (factory == null) throw new IOException("Cannot get socket factory"); //$NON-NLS-1$ return factory; } This means that Activator.getDefault().getSSLSocketFactory() is returning null. Here's the code in getSSLSocketFactory(): public SSLSocketFactory getSSLSocketFactory() { if (sslSocketFactoryTracker == null) { sslSocketFactoryTracker = new ServiceTracker(this.context, SSLSocketFactory.class.getName(), null); sslSocketFactoryTracker.open(); } return (SSLSocketFactory) sslSocketFactoryTracker.getService(); } which obviously gets the SSLSocketFactory service instance. This instance is created/registered when the org.eclipse.ecf.ssl bundle is started...i.e. here: org.eclipse.ecf.internal.ssl.ECFTrustManager.start(BundleContext) It seems as if the SSLSocketFactory service is not getting registered...and therefore the org.eclipse.ecf.ssl fragment is not getting started? I can't immediately tell why this is not happening in the test environment...as it looks like the org.eclipse.ecf.ssl fragment is present. Is it possible that it 'org.eclipse.ecf.ssl' and/or 'org.eclipse.ecf.provider.filetransfer.ssl' are not there for the test? I've verified the SSLSocketFactory service does get registered for me locally in test runs. Any ideas? > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransfer.<init>(HttpClientRetrieveFileTransfer. > java:176) > at > org.eclipse.ecf.provider.filetransfer.httpclient4. > HttpClientRetrieveFileTransferFactory. > newInstance(HttpClientRetrieveFileTransferFactory.java:22) > at > org.eclipse.ecf.internal.provider.filetransfer.Activator. > getFileTransfer(Activator.java:608) > at > org.eclipse.ecf.provider.filetransfer.retrieve.MultiProtocolRetrieveAdapter. > sendRetrieveRequest(MultiProtocolRetrieveAdapter.java:92) > at > org.eclipse.ecf.tests.filetransfer.URLRetrieveTest. > testReceive(URLRetrieveTest.java:132) > at > org.eclipse.ecf.tests.filetransfer.URLRetrieveTest. > testReceiveFile1(URLRetrieveTest.java:169) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl. > java:25) > at java.lang.reflect.Method.invoke(Method.java:592) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at > org.eclipse.ecf.tests.filetransfer.AbstractFileTransferTestCase. > runBare(AbstractFileTransferTestCase.java:38) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference. > run(JUnit3TestReference.java:130) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java: > 38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. > runTests(RemoteTestRunner.java:467) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. > runTests(RemoteTestRunner.java:683) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner. > java:390) > at > org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner. > main(RemotePluginTestRunner.java:62) > at > org.eclipse.pde.internal.junit.runtime.CoreTestApplication. > run(CoreTestApplication.java:23) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl. > java:25) > at java.lang.reflect.Method.invoke(Method.java:592) > at > org.eclipse.equinox.internal.app.EclipseAppContainer. > callMethodWithException(EclipseAppContainer.java:587) > at > org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java: > 198) > at > org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher. > runApplication(EclipseAppLauncher.java:110) > at > org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher. > start(EclipseAppLauncher.java:79) > at > org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353) > at > org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl. > java:25) > at java.lang.reflect.Method.invoke(Method.java:592) > at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629) > at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584) > at org.eclipse.equinox.launcher.Main.run(Main.java:1438) > at org.eclipse.equinox.launcher.Main.main(Main.java:1414) I think I've got it. I added these two fragments: org.eclipse.ecf.ssl org.eclipse.ecf.provider.filetransfer.ssl to the feature: org.eclipse.ecf.tests.filetransfer and now the httpclient4 tests are all passing just fine: https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/44/ Does anyone see anything wrong with this? (including these fragments in the feature simply makes sure that they appear in the workspace, and that they also appear in the test runtime environment). And when these fragments are present, the SSLSocketFactory is registered as a service (by the org.eclipse.ecf.ssl fragment), and so the tests all pass now. Yay. If this arrangement sits ok with everyone then I'll resolve this bug. Note that as of now people can get the httpclient4/httpcomponents build repo here: https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/ or http://download.ecf-project.org/repo/C-HEAD-filetransfer.httpclient4.feature/lastSuccessful/ (In reply to comment #49) > Does anyone see anything wrong with this? (including these fragments in the > feature simply makes sure that they appear in the workspace, and that they > also appear in the test runtime environment). And when these fragments are > present, the SSLSocketFactory is registered as a service (by the > org.eclipse.ecf.ssl fragment), and so the tests all pass now. Yay. The tests.feature is not intended to be part of our distribution. We can include whatever we want. That said, why don't we include org.eclipse.ecf.ssl, org.eclipse.ecf.provider.filetransfer.ssl in the httpclient4 feature. After all, they seem to be essentially required (most consumers probably want SSL support). (In reply to comment #50) > That said, why don't we include > org.eclipse.ecf.ssl, > org.eclipse.ecf.provider.filetransfer.ssl in the httpclient4 feature. After > all, they seem to be essentially required (most consumers probably want SSL > support). If they are both used by httpclient and httpclient4 we might get into p2 resolution problems if the two are build separately. Message like: htppclient4 requires ...ssl.20121010 but another version is already installed In that case a filetransfer.ssl.feature is required that gets imported by both httpclient* features. In either case, the ssl plugins need to be referenced by httpclient4 either included or as a dependend feature. (In reply to comment #51) > (In reply to comment #50) > > That said, why don't we include > > org.eclipse.ecf.ssl, > > org.eclipse.ecf.provider.filetransfer.ssl in the httpclient4 feature. After > > all, they seem to be essentially required (most consumers probably want SSL > > support). > > If they are both used by httpclient and httpclient4 we might get into p2 > resolution problems if the two are build separately. Message like: > > htppclient4 requires ...ssl.20121010 but another version is already installed Since these .ssl fragments will likely be coming with Eclipse...won't this become a problem if these fragements are actually explicitly included in httpclient4? > > In that case a filetransfer.ssl.feature is required that gets imported by > both httpclient* features. > > In either case, the ssl plugins need to be referenced by httpclient4 either > included or as a dependend feature. I would be ok with creating a filetransfer.ssl.feature...although I'm not sure how it would affect the existing filetransfer features...and how it would affect p2/platform...as well as installing into Eclipse (given the existing feature structure). Hey everyone, I've looked at this issue and I strongly believe that SSL support deserves its own feature, with direct requirement from org.eclipse.ecf.filetransfer.httpclient.feature and org.eclipse.ecf.filetransfer.httpclient.feature4 to it. Actually, the platform itself (P2) does not install any of those features - it includes only plugins :-) Having a strong dependency between those features secures installation via P2 into platform - it will not be anymore complicated than it is now (just two features will be installed instead of now). Actually, I think that this bug may be closed as soon as the feature and dependencies are created... Shall I provide a patch for that? (In reply to comment #53) > Hey everyone, > > I've looked at this issue and I strongly believe that SSL support deserves > its own feature, with direct requirement from > org.eclipse.ecf.filetransfer.httpclient.feature and > org.eclipse.ecf.filetransfer.httpclient.feature4 to it. > > Actually, the platform itself (P2) does not install any of those features - > it includes only plugins :-) > > Having a strong dependency between those features secures installation via > P2 into platform - it will not be anymore complicated than it is now (just > two features will be installed instead of now). > > > Actually, I think that this bug may be closed as soon as the feature and > dependencies are created... > > Shall I provide a patch for that? Hi Daniel, The httpclient4 provider does include support for ssl...it's in the org.eclipse.ecf.filetransfer.httpclient4.ssl fragment. This fragment is included in the o.e.e.f.httpclient4.feature feature. The reason that this ssl support code is in a fragment is historical...the reason being that originally, the requirement was that the ECF filetransfer code had to run on CDC 1.0 execution environment, which didn't/doesn't include javax.security. In our own test code, this ssl code seems to be operating fine, although admittedly we haven't had lots of opportunity to test under different network conditions (proxies particularly). If you would like to review, update, or otherwise improve/extend whats there, you are most welcome to do so. I am not the author of the *.ssl code...rather it was authored by some folks at IBM...I think including Tom Watson. See bug 224196 and bug 263724. In any case, the source code for the httpclient4 plugins and ssl fragments are located here: http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/providers/bundles If you are intending other changes/additions to p2 WRT security (not just ssl transport for file retrieve) then I think it should probably be done within the p2 domain rather than ECF/filetransfer. Hey Scott, There may be some miscommunication going here. I have noticed that httpclient4 feature has no org.eclipse.ecf.ssl bundle, which httpclient has (and I think this is also desribed in comments 47-49). I pointed that this situation may cause problems for users that will install httpclient4 feature only - as not all defacto required dependencies will be fetched. Another question is: what else needs to be done to close this bug? I'm actually interested in working on bug 337449 which is actually about removing discontinued org.apache.commons.httpclient from platform (P2), which is blocked by this bug. Hi Scott, (In reply to comment #55) > Hey Scott, > > There may be some miscommunication going here. > > I have noticed that httpclient4 feature has no org.eclipse.ecf.ssl bundle, > which httpclient has (and I think this is also desribed in comments 47-49). > > I pointed that this situation may cause problems for users that will install > httpclient4 feature only - as not all defacto required dependencies will be > fetched. Ok. That's just a bug in the httpclient4 feature. One of the releng issues that ECF has to deal with is that since p2/Eclispe distributes some bundles from ECF (e.g. org.eclipse.ecf.ssl) in p2 features, the creation of our other features is complicated somewhat. I've opened bug 397594 to track this. > > Another question is: > what else needs to be done to close this bug? I would say testing...particularly of proxy scenarios...although I'm open to doing that on another bug...if people would rather do that. > > I'm actually interested in working on bug 337449 which is actually about > removing discontinued org.apache.commons.httpclient from platform (P2), > which is blocked by this bug. Right. I certainly want to move on to httpclient4...and we have a codebase (aka ECF provider) for doing so. However, from my chair there needs to be some additional testing of the httpclient4 provider...before we even put it in a p2 milestone...because there could be regression problems in httpclient4...and/or the new provider code. I'll try to build Eclipse with httpclient4 and make it available for testing purposes. In response to the request posted at http://bit.ly/WuR9B6 I have downloaded the client for macosx.cocoa.x86_64 and proceeded to download my own plugin product from its update site that requires GEF as well. It all worked without any problems. Thank you Christopher! Tried the sdk_win32.win32.x86 build from fedorapeople, and installed a few features - through a proxy, worked fine (I confirmed it did pick up the proxy settings) (In reply to comment #59) > Tried the sdk_win32.win32.x86 build from fedorapeople, and installed a few > features - through a proxy, worked fine (I confirmed it did pick up the > proxy settings) Thanks for your report. Do you know which type of proxy it was? Did the proxy require authentication? (In reply to comment #60) > (In reply to comment #59) > > Tried the sdk_win32.win32.x86 build from fedorapeople, and installed a few > > features - through a proxy, worked fine (I confirmed it did pick up the > > proxy settings) > Thanks for your report. Do you know which type of proxy it was? Did the > proxy require authentication? Sorry - no and no :-( Guess i could set up a proper proxy instead. Do you know of any update sites which may be using https? Repository of ADT https://dl-ssl.google.com/android/eclipse/ (In reply to comment #61) > > Guess i could set up a proper proxy instead. Do you know of any update sites > which may be using https? FYI - it seems BASIC auth does not work with the httpclient4 provider, see bug 402740 Is there anything left to do here? BTW, I don't believe that this bug depends on bug 334937 (a Mylyn bug). (In reply to comment #64) > Is there anything left to do here? > Well, there's always testing :) More seriously, I just want to be explicit, org.eclipse.ecf.provider.filetransfer.httpclient4 is the only ecf httpclient to be used, right? I still see org.eclipse.ecf.provider.filetransfer.httpclient in some bundle manifests, and am assuming they are just old ones that are not used or built any longer. (I "see" them, just by searching what I happen to have in my workspace, I haven't checked a built SDK or our repository yet ... just wanted to be sure I was clear on the concept. Thanks. I would be inclined to leave this bug open (and more importantly bug 337449 as well) while testing goes on. If the past is any guide, there will be individual/new issues with proxy handling (in particular) once experience on a larger number of networks is obtained. Removed dependency on mylyn bug. Not sure how it got there. WRT including the old (httpclient 3.1 provider). It's the platform's choice, of course, but it might be worthwhile to continue to include the 3.1-based provider (and the url-connection-based provider) for at least a little while...so that in case emergencies are found that people can downshift to using the 3.1 provider. The Apache httpclient codebase, however, is > 500k...so that could be an issue. Resolving as fixed. As per bug 337449 the httpclient 4.1-based provider is now being used in p2/Equinox/Eclipse. |