Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 330198 - Select channel connector fails to download HTTPS page.
Summary: Select channel connector fails to download HTTPS page.
Status: RESOLVED WORKSFORME
Alias: None
Product: Jetty
Classification: RT
Component: client (show other bugs)
Version: 7.2.0   Edit
Hardware: PC Windows Server 2008
: P3 major (vote)
Target Milestone: 7.2.x   Edit
Assignee: Jan Bartel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-14 09:15 EST by Matt Painter CLA
Modified: 2011-09-20 13:24 EDT (History)
4 users (show)

See Also:


Attachments
Debug log (266.25 KB, text/plain)
2010-11-14 09:15 EST, Matt Painter CLA
no flags Details
Log of the hanging HTTPS request with javax.debug=all (22 bytes, application/zip)
2011-01-13 05:38 EST, Matt Painter CLA
no flags Details
Log of hanging HTTPS request (308.46 KB, text/plain)
2011-05-18 08:58 EDT, Matt Painter CLA
no flags Details
jetty-client-example project (1.76 KB, application/x-gzip)
2011-09-20 13:18 EDT, Joakim Erdfelt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Painter CLA 2010-11-14 09:15:57 EST
Created attachment 183093 [details]
Debug log

When using HttpClient.CONNECTOR_SELECT_CHANNEL, downloading URL:

https://foretagsfakta.bolagsverket.se/fpl-dft-ext-web/home.seam?rad=16&actionMethod=home.xhtml%3Adokumentmanager.hamtaArtikelkategorier&cid=5084

hangs.

Using the socket channel the download completes successfully.
Comment 1 Jan Bartel CLA 2010-11-14 22:05:53 EST
I'm downgrading this from critical to major, as this is working under unix. It may be a windows-specific ssl issue.

Greg, can you test under windows?

thanks,
Jan
Comment 2 Jesse McConnell CLA 2011-01-12 12:35:10 EST
can you rerun this debug log with -Djavax.net.debug=all

that should give us an idea what is going on with this ssl handshake issue

post that log and we and take a further look

cheers
Comment 3 Matt Painter CLA 2011-01-13 04:27:59 EST
I can no longer replicate this with 8.0.0M2
Comment 4 Matt Painter CLA 2011-01-13 04:44:37 EST
I can no longer replicate this with 8.0.0M2(In reply to comment #3)
> Apologies, I can. It seems someone has patched Jetty internally, let me try to track down a patch file for what they have done.
Comment 5 Matt Painter CLA 2011-01-13 05:38:12 EST
Created attachment 186712 [details]
Log of the hanging HTTPS request with javax.debug=all

Log of the failing HTTPS download, as requested.
Comment 6 Matt Painter CLA 2011-01-13 05:39:48 EST
(In reply to comment #4)
> I can no longer replicate this with 8.0.0M2(In reply to comment #3)
> > Apologies, I can. It seems someone has patched Jetty internally, let me try to track down a patch file for what they have done.

The patch internally was using the SOCKET connector, which makes it irrelevant. Tried switching back to SELECT and same behaviour. Just posted the resulting log.
Comment 7 Greg Wilkins CLA 2011-02-23 00:16:08 EST
Matt,

we have made a number of SSL inspired changes to the select handling because of other lock ups found.

Could you retest 7.3.0 (or better yet trunk) and confirm/deny you are still seeing this.

I will close for now, reopen if you confirm it
Comment 8 Matt Painter CLA 2011-05-18 08:53:03 EDT
This is still happening sometimes with 8.0.0M2.

Another URL which it is occurring for is:

https://www.registro-publico.gob.pa/scripts/nwwisapi.dll/conweb/MESAMENU?TODO=MER4&FROM=&TO=&START=1&ID=

This link between these two URLs is that they both use Connection:close in the headers.

Attaching a new log.
Comment 9 Matt Painter CLA 2011-05-18 08:58:31 EDT
Created attachment 195967 [details]
Log of hanging HTTPS request
Comment 10 Greg Wilkins CLA 2011-07-11 03:23:11 EDT
Jan,

many SSL issues have been fixed in jetty-7.4.4, which should now be merged with your latest jetty-8 trunk.  Can you produce a snapshot release to test
Comment 11 Jan Bartel CLA 2011-07-11 22:11:24 EDT
Matt et al,

I have pushed a snapshot: jetty-8.0.0.RC0-SNAPSHOT.

Ready for you to re-test.

thanks
Jan
Comment 12 Jan Bartel CLA 2011-07-28 22:43:20 EDT
Matt,

Can you please re-test with the indicated snapshot of jetty-8?

thanks
Jan
Comment 13 Matt Painter CLA 2011-07-29 05:16:23 EDT
Sure, I will look at this next week.

(In reply to comment #12)
> Matt,
> 
> Can you please re-test with the indicated snapshot of jetty-8?
> 
> thanks
> Jan
Comment 14 Jan Bartel CLA 2011-08-15 02:22:06 EDT
Matt,

Any feedback?

thanks
Jan
Comment 15 Matt Painter CLA 2011-08-15 05:48:08 EDT
Apologies, this got pushed out due to other work commitments.

It is in my work stack, but not for this week. I will try to schedule it for next week.

Many thanks

(In reply to comment #14)
> Matt,
> 
> Any feedback?
> 
> thanks
> Jan
Comment 16 Matt Painter CLA 2011-08-18 13:08:36 EDT
(In reply to comment #15)

I've managed to have a quick look at this, and it currently isn't great news.

with 8.0.0-RC0 I cannot download the first page I tried:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=298502

either directly or through a proxy. It just hangs.

If I enable a proxy, I do not see the request come through.

What's more, it seems to be using a SOCKET conenction, even though client.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL); is called:

ssl - [Session-1, SSL_NULL_WITH_NULL_NULL] channel=java.nio.channels.SocketChannel

If I can provide any other feedback do let me know :)

Many thanks

Matt

17:03:54.282 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Downloading https://bugs.eclipse.org/bugs/show_bug.cgi?id=298502
17:03:54.310 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Exchange : DownloadExchange@1b26af3=GET//bugs.eclipse.org:443/bugs/show_bug.cgi?id=298502#START(1313687034311ms)
17:03:54.445 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Exchange sent
17:03:54.446 [qtp32582734-8] DEBUG org.eclipse.jetty.util.log - Channels with connection pending: 0
17:03:54.473 [qtp32582734-8] DEBUG ssl - [Session-1, SSL_NULL_WITH_NULL_NULL] channel=java.nio.channels.SocketChannel[connected local=/192.168.0.31:56243 remote=bugs.eclipse.org/206.191.52.49:443]
Comment 17 Jan Bartel CLA 2011-09-04 20:44:04 EDT
Matt,

We've released jetty-8.0.0 in the meanwhile. Can you try with that, and then post your code and the debug log output?

thanks
Jan

(In reply to comment #16)
> (In reply to comment #15)
> 
> I've managed to have a quick look at this, and it currently isn't great news.
> 
> with 8.0.0-RC0 I cannot download the first page I tried:
> 
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=298502
> 
> either directly or through a proxy. It just hangs.
> 
> If I enable a proxy, I do not see the request come through.
> 
> What's more, it seems to be using a SOCKET conenction, even though
> client.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL); is called:
> 
> ssl - [Session-1, SSL_NULL_WITH_NULL_NULL]
> channel=java.nio.channels.SocketChannel
> 
> If I can provide any other feedback do let me know :)
> 
> Many thanks
> 
> Matt
> 
> 17:03:54.282 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Downloading
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=298502
> 17:03:54.310 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Exchange :
> DownloadExchange@1b26af3=GET//bugs.eclipse.org:443/bugs/show_bug.cgi?id=298502#START(1313687034311ms)
> 17:03:54.445 [main] DEBUG c.k.g.downloader.impl.AsyncDownload - Exchange sent
> 17:03:54.446 [qtp32582734-8] DEBUG org.eclipse.jetty.util.log - Channels with
> connection pending: 0
> 17:03:54.473 [qtp32582734-8] DEBUG ssl - [Session-1, SSL_NULL_WITH_NULL_NULL]
> channel=java.nio.channels.SocketChannel[connected local=/192.168.0.31:56243
> remote=bugs.eclipse.org/206.191.52.49:443]
Comment 18 Joakim Erdfelt CLA 2011-09-20 13:14:56 EDT
Tests on Windows 7.

$ mvn --version
Apache Maven 3.0.2 (r1056850; 2011-01-08 17:58:10-0700)
Java version: 1.6.0_27, vendor: Sun Microsystems Inc.
Java home: C:\Program Files\Java\jdk1.6.0_27\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"


$ java -jar target/jetty-client-examples-0.0.1-SNAPSHOT-shaded.jar "https://foretagsfakta.bolagsverket.se/fpl-dft-ext-web/home.seam?rad=16&actionMethod=home.xhtml%3Adokumentmanager.hamtaArtikelkategorier&cid=5084" bolagsverket.se.log
Writing to file: C:\code\jetty-client-examples\bolagsverket.se.log
Sending request
Waiting...
[RESP STATUS] HTTP/1.1 302 Moved Temporarily
[RESP HEADER] Date: Tue, 20 Sep 2011 17:10:41 GMT
[RESP HEADER] Server: Apache
[RESP HEADER] X-Powered-By: Servlet 2.4; JBoss-4.3.0.GA_CP03 (build: SVNTag=JBPAPP_4_3_0_GA_CP03 date=200810241616)/JBossWeb-2.0
[RESP HEADER] Set-Cookie: JSESSIONID=2163D50A6EB9646C7733F35220506254.prod_fpl_node01; Path=/; Secure
[RESP HEADER] X-Powered-By: JSF/1.2
[RESP HEADER] Location: https://foretagsfakta.bolagsverket.se/fpl-dft-ext-web/error.seam;jsessionid=2163D50A6EB9646C7733F35220506254.prod_fpl_node01?cid=53326
[RESP HEADER] Content-Length: 0
[RESP HEADER] Content-Type: text/plain
Download Complete
Downloaded 0 bytes
Done

No failure, but it didn't download (as the URL results in a redirect)

$ java -jar target/jetty-client-examples-0.0.1-SNAPSHOT-shaded.jar "https://foretagsfakta.bolagsverket.se/fpl-dft-ext-web/error.seam;jsessionid=2163D50A6EB9646C7733F35220506254.prod_fpl_node01?cid=53326" bolagsverket.se.log
Writing to file: C:\code\jetty-client-examples\bolagsverket.se.log
Sending request
Waiting...
[RESP STATUS] HTTP/1.1 200 OK
[RESP HEADER] Date: Tue, 20 Sep 2011 17:13:38 GMT
[RESP HEADER] Server: Apache
[RESP HEADER] X-Powered-By: Servlet 2.4; JBoss-4.3.0.GA_CP03 (build: SVNTag=JBPAPP_4_3_0_GA_CP03 date=200810241616)/JBossWeb-2.0
[RESP HEADER] X-Powered-By: JSF/1.2
[RESP HEADER] Transfer-Encoding: chunked
[RESP HEADER] Content-Type: text/html;charset=UTF-8
Download Complete
Downloaded 5,201 bytes
Done

The redirected / Location URL works just fine.
Comment 19 Joakim Erdfelt CLA 2011-09-20 13:18:36 EDT
Created attachment 203701 [details]
jetty-client-example project

Adding sample project - /jetty-client-examples/ used for testing.
Comment 20 Joakim Erdfelt CLA 2011-09-20 13:24:30 EDT
Sample project on Linux & Windows & OSX with both stated URLs work on latest Jetty 7.x and Jetty 8.x trees.