Bugzilla – Full Text Bug Listing
| Summary: | Content-Length not passed to wrapped response in GZipFilter | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [RT] Jetty | Reporter: | Mike Pilone <mpilone> | ||||||
| Component: | server | Assignee: | Michael Gorovoy <mgorovoy> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | normal | ||||||||
| Priority: | P3 | CC: | edi.weissmann+eclipse, janb, jesse.mcconnell, jetty-inbox, mgorovoy | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | 7.5.x | ||||||||
| Hardware: | PC | ||||||||
| OS: | Windows Vista | ||||||||
| Whiteboard: | |||||||||
| Attachments: |
|
||||||||
|
Description
Mike Pilone
Will look into this shortly. Upon looking over the GzipFilter code, it appears that if the MIME type of the requested content is added to 'exclude' init parameter of the filter, it would not even enter the code you are referring to (see GzipFilter.java, line 129). Please configure this parameter in your application while I'm further investigating this issue. After looking further into this issue, I was able to confirm that GzipFilter correctly passes Content-Length header. Method GzipResponseWrapper.getOutputStream() (lines 294-307) is called from line 248 of method GzipStream.doNotGzip() (lines 242-261) therefore the output stream is always present when line 252 inside the latter method is processed. I've committed my test harness that tests for this condition that you could examine here: http://goo.gl/uIcVu. Please feel free to re-open this bug if you are going to be able to create a test harness that demonstrate the issue you are experiencing. The problem appears to be related to the size of the response. If the response gets large enough and forces a buffer flush, the problem appears. I modified the original test case to show the issue. The modified test case uses a simple ContentLengthServlet to keep things extremely simple. It also has a simple loop that generates a much larger block of content, which more accurately represents what happens when a real mp3/mp2 file is retrieved. These files are usually in the 5MB - 30MB range. Interestingly the test passes if the doGetStreamBeforeContentLength method is used in the test ContentLenghServlet rather than the doGetStreamAfterContentLength. This is because of the way the GZipFilter handles the content-length. If the internal GZipStream is created before the content length header is set, the header is properly passed to the original response. However if the determination of noGzip is done before the GZipStream is created, the content-length header is not passed correctly. Note: you could probably do this with the DefaultServlet, but I wanted to keep it simple for my own debugging. -mike Created attachment 201075 [details]
Slightly modified test case with larger test data set.
Created attachment 201076 [details]
Simple servlet which demonstrates that the results change based on the order of the getOutputStream call.
I looked into using the exclude option. It isn't clearly documented but the actual init parameter is called "excludedAgents" and it is based on UserAgent and not mime-type. This should probably be added to the GzipFilter class docs. In this particular case it might actually be a decent workaround because I know all the failing requests are coming from Windows Media Player for streaming but in the long run it wouldn't help other clients who aren't getting a Content-Length header. Thanks for looking into it so quickly. You are correct, the parameter name is 'excludeAgents' and it takes user-agent values. Sorry for the confusion. What causes the issue that you are experiencing to occur is that your servlet is setting response headers, either directly or indirectly, *before* it calls Response.getOutputStream(). This effectively disables GzipFilter because the response is already committed by the time GzipResponseWrapper could create a GzipOutputStream. Thus GzipResponseWrapper bails out at line 299. I'm going to add code to set the content length in wrapped Response object if the wrapper has to bail out. You will need to modify any of your servlets that set request headers before calling Response.getOutputStream() or Response.getWriter() to set them afterwards in order to be able to take full advantage of GzipFilter's capabilities. -Michael For the record, the magic content length value is 8192 and it is the size of internal buffer inside GzipResponseWrapper. If the content length is less then the size of the buffer, the decision 'to gzip, or not to gzip' is made at the time the stream is closed. Otherwise, it is made as soon as buffer size is exceeded. Depending on the sequence of calls made to Response.getOutputStream()/getWriter() as well as Response.setContentType() and Response.setContentLength() the GzipFilter was failing to pass the content length at two different places inside two different classes. I've finally been able to track it all down and fix it for good. Hi, In which jetty version is this bug fixed? I'm still able to reproduce it in 7.5.4 Thanks, Eduard going by dates 7.5.4 was out 24 10 2012 so I would think it would be in that release. did you verify that the comments in 8 and 9 did not apply to your issue?...if yes the feel free to reopen with additional information Err, that was a year earlier: 24 10 2011 :) Jan (In reply to comment #11) > going by dates 7.5.4 was out 24 10 2012 so I would think it would be in that > release. > > did you verify that the comments in 8 and 9 did not apply to your > issue?...if yes the feel free to reopen with additional information |