Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 313639 - File Upload Problem with large files (> 1,5GB)
Summary: File Upload Problem with large files (> 1,5GB)
Status: RESOLVED NOT_ECLIPSE
Alias: None
Product: Jetty
Classification: RT
Component: server (show other bugs)
Version: unspecified   Edit
Hardware: PC All
: P3 critical (vote)
Target Milestone: 7.0.2.RC0   Edit
Assignee: Greg Wilkins CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-19 19:45 EDT by Jose M. Alcaraz CLA
Modified: 2010-05-24 07:46 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jose M. Alcaraz CLA 2010-05-19 19:45:56 EDT
Build Identifier: Jetty 7.0.2

The problem is the way in which the server manages the sockets. It believes on the "content-size" value provided by the browser. However, if you submit large files (more than 1,5Gb) on IE, Safari and Filezila this content-lenght has a wrong value. (There is no problems with Chrome). 

Then, the Jetty server reset the client communication in between of the file submission causing a total lost of the information. It happen around 1'5 Gb so in case you like to reproduce this error. You may submit a 2 Gb file using any of these browsers. 

Reproducible: Always

Steps to Reproduce:
1. Create a POST Form with the "file" field. 
2. Create a servlet to response with a dummy page but use the multi-part filter to manage the file upload ( I have tried both with and without filter doing the manual parsing of the multi-part encoding and both fails)
3. Submit a 1,6 GB file or more 
4. As a result, you can see how the file size is being increated on the server side. However, after 1,5GB the size of the file suddently become 0 bytes.
Comment 1 Greg Wilkins CLA 2010-05-20 04:35:31 EDT
Jose,

are you saying that IE, safari and Filezilla are sending a wrong content-length,
or that Jetty is reading the content-length incorrectly?

If an incorrect content-length is sent, then there is not much that Jetty can do. You will need to raise bugs on those browsers.
Comment 2 Jose M. Alcaraz CLA 2010-05-20 09:14:21 EDT
(In reply to comment #1)
> Jose,
> are you saying that IE, safari and Filezilla are sending a wrong
> content-length,
> or that Jetty is reading the content-length incorrectly?
> If an incorrect content-length is sent, then there is not much that Jetty can
> do. You will need to raise bugs on those browsers.

Hi Greg, 

thanks from you quick respose. Yes, I guess that the problem is ralated to a wrong content-lenght provided by the browsers or a bad interpretation of this value in the Jetty server. In fact, I guess that I have identified the exact class in which Jetty uses this value to manage the HTTP strem. It is the "HttpGenerator" class. 

If you like I can give you a servlet which processes a simple upload file and a jsp form which perform on the POST on the client side and you will discover that you will lost all the information in the client side because of the way in which the server manages the stream.


P.S. Steve Loughran sends you grettings from HP Labs :)

Thanks you very much for your time with this issue.

Regards, 
Jose M. Alcaraz
Automated Infrastructure Lab
HP Labs
Bristol
Comment 3 Jan Bartel CLA 2010-05-20 09:37:12 EDT
Jose,

If you have an example, please go ahead and attach to this issue.

However, I'm still not clear on what you're saying? It is the browser that sets the content length isn't it? You can use tcpdump or wireshark to check that the content-length header is correct on the wire.

thanks
Jan
Comment 4 Steve Loughran CLA 2010-05-20 11:55:28 EDT
If some browsers can generate >2GB files in forms which jetty handles (e.g. Chrome), and some don't, then it's the fault of the client for using signed integers for their content length. 

It sounds like IE does this, and so does the java.net HttpUrlConnection, though if you go to chunked everything works there. You can't ignore the content-length field and say "oh, they sent more, let's include the extra stuff", as that could be a sign of trouble all of its own. Same for clients, any mismatch between what was promised and what was delivered is trouble and should be acted on, not ignored.
Comment 5 Greg Wilkins CLA 2010-05-21 07:07:54 EDT
I'm testing this at the moment against jetty 7.1 (trunk).

There was a problem fixed in 7.1.1 about converting large content lengths into a long, but I don't see how that could directly effect file uploads...  although that depends on the exact handling.

Are you using the jetty multipart filter?    attaching your example would be good.

Also if you could capture the headers of any POST requests sent from a good browser and a bad browser, that would be very very helpful!  Use tcpdump or wireshark for that.

cheers
Comment 6 Greg Wilkins CLA 2010-05-21 09:05:55 EDT
Wow multipart filter is slow!    

But I've had no problem with a 1.9Gb file from FF and jetty 7.1.
I'll test a 2.1Gb next (as that will overflow a signed int).

cheers
Comment 7 Greg Wilkins CLA 2010-05-21 09:18:05 EDT
hmm firefox refuses to send a file bigger than 2GB.

So IE is close to the limit anyway.  I would really like to see the wireshark captures from IE.

I don't have a virtual machine big enough to play with 2GB files and IE.
Comment 8 Greg Wilkins CLA 2010-05-24 07:46:22 EDT
I'm closing this unless there is some indication that it is jetty that is in the wrong.   A wireshark dump of the correct headers being handled wrongly would be good to show this.

cheers