Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 346524 - Support arbitrary ZLib windows sizes (loose object signature check is too restrictive)
Summary: Support arbitrary ZLib windows sizes (loose object signature check is too res...
Status: NEW
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 0.12   Edit
Hardware: All other
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-19 14:47 EDT by Emilian Miron CLA
Modified: 2013-03-30 19:32 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Emilian Miron CLA 2011-05-19 14:47:48 EDT
When trying to use JGit on Android the platform limits the window size of ZLib deflators to less than the usual 2^15. Therefore, the loose objects written by JGit do not have the signature first byte equal to 0x78 because of the decreased window size.

In class org.eclipse.jgit.storage.file.UnpackedObject the following method checks the signature for loose objects versus packs:

	private static boolean isStandardFormat(final byte[] hdr) {
		// Try to determine if this is a standard format loose object or
		// a pack style loose object. The standard format is completely
		// compressed with zlib so the first byte must be 0x78 (15-bit
		// window size, deflated) and the first 16 bit word must be
		// evenly divisible by 31. Otherwise its a pack style object.
		//
		final int fb = hdr[0] & 0xff;
		return fb == 0x78 && (((fb << 8) | hdr[1] & 0xff) % 31) == 0;
	}


If fb == 0x78 is modified to ((fb & 0x0f) == 0x08), this will only check the CM (compression method) flag in the ZLib header for 0x8 which is DEFLATE and will not restrict the window size which is in the other nibble. 

For reference, this is the specification for Zlib:
http://www.gzip.org/zlib/rfc-zlib.html

Git's format is described here:
http://book.git-scm.com/7_how_git_stores_objects.html
http://book.git-scm.com/7_the_packfile.html

Since pack files start with "PACK", the check above does not yield positive for packs.

Also note that this conforms to Git's storage format guidelines, since there is no mention of an exact window size anywhere.
Comment 1 Nick Palmer CLA 2011-08-31 07:55:38 EDT
This is unfortunately not a complete patch as repos written on android with this patch are not actually readable by Git. I have not had time to look at the Git source but presumably it is doing the same.

I believe that this bug is caused by the same problem:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=347485
Comment 2 Robin Rosenberg CLA 2013-03-30 19:32:05 EDT
Isn't this fixed by https://git.eclipse.org/r/#/c/4071/ ?