Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 321195 - Performance of checkout
Summary: Performance of checkout
Status: CLOSED WORKSFORME
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 0.9.0   Edit
Hardware: PC All
: P3 major (vote)
Target Milestone: 0.9.0   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-29 04:51 EDT by Mathias Kinzler CLA
Modified: 2010-08-30 03:44 EDT (History)
0 users

See Also:


Attachments
Standard patch format with debug output (1.25 KB, patch)
2010-07-30 10:36 EDT, Mathias Kinzler CLA
no flags Details | Diff
Git patch format with debug output (1.66 KB, text/plain)
2010-07-30 10:37 EDT, Mathias Kinzler CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mathias Kinzler CLA 2010-07-29 04:51:20 EDT
When cloning the gaelyk repository from www.github.com, the performance of the checkout is dramatically bad: pretty much the first file is the jar file found at

core\lib\appengine-api-1.0-sdk-1.3.5.jar

It is created, but then it grows at a rate of a few dozen bytes per second.
This can be reproduced both on Windows and Linux (Ubuntu 10.04 LTS on PC) and is independent of whether the clone is via http or git:

http://github.com/glaforge/gaelyk.git
git://github.com/glaforge/gaelyk.git

There appear huge numbers of Inflater calls.
Comment 1 Mathias Kinzler CLA 2010-07-30 10:35:48 EDT
I diggend into this by adding some debug output. It turns out that DeltaStram.skip() is horribly inefficient. I attach patch files (one in the generic patch format, one in git patch format) which add the debug output.
The typical output looks like:

Read during 2 skips: 244616
Read outside skips: 512
Read during 3 skips: 489684
Read outside skips: 512
Read during 3 skips: 2415688
Read outside skips: 512
Read during 1 skips: 274937
Read outside skips: 512
Read during 2 skips: 489684
Read outside skips: 512
Read during 2 skips: 2122925
Read outside skips: 512
Read during 2 skips: 450521
Read outside skips: 512
Read during 1 skips: 240251
Read outside skips: 512

(only a very small number of DeltaStream instances shown). The Inflater does around 400 to 500 times more work during skip compared to the actual read work.
I didn't quite understand how this happens, but it looks like some weird kind of recursion.
Comment 2 Mathias Kinzler CLA 2010-07-30 10:36:33 EDT
Created attachment 175582 [details]
Standard patch format with debug output
Comment 3 Mathias Kinzler CLA 2010-07-30 10:37:00 EDT
Created attachment 175583 [details]
Git patch format with debug output
Comment 4 Mathias Kinzler CLA 2010-08-30 03:10:23 EDT
Does http://egit.eclipse.org/r/#change,1441 fix this?
Comment 5 Mathias Kinzler CLA 2010-08-30 03:44:53 EDT
Does not seem to be reproducible anymore. Not sure if http://egit.eclipse.org/r/#change,1441 fixed this (the clone went through without it), but I could clone this repository without problems.