Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 321195

Summary:

Performance of checkout

Product:

[Technology] JGit

Reporter:

Mathias Kinzler <mathias.kinzler>

Component:

JGit

Assignee:

Project Inbox <jgit.core-inbox>

Status:

CLOSED WORKSFORME

QA Contact:

Severity:

major

Priority:

Version:

0.9.0

Target Milestone:

0.9.0

Hardware:

OS:

All

Whiteboard:

Attachments:

Description	Flags
Standard patch format with debug output	none
Git patch format with debug output	none

Description Mathias Kinzler

2010-07-29 04:51:20 EDT

When cloning the gaelyk repository from www.github.com, the performance of the checkout is dramatically bad: pretty much the first file is the jar file found at

core\lib\appengine-api-1.0-sdk-1.3.5.jar

It is created, but then it grows at a rate of a few dozen bytes per second.
This can be reproduced both on Windows and Linux (Ubuntu 10.04 LTS on PC) and is independent of whether the clone is via http or git:

http://github.com/glaforge/gaelyk.git
git://github.com/glaforge/gaelyk.git

There appear huge numbers of Inflater calls.

Comment 1 Mathias Kinzler

2010-07-30 10:35:48 EDT

I diggend into this by adding some debug output. It turns out that DeltaStram.skip() is horribly inefficient. I attach patch files (one in the generic patch format, one in git patch format) which add the debug output.
The typical output looks like:

Read during 2 skips: 244616
Read outside skips: 512
Read during 3 skips: 489684
Read outside skips: 512
Read during 3 skips: 2415688
Read outside skips: 512
Read during 1 skips: 274937
Read outside skips: 512
Read during 2 skips: 489684
Read outside skips: 512
Read during 2 skips: 2122925
Read outside skips: 512
Read during 2 skips: 450521
Read outside skips: 512
Read during 1 skips: 240251
Read outside skips: 512

(only a very small number of DeltaStream instances shown). The Inflater does around 400 to 500 times more work during skip compared to the actual read work.
I didn't quite understand how this happens, but it looks like some weird kind of recursion.

Comment 2 Mathias Kinzler

2010-07-30 10:36:33 EDT

Created attachment 175582 [details]
Standard patch format with debug output

Comment 3 Mathias Kinzler

2010-07-30 10:37:00 EDT

Created attachment 175583 [details]
Git patch format with debug output

Comment 4 Mathias Kinzler

2010-08-30 03:10:23 EDT

Does http://egit.eclipse.org/r/#change,1441 fix this?

Comment 5 Mathias Kinzler

2010-08-30 03:44:53 EDT

Does not seem to be reproducible anymore. Not sure if http://egit.eclipse.org/r/#change,1441 fixed this (the clone went through without it), but I could clone this repository without problems.