Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 363603

Summary: use fetchCacheLocation to reuse git clones across builds
Product: [Eclipse Project] Platform Reporter: Kim Moir <kim.moir>
Component: RelengAssignee: Platform-Releng-Inbox <platform-releng-inbox>
Status: CLOSED WONTFIX QA Contact:
Severity: normal    
Priority: P3 CC: aniefer, daniel_megert, david_williams, gunnar, pwebster
Version: 3.8   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard: stalebug

Description Kim Moir CLA 2011-11-11 12:16:55 EST
See Andrew's blog for details

http://aniefer.blogspot.com/2011/01/building-from-git.html

This should make the build faster as we will only have to replicate clone deltas instead of full repos.
Comment 1 Kim Moir CLA 2012-01-25 17:03:16 EST
I talked to Andrew and you should set a fetchCacheLocation for every type of build.  (I,M,N)  Apparently there is a bug in the EGit fetch factory that it doesn't switch to the correct branch sometimes if you are pulling multiple branches from the same repo.
Comment 2 Kim Moir CLA 2012-01-26 15:29:44 EST
Okay I should have implemented this a long time ago.

Some numbers
Without fetchCacheLocation (times from start of build, no signing)
build source zips: 34 minutes
build tests:       1h 4minutes
Finish build:      2h 30minutes   

Some numbers
With fetchCacheLocation and git repos content primed(times from start of build, no signing)
build source zips: 18 minutes
build tests:       44 minutes
Finish build:      1h 37minutes   

Total savings: 
53 minutes

Wow. Thanks Andrew for implementing this feature in the fetch factory!

I've implemented this for the integration and nightly builds.  I can't do this for 3.7.2 yet since we are in the endgame and changes require PMC approval etc.
Comment 3 Kim Moir CLA 2012-01-26 16:15:32 EST
Not sure if bug 351384 is relevant.   Was talking to Ian and he mentioned he had issues with the caching but this may just have been related to how he specified his branch.
Comment 4 Andrew Niefer CLA 2012-01-26 16:35:39 EST
bug 347151 is the egit bug, it does a fetch but no merge
Comment 5 Gunnar Wagenknecht CLA 2012-01-27 04:14:41 EST
(In reply to comment #4)
> bug 347151 is the egit bug, it does a fetch but no merge

Did the fetch factory already moved back to PDE?
Comment 6 Kim Moir CLA 2012-01-27 11:32:36 EST
Gunnar, no the fetch factory hasn't moved to PDE.  I don't think we have any committers to work on it.

Andrew, from reading bug 347151, it's not clear to me that this causes an issue for our builds.  I've configured our build so there are directories for the Git cache for every stream (M, N, I) but these directories are reused when that build runs.  You seem to suggest in that bug that a new directory be used for the cache for every build which seems to negate the advantage of having a cache in the first place.
Comment 7 Kim Moir CLA 2012-01-30 10:16:41 EST
This seems to be working.  Closing.
Comment 8 Kim Moir CLA 2012-01-31 09:00:47 EST
This changed caused problems in bug 369409.  Thus I'm reverting it.
Comment 9 David Williams CLA 2012-06-20 21:35:30 EDT
upsetting 3.8 M6 target since this wasn't fixed in 3.8.
Comment 10 Lars Vogel CLA 2019-11-27 07:13:11 EST
This bug hasn't had any activity in quite some time. Maybe the problem got
resolved, was a duplicate of something else, or became less pressing for some
reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it.
The information can be, for example, that the problem still occurs, that you
still want the feature, that more information is needed, or that the bug is
(for whatever reason) no longer relevant.

If the bug is still relevant, please remove the stalebug whiteboard tag.
Comment 11 Alexander Kurtakov CLA 2020-01-15 08:57:09 EST
Infra is moving to containerized builds so the idea of reusing git clones is a no-go.