Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 344664 - tycho downloads content.xml and artifacts.xml on every build
Summary: tycho downloads content.xml and artifacts.xml on every build
Status: RESOLVED WORKSFORME
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Tycho (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-04 04:27 EDT by Dimo CLA
Modified: 2021-04-28 16:55 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dimo CLA 2011-05-04 04:27:15 EDT
Build Identifier: 

I am using a local mirror of the eclipse p2 helios repository configured like this in my pom.xml:
--cut--
		<repository>
			<id>eclipse-helios</id>
			<layout>p2</layout>
 			<url>http://mirror/eclipse-helios/</url>
 			<snapshots>
 				<enabled>false</enabled>
 				<updatePolicy>never</updatePolicy>
			</snapshots>
 			<releases>
 				<enabled>true</enabled>
 				<updatePolicy>never</updatePolicy>
 			</releases>
		</repository>
--cut--

Each time I build my project using tycho 0.11.0 it downloads the content.xml and artifacts.xml files from the p2 repository. The files are about 70 MB and this takes about 2-3 minutes just for that step - which is unacceptable.

I have tried both Nexus Pro 1.9.1 and apache2 to host the helios p2 repository mirror. The same behavior for both.

What I check the load balancer's logs, I see that tycho first issues a HEAD request and then a GET request. The HEAD request is fast and responds with correct Last-Modified and ETag headers (result from apache2):
--cut--
root@build:~# curl --head 'http://mirror/eclipse-helios/content.xml'
HTTP/1.1 200 OK
Date: Wed, 04 May 2011 09:19:17 GMT
Server: Apache/2.2.16 (Ubuntu)
Last-Modified: Wed, 04 May 2011 09:15:49 GMT
ETag: "20edd-3d402cb-4a26fb3df8d5a"
Accept-Ranges: bytes
Content-Length: 64225995
Connection: close
Content-Type: application/xml

--cut--

Despite that, tycho still redownloads the files.

I have also checked in my local m2 repository - the files are there.

Reproducible: Always
Comment 1 Igor Fedorenko CLA 2011-05-04 07:25:57 EDT
Please provide complete standalone project and steps to reproduce the problem.
Comment 2 Tobias Oberlies CLA 2011-06-06 07:16:31 EDT
It has been claimed that p2 is the culprit for this behaviour (I don't remember the bug number), but p2 has proven that they in fact only do HEAD requests to check that the cache is up to date. Nevertheless, it is still possible, that Tycho still sends out GETs:
1) Tycho's p2 version could be too old (cf. bug 342554)
2) Tycho prevents that p2 properly caches p2 repositories. (I wouldn't be surprised: p2's IProvisioningAgent is reponsible for maintaining the cache, and it seems a bit random how Tycho uses/creates IProvisioningAgent instances.)

However, all this theoretic analysis doesn't get us anywhere. As long as there are no detailed instructions for reproducing the issue, we won't see any progress here.
Comment 3 Tobias Oberlies CLA 2011-06-16 11:39:48 EDT
With the current 0.13.0-SNAPSHOT, I cannot reproduce the problem: I traced the http communication as described in bug 347403#c7, and there are only HEAD requests for the content.jar/artifact.jars.

Note that there is still the related issue, that Tycho does HEAD requests for the artifacts.xml, even if this is not necessary (see bug 347477).