Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 236173 - linux drops in I20080606-1637are missing eclipse/eclipse and eclipse.ini
Summary: linux drops in I20080606-1637are missing eclipse/eclipse and eclipse.ini
Status: RESOLVED DUPLICATE of bug 236157
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 blocker (vote)
Target Milestone: ---   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 236165 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-06-07 08:10 EDT by Kim Moir CLA
Modified: 2008-06-11 10:31 EDT (History)
3 users (show)

See Also:


Attachments
artifact repo prefs (32.10 KB, text/plain)
2008-06-07 16:45 EDT, DJ Houghton CLA
no flags Details
metadata repo prefs (9.55 KB, text/plain)
2008-06-07 16:47 EDT, DJ Houghton CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kim Moir CLA 2008-06-07 08:10:38 EDT
Also, there are logs when running the director 
!SESSION 2008-06-06 19:31:50.963 -----------------------------------------------
eclipse.buildId=@build@
java.version=1.5.0_14
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
Framework arguments:  -application org.eclipse.equinox.p2.director.app.application -flavor tooling -installIU org.eclipse.sdk.ide -version 3.4.0.I20080606-1637 -p2.os aix -p2.ws motif -p2.arch ppc -roaming -profile SDKProfile -profileProperties org.eclipse.update.install.features=true -metadataRepository file:/builds/transfer/files/testUpdates -artifactRepository file:/builds/transfer/files/testUpdates -metadataRepositoryName Eclipse Project Test Site -artifactRepositoryName Eclipse Project Test Site -destination /builds/I200806061637/org.eclipse.releng.eclipsebuilder/../src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse -bundlepool /builds/I200806061637/org.eclipse.releng.eclipsebuilder/../src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse
Command-line arguments:  -os linux -ws gtk -arch x86 -application org.eclipse.equinox.p2.director.app.application -consoleLog -flavor tooling -installIU org.eclipse.sdk.ide -version 3.4.0.I20080606-1637 -p2.os aix -p2.ws motif -p2.arch ppc -roaming -profile SDKProfile -profileProperties org.eclipse.update.install.features=true -metadataRepository file:/builds/transfer/files/testUpdates -artifactRepository file:/builds/transfer/files/testUpdates -metadataRepositoryName Eclipse Project Test Site -artifactRepositoryName Eclipse Project Test Site -destination /builds/I200806061637/org.eclipse.releng.eclipsebuilder/../src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse -bundlepool /builds/I200806061637/org.eclipse.releng.eclipsebuilder/../src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse

!ENTRY org.eclipse.equinox.p2.engine 4 4 2008-06-06 19:32:19.222
!MESSAGE An error occurred while collecting items to be installed
!SUBENTRY 1 org.eclipse.equinox.p2.artifact.repository 4 0 2008-06-06 19:32:19.222
!MESSAGE Error closing the output stream for org.apache.ant/osgi.bundle/1.7.0.v200803061910 on repository file:/builds/I200806061637/src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse/. 
!STACK 0
java.io.IOException: Error unzipping /tmp/org.apache.ant_1.7.0.v20080306191034693.jar: Invalid zip file format
	at org.eclipse.equinox.internal.p2.core.helpers.FileUtils.unzipFile(FileUtils.java:31)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository$ZippedFolderOutputStream.close(SimpleArtifactRepository.java:146)
	at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository$ArtifactOutputStream.close(SimpleArtifactRepository.java:75)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle(MirrorRequest.java:164)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:146)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:94)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:477)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:64)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
!SUBENTRY 1 org.eclipse.equinox.p2.artifact.repository 4 0 2008-06-06 19:32:19.223
!MESSAGE Error closing the output stream for org.eclipse.core.runtime.compatibility.registry/osgi.bundle/3.2.200.v20070717 on repository file:/builds/I200806061637/src/I20080606-1637/p2temp/equinox.p2.build/sdk.install.aix.motif.ppc/eclipse/. 
!STACK 0

http://download.eclipse.org/eclipse/equinox/drops/I20080606-1637/testresults/1212795111333.log
Comment 1 Kim Moir CLA 2008-06-07 08:12:20 EDT
main directory of p2 logs
http://download.eclipse.org/eclipse/equinox/drops/I20080606-1637/logs.php
Comment 2 Pascal Rapicault CLA 2008-06-07 09:59:26 EDT
Two observations: 
 - all the errors in the logs are for bundles or features that ends up in the folder shape once installed 
 - not all the bundles that ends up in this shape appears in the log 
Comment 3 DJ Houghton CLA 2008-06-07 16:45:27 EDT
Created attachment 104092 [details]
artifact repo prefs
Comment 4 DJ Houghton CLA 2008-06-07 16:47:10 EDT
Created attachment 104093 [details]
metadata repo prefs

Here are the preferences for the director on the build machine.

Note that there are 4 important metadata repos: the Ganymede site, Eclipse project updates, and 2 from the build.

Note that the 2 local build ones are missing from the list of artifact repos.
Comment 5 Pascal Rapicault CLA 2008-06-08 10:19:00 EDT
The messages in the log are confusing. 

>An error occurred while collecting items to be installed
>"/tmp/org.apache.ant_1.7.0.v20080306191034693.jar: Invalid zip file format" 
  should read "the file org.apache.ant could not be found in any of the repositories, which would correspond with DJ's findings in comment #4.

What is still a mystery is how come some builds succeeded and other did not. Maybe  repos are being removed mysteriously.
Comment 6 John Arthorne CLA 2008-06-08 11:31:05 EDT
It feels like we should attempt another build so we can figure out if this is an intermittent problem that we were just lucky not to encounter before, or if it's a very recent regression.
Comment 7 Kim Moir CLA 2008-06-08 16:20:16 EDT
Regarding comment #5

Certainly the Ganymede repos are cleaned quite frequently during the Ganymede build process.  

Regarding comment #6
I ran a test build yesterday morning. Same problem with missing files.

Andrew sent me an email as follows

I think they may be coming from the pre-populated repos in combination with https://bugs.eclipse.org/bugs/show_bug.cgi?id=235491

One thing to try would be to delete 
/builds/I200806070822/src/I20080607-0822/p2temp/equinox.p2.build/director/eclipse/configuration/.settings/*
before you start running the director

I'm trying Andrew's suggestion now on yesterday's test build.
Comment 8 Kim Moir CLA 2008-06-08 19:13:09 EDT
The test build still seems to have the same problem.
Comment 9 John Arthorne CLA 2008-06-09 02:12:22 EDT
I think Pascal already figured this out, but the spew of errors in the log was caused by our fix for bug 233214. With our "fix", we were no longer closing processing steps in the case of downloading a folder-based descriptor. Since the signature verifier step only copies the signed jar to the actual destination when closed, you end up with the zero-length jars for unzipped artifacts like org.apache.ant. Then when MirrorRequest attempts to close the destination stream (in the stack traces above), it attempts to unzip the zero-length file and fails.

I can consistently reproduce this, then when I apply Pascal's patch in bug 236157 comment 2, the problem goes away because we properly close the processing steps.  I don't actually see how this causes the missing Linux GTK root files, but it's quite possible it is somehow behind it.
Comment 10 John Arthorne CLA 2008-06-09 02:41:06 EDT
*** Bug 236165 has been marked as a duplicate of this bug. ***
Comment 11 Pascal Rapicault CLA 2008-06-11 10:31:52 EDT

*** This bug has been marked as a duplicate of bug 236157 ***