Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 361842

Summary:	map files have invalid repos URLs while on build machine
Product:	[Tools] Orbit	Reporter:	David Williams <david_williams>
Component:	releng	Assignee:	David Williams <david_williams>
Status:	RESOLVED FIXED	QA Contact:	Project Inbox <orbit.releng-inbox>
Severity:	normal
Priority:	P3
Version:	unspecified
Target Milestone:	---
Hardware:	PC
OS:	Linux
Whiteboard:

Description David Williams

2011-10-24 13:56:50 EDT

As noticed in bug 361830

The builds on "build.eclipse.org" do not really produce valid repo locations
for the maps files, while still on the build machine location ... they point to 

http://download.eclipse.org/tools/orbit/committers/drops/S20111018035124/repository
(which now is inaccurate, and points to nothing)

Would have to be 

http://build.eclipse.org/orbit/committers/orbit-I/20111018035124/S20111018035124/repository/

To fix this would be non trivial. 
Would have to fix the script that creates the map files (which is some custom java code ant task, I believe ... perhaps there's a properties file?)
plus, 
would have to fix up the "promote" script that does string replaces to fix up URLs when going from 'build' machine to 'downloads' machine. 

I'm not positive this has to be fixed, per se, but, might want to at least have a warning on the build page saying they can not used.

Plus, I guess if they are inaccurate, might want to put in the URL that would eventually be accurate, once promoted to downloads machine? Instead of a URL that is always inaccurate.

Comment 1 David Williams

2011-10-24 14:26:54 EDT

I do recall a few times using "committer builds map files" a few times to check some new contribution, before a build was promoted to downloads, and while not recommended, it does seem this should be "fixed" to be accurate. 

From what I've seen so far, there are properties in the ant task that generates these map files. Currently, they are 

        <eclipse.mapGenerator
            root="${buildDirectory}/${buildLabel}/repository/plugins"
            addressPrefix="http://download.eclipse.org/tools/orbit/committers/drops/${buildLabel}/repository/plugins/"
            inputFile="${buildDirectory}/finalPluginsVersions.properties"
            unzipinputFile="${buildDirectory}/bundlesToUnzip.properties"
            outputFile="${buildDirectory}/${buildLabel}/orbitBundles-${buildLabel}.map"
            p2MapFile="${buildDirectory}/${buildLabel}/orbitBundles-${buildLabel}.p2.map"
            p2Repository="http://download.eclipse.org/tools/orbit/committers/drops/${buildLabel}/repository"/>


So, there are parts of the URLs that are "hard coded" there.

Comment 2 David Williams

2011-10-24 21:04:45 EDT

This was fairly easy to fix, changing the properties in the 
eclipse.mapGenerator task, as follows. 

        
    	 <property 
    	 	name="mapPrefixHost"
    	 	value="http://build.eclipse.org" />
    	 <property
    	       name="mapRelativeDir"
    	 	value="orbit/committers/${projectname}/${cctimestamp}" />
    	
        <eclipse.mapGenerator
            root="${buildDirectory}/${buildLabel}/repository/plugins"
            addressPrefix="${mapPrefixHost}/${mapRelativeDir}/${buildLabel}/repository/plugins/"
            inputFile="${buildDirectory}/finalPluginsVersions.properties"
            unzipinputFile="${buildDirectory}/bundlesToUnzip.properties"
            outputFile="${buildDirectory}/${buildLabel}/orbitBundles-${buildLabel}.map"
            p2MapFile="${buildDirectory}/${buildLabel}/orbitBundles-${buildLabel}.p2.map"
            p2Repository="${mapPrefixHost}/${mapRelativeDir}/${buildLabel}/repository"/>


It is a bit funky that is now makes an assumption on the "cctimestamp" (the build timestamp from CruiseControl) simply because that's part of the "archive output" path ... in other words, if we ever change to use hudson, or something else, we'll have to fix this part of the script to be more general. 

One advantage, though, is pulling out significant parts of the URL, "mapPrefixHost" and "mapRelativeDir"  means someone could "build locally" and supply these variables in their 'localBuildProperties' so the final maps would point to their local machine. 

Also, I've fixed the "promote.sh" script to be specifically for this new format. I've used regex's '.*' and the "found" directory (dropDirName) to avoid complete "hard coding" but this script is still tightly coupled to the exact format used/created by the builder. Here's the relevant excerpt.  

    # now fix up map files, the one for P2 and HTTP GET

    fromString="http://build.eclipse.org/.*${dropDirName}"
    toString="http://download.eclipse.org/tools/orbit/downloads/drops/${dropDirName}"
    
    replaceCommand="s!${fromString}!${toString}!g"

    # remember TODIR already has a slash
    perl -w -pi -e ${replaceCommand} ${TODIR}${dropDirName}/*.map


I've tested with a new I-build (both building and promoting) and will rebuild the M-build so its map files, on the build machine, correctly point to build machine URL. We'll should, as always, double check once its promoted, but should be fine.