Community
Participate
Working Groups
From Shawn: I am trying to use the Subversive Fetch Factory. Per https://bugs.eclipse.org/bugs/show_bug.cgi?id=227115, it is ready to use. I have concluded that there are two possible bugs/enhancements that I'd like you to confirm before I create a bugzilla item. The first one was not supporting force in the SVN ant task for the export command. This one is easy and trivial. // in SVNTask.java else if (SVNTask.CMD_EXPORT.equals(this.command)) { cmdLine = "svn export "; // 4/8/2009 1:14PM Shawn: Added force when exporting if (this.force == true) { cmdLine += " --force "; } if (this.rev != null) { cmdLine += " -r " + this.rev; } cmdLine += " " + this.SPACE_WRAP_CHAR + this.url; if (this.pegRev != null) { cmdLine += "@" + this.pegRev; } cmdLine += this.SPACE_WRAP_CHAR; cmdLine += " " + this.SPACE_WRAP_CHAR + this.dest + this.SPACE_WRAP_CHAR; cmdLine += this.getCreadentialsPart(); cmdLine += " -q --non-interactive"; } The second one is more complicated. The Eclipse BaseBuilder uses Subversive Fetch Factory to calculate tag. In the IFetchFactory, it is specified explicitly that the value of tag can't contain characters like "/". The Subversive Fetch Factory didn't guard this constraint. It set the whole thing "tags/nightly/20093...." as the value of this key. This leads to build errors. The following is an excerpt from the IFetchFactory interface /** Key used to store the value of the tag that will be used to fetch the element. * <p> * The grammar of the expected value is limited to: * <pre> * value::= (alpha|digit|'_'|'-')+ * digit ::= [0..9] * alpha ::= [a..zA..Z] * </pre> * */ public static final String KEY_ELEMENT_TAG = "tag"; //$NON-NLS-1$ The following is the proposed modified code for the class SVNFetchFactory /******************************************************************************* * Copyright (c) 2005-2006 Polarion Software. * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html * * Contributors: * Alexander Gurov (Polarion Software) - initial API and implementation *******************************************************************************/ package org.eclipse.team.svn.pde.build; import java.util.HashMap; import java.util.Map; import java.util.Properties; import java.util.StringTokenizer; import org.eclipse.core.runtime.CoreException; import org.eclipse.core.runtime.IPath; import org.eclipse.pde.build.Constants; import org.eclipse.pde.build.IAntScript; import org.eclipse.pde.build.IFetchFactory; import org.eclipse.pde.internal.build.Utils; /** * Implementation of SVN fetch task factory for PDE build * * @author Alexander Gurov */ public class SVNFetchFactory implements IFetchFactory { public static final String TARGET_FETCH_FROM_SVN = "FetchFromSVN"; public static final String MAP_ENTRY_SEPARATOR = ","; public static final String VALUE_PAIR_SEPARATOR = "="; public static final String OVERRIDE_TAG = "SVN"; public static final String KEY_URL = "url"; public static final String KEY_PEG = "peg"; public static final String KEY_REVISION = "revision"; public static final String KEY_PATH = "path"; public static final String KEY_USERNAME = "username"; public static final String KEY_PASSWORD = "password"; public static final String PROP_FILETOCHECK = "fileToCheck"; public static final String PROP_ELEMENTNAME = "elementName"; public static final String PROP_DESTINATIONFOLDER = "destinationFolder"; public static final String PROP_URL = SVNFetchFactory.KEY_URL; public static final String PROP_PEG = SVNFetchFactory.KEY_PEG; public static final String PROP_REVISION = SVNFetchFactory.KEY_REVISION; public static final String PROP_TAG = IFetchFactory.KEY_ELEMENT_TAG; public static final String PROP_PATH = SVNFetchFactory.KEY_PATH; public static final String PROP_USERNAME = SVNFetchFactory.KEY_USERNAME; public static final String PROP_PASSWORD = SVNFetchFactory.KEY_PASSWORD; private static String overrideTag = ""; public SVNFetchFactory() { } public void addTargets(IAntScript script) { script.printTargetDeclaration(SVNFetchFactory.TARGET_FETCH_FROM_SVN, null, null, "${" + SVNFetchFactory.PROP_FILETOCHECK + "}", null); this.printSVNTask("export", "${" + SVNFetchFactory.PROP_URL + "}/${" + SVNFetchFactory.PROP_TAG + "}/${" + SVNFetchFactory.PROP_PATH + "}", "${" + SVNFetchFactory.PROP_PEG + "}", "${" + SVNFetchFactory.PROP_REVISION + "}", "${" + SVNFetchFactory.PROP_DESTINATIONFOLDER + "}/${" + SVNFetchFactory.PROP_ELEMENTNAME + "}", "${" + SVNFetchFactory.PROP_USERNAME + "}", "${" + SVNFetchFactory.PROP_PASSWORD + "}", script); script.printTargetEnd(); } public void generateRetrieveElementCall(Map entryInfos, IPath destination, IAntScript script) { String type = (String) entryInfos.get(KEY_ELEMENT_TYPE); String element = (String) entryInfos.get(KEY_ELEMENT_NAME); HashMap<String, String> params = new HashMap<String, String>(); IPath locationToCheck = (IPath)destination.clone(); if (type.equals(ELEMENT_TYPE_FEATURE)) { locationToCheck = locationToCheck.append(Constants.FEATURE_FILENAME_DESCRIPTOR); } else if (type.equals(ELEMENT_TYPE_PLUGIN)) { locationToCheck = locationToCheck.append(Constants.PLUGIN_FILENAME_DESCRIPTOR); } else if (type.equals(ELEMENT_TYPE_FRAGMENT)) { locationToCheck = locationToCheck.append(Constants.FRAGMENT_FILENAME_DESCRIPTOR); } else if (type.equals(ELEMENT_TYPE_BUNDLE)) { locationToCheck = locationToCheck.append(Constants.BUNDLE_FILENAME_DESCRIPTOR); } params.put(SVNFetchFactory.PROP_ELEMENTNAME, element); params.put(SVNFetchFactory.PROP_FILETOCHECK, locationToCheck.toString()); params.put(SVNFetchFactory.PROP_DESTINATIONFOLDER, destination.removeLastSegments(1).toString()); params.put(SVNFetchFactory.PROP_URL, (String)entryInfos.get(SVNFetchFactory.KEY_URL)); if (entryInfos.containsKey(SVNFetchFactory.KEY_PEG)) { params.put(SVNFetchFactory.PROP_PEG, (String)entryInfos.get(SVNFetchFactory.KEY_PEG)); } else { params.put(SVNFetchFactory.PROP_PEG, "HEAD"); } if (entryInfos.containsKey(SVNFetchFactory.KEY_REVISION)) { params.put(SVNFetchFactory.PROP_REVISION, (String)entryInfos.get(SVNFetchFactory.KEY_REVISION)); } else { params.put(SVNFetchFactory.PROP_REVISION, "HEAD"); } params.put(SVNFetchFactory.PROP_TAG, SVNFetchFactory.overrideTag); params.put(SVNFetchFactory.PROP_PATH, (String)entryInfos.get(SVNFetchFactory.KEY_PATH)); String username = (String)entryInfos.get(SVNFetchFactory.KEY_USERNAME); params.put(SVNFetchFactory.PROP_USERNAME, username != null ? username : ""); String password = (String)entryInfos.get(SVNFetchFactory.KEY_PASSWORD); params.put(SVNFetchFactory.PROP_PASSWORD, password != null ? password : ""); this.printAvailableTask(locationToCheck.toString(), locationToCheck.toString(), script); if (IFetchFactory.ELEMENT_TYPE_PLUGIN.equals(type) || IFetchFactory.ELEMENT_TYPE_FRAGMENT.equals(type)) { this.printAvailableTask(locationToCheck.toString(), locationToCheck.removeLastSegments(1).append(Constants.BUNDLE_FILENAME_DESCRIPTOR).toString(), script); } script.printAntCallTask(SVNFetchFactory.TARGET_FETCH_FROM_SVN, true, params); } public void generateRetrieveFilesCall(Map entryInfos, IPath destination, String[] files, IAntScript script) { String rootUrl = (String)entryInfos.get(SVNFetchFactory.KEY_URL); String pegRev = (String)entryInfos.get(SVNFetchFactory.KEY_PEG); String rev = (String)entryInfos.get(SVNFetchFactory.KEY_REVISION); String tag = SVNFetchFactory.overrideTag; String path = (String)entryInfos.get(SVNFetchFactory.KEY_PATH); String baseUrl = rootUrl + "/" + tag + "/" + path + "/"; String dest = destination.toString(); String username = (String)entryInfos.get(SVNFetchFactory.KEY_USERNAME); String password = (String)entryInfos.get(SVNFetchFactory.KEY_PASSWORD); for (String fileName : files) { this.printSVNTask("cat", baseUrl + fileName, pegRev, rev, dest + "/" + fileName, username, password, script); } } /* * Map file entry format: * mapEntry * : elementType '@' elementID (',' elementVersion)? = svnContent * ; * elementType * : 'bundle' | 'feature' | 'plugin' | 'fragment' * ; * elementID * : ... //plug-in, feature, fragment or bundle ID * ; * elementVersion * : ... //plug-in, feature, fragment or bundle version * ; * svnContent * : 'SVN' (',' arg)+ * ; * arg * : key '=' value * ; * key * : 'url' // project root URL * | 'tag' // optional tag name (trunk, tags/some_name etc.) * | 'path' // optional element, path relative to project root URL * | 'revision' // optional element, revision * | 'peg' // optional element, peg revision * | 'username' * | 'password' * ; */ public void parseMapFileEntry(String rawEntry, Properties overrideTags, Map entryInfos) throws CoreException { String []arguments = Utils.getArrayFromStringWithBlank(rawEntry, SVNFetchFactory.MAP_ENTRY_SEPARATOR); // check entry count here.... for (String argument : arguments) { int idx = argument.indexOf(SVNFetchFactory.VALUE_PAIR_SEPARATOR); if (idx != -1) { String key = argument.substring(0, idx); String value = argument.substring(idx + 1); entryInfos.put(key, value); } } if (overrideTags != null) { SVNFetchFactory.overrideTag = overrideTags.getProperty(SVNFetchFactory.OVERRIDE_TAG); if (SVNFetchFactory.overrideTag != null && SVNFetchFactory.overrideTag.length() > 0) { StringTokenizer st = new StringTokenizer(SVNFetchFactory.overrideTag, "/"); String lastToken = null; while (st.hasMoreTokens() == true) { lastToken = st.nextToken(); } entryInfos.put(IFetchFactory.KEY_ELEMENT_TAG, lastToken); } } // handle optional path String path = (String)entryInfos.get(SVNFetchFactory.KEY_PATH); if (path == null) { entryInfos.put(SVNFetchFactory.KEY_PATH, entryInfos.get(IFetchFactory.KEY_ELEMENT_NAME)); } // handle optional tag String tag = (String)entryInfos.get(IFetchFactory.KEY_ELEMENT_TAG); if (tag == null) { entryInfos.put(IFetchFactory.KEY_ELEMENT_TAG, ""); } } protected void printSVNTask(String command, String url, String pegRev, String rev, String dest, String username, String password, IAntScript script) { script.printTabs(); script.print("<svn"); script.printAttribute("command", command, false); script.printAttribute("url", url, false); script.printAttribute("pegRev", pegRev, false); script.printAttribute("rev", rev, false); script.printAttribute("dest", dest, false); script.printAttribute("username", username, false); script.printAttribute("password", password, false); script.println("/>"); } protected void printAvailableTask(String property, String file, IAntScript script) { script.printTabs(); script.print("<available"); script.printAttribute("property", property, true); script.printAttribute("file", file, false); script.println("/>"); } }
From Shawn: I found the 3rd possible bug/enhancement for the Subversive Fetch Factory. Original code doesn't support same plugin with multiple plugins to be downloaded into ${buildDirectory}/plugins/ example: com.shawn.bundles.axis (1.2.1) com.shawn.bundles.axis (1.4.0) both specified in feature.xml and .map The fetch script generated by original code always trying to download to one folder called ${buildDirectory}/plugins/com.shawn.bundles.axis Unlike CVSFetchFactory, if you specify a version to the plugin in the map, for example, 'plugin@xxx,1.2.1=...' it will download to ${buildDirectory}/plugins/com.shawn.bundles.axis_1.2.1 and ${buildDirectory}/plugins/com.shawn.bundles.axis_1.4.0 respectively. This is needed when we want to build multiple versions of the same bundle in a PDE project like Orbit or Shawn Commons. // original code // params.put(SVNFetchFactory.PROP_ELEMENTNAME, element); The patch will be attached.
Created attachment 132943 [details] Proposed patch to fix 3 problems Proposed patch to fix 3 problems: 1. Multiple versions in the same PDE build 2. Correct tag set for IFetchFactory 3. SVN force
Shawn, thanks a lot for provided patch. I applied the following: 1. Multiple versions in the same PDE build Done 2. Correct tag set for IFetchFactory I didn't applied it, because 'tag' is used in forming url to repository and if we change it in some way, url will not be correct. I mean following by "tag is used in forming url", e.g.: If I have a map file with following entry: plugin@org.my.plugin,1.0.0=SVN,url=http://localhost:81/repos/first/ProjectsData/Subversive/PDE,path=org.my.plugin.v1 and I set 'fetchTag'(in build.properties) to value: fetchTag=SVN=trunk Then Url to plugin will be http://localhost:81/repos/first/ProjectsData/Subversive/PDE/trunk/org.my.plugin.v1 Here is how url in export command is formed: <svn command="export" url="${url}/${tag}/${path}" ... So if for example user set 'fetchTag' to "tags/nightly/20093" and we will leave only the last part of it, then url will not be correct. So I just added a validation of tag in code according to required pattern and if it's not valid, I just throw exception. 3. SVN force Done I didn't yet propagated last changes to pde update site, so if you have any remarks (probably to 2nd problem), feel free to discuss them and then I'll prepare update site (if you need it).