Community
Participate
Working Groups
This is in addition to bug 196056. It would be a good idea to add a mylyn buildId to the user agent so that it is easier to understand bandwidth usages of various mylyn versions.
### Eclipse Workspace Patch 1.0 #P org.eclipse.mylyn.web.core Index: src/org/eclipse/mylyn/web/core/WebClientUtil.java =================================================================== RCS file: /cvsroot/tools/org.eclipse.mylyn/org.eclipse.mylyn.web.core/src/org/eclipse/mylyn/web/core/WebClientUtil.java,v retrieving revision 1.14 diff -u -r1.14 WebClientUtil.java --- src/org/eclipse/mylyn/web/core/WebClientUtil.java 13 Sep 2007 22:51:43 -0000 1.14 +++ src/org/eclipse/mylyn/web/core/WebClientUtil.java 2 Oct 2007 20:26:25 -0000 @@ -24,6 +24,9 @@ import org.apache.commons.httpclient.protocol.ProtocolSocketFactory; import org.eclipse.core.net.proxy.IProxyData; import org.eclipse.core.net.proxy.IProxyService; +import org.eclipse.core.runtime.Plugin; +import org.eclipse.core.runtime.PluginVersionIdentifier; +import org.eclipse.core.runtime.internal.adaptor.PluginParser.PluginInfo; /** * @author Mik Kersten @@ -32,7 +35,13 @@ */ public class WebClientUtil { - public static final String USER_AGENT = "Eclipse Mylyn (HttpClient/3.0.1)"; + private static String os = System.getProperty("os.name") + "-" + + System.getProperty("os.version") + "-" + + System.getProperty("os.arch"); + + private static String build = (String) WebCorePlugin.getDefault().getBundle().getHeaders().get("Bundle-Version"); + + public static final String USER_AGENT = "Eclipse Mylyn (HttpClient/3.0.1) build: " + build + " [" + os + "]"; public static final int CONNNECT_TIMEOUT = 60000; @@ -205,9 +214,9 @@ /** * utility method, proxy should be obtained via TaskRepository.getProxy() - * + * * TODO: deprecate - * + * * @return proxy as defined in platform proxy settings property page, Proxy.NO_PROXY otherwise */ public static Proxy getPlatformProxy() { @@ -231,7 +240,7 @@ /** * utility method, proxy should be obtained via TaskRepository.getProxy() - * + * * @return proxy as defined in platform proxy settings property page, Proxy.NO_PROXY otherwise * @since 2.1 */
Somehow I couldn't attach a patch so I pasted it in previous comment. Not only do I attach the build id (==Bundle-version) but also the OS and Platform. It was easy, I did this before for a dedicated web client I made ;-)
OK, somebody else must change the header to include [patch]
Created attachment 79580 [details] patch to include build version and os/platform in identifier string A it works now!
Created attachment 79581 [details] mylyn/context/zip
From my local logfile 127.0.0.1 - - [02/Oct/2007:22:46:32 +0200] "POST /bugzilla/show_bug.cgi HTTP/1.1" 200 8011 "-" "Eclipse Mylyn (HttpClient/3.0.1) build: 2.1.0.v20070927-0900 [Mac OS X-10.4.10-i386]" Q.E.D.
Mylyn should respect the format for the user-agent header specified in RFC 2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.43 . I would suggest to change it to something like this: Eclipse Mylyn/2.1.0.v20070927-0900 HttpClient/3.0.1
(In reply to comment #7) > Mylyn should respect the format for the user-agent header specified in RFC > 2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.43 . I would > suggest to change it to something like this: > > Eclipse Mylyn/2.1.0.v20070927-0900 HttpClient/3.0.1 > Good suggestion! I found that when using a web stats package like awstats, including OS-platform-architecture helped in adding the user agent to the respective platfom/os aggregrations. So I think OS/Platform/architecture should remain a part of the user-agent string.
Created attachment 79592 [details] Patch using Steffen Pingels suggestions Good suggestions!
Created attachment 79593 [details] mylyn/context/zip
Exposing additional system specific information might raises privacy issues however the internal web-browser also does transmit the OS and version (at least on my system): User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20061201 Firefox/2.0.0.6 (Ubuntu-feisty) Maarten, could you revise your patch to match this format?
as I think you mean as in: client/version (platform;os;architecture) library/version or in this case: Eclipse Mylyn/2.1.0.v20070927-0900 (Mac OS X;10.4.10;i386) HttpClient/3.0.1
Created attachment 79701 [details] Taking on board Steffens suggestions mimic common web browsers format as much as possible
Created attachment 79702 [details] mylyn/context/zip
Created attachment 79727 [details] updated patch to use StringBuilder
Thanks for you contribution Andrey. I have changed the patch slightly to use a StringBuilder to create the user-agent string and added a sanity check in case the bundle does not have the version header. On my system this string is generated: Eclipse (Linux 2.6.20-16-lowlatency; i386) Mylyn/2.1.0.v20070927-0900 HttpClient/3.0.1 Rob, please review and apply.
Maarten, thanks for your contribution and please excuse the name mix-up. Rob, we *need* proper real-name support in the next release :).
Guys, you forgot the Java version :-)
Created attachment 79747 [details] added Java version to user-agent The update manager uses "Java/1.5.0_11" as the user-agent. I have modified the patch to include that as well: Java/1.5.0_11 (Linux 2.6.20-16-lowlatency; i386) Eclipse Mylyn/2.1.0.v20070927-0900 HttpClient/3.0.1
Created attachment 79748 [details] mylyn/context/zip
(In reply to comment #19) > Java/1.5.0_11 (Linux 2.6.20-16-lowlatency; i386) Eclipse > Mylyn/2.1.0.v20070927-0900 HttpClient/3.0.1 This probably already went too far, but you also missing Eclipse version and vendor for the JVM. :-)
(In reply to comment #17) > Rob, we *need* proper real-name support in the next release :). +1 There is no way I'm letting another release go out without real name support... and you can quote me on that. :) Great work on this and all the config caching improvements Maarten! Patch applied, ip log updated.
(In reply to comment #22) > Great work on this and all the config caching improvements Maarten! Patch > applied, ip log updated. Umm... Rob, what about java vendor info and eclipse version? Maybe you marked this as resolved too fast.
Great stuff guys. I would like us to clean up the String a bit in order to make it friendlier to webmasters' eyeballs and parsing tools, and to include the product version (i.e. Eclipse). I agree that it would be nice to add the "java.vendor" version, not sure if I got the syntax right though (are we allowed a second slash). How about the following? Note that I ordered it by the stack, inspired by http://www.mozilla.org/build/revised-user-agent-strings.html (but I don't understand why they don't break out the OS). Mylyn/2.1.0 Eclipse/3.3.0 HttpClient/3.0.1 Java/1.5.0_11 (Sun) Linux/2.6.20-16-lowlatency (i386; en) Then if someone redistributed Mylyn they could do this: Mylyn/2.1.0 Eclipse/3.3.0 HttpClient/3.0.1 Java/1.5.0_11 (Sun) Linux/2.6.20-16-lowlatency (i386; en) MegaCorpTool/1.0 (MegaCorp, Inc.)
(In reply to comment #23) > Umm... Rob, what about java vendor info and eclipse version? Maybe you marked > this as resolved too fast. My bad, will reopen for comments on Mik's proposed changes...
(In reply to comment #24) > Note that I ordered it by > the stack, inspired by > http://www.mozilla.org/build/revised-user-agent-strings.html (but I don't > understand why they don't break out the OS). Maybe you should also be sensitive about their concerns. * There is concern that giving the operating system version on Windows reveals too much information about a system (such as potential security holes). https://bugzilla.mozilla.org/show_bug.cgi?id=57555 * There is concern that revealing localization information is a violation of privacy. https://bugzilla.mozilla.org/show_bug.cgi?id=55366
(In reply to comment #24) > Mylyn/2.1.0 Eclipse/3.3.0 HttpClient/3.0.1 Java/1.5.0_11 (Sun) > Linux/2.6.20-16-lowlatency (i386; en) But when we couldn't tell from release builds of Mylyn and Eclipse and the dev builds.
(In reply to comment #27) > (In reply to comment #24) > > Mylyn/2.1.0 Eclipse/3.3.0 HttpClient/3.0.1 Java/1.5.0_11 (Sun) > > Linux/2.6.20-16-lowlatency (i386; en) > > But when we couldn't tell from release builds of Mylyn and Eclipse and the dev > builds. Do we really need all this information ? As far as I remember from the conference call. IMO, mylyn version number is all that matters, anything else will be too difficult and not solve *any* practical purpose as far as any sort of reporting is concerned. This, I think should be Denis' call. Denis, any thoughts?
(In reply to comment #28) > not solve *any* practical purpose as far as any sort of reporting is concerned. There is *very* practical purpose. Given that Mylyn is included into almost all default packages, this info will practically give a real time picture of what users are actually running. Getting such report is simply a matter of running some log analyzer.
> IMO, mylyn version number is all that matters, anything else will be too > difficult and not solve *any* practical purpose as far as any sort of reporting > is concerned. Agreed. I also agree with Gunnar's statements in comment 26. (In reply to comment #29) > this info will practically give a real time picture of what > users are actually running. Getting such report is simply a matter of running > some log analyzer. Actually, I don't typically log the user agent on bugs.eclipse.org. I enabled it some time ago to troubleshoot some performance problems, but I will re-disable it. At any rate, our privacy policy and terms of use have restrictions as to what we can and cannot do with that information. IMHO, Mylyn/2.1.0 Eclipse/3.3.0 would be sufficient -- but that's just me.
(In reply to comment #29) > (In reply to comment #28) > > not solve *any* practical purpose as far as any sort of reporting is concerned. > > There is *very* practical purpose. Given that Mylyn is included into almost all > default packages, this info will practically give a real time picture of what > users are actually running. Getting such report is simply a matter of running > some log analyzer. > Agree, also note that eclipse.org is NOT THE ONLY bugzilla accessed using Mylyn. It is just another browser and should tell a host as much. Some sites do run log analyzers and it provides useful information. The above is a case for moving the code into the main plugin and making it API, getMylynClientString() sot that JIRA, Trac, generic webconnector and all the other can use it as well. Somehow I always hope it will reveal that us Mac users are NOT A MINORITY so its worthwile to fix bug 68111 ;-)
> > There is *very* practical purpose. Given that Mylyn is included into almost all > > default packages, this info will practically give a real time picture of what > > users are actually running. Getting such report is simply a matter of running > > some log analyzer. > > > Agree, also note that eclipse.org is NOT THE ONLY bugzilla accessed using > Mylyn. The current user-agent string already identities Mylyn. The goal of this bug report is to determine what other additional information is useful in that string. > It is just another browser and should tell a host as much. Some sites do run > log analyzers and it provides useful information. There is a difference between useful in the sense of interesting and what is required on a technical level. The purpose of the user-agent is to allow web-servers to respond differently if a certain version of a product requires it. Therefore any information other than the Mylyn and HttpClient version seems unnecessary and in the case of potentially private information such as the locale there should be an option to disable inclusion in the user-agent. If there is a need to collect statistics about the Eclipse user base I suggest to implement a separate plug-in that will make this explicit to the user or possibly include this in the Mylyn monitor plug-in. > The above is a case for moving the code into the main plugin and making it API, > getMylynClientString() sot that JIRA, Trac, generic webconnector and all the > other can use it as well. All connectors should invoke WebClientUtil.setupHttpClient() which configures the user-agent. Please file a bug report if that does not work properly.
(In reply to comment #32) > > It is just another browser and should tell a host as much. Some sites do run > > log analyzers and it provides useful information. > There is a difference between useful in the sense of interesting and what is > required on a technical level. Given the frequent Mylyn releases you won't be able to tell if users are running version with some particular patch/change or not, and it won't allow to do the same for Eclipse version. > The purpose of the user-agent is to allow web-servers to respond differently if a > certain version of a product requires it. Therefore any information other than the > Mylyn and HttpClient version seems unnecessary and in the case of potentially > private information such as the locale there should be an option to disable > inclusion in the user-agent. BTW, locale is always passed by a web browser in a completely different header, so there is no privacy issue there. > If there is a need to collect statistics about the Eclipse user base I suggest > to implement a separate plug-in that will make this explicit to the user or > possibly include this in the Mylyn monitor plug-in. Why that would be necessary if existing path could be used? I do understand privacy issues Denis was referring to, but that is a matter of keeping that information private, as long as only aggregated data published.
-1 for differentiating minor-minor-minor Eclipse and Mylyn date String, and keeping the version String at something like Mylyn/2.1.0. Length is important because some loggers truncate the user agent string by default. Also, in many cases we will be able to recognize that it is an integration build because it's version will be ahead (e.g. 2.2.0 for this week's build). I agree with prioritizing privacy concerns since that's a guiding principle for Mylyn, but I think that we should be matching the current browser implementations since they represent the state of the art (afaik Mozilla still has OS and localization, MSIE has that as well as CLR version, and WebKit works like Mozilla). I'm happiest with our following the Mozilla pattern, so if anyone knows that they are or will soon be logging a subset of this, please post and we can prune accordingly. On quick read of the bugs linked in comment#26 wasn't clear to me from the bug reports if/when they would change. But if enough people feel strongly about this we can remove this info, at the cost of being less standard. Combining feedback above, and looking over some http log files for formatting and other conventions, here is a summary of important info and explanations of why it is useful diagnosing how the Mylyn client is accessing a server. 1) Mylyn: know we need it, need to include connector since it controls the connection and in some cases will be more relevant than the Mylyn version 2) Eclipse: we depend on org.eclipse.core.net which can affect connections, have folded in the redistribution into the comment, but the redistribution is very important to include because it can affect how Mylyn behaves and who supports it 3) HttpClient: used for most connections 4) Java: java.net is used by HttpClient, can be used by us (though we avoid it) 5) OS: following browser convention So I'm still ending up with the following String: Mylyn/2.1.0 (Rally Connector 1.0) Eclipse/3.3.0 (JBuilder 2007) HttpClient/3.0.1 Java/1.5.0_11 (Sun) Linux/2.6.20-16-lowlatency (i386; en) If anyone would like to provide a shortened version please post arguments for removing one of the 1-5 entries and the resulting string. Also please vote on the privacy issue. My vote is: +1 for consistency with latest Mozilla implementation, since in my opinion it sets the de facto standard for finessing the privacy vs. utility tradeoff
+1 for consistency with latest Mozilla implementation, as that is: - most likely to be parsable by all web stats packages: awstats, webalyzer, etc. and provide desired feedback to host owner - most usable for traffic reduction/optimization efforts using mod_rewrite etc.
(In reply to comment #34) > 1) Mylyn: know we need it, need to include connector since it controls the > connection and in some cases will be more relevant than the Mylyn version Retrieve version info from WebcorePlugin Bundle > 2) Eclipse: we depend on org.eclipse.core.net which can affect connections, have > folded in the redistribution into the comment, but the redistribution is very > important to include because it can affect how Mylyn behaves and who supports it Retrieve version info from osgi.framework.version=3.3.1.R33x_v20070828 > 3) HttpClient: used for most connections Retrieve from HttpClientParams.getDefaultParams().getParameter(HttpClientParams.USER_AGENT); > 4) Java: java.net is used by HttpClient, can be used by us (though we avoid it) create from: java.vendor=Apple Computer, Inc. java.version=1.5.0_07 > 5) OS: following browser convention create form: os.arch=i386 os.name=Mac OS X os.version=10.4.10 So everything can be done, except: - connector identifier, this must go into setupHttpClient as a parameter - redistribution, is there a retrievable property for that too, like
Created attachment 79888 [details] Patch to (almopst) create a User Agent string as Mik suggests Still Missing: - redistribution - connector info
Created attachment 79940 [details] Patch that (almost, but now correctly) implements Mik's preference Missing: - Task repository client ID - Redistribution ID
Maarten: great stuff! I'll let Rob take over on applying the patch. (In reply to comment #38) > - Task repository client ID Seems like WebClientUtil will have to be setup by the connector to either take an instance of the connector plug-in or a version String. Then the version can be retrieved from the bundle. > - Redistribution ID Seems like the following property is best, since it is automatically set in RCP apps: eclipse.product=org.eclipse.platform.ide
Path applied, ip log updated.
distro and connector left for later
(In reply to comment #41) > distro and connector left for later Created bug 206496
(In reply to comment #42) > (In reply to comment #41) > > distro and connector left for later > > Created bug 206496 connector requires API change as it has to be passed into setupHttpClient(), so needs to be done for all callers: bugzilla, trac, monitor, jira, webrepository, etc. When is that feasible? distro: we need some dfeedback on how distro's show up in the properties. Ask on the newsgroup? Should these be sub tasks?
