Bugzilla – Full Text Bug Listing
| Summary: | Make the Installer the default download | ||
|---|---|---|---|
| Product: | Community | Reporter: | Mike Milinkovich <mike.milinkovich> |
| Component: | Website | Assignee: | Eric Poirier <eric.poirier> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | chris.guindon, denis.roy, Ed.Merks, ian.skerrett, mknauer, stepper |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Windows 7 | ||
| See Also: |
https://git.eclipse.org/r/44781 https://git.eclipse.org/c/www.eclipse.org/downloads.git/commit/?id=98bf22ad1262ab2436ea94f92201848e60932762 https://git.eclipse.org/r/55760 https://git.eclipse.org/c/www.eclipse.org/downloads.git/commit/?id=128cdc233ad97357224d531b30bef8495e55fb41 |
||
| Whiteboard: | |||
|
Description
Mike Milinkovich
How about we keep the packages on the page but we hide them by default? We would only show the installer by default but we could include a new "Show all Packages" button. If the user clicks on the "Show all Packages", all the packages would be displayed on the page. We could toggle the "Show all Packages" button to hide all the packages if we are currently showing the full list of packages. Another UI option would be to use tabs: http://getbootstrap.com/javascript/#tabs (In reply to Christopher Guindon from comment #1) > How about we keep the packages on the page but we hide them by default? > > We would only show the installer by default but we could include a new "Show > all Packages" button. I think that's a great compromise. I don't have any strong opinions about the UI. I suggest that we consciously experiment with different options and see what works best. So go ahead and try something :) New Gerrit change created: https://git.eclipse.org/r/44781 Here's a quick prototype of my suggestion: http://staging.eclipse.org/downloads/index-developer.php My only concern here is that, we are hiding the promoted download by default. Should we include the promoted download under the installer? (In reply to Christopher Guindon from comment #6) > Should we include the promoted download under the installer? I'll let Ian answer that question, but makes sense to me. Oddly, even if I click on "Show all packages", I don't see the sponsored download. (In reply to Mike Milinkovich from comment #7) > (In reply to Christopher Guindon from comment #6) > > Should we include the promoted download under the installer? > > I'll let Ian answer that question, but makes sense to me. > > Oddly, even if I click on "Show all packages", I don't see the sponsored > download. Ad-block might be removing it for you. Hi Ian, I just noticed that over the weekend approximately 2000 people have downloaded M6 the traditional way, which is what we wanted to avoid. Chris told me that he's currently waiting for your answer to the question of how to deal with promoted downloads comments #6 and #7). Please let us know ;-) Chris and I talked. Here is what we agreed to do. We will have two download pages so we can actually do A/B testing. Page A will be the current Milestone 6 download page. Page B will show the Installer, Promoted Download and a 'Show All Packages' link. Page A and B will be randomly severed to the end user. We will try to make it 50-50 but will count impressions. We will measure the following: - Number of Installer downloads. This is currently not being done since the Installer is not a Package. - We will track the number of people that select 'Show All Packages' on Page B - We will also track the click thru rate of the Promoted Download for each page. (In reply to Ian Skerrett from comment #10) > We will have two download pages so we can actually do A/B testing. Page A > will be the current Milestone 6 download page. Page B will show the > Installer, Promoted Download and a 'Show All Packages' link. But both of those versions would be the Developer Builds tab that virtually no one uses, right? Or would one of those versions be the actual downloads page? Perhaps I'm mistaken, but I think Mike's suggestion is to make your version B what is seen on the main downloads page seen by millions (not the Developer Builds). (In reply to Denis Roy from comment #11) > (In reply to Ian Skerrett from comment #10) > > We will have two download pages so we can actually do A/B testing. Page A > > will be the current Milestone 6 download page. Page B will show the > > Installer, Promoted Download and a 'Show All Packages' link. > > But both of those versions would be the Developer Builds tab that virtually > no one uses, right? Or would one of those versions be the actual downloads > page? > > Perhaps I'm mistaken, but I think Mike's suggestion is to make your version > B what is seen on the main downloads page seen by millions (not the > Developer Builds). I would never claim to speak for Mike but if we want to do A/B testing we need to compare two different Milestone download pages. (In reply to Denis Roy from comment #11) > Perhaps I'm mistaken, but I think Mike's suggestion is to make your version > B what is seen on the main downloads page seen by millions (not the > Developer Builds). That is the eventual goal, once we ship Mars. The idea is that between now and the final Mars release we conduct some experiments to ensure that we are constructing the page the right way. That way we could actually have some data and experience, rather than just opinions about the right construction for the page. We should hopefully be able to run several different experiments in the M6/M7/RCx cycles. Are any decisions outstanding or can we go on with this now? (In reply to Eike Stepper from comment #14) > Are any decisions outstanding or can we go on with this now? AFAICT we should try out the new approach ASAP. (In reply to Ian Skerrett from comment #10) > Chris and I talked. Here is what we agreed to do. > > We will have two download pages so we can actually do A/B testing. Page A > will be the current Milestone 6 download page. Page B will show the > Installer, Promoted Download and a 'Show All Packages' link. Page A and B > will be randomly severed to the end user. We will try to make it 50-50 but > will count impressions. > > We will measure the following: > - Number of Installer downloads. This is currently not being done since the > Installer is not a Package. > - We will track the number of people that select 'Show All Packages' on Page > B > - We will also track the click thru rate of the Promoted Download for each > page. I just made an update for staging, this is what I did: http://staging.eclipse.org/downloads/index-developer.php 1. I created a unique Eclipse campaign link for each oomph download link. We will be able to track the download count for oomph on test page A(new) and test page B(old) separately. 2. http://staging.eclipse.org/downloads/index-developer.php will now redirect you to test page A (new) http://staging.eclipse.org/downloads/index-developer-oomph.php or test page B (old) https://staging.eclipse.org/downloads/index-developer-default.php I created two pages because we want to leverage google analytics in-page analytics. 3. I am now hiding the packages with css to fix the flicking on page load for the new download page. 4. I am now tracking impressions of test page A and test page B. 5. The promoted download links is listed under oomph on the new download page. Made another small change to my patch: 1. the copyright year was updated. 2. If the user has JavaScript disabled and click on the show all packages link, he will be redirected to the default download page that shows all the packages. I am ready to go live with this. > I just made an update for staging, this is what I did:
> http://staging.eclipse.org/downloads/index-developer.php
It seems to work as expected. +1 for pushing this out.
Gerrit change https://git.eclipse.org/r/44781 was merged to [master]. Commit: http://git.eclipse.org/c/www.eclipse.org/downloads.git/commit/?id=98bf22ad1262ab2436ea94f92201848e60932762 (In reply to Denis Roy from comment #18) > > I just made an update for staging, this is what I did: > > http://staging.eclipse.org/downloads/index-developer.php > > It seems to work as expected. +1 for pushing this out. It's live! https://www.eclipse.org/downloads/index-developer.php https://www.eclipse.org/downloads/index-developer-oomph.php https://www.eclipse.org/downloads/index-developer-default.php (In reply to Christophér Guindon from comment #20) > (In reply to Denis Roy from comment #18) > > > > It seems to work as expected. +1 for pushing this out.' I just noticed that the URL used was [1]. The reference to Oomph should be removed, as this is being re-branded as the Eclipse installer. [1] http://www.eclipse.org/downloads/index-developer-oomph.php ~~~~~ (In reply to Mike Milinkovich from comment #21) > (In reply to Christophér Guindon from comment #20) > > (In reply to Denis Roy from comment #18) > > > > > > It seems to work as expected. +1 for pushing this out.' > > I just noticed that the URL used was [1]. The reference to Oomph should be > removed, as this is being re-branded as the Eclipse installer. > > [1] http://www.eclipse.org/downloads/index-developer-oomph.php > ~~~~~ Done: http://git.eclipse.org/c/www.eclipse.org/downloads.git/commit/?id=8075ac775ab2673935c2e77c5a54d17c2e059a45 http://www.eclipse.org/downloads/index-developer-installer.php I think we are done here. Thanks everyone! I thought I share some numbers with you: As of today the M7 packages have been downloaded 2057 times. Our M7 installers have been downloaded 2038 times. In addition our nightly builds have been downloaded 2598 times. Makes a total of 4636. I assume that a considerable number of installer users (especially on Linux and Mac) keep their installers and just use its p2 self-update), so it's hard to compare these numbers with the traditional package downloads. For how long do you plan to keep this random offering of the traditional package downloads in addition to the installer download? (In reply to Eike Stepper from comment #24) > For how long do you plan to keep this random offering of the traditional > package downloads in addition to the installer download? I am not sure what you are referring to? Are you asking about the "Show all packages" option that is shown below the installer? Currently https://www.eclipse.org/downloads/index-developer-default.php randomly shows either: 1) The installer and a "Show all packages" link, or 2) The installer and a full list of the package downloads. That's probably because of Ian's A/B testing comment #10. I wanted to know if it's planned to change that to always show 1) and if so, when ;-) (In reply to Eike Stepper from comment #26) > Currently https://www.eclipse.org/downloads/index-developer-default.php > randomly shows either: > > 1) The installer and a "Show all packages" link, or > 2) The installer and a full list of the package downloads. > > That's probably because of Ian's A/B testing comment #10. > > I wanted to know if it's planned to change that to always show 1) and if so, > when ;-) We haven't decided. The last time I looked at the stats there wasn't a huge difference between the A or B. I expect we will do a number of different experiments up until the release. We're very interested in broad community testing of the new install experience before Mars comes. Personally I have no idea what A/B testing is or what value it provides. What other experiments do you have in mind and when could they possibly start and end? Chris and I looked at the Google Analytics for the A/B testing we did for the two versions of the Developer Build download page: A: Just Installer and Promoted Download listed but allows to Show All packages https://www.eclipse.org/downloads/index-developer-installer.php B: All package listed https://www.eclipse.org/downloads/index-developer-default.php According to Google Analytics, on Page A more than 42% left this page without downloading the installer or a package. For Page B, only 28% left this page without downloading the instalelr or a package. When people actually download something, on Page A the installer download represents the bulk of the downloads, over 80%. The next most popular download is the Java EE package. For Page B, the JavaEE package and the Installer have close to equal number of downloads. One conclusion I take from these results is that when we only present the Installer download, people tend to get confused and leave the page. I believe we need to do more to improve educating people on why the Installer is worth downloading. I will work with Chris on this. I also need to look at the impact on the Promoted Download. Re-opening since I don't believe we are done. We are doing this today, after some discussion. Eric will take the Installer block from the Installer page, and put it over the list of packages. New Gerrit change created: https://git.eclipse.org/r/55760 Gerrit change https://git.eclipse.org/r/55760 was merged to [master]. Commit: http://git.eclipse.org/c/www.eclipse.org/downloads.git/commit/?id=128cdc233ad97357224d531b30bef8495e55fb41 The installer is now on the default downloads page. I also removed the installer link from the sidebar and deleted the installer page. I'm now closing this bug. Thanks! |