Bugzilla – Full Text Bug Listing
| Summary: | RSS feed for official eclipse.org releases | ||
|---|---|---|---|
| Product: | Community | Reporter: | Bjorn Freeman-Benson <bjorn.freeman-benson> |
| Component: | CommitterTools | Assignee: | Eclipse Webmaster <webmaster> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | antoine, karl.matthias, nboldt, remy.suen |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Bug Depends on: | |||
| Bug Blocks: | 126471, 130150 | ||
|
Description
Bjorn Freeman-Benson
I think this is a great idea, certainly one that the entire community could benefit from, but it sounds like an uphill battle because of the lack of consistency. What would it take to "force" the projects to comply with a set of policies for naming download files? Also, do you want all the individual files in this RSS, or just the directory to the actual release? Other than that, as a first step, I think someone should map the project vs. convention here in this bug just to get an idea of what's what. For instance: Eclipse: /eclipse/downloads/drops/R* CDT: cdt/releases/*/dist/* Thoughts? Adding Karl as he's likely the one who will implement this. I can work towards convention and with the standard build infrastructure it will be more so, but in the meantime :-( I was just looking for the project -> release -> link to a way to download it (an update or the download.php redirect for the file) I'll talk to you about this when I'm back in Portland, Bjorn. The other approach is to record details about each build in your own feed(s), and publish those feed(s) in a central place or have them aggregated by a centralizing script. There are currently at least three projects that publish RSS feeds for their builds, with more coming soon: http://wiki.eclipse.org/index.php/Eclipse_Build_Available_RSS_Feeds#Actual_Feeds Projects can choose to publish using that format, and then provide a link to their feed(s) on their downloads/home pages, like EMF does (see the "Atom 1.0" icons): http://www.eclipse.org/emf/downloads/ Ideally, there'd be a way for projects to "register" their feeds w/ the foundation so that anyone can find the full list of feeds from a single page, like is now possible for newsgroups and mailing lists. Perhaps a new entry in our project-info.xml files could contain feed URL(s), which could then be aggregated by a page like http://www.eclipse.org/feeds/. Since the feeds also contain links to downloads, release notes, update manager sites, and Europa status, aggregated feeds could be morphed into a sort of Europa dashboard for watching all the participating projects through one tabular or tree-view portal. $0.02. ;-) It would be nice to have some automated way to just find new releases but with the huge number of files and differing paths that might not work. I know Nick's solution might not cover everything Bjorn is looking for but it might be a more supportable solution. I just had an idea about how we can make this work somewhat automatically and sanity checked it with Denis. We can pretty easily build an RSS feed of the new files from the last 24 hours and publish that. The filenaming scheme is not universal, though, so we're likely to pick up lots of uninteresting files and will need to use some regular expressions to do some minimal sanitization. We wouldn't be able to get any kind of a file description but we could group files by project and could try to filter to releases when the project layout makes it obvious. I'll see if I can whip up an example in the next week or so and we can talk about it. Ideally everyone would use the same project release structure but I guess that's asking a lot at this point. :) (In reply to comment #7) This is brilliant. The download file index table is a great solution in that it only contains relevant "downloadable" files (.jar, .zip, etc) whereas scanning the downloads directory tree will require you to skip php, jpg, html files etc.... Gee, thanks. :) (In reply to comment #5) Yes, if everyone used build RSS feeds, we could do it that way. But _I_ need a solution that handles the case where people don't create build RSS feeds (more history: I need a way to verify that Releases have gone through the Release Review , especially the Eclipse Legal review). However, I think we could split this bug into two pieces: (1) For the EMO (me) based on comment #7. (2) For the world based on comment #5. That would encourage projects to publish their build RSS stream for sure - it would also mean that the EMO version doesn't need to be as refined or clever. There is a Real Need for this - not only for EMO, but for the general public as well, and even within our committer community (projects want to know when other projects release). Until we have a centralized publishing mechanism, as proposed in comment 5, I think the total lack of naming consistency across all projects makes a screenscrape approach a) inaccurate and b) unreliable. Actually, I think not having a centralized build publisher is a blocker. I'd like to suggest all projects be required to publish release information. Comment 5 seems to be a good solution - although I think each project should simply have a builds.xml file at the root of its downloads directory. Bjorn, what are your thoughts on requiring all projects to publish a build feed? (In reply to comment #11) > There is a Real Need for this - not only for EMO, but for the general public as > well, and even within our committer community (projects want to know when other > projects release). FYI, I've updated the wiki a little to make getting started a bit easier (and fixed old references to where the EMF feed *used* to be before it moved to Modeling). If people want control over how/where they publish their feeds (rather than being told "put it here: _____", we'll just need them to do something to tell /some tool/ where the feeds are. Could be a project-info metafile, could be a dev.eclipse.org committer tool... whatever's the path of least resistance will ease the adoption. (In reply to comment #11) > Bjorn, what are your thoughts on requiring all projects to publish a build > feed? I think that this is a reasonable request although we should probably do this: 1. support an optional build rss feed 2. request the planning council to endorse a required build rss feed 3. suggest how to integrate the rss generating into standard eclipse build configurations 4. then, if the community agrees that this is a good idea, start to require it (In reply to comment #13) > 3. suggest how to integrate the rss generating into standard eclipse build > configurations http://wiki.eclipse.org/index.php/Eclipse_Build_Available_RSS_Feeds provides details on where to get code from org.eclipse.releng.basebuilder, how to integrate it into teams' existing build/promote scripts, feeds already being published, and links to CVS for script/property file examples. If it needs to be rewritten to be more concise or more step-by-step, I'm open to assisting w/ that effort. (In reply to comment #13) > 1. support an optional build rss feed By this I meant to say "yes, let's have a central RSS feed for official release files for all eclipse.org projects. This feed will be generated by aggregating all the builds.xml feeds from all the projects. Those builds.xml will be optional at first (and thus may be missing), although should become required after some time." Bjorn is it your intention that we would put those "builds.xml" files into the Project Info database instead, now that we're working on that? (In reply to comment #16) Yes. It will be the "releases" key. I have built an RSS feed from the information contained in the ProjectInfo tables and it's now available for testing at: https://dev.eclipse.org/testcommitters/committertools/rssfeed_downloads.php This is not where it will live, but I want to make sure this accomplishes what we're talking about before we push it live. The current feed covers the previous six months worth of information. Clearly there is a bit of a data validation problem caused by the fact that the current data set is from the hand-typed XML files. Hence the file released in November 2007. When we build the portal component to allow projects to manage this data, we'll get tighter control over this and should be able to eliminate problems like that. I would also like to add a field that gives us a better description of what each release contains so that it can be syndicated as well. For now this is the data we have. Ok I have released the feed based on the existing information in the ProjectInfo database. This will shortly be update-able by the projects when we release a MyFoundation Portal component to control this information. The RSS feed is available from the downloads page and shows up both in the title bar and in the URL window on supported browsers. Closing RESOLVED/FIXED. (In reply to comment #18) > I have built an RSS feed from the information contained in the ProjectInfo > tables and it's now available for testing at: How about syndicating multi-project data from actual (generated) feeds, instead of manual project-info keywhackery files? http://wiki.eclipse.org/Eclipse_Build_Available_RSS_Feeds#Actual_Feeds If everyone was using that format you could filter on type="R" for releases, type="S" for milestones/RCs/stables, and provide a feed for each type of release. FWIW, these feeds already work for syndication, like on my blog where I list both Eclipse and EMF builds: http://divby0.blogspot.com/ (see "Eclipse Build Feeds"). (yes, this is basically the same $0.02 as in comment 5.) |