Bugzilla – Full Text Bug Listing
| Summary: | Remove direct eclipse signing requirement. | ||
|---|---|---|---|
| Product: | Community | Reporter: | Jesse McConnell <jesse.mcconnell> |
| Component: | Architecture Council | Assignee: | eclipse.org-architecture-council |
| Status: | CLOSED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | bugs.eclipse.org, contact, david_williams, denis.roy, d_a_carver, gunnar, joakim.erdfelt, john.arthorne, jvermillard, kim.moir, mary, mike.milinkovich, mknauer, mober.at+eclipse, overholt, rsternberg, sbouchet, slewis, wayne.beaton |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Mac OS X - Carbon (unsup.) | ||
| Whiteboard: | stalebug | ||
| Bug Depends on: | |||
| Bug Blocks: | 438696 | ||
|
Description
Jesse McConnell
There is no specific requirements regarding JAR signing in the EDP. This is a requirement of participation in the release train. Regardless, I think it's a good idea to have this discussion. Do you have a pointer to more information about what Apache does? I'm moving this bug to Architecture Council so that we can have them weigh in on this discussion. http://www.apache.org/dev/release-signing.html more specifically I am thinking along the lines of http://www.apache.org/dev/release-signing.html#web-of-trust IMO any p2 repo generated and available from eclipse ought to be signed by _someone_ and following a web of trust model seems like a good idea. thanks for the consideration of the issue wayne I'm not opposed to the web of trust, but I worry that we've only just gotten to the point where a lot of builds are done at eclipse.org and that if we accept this, we'd be moving away from that, back to random build machines. My main concern is ensuring that builds are reproducible. I agree with Andrew. It would be better to address the bottleneck problem with respect to signing with more hardware resources to allow teams to sign builds in a more timely manner. teasing apart the concerns here... build reproducibility should have little to do with signing, building with the continuous integration setup at eclipse is a fine goal and uncovers many issues...but should be wholly separate from the signing discussion building a release should not be a continuous integration process...it is a deliberate point in time that leaders deems the project ready. At that point all manner of things should happen, the SCM should have tags set with the relevant information, the artifacts produced should be signed and checksummed and all that jazz. then the resulting artifacts should be put somewhere durable where they remain unchanged, be it a p2 repo or a maven repo what I see as an issue under the current CI centric view of things is that projects seem to take arbitrary points in time, the output from the CI system and call it a 'release' and unless everyone is doing something with tagging in their SCM's that I don't know about automatically via hudson I don't really see how these things are 'reproducible' in the release engineering sense. There is some rejiggering and renaming of p2 repositories and whatnot...but not necessarily reproducible from a clean source checkout. release engineering and poaching working signed builds from the CI are not the same thing allowing for a web of trust model like this could promote good release engineering, especially if you encouraged signed released p2 repositories to have corresponding tags in SCM, which it really should be the case for releases. I am only tangentially aware of some of these processes though, so perhaps I have missed something (In reply to comment #3) > I'm not opposed to the web of trust, but I worry that we've only just gotten to > the point where a lot of builds are done at eclipse.org and that if we accept > this, we'd be moving away from that, back to random build machines. My main > concern is ensuring that builds are reproducible. I agree with Jesse that ensuring that builds are reproducible is a separate thing from signing...just as an example...ECF has it's reproducible build on our own project machine: https://build.ecf-project.org/jenkins/ yet we still use the eclipse.org signing process. I would prefer the signing model that Jesse describes (web of trust) as it's simpler from the project's point of view. (In reply to comment #4) > I agree with Andrew. It would be better to address the bottleneck problem with > respect to signing with more hardware resources to allow teams to sign builds > in a more timely manner. If it's a choice between more resources for centralized hardware and more resources for projects (less time spent/simpler from releng point of view), I vote for more simplifying/streamlining the build/signing process (i.e. for less time/simpler for projects). I like the idea of delegating signing. It definitely allows to scale better. The web-of-trust model is great for trusting people. But... IMHO Eclipse is also attractive for enterprises. I wonder what happens if a corporate user goes to the about page of his products and reads "signed by somedude@luckydudes.org". Enterprises don't like that (IMHO). Thus, they still want a signature from Eclipse.org (I imagine). However, this makes Eclipse.org an CA. Running a CA is not a light undertaking. It requires quite some infrastructure, processes, audits and the like. (In reply to comment #5) > build reproducibility should have little to do with signing I think that's the case already. The Eclipse signing process works like a HSM. You upload your zip/jars to some storage and you'll get it back signed. That really works on its own and can be done by a human as part of a release process. It's just that a lot people have scripted it into their build systems. PDE Build folks tend to do it with Ant. Maven folks wrote their own plug-in for that. However, no one requires it to be part of your build process. Thus, I don't really see why that is an issue for you. > release engineering and poaching working signed builds from the CI are not the > same thing It sounds like your main issue is that a lot folks have scripted the signing process into their builds. Frankly, I don't see how a web-of-trust model would change that. People will end up putting their keys on build machines and script signing again. IMHO this is much more dangerous than have a HSM like solution offered by the Eclipse signing process. > I am only tangentially aware of some of these processes though, so perhaps I > have missed something I'm not sure how the Buckminster and Maven folks do it. The builds I saw so far assume a checkout happened before (eg., by some CI system) PDE Build has a built-in feature that encourages "tag-first-before-you-build". You explicitly have to turn it off. It's even so tightly integrated that artifact/bundle versions are generated based on SCM tag. For example, if the SCM tag doesn't change the bundle version stays the same. The Equinox, Platform and e4 teams exercise that extensively. A lot effort has been put into the tools and processes recently in order to make it work with Git. If there's any way that we can improve the signing process while not compromising our PK, security or signing integrity, I'm all ears. > It would be better to address the bottleneck problem with
> respect to signing with more hardware resources
FWIW, as Indigo just shipped, I don't remember seeing, or reading about any bottlenecks despite being crippled by hardware failures throughout the month of June. One signer thread can only use one CPU core, and all our build servers are using some of the fastest CPUs out there. The key is to split huge signing tasks into smaller ones that can run in parallel, taking advantage of the high numbers of CPU cores available.
(In reply to comment #6) > If it's a choice between more resources for centralized hardware and more > resources for projects Judging by how many projects have migrated away from project-centric releng onto centralized releng, I would suggest that although some projects seem to enjoy tinkering with a releng system of their own, most do not. tycho needs a release plugin akin to the maven-release-plugin, which I suspect will exist at some point...then all of this gets a whole lot more normal and simple. honestly this argument on my side is a bit self-serving since it annoys me to no end that I can't use the artifacts we release into maven central for the eclipse builds like indigo and juno. Instead we have to wire up a separate build that consumes those artifacts and then rejiggers them for eclipse. They are already osgi bundles (felix runs them) and the artifact is signed with checksums, etc...but they are wholly inappropriate for the eclipse build. perhaps tangential to this discussion is the one of artifact/bundle signing vs class file signing as is being done right now. I can see that for the master download, the eclipse that everyone downloads, etc...but its a bit heavyweight to force anyone else consuming those artifacts to have to use SecureClassloader to load anything. My understanding from talking to Jeff McAffer a couple years ago there was a hack in equinox to speed up (ignore?) the secure classloader usage. but I digress, ultimately I would like to see us able to use a general signing solution as opposed to a bespoke one, and forcing every bundle of all 'releases' through build.eclipse.org (not to mentioned managed by each project) seems a little silly to me. The end all be all final product? sure...but every incremental step getting there is a bit much to ask of projects imo. Which leads to the point about projects moving to a centralized process, I think the reason behind that is that we are being asked to do strange magical things to our artifacts that fly against reason...and instead of having each project have to iron out some strange pack, sign, turn around, repack, do the hokey pokey, fix checksums in metadata files, then promote a p2 repo some arbitrary place and then have to go edit some map file to point at it...it makes sense for projects on limited budgets to try and shove that magic onto someone else to manage. I know it took me quite a while to wrap my head around it when we came to eclipse. Previous to that we just built jar/bundle files and put them in a centralized place...signed (the file not the classes) and checksummed. People know how and where to consume them and the onus was entirely on them to consume it. making the contribution to release train easier for projects would be nice....just make it so projects only need to produce artifact files that are signed and checksumed by someone that is trusted...and then during the aggregation process you can write logic once that packs and repacks and adjusts metadata, etc etc as a part of one process (In reply to comment #10) > (In reply to comment #6) > > If it's a choice between more resources for centralized hardware and more > > resources for projects > > Judging by how many projects have migrated away from project-centric releng > onto centralized releng, I would suggest that although some projects seem to > enjoy tinkering with a releng system of their own, most do not. I'm not sure it's a matter of whether they enjoy it or not...rather, I suspect it's a matter of resources and releng difficulty that moves many projects to use the centralized releng. (In reply to comment #5) > what I see as an issue under the current CI centric view of things is that > projects seem to take arbitrary points in time, the output from the CI system > and call it a 'release' and unless everyone is doing something with tagging in > their SCM's that I don't know about automatically via hudson I don't really see > how these things are 'reproducible' in the release engineering sense. There is > some rejiggering and renaming of p2 repositories and whatnot...but not > necessarily reproducible from a clean source checkout. I don't know what all projects are doing, but many projects do tag their source code (and the builder used to build it) each time they build. This does ensure that the build is entirely reusable. I think most projects aren't practicing a pure CI (build on every single commit) approach. On large projects where the build and tests take hours and there are dozens of commits a day, it doesn't scale (well, maybe if Amazon was doing our builds ;). The philosophy being used in the Eclipse SDK, which I suspect some other projects are following, is that every scheduled build is a potential release. I.e., the approach isn't to take a random CI build and call it a release, but rather to perform the exact same build whether it is the first build of the release or the last. The problem with doing different build steps for regular builds vs releases is that you are no longer testing exactly what you are releasing. For example when we introduced signing, we found several bugs and performance problems that would have been discovered too late if signing was only done at release time: http://wiki.eclipse.org/JAR_Signing#What_effect_does_signing_have_on_runtime_behaviour.3F By doing the exact same build every time, it helps ensure we are continuously testing and validating the same software that we are releasing to others. Anyway, I just wanted to shed some light on the reasoning behind the current build approach being used by many projects. That is just a minor point in a great overall discussion. There certainly might be ways the signing process could be simplified or federated to make life easier for projects. And there may be room to reduce signing impact by signing less builds (for example signing only weekly builds rather than daily builds). (In reply to comment #9) > > It would be better to address the bottleneck problem with > > respect to signing with more hardware resources > > FWIW, as Indigo just shipped, I don't remember seeing, or reading about any > bottlenecks despite being crippled by hardware failures throughout the month of > June. One signer thread can only use one CPU core, and all our build servers > are using some of the fastest CPUs out there. The key is to split huge signing > tasks into smaller ones that can run in parallel, taking advantage of the high > numbers of CPU cores available. FWIW, The idea that the Eclipse Release Train is the only way projects at eclipse do their releases is wrong. There are more and more projects that release with no dependency to the Eclipse Release Train or the Eclipse Runtime. Failure to ignore this fact will result in lopsided discussion on the topic brought up in this Bug. (In reply to comment #14) > > FWIW, The idea that the Eclipse Release Train is the only way projects at > eclipse do their releases is wrong. I'm not sure how you've interpreted that from this discussion, since that was clearly not said. I brought up Indigo in the context of a signing _bottleneck_ since, if there was a time when signing could become a bottleneck, surely it would be at a time when 60 projects are all working towards a common release date. > Failure to ignore this fact will result in lopsided discussion on the topic > brought up in this Bug. And that is why you're here to provide more insight into the discussion with positive, constructive comment, right?
> I'm not sure how you've interpreted that from this discussion, since that was
> clearly not said. I brought up Indigo in the context of a signing _bottleneck_
> since, if there was a time when signing could become a bottleneck, surely it
> would be at a time when 60 projects are all working towards a common release
> date.
well, he is correct reading my response since I certainly made that assumption with the pack sign repack jingle. I think that is a requirement strictly of contributing to the release train..
or not, I get confused some times about that
(In reply to comment #16) > > I'm not sure how you've interpreted that from this discussion, since that was > > clearly not said. I brought up Indigo in the context of a signing _bottleneck_ > > since, if there was a time when signing could become a bottleneck, surely it > > would be at a time when 60 projects are all working towards a common release > > date. > > well, he is correct reading my response since I certainly made that assumption > with the pack sign repack jingle. I think that is a requirement strictly of > contributing to the release train.. > > or not, I get confused some times about that Actually, anything regardless of it being on the release train or not, if it is officially released it needs to go through the signing process. So all projects that do a public release from eclipse need to sign their stuff. (In reply to comment #17) > Actually, anything regardless of it being on the release train or not, if it is > officially released it needs to go through the signing process. So all > projects that do a public release from eclipse need to sign their stuff. This is incorrect. See comment #1. The EDP says nothing about signing. This is only a release train requirement. (In reply to comment #18) > (In reply to comment #17) > > Actually, anything regardless of it being on the release train or not, if it is > > officially released it needs to go through the signing process. So all > > projects that do a public release from eclipse need to sign their stuff. > > This is incorrect. See comment #1. The EDP says nothing about signing. This is > only a release train requirement. that is what I was thinking...we were talking about that with glyn on the virgo project and it came up as one of the marks against contributing to the release train (In reply to comment #18) > This is incorrect. See comment #1. The EDP says nothing about signing. This is > only a release train requirement. Correct. Signing is a simultaneous release requirement. I'm not sure if it's explicit anywhere (at least I can't find any explicit statement other than those that apply directly to the Eclipse project), but I believe that it's generally accepted/expected that any signed bundles distributed from eclipse.org need to be signed with the eclipse.org certificate. Is this controversial? (In reply to comment #20) > (In reply to comment #18) > > This is incorrect. See comment #1. The EDP says nothing about signing. This is > > only a release train requirement. > > Correct. Signing is a simultaneous release requirement. > > I'm not sure if it's explicit anywhere (at least I can't find any explicit > statement other than those that apply directly to the Eclipse project), but I > believe that it's generally accepted/expected that any signed bundles > distributed from eclipse.org need to be signed with the eclipse.org > certificate. > > Is this controversial? operative word being 'bundles' there that is my understanding have accepted that as the case thus far Ultimately I would like to see that eased up so that they just needed to be signed by a key that is trusted by the EFPK Sorry, the way I understood signing or doing a "release" from eclipse.org was that had to be signed. If it doesn't have to be signed, then I'm not sure why the release train has the requirement for things to be signed? Anyways, I'll be quiet now and go back to vacation. We discussed this on the Architecture Council today: http://wiki.eclipse.org/Architecture_Council/Meetings/September_15_2011 Obviously there is value in having some confirmation that downloaded bits are authentic, and the current scheme of signing at Eclipse.org only makes things very easy to understand (if signed by Eclipse it's built at Eclipse). More distributed trust will be interesting, but it's not easy to get right. If any committer can sign an official Eclipse.org download, then who validates that committer's build environment? Things get more complex very quickly. Mary Ruddy (who's subject expert as lead of the Higgins project) suggested the Eclipse Foundation to officially own and develop a policy for delegation of signing rights. On a more practical base, it may make sense to allow having I-builds signed by any committer but require releases to be signed by Eclipse.org and built at Eclipse.org -- looks like the extra simplicity validating authenticity is worth living with the "bottleneck problem" of signing around a release. Comments welcome. I should also mention that the primary discussion to have before we go into the technics of signing is, how trustworthy we do want the Eclipse.org downloads to be. Is there value for the Eclipse Foundation or its members in having Eclipse.org downloads validated / guaranteed to be authentic, and to what level do we need to ensure that authenticity? In the end, product providers are responsible for their products, Eclipse just provides frameworks and tools. (In reply to comment #23) <stuff deleted> > On a more practical base, it may make sense to allow having I-builds signed by > any committer but require releases to be signed by Eclipse.org and built at > Eclipse.org Signing at Eclipse.org is already the way things are done of course...but for ECF (and other projects I would guess)...requiring releases to be *built* at Eclipse.org would be unacceptable. As a project team, we've gone to a great deal of trouble to a) marshal resources for a project/community build server; b) setup/configure/run/secure that server so that it meets our project and our consumers' ongoing needs. If the arch council or foundation now tells us that we couldn't use that server for release builds (and thereby forcing us to use build.eclipse.org...or whatever), it would effectively make us redo that work...and tantamount to forcing the project out of the simultaneous release and/or the foundation (since I expect any such requirement/mandate to be totally unresourced). >-- looks like the extra simplicity validating authenticity is worth > living with the "bottleneck problem" of signing around a release. As a project lead, I don't really agree with this...and I suspect others (perhaps not even privy/interested in arch council or this bug) won't/don't as well. If this is a council-level conclusion...why would they come to this conclusion? > the current scheme of signing at Eclipse.org only makes things
> very easy to understand (if signed by Eclipse it's built at Eclipse).
Just a nit: signed by Eclipse means it was signed at Eclipse, not necessarily built here. One could build on a virus-infested machine, upload to Eclipse and sign.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie. I still believe this is a much better approach but we dropped from the release train ages ago so I'll just close this. |