Bugzilla – Full Text Bug Listing
| Summary: | Practice of Exporting all packages | ||
|---|---|---|---|
| Product: | Community | Reporter: | David Carver <d_a_carver> |
| Component: | Architecture Council | Assignee: | eclipse.org-architecture-council |
| Status: | RESOLVED INVALID | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | caniszczyk, david_williams, digulla, jeffmcaffer, john.arthorne, mober.at+eclipse, pwebster |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Linux | ||
| Whiteboard: | stalebug | ||
|
Description
David Carver
I can write a really long diatribe on this topic, but all packages should be exported coming imho. The simple answer is that it helps with API evolution. If people can't see anything, they won't have feedback about it. FYI, You can run Equinox in "strict mode" which effectively makes anything marked x-internal as a hidden package. http://aniszczyk.org/2008/03/06/osgi-equinox-and-strict-mode/ Good luck getting anything to run in this pure mode :) However, this shouldn't prevent us from discussing the topic on the next architecture council meeting. This should be a fun one. (In reply to comment #1) > I can write a really long diatribe on this topic, but all packages should be > exported coming imho. The simple answer is that it helps with API evolution. If > people can't see anything, they won't have feedback about it. > > FYI, You can run Equinox in "strict mode" which effectively makes anything > marked x-internal as a hidden package. > > http://aniszczyk.org/2008/03/06/osgi-equinox-and-strict-mode/ > > Good luck getting anything to run in this pure mode :) > > However, this shouldn't prevent us from discussing the topic on the next > architecture council meeting. > > This should be a fun one. The counter argument being that projects should be consumers of their own API, so that you truely get a community wide API. If you need it, more than likely Adopters may need it as well. (In reply to comment #2) > The counter argument being that projects should be consumers of their own API, > so that you truely get a community wide API. If you need it, more than likely > Adopters may need it as well. Not true imho. It's impossible to get API right the first time and in Eclipse, once things are API you're stuck. You need to iterate over API to get it right. The lower you go down the stack for a platform technology, the more critical API is. (In reply to comment #3) > (In reply to comment #2) > > The counter argument being that projects should be consumers of their own API, > > so that you truely get a community wide API. If you need it, more than likely > > Adopters may need it as well. > > Not true imho. > > It's impossible to get API right the first time and in Eclipse, once things are > API you're stuck. > > You need to iterate over API to get it right. > > The lower you go down the stack for a platform technology, the more critical > API is. I think we are in agreement there as well. How strict does the x-friends go. Can you limit it to only friends seeing the internals, and nobody else being able to access them, even though they are visible? The problem I guess I'm trying to help reduce is end users getting plugins that don't work across eclipse because an adopter used an API that was internal. Managing who can and can not see them is one of the practices that I think we need to develop. Also, the more you consume your own API or write code from an API consumers perspective, the more useful the API you eventually create. You are eating your own dog food so to speak. I'm looking for more of a Guideline for a best practice, not a hard set rule. I think the current approach is to open and just shifts the burden of responsibility, plus I don't think in many projects (WTP in particular) it encourages committers to create API, since everything is exposed. For another project policy, see http://wiki.eclipse.org/WTP_Policy_on_Package_Visibility (There's no hard and fast rule ... just acknowledges its sometimes appropriate to not make everything visible, and that projects can decide for themselves). I've been through this discussion at least three or four times now. Each time quite a number of people in the community were, lets say, "outraged", at the idea of not having access to the internals. The reasoning was along the following lines 1) it allows and encourages people who want to experiment and push the envelope. This furthers the development and discovery of API. 2) many people just need to get their job done and simply cannot wait for API to harden 3) producers seldom know all the usecases for their API and need time and help to make it right. As a result of these community requirements we put in place the x-internal and x-friends mechanisms as well as compiler support for discouraged access and OSGi resolver support for strict mode resolution. With this support in place there is literally no basis on which a project or consumer can claim that a) they were not able to define their API clear enough or b) they didn't know that what they were using was not API. One additional concern that I have with not exporting is that now you actually elevate the internal stuff to be somewhat closer to API. Since there is something "worse" than internal (i.e, not exported stuff), this exported but internal stuff must be somehow more stable/interesting/useful. When in fact that is not necessarily the statement. Personally the rule/guideline should be simple and clear otherwise the producers and consumers alike get confused. There are a great many consumers out there with 10 years of expecting access to everything. Please tread lightly here. (In reply to comment #6) > As a result of these community requirements we put in place the x-internal and > x-friends mechanisms as well as compiler support for discouraged access and > OSGi resolver support for strict mode resolution. With this support in place > there is literally no basis on which a project or consumer can claim that a) > they were not able to define their API clear enough or b) they didn't know that > what they were using was not API. > One additional concern that I have with not exporting is that now you actually > elevate the internal stuff to be somewhat closer to API. Since there is > something "worse" than internal (i.e, not exported stuff), this exported but > internal stuff must be somehow more stable/interesting/useful. When in fact > that is not necessarily the statement. My take on this is that if you are using it yourself to create the plugin, then it probably needs to be API. If you need it or one of your plugins needs it, then more than likely a consumer of your project is going to need it as well. And personally, if you are using internals in your plugins and having to use x-friends, then you potentially have something that really should be API. > Personally the rule/guideline should be simple and clear otherwise the > producers and consumers alike get confused. There are a great many consumers > out there with 10 years of expecting access to everything. Please tread > lightly here. I agree that we have historical reasons why things were done and that whatever guidelines are established from the Architecture Council should be clear and concise. But I also think how and when something should be considered Stable API is going to have to be a project level determination. Artificial constructs like internal package, x-friends, x-internal, are all just work arounds. Saying something is internal doesn't address the fact that Adopters will use it, and that compatibility from release to release on plugins built on top of the frameworks will vary greatly. The vary ability is not a good thing from an end users point of view. Who is to blame, committers will say adopters, and adopters may say committers because they didn't provide public API that did what they wanted. Adopters can and will access stuff we label as internal. If they have access to it, they are less likely to request it be API. I've seen the case with many adopters where they happily use internals, but never ask for anything to be made API. I would assume that this is more common than the alternate where API is requested. Regardless of how we want to label something, once somebody is using something it becomes very difficult to change it with out breaking somebody. If it is really ment to be internal, my preference would be not to even expose it for consumption. Otherwise, projects ideally should consume their own dog food. I.e. use their own API. (In reply to comment #7) > My take on this is that if you are using it yourself to create the plugin, then > it probably needs to be API. API == Application Programmer Interface. You are a plugin programmer. Your needs may coincide and it is great when it does but there is all sorts of code in bundles that operates on internals and does not call the bundle API. > If you need it or one of your plugins needs it, > then more than likely a consumer of your project is going to need it as well. > And personally, if you are using internals in your plugins and having to use > x-friends, then you potentially have something that really should be API. I'm all for eating your own dog food. Makes good sense. However, simply because you have a deployment requirement to split some function into two bundles, for example, does not mean that you should need/have/want to create *A*PI between to facilitate that. API is, by definition, application programmer facing, durable, robust and lots of other good stuff. Two collaborating friend bundles are none of these (at least not necessarily). For example, API should not be influenced or driven by packaging decisions or requirements. > I agree that we have historical reasons why things were done and that whatever The reasons are not "historical" so much as the reasons have been around a long time and continue to be present. They are no less valid or relevant now than they were when the approach was put in place. > I also think how and when something should be considered Stable > API is going to have to be a project level determination. Absolutely. Nothing about x-internal and x-friend counters that. In fact, they empower teams to better describe their API and non-API while still empowering the community to make informed decisions. > Artificial constructs like internal package, x-friends, x-internal, are all just work > arounds. Not sure what is "artificial" about these. Several modularity systems have friend-like constructs. Even Java has package-private, something between private and public that recognizes the need for close cooperation between distinct entities. So these are workarounds in as much as the binary export or not approach is insufficient for the requirements at hand. Do you have an alternative approach for addressing the requirements? > Saying something is internal doesn't address the fact that Adopters > will use it, and that compatibility from release to release on plugins built on > top of the frameworks will vary greatly. The vary ability is not a good thing > from an end users point of view. Who is to blame, committers will say > adopters, and adopters may say committers because they didn't provide public > API that did what they wanted. The rule is very simple: if it is x-internal or x-friend it is NOT API. No ambiguity. To imply that something that is exported with these annotations is "sort of API" or that "it will change less" weakens the power of our API definitions. If you use internals, chances are that you will be broken from release to release so you can make no assumptions. PDE and JDT warn you in several ways. Its not about blame, its about informed consent. People who want to stay in-bounds should set the discouraged access compiler preference to Error and deal with their red Xs. They can live in blissful ignorance of x-internal and x-friend packages. > Adopters can and will access stuff we label as internal. Not if they care to set the appropriate compiler preferences. > If they have access > to it, they are less likely to request it be API. My experience is the opposite. > I've seen the case with > many adopters where they happily use internals, but never ask for anything to > be made API. I would assume that this is more common than the alternate where > API is requested. Then they are happily broken from release to release. What is the scenario for them if the packages were simply not exported? They would not use the project code because the function they need was simply not available. They could request it and wait for the next release, hack the manifest, make a fragment, ... These are all big inhibitors to adoption. So they just go away and you still don't get the feedback. I guess it does solve the problem of adopters using non-API though :-) > Regardless of how we want to label something, once somebody is using something > it becomes very difficult to change it with out breaking somebody. If you tell someone that they will be broken (by marking a package x-internal for example) then the contract is very clear and the consumer was fully empowered to use or not use the function. > If it is > really ment to be internal, my preference would be not to even expose it for > consumption. All I can say is that producers often have no idea that their consumers will want to do something. This has frankly been one of the greatest joys of my experience with Eclipse and has fostered evolutions such as RCP! Further, producers seldom get it (anything) right the first time. Without access to the internals you both stifle innovation and hamper consumers who are just trying to get their job done. Perhaps by my own hand this discussion has morphed into something akin to "ban x-internal/x-friend". My apologies if this has been unwarranted. I'll be the first to say that x-internal as a boolean has problems and there is a bug report around somewhere that talks about replacing it with x-api (or some such) that would more completely describe the status of the exported package (internal, provisional, whatever). Absolutely. However even with these improvements I would still argue that everything should be exported in support of the usecases you DON'T know about and encourage the brave to experiment and innovate. Our tooling fully facilitates those who want/need to be less adventurous. Full disclosure. Informed consent. (In reply to comment #8) > (In reply to comment #7) > > My take on this is that if you are using it yourself to create the plugin, then > > it probably needs to be API. > > API == Application Programmer Interface. You are a plugin programmer. Your > needs may coincide and it is great when it does but there is all sorts of code > in bundles that operates on internals and does not call the bundle API. I think this summarizes our fundamental differences. I don't make a distinction between a plugin/bundle/framework/application. To me it all is API because you have to use methods and classes to get any of them to work. In fact the general term "application program" can mean a wide variety of things depending on the view point taken. The Wikipedia definition for API is pretty general: http://en.wikipedia.org/wiki/Application_programming_interface The other issue on x-friends is that I can fundamentally override all those warning messages with my projects bundle settings. Yes it's my choice, but then again it does not encourage users to request API in my experience. If you start out with the intent to create API, and your own bundles only access exported classes (i.e. no internal code), then you are working from the same contract that you expect your users to work from as well. There will always be internal code, how and if you should expose that code is the difference we have. If you don't expose the api, an adopter can always get the source code and see what it is doing and create something themselves, or ask for something to be made API. Yup, we have a difference of opinion on this. WRT API, the Wikipedia entry says "API) is an interface that software programs implement in order to allow other software to interact with it" For me this hinges on "other software". If my component happens to need to be shipped as two or more bundles, are those bundles "other" within the group? Are the interaction points needed between the members of the group really things that general application programmers are expected (should be allowed) to call? Maybe. Maybe not. Similarly, API typically has to have all manner of checks and synchronization and ... as the guy writing a bundle (or set of bundles) you know what is going on and are in control of the contracts. Within one bundle it is quite frequent that the bundle's code not use its own API so as to be more efficient, not cause deadlock, ... The same may well be true between closely collaborating bundles (e.g., the group from above). That is really the friends topic. On the x-internal topic, seems we also have to agree to disagree. I'm a power developer. I routinely copy large swats of Eclipse code into my projects since the API sti*** ... doesn't cut it for me. Too much private, too many final, too much static. I've stopped to file bug reports because they rarely get the attention I need and I can solve my problems more easily without trying to persuade some id**** ... someone of my use case and why it's valid. Requesting API is just too much effort; mostly because the Eclipse guys are already at their limit and can't be bothered with more work. I see two sides here. As a power developer, I have to make my software work and to hell with the Eclipse rules. A lot of code in Eclipse is unnecessarily coupled and I have to break these dependencies somehow to reuse the code. One major pain for me is OSGi. My apps must start fast (< 5s) and in this scenario, OSGi doesn't work today. The Eclipse developers, OTOH, have to care about their time. If they jumped at my every whim, they wouldn't get any other work done. In the end, I think we need a better way to migrate API. If you change something internally, it should be 100% clear where that new code went. Use @deprecated, for example, plus a comment where the new stuff is and how to use it. This is vital information for outsiders and insiders, who revisit the code after some time. I know @deprecated doesn't take an argument but maybe you should enforce one. I'm not a huge fan of keeping things around. Dead is dead and should be buried somewhere (i.e. code should be removed). But it's a fact that today, it's too hard to follow internal changes from the outside. On top of that, we have the usual laziness of plugin developers to update their code for new Eclipse releases. I don't think that there is a solution. Either someone maintains his work or they don't. Not much Eclipse could do here except maybe a magic bytecode rewriter that can replace obsolete APIs in existing code ... Conclusion: If you dare to make stuff more private, that will only make me write a script that pulls Eclipse from the VCS, remove all those private statements and build something that's useful for me. If that means to fork OSGi and to hack bytecode, then that's what's going to happen. As a (re-)user of Eclipse code, "private" just means "Eclipse developers are free to change this in the future". It means more maintenance effort on my side which is OK. I get Eclipse for free, so I pay the price somewhere else. It never means "touch it and you'll die". Since Eclipse is written in Java, we all know that designing a good API takes a lot of time and effort, so when push comes to pull, that doesn't happen. I prefer a new version of Eclipse on time that I can hack with a chainsaw than a great API but half the amount of features. Interesting point of view. Aaron, where can we download your hacked code to see what kind of changes you have made? (In reply to comment #12) > Interesting point of view. Aaron, where can we download your hacked code to > see what kind of changes you have made? That's for internal projects but I can explain what I did and maybe send diffs later. Here are some examples: I wanted to give the user a list to select from. ElementListSelectionDialog seemed like a perfect solution. But I can't use it without a Workbench. After some digging through the code, I found that FilteredList uses an internal private class TableUpdateJob which extends WorkbenchJob. Since there is no way to intercept the creation of this class, the whole dialog is useless. The fix was to simply remove "extends WorkbenchJob" and add three methods to satisfy the API. End result: To reuse the dialog, I had to copy six files and add ten lines of code. Another example: The standard error dialog doesn't show the stack trace. Since the methods which build the message are private, I can't override this. Next example: The cursor handling in the StyledText class is deeply ingrained in the code. This didn't allow me to add a listener for cursor movements. After a lengthy discussion with the author, I refactored the code to solve my problem and the author added an API to listen for cursor events. My solution was to move all cursor movement into a distinct class which was reusable, he didn't like it because of backwards compatibility and performance reasons. After I had wasted so much time, I didn't have a second look at the API so I have no idea whether I could use it or not. Last example: The numbering of lists in StyledText is broken (it starts with 0 instead of 1). To fix this, I had to copy over 50 files into my project. The fix was to add "+1" in a single line of code. After that, we had a discussion over several days whether this fix was "too dangerous" for 3.5.1 until I got so fucking mad that I pulled out of the bug. I don't know whether it has been fixed nor do I care. I understand that you guys are protective of your API and your precious resources and that there are reasons why things happened the way they did but the net result is that it has become more cheap (in terms of time and sanity) for me to just copy the code and make the fix for myself. I do still file the odd bug report but my resolve to follow them through has become unsteady. Conclusion: There are idiots out there and power users. Both behave the same way (at least through the lens of an email) but one group knows what they're doing. If you make your code un-reusable (by making everything package private or really private, use of the broken Java singleton pattern, abuse of the factory pattern), you annoy both. While it won't stop with the idiots pulling strings they should leave alone, it alienates the power users who see that there is a simple solution (change a few lines of code) to solve their issues. Then, they weight that against the effort to make you understand they're not idiots and change your ways. I think everyone here can do the math. This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie. Nothing happening here. |