Bugzilla – Full Text Bug Listing
| Summary: | Include the JST Server features in the Web Tools Platform (WTP) 3.2.x update site category | ||
|---|---|---|---|
| Product: | [WebTools] WTP Releng | Reporter: | Larry Isaacs <larryisaacs> |
| Component: | releng | Assignee: | David Williams <david_williams> |
| Status: | RESOLVED FIXED | QA Contact: | David Williams <david_williams> |
| Severity: | normal | ||
| Priority: | P2 | CC: | kaloyan, mauromol |
| Version: | 3.10 | Flags: | david_williams:
pmc_approved?
(david_williams) david_williams: pmc_approved? (raghunathan.srinivasan) david_williams: pmc_approved? (naci.dai) deboer: pmc_approved+ david_williams: pmc_approved? (neil.hauge) kaloyan: pmc_approved+ |
| Target Milestone: | 3.10.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | PMC_approved | ||
| Attachments: | |||
|
Description
Larry Isaacs
I'll help. This was an oversight. Created attachment 177182 [details]
Possible patch to add JST server features to WTP 3.2.x update site categories
It looks like these are the files that need modification.
In testing what gets installed with the JST server features added to the current WTP 3.2.x categories, there is still some descrepancy between what is contained in the WTP 3.2.x zip. It appears that also including the JST Web UI feature addresses most of what is missing. I can update the patch to add this feature if you like.
If matching the zip is desired, the "Eclipse Faceted Project Framework JDT Enablement" (org.eclipse.jst.common.fproj.enablement.jdt) feature is missing, though its plug-ins get installed via dependencies.
I have (I think) updated the category metadata for the 3.2.1 release. I did add the JST Web UI feature ... I'm surprised JEE stuff works at all without that ... magic of p2, I guess. I'd prefer to change only the 3.2.1 one, following the usual guideline of "change as little as possible" ... especially for an already released repo! The only reason we may do that at all is since it is a pretty bad bug. My first choice would be to fix in 322 release and leave it at that (There is, a work around, after all, though its not a very good one ... just to uncategorize and pick the features you want/need). But, I say all this since I'd like to hear your advice. What I've done so far is made a "test" repository at http://download.eclipse.org/webtools/downloads/drops/R3.2.1/R-3.2.1-20100730021206/testrepository/ But ... I tried it real quick and the server features didn't seem to show up, and when I tried to install "JST Web UI" it said I had some conflicting dependencies ... probably since I started off with the JEE IDE Package. But, if you want to try ant test one, and report back, that'd be great. Advice welcome. Thanks so much. Created attachment 177216 [details]
Additional patch to update committed versions.
Regrettably, in spite of trying to be careful, I still manage to miss the obvious. There is a typo in each of the category names for the JST server fetures. While I managed to update the "(?)" to be "(WTP)" and "(WTP SDK)", I failed to correct the "Test" in the "Web Tools Platform" part of the category name. I believe this patch corrects this for the wtp320category.xml and wtp321category.xml files. Also, the wtp322category.xml had a duplicate "org.eclipse.jst.server_adapters.sdk.feature" and was missing the "org.eclipse.jst.server_adapters.ext.feature". I guess it's hard to get these right by hand. The patch includes fixing this and bumping the version of "org.eclipse.jst.server_adapters.ext.feature" and "org.eclipse.jst.server_adapters.ext.sdk.feature" to 3.2.2 since I have already updated those features to that version for a Tomcat change in WTP 3.2.2. I'll be happy to test whenever a new repository is ready.
Forgot to answer the request for advice. Since I tend to be error prone, at least in situations where I'm unable to test my work, I tend to avoid giving advice. :) I agree there isn't much point to fixing 3.2.0. I would vote in favor of fixing 3.2.1 since it will be about a month before 3.2.2 is ready and it might avoid some questions on the WTP newgroup. Also, for reasons I don't understand, if you install the Helios Java EE package, and run Check for Updates, it says that no updates were found. I couldn't figure out how to update to WTP 3.2.1 except to use Install New Software. By fixing the 3.2.1 category, selecting that in order to update a Java EE package will do what is expected. Unfortunately, it's hard to know how many have already done that and don't realize their WTP is not fully updated to 3.2.1. > avoid some questions on the WTP newgroup. Also, for reasons I don't
> understand, if you install the Helios Java EE package, and run Check for
> Updates, it says that no updates were found. I couldn't figure out how to
> update to WTP 3.2.1 except to use Install New Software. By fixing the 3.2.1
> category, selecting that in order to update a Java EE package will do what is
> expected. Unfortunately, it's hard to know how many have already done that and
> don't realize their WTP is not fully updated to 3.2.1.
My own experience, instead, is the following. The Helios Update site does not contain WTP 3.2.1, so if you want to install WTP 3.2.1 on an Eclipse Helios installation without WTP at all, you have two options:
1) install WTP 3.2.0, then upgrade to 3.2.1: if I understood it well, because of this bug the user should better install WTP 3.2.0 from the Helios Update Site; but, since WTP 3.2.1 is not available there, he then has to check that the WTP Update Site is enabled in the Preferences; if it is enabled, checking for updates will reveal WTP 3.2.1; this worked for me (actually, this fixed all the problems I've had, as reported in the newsgroup)
2) install WTP 3.2.1 directly: since WTP 3.2.1 is not in the Helios Update Site, the user has to first add the WTP Update Site in his/her available update sites; however, because of this bug, I had problems to install in this way because some bundles were missing (JST Server Adapters, JST Web UI and Eclipse Faceted Project Framework JDT Enablement); so I tried to install the missing bundles from WTP 3.2.0 on the Helios Update Site, assuming those bundles were not updated; most of them did install, but I ended up with a mixed installation of WTP 3.2.0 and 3.2.1 and some features didn't work
HTH.
Thanks again Larry. On second though, I think if we fix either, we may have to fix 3.2.0 as well ... since that is the one that "matches" the EPP JEE package. (e.g. if someone wanted 'source' for that they had from epp package .... ) So, I (once again) blindly applied patches and generated new categories for test repositories. JST Server adatpers now shows up ... there is some "uncategorized" one ... not sure what that is from ... but, thought I'd just update status and let you test/look if you'd like. http://download.eclipse.org/webtools/downloads/drops/R3.2.0/R-3.2.0-20100615235519/testrepository http://download.eclipse.org/webtools/downloads/drops/R3.2.1/R-3.2.1-20100730021206/testrepository To test, I recommend disabling all other repos, have only these two enabled, and then "select all" repos when installing. That's how you'll get the "full effect" ... if we replaced fix the "main" (single) webtools/helios/repository. As for update vs. install ... yes, update only checks/sees the "root" feature that was installed ... and for EPP, that's the main "EPP JEE feature". So that won't be updated until SR1. HTH Created attachment 177240 [details]
Another additional patch to committed versions.
This patch should fix the "uncategorized" feature in the WTP 3.2.0 file. I found one other error in the WTP 3.2.2 file as well. Since the WTP 3.2.1 file seemed okay, I tested installing WTP 3.2.1 and WTP SDK 3.2.1. The result looked correct given that since the "Eclipse Faceted Project Framework JDT Enablement" isn't included in the category, it's source plug-ins won't be installed for the WTP SDK unless selected directly. I can try to patch that for the WTP 3.2.2 file, though I'm not sure how many tries it will take me to get it right.
Thanks for reminding me about update works with EPP. It seems I don't think about that often enough to remember.
Thanks once more. I too had discovered some errors/issues, but ... good news! ... I've discovered you don't need to specify version ... that if you use "0.0.0.qualifier" it gets the latest in the repo and fills it in. 1) I think I tried that before with just "0.0.0" as with features, and it didn't work. 2) I can't find that documented anywhere!? (and opened bug 323432 for that). So, I've regen'd the test repos, and even made a test composite to make it easier to test? Please give this repo a try, for various scenarios (e.g. starting with EPP, starting from custom built WTP, etc.) , and let me know if things work as you expect. (Wouldn't hurt to try installing the tests too ... I know we've paid less attention to those), http://download.eclipse.org/webtools/testrepository/helios/ If you two (Larry and Mauro) can confirm this works well ... and maybe a few of your newsgroup friends too? :) then I'll roll this out to the "normal" repo. Thanks much, Thanks, P.S. the plan is to eventually get ride of these many individual features in categories, and just have some commonly requested collections ... JavaScript, XML, (non Java) web development, and JEE development. [I'll be expecting advice there too Larry :) Created attachment 177351 [details]
Yet another patch to committed versions.
After encountering some issues, I found that there was some duplication in the current category files. This patch corrects that. It doesn't appear this would cause the following issues.
Using the "webtools/testrepository/helios/" repository, I'm not seeing the JST server and JST Web UI features listed for the "Web Tools Platform (WTP) 3.2.0" category. It also isn't showing the "Eclipse Faceted Project Framework JDT Enablement SDK" feature that was added in the last commit. The same is true for the "Web Tools Platform SDK (WTP SDK) 3.2.0" category.
The "Web Tools Platform (WTP) 3.2.1" category is showing the JST server and JST Web UI features, but not the "Eclipse Faceted Project Framework JDT Enablement SDK" feature. The same is true for the "Web Tools Platform SDK (WTP SDK) 3.2.1" category. This would seem to suggest that something may have gone wrong with the most recent "regen".
David, can you check to see if what is on the test server confirms these issues, or if is there simple way I can inspect those files investigate this. The repository may not be worth more testing until this is straightened out. Thanks.
Thanks Larry, I did figure out several things I did wrong, and have re-gen'd them ... but, they still don't look right (especially 3.2.0) so I need to go through it again (when I'm more awake :) ... just wanted to give a quick status in case you (or others) saw changes in testrepository. But ... as you said ... I don't think it's ready for testing yet. I'll take another look in about day. Thanks, (In reply to comment #11) > I don't think it's ready for testing yet. Well ... I think its ready after all ... Not sure if it took time to be "visible" from server side ... or if restarting my computer just really cleaned out my local p2 caches really well ... but, this morning, all features show up as I expected. So, please do take a look. Just to document it, one of the mistakes I made in deciding that leaving version as "0.0.0.qualifier" _appeared_ to work because I forget to "start fresh" and categories are added to repo in a "cumulative" way ... in my initial tests, I forgot to start afresh, (and appeared to work) but when I regen'd, I started fresh, and it didn't work. This may be complicated by some of the client side cache stuff ... not sure how to be positive its all be cleared. But, in the course of looking at all that, I looked at the code for 'CategoryPublisher' ... I was thinking of writing our own custom app to do it, so we wouldn't have to specify all the versions "by hand". As far as I can tell from the code, the way it works is that a p2 query is formed from the category.xml feature entry ... then that query is used against the artifact repo to find all that matches, and the results are put in the category entry for that feature. Once conclusion is that the URL doesn't seem to be used at all (could be, somehow ... but wasn't obvious). More significantly, an version such as "3.2.1.qualifier" forms a query to find all versions that match the range, effectively, as "3.2.1.*", so 0.0.0.qualifier would seldom match anything. BUT, what I learned from the code is that the absence of a version at all, it what causes a special query to be formed to "match anything". So, unbelievably, when a repo contains only one version of a feature, a simple entry such as the following suffices to find what's there. <feature id="org.eclipse.wst.xml_sdk.feature"> <category name="Web Tools Platform SDK (WTP SDK) 3.2.1"/> </feature> So, give it a sanity check. If it checks out, I suspect I should try again create things from afresh ... just to be sure ... but I think we are on to something. Sorry for the delay. I'm having to work the testing in amongst many distractions. I've completed testing the 3.2.0 categories, adding it to an EEP and plain Eclipse SDK. The Web Tools Platform (WTP) 3.2.0 and Web Tools Platform SDK (WTP SDK) 3.2.0 install the appropriate features and plug-ins to match what you would get from the zips. There is a discrepancy with the Web Tools Tests (WTP Tests) 3.2.0 category. It appears that because the JST Web Test feature doesn't include any dependent features like the JST Web UI and JST Web Plug-in Developer Resources features do, the WST Web Services Tests and the XSL Tests features don't get installed. I'm not sure if these missing test features is supposed to be handled within the JST Web Tests feature or the category file. This may not be worth fixing until WTP 3.2.2. I'll start testing the 3.2.1 categories next. For the 3.2.1 categories, I get the same results as the 3.2.0 categories except that the Web Tools Platform Tests (WTP Tests) 3.2.1 category gets stuck while installing to a fresh EPP, repeatedly "Fetching ..." a plug-in. It has been three different plug-ins from four different tries. I'm not sure what is going on since this category installs fine on a fresh Eclipse SDK and installed fine on the EPP and Eclipse SDK I had previously added the 3.2.0 categories. It would seem the problem isn't the repository. I'll see if can find the time to dig a little deeper into what the problem is. It could be some quirk local to my system. The problem I had adding the Web Tools Platform Tests (WTP Tests) 3.2.1 category to an EPP (with the 3.2.1 WTP and WTP SDK already added) appears to be caused by the fact that I forgot to disable the current WTP update site. If I disable that update site the 3.2.1 WTP Tests installed like the others, i.e. installs fine except the WST Web Services Tests and the XSL Tests features don't get installed. Other than this, the test repository looks good. Thanks. status: I removed the "testrepository" (I wasn't thinking ahead there ... poor choice of names to have long term) I created some new repositories, with the fixed/improved categories, called repositoryb (note the b). Plus, I "manually" fixed those in our current candidate release build, so this repo should be correct: http://download.eclipse.org/webtools/downloads/drops/R3.2.2/M-3.2.2-20100910064233/repository What I propose, is we make the "repositoryb" repo the "official" repositories (though I'll leave the low level ones in place, as is, since someone might use those for builds, or something, in theory) and rename it to "repository" once approved by PMC. Then, as we roll out WTP 3.2.2, it'll show three releases in our repository. The runtime and sdk categories have 21 features each. The test category will have 18 features. (In reply to comment #16) > > I created some new repositories, with the fixed/improved categories, called > repositoryb (note the b). > I meant to give the URL: http://download.eclipse.org/webtools/repositoryb/helios/ Once approved, I'll fix the "official" repository at http://download.eclipse.org/webtools/repository/helios/ to point to the fixed repositories (so, minimal change from "b" versions. For the record, our repositories are about 300M each, so having two of them on disk (and on mirrors) isn't great, but not obviously too much. If anyone is concerned about that, I could "write over" the existing repos at 'repository' and just have one .... but, I'd prefer not to write over what we released, unless urgent. Note to self: once I understand p2 better (and its APIs, Ant tasks, etc.), we could probably only duplicate the "content metadata" (for categories) and leave only one copy of artifacts repository. I have, btw, fixed our builds to automatically create the repo categories correctly on each build, so in future, won't have to do that manual step, at end (though there's still a couple of other manual steps). During our build, each major component of the build produces its own small repo, with its own categories, and at the end, mirror those to a final repository. FWIW, to get the _categories_ to mirror wasn't easy for me to figure out, but finally saw a hint in an example, and its very easy: you can mirror based on properties, not just IU IDs, so had to add one statement to the list of IUs mirrored: <iu query="property[@name='org.eclipse.equinox.p2.type.category']"/> PMC members, I'm targetting for 3.2.2, and asking for your review and approval, but there's no code changes, no new builds required. This is asking for your approval simply because I'm proposing "changing" the released repositories -- and that always takes PMC review and approval. It is a change in categories metadata only, does not (should not) effect code or artifacts in anyway. Also, would not effect any scripts which pull the features directly ... just adds more features to our simple categories. (In our "retention policy", http://wiki.eclipse.org/WTP/Retention_Policy we even say we might change categories, over time, though "fixes" wasn't what was in mind when that was written). The motivation for this fix is at 2 levels: First, to fix the current 3.2.2 categories, which is done, and which would not need PMC approval ... but, I've found some cases where things do not install (update) correctly/easily unless the preceding version is installed. Hence, I think best to make the categories of each release match each other and avoid one source of confusion. Created attachment 178783 [details]
small picture of the install dialog with all 21 features listed.
In case it helps, this is what the 'repository' will look like once we change from 'repositoryb'. Just in case it helps visualize what this is about.
I've fixed the two existing release repositories. And since we had to respin, I even confirmed the script to categorize during the build works as expected. |