Bugzilla – Full Text Bug Listing
| Summary: | [Intro] o.e.pde can not be jarred | ||
|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Mazen Faraj <mfaraj> |
| Component: | User Assistance | Assignee: | Mazen Faraj <mfaraj> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | dj.houghton, jeffmcaffer, wassim.melhem |
| Version: | 3.1 | ||
| Target Milestone: | 3.1 M7 | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Whiteboard: | |||
| Bug Depends on: | |||
| Bug Blocks: | 90086 | ||
|
Description
Mazen Faraj
what is the technical limitation that prevents the jarring? Intro gathers content from various plugins and creates an HTML file in memory that is rendered in the SWT Browser. Relative resource that are used by CSS files can not be resolved in the case when these content plugins are jarred. Hence the requirement. There is one thing that I want to convince myself with first (Im investigating), but for now, please unjar pde to prevent Intro from being broken. Konrad, please reverse the jarring process for now. Jeff, how does this limitation fit into the "jars are the path to Nirvana" theory? The following branding plug-ins: org.eclipse.platform org.eclipse.jdt org.eclipse.pde cannot be JARd because they contain web resources (css files, images etc.) that the generated welcome HTML pages reference. Embedded browsers we use do not understand jar protocol - we must have these resources on the file system. Note that we have the same problem with the About informatoin. The approach
there was to simply extract the files before launching the browser. For
example,
URL url = Platform.asLocalURL(bundle.getEntry("welcome_files/"));
browser.open(url);
Extracts (if needed) all the files in welcome_files and then launches the
browser on them.
Presumeably this is only needed if someone actually visits the welcome pages...
True but welcome pages are loaded on first Eclipse startup (always), so unzipping the files adds to the overall startup time. The welcome support etc should use the coding pattern outlined to ensure that plugins contriubting intro/welcome content CAN be jard. There is a separate question as to whether or not a particular plugin should be JARd. For the Platform perhaps it should not. Perhaps JDT etc can be jar'd. Since these kinds of scenarios are not unique (i.e. inability to use resources in a JAR), will platform provide some helper methods/classes to unzip zip entries before use? Sorry, I can see that you already answered this in the comment #5. (In reply to comment #5) Jeff, Intro already loads each and every resource using the above trick. Because PDE does not behave like real runtime, there is a difference in behavior when testing. Let me explain: Each intro resource is loaded using Platform.find() + Platform.asLocalURL() + toExternalString(). When a plugin is not jarred, this returns the true location of the resource in the plugin. This means that relative resource are properly resolved. When the plugin is jarred, asLocalURL copies the files into the config area, but relative resource are not copied. This is what breaks Intro. Content can be read, and is read properly when resources are jarred. BUT relative resources are not properly resolved. Now, are you saying that if I call asLocalURL() on a folder path, the logic is to extract all resources under that folder? If yes, then this may be a trick that I can do, BUT this is bad for performance. Not only are we adding logic to unjar the resources, which gives overhead, we endded up with the plugin unzipped anyway, so why jar in the first place Mazen, we should be able to support this scenario and then provide numbers to demonstrate that the extra step is not worth paying i.e. we need to argue that JARing plug-ins with intro content is a performance hit with real numbers. Dejan, It is taken for granted that it is a performnace hit. Think about it, each introContent.xml and each extension that we read it extracted from the jar. So for sure there will be a hit. Whether that hit is big or not is solely determined by the size of the file. I know. So can be back it by numbers? We will leave org.eclipse.pde jarred, until Intro finishes the investigation and provides tangible data that branding plugins must not be jarred. wassim, please do not jar o.e.pde. I will relay on Jeff's help/pointers to make Intro work with jarred content plugins. And I will add performance numbers in this defect. But, for now, Intro is broken with jarred content plugins, so please unjar. I will ping you when this issue is resolved and ask you to please jar pde, if this is the route we end up taking. re comment 10. why would the whole plugin be extracted? Only the resources for the intro content need to be extracted. If they are all together in one directory, just extract that dir? We cannot not jar o.e.pde. I has already been jarred over a week ago. I unjarred it not especially for Mazen, but I do not buy the performance argument yet. A plugin may contain many files and only small portion of them being intro. It should be per plug-in decision whether it should be jarred or not. For plugin like o.e.pde probably not many users are actually interested in its intro content, while all of them will have to go through installation and startup. These will need to be considered, not just the time to extract intro files alone. Now that the o.e.pde is unjarred, I hope to hear back and "for now" does not mean forever. Konrad, thanks for unjarring. Hopefully this is not "forever" (Im sure you meant it in a nice way! :-) ) You are correct in saying the o.e.pde (or any plugin for that matter) should not be affected by Intro content. This is what I am trying to resolve with Jeff. Note that o.e.pde is a special case - for historic reasons we needed to keep the builders and natures in it (because nature and builder ids are APIs and we could not move them into o.e.pde.core). Ideally, this plug-ins would only contain PDE branding info (like o.e.platform or o.e.jdt). (In reply to comment #16) Jeff, Javadocs for asLocalURL() say : "If the specified is a plug-in-relative URL of a directory,an exception is thrown". Is this still a true statement? Can I simply call asLocalURL() on a directory and you guys extract and copy all content to config area? I can put the restriction to make all Intro content under one directory and the first intro call to any resource under that directrory will extract all content to the config area (I can force a call to the parent folder of an Intro resource) but I need a blessing from you that it is ok to use the APIs like this. As Jeff mentions in comment #5, this is what we do for the about files. Sometimes they contain relative links, files, etc so we now have the main about.html in the plug-in root and then put all the relative files in a directory called about_files. Check out AboutPluginsDialog#getMoreInfoURL. Jeff: I believe that the Platform#asLocalURL spec for dirs is incorrect. Comments? thanks DJ, I will try that. Maybe someone can update the follwowing page: http://dev.eclipse.org/viewcvs/index.cgi/~checkout~/platform-core- home/documents/3.1/run_from_jars.html It says that Intro content plugins can not be jarred. The doc is incorrect. See bug 88267. added logic to intro model loading that extracts the parent folder of any
content (ie: XML DOM) that intro loads. This means that whenever we load a
content file, or config extension, or XHTML content file, we extract the
parent folder assuming that allo resources are relative to the content file.
works like a charm. I will do some performance impact testing to see how much
overhead this means.
Jeff,
I need your oath of faith on the following:
1) calling asLocalURL("folderPath") on a folder that has already been
extracted by the api has no/very very minimal perfromance impact.
2) what happens when you call asLocalURL("folderPath") on a folder that is not
in a jar? The javadocs where wrong, and so I want to be sure. Reason I ask, I
now always do a call to asLocalURL("folderPath") on the parent folder of the
resource Im loading. If the resource is not in a jar, this is a redundany and
overhead call. Come to think about it, does a bundle know/care if it is
jarred? can we query the platform if a bundle is jarred or not?
1) yes 2) nothing. it just works. No, you cannot tell directly that a plugin is jar'd. You should not care. Note that people should be encouraged to NOT put their intro content at the root of the plugin. This new policy will then cause the entire plugin to be extracted. (In reply to comment #25) > Note that people should be encouraged to NOT put their intro content at the > root of the plugin. This new policy will then cause the entire plugin to be > extracted. that right, and yes we want to avoid that. It will need to be properly documented. Jeff/Wassim, I just want to point out something that Im not sure if its a bug or not. Try this: self host, and in your Configuration tab make sure you do not choose to "Clear config area...". Now check the config area for files that have been extracted by runtime (eg: swt.dll and Intro stuff). On each run, a new bundle number is generated and this means that you are not really "reusing" the config area. This seems to only happen for file extracted by asLocalURL(). Other config area files are not regenerated. Does this make sense? is this how you guys want it? Good observation. This is actually less than optimal as it will fill up the disk and takes more time. I suspect that this is actually an update issue. If they think that the plugin has changed then they uninstall and reinstall. That causes the bundle id (the long) to be changed and thus new dirs to be used. On the one hand this is wasteful. On the other it avoids some level of cache coherence problems. We could likely do better. code to handle jarring of intro plugins is now in Iv20050420. I updated the pde, jdt, and platform plugins to include intro content just in one directory called intro. Wassim/Konrad, thank you for waiting. You can now jar o.e.pde. Do you think that the directory should be named intro_files to be consistent with the about_files? DJ, Is the "about_files" name hardcoded? In the case of Intro, its not. I extract the parent folder that holds the introContent.xml, whatever that happens to be. So it can be anything. Yep, its hardcoded. when in Rome, do what the Romans do :-) use the same trick for the about.html file, though it might be overkill for this one file. Peformance numbers: ------------------- Time to open Intro view on a cold start: ~750 ms (see bug 90086 for details) Calling asLocalURL() on 23Kbytes (intro content folder in jdt: 18 files/3 folders) takes ~ 60ms Calling asLocalURL() on 14Kbytes (intro content folder in pde: 12 files/3folders) takes ~ 40ms Performace hit for using jarred SDK plugins: 13.3% increase in startup time. Jeff/Dejan: this performance hit is quite a bit. I'll take you input as to wether we jar pde/jdt or not. I havent even factored in platform because Jeff mentioned that jarring platform is debatable. also, there is a random 10ms time when calling asLocalURL() on a folder that has already been extracted. So we have to factor this in, intrapolated to the number of plugins that have intro content. Thank you, Mazen. I have jarred o.e.pde plug-in again. re comment 33. yes, we could be more relaxed on this. We still want a convention that we always use about_files. Intro content should be wherever it wants. It is a perforance thing to put it all togehter in one sub dir. re comment 34, that is 13% of Intro startup right, not total startup. The real number is about 100ms. What is the impact on second start (both warm and cold). First start cold is going to be relatively slow. Cold second start is pretty interesting. (In reply to comment #37) > re comment 34, that is 13% of Intro startup right, not total startup. The real > number is about 100ms. What is the impact on second start (both warm and > cold). First start cold is going to be relatively slow. Cold second start is > pretty interesting. thats correct, its about 13% of the Intro startup time (time between start on Intro plugin to right after UI id displayed). Cold start (ie: clean config area, never ran it before) 50 and 70 ms (with the just newly updated jdt and pde intro plugins). Cold second startup is actually a lot better. There are random/inconsistant 10ms being used by calls to asLocalURL on a folder that has already been extracted. Intro assumed that a call to asLocalURL is very close to a no-op when the folder is alredy extracted. This is the case, most of the times. ok. it would be interesting to know what the cost is if platform were jar'd. the platform plugin has considerably more intro content. Again this would be a factor only on the very first start. As it stands here, the performance is reasonable. The 10ms variation would be interesting to analyze/identify. wow. So your simple "it would be interesting" comment made me do a model design change! Here is the reason: jarring platform works, yet the full plugin was being extracted to the config area. What had happened is that Intro model used to _always_ resolve relative resource in the intro content file relative to the plugin install directory (root). To satisfy the SWT browser, we needed to always set the base element of the in memory html file. So we set it to the plugin install directory of the owner plugin. This meant that the full jarred plugin will always be extracted if it had intro content. This is far from ideal. Redesigned the model to introduce the concept of "base". Relative resources in an intro xml content file are now resolved relative to the location of the file (rather than plugin root). This applies to configExtensions as well. Meaning, configExensions loaded from another plugin are resolved relative to the loaction of the configExtension content file. Also, intro content files forked out using the "content" attribute are in turned resolved in the same manner. This change has the added advantage that existing Intro plugins that need to be jarred have far less work to do to enable jarring. If resources where still being resolved against the root of the plugin, moving intro content into a directory meant that each relative resource in all content files will have to be updated. Too tedious. With the new base model, all intro content is one relative block and it can be renamed, moved or whatever, and resources will be resolved. NOW... Jeff, to answer your question. When platform is jarred, here are the perf numbers: <b>Performance numbers for o.e.platform:</b> Calling asLocalURL() on 805Kbytes (intro content folder in platform: 97 files/14 folders) takes ~ 350ms. Performace hit for using jarred o.e.platform plugin: 46.7% increase in startup time. So, in summary of all this defect, jarring all three intro content plugins gives a steep performance hit, but its only on a first ever cold start. Please let me know if you want me to jar the platform plugin or not. I would recommend against jarring it, and against jarring any intro content plugin. o.e.pde & o.e.jdt are jarred. o.e.platform remains unjarred. Intro fully supports jarring now, so the client decides whether or not to jar intro content plugins. Jeff, Im assuming that you guys want platform unjarred, which is great with Intro. So Im closing this bug with the status quo. If otherwise, please reopen. |