Bug 37676

Comment 1 Konrad Kolosowski 2003-06-30 13:41:14 EDT

We investigated several alternatives for a solution.  Two main ones:

1.  Allow documentation plug-ins to register themselves as web applications.  
The help system would be responsible for starting the web applications on 
demand (as a first request for a document from a particular plug-in is 
received.  Documentation plug-ins could contribute both as static and dynamic, 
with documents as flat files, in doc.zip, or generated by servlets.  This 
solution has advantage of being able to design a plug-in as a standard j2ee 
application, AND we discovered following disadvantages:
a)Plug-ins could not contribute documents as JSPs, only Servlets were possible 
as there is no compiler able to compile JSPs available in Eclipse Platform.  
JSPs would need to be pre-compiled at build time.
b)Build of any documentation plug-ins taking advantage of dynamic documents 
would be complicated for servlets and JSPs as Eclipse does not provide J2EE 
component.  Help currently compiles servlets at build time with custom build 
script, since application plug-in is replaceable and cannot be dependent 
upon.  Pre-compiling JSPs involves custom script calling optional ant tasks 
that will require carefully crafted classpaths, that PDE does not generate.
c)There would be many web application running in Eclipse, each of them having 
its own web container and resources.  That would increase memory requirements.
d)For documentation webapps to serve requests, the requests had to be 
delivered by help to them.  That could be accomplished by help opening local 
HTTP connection from help application to document applications for every 
document.  That would impact performance significantly.  Another way was to 
forward request between applications, but that cannot be done for accessing 
documents where there is not request, for example during search indexing.
e)With doc plug-ins running its own web application there is a very high 
chance of documents not being found correctly or non standardized errors being 
displayed.  Help plug-ins currently searches doc.zip and file system for files 
in the correct locales.  If documentation plug-ins were to obtain dynamic 
documents themselves it is very difficult to ensure that they would not search 
static documents and not display any error pages, so that help could try to 
search documents themselves.
f)Allowing cross-webapp forwarding cannot be specified by a web application.  
It would need to be enabled in Tomcat plug-in for help webapp, making Tomcat 
plugin specifically designed for help.  There would likely be issues with 
cross application contexts when Tomcat was replaced by a different plug-in 
contributed as application server for the platform.
g)We are using embedded Tomcat, which does not provide good sets of API to 
contribute servlet mapping and filters from the code.  Help applies filters to 
document contents.  That filtering is done in help servlets.  It does not work 
if request are forwarded.
h)Dynamic documents generated using different technologies (other than J2EE) 
would still need to wrap their code to deal in J2EE web application 
environment.

Second solution, much more generic, is to allow documentation to contribute a 
DocumentProducer class that does not have to depend on J2EE classes, but can 
return Input stream for required documents.  This allows using simple java 
classes to generate HTML (or other formats e.g. GIF).  Other technologies that 
generate HTML can be contributed by plugging in some wrapper as 
DocumentProducer.  This solution, although simpler, does not eliminate 
possibility of documentation plug-ins using servlets or JSPs.  If there is 
such need, they can contribute a DocumentPrducer that starts web application 
for a plug-in when it is first called, opens local HTTP connection, and 
returns InputStream to help.  This ensures that web application and local TCP-
IP communication occurs only for these dynamic documentation plug-ins that 
actually use J2EE technologies for HTML generation.

After investigating possibilities, trying to prototype both, help team decided 
to go with the second one.

I have released the initial code into the repository.
The API consist of Interface:
	package org.eclipse.help;
	public interface IHelpContentProducer {
		public InputStream getInputStream(String pluginID, String 
href, Locale locale)
and an extension point:
	org.eclipse.help.contentProducer
The implementation in help has been released as well, and will be in this week 
integration build.  Basic documentation for APIs has been released.  Better 
documentation in on-line help is in work.

If you have any suggestions for improvements in any area, please comment.

Comment 2 David J. Orme 2003-06-30 16:44:55 EDT

*** Bug 9409 has been marked as a duplicate of this bug. ***

Comment 3 mary 2003-06-30 17:08:36 EDT

This looks pretty good, but I have a question. You call my producer with an 
href from my toc.xml file. I give you an open stream to that file, say an html 
file on the local disk. What about relative URLs within the html file, say to 
gif's or other documents? Will you call my producer with those? Or are they 
relative to the original file?

Comment 4 Konrad Kolosowski 2003-06-30 17:47:13 EDT

Mary,

The producer is called with href as from TOC, but parsed into plug-in and 
path.  For example if TOC contained href="plugin1/path1/document1.html" than 
producer is called with pluginID=="plugin1", and name=="path1".  You would 
typically care about name and ignore pluginID. 

Having pluginID is useful if you want to reuse the same producer by different 
plug-ins.  If plugin2 can use the same producer, you can contribute producer 
from plugin3 that is required by plugins 1 and 2.  The producer can then 
figure out which plug-in documents it is serving by looking at the pluginID 
parameter.

There is nothing special done for relative links in the content.  For example 
user browses to plugin1/path1/document1.html.  Your producer in plugin1 is 
called.  If the document generated contains link <img src="gif1.gif"/> it will 
be interpreted in the browser as relative, and it will create an absolute 
URL.  Help system will in turn end up trying to resolve 
plugin1/path1/gif1.gif, so it will call your producer, because plugin1 
contributed one.
If on another hand you have a link pointing to another plugin, for 
example: ../../plugin2/path2/document2.html, the producer in plug-in 1 will 
not be called.  If plugin2 contributes a producer (it can be the same class, 
as explained above), it will be called.

Comment 5 Konrad Kolosowski 2003-07-02 11:18:39 EDT

Available for 3.0 M2

Comment 6 Ed Burnette 2003-09-18 11:43:33 EDT

How does this tie in with searching - can you search a dynamic page?

Comment 7 Konrad Kolosowski 2003-09-18 12:02:44 EDT

Yes.  IHelpContentProducer is called during indexing as well, if the URL is 
included in one of the TOC.  Whatever content it returns is going to be 
indexed.
Obviously those topics that exist in the TOC should not have its visible 
content changed drastically between times when topic is viewed.  This would be 
confusing to the user regardless of search.

Comment 8 James Willans 2004-01-20 05:12:06 EST

I've tried this mechanism out and it works well - many thanks.  Am I right in 
thinking that the indexing of help pages only gets called once when a new help 
contribution is made?  If this is the case then I would like to see a button on 
the help window that forces the reindexing of pages.  This is motivated by the 
fact that dynamic content may evolve big time and the index would never be a 
true representation of the available content.

Another suggestion I have is that the interface defines a further two methods, 
these might be called "HelpInit()" and "HelpDispose()", which are called when 
the help browser starts and ends respectively.  The idea being that a dynamic 
content provider might need to do some preparation in order to provide the help 
in HelpInit()  (of course you could do this in the first calling 
of "getInputStream" but I think making this explicit is more elegant), but more 
importantly the provider may want to do some clearing up after the content 
provider has done its job using "HelpDispose()".  To give a more concrete 
example, imagine help content saved in a database, the "HelpInit()" method 
would make the connection to the database - the "HelpDispose()" would perform 
the important disconnection.

Comment 9 Konrad Kolosowski 2004-01-20 14:56:18 EST

Yes.  Indexing happens only once for each new plugin or its version.  I do not 
think button for the user to trigger reindexing is a good idea.
Changing content returned by dynamic document too much is going to confuse the 
user.  The main idea should stay the same, and that would require reindexing.
Even if document changed a lot, placing a button on the UI requires a user to 
be aware that documents have changed without them installing new plug-ins.  
The indexing functionality should be hidden as much as possible from the 
user.  It is a necessary step dictated by technology, not something user wants 
to do.  In addition, having part of documentation being dynamic should not 
caues all index to be rebuilt.  This may take a long time, that is painfull 
even if happening once.
If something has to be done to reflect a change in dynamic document behavior 
in the index, which I do not think is necessary, it would be better to provide 
some callback mechanism where documentation plug-in may indicate a need for 
reindexing.  Please open a separate feature request, if you feel strongly 
about reindexing.

Regarding HelpInit() - I do not think it is buying anything.  If you want to 
do some preparation you can do it in Plugin.startup().  I think it is more 
appropriate to do it lazily in IHelpContentProducer.getInputStream().  
Providing another method HelpInit() on the producer will require HelpInit() to 
be called just before getInputStream().  If HelpInit was not a method on 
IHelpContent producer, but more global, it would actually hurt to have it as 
it would initilize content producers that are not actually being used.

Regarding HelpDispose(), help does not usually know when the browser has been 
closed.  And in the event it was, openning help for a second time should be 
immidiate, so calling HelpDispose() should not happen.  If you need to release 
resources that can be recreated fast, do it before returning from 
getInputStream.  If it is something like connection to the database, you can 
release these in your Plugin.shutdown().
Another solution for you is to implement sort of a cache for expensive 
resources.  You can release your database connections when you feel (your need 
to use your judgement here) they are no longer needed.  Help does not know 
when it done with completely and will not be used anymore, so it cannot 
provide such infomration to your content producer.