Community
Participate
Working Groups
It would be nice if we could leverage WTPs XML editor, and enhance it or embed it with the existing XQDT DLTK xquery editor. This would give us all the content completion, validation, and other goodness from the existing XML editor, along with the goodness of the XQuery syntax. I'm going to copy Roy Ganor from the PDT project on this as this along the lines that they did with the WTP HTML editor and their DLTK based PHP editor. So he might be able to help out with how we should proceed.
Adding Roy for some input on how PDT integrated PHP and the existing WTP HTML editor with DLTK.
Here is a starting point from the PHP Editor. http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.pdt/plugins/org.eclipse.php.ui/src/org/eclipse/php/internal/ui/editor/PHPStructuredEditor.java?root=Tools_Project&view=markup It shows how to integrate some of DLTK and SSE. There is more that needs to be done, but this is the main class. You'll also want to look at there plugin.xml file for the extension points as well.
Created attachment 154357 [details] First Cut at XQuery SSE integration
I've attached a start of trying to get this to work. What I've seen of the analysis of what PDT has done is modify the JFLex 1.2.2 based XML grammar to enhance it with some PHP information. I've included the a copy in the org.eclipse.wst.xquery.core.sse plugin. Nitin I think we'll need your help in getting the appropriate XQuery related content regions setup. This implementation will need a special source parser that uses this new tokenizer to setup the regions. SSE uses the regions to help determine various items like syntax highlighting, and content assistance. Everything thing really hinges around the tokenizer and source parser.
Added Lionel to this discussion. Lionel can you please add the two plug-ins as an attachment to this bug? Some guidelines on requirements and set-up might be also useful. Thanks!
I should be able to attach them hopefully sometimes next week (legal stuff going on).
Created attachment 169987 [details] First shot at providing an XQuery Editor on top of SSE
To try out the XQDT-SSE plugins, just unpack the zip file into your workspace. They depend on the following plugins (other than the platform plugins): - org.eclipse.wst.xquery.sse.core;bundle-version="0.0.1", - org.eclipse.wst.sse.ui;bundle-version="1.1.102", - org.eclipse.wst.sse.core;bundle-version="1.1.402", - org.eclipse.dltk.debug;bundle-version="1.0.0" - org.eclipse.wst.validation;bundle-version="1.2.104" The XML SSE plugin should also be available, even though the code does not depend on it (plugin.xml should be cleaned up). These plugins supports the following functionalities: - Contextual Syntax highlighting for full XQuery 1.0 - Syntax highlighting preferences - Partial XQuery 1.0 syntax checking - Line breakpoint for DTLK There is also placeholders for source code formatting and content assist.
(In reply to comment #8) > To try out the XQDT-SSE plugins, just unpack the zip file into your workspace. > They depend on the following plugins (other than the platform plugins): > - org.eclipse.wst.xquery.sse.core;bundle-version="0.0.1", > - org.eclipse.wst.sse.ui;bundle-version="1.1.102", > - org.eclipse.wst.sse.core;bundle-version="1.1.402", > - org.eclipse.dltk.debug;bundle-version="1.0.0" > - org.eclipse.wst.validation;bundle-version="1.2.104" > > The XML SSE plugin should also be available, even though the code does not > depend on it (plugin.xml should be cleaned up). > > These plugins supports the following functionalities: > - Contextual Syntax highlighting for full XQuery 1.0 > - Syntax highlighting preferences > - Partial XQuery 1.0 syntax checking > - Line breakpoint for DTLK > > There is also placeholders for source code formatting and content assist. Thanks, Lionel. This looks like an execellent start. I haven't tried this yet, but how does it handle mixed syntaxes like XML and XQuery together. Or is this just XQuery syntax, and we need to add in the XML through the embedded class? sorta like how the CSS support is added to JSP's editors?
> but how does it handle mixed syntaxes like XML and XQuery together. Or is > this just XQuery syntax, and we need to add in the XML through the embedded > class? > The tokenizer handles the mix. I don't know how to use the embedded XML class (yet). There are several issues like the XML in XQuery is not full XML, and XQuery expressions can be nested into the XML. The embedded XML needs a way to call back the XQuery plugin when, for instance, it sees { } in an attribute value. Another issue, functional this time, is whether or not it makes sens to provide a DOM view of the XQuery XML fragments. I hope that we will be able to share code between the 2 projects, specially for quick assist and code formatting. And hopefully more than that. > sorta like how the CSS support is added to JSP's editors? I think embedding CSS is easier than embedding XML in XQuery, due to the nested nature of XQuery (XQuery -> XML -> XQuery -> ...).
(In reply to comment #10) > > but how does it handle mixed syntaxes like XML and XQuery together. Or is > > this just XQuery syntax, and we need to add in the XML through the embedded > > class? > > > > The tokenizer handles the mix. If the information is setting the appropriate Regions, and content types. Then enabling the appropriate content assist should be just a matter of making sure that it is available and called when the Content Assist classes are invoked from the Source Editor Configuration. This is what we currently do with the wst.xsl editor, we make sure that both the existing XML content assistance is invoked, and the XSL content assistance is invoked as well. wst.xsl added a content assist extension point where adopters could include their own content assistance into the editor with out having to create a custom editor. I'll take a look at the existing code when I get a chance and see what would be involved. We may also want to look at leveraging the Semantic Highlighting ability that was recently added to SSE to provide content specific syntax highlighting depending on the content type and region.
> If the information is setting the appropriate Regions, and content types. Then > enabling the appropriate content assist should be just a matter of making sure > that it is available and called when the Content Assist classes are invoked > from the Source Editor Configuration. > The region types in XQDT are the same as the region types in the XML plugin. That's a start. We need then to look at the content types. > We may also want to look at leveraging the Semantic Highlighting > ability that was recently added to SSE to provide content specific syntax > highlighting depending on the content type and region. Can you give me an example of semantic highlighting for XML?
(In reply to comment #12) > > If the information is setting the appropriate Regions, and content types. Then > > enabling the appropriate content assist should be just a matter of making sure > > that it is available and called when the Content Assist classes are invoked > > from the Source Editor Configuration. > > > > The region types in XQDT are the same as the region types in the XML plugin. > That's a start. We need then to look at the content types. > > > We may also want to look at leveraging the Semantic Highlighting > > ability that was recently added to SSE to provide content specific syntax > > highlighting depending on the content type and region. > > Can you give me an example of semantic highlighting for XML? For some examples of it being used: http://dev.eclipse.org/viewcvs/index.cgi/sourceediting/plugins/org.eclipse.wst.xsl.ui/?root=WebTools_Project The extension point configuration looks like: <extension point="org.eclipse.wst.sse.ui.semanticHighlighting"> <highlighting class="org.eclipse.wst.xsl.ui.internal.style.XSLTagNameSemanticHighlighting" target="org.eclipse.core.runtime.xml, org.eclipse.wst.xml.core.xslsource" styleStringKey="xsltagName"> </highlighting> </extension> <extension point="org.eclipse.wst.sse.ui.semanticHighlighting"> <highlighting class="org.eclipse.wst.xsl.ui.internal.style.XSLAttrNameSemanticHighlighting" target="org.eclipse.core.runtime.xml, org.eclipse.wst.xml.core.xslsource" styleStringKey="xsltagAttributeName"> </highlighting> </extension> <extension point="org.eclipse.wst.sse.ui.semanticHighlighting"> <highlighting class="org.eclipse.wst.xsl.ui.internal.style.XSLAttrValueSemanticHighlighting" target="org.eclipse.core.runtime.xml, org.eclipse.wst.xml.core.xslsource" styleStringKey="xsltagAttributeValue"> </highlighting> </extension> <extension point="org.eclipse.wst.sse.ui.semanticHighlighting"> <highlighting class="org.eclipse.wst.xsl.ui.internal.style.XSLTagDelimsSemanticHighlighting" target="org.eclipse.core.runtime.xml, org.eclipse.wst.xml.core.xslsource" styleStringKey="xsltagBorder"> </highlighting> </extension> This uses an existing preference string to retrieve the existing color code values to be used to highlight the code. The targets are the content types. If you have the Regions the same for XML, then you should be good on the content types. You can take a look at the class files above for how we implemented Semantic Highlighting for XSL using the extension points above.
(In reply to comment #12) > > If the information is setting the appropriate Regions, and content types. Then > > enabling the appropriate content assist should be just a matter of making sure > > that it is available and called when the Content Assist classes are invoked > > from the Source Editor Configuration. > > > > The region types in XQDT are the same as the region types in the XML plugin. > That's a start. We need then to look at the content types. > > > We may also want to look at leveraging the Semantic Highlighting > > ability that was recently added to SSE to provide content specific syntax > > highlighting depending on the content type and region. > > Can you give me an example of semantic highlighting for XML? I managed to play with this a little bit today. In the XQueryStructuredTextViewerConfiguration, you can leverage the existing XMLLineStyleProvider from wst.xml.ui. This will provide the syntax coloring that comes from the core XML editor from WTP. However when adding it the syntax coloring for the XQuery syntax gets disabled. But it does show at least that the correct partitions are being added and the correct regions. I think we can reuse or extended where needed much of the existing XML Editors core functionality. For XML Content Assistance, and other areas. I'll take a shot at trying to add Semantic Highlighting based on the work you provided. I believe Gabriel was going to submit a CQ for this after another patch submission. Gabriel what is the status of the CQ?
I should be able to submit another patch this coming Friday. I think it's fine to have XQDT handling the XML syntax coloring since it's only few lines of codes. The big win will be if we can integrate the XML content assist. But this is still a long shot, since the XML content assist requires DOM models. The XQDT SSE source parser currently does not build those models and letting the XML plugin doing the parsing won't work as it does not know when to call the XQuery parsing for the xquery fragments...
(In reply to comment #15) > I should be able to submit another patch this coming Friday. > > I think it's fine to have XQDT handling the XML syntax coloring since it's only > few lines of codes. > > The big win will be if we can integrate the XML content assist. But this is > still a long shot, since the XML content assist requires DOM models. The XQDT > SSE source parser currently does not build those models and letting the XML > plugin doing the parsing won't work as it does not know when to call the XQuery > parsing for the xquery fragments... Well, once we have your code going through IP review and into the repo, I can probably see about getting the later part with XML content assistance working. I think we can do it but as you said it'll be a bit of work. The reason I'd prefer to leverage the existing XML syntax highlighting is this way if people make changes in the XML Editor's preferences it automatically filters down to XQuery as well. This is the way it is done with XSL editor so that it inherits any general XML color changes, and then has it's own more limited color settings for XSL specific syntax. From what I'm seeing it won't be that difficult to leverage the existing color syntax highlighting from XML into the new editor's code.
Added a CQ for this: CQ 4274 https://dev.eclipse.org/ipzilla/show_bug.cgi?id=4274
Created attachment 172550 [details] XQDT SSE plugins XQDT editor based on SSE, ready for CQ. In addition to the previously described features, this contribution includes: - A lexer for XQuery Update 1.0 - A incremental parser for XQuery 1.0. Build XML plugin DOM models. - Better variable reference completion - Template-based completion (coming from the XQDT-DTLK editor) (still quite rudimentary) Gabriel, you can go ahead and submit it to CQ.
The Attached the latest patch from Lionel to the CQ Bug: https://dev.eclipse.org/ipzilla/show_bug.cgi?id=4274
I confirm that: 1. I wrote 100% of the content I'm contributing (contained in the attachment to the CQ 4274) 2. I has the rights to donate the content to Eclipse 3. I contributes the content under the EPL Lionel
I created a new branch called xquery-dev-sse. There you should find the 2 plugins containing the SSE-editor. I'm new to GIT so let me know if something went wrong.
(In reply to comment #21) > I created a new branch called xquery-dev-sse. There you should find the 2 > plugins containing the SSE-editor. > > I'm new to GIT so let me know if something went wrong. Actually, no need to create a new branch for it. Just add the two new plugins to the existing xquery-dev branch. We need to integrate those changes anyways.
(In reply to comment #22) > (In reply to comment #21) > > I created a new branch called xquery-dev-sse. There you should find the 2 > > plugins containing the SSE-editor. > > > > I'm new to GIT so let me know if something went wrong. > > Actually, no need to create a new branch for it. Just add the two new plugins > to the existing xquery-dev branch. We need to integrate those changes > anyways. I merged the xquery-dev-sse branch into xquery-dev. I was surprised to see that the merge actually happened directly in the bare repository, not locally. Maybe because this time I used eGIT and not the command line..
(In reply to comment #23) > (In reply to comment #22) > > (In reply to comment #21) > > > I created a new branch called xquery-dev-sse. There you should find the 2 > > > plugins containing the SSE-editor. > > > > > > I'm new to GIT so let me know if something went wrong. > > > > Actually, no need to create a new branch for it. Just add the two new plugins > > to the existing xquery-dev branch. We need to integrate those changes > > anyways. > > I merged the xquery-dev-sse branch into xquery-dev. > > I was surprised to see that the merge actually happened directly in the bare > repository, not locally. Maybe because this time I used eGIT and not the > command line.. Yeah, probably because you didn't have a version tracked locally before doing the merge. But could be a bug as well. I went ahead and added the necessary poms to those projects and added them to the master pom as well. So those plugins are building. I'll add them to the core feature so they appear in the update site. They are building with the other plugins in the Hudson build now. Eventually we are going to want to change the version number of these plugins to match the others but we'll hold off on that until there is a bit more functionality.
I just pushed some changes to add the Xquery SSE plugins so they come in with the other core editors when you install the main Xquery Feature from an update site. This change will appear in the next CI build.
Closing as the contribution is now in the Git repository. Will open smaller tasks on this topic..
Bulk close of old resolved bugs.
