Community
Participate
Working Groups
The test input: org.eclipse.cosmos.rm.validation.tests\src\test-resources\others\remote-document.xml is not up-to-date based on the latest SML 1.1 specification. The definition of the locator element used to remotely reference instance documents has changed. The definition is now: <locator> <documentURI/> ? xs:any <!—A reference to a definition document goes here> </locator> I've also noticed that the validator in Eclipse generates the following error when definition documents are included in an SML-IF document using locator in the instance section: "The markup in the document preceding the root element must be well-formed. remote-document.xml" **************************************************************************** For example, the following document passes: <?xml version="1.0" encoding="UTF-8"?> <model xmlns="http://www.w3.org/2008/03/sml-if"> <identity> <name>TestDocumentLocator</name> <version/> <displayName/> <description>Tests the correct implementation of the locator element</description> <baseURI> http://interop.serviceml.org </baseURI> </identity> <instances schemaComplete="true"> <document> <locator xlink:href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co" xmlns:xlink="http://www.w3.org/1999/xlink"/> </document> </instances> </model> **************************************************************************** But the following with definition files fails with the above noted error: <?xml version="1.0" encoding="UTF-8"?> <model xmlns="http://www.w3.org/2008/03/sml-if"> <identity> <name>TestDocumentLocator</name> <version/> <displayName/> <description>Tests the correct implementation of the locator element</description> <baseURI> http://interop.serviceml.org </baseURI> </identity> <definitions schemaComplete="true"> <document> <data> <xs:schema elementFormDefault="qualified" targetNamespace="urn:university" xmlns:sml="http://www.w3.org/2008/03/sml" xmlns:tns="urn:university" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:import namespace="http://www.w3.org/2008/03/sml" schemaLocation="http://www.w3.org/2008/03/sml-schema.xsd"/> <xs:element name="EnrolledCourse"/> <xs:element name="Student" type="tns:StudentType"/> <xs:element name="Course" type="tns:CourseType"/> <xs:element name="Courses"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" minOccurs="0" name="Course" type="tns:CourseType"/> </xs:sequence> </xs:complexType> </xs:element> <xs:complexType name="CourseType"> <xs:sequence> <xs:element name="Name" type="xs:string"/> </xs:sequence> </xs:complexType> <xs:complexType name="StudentType"> <xs:sequence> <xs:element name="ID" type="xs:string"/> <xs:element minOccurs="0" name="EnrolledCourses"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" ref="tns:EnrolledCourse"/> </xs:sequence> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:schema> </data> </document> </definitions> <instances schemaComplete="true"> <document> <locator xlink:href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co" xmlns:xlink="http://www.w3.org/1999/xlink"/> </document> </instances> </model> **************************************************************************** If multiple document instance files are included, then the following error is produced: "Cannot find the declaration of element 'document'." Test input: <?xml version="1.0" encoding="UTF-8"?> <model xmlns="http://www.w3.org/2008/03/sml-if"> <identity> <name>TestDocumentLocator</name> <version/> <displayName/> <description>Tests the correct implementation of the locator element</description> <baseURI> http://interop.serviceml.org </baseURI> </identity> <definitions schemaComplete="true"> <document> <data> <xs:schema elementFormDefault="qualified" targetNamespace="urn:university" xmlns:sml="http://www.w3.org/2008/03/sml" xmlns:tns="urn:university" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:import namespace="http://www.w3.org/2008/03/sml" schemaLocation="http://www.w3.org/2008/03/sml-schema.xsd"/> <xs:element name="EnrolledCourse"/> <xs:element name="Student" type="tns:StudentType"/> <xs:element name="Course" type="tns:CourseType"/> <xs:element name="Courses"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" minOccurs="0" name="Course" type="tns:CourseType"/> </xs:sequence> </xs:complexType> </xs:element> <xs:complexType name="CourseType"> <xs:sequence> <xs:element name="Name" type="xs:string"/> </xs:sequence> </xs:complexType> <xs:complexType name="StudentType"> <xs:sequence> <xs:element name="ID" type="xs:string"/> <xs:element minOccurs="0" name="EnrolledCourses"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" ref="tns:EnrolledCourse"/> </xs:sequence> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:schema> </data> </document> </definitions> <instances schemaComplete="true"> <document> <data> <Student xmlns="urn:university" xmlns:sml="http://www.w3.org/2008/03/sml"> <ID>1000</ID> <EnrolledCourses> <EnrolledCourse sml:ref="true"> <sml:uri> /Universities/MIT/PHY101.xml </sml:uri> </EnrolledCourse> </EnrolledCourses> </Student> </data> </document> <document> <locator xlink:href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co" xmlns:xlink="http://www.w3.org/1999/xlink"/> </document> <document> <docinfo> <aliases> <alias>http://interop.serviceml.org/Universities/MIT/PHY101.xml</alias> </aliases> </docinfo> <data> <Course xmlns="urn:university" xmlns:sml="http://www.w3.org/2008/03/sml"> <Name>Physics101</Name> </Course> </data> </document> </instances> </model>
Mark, please continue to keep us updated if you are blocked in any way, things like setup issues.
I am unable to replicate this bug as described. I get two errors when I try to validate the document in question. The first error I am getting with every SML-IF document I try to validate: src-resolve: Cannot resolve the name 'xml:lang' to a(n) 'attribute declaration' component The error is always referenced at some line number greater than the number of lines in the actual file. David and I discussed this at some length earlier today, and he is going to investigate further. This may be some other defect, or it may be some issue with my environment. The second error does seem relevant to this defect: 'The referenced document on line number 67 cannot be resolved. Make sure the document exists and is valid.' This sounds much more like the behavior we would expect. The file I am validating is remote-document.xml, as described in the original report. I have tried removing the definitions file as described, and I have tried having the locate tag in the documents section both by itself and with another document defined within a data tag, but cannot get any errors other than the ones described above. Interestingly, the error is referencing the http://www.w3.org/1999/xlink URL, and not the URL pointing to the SML document: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co. To see this, simply move the attributes to different lines and revalidate. I'm going to see if I can step through the validation code in debug mode to figure out what is actually happening at the point the error is discovered, but wanted to document my findings since they seem to be at odds with the reported behavior. As a side note, I noticed that there is no undo stack in the editor. Is this intentional, or a bug, or perhaps another environmental issue specific to me?
(In reply to comment #2) > As a side note, I noticed that there is no undo stack in the editor. Is this > intentional, or a bug, or perhaps another environmental issue specific to me? Thanks for the detailed report, Mark. I hope to dig into it a bit today. Re: undo stack, you mean there is no way to undo changes in the Source tab of the editor, correct? Yes, there is more that can be done with the editor there. Ideally, we should be able to leverage the XML editor from WTP, if WTP is loaded, and gain their undo stack, outliner, and other features "for free", since we would rather reuse than reinvent. I'd be happy to assign that work to you after this one is complete, if you like. ;-)
Incorporating the XML editor from WTP sounds like a useful (for everyone) and educational (for me) thing to do, so I'll gladly take the task as soon as everyone is satisfied with the resolution of this one. I hope to have a little more status regarding the validation error soon.
Regarding the following error: src-resolve: Cannot resolve the name 'xml:lang' to a(n) 'attribute declaration' component Can you see if you can access: http://www.w3.org/ If you run the SML tests too often, w3 will block your IP address from accessing the schema file required to validate the XML content.
That is a great thought, but I'm pretty sure it isn't the cause here. I'm able to access w3.org with no problems. Well, actually, you may be on to something here. I'm able to access it with no problems through my browser, which goes through a proxy server. Is there a configuration setting somewhere that allows me to specify proxy server settings when the validator is fetching things? Are you behind a proxy server, and do you get the same error message?
(In reply to comment #5) > Regarding the following error: > > src-resolve: Cannot resolve the name 'xml:lang' to a(n) 'attribute > declaration' component > > Can you see if you can access: http://www.w3.org/ > If you run the SML tests too often, w3 will block your IP address from > accessing the schema file required to validate the XML content. Ali, I thought we had the SML schema locally in the plugin, so that Internet access isn't required for validation?
Ok, more information: Debugging reveals that the code is actually failing not while trying to access w3.org (and why would it, because aren't we just using that in a namespace declaration here?), but while trying to access: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co The error message is a bit misleading, because, as mentioned before, if the attributes are on different lines, the error message points to the xmlns:xlink attribute, and not the xlink:href attribute. I have no explanation for this, but for sure, the problem is actually trying to load the URL pointing to the actual remote document. The original exception is thrown from URL's openStream method (see line 216 of SMLValidatorUtil), and the message in the exception is java.net.ConnectException: Connection Refused This is beginning to look more and more like an environmental problem specific to my environment. My next thought is to further explore the proxy server issue.
Some quick and dirty code changes verify that Ali's idea was correct: public static Node retrieveRemoteDocument(String uri) throws RemoteRetrievalException { Exception exception = null; InputStream stream = null; try { URL locator = new URL(uri); //stream = locator.openStream(); Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("inetgw.unx.sas.com", 80)); stream = locator.openConnection(proxy).getInputStream(); After hard-coding my local proxy server to get the connection, the error goes away and everything validates (well, I still have the odd "src-resolve: Cannot resolve the name 'xml:lang' to a(n) 'attribute declaration' component" issue, but I'm fairly certain that is a different issue). So the question becomes: How do we give users the ability to specify a proxy server when trying to validate remote document references? Shouldn't this be a global setting somewhere? Any thoughts?
Clearly I have been a little trigger happy when updating defects. Of course, Eclipse has a preference for proxy settings under General -> Network Connections. I stripped out my hard-coded proxy setting that I was using for debugging, set up the proxy server on the launched version of Eclipse I was debugging against, and everything worked differently, if not correctly. There were no issues related to the locator element, and the "src-resolve" message went away. Now there is just a single error: "cvc-elt.1: Cannot find the declaration of element 'document'" on line 81 of the document, which looks like: 79: <document> 80: <docinfo> 81: <aliases> 82: 83: <alias>http://interop.serviceml.org/Universities/MIT/PHY101.xml</alias> 84: </aliases> 85: </docinfo> 86: <data> 87: <Course xmlns="urn:university" xmlns:sml="http://www.w3.org/2008/03/sml"> Like the other issues I have come through so far, this seems to be different from the originally reported issues. More debugging to come, but should anything be done to address the difficulties in figuring out the original problems I've had? This might be as simple as enhancing the error message to say something like, "The document referenced (http://...) could not be retrieved. Please check your network connections (including proxy settings, if applicable) to ensure that they are correct.
(In reply to comment #10) > Like the other issues I have come through so far, this seems to be different > from the originally reported issues. More debugging to come, but should > anything be done to address the difficulties in figuring out the original > problems I've had? This might be as simple as enhancing the error message to > say something like, "The document referenced (http://...) could not be > retrieved. Please check your network connections (including proxy settings, if > applicable) to ensure that they are correct. Yes, I like the new message wording, although I'd still like to keep the line number in the message to help the user locate the error. Feel free to go ahead and make that change as part of this defect, and let me know if you have any questions about how to do message strings with variables.
Mark, when trying to validate the file in question, did you actually change the file to match the SML 1.1 spec, or were you validating it as it exists currently in CVS? If I validate it using the JUnit tests in org.eclipse.cosmos.rm.internal.validation.core.TestDatastructureBuilderImpl.testRemoteDocuments(), it passes, but from what I understand Ali to have said, the .xml file needs to change to use the new locator syntax. If I validate the file from the UI (select file > validate sml-if), I get three errors. We really need to fix the problems with interactive validation.
I've been totally concentrating on the validation message part and completely overlooked the schema change part. I've been validating against the file as it exists in CVS. So to match the 1.1 specification, should the remote document part should change from: <document> <locator xlink:href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co" xmlns:xlink="http://www.w3.org/1999/xlink"/> </document> to <document> <locator> <documentURI> http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co </documentURI> </locator> </document> Am I reading that right? With regards to the interactive validation, I think I am making some strides. The problems we are seeing (I think) are a result of issues in ElementSourceBuilder. It appears that when the locator element is present, the line counts around the various documents are getting hosed, and so the getElementSource method is setting up the segments with unbalanced tags for the parts which come after the document element containing the locator... Hence David's "The markup in the document preceding the root element must be well-formed". I still haven't had time to monkey with the code enough to work out a fix, but I'm pretty sure that's the problem.
Looks like someone else is having a similar problem as Mark: http://dev.eclipse.org/newslists/news.eclipse.technology.cosmos/msg00165.html
Ali, This might be my XML Schema ignorance here, but I don't understand this notation: <documentURI/> ? xs:any <!-- A URI or IRI that points to a definition document goes here --> I would have expected to instead see: <documentURI> ? xs:any <!-- A URI or IRI that points to a definition document goes here --> </documentURI> The way the notation reads in the spec, it seems like you would have this as an example: <locator> <documentURI/> http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests/src/test-resources/others/sampleLocatorFile.xml?root=Technology_Project&view=co </locator> Is putting the /> there some sort of shorthand notation in the schema to indicate that whatever is following goes inside that element?
OK, I looked at it further and it seems that the <documentURI/> was "pseudo schema". Reading through the actual schema, it would seem that Mark's change to the sample is correct. Mark, when you make that change and run the JUnit, how does it change things? I'm assuming we'll need to change our SAX handlers to parse that correctly.
Just to help you out, Mark, it looks like the place we handle the <locator> element is: org.eclipse.cosmos.rm.internal.validation.core.FoundationBuilder.startElement(String, String, String, Attributes) This will need to change to look for a documentURI element and to handle its contents. Currently it looks for the URI within the locator's attributes, as per the old syntax. If you don't have a link to the spec, this is described in the SML-IF spec here: http://www.w3.org/TR/sml-if/
Created attachment 104368 [details] patch for the FoundationBuilder class
Created attachment 104369 [details] Patch for the ISMLConstants class Adds additional state storage needed for SML-IF 1.1 validation conformance
Created attachment 104370 [details] Patch for remote-document.xml test file Changes the format of a document instance which references a remote document so that it conforms to the updated SML 1.1 specification.
Expanded the SAX parsing mechanisms to accommodate the SML 1.1 spec changes to the locator element (which now has a documentURI element embedded, rather than having a URI attribute and a URI namespace attribute). Expanded ISMLConstants to add new states for being inside the locator and documentURI elements, and introduced a local StringBuffer to append to whenever the characters method was invoked within that state. Altered the retrieveRemoteDocument method to accept a string (containing the referenced URI) as a parameter, rather than an Attribute, and moved the invocation of retrieveRemoteDocument from startElement to endElement, since we should now wait until the close of the documentURI tag to ensure we have captured all of the URI. The JUnit test TestDataStructureImpl.testRemoteDocuments() now passes when the new format is encountered. In the process of debugging this defect, numerous other related UI and documentation issues were discovered, but those will be entered as separate defects.
Mark, I was just about to commit your changes, and ran the full suite of validator tests at the last minute. It appears that your changes introduce a new test failure. When I remove your patches, all the validation tests pass again. Sorry to be the bearer of bad news regarding your first SML bug fix. :-) Can you take a look at it? Here is the failure I'm seeing: AllTests - validation AllTests TestMainValidator testIncompleteInValid(org.eclipse.cosmos.rm.internal.validation.core.TestMainValidator) junit.framework.AssertionFailedError: The actual file has less content than expected at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertNotNull(Assert.java:217) at org.eclipse.cosmos.common.tests.CommonTestUtility.compareFiles(CommonTestUtility.java:77) at org.eclipse.cosmos.common.tests.CommonTestUtility.compareFiles(CommonTestUtility.java:58) at org.eclipse.cosmos.common.tests.CommonTestUtility.compareFiles(CommonTestUtility.java:52) at org.eclipse.cosmos.common.tests.CommonTestUtility.compareFiles(CommonTestUtility.java:38) at org.eclipse.cosmos.rm.internal.validation.core.AbstractMainValidator.startValidation(AbstractMainValidator.java:73) at org.eclipse.cosmos.rm.internal.validation.core.AbstractMainValidator.startValidation(AbstractMainValidator.java:48) at org.eclipse.cosmos.rm.internal.validation.core.AbstractMainValidator.startValidation(AbstractMainValidator.java:41) at org.eclipse.cosmos.rm.internal.validation.core.AbstractMainValidator.startValidation(AbstractMainValidator.java:36) at org.eclipse.cosmos.rm.internal.validation.core.TestMainValidator.testIncompleteInValid(TestMainValidator.java:499) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:618) at junit.framework.TestCase.runTest(TestCase.java:164) at junit.framework.TestCase.runBare(TestCase.java:130) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:120) at org.eclipse.hyades.test.common.junit.HyadesTestCase.run(HyadesTestCase.java:172) at junit.framework.TestSuite.runTest(TestSuite.java:230) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.runTest(HyadesTestSuite.java:457) at junit.framework.TestSuite.run(TestSuite.java:225) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.doRun(HyadesTestSuite.java:425) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.run(HyadesTestSuite.java:338) at junit.framework.TestSuite.runTest(TestSuite.java:230) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.runTest(HyadesTestSuite.java:457) at junit.framework.TestSuite.run(TestSuite.java:225) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.doRun(HyadesTestSuite.java:425) at org.eclipse.hyades.test.common.junit.HyadesTestSuite.run(HyadesTestSuite.java:338) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
Also, Mark, you'll want to add a line to the copyright statement of each class you change to indicate the SAS copyright statement. You can find an example here, just add a line for SAS under contributors: http://wiki.eclipse.org/COSMOS_dev_process#3.__Complete_Coding.2FImplementation
Created attachment 104669 [details] patch for another test file for testing project
Created attachment 104670 [details] patch for yet another test file for testing project
Received from Mark in email: After a bunch of tail chasing through the debugger, I realized that the issue was that one of the test files still had the old 1.0 format for a locator reference. Updating that file (there are two copies of it in different places, so I updated them both), seems to fix the problem. I have attached patches for both copies. If this works and all the tests pass for you (see notes below), feel free to commit and let me know, and I'll update the defect with the new patches. I'm still getting two failures when I run AllTests. I never had gotten AllTests to run completely through, and before I submitted last time, I just assumed (hastily) that all of the failures were pre-existing failures not introduced by my code change. For sure the one you mentioned was, and I have that cleared up, but the other two seem unrelated, so I'm wondering if they are some environmental wrinkles I haven't ironed out yet. If they're not occuring for you, I think we can assume that is the case, and I'll try to figure them out later. I've also attached screenshots of them, for completeness (The junit output editor won't let me cut and paste its output). Let me know what turns up for you.
Mark, I applied your patched test files. What I found was that when I run the validation tests using the IBM JRE 1.5, all the tests pass. When I run using Sun JRE 1.5, I get the exact same two failures as you. We will go ahead and consider this current bug as fixed, and I will open a new bug for the Sun JRE issue.
closing
