Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 230924 - SAXParseException resolving update site mirrors
Summary: SAXParseException resolving update site mirrors
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P1 critical (vote)
Target Milestone: 3.4 RC1   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 232421 (view as bug list)
Depends on:
Blocks: 272414
  Show dependency tree
 
Reported: 2008-05-07 12:08 EDT by Nick Boldt CLA
Modified: 2009-04-16 14:22 EDT (History)
8 users (show)

See Also:
dj.houghton: review+


Attachments
Fix (1.16 KB, patch)
2008-05-12 16:36 EDT, John Arthorne CLA
no flags Details | Diff
Patch that corrects the problem (1.17 KB, patch)
2009-04-15 19:27 EDT, Thomas Hallgren CLA
no flags Details | Diff
Sorry, that one was outdated. Here is one that's current. (1.17 KB, patch)
2009-04-15 19:28 EDT, Thomas Hallgren CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2008-05-07 12:08:43 EDT
Steps to repro:

Install Eclipse 3.4M7, then add some update sites:

http://download.eclipse.org/modeling/emf/updates/milestones/
http://download.eclipse.org/modeling/mdt/updates/milestones/
http://download.eclipse.org/modeling/emft/updates/milestones/

Then install something, like the OCL SDK.

I think you're missing an ampersand when you assemble the URL below. 

Compare:

BAD: http://www.eclipse.org/downloads/download.php?file=/modeling/mdt/updates/format=xml&countryCode=us&timeZone=-5&responseType=xml
GOOD: http://www.eclipse.org/downloads/download.php?file=/modeling/mdt/updates/&format=xml&countryCode=us&timeZone=-5&responseType=xml

full error message:

!ENTRY org.eclipse.equinox.p2.artifact.repository 4 0 2008-05-07 10:58:50.645
!MESSAGE Error processing mirrors URL: http://www.eclipse.org/downloads/download.php?file=/modeling/mdt/updates/milestones/format=xml&countryCode=us&timeZone=-5&responseType=xml
!STACK 0
org.xml.sax.SAXParseException: The reference to entity "lang" must end with the ';' delimiter.
        at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:264)
        at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:292)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:146)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.computeMirrors(MirrorSelector.java:120)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.initMirrors(MirrorSelector.java:187)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.selectMirror(MirrorSelector.java:227)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorLocation(MirrorSelector.java:165)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getMirror(SimpleArtifactRepository.java:428)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:408)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:452)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:132)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:109)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:441)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:64)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Comment 1 John Arthorne CLA 2008-05-12 16:35:42 EDT
Wow, mirroring is currently broken because of this bug.
Comment 2 John Arthorne CLA 2008-05-12 16:36:39 EDT
Created attachment 99810 [details]
Fix

Off by one error.
Comment 3 Pascal Rapicault CLA 2008-05-12 16:43:42 EDT
Reviewed patch, +1
Comment 4 John Arthorne CLA 2008-05-12 17:03:13 EDT
David, I believe mirroring won't work on the Ganymede site due to this bug. There are two options for a quick fix:

 1) Re-run p2 enablement with a fixed generator
 2) Manually fix the mirrorsURL argument in content.xml and artifacts.xml (add the '&' character in the right place.

I.e., change this:

<property name='p2.mirrorsURL' value='http://www.eclipse.org/downloads/download.php?file=/releases/ganymede/staging/format=xml&amp;protocol=http'/>

to:

<property name='p2.mirrorsURL' value='http://www.eclipse.org/downloads/download.php?file=/releases/ganymede/staging/&amp;format=xml&amp;protocol=http'/>


Mirroring likely isn't too important for a milestone as far as bandwidth use, but it would be good to get wider testing with mirroring enabled.
Comment 5 John Arthorne CLA 2008-05-12 17:40:54 EDT
Fix released.
Comment 6 David Williams CLA 2008-05-12 18:18:33 EDT
I fixed the ganymede released (not staging) jars "by hand". 

Let me know when this makes it into build suitable to use in our build.eclipse.org scripts, so the "staging" versions will be correct for next time. 

Comment 7 John Arthorne CLA 2008-05-12 18:38:52 EDT
Thanks David!
Comment 8 Eike Stepper CLA 2008-05-21 14:59:25 EDT
With the RC1 platform I just got many errors like the one below. Nick suggested to reopen this bug...

---

eclipse.buildId=I20080516-1333
java.version=1.6.0_05
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=de_DE
Command-line arguments:  -os win32 -ws win32 -arch x86


Error
Wed May 21 20:45:02 CEST 2008
Error processing mirrors URL: http://www.eclipse.org/downloads/download.php?file=/modeling/emft/updates/milestones/format=xml&countryCode=de&timeZone=1&responseType=xml

org.xml.sax.SAXParseException: The reference to entity "lang" must end with the ';' delimiter.
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.computeMirrors(MirrorSelector.java:125)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.initMirrors(MirrorSelector.java:192)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.selectMirror(MirrorSelector.java:232)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorLocation(MirrorSelector.java:170)
at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getMirror(SimpleArtifactRepository.java:428)
at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:408)
at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:452)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:132)
at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:109)
at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:441)
at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:64)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Comment 9 Eike Stepper CLA 2008-05-21 15:04:31 EDT
I forgot to describe how to reproduce:

1) install fresh eclipse RC1
2) install emf RC1 by unzipping (don't think that this is really needed)
3) download+install net4j RC1a via p2 UI (from http://download.eclipse.org/modeling/emft/updates/milestones/)
Comment 10 John Arthorne CLA 2008-05-21 15:37:57 EDT
Nick, have you regenerated the EMF repository with a version of the generator that contains this fix? The bug is fixed, but any repository that was created with the buggy generator will still have the error.
Comment 11 Pascal Rapicault CLA 2008-05-21 15:51:14 EDT
Note that the metadata generator in the base builder has been updated to RC1 yesterday.
Comment 12 Nick Boldt CLA 2008-05-21 17:24:29 EDT
(In reply to comment #11)
> Note that the metadata generator in the base builder has been updated to RC1
> yesterday.

Was it tagged and released? I see no update here:

http://wiki.eclipse.org/index.php/Platform-releng-basebuilder#org.eclipse.releng.basebuilder_update_history

(In reply to comment #10)
> Nick, have you regenerated the EMF repository with a version of the generator
> that contains this fix? The bug is fixed, but any repository that was created
> with the buggy generator will still have the error.

Yes. My process deletes the existing content.jar and artifacts.jar before creating  new ones, so either the jars exist after being created w/ the M7_34 basebuilder's org.eclipse.equinox.p2.metadata.generator.EclipseGenerator, or they're not there. 

You can see the contents of these jars here: 

http://emft.eclipse.org/modeling/emft/updates/milestones/artifacts.jar
http://emft.eclipse.org/modeling/emft/updates/milestones/content.jar

Unpacking them, I see:

    <property name='p2.mirrorsURL' value='http://www.eclipse.org/downloads/download.php?file=/modeling/emft/updates/milestones/format=xml'/>

in both the artifacts.xml and content.xml files.

Comment 13 John Arthorne CLA 2008-05-21 18:39:55 EDT
Ok, marking fixed again. You're not running a generator with the fix. I don't know the schedule on which the base builder is tagged ... ask Kim.
Comment 14 John Arthorne CLA 2008-05-30 16:24:25 EDT
*** Bug 232421 has been marked as a duplicate of this bug. ***
Comment 15 Martin Oberhuber CLA 2008-06-06 09:42:39 EDT
Since I just ran into this with Eclipse 3.4RC3, I'd like to clarify again:

1. The bug is in an incorrectly created artifacts.jar / artifacts.xml
2. That file is generated by the p2.metadata.generator
3. When using basebuilder, your build system needs a newer version than
   basebuilder M7_34

I've just upgraded to basebuidler RC2_34 which is current according to the Wiki, then re-generated my P2 Metadata, and voila the bug is gone. Thanks!
Comment 16 Thomas Hallgren CLA 2009-04-15 19:26:34 EDT
This bug seems to have resurfaced in 3.5. I was just bitten by it.
Comment 17 Thomas Hallgren CLA 2009-04-15 19:27:15 EDT
Created attachment 132001 [details]
Patch that corrects the problem
Comment 18 Thomas Hallgren CLA 2009-04-15 19:28:55 EDT
Created attachment 132002 [details]
Sorry, that one was outdated. Here is one that's current.
Comment 19 Thomas Hallgren CLA 2009-04-15 19:41:11 EDT
Just to clarify. The problem is when the repository is downloaded, not when it's generated. The mirrorsURL in the repository does not contain the responseType=xml entry.
Comment 20 John Arthorne CLA 2009-04-16 09:49:33 EDT
Thomas, I don't see the relationship between your new patch and the original bug here. This bug was about an indexing error that resulted in an '&' sign being missing from the mirrors URL. Your patch seems to be changing "responseType=xml" to "format=xml". Were you getting a SAXParseException, or what was the symtom in your case?
Comment 21 Thomas Hallgren CLA 2009-04-16 10:06:58 EDT
This is the symptom (and also the similarity) in the stack-trace below. The cause is different, since this actually the MirrorSeletctor appending the wrong thing rather then the generator doing it.

If your mirrorsURL already has a format=xml declared, you'll never notice. But if you lack that declaration, then the download.php page will not see it at all because it doesn't consider the responseType=xml incorrectly added by the MirrorSelector.


!MESSAGE Error processing mirrors URL:
http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/tools-3.5/&protocol=http&countryCode=us&timeZone=-5&responseType=xml
!STACK 0
org.xml.sax.SAXParseException: The reference to entity "lang" must end with the
';' delimiter.
        at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown
Source)
        at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown
Source)
        at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.computeMirrors(MirrorSelector.java:135)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.initMirrors(MirrorSelector.java:196)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.selectMirror(MirrorSelector.java:246)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorLocation(MirrorSelector.java:174)
        at
org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getMirror(SimpleArtifactRepository.java:492)
        at
org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:448)
        at
org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:516)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.getArtifact(MirrorRequest.java:195)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle(MirrorRequest.java:170)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:154)
        at
org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:95)
        at
org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:505)
        at
org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:64)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Comment 22 John Arthorne CLA 2009-04-16 14:19:42 EDT
Thanks for clarifying Thomas. I have entered bug 272539 for this.
Comment 23 John Arthorne CLA 2009-04-16 14:22:07 EDT
Marking this bug fixed again because a code change was made to address this bug in 3.4 RC1.