Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 136219

Summary: convert bug parser to use xml and rdf sources
Product: z_Archived Reporter: Robert Elves <robert.elves>
Component: MylynAssignee: Robert Elves <robert.elves>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P2 CC: ekuleshov, mik.kersten, robert.elves
Version: unspecified   
Target Milestone: 0.6   
Hardware: PC   
OS: All   
Whiteboard:
Attachments:
Description Flags
Progress: Bugzilla server configuration retrieved from rdf.
none
mylar/context/xml
none
Progress: Missing components patch.
none
Fix npe in AbstractBugzillaWizardPage
none
Progress: fixes new bug submission errors
none
Provides default new bug attributes and OS/Platform detection fixed.
none
Missed file with error handler.
none
More cleaning up of attribute handling
none
mylar/context/xml/zip
none
mylar/context/zip
none
Bug reports read from xml stream
none
mylar/context/zip none

Description Robert Elves CLA 2006-04-11 18:56:35 EDT
Currently we rely on screen scraping html output from Bugzilla in order to get bug reports (their details, the available attributes and operations).  Bug reports are available in xml form by appending ctype=xml to the show_bug.cgi url (example: https://bugs.eclipse.org/bugs/show_bug.cgi?id=7016&ctype=xml ). The parameters such as OS, Product etc are also available via the config.cgi interface and can be retrieved in RDF format (example: https://bugs.eclipse.org/bugs/config.cgi?ctype=rdf ). We need to make use of these interfaces since they are not susceptible to changes in the presentation layer.

The only information that we will loose is the available operations.  An interim solution would be to make all operations available to the user.
Comment 1 Eugene Kuleshov CLA 2006-04-12 00:31:50 EDT
Interestingly, BugZilla integrated into CollabNet has similar feature, though URL is different: https://xcommonj-work.projects.dev2dev.bea.com/issues/xml.cgi?id=1
Comment 2 Mik Kersten CLA 2006-04-12 11:27:56 EDT
Yup, this stands a chance of giving us 'free' CollabNet integration (!).  I'm very happy that Rob and I sat in on the Bugzilla devs meeting yesterday since that helped point us in the right direction to taking baby steps towards a WS API.

Rob: what about getting back query results?
Comment 3 Eugene Kuleshov CLA 2006-04-12 11:35:55 EDT
There are some difference in tag names for task info, but generally 

CollabNet does have xml representation for query hits. I didn't noticed it before, but it seems that Bugzilla version on eclipse.org has support for atom feed. E.g.: https://bugs.eclipse.org/bugs/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=eu%40md.pp.ru&emailtype1=exact&emailassigned_to1=1&emailreporter1=1&remaction=&query_format=advanced&ctype=atom
Comment 4 Eugene Kuleshov CLA 2006-04-12 11:45:29 EDT
BTW, there is also a CVS format avialable on both BugZilla and CollabNet. Interesting that issue descriptions are not truncated up there as opposed to html query result page...
Comment 6 Eugene Kuleshov CLA 2006-04-12 14:57:40 EDT
I'd like to suggest Rome as an rss/rdf/atom parser. So, it could be also used for raw RSS integration. See https://rome.dev.java.net/
Comment 7 Robert Elves CLA 2006-04-24 14:15:07 EDT
Note: There is an open Bugzilla report ( https://bugzilla.mozilla.org/show_bug.cgi?id=264785 ) related to the issue of special characters not being escaped in the RDF/XML export facility.
Comment 8 Robert Elves CLA 2006-04-24 20:55:07 EDT
Created attachment 39378 [details]
Progress: Bugzilla server configuration retrieved from rdf.
Comment 9 Robert Elves CLA 2006-04-24 21:27:29 EDT
Created attachment 39381 [details]
mylar/context/xml

rdf configuration context
Comment 10 Robert Elves CLA 2006-04-25 12:59:52 EDT
Created attachment 39429 [details]
Progress: Missing components patch.
Comment 11 Mik Kersten CLA 2006-04-25 13:07:52 EDT
Patch applied and verified via adding new query to clean workspace.

Rob, you can do the bug stuff on this report, but create a new one for the RDF-based search.
Comment 12 Mik Kersten CLA 2006-04-25 16:49:44 EDT
When trying to add a new bug report I get the following error after selecting a product:

java.lang.NullPointerException
	at org.eclipse.mylar.internal.bugzilla.ui.wizard.AbstractBugzillaWizardPage.setPlatformOptions(AbstractBugzillaWizardPage.java:742)
	at org.eclipse.mylar.internal.bugzilla.ui.wizard.AbstractBugzillaWizardPage.createControl(AbstractBugzillaWizardPage.java:382)
	at 
Comment 13 Mik Kersten CLA 2006-04-25 16:55:46 EDT
It's no longer possible to submit bugs.  This error gets logged, and Eclipse has to be killed:

java.lang.NullPointerException
at org.eclipse.mylar.internal.bugzilla.ui.wizard.AbstractBugzillaWizardPage.setPlatformOptions(AbstractBugzillaWizardPage.java:742)
at org.eclipse.mylar.internal.bugzilla.ui.wizard.AbstractBugzillaWizardPage.createControl(AbstractBugzillaWizardPage.java:382)
at org.eclipse.jface.wizard.WizardDialog.updateForPage(WizardDialog.java:1046)
at org.eclipse.jface.wizard.WizardDialog.access$2(WizardDialog.java:1038)
at org.eclipse.jface.wizard.WizardDialog$4.run(WizardDialog.java:1028)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
at org.eclipse.jface.wizard.WizardDialog.showPage(WizardDialog.java:1026)
at org.eclipse.jface.wizard.WizardDialog.nextPressed(WizardDialog.java:757)
Comment 14 Robert Elves CLA 2006-04-25 17:54:38 EDT
Created attachment 39484 [details]
Fix npe in AbstractBugzillaWizardPage

Not completely sure how this came about. Here is a patch that will check for the npe.  Apply the patch then update your products and try again.
Comment 15 Mik Kersten CLA 2006-04-25 17:57:51 EDT
But are you able to submit bugs successfully?  I don't think it was just the NPE preventing that.  Also, just checking for the NPE will fail to set the platform opions correctly, so that can't be right.
Comment 16 Robert Elves CLA 2006-04-25 18:04:53 EDT
You are right. Please ignore the previous patch. New one to follow...
Comment 17 Mik Kersten CLA 2006-04-25 18:08:39 EDT
I'm also seeing these scary messages printed to my console now and then, and think they might be coming from the Bugzilla support since I saw them way back.  Watch for your runtime workspace generating them.

[Fatal Error] :-1:-1: Premature end of file.
Comment 18 Robert Elves CLA 2006-04-25 21:19:59 EDT
Created attachment 39498 [details]
Progress: fixes new bug submission errors

I did manual tests on 2.20.1 and 2.22. Let me know if you still have problems with eclipse.org.
Comment 19 Mik Kersten CLA 2006-04-25 21:33:57 EDT
Patch applied and verified (new bug submitted succesfully to eclipse.org).

Problems (hopefully easy to fix):

- Instead of the default values being selected for attributes, the top values are now selected.  This is bad for us because every bug will be submitted as a P1 "blocker" ;)  Need to restore default values.  If this is not exposed via config.cgi, you may be forced to select the middle value for priority and severity, and the last value for version, and the top value for component.

- OS and Platform selection is not working anymore.  Probably because it's matching the wrong strings?


Comment 20 Robert Elves CLA 2006-04-26 17:12:46 EDT
Created attachment 39599 [details]
Provides default new bug attributes and OS/Platform detection fixed.

Due to the inconsistent state of my working environment I need you to run all tests, apply the patch, and run them again. Thanks.
Comment 21 Robert Elves CLA 2006-04-26 20:33:31 EDT
Created attachment 39615 [details]
Missed file with error handler.
Comment 22 Mik Kersten CLA 2006-04-26 20:36:36 EDT
Patches applied.
Comment 23 Robert Elves CLA 2006-04-28 16:29:13 EDT
Created attachment 39833 [details]
More cleaning up of attribute handling
Comment 24 Robert Elves CLA 2006-04-28 16:35:38 EDT
Created attachment 39835 [details]
mylar/context/xml/zip

context for attributes
Comment 25 Eugene Kuleshov CLA 2006-04-29 14:30:58 EDT
Strange, but I don't see any attached context for this report. Using Mylar 0.5.1
Comment 26 Mik Kersten CLA 2006-04-29 14:36:00 EDT
We changed the "Attachment" description to mylar/context/zip, and this only has the old descriptions (e.g. mylar/context/xml/zip).

Btw, I'm about to go offline until Wed.
Comment 27 Eugene Kuleshov CLA 2006-04-29 14:53:01 EDT
(In reply to comment #26)
> We changed the "Attachment" description to mylar/context/zip, and this only has
> the old descriptions (e.g. mylar/context/xml/zip).

which report has new ones?

> Btw, I'm about to go offline until Wed.

too bad. :-(

Comment 28 Eugene Kuleshov CLA 2006-04-29 14:58:43 EDT
(In reply to comment #26)
> We changed the "Attachment" description to mylar/context/zip, and this only has
> the old descriptions (e.g. mylar/context/xml/zip).

Actually, shouldn't be both descriptions supported?

Comment 31 Robert Elves CLA 2006-05-02 15:04:24 EDT
Created attachment 40093 [details]
mylar/context/zip

RDF Configuration retrieval
Comment 32 Mik Kersten CLA 2006-05-04 17:17:15 EDT
Regarding the PS, a slicker way of doing that would probably be to make the Bugzilla editor's "paste" be smarter about pasting a refernce to the report and only paste "bug 123" since that follows the hyperlink policy.  Please submit a new bug for that.
Comment 33 Robert Elves CLA 2006-05-08 19:05:11 EDT
Created attachment 40670 [details]
Bug reports read from xml stream
Comment 34 Robert Elves CLA 2006-05-08 19:06:49 EDT
Created attachment 40671 [details]
mylar/context/zip

BugReports (BugzillaReport) parsed from xml
Comment 35 Mik Kersten CLA 2006-05-09 00:39:01 EDT
Line wrapping needs to be added back.  Perhaps now it can be dynamic (i.e. scaled to window width)?
Comment 36 Mik Kersten CLA 2006-05-12 22:28:32 EDT
Rob, this is working great, we just need to fix up the extra spacing in the editor.
Comment 37 Robert Elves CLA 2006-05-16 16:34:05 EDT
Fixed. See bug# 142084 for discussion regarding bugzilla comment layout.