Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 115941

Summary: provide web forum based issue provider
Product: z_Archived Reporter: Erkki Lindpere <villane>
Component: MylynAssignee: Mik Kersten <mik.kersten>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: P4 CC: ekuleshov, remy.suen
Version: unspecifiedKeywords: helpwanted
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:

Description Erkki Lindpere CLA 2005-11-10 20:08:38 EST
I have some half-complete code that can communicate with web forums. It has a
generic API and defines an extension point so implementations could be written
for any specific forum (that is somewhat similar to the popular ones like
vBulletin and phpBB).

I want to use this code to create an additional provider for Mylar that could
import topics/threads from a bulletin board into the task list.

At first I'm planning to experiment with creating a simple query that would work
similarly to a Bugzilla Query and would create tasks based on topics read from a
selected (sub)forum.

What extension points or interfaces do I need to implement to do this?

See also Bug 102848
Comment 1 Mik Kersten CLA 2005-11-22 20:28:31 EST
The way to do this would be to start out by mimicking the way that Bugzilla queries work.  You'll notice that the task list has an org.eclipse.mylar.tasklist.providers extension point that requires externalizer and handler classes for dealing with saving state and task actions.  I hope that these extension points and APIs will get simplified some, and any progress on that will be posted on bug 102848.
Comment 2 Mik Kersten CLA 2006-05-23 15:24:11 EDT
Erkki, are you still intersted in this?  We could make it a connector that people vote for (see bottom of http://www.eclipse.org/mylar/start.php) but I think it would probably need to target a specific forum.
Comment 3 Erkki Lindpere CLA 2006-05-23 16:38:23 EDT
I am still interested, but the forum client project hasn't been in very active developement due to various issues. I do want to pick it up again, though.
If I'd have to choose a particular forum, I think I'd go with phpBB 2 since it's open source and quite popular, and my implementation for the generic client is more complete than the vBulletin one.
Maybe I'll start messing with this in the coming weekend...
Comment 4 Erkki Lindpere CLA 2006-05-24 16:42:14 EDT
Just letting you know I already started working on the phpBB connector (I've been getting somewhat bored lately and didn't immediately remember that I had this thing planned so Mik's reminder had excellent timing :)). Already a repository can be added and the connection settings validated.

I have some questions regarding how to best implement this for Mylar:
My phpBB client implementation uses Jakarta Commons HttpClient (3.0) instead of the java.net classes. Would this be ok if the provider would eventually be included in Mylar? It currently also uses Log4J, I haven't looked into how to best do logging in Eclipse/OSGi yet. HttpClient also requires Commons Codec and Commons Logging. Commons Logging and Log4J are provided by the Callisto update site as of RC3, so that leaves HttpClient and Codec, which I currently provide as custom osgi bundles.

Should I base the phpBB connector solely on the Bugzilla connector or should I also look into the Jira one?
Comment 5 Mik Kersten CLA 2006-05-24 18:42:21 EDT
The HttpClient and Commons dependencies are fine.  It would be best not to use Log4J but instead to use the MylarStatusHandler class.

You should not be depending on mylar.bugzilla, just on mylar.tasklist, and it's best to use mylar.jira as an example.  We'll be providing common editing (bug138043) and offline storage facilities (bug 143260) soon which will also result in some refactoring, so it's best to stay away from those features for now.

Keep me posted on how it goes!  Btw, what user communities would make use of the phpBB connector? 
Comment 6 Erkki Lindpere CLA 2006-05-25 03:37:11 EDT
(In reply to comment #5)

> The HttpClient and Commons dependencies are fine.  It would be best not to use
> Log4J but instead to use the MylarStatusHandler class.

Ok, but if I plan to use the same library outside of Mylar without rewriting it, maybe I could keep using Log4J and write a MylarStatusHandlerAppender for it? 

> Keep me posted on how it goes!  Btw, what user communities would make use of
> the phpBB connector? 

To be honest, I don't really know of any potential immediate users. I didn't really even think about this till now... I had a vague idea myself about using it myself privately in a project that I plan to open source later.
Comment 7 Erkki Lindpere CLA 2006-05-25 15:27:35 EDT
I've now made it possible to create queries by selecting a forum (similar to selecting the predefined filters for a Jira query). Maybe tomorrow or on the weekend I'll implement getting threads from a forum as issues. But this is just a prototype -- beyond that, I think this need some analysis to determine what kind of objects to treat as issues.

Also, I'll try to get this into some public SCM server once I have something usable.
Comment 8 Erkki Lindpere CLA 2006-05-26 20:01:51 EDT
Queries now give hits -- threads from the page 1 of the forum. The externalizer also now supports PHPBB queries & hits. Next I'll try to add support for adding new & existing tasks, refactor some things and try to get the source code to a public CVS server.
Comment 9 Mik Kersten CLA 2006-05-29 22:01:54 EDT
Sounds good Erkki, and if you end up using it yourself I'm sure more users will join you :)  Also, it will be useful for us to see how the UI works for this sort of provider.

If you assign me commit rights on the project that you create I can offer to keep the plug-in update as we refactor.  Of the hosting choices (SourceForge, Tigris, java.net, and CodeHaus) my choice would probably be Tigris.org because they provide Subversion and allow you to link to another issue tracker.
Comment 10 Erkki Lindpere CLA 2006-07-12 15:43:34 EDT
I created a project at tigris.org: http://bbapi.tigris.org/. I'll get the code in there in a couple of days.
Comment 11 Mik Kersten CLA 2006-07-12 16:07:36 EDT
Erkki, I've CC'd Eugene since he has been doing some work on HTML-based access to vBulletin.
Comment 12 Eugene Kuleshov CLA 2006-07-12 19:06:16 EDT
Here are the regexps for web repository provider from mylar sandbox plugin:

phpbb: <a href="viewtopic.php\?t=(.+?)" class="topictitle">(.+?)</a>
vBullettin: <a href=\"showthread.php\?.+?t=(\d+?)" id="thread_title_\1">(.+?)</a>
Comment 13 Erkki Lindpere CLA 2006-07-17 13:21:07 EDT
I'm not aware of the mylar sandbox plugin, but I'll take a look at it this week hopefully (I'm not able to spend very much time on this, sadly).

The phpBB and vBulletin BBAPI implementations are now in SVN (The patterns used are mostly in the org.tigris.bbapi.vbulletin.VBParser and org.tigris.bbapi.phpbb.PHPBBParser classes ). Note that these implementations are not complete. But I do have some old code that has more functions implementated but had a different API - I need to copy, paste & refactor that code.

I'll soon import the Mylar provider I was working on into SVN as well.
Comment 14 Erkki Lindpere CLA 2006-07-24 09:27:20 EDT
Ok, I took an initial look at the sandbox plugin (mostly the UI, and some of the code). That seems like a great way to do it, and my idea of writing a provider using BBAPI is much more complicated.

I haven't taken a look at how the templates were implemented yet... Would you somehow be able to add automatic login implementation with each template? The BBAPI providers for phpBB and vBulletin use HttpClient's Cookies mechanism for login (and also parse the cookies for the user id).

With the BBAPI, various features could be added later: like an editor that displays comments, attachements etc. But I'm willing to drop that idea for the moment and use the generic web provider (if it supported automatic login)

Also, I'm planning to refactor the BBAPI and introduce ECF as dependency. If I do that, the provider would then depend on ECF.
Comment 15 Eugene Kuleshov CLA 2006-07-24 10:45:34 EDT
(In reply to comment #14)
> Ok, I took an initial look at the sandbox plugin (mostly the UI, and some of
> the code). That seems like a great way to do it, and my idea of writing a
> provider using BBAPI is much more complicated.
> 
> I haven't taken a look at how the templates were implemented yet... Would you
> somehow be able to add automatic login implementation with each template? The
> BBAPI providers for phpBB and vBulletin use HttpClient's Cookies mechanism for
> login (and also parse the cookies for the user id).

Currently there is no support for logging in. If you have more specific details how this should be handled, please open separate report for web connector (at the moment you can use Mylar/Core module)
Comment 16 Erkki Lindpere CLA 2006-09-29 14:52:49 EDT
I'm sad to say that I'm no longer very interested in doing this connector. Firstly, I don't really have the time right now. Secondly, the generic web provider can already handle some forums. Thirdly, I think there is a lot more benefit in creating connectors for actual issue trackers rather than making a forum into an issue tracker.

If I have some time next year, or the one after that, I might take this up again if noone else does and if I'm still interested.
Comment 17 Mik Kersten CLA 2006-10-02 14:23:58 EDT
Those points make sErkki, thanks for keeping us posted.  I'll mark this for later so that we can revisit any forum-specific needs that the web connector does not handle post 1.0.  Right now we're all fully focused on supporting issue trackers, but post 1.0 we'll be able to step back and consider the broader set of items that might make sense to have in the Task List.
Comment 18 Steffen Pingel CLA 2009-08-20 13:21:02 EDT
Reopening to mark as resolved as per message from webmaster: http://dev.eclipse.org/mhonarc/lists/eclipse.org-committers/msg00778.html.
Comment 19 Steffen Pingel CLA 2009-08-20 13:25:05 EDT
Marking resolved.