Bugzilla – Full Text Bug Listing
| Summary: | Investigate "No Subject" posts from forum users | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Community | Reporter: | Eike Stepper <stepper> | ||||||||||||||||||
| Component: | Forums and Newsgroups | Assignee: | Forums and Newsgroups inbox <forums-inbox> | ||||||||||||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||||||||||||
| Severity: | normal | ||||||||||||||||||||
| Priority: | P3 | CC: | alex.lagarde, bugs.eclipse.org, caniszczyk, daniel_megert, david_williams, deepakazad, eclipse-bugs, Ed.Merks, gunnar, henrik.lindberg, mariot.chauvin, martin.fluegge, Michal.Tkacz, pwebster, rsternberg, ruediger.herrmann, rumburak.eclipse, saurav.sarkar1, sbouchet, sebastian.zarnekow, sven.efftinge, thatnitind, webmaster, wim.jongman | ||||||||||||||||||
| Version: | unspecified | ||||||||||||||||||||
| Target Milestone: | --- | ||||||||||||||||||||
| Hardware: | PC | ||||||||||||||||||||
| OS: | Windows 7 | ||||||||||||||||||||
| Whiteboard: | |||||||||||||||||||||
| Bug Depends on: | 348464, 376695 | ||||||||||||||||||||
| Bug Blocks: | |||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||
|
Description
Eike Stepper
This synchronization issue between NNTP and web interface is also problematic in the opposite way : For example, the question of Stephane Fournier entitled "[CDO] Revert an dirty object to its previous clean state" (01/06/2011) is visible from NNTP but not from the web interface. Created attachment 198186 [details]
Screen capture showing just how out of whack the EMF newsgroup is
Things to note. The post from Hallvard titled "Model browser ala MacOSX' finder" is in the newsgroup, but not in the forum. The post from Briem.lars is in the forum but not the newsgroup. The thread "Trouble with ecore:reference" shows no response in the forum but is answered in the newsgroup.
It's impossible to manage proper service for the community with these two inconsistent streams... It would be better to have two independent mechanisms than this broken version of a single mechanism.
A thought that keeps brewing in my mind is just to move everything to mail lists. It's easier to deal with, projects would have a mail list... e.g., emf-users (In reply to comment #3) > A thought that keeps brewing in my mind is just to move everything to mail > lists. It's easier to deal with, projects would have a mail list... e.g., > emf-users -1 on the mailing list idea. Mailing lists are just too intrusive, especially, I think, to the "modern" (ie, younger) web consumer crowd. Web forums are ubiquitous and expected in this day and age, IMO. My observation is that these synchronization problems only started after the recent upgrade to FUD Forum. Is that correct? Yes a huge -1 on mail. I get way to much and I'd not be able to follow dozens of groups as I do now. Yes, the big mess seems to be a recent thing. When we had web-based newsgroup posts, things worked fine. How about something similar to Stackoverflow? There are things like Shapado that runs http://ask.debian.net/ that we could potentially use. We could just shut off NNTP because kids these days have no idea what that is anyway. (In reply to comment #7) > How about something similar to Stackoverflow? > > There are things like Shapado that runs http://ask.debian.net/ that we could > potentially use. We could just shut off NNTP because kids these days have no > idea what that is anyway. See Bug 348464 that this one depends on. I'm one of those die-hard NNTP users, as are several other VERY prominent forums contributors. If I had my 'druthers, I'd say investigating the FUDForum bugs would be the first step; only if that is a dead-end or prohibitively expensive would I entertain the idea of shutting down NNTP. I'm not sure most of the folks posting questions are kids. Not only that, I thought kids these days are so sophisticated and intelligent that there are no barriers for them... Let's just say NNTP isn't used by many people these days and acts as a barrier to entry for people wanting to ask questions and is a maintenance issue for the foundation. (In reply to comment #10) > Let's just say NNTP isn't used by many people these days and acts as a barrier > to entry for people wanting to ask questions and is a maintenance issue for the > foundation. NNTP has proper clients, however. One of the main problems I have with the forum software is it totally destroys context and workflow if you try and answer more than one question per day. ex: go to the group of your choice, start reading posts. If you use Prev: and Next: that works fine until your respond to a post. The software immediately puts it at the top of the stack (no more next button). If you try and go up to the threads view so you can find the next post (using breadcrumbs at the top or the "Go" combo at the bottom), it goes back to page one, not the page you started on. Try and go back through your post operations a few time to get to the correct forum thread view page, and that'll make it confused as well. PW > ex: go to the group of your choice, start reading posts. If you use Prev: and
> Next: that works fine until your respond to a post. The software immediately
FWIW, I am very active on non-tech, non-Eclipse related forums that do not use NNTP at all.
On those threads I want to read, and perhaps even post, I 'open in new tab' each thread I want to read and don't switch to the other tabs until I'm done scanning the index.
After that, I go to the tabs, read, answer close as necessary.
(In reply to comment #12) > After that, I go to the tabs, read, answer close as necessary. I used to do that with the older version of the forum software but stopped for some reason (it must have got confused). I'll try again with the newer version. I realize we're constrained by what open source software there is available, but I find the forum software navigation and lost-context to be extremely painful (hence why I wouldn't want to walk away from NNTP). PW Yes, NNTP has proper clients and it has a P which makes automation possible. I would hate to see NNTP go because it severely limits integration of forums. I really like NNTP. I like it so much that I use http://gmane.org/ to read *any* mailing list via NNTP. If one is missing I usually submit a subscription requests and they'll add it. I understand that maintaining three different systems (forums, news, mailing lists) for the same purpose (direct communication) actually is overhead with room for improvements. I noticed other communities are using services like http://www.nabble.com/ for providing different communication channels on top of mailing lists. Maybe that's a way to remove overhead from Eclipse webmasters? I use google groups in combination with gmane to provide web, mail, RSS, and NNTP access to the same content. Like Gunnar, I also like gmane and read many mailinglists via NNTP. HI, this is also a major issue with my way of dealing the community messages. Since the forum upgrade, i cannot be sure my replies will be visible aeverywhere , and in the other hand, be sure i will not miss any messages that implies projects i monitor. I used NNTP for years and will not change it for now.. *** Bug 349905 has been marked as a duplicate of this bug. *** I'm bumping up the priority and importance of this, as it definitely has a big impact on the community support and we're quickly approaching a major release. I know webmasters are swamped this time of year, but this must be addressed ASAP, in my opinion. By the way, let's keep discussion about the merits and potential deprecation of NNTP over on Bug Bug 348464 and leave this one to focus on the synchronization problems. Is there some sort of vote going on? Community feedback? If so, I'd certainly like to see NNTP continue. I use that all the time, almost never forums. I've never understood how people can use forums, to be honest (on a frequent, regular basis). But, admit, I find lots of web interfaces clunky, and lacking in personal customizations, filters, etc. I use Thunderbird client ... sounds like I better try gmane? :) Plus ... it seems odd "we" would consider sunsetting the NNTP service that's worked for years ... because of bugs in the forum software? If I'm reading between the lines, that seems to be what's being said. But, I'll admit, I'm not sure. In fact, I am surprised there's not more said from Eclipse IT (webmasters) ... I must be missing something. Is there a plan to fix this? Is it seriously being proposed just to drop NNTP simply because of synchronization problems? As always, I'm more sympathetic than I may sound. If there's justifiable business reasons to do one thing or another, then that trumps my personal preference ... but, I know for me personally, it'd take some effort and time to learn how to use the web-based solutions (i.e. costs may be just shifted, not reduced). [Oh, thanks Eric, right as I was submitting this, saw your addition pointing to the "deprecate nntp" bug. thanks for that.] (oops, I accidentally reset importance/severity that Eric set, so setting back to what he had). (In reply to comment #21) > Plus ... it seems odd "we" would consider sunsetting the NNTP service that's > worked for years ... because of bugs in the forum software? No, we would consider sunsetting NNTP because of the vast decline in posts from nntp clients vs. forum posts. > In fact, I am surprised there's not more said from Eclipse IT > (webmasters) ... I must be missing something. Thee hardware outages in three weeks before Indigo. I guess you did miss something. > Is there a plan to fix this? I've had a quick look, and I could not find the problem. Until I can find the problem, it's hard to plan for a fix. > Is it seriously being proposed just to drop NNTP simply because of > synchronization problems? Yes. I rarely open bugs just to joke around. > If there's justifiable business reasons to do one thing or another > then that trumps my personal preference Well, in my mind, cutting one service that needs to be maintained is always a justifiable business reason. That doesn't mean it's a good one. (In reply to comment #23) > No, we would consider sunsetting NNTP because of the vast decline in posts from > nntp clients vs. forum posts. Is this based on absolute numbers or just the the ratio between forums/nntp? I could image that posts have increased generally, but since forums are more prominently advertised on our page, newcomers go there first. E.g. forums are linked directly on the landing page, though NNTP isn't. Markus (In reply to comment #23) > No, we would consider sunsetting NNTP because of the vast decline in posts from > nntp clients vs. forum posts. Please see bug 348464 comment 7. I'm afraid that if we cut-off NNTP we'll have forums with a lot questions but a vast decline in answers. > Is this based on absolute numbers or just the the ratio between forums/nntp? My methodology is discussed on bug 348464 comment 0. > but since forums are more > prominently advertised on our page, newcomers go there first. The gateway to both forums and NNTP is www.eclipse.org/forums. That landing page features a link to each web forum, and an [nntp] link as well, so I don't buy the argument. Of course, linking to a news:// URL is pointless since a) most browsers do not support it directly without a helper app and b) our NNTP sever is password-protected, which would cause newcomers even more confusion. > Please see bug 348464 comment 7. I'm afraid that if we cut-off NNTP we'll have
> forums with a lot questions but a vast decline in answers.
Yes, I've seen that... hence the reason the bug is a question. Since the answer appears to be 'no' I suggest we close that bug as WONTFIX.
(In reply to comment #27) > Yes, I've seen that... hence the reason the bug is a question. Since the > answer appears to be 'no' I suggest we close that bug as WONTFIX. Well there is still the option to NOT deprecate NNTP but to let an external service host it. (In reply to comment #26) > > Is this based on absolute numbers or just the the ratio between forums/nntp? > > My methodology is discussed on bug 348464 comment 0. It only shows the ratio between forums vs. NNTP. But has NNTP usage died down in absolute numbers over the years? > The gateway to both forums and NNTP is www.eclipse.org/forums. That landing > page features a link to each web forum, and an [nntp] link as well, so I don't > buy the argument. Of course, linking to a news:// URL is pointless since a) > most browsers do not support it directly without a helper app and b) our NNTP > sever is password-protected, which would cause newcomers even more confusion. By landing page I mean eclipse.org/ which linkes "Forums" in the toolbar. Same for the forums page itself, that has "Forums" everywhere. And then you need a password to access NNTP (even for read only access). Might this be a reason newcomers choose a web based front end? People need to be aware that their questions are likely to be overlooked by the folks who answer them and that they're likely not to see the answers that are being posted. That's obviously totally bad news made worse by there being a new release, so I'm bumping this back to what Eric did earlier. (In reply to comment #30) > People need to be aware that their questions are likely to be overlooked by the > folks who answer them and that they're likely not to see the answers that are > being posted. That's obviously totally bad news made worse by there being a > new release, so I'm bumping this back to what Eric did earlier. +1. www.eclipse.org was unable to connect to the NNTP server as a result of the routing issues we'd been having, and was failing silently. I've fixed the routing issue, and they should now be able to connect. NNTP posts will be gradually imported into the Forum, and new forum posts will (should?) make their way into NNTP immediately. On eclipse.test I've posted several messages/replies using both the forums and NNTP, and it all worked. If you can, please keep your eyes opened to see if these shenanigans persist. Just a reminder -- forum posts are sent to NNTP immediately, but NNTP posts are pulled in batches every couple of minutes. Are you saying the backlog of missing things should correct itself now? That doesn't appear to be the case, at least for EMF. I.e., if I look the EMF newsgroup/forum at http://www.eclipse.org/forums/index.php/f/108/ I don't see that yesterday's answer to "Translatable EAttribute default values" from NNTP has shown up in the forum and none of the things in the forum that never showed up in NNTP have shown up in NNTP yet. E.g., "Best Practice for building and pasting data structures". NNTP answers from today haven't shown up either, e.g., "XSD2ECORE Many "Types" generated" in the forum should see an answer. Maybe it will show up but it's been 15 minutes; how long should it take? > Are you saying the backlog of missing things should correct itself now?
The backlog of unimported NNTP posts should all be done. The forum maintains a pointer to which article it's up to in the NNTP newsgroup; I have a feeling that the pointer may have changed or became stale after days/weeks of silent failure.
From here, one thing that can be done is to delete the most badly corrupted forums and completely reimport them from NNTP. Is this something you'd like to see done to EMF?
Actually, the EMF forum is not updating at all, since there's a SQL error when importing it. I'll backtrace and try to see what the problem is. Ed, about 2 months' worth of EMF nntp articles have imported into the EMF forum. Can you tell me if this is what was expected? The contents of the forum look quite different now. I'll monitor in the next days to see if things appear to be syncing properly now. Thanks so much for looking into it immediately!! If you think re-importing would improve the threading structure, that would be a good thing to try, yes. Created attachment 198885 [details]
More sync problems
There is obviously (still?) an update problem between forum and NNTP. The text of the forum (right side) notes an update, NNTP didn't see this. The thread was started after the updates mentioned by Denis.
There are lots of issues in the eclipse.modeling.tmf forum. Almost everyday there is a handful of posts where threading is broken, and it is made worse when people do not set the subject (i.e. "Re: (no subject)" pops up here and there with comments like "hi, no that does not work" :) @Ed: Did reimporting help for the EMF newsgroup? The problem still persists and it is severe. Today I answered several newsgroup questions just because I did not see that someone else already answered them in the same thread. Essentially the concept of a thread does not exist anymore. Is here any progress being made? Can we just restore the state when newsgroups worked, the state we used to have for many years? Eike, There may be a specific problem with your newsgroup. If you can remind me which one it is, I can delete it and re-create it from NNTP. I participate on the newcomers forum AND newsgroup quite frequently, and although there is the occasional threading glitch, for the most part it works just fine. FWIW, I do acknowledge that there are occasional threading glitches. There's not much I can do -- I didn't write this software, there are no viable alternatives out there and both systems are expected to coexist. Created attachment 200549 [details] Broken NNTP threads The problem is NNTP itself. As you can see in the screen capture, the "EMF - Is it the future?" thread is completely fragmented. Didn't you have a theory about that? https://bugs.eclipse.org/bugs/show_bug.cgi?id=347991 Could we disable that function? Is it a known issue that's perhaps addressed in the latest version? Or needs to be reported so it will be addressed? Also note that lately sikriti seems to be the name for most people who post. What's with that? It seems a new problem. The webview shows Stacey correctly. Created attachment 200581 [details] screenshot of missing name in reply (In reply to comment #45) > Also note that lately sikriti seems to be the name for most people who post. > What's with that? It seems a new problem. The webview shows Stacey correctly. Yes, i've seen that too. i only use NNTP to reply to people, and my name is never shown in the forums ( see attached screenshot ) I am registered with the same email in both NNTP and forums Created attachment 200601 [details] Screenshot of proper names > Didn't you have a theory about that? > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=347991 I tested it real quickly using eclipse.test and the quick reply threaded correctly. I should close that bug. > Also note that lately sikriti seems to be the name for most people who post. > What's with that? It seems a new problem. I must be special. I see the names correctly in NNTP using thunderbird. /scratches head Hi, I notice that my replies to forum postings in the newsgroup get a (no subject) header. http://www.eclipse.org/forums/index.php/t/225447/ The Subject is set correctly. Regards, Wim (In reply to comment #48) This incorrect subject is my own mistake. (In reply to comment #49) > (In reply to comment #48) > > This incorrect subject is my own mistake. Well it was but many other newsreaders also seem to suffer from this. I have filed a new bug for this: https://bugs.eclipse.org/bugs/show_bug.cgi?id=354705 Created attachment 201794 [details]
Mozilla's Bugzilla newsgroup
Attached is a screenshot of the Bugzilla support NG at mozilla. They have a similar NNTP<->Forums bridge, and you can see they also have the occasional threading issue.
Created attachment 201795 [details]
Eclipse Newcomers group featuring posts from 2008
I thought this was cool... Some user must have found threads from 2008 and 2008 by using a web search and bumped them back to new topics. This is unheard of using NNTP only. The threading of these ancient threads is perfect, even in the Forum view.
Created attachment 201796 [details]
EMF newsgroup
This is a snapshot of the EMF newsgroup. It looks quite clean.
I think the dust has settled and we're in good shape. The forum upgrade, followed by some hardware failures led to some odd behaviour in the forums. Since the underlying issues have been fixed, I'm lowering the severity of this. Will the threading always be perfect? No. Does the forum occasionally double-post a message? Sure. But for the most part, I think it works nicely. I will use this bug to investigate why some NNTP posts have a No Subject title. The forum again is missing posts that were sent via NNTP. On this thread [1] in the RAP forum, the most recent post is from Sept 30, whereas the most recent NNTP post is from Oct 3rd. [1] http://www.eclipse.org/forums/index.php/mv/tree/204266/ > The forum again is missing posts that were sent via NNTP. > On this thread [1] in the RAP forum, the most recent post is from Sept 30, > whereas the most recent NNTP post is from Oct 3rd. > > [1] http://www.eclipse.org/forums/index.php/mv/tree/204266/ The forum got confused and created a new thread at some point. This new thread contains all the messages. http://www.eclipse.org/forums/index.php/t/243430/ There must be a bug in the NNTP code. I'll create a new forum for myself, link it to eclipse.technology.rap and run through the import process with some debugging to see where it failed to detect the reply code. > The forum got confused and created a new thread at some point.
I think I've found the issue. While importing into a test forum, the proper threading of that specific thread was preserved. After looking at the code, it appears a database call is issued to determine if the message being imported is a reply to something else. If our database slave is lagged even by a few seconds, the database call will return null and the importer will start a new thread.
I've forced the database call to use the master instead when determining if a NNTP message is a reply.
(In reply to comment #57) > > The forum got confused and created a new thread at some point. > [ ... ] > I've forced the database call to use the master instead when determining if a > NNTP message is a reply. Thanks for the quick solution :) *** Bug 354705 has been marked as a duplicate of this bug. *** Another thing I'll set up is a mechanism to detect if the slave is lagged (or off for some reason). We had this before and it worked well, and it will also benefit Bugzilla and Wiki. > Another thing I'll set up is a mechanism to detect if the slave is lagged (or
> off for some reason).
I just finished putting that in place. That should go a long way in helping the forum stay in line with the newsgroups.
I've resolved the NNTP sync issues here as part of bug 376695. |