Read certain forums with Gnus (Canopus)

Read certain forums with Gnus (Canopus)

[Harry Putnam]

Can anyone here tell me if it is possible to read forums at
www.canopus.com using gnus somehow?

Like here:
   http://forum.canopus.com/ubbthreads.php

The setup in these forums and others I've browsed, is so clunky since
it is web based, that it makes doing serious reading very time
consuming.  Something like saving snippets of info isn't even
practicle since it is so cumbersome to do.

Even if it involved using something like wget to grab it all first, it
might be worth doing if gnus could be brought to bear somehow.

Re: Read certain forums with Gnus (Canopus)

[Jonas Steverud]

Harry Putnam <reader@newsguy.com> writes:

> The setup in these forums and others I've browsed, is so clunky since
> it is web based, that it makes doing serious reading very time
> consuming.  Something like saving snippets of info isn't even
> practicle since it is so cumbersome to do.

<rant>In my not very humble opinion are the web based forums a step
ten to fifteen year back in time. The functionality of them are so
limited that the sheer frustration of even following the most
interesting thread is so overwhelming that I have to have a lie down
and curse the developers ancestors down to and beyond the Stone
Age. No threading (a "thread" is the same as a "all articles with the
same subject sorted in chronological order"). Either all articles are
marked as read when you log off or you have to "read" a "thread" to
mark it read - there is no "mark this as read" which makes it very
hard to follow interesting discussions. And so on. I don't think
USENET was this bad even in the beginning.</rant>

> Even if it involved using something like wget to grab it all first, it
> might be worth doing if gnus could be brought to bear somehow.

I think it is possible using W3 and the nnweb interface - I have not
looked into neither but the forums are web based and not very
obfuscated. The problem is that the design of the forum differ from
place to place so if there are three forums forum.x.com,
www.y.com/forums and foobar.z.com/chat, there is a risk we need to
make nnforumsX, nnforumsY, and nnforumsZ. Even of all use UBBTHreads
or phpthread (or whatever the other larger project is called).

The lack of proper threads makes it hard as well - we probably need to
make some cleaver trick to give each article an article number.

But I heart fully agree, a backend for Gnus for this would be a gift
from the gods. I don't know how much I could help since I got to many
hobbies and projects myself already - but all papers are in due order,
so it would be possible for me to lend a hand.

On possibility would be to work with the forum system developers to
actually have a "Gnus plugin" - i.e. have a plain version of the
threads as well, maybe even a XML file. That way other programs might
be developed to interface the forum systems. E.g. if forum.x.com/ is a
forum system with a number of "subgroups" a, b, c and d
(forum.x.com/a, forum.x.com/b, ...) then one can access this bare bone
human-in-readable file by replacing forum.x.com/a/index.html with
forum.x.com/a/threadindex.xml and then forum.x.com/a/threads.xml or
forum.x.com/a/someinterestingthread.xml . Those XML files will be
something similar to what is pulled from NNTP but formatted in XML. (I
think it will be easier to persuade the maintainers to format it in
XML then to format in - say - "NNTP", if you catch my drift.)

Just a thought from the top of my head.

I think it is time to end this mail before it gets out of hand. :-)

-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )

Re: Read certain forums with Gnus (Canopus)

[Adam Sjøgren]

On Fri, 07 Jan 2005 23:51:03 +0100, Jonas wrote:

> On possibility would be to work with the forum system developers to
> actually have a "Gnus plugin" - i.e. have a plain version of the
> threads as well, maybe even a XML file. That way other programs
> might be developed to interface the forum systems.

A third way is to talk the web-forum-operator into running papercut:
 <http://papercut.org/>

(and, perhaps, adding support to papercut for the forum-software in
question).

Although I suspect that that is more trouble for the operator, than it
would be for the forum-authors to add some sort of XML-based
interface. (XML being all the rage a couple of years back, and all).

Does anyone (ab)use RSS or Atom for something like this?


  Best regards,

-- 
 "Yeah, the revolution starts now                             Adam Sjøgren
  In your own backyard"                                  asjo@koldfront.dk

Re: Read certain forums with Gnus (Canopus)

[Jonas Steverud]

asjo@koldfront.dk (Adam Sjøgren) writes:

[...]
> Although I suspect that that is more trouble for the operator, than it
> would be for the forum-authors to add some sort of XML-based
> interface. (XML being all the rage a couple of years back, and all).

Most definitely! I had some trouble to convince a Swedish forum
operator (a game company) to include a plug in to send mail digests -
I've sent two mails with quite some comments and arguments and links
but all I got is one email telling me he had forwarded my (first)
email to the sysadmin.

If the extension is included in the forum system, it will be
automatically included in the next upgrade of the system and in all
new systems. I think we need to be a bit sneaky if we want to use
that approach. But I think it is wishful thinking at the moment -
lets concentrate on how to parse the existing web pages IMHO.

-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )

Re: Read certain forums with Gnus (Canopus)

[David Edmondson]

* reader@newsguy.com [2005-01-07 20:26:49]
> Can anyone here tell me if it is possible to read forums at
> www.canopus.com using gnus somehow?
> 
> Like here:
>    http://forum.canopus.com/ubbthreads.php

nnultimate is intended for this, I believe, though I've never used it.

dme.

Re: Read certain forums with Gnus (Canopus)

[Michael Widerkrantz]

Jonas Steverud <tvrud@bredband.net> writes:

> I think it is possible using W3 and the nnweb interface - I have not
> looked into neither but the forums are web based and not very
> obfuscated. The problem is that the design of the forum differ from
> place to place so if there are three forums forum.x.com,
> www.y.com/forums and foobar.z.com/chat, there is a risk we need to
> make nnforumsX, nnforumsY, and nnforumsZ. Even of all use UBBTHreads
> or phpthread (or whatever the other larger project is called).

In general, I agree with you about the horridness of web forums, but
there are better ways of getting at them then web scraping: Many of
these forums are based on some database. If you can get permission to
access the database in a more direct way, you can throw together an
NNTP gateway.

There's a pretty nice framework known as Papercut to help you do
this. It's written in Python and fairly easy to work with. I once did
a plugin to it to access Gnuheter, the Swedish free software news
site. My plugin worked last I tested it, but that, I'm afraid was some
time ago. What I did is here:

  http://hack.org/mc/hacks/papercut-gnuheter-1.1.tar.gz

-- 
M.C. Widerkrantz, http://hack.org/mc/
WAR IS PEACE. FREEDOM IS SLAVERY. ERASE IS BACKSPACE.