Web-based mailreading enhancement

Web-based mailreading enhancement

[Jan Vroonhof]

Given that red gnus is the Web-aware gnus. Would it be very
complitated to enhance nnweb to be able to read those webbases mailing
list archives (e.g. hypermail etc) as real groups?

Jan

P.S. I just remembered that it would also be nice if we could use
another group as a mailgroups spool directory.

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Given that red gnus is the Web-aware gnus. Would it be very
> complitated to enhance nnweb to be able to read those webbases mailing
> list archives (e.g. hypermail etc) as real groups?

That would be *very* nice.  I remember trawling through the mailnews-l
Web archive with (spit) Netscape, and being terribly frustrated and
wishing that I could just read it with Gnus.  However, I don't know
anything about what protocol the listserv Web thingies use.  Could
somebody look into it and let me know how to request an overview of
the articles in the "groups"?

> P.S. I just remembered that it would also be nice if we could use
> another group as a mailgroups spool directory.

I don't quite follow...

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Web-based mailreading enhancement

[Wesley.Hardaker]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> That would be *very* nice.  I remember trawling through the mailnews-l
> Web archive with (spit) Netscape, and being terribly frustrated and
> wishing that I could just read it with Gnus.  However, I don't know
> anything about what protocol the listserv Web thingies use.  Could
> somebody look into it and let me know how to request an overview of
> the articles in the "groups"?

Unfortunately, most mailing lists differ in the web backend they use.
(or does listserv come with a standard?)  The last time I looked for
mail-to-web type software the list was something like 10-20 packages
long...

> > P.S. I just remembered that it would also be nice if we could use
> > another group as a mailgroups spool directory.
> 
> I don't quite follow...

And I thought it was just me.  "Oh well, I'm sure Lars will
understand" was went through my mind...

Wes

Re: Web-based mailreading enhancement

[Steinar Bang]

[snip! on reading web mailing list archives with Gnus]

>>>>> Wesley.Hardaker@sphys.unil.ch:

> Unfortunately, most mailing lists differ in the web backend they
> use.  (or does listserv come with a standard?)  The last time I
> looked for mail-to-web type software the list was something like
> 10-20 packages long...

Well, if we could support the 1 or 2 most used (starting with
hypermail), I think that would get us pretty far.  Most of the
archives I've seen, have been hypermail archives.

There *may* be a problem with missing information in the generated
HTML code, however...


- Steinar

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

Wesley.Hardaker@sphys.unil.ch writes:

> Unfortunately, most mailing lists differ in the web backend they use.
> (or does listserv come with a standard?) 

I think the "listserv" mailing list thingie comes with a standard
list-to-web package.  I have at least seen three separate sites that
had excactly the same layout and commands and stuff.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Web-based mailreading enhancement

[Steinar Bang]

>>>>> Steinar Bang <sb@metis.no>:

> [snip! on reading web mailing list archives with Gnus]
>>>>> Wesley.Hardaker@sphys.unil.ch:

>> Unfortunately, most mailing lists differ in the web backend they
>> use.  (or does listserv come with a standard?)  The last time I
>> looked for mail-to-web type software the list was something like
>> 10-20 packages long...

> Well, if we could support the 1 or 2 most used (starting with
> hypermail), I think that would get us pretty far.  Most of the
> archives I've seen, have been hypermail archives.

> There *may* be a problem with missing information in the generated
> HTML code, however...

I looked at some stuff generated by hypermail.  Much of this auxillary
information can be found in SGML/HTML comments at the start.  Head
(slightly edited to protect the innocent) of a typical hypermail file:

<!-- received="Fri Oct 11 14:42:52 1996 MET DST" -->
<!-- sent="Fri, 11 Oct 1996 14:42:36 +0200" -->
<!-- name="Steinar Bang" -->
<!-- email="sb@metis.no" -->
<!-- subject="Something testing" -->
<!-- id="199610111242.NAA20690@client.metis.no" -->
<!-- inreplyto="" -->
<title>Somestupidarchivenamehere: Something testing</title>
<h1>Something testing</h1>
Steinar Bang (<i>sb@metis.no</i>)<br>
<i>Fri, 11 Oct 1996 14:42:36 +0200</i>
<p>
<ul>
<li> <b>Messages sorted by:</b> <a href="date.html#290">[ date ]</a><a href="index.html#290">[ thread ]</a><a href="subject.html#290">[ subject ]</a><a href="author.html#290">[ author ]</a>
<!-- next="start" -->
<li> <b>Previous message:</b> <a href="0289.html">Hogne Tjemsland: "Re: C++ file extensions"</a>
<!-- nextthread="start" -->
</ul>
<!-- body="start" -->

The rest is HTML'ized code of the original message text body.  Message
paragraphs are turned into HTML paragraphs.  Each line ends with a
<br> element to preserve line length.

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@metis.no> writes:

> I looked at some stuff generated by hypermail.  Much of this auxillary
> information can be found in SGML/HTML comments at the start.  Head
> (slightly edited to protect the innocent) of a typical hypermail file:
> 
> <!-- received="Fri Oct 11 14:42:52 1996 MET DST" -->
> <!-- sent="Fri, 11 Oct 1996 14:42:36 +0200" -->
> <!-- name="Steinar Bang" -->
> <!-- email="sb@metis.no" -->
> <!-- subject="Something testing" -->
> <!-- id="199610111242.NAA20690@client.metis.no" -->
> <!-- inreplyto="" -->

Well, if I want "headers" from all articles from
Somestupidarchivenamehere, can Hypermail give me that?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Web-based mailreading enhancement

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@ifi.uio.no>:

> Steinar Bang <sb@metis.no> writes:
>> I looked at some stuff generated by hypermail.  Much of this auxillary
>> information can be found in SGML/HTML comments at the start.  Head
>> (slightly edited to protect the innocent) of a typical hypermail file:
>> 
>> <!-- received="Fri Oct 11 14:42:52 1996 MET DST" -->
>> <!-- sent="Fri, 11 Oct 1996 14:42:36 +0200" -->
>> <!-- name="Steinar Bang" -->
>> <!-- email="sb@metis.no" -->
>> <!-- subject="Something testing" -->
>> <!-- id="199610111242.NAA20690@client.metis.no" -->
>> <!-- inreplyto="" -->

> Well, if I want "headers" from all articles from
> Somestupidarchivenamehere, can Hypermail give me that?

Depends on how detailed info you want.  I think this is the closest
you get (date.html from the archive, reverse sorted (ie. newest at the
top)).


<title>Somestupidarchivenamehere by date</title>
<a name="start"><h1>Somestupidarchivenamehere by date</h1></a>
<ul>
<li> <b>Messages sorted by:</b> <a href="index.html#start">[ thread ]</a><a href="subject.html#start">[ subject ]</a><a href="author.html#start">[ author ]</a>
<li> <b><a href="..">Other mail archives</a></b>
</b>
</ul>
<p>
<b>Starting:</b> <i>Wed 18 Oct 1995 - 15:39:47 MET DST</i><br>
<b>Ending:</b> <i>Fri 11 Oct 1996 - 00:00:-33428 MET DST</i><br>
<b>Messages:</b> 291
<p>
<ul>
<li> <a href="0290.html"><b>Something testing</b></a> <a name="290"><i>Steinar Bang</i></a>
<li> <a href="0289.html"><b>Re: C++ file extensions</b></a> <a name="289"><i>Hogne Tjemsland</i></a>
<li> <a href="0288.html"><b>Another silly message</b></a> <a name="288"><i>Trond Kristiansen</i></a>


If you need all the info in the comments heading each message, I see
no other way, except to actually *fetch* that message.

Sounds awfully heavy to me...

Re: Web-based mailreading enhancement

[William Perry]

Lars Magne Ingebrigtsen writes:
>Steinar Bang <sb@metis.no> writes:
>
>> I looked at some stuff generated by hypermail.  Much of this auxillary
>> information can be found in SGML/HTML comments at the start.  Head
>> (slightly edited to protect the innocent) of a typical hypermail file:
>> 
>> <!-- received="Fri Oct 11 14:42:52 1996 MET DST" -->
>> <!-- sent="Fri, 11 Oct 1996 14:42:36 +0200" -->
>> <!-- name="Steinar Bang" -->
>> <!-- email="sb@metis.no" -->
>> <!-- subject="Something testing" -->
>> <!-- id="199610111242.NAA20690@client.metis.no" -->
>> <!-- inreplyto="" -->
>
>Well, if I want "headers" from all articles from
>Somestupidarchivenamehere, can Hypermail give me that?

  I don't think you can.  You might be able to do something heinous with
the information in the big 'summary' view that hypermail can use.  It gives
you the URLs, the subject, and I think the author's 'nice' name (William
Perry instead of 'wmperry@aventail.com').

  I don't think you would want to do this with regexp's on the HTML though.
The 'threading' view is done by using nested lists.  Relatively easy to
handle from the parser (its even iterative! ;) though.

  If you want, I can mail you the output of the relevant chunks of the
parse tree of the oct96 linux kernel mailing list.

-Bill P.

Re: Web-based mailreading enhancement

[Joe Wells]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  Lars> Steinar Bang <sb@metis.no> writes:
  >> I looked at some stuff generated by hypermail.  Much of this auxillary
  >> information can be found in SGML/HTML comments at the start.  Head
  >> (slightly edited to protect the innocent) of a typical hypermail file:
  >> 
  >> <!-- received="Fri Oct 11 14:42:52 1996 MET DST" -->
  >> <!-- sent="Fri, 11 Oct 1996 14:42:36 +0200" -->
  >> <!-- name="Steinar Bang" -->
  >> <!-- email="sb@metis.no" -->
  >> <!-- subject="Something testing" -->
  >> <!-- id="199610111242.NAA20690@client.metis.no" -->
  >> <!-- inreplyto="" -->

  Lars> Well, if I want "headers" from all articles from
  Lars> Somestupidarchivenamehere, can Hypermail give me that?

You could fetch all of the "articles" to get the "header" information from
the comments placed at the head of each article.  It would be kind of
slow.  One problem is that you won't be able to use the W3 parser to get
the information, since it will discard the comments.

-- 
Joe Wells <jbw@cs.bu.edu>

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@metis.no> writes:

> Depends on how detailed info you want.  I think this is the closest
> you get (date.html from the archive, reverse sorted (ie. newest at the
> top)).

Do you have the URL of a likely Hypermail archive so that I can play
with it some?

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Web-based mailreading enhancement

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@ifi.uio.no>:

> Steinar Bang <sb@metis.no> writes:
>> Depends on how detailed info you want.  I think this is the closest
>> you get (date.html from the archive, reverse sorted (ie. newest at the
>> top)).

> Do you have the URL of a likely Hypermail archive so that I can play
> with it some?

The only one I had handy, was the archive for the now discontinued
DSSSL Lite mailing list.
	<URL:http://www.falch.no/people/pepper/DSSSL-Lite/archives/>
(their connection used to be sort of thin, so if you plan to access it
a lot, it may be an idea to do it through a caching proxy.  If you
don't have one near you, you could try proxy.eunet.no port 8080).

This archive was created back in November of 1994.  I don't know if
hypermail has seen some upgrades since then.


- Steinar

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

William Perry <wmperry@aventail.com> writes:

>   I don't think you can.  You might be able to do something heinous with
> the information in the big 'summary' view that hypermail can use.  It gives
> you the URLs, the subject, and I think the author's 'nice' name (William
> Perry instead of 'wmperry@aventail.com').

That's not overly much, but it would allow generating kinda nice
summary buffers.

>   I don't think you would want to do this with regexp's on the HTML though.
> The 'threading' view is done by using nested lists.  Relatively easy to
> handle from the parser (its even iterative! ;) though.

I should probably junk the regexp'y HTML "parsing" done now and rely
on w3 instead.  That would be much more resist{a,e}nt to changes in output
from the search engines and would be nicer all over.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Web-based mailreading enhancement

[Raja R Harinath]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
[snip]
> Do you have the URL of a likely Hypermail archive so that I can play
> with it some?

I was looking for mail-to-web converters recently.  So, I have some links:

The Hypermail software can be found at:

  <URL:http://www.eit.com/software/hypermail/hypermail.html>

If you want a sample of how it works, you should probably check out

  <URL:http://homer.ncm.com/>

which maintains a gateway for various linux-related mailinglists.

I found the following sites:

  <URL:http://www.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/HTML_Converters/>
  <URL:http://union.ncsa.uiuc.edu/HyperNews/get/www/html/converters.html>

which had a some more links to mail-to-web converters, but Hypermail
seems to be the most popular.

- Hari

-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu
"When all else fails, read the instructions."      -- Cahn's Axiom
"Our policy is, when in doubt, do the right thing."   -- Roy L Ash

Re: Web-based mailreading enhancement

[William Perry]

Lars Magne Ingebrigtsen writes:
>Steinar Bang <sb@metis.no> writes:
>
>> Depends on how detailed info you want.  I think this is the closest
>> you get (date.html from the archive, reverse sorted (ie. newest at the
>> top)).
>
>Do you have the URL of a likely Hypermail archive so that I can play
>with it some?

http://www.uwsg.indiana.edu/hypermail/linux/kernel/9610/index.html

Bill P.

Re: Web-based mailreading enhancement

[Steven L Baur]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

Lars> Do you have the URL of a likely Hypermail archive so that I can
Lars> play with it some?

How about the Hypermail archives of this mailing list?
	http://www.miranova.com/gnus-list/

What's extremely obnoxious about converting hypermail into a Gnus
group is that the input to hypermail is a Unix mailbox ...

I break the archive into months and save (compressed right now) the
input mailbox for the month.  Could you do something clever if I made
the full mailboxes available?

The size of September 1996 is 723,350 uncompressed, 161,324 compressed.
-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be billed at $250/message.
What are the last two letters of "doesn't" and "can't"?
Coincidence?  I think not.

Using groups as spool

[Jan Vroonhof]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> > P.S. I just remembered that it would also be nice if we could use
> > another group as a mailgroups spool directory.
> 
> I don't quite follow...

Let me clarify: Now the back-ends go getch their new mails from some
spool file. They fetch messages one by one and move them to the
respective group.

Suppose that instead they would take all the new messages in a group A
as the "spool" for B:
i.e. do the following:
foreach new message M in A do
  mark M als read
  copy M to the B as a new message
end

Now this may seem a silly exercise but it is nice if A is say a mail
archive on some slow ftp server and B is a say a local nnfolder group.


Jan

Re: Using groups as spool

[Kai Grossjohann]

>>>>> Jan Vroonhof writes:

  Jan> Suppose that instead they would take all the new messages in a group A
  Jan> as the "spool" for B:
  Jan> i.e. do the following:
  Jan> foreach new message M in A do
  Jan>   mark M als read
  Jan>   copy M to the B as a new message
  Jan> end

Would "B r" (respooling) do the trick?

kai
-- 
Life is hard and then you die.

Re: Web-based mailreading enhancement

[Lars Magne Ingebrigtsen]

Steven L Baur <steve@miranova.com> writes:

> I break the archive into months and save (compressed right now) the
> input mailbox for the month.  Could you do something clever if I made
> the full mailboxes available?

Uhm...  No, not really.  To do anything useful with those, Gnus would
have to load the full mailboxes over the net, which is overkill.
Reading the ding archives as an nndir group would probably be easier. 

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."