Re: Broken threading [was: Unbelievable.]

Re: Broken threading [was: Unbelievable.]

[Thomas Yan]

Tom Hardy <rhardy@mailandnews.com> writes:
> 
> Thomas Yan <tyan@twcny.rr.com> writes:
> 
> > Christopher K Davis <ckd-post@ckdhr.com> writes:
> > 
> > > Tom Hardy <rhardy@mailandnews.com> writes:
> > > 
> > > [Gnus hates]
> > > >> <n4ooiuov4co <1ffgkur.fqzexq13c0u9gN%marykay@kare.ws>
> > > 
> > > > It's a known problem with with Gnus, fixed with development
> > > > versions beyond the 5.8.* series.  I know because I reported it
> > > > myself (but I don't keep up Gnus development).
> > >
> > > Backported patch against 5.8.8 follows.  
> 
> Heh, I just noticed that it's not fixed as of 5.9.0.  I guess I'll be
> patching too.  One other thing--I started a followup to Mary Kay's
> post (Message-ID quoted above), and get 23 references.  Wouldn't it be
> a good idea to trim the references down to size at that point?  I can
> do it by hand, but that sort of thing can be done improperly, and
> mostly doesn't get done.

That would probably be good to have as an option.  I forget the recipe
I've seen in the past.  Something like: Keep the first (oldest)
reference and the last (newest) 3 references.

> > > (Note that recompiling the Gnus elc files can be tricky.  What
> > > worked for me: patch, load gnus-util.el, touch gnus-sum.el,
> > > byte-recompile-directory)
> 
> Stupid question time--how do you apply a patch?

Here's how to apply this one.

0. Find the gnus elisp files, maybe somehwere like
   /usr/local/share/emacs/lisp/gnus.

1. Copy the patch into a file, the "patchfile".
   At the command line, use the "patch" command to apply the changes
   specified in the patchfile to gnus-util.el.
OR
   Open gnus-util.el in emacs and apply the changes by hand.
   (See below.) This is not too onerous: there are only a few changes.
      
2. In emacs, use M-x load-file to load gnus-util.el.

3. At the command line, use the "touch" command to change the
   modification date of gnus-sum.el.  (Or open the file in emacs
   and make and save a cosmetic change, e.g. add an extra line at
   the end.)

4. In emacs, use M-x byte-recompile-directory to recompile the
   elisp files in the gnus elisp directory.  (If you run this
   command in a buffer with one of the gnus elisp files, 
   e.g. gnus-util.el, then emacs will prompt you with the correct
   directory as the default choice.)

-snip-

> > The next annoyance I want to deal with is to have Gnus look at both
> > References: and In-Reply-To: lines so that it does a better job of
> > threading mailing lists.
> 
> Are you submitting your changes back to the developers list?

Not so far.  I'm using the fink port of Emacs 20.7.1 to Mac OS X,
running something that calls itself Gnus 5.7, which is confusing since
the manual at www.gnus.org seemed to say it was not publically
released.

Anyway, my changes have been to try to duplicate some of the features
listed in that manual or have been quick and dirty minor kludges, so I
don't think they warrant submitting.

I guess I should try and see sometime "soon" if upgrading to 5.8.8 or
a later version poses any problems, or at least look at the code to
see if my In-Reply-To problem has already been solved.  If not, and I
solve it, *that* may be worth submitting.

That, plus temporary scorefile entries with an expiration date instead
of temporary entries that expire if they haven't been used "recently".

> Here's the patch in question once again:

Here's an explanation of how to apply
 
> --- gnus-util.el.orig	Wed May 15 19:03:45 2002
> +++ gnus-util.el	Thu Jul 18 22:54:21 2002

These name a file containing the old version and
[the new modified file] / [your file to be modified].

> @@ -379,7 +379,7 @@
>    "Return a list of Message-IDs in REFERENCES."
>    (let ((beg 0)
>  	ids)

These provide the context of a modification: line numbers, and some
surrounding, unchanged text that occurs just before the change.

> -    (while (string-match "<[^>]+>" references beg)

The original text to be changed, with "-" stuff in front of each line.

> +    (while (string-match "<[^<]+[^< \t]" references beg)

The replacement text, with "+" stuffed in front of each line.  If your
are applying the changes manually, remember to delete the +'s!

>        (push (substring references (match-beginning 0) (setq beg (match-end 0)))
>  	    ids))
>      (nreverse ids)))

More context: unchanged text that occurs just after the change.

Re: Broken threading [was: Unbelievable.]

[J.B. Moreno]

In article <m37kjo7i2i.fsf@jeeves.decathlon>, 
 Tom Hardy <rhardy@mailandnews.com> wrote:

> "J.B. Moreno" <planb@newsreaders.com> writes:
> 
> >  Thomas Yan <tyan@twcny.rr.com> wrote:
> > 
-snip what to do with References-
> > > Something like: Keep the first (oldest) reference and the last
> > > (newest) 3 references.
> > 
> > That is the minimum GNKSA requirement
-snip-
> > Both the GNKSA and USEFOR recommend that trimming be done by removing
> > the 2nd to oldest ID until you've reached whatever limit you are using.
> 
> Here it differed.  Gnus removed the 6th reference, not the 2nd.

Compliant with a previous draft from USEFOR -- it was thought by some
that it would help with threading (it doesn't).

> I had thought to remove more in the hope of avoiding the truncation
> problem with older newsreaders.  Bad idea?

Not necessarily -- if you comply with the minimum GNKSA requirement,
then threading will work in the vast majority of cases, as will a good
percentage of the other things done to/with the References header.

Of course, the more you leave in the more cases that it will work with,
and in *this* particular case I don't think older software was the
problem -- I think that Dorothy tried to trim it by hand and messed up.

Also note that there is no sure fire way of avoiding the truncation --
a previous version of Mac OE had a problem with long id's (for instance
those from pine) and corrupted the immediate precursor.

My recommendation is to stick to not causing any problems yourself and
removing any known problems, and leave it at that (letting Gnus handle
the rest).

-- 
J.B. Moreno

Re: Broken threading [was: Unbelievable.]

[Tom Hardy]

In article <240720022348365027%planb@newsreaders.com>, 
"J.B. Moreno" <planb@newsreaders.com> writes:

> In article <m3fzy99j2f.fsf@defun.localdomain>,
>  Jesper Harder <harder@myrealbox.com> wrote:

>> Thomas Yan <tyan@twcny.rr.com> writes:
>> 
> -snip-
>> > You don't have a problem with some mail programs that see a
>> References > header, but don't know what it is and therefore stick
>> it in the reply?
>> 
>> Ah, I see.  I haven't encountered that problem.
>> 
>> So what you probably want is to check if the message-id in
>> In-Reply-To is equal to the last message-id in References -- and if
>> it isn't then use the one from In-Reply-To instead.

> That might work -- then again it might not.  I once did some local
> gatewaying in order to read a mailing list with a threaded view, and
> the only thing that was reliable was that programs repeated their
> mistakes.  You are describing a "generic" method, which did indeed
> work most of the time, but I had to have a whole bunch of special
> cases (it wouldn't surprise me to learn that some of them were
> developed by throwing darts to see what they should do),

I thought I would look at cases just for fun.  Tell me if I am all
wet or otherwise wasting my time.

   I.  Mailer updates References and In-Reply-To, in which case they
       both work, but references are more versatile.  In this case,
       In-Reply-To will be identical to the last reference.

  II.  Mailer does neither.  Anything goes, but it is likely to look
       like the first case.

 III.  Mailer updates In-Reply-To, ignores References.  In-Reply-To
       will not be found in References.

  IV.  Mailer updates References, ignores In-Reply-To.  In-Reply-To
       will be found in References, but it won't be the *last*
       reference.  (Does this happen?)

   V.  Other cases, such as one or the other header (or both) is
       missing.

In case V, you use what is available, including Subject, if
necessary.  You add/update References if posting, both headers if
mailing.  (Done, presumably)

In cases I-IV, you have both headers.  Check if In-Reply-To is found
anywhere in References.  If so, you can use References.  If
In-Reply-To != last reference, that only means In-Reply-To was not
updated.

Only in the case where In-Reply-To is not found anywhere in References
do you use In-Reply-To.  This is case III.  When posting, you could
slip In-Reply-To into References ahead of the Message-ID.

I suppose there is the case where In-Reply-To is ignored by dozens of
newsreaders and yet References get trimmed to the point where
In-Reply-To is no longer present.  Or a newsreader like AOL gives you
one reference, yet passes along an old In-Reply-To.  (Don't know.)

So I suppose I could modify the rule: If In-Reply-To is not found
anywhere in References and there are at least four (?) References, use
In-Reply-To.

Does that make sense?  Am I wasting my time? (Separate question, as I
may be going over old ground.)

-- 
    Tom Hardy   <*>
rhardy@mailandnews.com
   rhardy@visi.com

Re: Broken threading [was: Unbelievable.]

[J.B. Moreno]

In article <m3vg6v8j0u.fsf@jeeves.decathlon>, 
 Tom Hardy <rhardy@mailandnews.com> wrote:

Sorry for the late reply, was offline for a week and then didn't read
the group for a week..

> "J.B. Moreno" <planb@newsreaders.com> writes:

> > That might work -- then again it might not.  I once did some local
> > gatewaying in order to read a mailing list with a threaded view, and
> > the only thing that was reliable was that programs repeated their
> > mistakes.  You are describing a "generic" method, which did indeed
> > work most of the time, but I had to have a whole bunch of special
> > cases (it wouldn't surprise me to learn that some of them were
> > developed by throwing darts to see what they should do),
> 
> I thought I would look at cases just for fun.  Tell me if I am all
> wet or otherwise wasting my time.
> 
>    I.  Mailer updates References and In-Reply-To, in which case they
>        both work, but references are more versatile.  In this case,
>        In-Reply-To will be identical to the last reference.
> 
>   II.  Mailer does neither.  Anything goes, but it is likely to look
>        like the first case.
> 
>  III.  Mailer updates In-Reply-To, ignores References.  In-Reply-To
>        will not be found in References.
> 
>   IV.  Mailer updates References, ignores In-Reply-To.  In-Reply-To
>        will be found in References, but it won't be the *last*
>        reference.  (Does this happen?)

Yes, don't ask me why.

>    V.  Other cases, such as one or the other header (or both) is
>        missing.

Skipping the fact that V covers everything, I-IV is pretty much what
you'd expect and is pretty much what *most* mailers do -- but it's by
no means certain.

One of the special cases that I had to keep an eye out for was where
In-Reply-To was accurate, but References wasn't (I remember two
variation on that, one where the precursor was simply left off, and one
where it was inserted as the 2nd id in the References header, which
required an especially stupid programmer IMO).  Basically anything can
go wrong.

And then there's stupid users who for some reason reply to a message
and will manually cut-n-paste so that it appears to be a reply to
another message (and no, I don't mean where they are replying to two
messages at once), which you have to watch out for because you don't
want to mistake it for something else and try to fix it based upon the
client.

-- 
J.B. Moreno