broken Eudora 4.1 problems

broken Eudora 4.1 problems

[Wes Hardaker]

With the recent onslaught of mail sent by users of the new Eudora 4.1, 
I've noticed it's broken in two painful ways:

1) The In-Reply-To: header contains the reference that the mailer is
   replying to.  But, the references header doesn't contain the
   parent, only the grandparent and above.  Hence, the
   references/in-reply-to headers look like:

     In-Reply-To: ref1
     References:  ref2 ref3 ref4

   Which makes gnus thread it as:

     ref4
       ref3
         ref2
         ref1

   Instead of:
   
     ref4
       ref3
         ref2
           ref1

   sigh...  We should probably watch out for this?  I bet if we
   default to handling it as Eudora generates it, other mailers will
   have both fields but encoded properly and that won't work either?

2) Somehow, I think (I want to sit in front of a Eudora 4.1 and test
   this theory before believing it entirely) that Eudora 4.1 changes
   the MsgID for each copy of a message it sends out.  This means that 
   a message sent To: Lars, ding@gnus.org would be sent twice (as is
   normal), but with different msgIDs rather than the same one.
   Hence, this breaks all semblances of msgID caching (I have 2 in
   place, one in gnus and one in procmail).  This, sadly, is probably
   impossible to fix.  Has anyone else noticed this problem though?

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: broken Eudora 4.1 problems

[Conrad Sauerwald]

Wes Hardaker <wjhardaker@ucdavis.edu> writes:

> With the recent onslaught of mail sent by users of the new Eudora 4.1, 
> I've noticed it's broken in two painful ways:
> 
> 1) The In-Reply-To: header contains the reference that the mailer is
>    replying to.  But, the references header doesn't contain the
>    parent, only the grandparent and above.  Hence, the
>    references/in-reply-to headers look like:

Isn't that the general brokenness when newsreaders meet mailreaders?
I think that's where even the RFCs contradict, but I'll leave that to
the experts.  

It just reminded me of another big pain when talking to people who use
Eudora: it carries over the entire string that Gnus puts in the
in-reply-to (only meant for humans) and treats it like a message-id.
Wouldn't it be lovely if there was a switch for poor morons like me
who really have no problem having a in-reply-to which looks more like
the one in mutt where the human readable part is preceded by the
message-id and a semicolon (which Eudora magically does understand)

Re: broken Eudora 4.1 problems

[Conrad Sauerwald]

Wes Hardaker <wjhardaker@ucdavis.edu> writes:

> With the recent onslaught of mail sent by users of the new Eudora 4.1, 
> I've noticed it's broken in two painful ways:
> 
> 1) The In-Reply-To: header contains the reference that the mailer is
>    replying to.  But, the references header doesn't contain the
>    parent, only the grandparent and above.  Hence, the
>    references/in-reply-to headers look like:

Isn't that the general brokenness when newsreaders meet mailreaders?
I think that's where even the RFCs contradict, but I'll leave that to
the experts.  

It just reminded me of another big pain when talking to people who use
Eudora: it carries over the entire string that Gnus puts in the
in-reply-to (only meant for humans) and treats it like a message-id.
Wouldn't it be lovely if there was a switch for poor morons like me
who really have no problem having a in-reply-to which looks more like
the one in mutt where the human readable part is preceded by the
message-id and a semicolon (which Eudora magically does understand)

Re: broken Eudora 4.1 problems

[Stephen Zander]

>>>>> "Wes" == Wes Hardaker <wjhardaker@ucdavis.edu> writes:
    Wes>    sigh...  We should probably watch out for this?  I bet if
    Wes> we default to handling it as Eudora generates it, other
    Wes> mailers will have both fields but encoded properly and that
    Wes> won't work either?

Yes, I'd noticed this also.  I was thinking of writing a custom
threading defun that check the X-Mailer header but haven't got round
to it.

-- 
Stephen
---
It should be illegal to yell "Y2K" in a crowded economy.  :-) -- Larry Wall

Re: broken Eudora 4.1 problems

[Lars Magne Ingebrigtsen]

Wes Hardaker <wjhardaker@ucdavis.edu> writes:

>    sigh...  We should probably watch out for this?  I bet if we
>    default to handling it as Eudora generates it, other mailers will
>    have both fields but encoded properly and that won't work either?

Yup.  There's nothing that can be done about this.  Eudora 4.1 sounds
like it is broken.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: broken Eudora 4.1 problems

[Lars Magne Ingebrigtsen]

Conrad Sauerwald <conrad@stack.nl> writes:

> Isn't that the general brokenness when newsreaders meet mailreaders?
> I think that's where even the RFCs contradict, but I'll leave that to
> the experts.  

They do kinda contradict.  Quoth RFC822:

     4.6.2.  IN-REPLY-TO

             The contents of this field identify  previous  correspon-
        dence  which this message answers.  Note that if message iden-
        tifiers are used in this  field,  they  must  use  the  msg-id
        specification format.

     4.6.3.  REFERENCES

             The contents of this field identify other  correspondence
        which  this message references.  Note that if message identif-
        iers are used, they must use the msg-id specification format.

So, if we take RFC822 as gospel, then what Eudora 4.1 does is right.
However, nobody has ever implemented this (before now, it seems), and
all mailers I've heard of that do References implement RFC1036
References and if the do In-Reply-To, they do RFC822 In-Reply-To.

DRUMS makes what Eudora does invalid.

> It just reminded me of another big pain when talking to people who use
> Eudora: it carries over the entire string that Gnus puts in the
> in-reply-to (only meant for humans) and treats it like a message-id.

That, however, is undisputably a bug.

> Wouldn't it be lovely if there was a switch for poor morons like me
> who really have no problem having a in-reply-to which looks more like
> the one in mutt where the human readable part is preceded by the
> message-id and a semicolon (which Eudora magically does understand)

No.  There are limits to how much brokeness I have the stomach to
cater to.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: broken Eudora 4.1 problems

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> So, if we take RFC822 as gospel, then what Eudora 4.1 does is right.
> However, nobody has ever implemented this (before now, it seems),

Hasn't Eudora had this behaviour all along?  It has threaded to the
grandparent for as long as I can remember.

> and all mailers I've heard of that do References implement RFC1036
> References and if the do In-Reply-To, they do RFC822 In-Reply-To.

> DRUMS makes what Eudora does invalid.

OK.  Er... me stupid... what's DRUMS...?

[snip!]

> No.  There are limits to how much brokeness I have the stomach to
> cater to.

Would it be possible to detect Eudora behaviour or use the X-Mailer
header and make articles from Eudora be threaded correctly?

Or is that also beyond the "brokenness catering" limit?

Hm... I could actually do this with procmail, I think.  Fix the
article before Gnus is even seeing it...

Re: broken Eudora 4.1 problems

[Steinar Bang]

>>>>> Steinar Bang <sb@metis.no>:

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:
>> So, if we take RFC822 as gospel, then what Eudora 4.1 does is right.
>> However, nobody has ever implemented this (before now, it seems),

> Hasn't Eudora had this behaviour all along?  It has threaded to the
> grandparent for as long as I can remember.

Windows Eudora Light 3.05 has the very same behaviour.

[snip!]
> Would it be possible to detect Eudora behaviour or use the X-Mailer
> header and make articles from Eudora be threaded correctly?

> Or is that also beyond the "brokenness catering" limit?

> Hm... I could actually do this with procmail, I think.  Fix the
> article before Gnus is even seeing it...

Here's something (caveat emptor):

	#
	# "Fix" the broken references line from Eudora mailers, by
	# renaming it
	#
	:0 fhw
	* X-Mailer:.*Eudora
	| sed -e "s/^References:/Broken-Eudora-References:/g"

I'm renaming the useless References field from Eudora mailers, because 
there's a message-id in the in-reply-to header that Gnus can use to
thread the message.

Re: broken Eudora 4.1 problems

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@metis.no> writes:

> OK.  Er... me stupid... what's DRUMS...?

The IETF working group that's, er, working on RFC822bis.

> Would it be possible to detect Eudora behaviour or use the X-Mailer
> header and make articles from Eudora be threaded correctly?
> 
> Or is that also beyond the "brokenness catering" limit?

I think so.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: broken Eudora 4.1 problems

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Steinar Bang <sb@metis.no> writes:
>> OK.  Er... me stupid... what's DRUMS...?

> The IETF working group that's, er, working on RFC822bis.

Ah...

>> Would it be possible to detect Eudora behaviour or use the X-Mailer
>> header and make articles from Eudora be threaded correctly?
>> 
>> Or is that also beyond the "brokenness catering" limit?

> I think so.

No matter.  The procmail filter
	#
	# "Fix" the broken references line from Eudora mailers, by
	# renaming it
	#
	:0 fhw
	* X-Mailer:.*Eudora
	| sed -e "s/^References:/Broken-Eudora-References:/g"
seems to be working.

"^" now works on articles received from Eudora, and they thread
nicely. 

Re: broken Eudora 4.1 problems

[Per Abrahamsen]

Steinar Bang <sb@metis.no> writes:

> Would it be possible to detect Eudora behaviour or use the X-Mailer
> header and make articles from Eudora be threaded correctly?

Yep.  Lars, I think Gnus should include the function below (or
something similar), but not call it.  Let it be up to the users to add
it to the hook.

(defun gnus-fix-eudora-headers ()
  "Eudora has a broken references line, but an OK in-reply-to."
  (goto-char (point-min))
  (when (re-search-forward "^X-Mailer:.*Eudora" nil t)
    (goto-char (point-min))
    (when (re-search-forward "^References:" nil t)
      (beginning-of-line)
      (insert "X-Gnus-Broken-Eudora-"))))

(add-hook 'nnmail-prepare-incoming-header-hook 'gnus-fix-eudora-headers)

Re: broken Eudora 4.1 problems

[Wes Hardaker]

>>>>> On 27 Jan 1999 11:20:48 +0100, Steinar Bang <sb@metis.no> said:
Steinar> Here's something (caveat emptor):

Steinar> #
Steinar> # "Fix" the broken references line from Eudora mailers, by
Steinar> # renaming it

Cool.  Thanks!  (a very simple, but easily working script).

I hate destroying those decent references though...  It'd be tempting 
to take the in-reply-to header and shove its contents in front of the
rest of the references header...  A bit more complex in procmail, but
doable.

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: broken Eudora 4.1 problems

[Wes Hardaker]

>>>>> On 27 Jan 1999 06:03:41 +0100, Lars Magne Ingebrigtsen <larsi@gnus.org> said:

Lars> Yup.  There's nothing that can be done about this.  Eudora 4.1
Lars> sounds like it is broken.

There's *always* something that can be done...

("X-Mailer: Eudora" "/dev/null") comes to mind...

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: broken Eudora 4.1 problems

[Steinar Bang]

>>>>> Wes Hardaker <wjhardaker@ucdavis.edu>:

> I hate destroying those decent references though... 

Agreed.

> It'd be tempting to take the in-reply-to header and shove its
> contents in front of the rest of the references header...  A bit
> more complex in procmail, but doable.

Yup.  That's what I planned on doing, when it struck me that the
simpler approach was more easily doable and less error prone in an
experimentation phase.

Would be easier to do something like this in Per Abrahamsen's elisp
function I think (...if I could just understand the mystery of emacs
lisp function parameter numbering...).

Re: broken Eudora 4.1 problems

[Wes Hardaker]

>>>>> On 27 Jan 1999 17:43:54 +0100, Steinar Bang <sb@metis.no> said:

Steinar> Would be easier to do something like this in Per Abrahamsen's
Steinar> elisp function I think (...if I could just understand the
Steinar> mystery of emacs lisp function parameter numbering...).

yep, but I use procmail to deliver to multiple places (imap folders
and pop->gnus), so its the best place for installing filters: ahead of 
everything else.

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Re: broken Eudora 4.1 problems

[Lars Magne Ingebrigtsen]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Yep.  Lars, I think Gnus should include the function below (or
> something similar), but not call it.  Let it be up to the users to add
> it to the hook.
> 
> (defun gnus-fix-eudora-headers ()

I've now added this to 0.75 under the name
`nnmail-fix-eudora-headers'. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen