feature request: work around b0rked Pegasus Mail headers

feature request: work around b0rked Pegasus Mail headers

[Matthias Andree]

Greetings,

Pegasus Mail (a popular Window$ mail software) has, among others, for a
long time up to and including the current 4.31 version, generated
improper headers on replies; replying to a message with (header #1)

Message-ID: <foo@example.org>
References: <bar@example.org>

It would generate (header #2):

Message-ID: <new@example.org>
References: <bar@example.org>
In-Reply-To: <foo@example.org>

Where the direct reference was missing from References.
This causes Gnus to attach Pegasus Mail replies too early in the threads
(of, for instance, mailing lists).

May I suggest to change Gnus in a way that it checks if the In-Reply-To
ID is present in References: and if it's not, internally treat header #2
as though it had contained References: <bar@example.org> <foo@example.org>
to ensure proper threading?

Thanks,

-- 
Matthias Andree

Re: feature request: work around b0rked Pegasus Mail headers

[Reiner Steib]

On Fri, Jun 16 2006, Matthias Andree wrote:

> Pegasus Mail (a popular Window$ mail software) has, among others, for a
> long time up to and including the current 4.31 version, generated
> improper headers on replies; replying to a message with (header #1)
>
> Message-ID: <foo@example.org>
> References: <bar@example.org>
>
> It would generate (header #2):
>
> Message-ID: <new@example.org>
> References: <bar@example.org>
> In-Reply-To: <foo@example.org>
>
> Where the direct reference was missing from References.
> This causes Gnus to attach Pegasus Mail replies too early in the threads
> (of, for instance, mailing lists).
>
> May I suggest to change Gnus in a way that it checks if the In-Reply-To
> ID is present in References: and if it's not, internally treat header #2
> as though it had contained References: <bar@example.org> <foo@example.org>
> to ensure proper threading?

I think this could be done similar to `nnmail-fix-eudora-headers':

,----[ (info "(gnus)Washing Mail") ]
|     `nnmail-fix-eudora-headers'
|           Eudora produces broken `References' headers, but OK
|           `In-Reply-To' headers.  This function will get rid of the
|           `References' headers.
`----

Would doing the same for Pegasus be sufficient?  Could you please
provide some sample headers (including X-Mailer or User-Agent,
Reply-To and References) for such articles?

Would someone like to work on `nnmail-fix-pegasus-headers'?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: feature request: work around b0rked Pegasus Mail headers

[Reiner Steib]

On Tue, Jun 20 2006, Reiner Steib wrote:

> I think this could be done similar to `nnmail-fix-eudora-headers':
>
> ,----[ (info "(gnus)Washing Mail") ]
> |     `nnmail-fix-eudora-headers'
> |           Eudora produces broken `References' headers, but OK
> |           `In-Reply-To' headers.  This function will get rid of the
> |           `References' headers.
> `----
>
> Would doing the same for Pegasus be sufficient?  Could you please
> provide some sample headers (including X-Mailer or User-Agent,
> Reply-To and References) for such articles?
>
> Would someone like to work on `nnmail-fix-pegasus-headers'?

Done:

,----[ lisp/ChangeLog ]
| 	* nnmail.el (nnmail-broken-references-mailers): New variable.
| 	(nnmail-ignore-broken-references): New function generalizing
| 	nnmail-fix-eudora-headers.
| 	(nnmail-fix-eudora-headers): Now obsolete.
`----

,----[ (info "(gnus)Washing Mail") ]
|     `nnmail-ignore-broken-references'
|           Some mail user agents (e.g. Eudora and Pegasus) produces
|           broken `References' headers, but correct `In-Reply-To'
|           headers.  This function will get rid of the `References'
|           header if the headers contain a line matching the regular
|           expression `nnmail-broken-references-mailers'.
`----

I don't use nnmail-fix-eudora-headers/nnmail-ignore-broken-references
so please test if it works correctly.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: feature request: work around b0rked Pegasus Mail headers

[Matthias Andree]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Tue, Jun 20 2006, Reiner Steib wrote:
>
>> I think this could be done similar to `nnmail-fix-eudora-headers':
>>
>> ,----[ (info "(gnus)Washing Mail") ]
>> |     `nnmail-fix-eudora-headers'
>> |           Eudora produces broken `References' headers, but OK
>> |           `In-Reply-To' headers.  This function will get rid of the
>> |           `References' headers.
>> `----
>>
>> Would doing the same for Pegasus be sufficient?  Could you please
>> provide some sample headers (including X-Mailer or User-Agent,
>> Reply-To and References) for such articles?
>>
>> Would someone like to work on `nnmail-fix-pegasus-headers'?
>
> Done:
>
> ,----[ lisp/ChangeLog ]
> | 	* nnmail.el (nnmail-broken-references-mailers): New variable.
> | 	(nnmail-ignore-broken-references): New function generalizing
> | 	nnmail-fix-eudora-headers.
> | 	(nnmail-fix-eudora-headers): Now obsolete.
> `----
>
> ,----[ (info "(gnus)Washing Mail") ]
> |     `nnmail-ignore-broken-references'
> |           Some mail user agents (e.g. Eudora and Pegasus) produces
> |           broken `References' headers, but correct `In-Reply-To'
> |           headers.  This function will get rid of the `References'
> |           header if the headers contain a line matching the regular
> |           expression `nnmail-broken-references-mailers'.
> `----
>
> I don't use nnmail-fix-eudora-headers/nnmail-ignore-broken-references
> so please test if it works correctly.

Hm - this appears to be limited in effect.

If I understand this correctly, it is laundering incoming messages that
are being moved via the mailsource to the final location; however, my
system presorts messages with maildrop and .forward+... files, thus the
"ignoring of broken references" needs to be done when the Group buffer
is assembled and not as incoming messages are sorted.

BTW, is the regexp (...broken-references-mailers) case sensitive?
Pegasus uses "X-mailer: Pegasus ..." with lower-case m (but testing with
upper-case M in an existing IMAP folder doesn't help either).

-- 
Matthias Andree

Re: feature request: work around b0rked Pegasus Mail headers

[Reiner Steib]

On Sun, Jun 25 2006, Matthias Andree wrote:

> If I understand this correctly, it is laundering incoming messages that
> are being moved via the mailsource to the final location; 

Yes.

> however, my system presorts messages with maildrop and
> .forward+... files, 

I don't know maildrop, but maybe it'd  `ignore-broken-references'

> thus the "ignoring of broken references" needs to be done when the
> Group buffer is assembled and not as incoming messages are sorted.

I couldn't quickly locate the right place where this could be done
(probably you'd need to add In-Reply-To to `gnus-extra-headers').
Anyone?

> BTW, is the regexp (...broken-references-mailers) case sensitive?
> Pegasus uses "X-mailer: Pegasus ..." with lower-case m (but testing with
> upper-case M in an existing IMAP folder doesn't help either).

'nnmail-prepare-incoming-header-hook' is called with
`case-fold-search' bound to t in `nnmail-process-unix-mail-format'
(and other places).  So it's not treated case sensitively.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: feature request: work around b0rked Pegasus Mail headers

[Reiner Steib]

On Sun, Jun 25 2006, Reiner Steib wrote:

> On Sun, Jun 25 2006, Matthias Andree wrote:
>> however, my system presorts messages with maildrop and
>> .forward+... files, 
>
> I don't know maildrop, [...]

... but maybe it's possible to ignore the broken references on mail
delivery?  Using reformail?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: feature request: work around b0rked Pegasus Mail headers

[Matthias Andree]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Sun, Jun 25 2006, Reiner Steib wrote:
>
>> On Sun, Jun 25 2006, Matthias Andree wrote:
>>> however, my system presorts messages with maildrop and
>>> .forward+... files, 
>>
>> I don't know maildrop, [...]
>
> ... but maybe it's possible to ignore the broken references on mail
> delivery?  Using reformail?

Well, I don't plan to modify the messages, I just want Gnus to ignore
(or correct) broken references internally when threading.

-- 
Matthias Andree