Gnus generates an invalid To header

Gnus generates an invalid To header

[Tassilo Horn]

Hi,

I have a mail with these header lines:

--8<---------------cut here---------------start------------->8---
Reply-To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
From: <kontakt@tieraerztliche-klinik.com>
--8<---------------cut here---------------end--------------->8---

When I want to reply to it (R or r) Gnus generates the following To
header:

--8<---------------cut here---------------start------------->8---
To: "kontakt@/usr/bin/idn: tld_check_4z: missing inputtieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
--8<---------------cut here---------------end--------------->8---

I'm not too familiar with the specs, but I guess it's not Gnus' but the
senders' fault, e.g. it has to be

--8<---------------cut here---------------start------------->8---
Reply-To: "Tierklinik Kaiserberg Duisburg" <kontakt@tieraerztliche-klinik.com>
--8<---------------cut here---------------end--------------->8---

Right?

Bye,
Tassilo

Re: Gnus generates an invalid To header

[Reiner Steib]

On Sat, Nov 24 2007, Tassilo Horn wrote:

> Reply-To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
> From: <kontakt@tieraerztliche-klinik.com>
>
> When I want to reply to it (R or r) Gnus generates the following To
> header:
>
> To: "kontakt@/usr/bin/idn: tld_check_4z: missing inputtieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>

This looks like the output of the IDNA program (/usr/bin/idn).  I
don't have IDNA installed, so I get (disabling it does the same [1]):

,----
| To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
`----

This is exactly the bogus Reply-To the sender requested.

By edebugging `message-idna-to-ascii-rhs-1', I found that it calls
(idna-to-ascii rhs) with `rhs' = "" because the right hand side of
"tierklinikkaiserbergduisburg" is the empty string.  Maybe message
should just return the same header instead if calling `idna-to-ascii'
if `rhs' is empty?

Throwing an error would not be correct, because "To: foo" is valid
(local user / domain added by the MTA).

> I'm not too familiar with the specs, but I guess it's not Gnus' but the
> senders' fault, e.g. it has to be
>
> Reply-To: "Tierklinik Kaiserberg Duisburg" <kontakt@tieraerztliche-klinik.com>

Yes.  "kontakt@tieraerztliche-klinik.com" is a comment.

Bye, Reiner.

[1]
,----[ M-x apropos-variable RET idna RET ]
| gnus-use-idna
|   Variable: Whether IDNA decoding of headers is used when viewing messages.
| message-use-idna
|   Variable: Whether to encode non-ASCII in domain names into ASCII according to IDNA.
`----
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Gnus generates an invalid To header

[Tassilo Horn]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

Hi Reiner,

>> Reply-To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
>> From: <kontakt@tieraerztliche-klinik.com>
>>
>> When I want to reply to it (R or r) Gnus generates the following To
>> header:
>>
>> To: "kontakt@/usr/bin/idn: tld_check_4z: missing inputtieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
>
> This looks like the output of the IDNA program (/usr/bin/idn).  I
> don't have IDNA installed, so I get (disabling it does the same [1]):
>
> ,----
> | To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
> `----
>
> This is exactly the bogus Reply-To the sender requested.
>
> By edebugging `message-idna-to-ascii-rhs-1', I found that it calls
> (idna-to-ascii rhs) with `rhs' = "" because the right hand side of
> "tierklinikkaiserbergduisburg" is the empty string.  Maybe message
> should just return the same header instead if calling `idna-to-ascii'
> if `rhs' is empty?

With "the same header" you mean the complete bogus Reply-To header?  I
don't think that would be good, because then it would possibly be sent
to a local user.  And sending to a wrong recipient is much worse than
not sending at all.

Bye,
Tassilo

Re: Gnus generates an invalid To header

[Elias Oltmanns]

Tassilo Horn <tassilo@member.fsf.org> wrote:
> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
> Hi Reiner,
>
>>> Reply-To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
>>> From: <kontakt@tieraerztliche-klinik.com>
>>>
>>> When I want to reply to it (R or r) Gnus generates the following To
>>> header:
>>>
>>> To: "kontakt@/usr/bin/idn: tld_check_4z: missing inputtieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
>>
>> This looks like the output of the IDNA program (/usr/bin/idn).  I
>> don't have IDNA installed, so I get (disabling it does the same [1]):
>>
>> ,----
>> | To: "kontakt@tieraerztliche-klinik.com" <Tierklinik Kaiserberg Duisburg>
>> `----
>>
>> This is exactly the bogus Reply-To the sender requested.
>>
>> By edebugging `message-idna-to-ascii-rhs-1', I found that it calls
>> (idna-to-ascii rhs) with `rhs' = "" because the right hand side of
>> "tierklinikkaiserbergduisburg" is the empty string.  Maybe message
>> should just return the same header instead if calling `idna-to-ascii'
>> if `rhs' is empty?
>
> With "the same header" you mean the complete bogus Reply-To header?  I
> don't think that would be good, because then it would possibly be sent
> to a local user.  And sending to a wrong recipient is much worse than
> not sending at all.

What Reiner suggests is fixing a bug on Gnus' part and a priori the
right thing to do.  Your concern is legitimate, but it is a feature
request rather than a bug fix.  After all, the sender is ultimately
responsible for setting a valid Reply-To header and, by failing to do
so, positively asks for trouble.  Also, Reiner's suggestion doesn't even
change anything on the MTA level since the non'comment part of both
example headers (Reiner's and yours) are identical.

That said, it might still be worthwhile to implement the feature you are
asking for.  I recollect two incidents where I tried to reply to emails
only to find out that the Reply-To header was bogus.  However, there
seem to be too many ways to screw the Reply-To header in a way that
results in a syntactically correct but effectively bogus header.  If a
user decides to use fully qualified email addresses only (after all,
there is bbdb and auto completion), gnus can generally warn before
shipping off messages to addresses without a domain part.  This is about
all I can think of right now to tacle this issue and its probably not a
good default setting.

Regards,

Elias

Re: Gnus generates an invalid To header

[Reiner Steib]

On Mon, Nov 26 2007, Elias Oltmanns wrote:

> Tassilo Horn <tassilo@member.fsf.org> wrote:
>> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>>> By edebugging `message-idna-to-ascii-rhs-1', I found that it calls
>>> (idna-to-ascii rhs) with `rhs' = "" because the right hand side of
>>> "tierklinikkaiserbergduisburg" is the empty string.  Maybe message
>>> should just return the same header instead if calling `idna-to-ascii'
>>> if `rhs' is empty?
>>
>> With "the same header" you mean the complete bogus Reply-To header?  

In the end, yes.  (More specifically, I meant
`message-idna-to-ascii-rhs-1'.)

>> I don't think that would be good, because then it would possibly be
>> sent to a local user.  And sending to a wrong recipient is much
>> worse than not sending at all.
[...]
> Your concern is legitimate, but it is a feature request rather than
> a bug fix.  After all, the sender is ultimately responsible for
> setting a valid Reply-To header and, by failing to do so, positively
> asks for trouble.

ACK

> That said, it might still be worthwhile to implement the feature you are
> asking for.  I recollect two incidents where I tried to reply to emails
> only to find out that the Reply-To header was bogus.  However, there
> seem to be too many ways to screw the Reply-To header in a way that
> results in a syntactically correct but effectively bogus header.  If a
> user decides to use fully qualified email addresses only (after all,
> there is bbdb and auto completion), gnus can generally warn before
> shipping off messages to addresses without a domain part.  This is about
> all I can think of right now to tacle this issue and its probably not a
> good default setting.

I agree.  Such a feature could be provides via
`message-syntax-checks'.  But it should be disabled by default, I
think.  Hm, probably it is already there: See
`message-bogus-recipient-p', `message-bogus-address-regexp', ...

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Avoid calling idna-to-ascii on ASCII domain names (was: Gnus generates an invalid To header)

[Reiner Steib]

[ Resending.  The first message didn't reach the list, AFAICS.
  In the meantime, I have installed the latter patch. ]

On Mon, Nov 26 2007, Elias Oltmanns wrote:

> Tassilo Horn <tassilo@member.fsf.org> wrote:
>> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>>> By edebugging `message-idna-to-ascii-rhs-1', I found that it calls
>>> (idna-to-ascii rhs) with `rhs' = "" because the right hand side of
>>> "tierklinikkaiserbergduisburg" is the empty string.  Maybe message
>>> should just return the same header instead if calling `idna-to-ascii'
>>> if `rhs' is empty?
[...]
> What Reiner suggests is fixing a bug on Gnus' part and a priori the
> right thing to do.  

This patch avoids calling `idna-to-ascii' if the domain part is empty:

--8<---------------cut here---------------start------------->8---
--- message.el	06 Nov 2007 00:13:10 +0100	7.228
+++ message.el	28 Nov 2007 22:15:16 +0100	
@@ -5547,7 +5547,9 @@
 			(mapcar 'downcase
 				(mapcar
 				 'car (mail-header-parse-addresses field))))))
-	(setq ace (downcase (idna-to-ascii rhs)))
+	(setq ace (if (equal rhs "")
+		      ""
+		    (downcase (idna-to-ascii rhs))))
 	(when (and (not (equal rhs ace))
 		   (or (not (eq message-use-idna 'ask))
 		       (y-or-n-p (format "Replace %s with %s in %s:? "
--8<---------------cut here---------------end--------------->8---

This one avoids calling `idna-to-ascii' if the domain part contains
only ASCII characters.  I think we should apply it.  Does anyone see a
problem with it?

--8<---------------cut here---------------start------------->8---
--- message.el	06 Nov 2007 00:13:10 +0100	7.228
+++ message.el	28 Nov 2007 22:21:47 +0100	
@@ -5547,7 +5547,9 @@
 			(mapcar 'downcase
 				(mapcar
 				 'car (mail-header-parse-addresses field))))))
-	(setq ace (downcase (idna-to-ascii rhs)))
+	(setq ace (if (string-match "\\`[[:ascii:]]+\\'" rhs)
+		      rhs
+		    (downcase (idna-to-ascii rhs))))
 	(when (and (not (equal rhs ace))
 		   (or (not (eq message-use-idna 'ask))
 		       (y-or-n-p (format "Replace %s with %s in %s:? "
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/