5.8.2 bug#1: test тестtéßt test

5.8.2 bug#1: test тестtéßt test

[Vladimir Volovich]

Hi,

when sending a message with this subject, gnus encodes non-ascii text
as encoded-word, and as a result, the space between a cyrillic word
(koi8-r) and a latin-1 word is lost. This rule was already discussed
in this list (in a different thread):

RFC 2047 says:

RFC-2047> When displaying a particular header field that contains
RFC-2047> multiple 'encoded-word's, any 'linear-white-space' that
RFC-2047> separates a pair of adjacent 'encoded-word's is ignored.
RFC-2047> (This is to allow the use of multiple 'encoded-word's to
RFC-2047> represent long strings of unencoded text, without having to
RFC-2047> separate 'encoded-word's where spaces occur in the unencoded
RFC-2047> text.)

The subject contained a cyrillic word and a latin-1 word separated by
a real space. Gnus encoded both words as encoded-word, but not encoded
a space between them. As a result, this space is ignored when
displaying the subject.

Best regards, Vladimir.

Re: 5.8.2 bug#2: test тестtéßt test

[Vladimir Volovich]

Hi,

when i try to send a mail which body contains a text which is in the
subject of this letter, i get the following error:

Signaling: (wrong-type-argument arrayp nil)
  get-charset-property(iso-8859-1 preferred-coding-system)
  #[(x) "\bС=┐\b
  mapcar(#[(x) "\bС=┐\b
  find-coding-systems-for-charsets((iso-8859-1 koi8-r))
  mm-find-mime-charset-region(264 476)
  mml-parse-1()
  mml-parse()
  mml-generate-mime()
  message-encode-message-body()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)
  call-interactively(message-send-and-exit)

This is on Emacs 20.5, gnus 5.8.2

Best regards, Vladimir.

Re: 5.8.2 bug#2: test ~~~~téßt test

[Shenghuo ZHU]

>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:

VV> Hi,
VV> when i try to send a mail which body contains a text which is in the
VV> subject of this letter, i get the following error:

VV> Signaling: (wrong-type-argument arrayp nil)
VV>   get-charset-property(iso-8859-1 preferred-coding-system)

[...]

VV>   find-coding-systems-for-charsets((iso-8859-1 koi8-r))
VV>   mm-find-mime-charset-region(264 476)
VV>   mml-parse-1()
VV>   mml-parse()
VV>   mml-generate-mime()
VV>   message-encode-message-body()
VV>   message-send-mail(nil)
VV>   message-send-via-mail(nil)
VV>   message-send(nil)
VV>   message-send-and-exit(nil)
VV>   call-interactively(message-send-and-exit)

VV> This is on Emacs 20.5, gnus 5.8.2

VV> Best regards, Vladimir.

Fixed in 5.8.3.

-- 
~{VlJ$;p~}

Re: 5.8.2 bug#1: test тестtéßt test

[Vladimir Volovich]

"VV" == Vladimir Volovich writes:

RFC-2047> When displaying a particular header field that contains
RFC-2047> multiple 'encoded-word's, any 'linear-white-space' that
RFC-2047> separates a pair of adjacent 'encoded-word's is ignored.
RFC-2047> (This is to allow the use of multiple 'encoded-word's to
RFC-2047> represent long strings of unencoded text, without having to
RFC-2047> separate 'encoded-word's where spaces occur in the unencoded
RFC-2047> text.)

 VV> The subject contained a cyrillic word and a latin-1 word
 VV> separated by a real space. Gnus encoded both words as
 VV> encoded-word, but not encoded a space between them. As a result,
 VV> this space is ignored when displaying the subject.

Here is an encoded subject:

Subject: 5.8.2 bug#1: test =?koi8-r?b?1MXT1A==?= =?iso-8859-1?q?t=E9=DFt?= test
                                                ^
                                                |
The question is: is this space -----------------+ a 'linear-white-space'?

If yes (and thus it should be ignored when displaying), then gnus'
encoded-word *encoder* is broken. Otherwise (if this is not a
'linear-white-space'), gnus' encoded-word *decoder* (i.e. function
which decodes subject and displays it in summary and article buffers)
is broken. In each case, this is a bug in gnus which should be fixed.

Best,
v.

Re: 5.8.2 bug#1: test тестtéßt test тест téßt

[Shenghuo ZHU]

>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:

[...]

Fixed in 5.8.3.

-- 
~{VlJ$;p~}

Re: 5.8.2 bug#1: test ^[,LbUab^[(Btéßt test тестtéßt

[Vladimir Volovich]

"ZSH" == Shenghuo ZHU writes:

 >>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:
 ZSH> [...]

 ZSH> Fixed in 5.8.3.

The subject of the letter which you sent to me looks like:

Subject: Re: 5.8.2 bug#1:
  test=?iso-8859-5?b?IOLV4eJ0?==?iso-8859-1?q?=E9=DFt?= test=?iso-8859-5?b?IOLV4eI=?==?iso-8859-1?q?_t=E9=DFt?=

Is it okay to use such long lines in headers? Shouldn't gnus break
long lines (between encoded-words)?

	Best regards, -- Vladimir.

Re: 5.8.2 bug#1: test тестtéßt test тест téßt

[Shenghuo ZHU]

>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:

VV> Is it okay to use such long lines in headers? 

No.

VV> Shouldn't gnus break long lines (between encoded-words)?

Fixed.

-- 
~{VlJ$;p~}

Re: 5.8.2 bug#1: test тестtéßt test тест téßt

[Shenghuo ZHU]

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:
VV> Is it okay to use such long lines in headers? 

ZSH> No.

VV> Shouldn't gnus break long lines (between encoded-words)?

ZSH> Fixed.

Maybe I should test it using iso-8859-5 instead of koi8-r.

-- 
~{VlJ$;p~}

Re: 5.8.2 bug#1 blah blah blah

[Florian Weimer]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> ZSH> Fixed.
> 
> Maybe I should test it using iso-8859-5 instead of koi8-r.

And `string-to-char-list' isn't available on Emacs 20.4.

Re: 5.8.2 bug#1 blah blah blah

[Hrvoje Niksic]

Florian Weimer <fw@s.netic.de> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > ZSH> Fixed.
> > 
> > Maybe I should test it using iso-8859-5 instead of koi8-r.
> 
> And `string-to-char-list' isn't available on Emacs 20.4.

AFAIK `string-to-char-list' was available in XEmacs only as a
compatibility function.  Maybe now we'll be able to delete it.

Re: 5.8.2 bug#1: test ^[,LbUab^[(Btéßt test тестtéßt

[Vladimir Volovich]

"ZSH" == Shenghuo ZHU writes:

 VV> Shouldn't gnus break long lines (between encoded-words)?
 ZSH> Fixed.

thank you

 ZSH> Maybe I should test it using iso-8859-5 instead of koi8-r.

hmm.. your message contains cyrillic encoded in iso-8859-5, not
koi8-r.

i have two questions:

1) why the subject of this letter is partially brokenly encoded by
   gnus (i'm writing this letter from pgnus 0.99)? (cyrillic word
   followed by latin-1 word without a space were encoded brokenly,
   but cyrillic word followed by latin-1 word with a space between
   them was encoded correctly. does the current version of gnus behave
   as buggy as pgnus 0.99 in this situation?)

2) when displaying subjects in an *Article* buffer, gnus displays
   several lines, instead of just one (as in the summary buffer). this
   violates interpretation of linear spaces given in RFC-2047, doesn't
   it?

	Best regards, -- Vladimir.

Re: 5.8.2 bug#1: test

[Shenghuo ZHU]

>>>>> "VV" == Vladimir Volovich <vvv@vvv.vsu.ru> writes:

ZSH> Maybe I should test it using iso-8859-5 instead of koi8-r.

VV> hmm.. your message contains cyrillic encoded in iso-8859-5, not
VV> koi8-r.

I sent two messages. The other is encoded in koi8-r.

VV> i have two questions:

VV> 1) why the subject of this letter is partially brokenly encoded by
VV> gnus (i'm writing this letter from pgnus 0.99)? (cyrillic word
VV> followed by latin-1 word without a space were encoded brokenly,
VV> but cyrillic word followed by latin-1 word with a space between
VV> them was encoded correctly. does the current version of gnus
VV> behave as buggy as pgnus 0.99 in this situation?)

I think it is the problem of rfc2047-dissect-region. The MIME charset
depended on the first non-ascii MULE charset in words, therefore, the
order is sensitive. Now, I've rewritten the function, fixing the bug.

VV> 2) when displaying subjects in an *Article* buffer, gnus displays
VV> several lines, instead of just one (as in the summary
VV> buffer). this violates interpretation of linear spaces given in
VV> RFC-2047, doesn't it?

I only found the explicit description about CRLF between
encoded-words, nothing for other cases, i.e. what you saw. If you can
find the description. Please post it.

-- 
~{VlJ$;p~}