washing quoted-printable bug again

washing quoted-printable bug again

[Vladimir Volovich]

Hi,

in pgnis-0.36, washing QP (W q) results in completely messed up
buffer. For example, characters are displayed as
\214<8-bit-char>\214<8-bit-char>... (afair, this is a result of
double-encoding).

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> in pgnis-0.36, washing QP (W q) results in completely messed up
> buffer. For example, characters are displayed as
> \214<8-bit-char>\214<8-bit-char>... (afair, this is a result of
> double-encoding).

Would these be articles encoded in QP without a CTE header?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> in pgnis-0.36, washing QP (W q) results in completely messed up
 >> buffer. For example, characters are displayed as
 >> \214<8-bit-char>\214<8-bit-char>... (afair, this is a result of
 >> double-encoding).

 LMI> Would these be articles encoded in QP without a CTE header?

Yes. Even without C-T header. After washing them, i get a buffer in
which coloured text is messed up, and some 8-bit chars are represented
as \214<8-bit-char>.

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> Yes. Even without C-T header. After washing them, i get a buffer in
> which coloured text is messed up, and some 8-bit chars are represented
> as \214<8-bit-char>.

If I try this with 0.38 (and with mm-default-coding-system set), then
things seem to work for me.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> Yes. Even without C-T header. After washing them, i get a buffer
 >> in which coloured text is messed up, and some 8-bit chars are
 >> represented as \214<8-bit-char>.
 LMI> If I try this with 0.38 (and with mm-default-coding-system set),
 LMI> then things seem to work for me.

Nope. Just worse: with 0.38 i again began to see \214<8-bit-char>
instead of russian letters in my nnmbox groups (at least for 8-bit
articles without c-t header).

The above bug with washing did not disappear either (i tested it in
`normal' nntp newsgroups), --- the behavior did not change at all.

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Vladimir Volovich]

"VV" == Vladimir Volovich writes:

 VV> Nope. Just worse: with 0.38 i again began to see \214<8-bit-char>
 VV> instead of russian letters in my nnmbox groups (at least for
 VV> 8-bit articles without c-t header).

I tested more, and it appears that gnus shows \214<8-bit-char>
combinations for all mails (in nnmbox backend) with 8-bit bodies
regardless of presence of C-T and C-T-T headers. I.e., the behavior
returned to original broken behavior. This broke either in 0.37 or in
0.38. In 0.36 it was Okay, afair.

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> I tested more, and it appears that gnus shows \214<8-bit-char>
> combinations for all mails (in nnmbox backend) with 8-bit bodies
> regardless of presence of C-T and C-T-T headers. I.e., the behavior
> returned to original broken behavior. This broke either in 0.37 or in
> 0.38. In 0.36 it was Okay, afair.

0.38 stopped switching on multibyte mode unless Emacs was already
multibytified.  This fixed some not-so-obscure problems.  I guess
there could still be problems is some of the buffers are multibyte and
others not.  Is your mbox buffer multibyte or unibyte, and how about
the other buffers?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 LMI> 0.38 stopped switching on multibyte mode unless Emacs was
 LMI> already multibytified. This fixed some not-so-obscure problems.
 LMI> I guess there could still be problems is some of the buffers are
 LMI> multibyte and others not.  Is your mbox buffer multibyte or
 LMI> unibyte, and how about the other buffers?

How can i test this to answer your questions? [as for mbox, when i try
to visit it with C-x C-f mbox when gnus is active, i get a question
"Revisit file non-literally?"]

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> How can i test this to answer your questions? 

Look at the mode lines in the buffers.  Unibyte buffers have "-:",
while multibyte have "1:" (or some other character before the colon). 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> How can i test this to answer your questions?
 LMI> Look at the mode lines in the buffers.  Unibyte buffers have
 LMI> "-:", while multibyte have "1:" (or some other character before
 LMI> the colon).

Well, i switched to mbox buffer, and it seems to be unibyte: "-:--".
The summary and article buffers in nnmbox groups are multibyte:
"-R:--". These buffers are broken as in early versions of pgnus: all
russian 8-bit characters are displayed as \214<8-bit-char>. It seems
that if the C-T-E of the message in russian was Q-P or base64 (i.e. in
the original article there were no 8-bit characters `as is'), then
this article is displayed correctly; 8-bit subjects in summary buffer
also became broken in 0.38 (\214<8-bit-char>) unless they were
word-encoded.

For nntp newsgroups, buffers are encoded in the same way, but there
russian characters are displayed well (as always). However, the bug
about which subject says (appearing of \214<8-bit-char> instead of
russian characters after washing) is present even in nnpt newsgroups.

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> Well, i switched to mbox buffer, and it seems to be unibyte: "-:--".
> The summary and article buffers in nnmbox groups are multibyte:
> "-R:--".

Hm.  Did you recompile Gnus properly with "make" in the main
directory?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> Well, i switched to mbox buffer, and it seems to be unibyte:
 >> "-:--".  The summary and article buffers in nnmbox groups are
 >> multibyte: "-R:--".
 LMI> Hm.  Did you recompile Gnus properly with "make" in the main
 LMI> directory?

Surely. Then i copied generated *.elc files to
/usr/share/emacs/20.3/lisp/gnus/ manually. What results are expected
for multi/unibyte tests?

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

>  LMI> Hm.  Did you recompile Gnus properly with "make" in the main
>  LMI> directory?
> 
> Surely. Then i copied generated *.elc files to
> /usr/share/emacs/20.3/lisp/gnus/ manually.

Don't do that.  Leave the .elc files where they are, and adjust the
load-path as discussed in the readme file.

> What results are expected for multi/unibyte tests?

I would expect to see "-:" in all the buffers.  "R:" means that
something is switching multibyte mode on in some buffers.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> What results are expected for multi/unibyte tests?
 LMI> I would expect to see "-:" in all the buffers.  "R:" means that
 LMI> something is switching multibyte mode on in some buffers.

What i see (in 0.40), is group, summary, article buffers are all "R:".
I'm not exactly sure, but i think that in 0.36 they were also "R:",
but things definitely worked fine. Note also, that in nntp newsgroups
i also see "R:" in all buffers (summary, article), but there russian
characters are displaying well.

What do you see if you set

(setq gnus-secondary-select-methods '((nnmbox "")))
(setq rfc2047-default-charset 'koi8-r) ; pgnus
(setq mm-default-coding-system 'koi8)

Are your summary and article buffers unibyte?? [i do not run emacs
with --unibyte option, should i?]

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> What i see (in 0.40), is group, summary, article buffers are all "R:".
> I'm not exactly sure, but i think that in 0.36 they were also "R:",
> but things definitely worked fine.

In 0.36, all buffers we multibyte, no matter what.

> Are your summary and article buffers unibyte?? [i do not run emacs
> with --unibyte option, should i?]

Try it, and see whether there are any differences.

Do you do anything like

(setq-default enable-multibyte-characters nil)

in your .emacs?  If you do, also try to take that out, and see whether
that makes any difference.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 >> What i see (in 0.40), is group, summary, article buffers are all
 >> "R:".  I'm not exactly sure, but i think that in 0.36 they were
 >> also "R:", but things definitely worked fine.
 LMI> In 0.36, all buffers we multibyte, no matter what.

 >> Are your summary and article buffers unibyte?? [i do not run emacs
 >> with --unibyte option, should i?]
 LMI> Try it, and see whether there are any differences.

yes, it makes differences. When running "emacs --unibyte", i get
latin1 text instead of russian characters (i.e. umlauts, acuted
letters, etc).  This is in all buffers: in nnmbox, and in nntp
newsgroups; i no longer can see russian letters even in nntp
newsgroups.

 LMI> Do you do anything like
 LMI> (setq-default enable-multibyte-characters nil)

No.

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> yes, it makes differences. When running "emacs --unibyte", i get
> latin1 text instead of russian characters (i.e. umlauts, acuted
> letters, etc).  This is in all buffers: in nnmbox, and in nntp
> newsgroups; i no longer can see russian letters even in nntp
> newsgroups.

What if you

(set-language-environment "Latin-5")

or whatever is the Russian language environment...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:

 LMI> What if you
 LMI> (set-language-environment "Latin-5")
 LMI> or whatever is the Russian language environment...

I have the following in my .emacs file:

(set-language-environment "Cyrillic-KOI8")

	Best regards, -- Vladimir.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> I have the following in my .emacs file:
> 
> (set-language-environment "Cyrillic-KOI8")

I'm still not able to reproduce this.  I start Emacs with --unibyte,
and I have

(set-language-environment "Cyrillic-KOI8")

in my .emacs file.  Still, no Gnus buffers show up with "R:".  Are you
totally, absolutely sure that nothing meddles with
`enable-multibyte-characters' -- perhaps some site-wide init file? 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> > I have the following in my .emacs file:
> > 
> > (set-language-environment "Cyrillic-KOI8")
> 
> I'm still not able to reproduce this.  I start Emacs with --unibyte,
> and I have
> 
> (set-language-environment "Cyrillic-KOI8")
> 
> in my .emacs file.  Still, no Gnus buffers show up with "R:".  Are you
> totally, absolutely sure that nothing meddles with
> `enable-multibyte-characters' -- perhaps some site-wide init file? 

Well, i just started "emacs --unibyte" with the following trivial
~/.emacs file (no ~/.gnus* files):

(set-default-font "fontset-standard")
(set-language-environment "Cyrillic-KOI8")
(load "files") ; partial bugfix for literal file visiting
(setq rfc2047-default-charset 'koi8-r)
(setq mm-default-coding-system 'koi8)
(setq gnus-select-method '(nntp "localhost"))
(setq gnus-secondary-select-methods '((nnmbox "")))

and buffers are unibyte "-:". But i'm not able to input/see russian
characters anymore! instead, i see latin1 (iso-8859-1) characters.
this is not with which i can live. :-) [and i tested again with my
"standard" .emacs running with --unibyte option, -- i got the same
result: all buffers were unibyte, but no russian characters]

And in general, is it good to run emacs with --unibyte? for example,
will japanese characters work?  if i preview japanese/chinese mails in
gnus running in "emacs --unibyte", i see only messy symbols.  imho,
gnus should work right without --unibyte too. and 0.36 worked fine,
--- things were broken in 0.38.

Best regards, vvv.

Re: washing quoted-printable bug again

[Steinar Bang]

>>>>> Vladimir Volovich <vvv@vvv.vsu.ru>:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> in my .emacs file.  Still, no Gnus buffers show up with "R:".  Are you
>> totally, absolutely sure that nothing meddles with
>> `enable-multibyte-characters' -- perhaps some site-wide init file? 

> Well, i just started "emacs --unibyte" with the following trivial
> ~/.emacs file (no ~/.gnus* files):

Have you tried starting it with
	emacs --unibyte --no-init-file --no-site-file
?

Or maybe
	emacs --unibyte --no-init-file --no-site-file --load=yoursimple.emacs
?

Re: washing quoted-printable bug again

[Vladimir Volovich]

Steinar Bang <sb@metis.no> writes:

> Have you tried starting it with
> 	emacs --unibyte --no-init-file --no-site-file --load=yoursimple.emacs
> ?

Yes, i tried this. The behavior is not quite satisfactory. When i'm using
fontset-standard, i cannot see and input russian characters. I cannot see
or input any multibyte characters (e.g. japanese).

If i run in multibyte mode, russian characters in nnmbox newsgroups look
like \204<8-bit-char>, if they were 8-bit in the original mail (and seem
to look right if they were somehow ascii-clean encoded, e.g. with QP or
base64 or encode-word).

What bothers me is that all WORKED JUST FINE in pgnus 0.36 (and maybe
0.37), and things were BROKEN in 0.38. Could we PLEASE restore the
previous behavior? PLEASE! :-)

Best regards, VVV.

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> What bothers me is that all WORKED JUST FINE in pgnus 0.36 (and maybe
> 0.37), and things were BROKEN in 0.38. Could we PLEASE restore the
> previous behavior? PLEASE! :-)

The previous behavior was to always force multibyte in all buffers,
which is not acceptable.

I've now found some bugs in this area -- could you see whether 0.43
fixes this problem for you?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> Well, i just started "emacs --unibyte" with the following trivial
> ~/.emacs file (no ~/.gnus* files):

[...]

> and buffers are unibyte "-:". But i'm not able to input/see russian
> characters anymore!

Hm.  Isn't koi8-r a unibyte charset encoding?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: washing quoted-printable bug again

[Vladimir Volovich]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> > and buffers are unibyte "-:". But i'm not able to input/see russian
> > characters anymore!
> 
> Hm.  Isn't koi8-r a unibyte charset encoding?

Yes, it is. But with fontset-standard, i get iso-8859-1 characters instead
of koi8-r when run in unibyte mode. when i switch to 8-bit koi8-r-encoded
font, i get russian letters visible. But i feel that this is not the
Right Thing (TM) to do, because e.g. i do not see any multibyte characters.
Gnus should run clean in a multibyte mode, shouldn't it? :-)

Best regards, VVV.

Re: washing quoted-printable bug again

[Vladimir Volovich]

Hi,

i checked again, and it appears that things broke in 0.37. Pgnus 0.36
works great in multibyte mode. I have NO problems in nnmbox with 0.36.
This is the last version of pgnus which which i can work (with all the
bugs of 0.36 built in :). :-(

	Best regards, -- Vladimir.