Brian May's `Re: Bugs in PGnus' article jams Gnus

Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hannu Koivisto]

Greetings,

Any attempt to read that article results in jam in my Emacs 20.3
with Gnus from CVS with file versions shown below.  If I break it
with debug-on-quit enabled (after waiting about one minute or so),
I get the following backtrace.

"""
Signaling: (quit)
  generate-new-buffer(" *mm*")
  mm-copy-to-buffer()
  mm-dissect-singlepart(("text/plain") base64 t nil nil)
  mm-dissect-buffer(t)
  mm-dissect-multipart(("multipart/mixed" (boundary . "=-=-=")))
  mm-dissect-buffer()
  gnus-display-mime()
  gnus-article-prepare-display()
  gnus-article-prepare(2893 nil)
  gnus-summary-display-article(2893 nil)
  gnus-summary-select-article(nil nil pseudo)
  gnus-summary-scroll-up(1)
* call-interactively(gnus-summary-scroll-up)
"""

Once I managed to get a backtrace that had one call more, which was
something like `generate-new-buffer(" *mm<820>*")', but I haven't
been able to replicate that.

It may be that the mail forwarder thing I'm using has broken it (it
has modified encodings from 8bit to base64), but I don't know MIME
enough to say for sure.  But even if it is broken, I think Gnus
shouldn't jam on it.

The article exactly as I see it is available at
<URL:http://www.iki.fi/azure/var/bugreports/problem-mail.txt>,
directly copied from my nnml folder directory.

Gnus file versions are (duh, I should have saved the exact checkout
time so that one could use -D (or is there some other handy way to
specify a point in version history when one can't write to the
repository and make tags?)):

ChangeLog 5.255
Makefile.in 5.5
base64.el 5.10
binhex.el 5.8
dgnushack.el 5.9
earcon.el 5.2
format-spec.el 5.1
gnus-agent.el 5.35
gnus-art.el 5.104
gnus-async.el 5.8
gnus-audio.el 5.3
gnus-bcklg.el 5.4
gnus-cache.el 5.18
gnus-cite.el 5.11
gnus-cus.el 5.17
gnus-demon.el 5.9
gnus-draft.el 5.12
gnus-dup.el 5.3
gnus-eform.el 5.3
gnus-ems.el 5.14
gnus-gl.el 5.4
gnus-group.el 5.34
gnus-int.el 5.12
gnus-kill.el 5.7
gnus-load.el 5.2
gnus-logic.el 5.6
gnus-mh.el 5.2
gnus-mlspl.el 5.5
gnus-move.el 5.2
gnus-msg.el 5.47
gnus-nocem.el 5.8
gnus-picon.el 5.17
gnus-range.el 5.9
gnus-salt.el 5.9
gnus-score.el 5.21
gnus-setup.el 5.3
gnus-soup.el 5.6
gnus-spec.el 5.7
gnus-srvr.el 5.17
gnus-start.el 5.46
gnus-sum.el 5.88
gnus-topic.el 5.12
gnus-undo.el 5.3
gnus-util.el 5.34
gnus-uu.el 5.19
gnus-vm.el 5.3
gnus-win.el 5.6
gnus-xmas.el 5.23
gnus.el 5.118
ietf-drums.el 5.5
imap.el 1.6
lpath.el 5.31
mail-parse.el 5.6
mail-prsvr.el 5.2
mail-source.el 5.29
mailcap.el 5.28
md5.el 5.3
message.el 5.81
messagexmas.el 5.4
messcompat.el 5.3
mm-bodies.el 5.45
mm-decode.el 5.67
mm-encode.el 5.17
mm-util.el 5.52
mm-uu.el 5.17
mm-view.el 5.39
mml.el 5.37
nnagent.el 5.5
nnbabyl.el 5.6
nndb.el 5.6
nndir.el 5.2
nndoc.el 5.19
nndraft.el 5.15
nneething.el 5.8
nnfolder.el 5.24
nngateway.el 5.4
nnheader.el 5.30
nnheaderxm.el 5.2
nnimap.el 1.6
nnkiboze.el 5.7
nnlistserv.el 5.4
nnmail.el 5.43
nnmbox.el 5.9
nnmh.el 5.10
nnml.el 5.19
nnoo.el 5.3
nnslashdot.el 5.32
nnsoup.el 5.10
nnspool.el 5.8
nntp.el 5.28
nnultimate.el 5.20
nnvirtual.el 5.11
nnwarchive.el 5.4
nnweb.el 5.20
parse-time.el 5.8
pop3.el 5.14
qp.el 5.13
rfc1843.el 5.6
rfc2045.el 5.2
rfc2047.el 5.30
rfc2104.el 1.3
rfc2231.el 5.9
score-mode.el 5.4
smiley.el 5.12
time-date.el 5.6
uudecode.el 5.8
webmail.el 5.5

-- 
Hannu

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Gunnar Evermann]

Hannu Koivisto <azure@iki.fi> writes:

> Any attempt to read that article results in jam in my Emacs 20.3
> with Gnus from CVS with file versions shown below.  If I break it
> with debug-on-quit enabled (after waiting about one minute or so),

For me it took about 80 seconds on XEmacs21.1.8 (performing dozens of
garbage collections in the process) and then displayed correctly (I
think).

  Gunnar

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Stein Arild Stromme]

[Gunnar Evermann]

| Hannu Koivisto <azure@iki.fi> writes:
| 
| > Any attempt to read that article results in jam in my Emacs 20.3
| > with Gnus from CVS with file versions shown below.  If I break it
| > with debug-on-quit enabled (after waiting about one minute or so),
| 
| For me it took about 80 seconds on XEmacs21.1.8 (performing dozens of
| garbage collections in the process) and then displayed correctly (I
| think).
| 
|   Gunnar

For me too it took 80 seconds, Emacs 20.4.  Can't say whether it
displayed correctly, though, many weird characters.
-- 
Stein A. Strømme   ---   Matematisk institutt, Universitetet i Bergen
e-post: stromme@math.uib.no   telefon: 5558 4825   telefax: 5558 9672

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hannu Koivisto]

Gunnar Evermann <ge204@eng.cam.ac.uk> writes:

| > with debug-on-quit enabled (after waiting about one minute or so),
| 
| For me it took about 80 seconds on XEmacs21.1.8 (performing dozens of

Well, that one minute was just the time after which I generated the
backtrace.  I did wait for ten minutes to see that it really is
stuck.  Of course it could finish after ten minutes and one second,
but, uh, if that kind of times are a feature, then at least there
should be some sort of process indicator.

-- 
Hannu

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Laura Conrad]

>>>>> "Hannu" == Hannu Koivisto <azure@iki.fi> writes:

    Hannu> Gunnar Evermann <ge204@eng.cam.ac.uk> writes: 


    Gunnar> | >with debug-on-quit enabled (after waiting about one
    Gunnar> minute or so), | | For me it took about 80 seconds on
    Gunnar> XEmacs21.1.8 (performing dozens of

    Hannu> Well, that one minute was just the time after which I
    Hannu> generated the backtrace.  I did wait for ten minutes to see
    Hannu> that it really is stuck.  Of course it could finish after
    Hannu> ten minutes and one second, but, uh, if that kind of times
    Hannu> are a feature, then at least there should be some sort of
    Hannu> process indicator.


I decided to upgrade to xemacs 21.1.8 last night, so I have tried this 
message on both versions.  It seems to hang 20.4 indefinitely (longer
than I wanted to wait), while 21.1.8 does indeed display eventually
(also longer than I WANTED to wait).

I wouldn't say there's NO process indicator; the cursor keeps changing 
from the watch to the regular and back again.

-- 
Laura (mailto:lconrad@world.std.com , http://www.world.std.com/~lconrad/ )
(617) 661-8097	fax: (801) 365-6574 
233 Broadway, Cambridge, MA 02139

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Per Abrahamsen]

Laura Conrad <lconrad@gamesville.com> writes:

> I decided to upgrade to xemacs 21.1.8 last night, so I have tried this 
> message on both versions.  It seems to hang 20.4 indefinitely (longer
> than I wanted to wait), while 21.1.8 does indeed display eventually
> (also longer than I WANTED to wait).

20.4 finished here (250MHz SPARC) after 96 seconds.

Tue Dec  7 16:53:02 MET 1999
Tue Dec  7 16:54:38 MET 1999

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hrvoje Niksic]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Laura Conrad <lconrad@gamesville.com> writes:
> 
> > I decided to upgrade to xemacs 21.1.8 last night, so I have tried this 
> > message on both versions.  It seems to hang 20.4 indefinitely (longer
> > than I wanted to wait), while 21.1.8 does indeed display eventually
> > (also longer than I WANTED to wait).
> 
> 20.4 finished here (250MHz SPARC) after 96 seconds.
> 
> Tue Dec  7 16:53:02 MET 1999
> Tue Dec  7 16:54:38 MET 1999

Let's see what XEmacs (21.2) spent time doing:

    Function Name                         Ticks    %/Total   Call Count
    ==================================    =====    =======   ==========
    generate-new-buffer-name              2303     45.876    6369
    (in garbage collection)               1662     33.108    
    get-buffer-create                     248      4.940     6976
    kill-buffer                           139      2.769     5763
    re-search-forward                     111      2.211     20685
    string-match                          63       1.255     45878
    put-char-table                        59       1.175     1822
    byte-code                             58       1.155     14892
    expand-file-name                      46       0.916     6072
    ietf-drums-token-to-list              30       0.598     3642
    ...
    -------------------------------------------------------------------
    Total                                 5020     100.00


    One tick = 1 ms

So all the time was spent in generate-new-buffer-name.  How weird!

The message has about 600 parts (why it was created so is a different
anomaly.)  The mm code in turn created several hundreds of buffers,
all using generate-new-buffer-name.  The algorithm of
generate-new-buffer-name is:

  count = 1;
  while (1)
    {
      sprintf (number, "<%d>", ++count);
      gentemp = concat2 (name, build_string (number));
      if (!NILP (ignore))
        {
          tem = Fstring_equal (gentemp, ignore);
          if (!NILP (tem))
            return gentemp;
        }
      tem = Fget_buffer (gentemp);
      if (NILP (tem))
	return gentemp;
    }

So, the more buffers you create, the more time it takes.  For the
number of parts here, it corresponds to tens of thousands of passes
through the above loop: and each pass actually conses up a string!

I think that's the source of the immense slowness.  I'll try to fix
it.

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "H" == Hannu Koivisto <azure@iki.fi> writes:

H> Greetings,
H> Any attempt to read that article results in jam in my Emacs 20.3
H> with Gnus from CVS with file versions shown below.  If I break it
H> with debug-on-quit enabled (after waiting about one minute or so),
H> I get the following backtrace.

[...]

H> Once I managed to get a backtrace that had one call more, which was
H> something like `generate-new-buffer(" *mm<820>*")', but I haven't
H> been able to replicate that.

There are 606 parts in the message! Gnus takes time to decode them.

H> It may be that the mail forwarder thing I'm using has broken it (it
H> has modified encodings from 8bit to base64), but I don't know MIME
H> enough to say for sure.  But even if it is broken, I think Gnus
H> shouldn't jam on it.

[...]

I believe mail forwarder did not do anything wrong. On the quimby's
newsgroup, the message is encoded in 8bit and has no modification
notes, which means the orignal message got 606 parts.

The question is whether Gnus jammed Brian May's `Re: Bugs in PGnus'
article. I believe it did. I guess Gnus could not figure out what
charset should be used, then divided the article into parts.

-- 
Shenghuo

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

[...]

ZSH> I believe mail forwarder did not do anything wrong. On the quimby's
ZSH> newsgroup, the message is encoded in 8bit and has no modification
ZSH> notes, which means the orignal message got 606 parts.

ZSH> The question is whether Gnus jammed Brian May's `Re: Bugs in PGnus'
ZSH> article. I believe it did. I guess Gnus could not figure out what
ZSH> charset should be used, then divided the article into parts.

Finally I figure out what's wrong.  

In XEmacs (From the header of the message, I know Brian May use it),
the message contains `\207', which is a `control-1' characters and
some other `latin-iso8859-1' characters, therefore Gnus used multiply
charsets to encode it. Actually, Gnus can handle it as `iso8859-1'. So
I fixed it in latest CVS.

In Emacs `\207' is an `ascii' characters, so such a message will not
happen in Emacs.

-- 
Shenghuo

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hannu Koivisto]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

| some other `latin-iso8859-1' characters, therefore Gnus used multiply
| charsets to encode it. Actually, Gnus can handle it as `iso8859-1'. So
| I fixed it in latest CVS.

Whee, thanks, now the article was displayed and it took only 50
seconds.

-- 
Hannu

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hrvoje Niksic]

Hannu Koivisto <azure@iki.fi> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> | some other `latin-iso8859-1' characters, therefore Gnus used multiply
> | charsets to encode it. Actually, Gnus can handle it as `iso8859-1'. So
> | I fixed it in latest CVS.
> 
> Whee, thanks, now the article was displayed and it took only 50
> seconds.

I believe Shenghuo refers to "fixed" as "fixed not to generate these
parts", not "fixed to have them displayed fast".

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Toby Speight]

ZSH> Shenghuo ZHU <URL:mailto:zsh@cs.rochester.edu>
>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:


0> In article <2npuwiwsoe.fsf@tiger.jia.vnet>, ZSH wrote:

ZSH> In XEmacs (From the header of the message, I know Brian May use
ZSH> it), the message contains `\207', which is a `control-1' characters
ZSH> and some other `latin-iso8859-1' characters, therefore Gnus used
ZSH> multiply charsets to encode it.  Actually, Gnus can handle it as
ZSH> `iso8859-1'.  So I fixed it in latest CVS.
ZSH>
ZSH> In Emacs `\207' is an `ascii' characters, so such a message will
ZSH> not happen in Emacs.

Are you saying that with your patch, Message/Gnus will send articles
labelled as 8859.1 but containing characters in the \200-\237 range
(therefore not IS-8859.1)?  Isn't this as rude as the Microsoft stuff
that labels its Windows encoding as 8859.1?

OTOH, what is the correct action to take when we see these characters?
In this case it was a result of embedding binary data (bytecode) into
a text message - AFAICT, MIME doesn't cover this; it expects you to
put it into a non-text part (application/x-emacs-trace anyone?).

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "Hrvoje" == Hrvoje Niksic <hniksic@iskon.hr> writes:

Hrvoje> Hannu Koivisto <azure@iki.fi> writes:
>> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
>> 
>> | some other `latin-iso8859-1' characters, therefore Gnus used multiply
>> | charsets to encode it. Actually, Gnus can handle it as `iso8859-1'. So
>> | I fixed it in latest CVS.
>> 
>> Whee, thanks, now the article was displayed and it took only 50
>> seconds.

Hrvoje> I believe Shenghuo refers to "fixed" as "fixed not to generate
Hrvoje> these parts", not "fixed to have them displayed fast".

Yes.

-- 
Shenghuo ZHU

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "Toby" == Toby Speight <Toby.Speight@streapadair.freeserve.co.uk> writes:

[...]

Toby> Are you saying that with your patch, Message/Gnus will send
Toby> articles labelled as 8859.1 but containing characters in the
Toby> \200-\237 range (therefore not IS-8859.1)?  Isn't this as rude
Toby> as the Microsoft stuff that labels its Windows encoding as
Toby> 8859.1?

Toby> OTOH, what is the correct action to take when we see these
Toby> characters?  In this case it was a result of embedding binary
Toby> data (bytecode) into a text message - AFAICT, MIME doesn't cover
Toby> this; it expects you to put it into a non-text part
Toby> (application/x-emacs-trace anyone?).

Yes. This is expected. 

But it is difficult to automatically detect what the the type of the
message is.

-- 
Shenghuo ZHU

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hannu Koivisto]

Hrvoje Niksic <hniksic@iskon.hr> writes:

| I believe Shenghuo refers to "fixed" as "fixed not to generate these
| parts", not "fixed to have them displayed fast".

Yes, I realized this like two seconds after hitting `C-c C-c' :)
Which makes me wonder why reading, too, works now :-o  Perhaps there
has been some other changes between the version I used and the
version I just checked out after reading Shenghuo's article.
Another difference is that since I'm now at home and operating via
19k modem, I'm running Emacs on tty instead of X.  It's hard to
imagine why that would make any difference, but, well, you never
know about these computers :)

-- 
Hannu

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> Toby> Are you saying that with your patch, Message/Gnus will send
> Toby> articles labelled as 8859.1 but containing characters in the
> Toby> \200-\237 range (therefore not IS-8859.1)?  Isn't this as rude
> Toby> as the Microsoft stuff that labels its Windows encoding as
> Toby> 8859.1?

This is ugly. Can we not at the very least ask the user what to do?

Jan

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hrvoje Niksic]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > Toby> Are you saying that with your patch, Message/Gnus will send
> > Toby> articles labelled as 8859.1 but containing characters in the
> > Toby> \200-\237 range (therefore not IS-8859.1)?  Isn't this as rude
> > Toby> as the Microsoft stuff that labels its Windows encoding as
> > Toby> 8859.1?
> 
> This is ugly. Can we not at the very least ask the user what to do?

I concur.  Labeling non-Latin stuff as Latin 1 is highly bogus.
Characters in the "Control-1" range can screw some terminals, when
dumped raw.  And some software _will_ dump them raw, when Gnus
mislabels them as Latin 1.

I'm really not sure what to do here.  Maybe we should ask at
comp.mail.mime?

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Brian May]

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

    ZSH> In XEmacs (From the header of the message, I know Brian May use it),
    ZSH> the message contains `\207', which is a `control-1' characters and
    ZSH> some other `latin-iso8859-1' characters, therefore Gnus used multiply
    ZSH> charsets to encode it. Actually, Gnus can handle it as `iso8859-1'. So
    ZSH> I fixed it in latest CVS.

    ZSH> In Emacs `\207' is an `ascii' characters, so such a message will not
    ZSH> happen in Emacs.

I remain skeptical that this has fixed the problem. What happens
if I send mail, with another control character, that really is
non-ASCII?

Why does Gnus have to split the message when it sees a 'bad' character?

Why can't it encode the entire message, in the same way?

Also, another question: when I saw the my message (from the URL
previously given), Gnus used 8 bit encoding. I thought 8 bit encoding
meant no encoding at all, and was considered a bad idea for non-ASCII
characters? Where am I wrong?

-- 
Brian May <bmay@csse.monash.edu.au>

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "Brian" == Brian May <bmay@csse.monash.edu.au> writes:

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

ZSH> In XEmacs (From the header of the message, I know Brian May use
ZSH> it), the message contains `\207', which is a `control-1'
ZSH> characters and some other `latin-iso8859-1' characters, therefore
ZSH> Gnus used multiply charsets to encode it. Actually, Gnus can
ZSH> handle it as `iso8859-1'. So I fixed it in latest CVS.

ZSH> In Emacs `\207' is an `ascii' characters, so such a message will
ZSH> not happen in Emacs.

Brian> I remain skeptical that this has fixed the problem. What
Brian> happens if I send mail, with another control character, that
Brian> really is non-ASCII?

`\207' is only an example. 

Brian> Why does Gnus have to split the message when it sees a 'bad'
Brian> character?

Now Gnus treats all `bad character' (can not find a MIME charset) as
ASCII. Emacs does this.

Brian> Why can't it encode the entire message, in the same way?

If characters are not in one charset.

Brian> Also, another question: when I saw the my message (from the URL
Brian> previously given), Gnus used 8 bit encoding. I thought 8 bit
Brian> encoding meant no encoding at all, and was considered a bad
Brian> idea for non-ASCII characters? Where am I wrong?

For Gnus, 8 bit encoding meant no encoding at all. Some MTA's do not
like 8 bit encoding, so they translate it to base64 or QP.  Any other
opinions?

-- 
Shenghuo

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> Now Gnus treats all `bad character' (can not find a MIME charset) as
> ASCII.

I still hate that.

How do I switch this off?

Jan

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Per Abrahamsen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > Now Gnus treats all `bad character' (can not find a MIME charset) as
> > ASCII.
> 
> I still hate that.
> 
> How do I switch this off?

What would `switch it off' imply?  Sending it (e.g. Lisp backtraces)
as application/octet-stream?

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> What would `switch it off' imply?

It could start by asking me.

> Sending it (e.g. Lisp backtraces)
> as application/octet-stream?

Maybe.. Or I could replace all the offending character by octal
sequences.

Jan

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Per Abrahamsen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Per Abrahamsen <abraham@dina.kvl.dk> writes:
> 
> > What would `switch it off' imply?
> 
> It could start by asking me.

About what?

PS: What is breaking your references header?

References: <877liqx43y.fsf@senstation.vvf.fi> <2n66ya1xv3.fsf@tiger.jia.vnet> <2npuwiwsoe.fsf@tiger.jia.vnet> <t4id7shj5z7.fsf@silas-2.cc.monash.edu.au> <2n9034ude3.fsf@tiger.jia.vnet> <byn1rk9qwj.fsf@urysohn.math.ethz.ch> <rj9034mcb2.fsf@zuse.dina.kvl.d
From: Jan Vroonhof <vroonhof@math.ethz.ch>
Date: 09 Dec 1999 16:46:33 +0100
In-Reply-To: Per Abrahamsen's message of "Thu, 09 Dec 1999 14:14:48 GMT"
User-Agent: Gnus/5.070099 (Pterodactyl Gnus v0.99) XEmacs/21.1 (Bryce Canyon)

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

I want to be told something like "Message contains characters with
unknown encoding. Really send ?"

> PS: What is breaking your references header?

Our mailserver. Or better the stupid program they have here to forward
to the real one. I have asked our sysadmins to fix.

Jan

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "Jan" == Jan Vroonhof <vroonhof@math.ethz.ch> writes:

Jan> I want to be told something like "Message contains characters with
Jan> unknown encoding. Really send ?"

Done.

-- 
Shenghuo

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Kai Großjohann]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> For Gnus, 8 bit encoding meant no encoding at all. Some MTA's do not
> like 8 bit encoding, so they translate it to base64 or QP.  Any other
> opinions?

How many MTAs are still around which do not grok 8bit characters and
just strip off the high bit?  If there are still some, it would be
better not to send 8bit mails.

Hm.  But I think Gnus doesn't send 8bit mails anyway?  At least not by
default? 

kai
-- 
A preposition is not a good thing to end a sentence with.

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Aaron M. Ucko]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Gro_johann) writes:

> How many MTAs are still around which do not grok 8bit characters and
> just strip off the high bit?  If there are still some, it would be
> better not to send 8bit mails.

Some of MIT's mailhubs still run Sendmail 5 and strip high bits. :-/

> Hm.  But I think Gnus doesn't send 8bit mails anyway?  At least not by
> default? 

I just sent a test message with 5.8.2 containing 8-bit characters in
both the headers and the body.  The headers were properly RFC
2047-encoded, but Gnus sent the body as 8bit. :-/

-- 
Aaron M. Ucko, KB1CJC <amu@mit.edu> (finger amu@monk.mit.edu)

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Lloyd Zusman]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> >>>>> "Jan" == Jan Vroonhof <vroonhof@math.ethz.ch> writes:
> 
> Jan> I want to be told something like "Message contains characters with
> Jan> unknown encoding. Really send ?"
> 
> Done.

... and now, I get this prompt whenever I send normal, valid email
with some special Spanish-language characters in it.  Previously, Gnus
used to accept these types of messages and send them properly (I'm not
sure how they were sent, but some sort of valid MIME-izing was done).

What can I do to inhibit this prompt and make Gnus behave for me like
it used to when I would send messages containing these special
characters?  Is there some variable I can set to tell Gnus that
the characters in my message are valid?

Thanks in advance.

-- 
 Lloyd Zusman
 ljz@asfast.com

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Shenghuo ZHU]

>>>>> "Lloyd" == Lloyd Zusman <ljz@asfast.com> writes:

Lloyd> ... and now, I get this prompt whenever I send normal, valid
Lloyd> email with some special Spanish-language characters in it.
Lloyd> Previously, Gnus used to accept these types of messages and
Lloyd> send them properly (I'm not sure how they were sent, but some
Lloyd> sort of valid MIME-izing was done).

Lloyd> What can I do to inhibit this prompt and make Gnus behave for
Lloyd> me like it used to when I would send messages containing these
Lloyd> special characters?  Is there some variable I can set to tell
Lloyd> Gnus that the characters in my message are valid?

(setq mml-confirmation-set '(unknown-encoding use-ascii))

-- 
Shenghuo

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Hrvoje Niksic]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> > PS: What is breaking your references header?
> 
> Our mailserver. Or better the stupid program they have here to
> forward to the real one. I have asked our sysadmins to fix.

Try:

    (setq message-cater-to-broken-inn nil)

...and see if your mailserver likes that better.

References (was: Re: Brian May's `Re: Bugs in PGnus' article jams Gnus)

[Per Abrahamsen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Per Abrahamsen <abraham@dina.kvl.dk> writes:
> 
> > PS: What is breaking your references header?
> 
> Our mailserver. Or better the stupid program they have here to forward
> to the real one. I have asked our sysadmins to fix.

It is not the only such broken injection agent in existence.  Maybe
`maxsize' and `cut' below should be made a user options, in order to
allow workarounds?

(defun message-shorten-references (header references)
  "Trim REFERENCES to be less than 31 Message-ID long, and fold them.
If folding is disallowed, also check that the REFERENCES are less
than 988 characters long, and if they are not, trim them until they are."
  (let ((maxcount 31)
	(count 0)
	(cut 6)
	refs)
    ;; ...
    (when message-cater-to-broken-inn
      (let ((maxsize 988)
#

Re: References (was: Re: Brian May's `Re: Bugs in PGnus' article jams Gnus)

[Hrvoje Niksic]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> It is not the only such broken injection agent in existence.  Maybe
> `maxsize' and `cut' below should be made a user options, in order to
> allow workarounds?

And maxcount, etc.  It might make sense.  But there would be many of
them.  Maybe a plist that describes how the references will be
massaged would be nice.

(defvar message-shorten-references-parameters
  '(maxcount 31 cut 6 maxsize 988 ...)   ; only with nicer names
  "... detailed explanation for those ...")

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Try:
> 
>     (setq message-cater-to-broken-inn nil)
> 
> ...and see if your mailserver likes that better.

Then I have to run CVS update first :-). But that cuts the lines to
998 chars doesn't it? That stupid program here cuts to 255 chars.

Jan

P.S. I think I'll walk up to the sysadmins office personally now.

Broken References (was: Brian May's `Re: Bugs in PGnus' ...)

[Toby Speight]

Jan> Jan Vroonhof <URL:mailto:vroonhof@math.ethz.ch>

0> In article <byso1b14bp.fsf@bolzano.math.ethz.ch>, Jan wrote:

Jan> Hrvoje Niksic <hniksic@iskon.hr> writes:

>> Try:
>>  (setq message-cater-to-broken-inn nil)
>> ...and see if your mailserver likes that better.

Jan> But that cuts the lines to 998 chars doesn't it?  That stupid
Jan> program here cuts to 255 chars.

AIUI (I haven't tried the code, but I remember reading the patch),
`message-cater-to-broken-inn' is true by default, which means
truncation.  Setting it false makes Message wrap (to much less
than 255 chars) instead of truncating.

BTW, perhaps the default should be true for news only, and false for
mail?

Re: References (was: Re: Brian May's `Re: Bugs in PGnus' article jams Gnus)

[Per Abrahamsen]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Per Abrahamsen <abraham@dina.kvl.dk> writes:
> 
> > It is not the only such broken injection agent in existence.  Maybe
> > `maxsize' and `cut' below should be made a user options, in order to
> > allow workarounds?
> 
> And maxcount, etc.  It might make sense. 

Well, if 

        (setq  message-cater-to-broken-inn nil)

solves the problem, then it is probably not worth bothering.

> But there would be many of
> them.  Maybe a plist that describes how the references will be
> massaged would be nice.
> 
> (defvar message-shorten-references-parameters
>   '(maxcount 31 cut 6 maxsize 988 ...)   ; only with nicer names
>   "... detailed explanation for those ...")

Putting on my customize hat, I'd much prefer three simple integer
options collected in a group.  The KISS principle for user options.

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Per Abrahamsen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> Then I have to run CVS update first :-). But that cuts the lines to
> 998 chars doesn't it? That stupid program here cuts to 255 chars.

But is that header length, or line length?

If it is line length, setting `message-cater-to-broken-inn' to nil
will solve the problem.

> P.S. I think I'll walk up to the sysadmins office personally now.

I have never experienced a sysadmin who would do anything (useful)
unless I stood next to him.

Re: Brian May's `Re: Bugs in PGnus' article jams Gnus

[Jan Vroonhof]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> But is that header length, or line length?

line length

> If it is line length, setting `message-cater-to-broken-inn' to nil
> will solve the problem.

I am running 'cvs update' now. However that will only work for
outgoing messages.

The problem is that we are running Obtuse smptpd 2.0 which
INTENTIONALLY cuts message headers to length 255.
(this restriction is not documented, and hard coded in the source).
Our sysadmins have promised a recompile after they have made sure we
never depend on that behaviour for security.

I wish they stopped using it. It is also forcing unnecessary 8bit->QP
conversions. 

We have been running it if for quite some time now (the log files have
comments about the lines being cut from at least June), so I am not sure why
we are noticing the problem only now.

Maybe I should lobby for message-cater-to-broken-inn to be on for news
only.

Jan