Problem saving attachment with latest CVS HEAD Gnus

Problem saving attachment with latest CVS HEAD Gnus

[Nelson Ferreira]

Hi all,

I am using SXEmacs 22.1.6 patch 97, which as some may know is a
descendent of XEmacs 21.4.

I noticed problems saving an attachment of mime type
application/vnd.ms-powerpoint encoded in base64

I tracked it down to mm-insert-part calling mm-string-to-multibyte.
Previously it was calling mm-string-as-multibyte and when I changed it
back, it now works again.

I tracked the changed to revision 7.47 of lisp/mm-decode.el

I am not sure what the right fix is in this case, but it seems odd
that a mime type that could be binary is being attempted to be
interpreted as multibyte.


Any thoughts ?

Best regards,

     Nelson

I can send a sample of the message I got the problem, but since it is
1.4M gziped I am refraining from doing so to all.

-- 
Nelson Ferreira

Re: Problem saving attachment with latest CVS HEAD Gnus

[Katsumi Yamaoka]

>>>>> In <m34pst303h.fsf.nfl._-nfl---656d6163732e676e75732e67656e6572616c_---.00@tuxie.homelinux.net>
>>>>>	Nelson Ferreira wrote:

> Hi all,

> I am using SXEmacs 22.1.6 patch 97, which as some may know is a
> descendent of XEmacs 21.4.

Hi,

I tested it with XEmacs 21.4.19 and 21,5-b27 that have been
built from the recent CVS.  I found no problem with saving of
ppt attachments so far.

> I noticed problems saving an attachment of mime type
> application/vnd.ms-powerpoint encoded in base64

> I tracked it down to mm-insert-part calling mm-string-to-multibyte.
> Previously it was calling mm-string-as-multibyte and when I changed it
> back, it now works again.

> I tracked the changed to revision 7.47 of lisp/mm-decode.el

Does SXEmacs distinguish multibyte string and unibyte string as
Emacs does?  Though I don't think so.  As far as I understand,
there is no difference between `mm-string-as-multibyte' and
`mm-string-to-multibyte' in XEmacs except that the later takes
more time than the former.

> I am not sure what the right fix is in this case, but it seems odd
> that a mime type that could be binary is being attempted to be
> interpreted as multibyte.

> Any thoughts ?

> Best regards,

>      Nelson

> I can send a sample of the message I got the problem, but since it is
> 1.4M gziped I am refraining from doing so to all.

Please send it to me.  If there's still no problem with it and
XEmacs, I will try installing SXEmacs.

Regards,

Re: Problem saving attachment with latest CVS HEAD Gnus

[Katsumi Yamaoka]

>>>>> In <b4mirh9kzlp.fsf@jpl.org> Katsumi Yamaoka wrote:

> Please send it to me.  If there's still no problem with it and
> XEmacs, I will try installing SXEmacs.

I got an example message containing a ppt attachment from Nelson.
I was able to save and also view (using OpenOffice) it with
XEmacs 21.4.19 and 21.5-b27.  So, I installed SXEmacs --
"steve@sxemacs.org--2006/sxemacs--main--22.1.6--patch-119", and
tried it again.  And I confirmed it works fine.

Can anyone imagine what happens with Nelson?  You may probably
ask him to send an example.

BTW, I realized the `mm-string-to-multibyte' function takes time
very much in (S)XEmacs if a given string is large.  The
definition using `mapconcat' is for Emacs 21, but is nonsense to
(S)XEmacs.  So, I'm going to change it so as to be an aliase to
`identity' in (S)XEmacs, later.

Regards,

Re: Problem saving attachment with latest CVS HEAD Gnus

[Dave Goldberg]

>>>>> On Mon, 20 Nov 2006 23:17:06 -0500, Nelson Ferreira <nelson.ferreira@ieee.org> said:

> Hi all,

> I am using SXEmacs 22.1.6 patch 97, which as some may know is a
> descendent of XEmacs 21.4.

> I noticed problems saving an attachment of mime type
> application/vnd.ms-powerpoint encoded in base64

> I tracked it down to mm-insert-part calling mm-string-to-multibyte.
> Previously it was calling mm-string-as-multibyte and when I changed it
> back, it now works again.

> I tracked the changed to revision 7.47 of lisp/mm-decode.el

> I am not sure what the right fix is in this case, but it seems odd
> that a mime type that could be binary is being attempted to be
> interpreted as multibyte.


> Any thoughts ?

Just a data point: yesterday, after a bit of a hiatus I did a cvs update and got mm-decode.el 7.47 and just about all base64 encoded attachments became unsaveable.  I run a MULE-enabled XEmacs 21.4.19 on cygwin.  I'm not at work at the moment so can't provide the exact configuration.  I guessed (apparently correctly) that the issue was with mm-decode.el and backed out to 7.46 and the problem has gone away.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Problem saving attachment with latest CVS HEAD Gnus

[Katsumi Yamaoka]

>>>>> In <x71wnx6m5t.fsf@davestoy.homelinux.org> Dave Goldberg wrote:

> Just a data point: yesterday, after a bit of a hiatus I did a cvs
> update and got mm-decode.el 7.47 and just about all base64 encoded
> attachments became unsaveable.  I run a MULE-enabled XEmacs 21.4.19 on
> cygwin.  I'm not at work at the moment so can't provide the exact
> configuration.  I guessed (apparently correctly) that the issue was
> with mm-decode.el and backed out to 7.46 and the problem has gone
> away.

How does the most recent trunk work?  I've checked in in mm-util.el
in order to make `mm-string-to-multibyte' be an alias to `identity'
in (S)XEmacs.  mm-decode.el 7.47 uses it while 7.46 used
`mm-string-as-multibyte', which breaks binary data in Emacs 23.
Though I'm not quite sure (because I cannot reproduce the problem
with (S)XEmacs), the most recent change in mm-util.el at least makes
XEmacs work fast and may possibly solve the problem.

Regards,

Re: Problem saving attachment with latest CVS HEAD Gnus

[Dave Goldberg]

> How does the most recent trunk work?  I've checked in in mm-util.el
> in order to make `mm-string-to-multibyte' be an alias to `identity'
> in (S)XEmacs.  mm-decode.el 7.47 uses it while 7.46 used
> `mm-string-as-multibyte', which breaks binary data in Emacs 23.
> Though I'm not quite sure (because I cannot reproduce the problem
> with (S)XEmacs), the most recent change in mm-util.el at least makes
> XEmacs work fast and may possibly solve the problem.

This does appear to work fine now.  I'm again able to save and view all attachments using the CVS head.

Thanks!
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Problem saving attachment with latest CVS HEAD Gnus

[Katsumi Yamaoka]

>>>>> In <x74pssrh6g.fsf@davestoy.homelinux.org>
>>>>>	Dave Goldberg <david.goldberg6@verizon.net> wrote:

>> How does the most recent trunk work?

> This does appear to work fine now.  I'm again able to save and
> view all attachments using the CVS head.

> Thanks!

Thanks for verifying.

I'd like to know the cause to produce such an obstacle.  Does
anyone know what kind of string or what situation breaks STRING
by the following expression in XEmacs?

(mapconcat 'char-to-string "STRING" "")

Re: Problem saving attachment with latest CVS HEAD Gnus

[Nelson Ferreira]

Hi Katsumi,

First of all, let me thank you for the prompt response you gave the
issue, even to the point of getting and installing SXEmacs to attempt
to reproduce it.

"Katsumi" == Katsumi Yamaoka <yamaoka@jpl.org>  escreveu:

  >>>>>> In <x74pssrh6g.fsf@davestoy.homelinux.org>
  >>>>>> Dave Goldberg <david.goldberg6@verizon.net> wrote:

  >>> How does the most recent trunk work?

  >> This does appear to work fine now.  I'm again able to save and
  >> view all attachments using the CVS head.

  >> Thanks!

  Katsumi> Thanks for verifying.

Sorry for the delay in verifying. The changes in CVS head do indeed
solve the issue on the afflicted version of SXEmacs.

  Katsumi> I'd like to know the cause to produce such an obstacle.  Does
  Katsumi> anyone know what kind of string or what situation breaks STRING
  Katsumi> by the following expression in XEmacs?

  Katsumi> (mapconcat 'char-to-string "STRING" "")

We (SXEmacs team) did manage to reproduce it with this little snippet,

(let ((str (make-string 1600000 ?x))) 
  (mapconcat #'(lambda (el) el) str ""))

The issue as we traced it, has to do with the available amount of stack
available for the alloca calls that are made in mapconcat, which for a
1.6 MB string is strenous :)

We fixed this in SXEmacs in
steve@sxemacs.org--2006/sxemacs--main--22.1.6--patch-120 and we will
forward the patch to the XEmacs folks.

We tested the previous Gnus version with the fixed SXEmacs and all
worked fine. It will of course, as you expected previously, take a
little longer.

Best regards,

     Nelson

-- 
Nelson Ferreira

Re: Problem saving attachment with latest CVS HEAD Gnus

[Katsumi Yamaoka]

>>>>> In <m3lkm4ytm8.fsf.nfl._-nfl---656d6163732e676e75732e67656e6572616c_---.00@tuxie.homelinux.net>
>>>>>	Nelson Ferreira wrote:

> Hi Katsumi,

> First of all, let me thank you for the prompt response you gave the
> issue, even to the point of getting and installing SXEmacs to attempt
> to reproduce it.

You're welcome.  SXEmacs which I've never explored satisfied my
curiosity. ;-)

[...]

> We (SXEmacs team) did manage to reproduce it with this little snippet,

> (let ((str (make-string 1600000 ?x)))
>   (mapconcat #'(lambda (el) el) str ""))

> The issue as we traced it, has to do with the available amount of stack
> available for the alloca calls that are made in mapconcat, which for a
> 1.6 MB string is strenous :)

It is platform-dependent, isn't it?  In the Fedora Core 6 system,
I didn't get a non-nil value using 22.1.6--patch-119 and this form:

(string-match
 "[^x]"
 (mapconcat #'char-to-string (make-string 3200000 ?x) ""))
 => nil

And I also got:

(length (mapconcat #'char-to-string (make-string 3200000 ?x) ""))
 => 3200000

Anyway I've removed such one which forces heavy labor to (S)XEmacs.

> We fixed this in SXEmacs in
> steve@sxemacs.org--2006/sxemacs--main--22.1.6--patch-120 and we will
> forward the patch to the XEmacs folks.

I'll install the new version.  Thanks.

> We tested the previous Gnus version with the fixed SXEmacs and all
> worked fine. It will of course, as you expected previously, take a
> little longer.

> Best regards,

>      Nelson

Regards,