no more character sets free

no more character sets free

[Didier Verna]

       Hey there,

for some time now, I've had problems when attaching PDF files (probably
large ones). Here's an example of the error I get when trying to send
the message. Any idea what that means ?

I'm Cc'ing xemacs-beta@ because I suspect this is realted to XEmacs more
than to Gnus itself.


Thanks !



Debugger entered--Lisp error: (invalid-operation "No more character sets free for this dimension" 1)
  insert-file-contents-internal("~/Attestation_Logement.pdf" nil nil nil nil undecided used-codesys)
  byte-code("..." [buffer-file-coding-system-for-read coding-system-for-read return-val visit filename coding-system run-hook-with-args insert-file-contents-access-hook run-hook-with-args-until-success insert-file-contents-pre-hook find-file-coding-system-for-read-from-filename raw-text find-coding-system lwarn notice "Invalid coding-system (%s), using 'undecided" undecided insert-file-contents-internal used-codesys start end replace] 9)
  insert-file-contents("~/Attestation_Logement.pdf" nil nil nil nil)
  mm-insert-file-contents("~/Attestation_Logement.pdf" nil nil nil nil t)
  mml-compute-boundary-1((part (type . "application/pdf") (filename . "~/Attestation_Logement.pdf") (disposition . "attachment") (tag-location . 2161) (contents . "")))
  mapc(mml-compute-boundary-1 ((part (type . "text/plain") (contents . "\n\n-- \nResistance is futile. You will be jazzimilated.\n\nScientific site:   http://www.lrde.epita.fr/~didier\nMusic (Jazz) site: http://www.didierverna.com\n"))))
  mml-compute-boundary-1((multipart (type . "mixed") (part (type . "text/plain") (contents . "I wrote:\n\n> I should be able to provide that next week (tuesday or so). Sorry for\n> the delay.\n\n  Hello,\n\nplease find the accomodation attestation for both of you attached below.\nGood luck with the visa !\n\n")) (part (type . "application/pdf") (filename . "~/Attestation_Logement.pdf") (disposition . "attachment") (tag-location . 2161) (contents . "")) (part (type . "text/plain") (contents . "\n\n-- \nResistance is futile. You will be jazzimilated.\n\nScientific site:   http://www.lrde.epita.fr/~didier\nMusic (Jazz) site: http://www.didierverna.com\n"))))
  mml-compute-boundary((multipart (type . "mixed") (part (type . "text/plain") (contents . "I wrote:\n\n> I should be able to provide that next week (tuesday or so). Sorry for\n> the delay.\n\n  Hello,\n\nplease find the accomodation attestation for both of you attached below.\nGood luck with the visa !\n\n")) (part (type . "application/pdf") (filename . "~/Attestation_Logement.pdf") (disposition . "attachment") (tag-location . 2161) (contents . "")) (part (type . "text/plain") (contents . "\n\n-- \nResistance is futile. You will be jazzimilated.\n\nScientific site:   http://www.lrde.epita.fr/~didier\nMusic (Jazz) site: http://www.didierverna.com\n"))))
  mml-generate-mime-1((multipart (type . "mixed") (part (type . "text/plain") (contents . "I wrote:\n\n> I should be able to provide that next week (tuesday or so). Sorry for\n> the delay.\n\n  Hello,\n\nplease find the accomodation attestation for both of you attached below.\nGood luck with the visa !\n\n")) (part (type . "application/pdf") (filename . "~/Attestation_Logement.pdf") (disposition . "attachment") (tag-location . 2161) (contents . "")) (part (type . "text/plain") (contents . "\n\n-- \nResistance is futile. You will be jazzimilated.\n\nScientific site:   http://www.lrde.epita.fr/~didier\nMusic (Jazz) site: http://www.didierverna.com\n"))))
  mml-generate-mime()
  message-encode-message-body()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit()
  gnus-draft-send(1 "nndraft:drafts" t)
  gnus-draft-send-message(nil)
  call-interactively(gnus-draft-send-message)


-- 
Resistance is futile. You will be jazzimilated.

Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com

no more character sets free

[Stephen J. Turnbull]

Didier Verna writes:
 > 
 >        Hey there,
 > 
 > for some time now, I've had problems when attaching PDF files (probably
 > large ones). Here's an example of the error I get when trying to send
 > the message. Any idea what that means ?
 > 
 > Debugger entered--Lisp error: (invalid-operation "No more character sets free for this dimension" 1)
 >   insert-file-contents-internal("~/Attestation_Logement.pdf" nil nil nil nil undecided used-codesys)

"undecided" is probably the culprit here.  I'd be interested to know
what "used-codesys" turns out to be.

I don't understand Aidan's JIT charset mechanism, so you'll need to
check with him on whether the following scenario is really possible.
PDF files are mostly ASCII for the first 2-4k, so they may be detected
as Shift_JIS or some such.  Then in attempting to convert the rest of
the file you'll encounter piles of random non-JIS characters, and
creating JIT charsets for those could fill up a bunch of one-byte
charsets pretty fast.

Even if that's not correct, it's worth trying something like
consing ("\\.pdf$" . binary) on to file-coding-system-alist to force
binary for PDFs.  There might be something more specific to
Gnus/message mode/mml, of course.

Re: no more character sets free

[Didier Verna]

Stephen J. Turnbull wrote:

> Even if that's not correct, it's worth trying something like consing
> ("\\.pdf$" . binary) on to file-coding-system-alist to force binary
> for PDFs.

  This "fixes" the problem indeed, thanks.

-- 
Resistance is futile. You will be jazzimilated.

Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com

Re: no more character sets free

[Stephen J. Turnbull]

Didier Verna writes:
 > Stephen J. Turnbull wrote:
 > 
 > > Even if that's not correct, it's worth trying something like consing
 > > ("\\.pdf$" . binary) on to file-coding-system-alist to force binary
 > > for PDFs.
 > 
 >   This "fixes" the problem indeed, thanks.

Cool!

I'm not sure where to go from there, though.  Suggestions welcome
(preferably on the XEmacs issue tracker, since I know I won't have
time to do anything about it for a while, and it doesn't seem hugely
pressing -- and no, I'm not sure adding (pdf . binary) as above is
a good idea; I have had occasion to look at PDFs as text).

Reply-To set to xemacs-beta; it's arguable that this is not a Gnus
bug, you guys can fix it or not as you judge.

Re: no more character sets free

[Aidan Kehoe]

 Ar an séú lá is fiche de mí Bealtaine, scríobh Stephen J. Turnbull: 

 > Didier Verna writes:
 > 
 >  > [...] I've had problems when attaching PDF files (probably large
 >  > ones). Here's an example of the error I get when trying to send the
 >  > message. Any idea what that means ?
 >  > 
 >  > Debugger entered--Lisp error: (invalid-operation "No more character sets free for this dimension" 1)
 >  >   insert-file-contents-internal("~/Attestation_Logement.pdf" nil nil nil nil undecided used-codesys)
 > 
 > "undecided" is probably the culprit here.  I'd be interested to know
 > what "used-codesys" turns out to be.
 > 
 > I don't understand Aidan's JIT charset mechanism, so you'll need to
 > check with him on whether the following scenario is really possible.
 > PDF files are mostly ASCII for the first 2-4k, so they may be detected
 > as Shift_JIS or some such.  Then in attempting to convert the rest of
 > the file you'll encounter piles of random non-JIS characters, and
 > creating JIT charsets for those could fill up a bunch of one-byte
 > charsets pretty fast.

You’re confusing my Unicode JIT charsets, which were always called that,
with the ISO-2022-compatible on-the-fly allocation of charsets once an
unknown leading byte is encountered, which are a similar concept but were
never called that. All the Unicode JIT charsets are of dimension two, for
example.

And yes, the autodection-as-iso-2022-compatible, then exhaustion of the
available characters sets is the culprit. It’s as well, too, without that
error things would have just been randomly, silently corrupted for Didier.

-- 
“Apart from the nine-banded armadillo, man is the only natural host of
Mycobacterium leprae, although it can be grown in the footpads of mice.”
  -- Kumar & Clark, Clinical Medicine, summarising improbable leprosy research

_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta@calypso.tux.org
http://calypso.tux.org/mailman/listinfo/xemacs-beta

Re: no more character sets free

[Aidan Kehoe]

 Ar an séú lá is fiche de mí Bealtaine, scríobh Stephen J. Turnbull: 

 > I'm not sure where to go from there, though.  Suggestions welcome
 > (preferably on the XEmacs issue tracker, since I know I won't have
 > time to do anything about it for a while, and it doesn't seem hugely
 > pressing -- and no, I'm not sure adding (pdf . binary) as above is
 > a good idea; I have had occasion to look at PDFs as text).

I think it’s a good idea. Text in PDF files is probably in a font encoding,
which we don’t understand, typically; and there will be lots of octets from
all over the eight-bit space for binary objects, and I’m not about to write
a PDF parser that understands what’s a binary object and what isn’t for the
sake of the autodetection.

 > Reply-To set to xemacs-beta; it's arguable that this is not a Gnus
 > bug, you guys can fix it or not as you judge.

Gnus needs to handle charsets on the MIME level, and it should be forcing a
known encoding when reading in things as MIME attachments that are not plain
text or, at a stretch, HTML. Is there a reason you’re not binding
coding-system-for-read, Gnus? Or is our XEmacs package version just out of
date?

-- 
“Apart from the nine-banded armadillo, man is the only natural host of
Mycobacterium leprae, although it can be grown in the footpads of mice.”
  -- Kumar & Clark, Clinical Medicine, summarising improbable leprosy research

_______________________________________________
XEmacs-Beta mailing list
XEmacs-Beta@calypso.tux.org
http://calypso.tux.org/mailman/listinfo/xemacs-beta

Re: no more character sets free

[Didier Verna]

Aidan Kehoe <kehoea@parhasard.net> wrote:

> Gnus needs to handle charsets on the MIME level, and it should be
> forcing a known encoding when reading in things as MIME attachments
> that are not plain text or, at a stretch, HTML. Is there a reason
> you’re not binding coding-system-for-read, Gnus? Or is our XEmacs
> package version just out of date?

  I haven't looked at it yet, but I generally use a fairly recent
version of it; not our package version.

-- 
Resistance is futile. You will be jazzimilated.

Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com