Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Sat, Jan 28 2006, Stephen J. Turnbull wrote:

>>>>>> "Mike" == Mike Kupfer <mike.kupfer@sun.com> writes:
>
>     Mike> I wasn't sure if I should start here or on the gnus bug
>     Mike> list...
>
> I would say gnus, since you get it with xemacs -vanilla (which means
> that latin-unity isn't involved).

I think some change from Aidan (added after the 5.10.6 release) loads
latin-unity from Gnus:

,----[ gnus/lisp/mm-util.el ]
| (defun mm-xemacs-find-mime-charset-1 (begin end)
|   "Determine which MIME charset to use to send region as message.
| This uses the XEmacs-specific latin-unity package to better handle the
| case where identical characters from diverse ISO-8859-? character sets
| can be encoded using a single one of the corresponding coding systems.
| [...]
|     ;; Load the Latin Unity library, if available.
|     (when (and (not (featurep 'latin-unity)) (locate-library "latin-unity"))
|       (ignore-errors (require 'latin-unity)))
`----

>     Mike> and there's a message in the minibuffer saying
>     Mike> Unknown charset: ISO-8859-1
>     Mike> With gnus 5.10.6 (XEmacs package 1.80) the From is rendered
>     Mike> as expected, with an accented character.
>
> I'm seeing a number of anomolies with Gnus, but since I'm running
> bleeding edge CVS, I've not been too eager to dive into Gnus code; I'm
> currently batting about 2 for 15 even localizing problems with Gnus.
>
> I would say that posting to ding is a good idea because they're more
> likely to be able to localize it with a small effort.  Even if they
> point the finger back here. ;-)

Sure. ;-)

BTW, it would be good if the XEmacs Gnus package could provide the
date when the package was synched with the Gnus v5-10 branch.  Without
a date, "5.10.7" is rather pointless.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Steve Youngs]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

* Reiner Steib <reinersteib+gmane@imap.cc> writes:

  > BTW, it would be good if the XEmacs Gnus package could provide the
  > date when the package was synched with the Gnus v5-10 branch.  Without
  > a date, "5.10.7" is rather pointless.

(package-get-info 'gnus 'date)
 => 2005-11-15

That would come pretty close. :-)

-- 
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
|                 I am Dyslexic of Borg.                   | 
|    Fusistance is retile. Your arse will be laminated.    |
|------------------------------------<steve@sxemacs.org>---|

[-- Attachment #2: Type: application/pgp-signature, Size: 256 bytes --]

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Sun, Jan 29 2006, Steve Youngs wrote:

> * Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>   > BTW, it would be good if the XEmacs Gnus package could provide the
>   > date when the package was synched with the Gnus v5-10 branch.  Without
>   > a date, "5.10.7" is rather pointless.
>
> (package-get-info 'gnus 'date)
>  => 2005-11-15
>
> That would come pretty close. :-)

Thanks.

Could you suggest a way to check the date of the package without
XEmacs?  Maybe an URL to the package ChangeLog or similar?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Stephen J. Turnbull]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

    Reiner> On Sun, Jan 29 2006, Steve Youngs wrote:

> (package-get-info 'gnus 'date)
>  => 2005-11-15
>
> That would come pretty close. :-)

    Reiner> Could you suggest a way to check the date of the package
    Reiner> without XEmacs?  Maybe an URL to the package ChangeLog or
    Reiner> similar?

In general, using XEmacs is the right way, because that will tell you
what the bug reporter has installed.  If you want to check what's
current, all of that information is in the package-index,

ftp://ftp.xemacs.org/pub/xemacs/packages/package-index.LATEST.gpg

a file of about 80kB, so probably not too expensive to return.  That's
just a LISP file containing an alist and some syntactic sugar, so you
can `load' it and extract only the package information you need with
any Emacsen.  You can also typically check out what was in the last
few releases by looking for the dated versions of package-index in the
same directory.

The package ChangeLog is available from ViewCVS at http://cvs.xemacs.org/.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

Reiner> Could you suggest a way to check the date of the package without
Reiner> XEmacs?  Maybe an URL to the package ChangeLog or similar?

If you've got the mail generated by report-xemacs-bug, it has the
version information for all the installed packages.  In this case, it's

(gnus ver: 1.87 upstream: 5.10.7)

You could then track this back with the package ChangeLog.

2005-11-15  Norbert Koch  <viteno@xemacs.org>

	* Makefile (VERSION): XEmacs package 1.87 released.

2005-11-15  Steve Youngs  <steve@sxemacs.org>

	* Sync with upstream stable branch.
	Please see the ChangeLog.upstream files for details.

2005-10-12  Norbert Koch  <viteno@xemacs.org>

	* Makefile (VERSION): XEmacs package 1.86 released.

Does that help?

cheers,
mike

where does package-get-info get its info from? (was: gnus 5.10.7 not rendering Latin-1 headers)

[Mike Kupfer]

SY> (package-get-info 'gnus 'date)
SY>  => 2005-11-15

Hmm.  When I do that, I get

(package-get-info 'gnus 'date)
"2003-10-13"

(package-get-info 'gnus 'author-version)
"5.10.2"

I thought it might be getting that from
$HOME/.xemacs/package-index.LATEST.gpg (which hasn't been updated since
22 September 2004).  But if I blow away that file and invoke
package-get-info in a new XEmacs process, it creates a new
package-index.LATEST.gpg and gives me the same results.

The packages that I'm actually running are from a sumo that I unpacked
in /usr/local/lib/xemacs/xemacs-packages.

mike

Re: where does package-get-info get its info from?

[Steve Youngs]

[-- Attachment #1: Type: text/plain, Size: 2876 bytes --]

* Mike Kupfer <kupfer@athyra.sfbay.sun.com> writes:

  SY>  (package-get-info 'gnus 'date)
  SY>  => 2005-11-15

  > Hmm.  When I do that, I get

  > (package-get-info 'gnus 'date)
  > "2003-10-13"

  > (package-get-info 'gnus 'author-version)
  > "5.10.2"

...because you have _that_ version installed. :-)

  > I thought it might be getting that from
  > $HOME/.xemacs/package-index.LATEST.gpg

That's exactly where the info originates from.  Well, some of the info
does.  Some of it comes from autoloaded forms in `_pkg.el' files in
each package lisp directory.

  > But if I blow away that file and invoke package-get-info in a new
  > XEmacs process, it creates a new package-index.LATEST.gpg and
  > gives me the same results.

Because the package-index file it created was the same as the one you
blew away.  It'd just be a copy of the file that is in
/usr/local/lib/xemacs-x.x.x/etc/.  And...

  > The packages that I'm actually running are from a sumo that I
  > unpacked in /usr/local/lib/xemacs/xemacs-packages.

...because you are a "Sumo-installer" and not a "PUI-installer", your
package-index file never gets updated.  Yes, I know that seems like a
bug or flaw in the way XEmacs handles package management.  But really
it isn't.

Sumo's are _not_ part of and have no knowledge of package management.
Installing a Sumo would be like getting a RPM package, converting it to
a .tar.gz and installing it with `tar zxf file.tar.gz'.  The package
would install and work fine, but the underlying RPM database would have
no knowledge of it.

So, the moral to this story is: If you want the XEmacs PUI tools to
give you the best results, always install/remove/update your packages
with PUI.  Sure, use the Sumo's to set up a virgin installation, but
once the Sumo is installed, continue maintaining the packages with PUI
From that point on.

I use the following incantation to keep all of my packages up to
date... 

  Tools -> Packages -> Set Download Site ...
  Tools -> Packages -> Update Installed Packages

This elisp should also work (untested)...

,----
| (defun sy-update-my-pkgs ()
|   "Just an easy way to update installed packages."
|   (interactive)
|   (let ((package-get-remote '("ftp.xemacs.org" "pub/xemacs/packages")))
|     (package-get-update-all)))
`----

And while I'm on a roll with the recipes, this will make
`package-get-info' use a package-index from a remote package
mirror... 

,----
| (let ((package-get-remote '("ftp.xemacs.org" "pub/xemacs/packages")))
|   (package-get-info 'gnus 'date nil 'remote))
`----


-- 
|---<Steve Youngs>---------------<GnuPG KeyID: A94B3003>---|
|                 I am Dyslexic of Borg.                   | 
|    Fusistance is retile. Your arse will be laminated.    |
|------------------------------------<steve@sxemacs.org>---|

[-- Attachment #2: Type: application/pgp-signature, Size: 256 bytes --]

Date of XEmacs' Gnus (5.10.7) package (was: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers)

[Reiner Steib]

On Mon, Jan 30 2006, Stephen J. Turnbull wrote:

>>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>     Reiner> On Sun, Jan 29 2006, Steve Youngs wrote:
>
>> (package-get-info 'gnus 'date)
>>  => 2005-11-15
>>
>> That would come pretty close. :-)
>
>     Reiner> Could you suggest a way to check the date of the package
>     Reiner> without XEmacs?  Maybe an URL to the package ChangeLog or
>     Reiner> similar?
>
> In general, using XEmacs is the right way, because that will tell you
> what the bug reporter has installed.  If you want to check what's
> current, all of that information is in the package-index,
>
> ftp://ftp.xemacs.org/pub/xemacs/packages/package-index.LATEST.gpg

(progn (url-insert-file-contents
	"ftp://ftp.xemacs.org/pub/xemacs/packages/package-index.LATEST.gpg")
       (re-search-forward "^(gnus"))

... fine:

,----
| (package-get-update-base-entry (quote
| (gnus
|   (standards-version 1.1
|    version "1.89"
|    author-version "5.10.7"
|    date "2006-01-04"
|    build-date "2006-01-04"
`----

On Tue, Jan 31 2006, Mike Kupfer wrote:

(Mike, your article has a bogus References header:

,----
| Message-ID: <200601302324.k0UNOLos019384@athyra.sfbay.sun.com>
| References: <reinersteib+gmane@imap.cc>
| [...]
| In-Reply-To: Message from Reiner Steib <reinersteib+gmane@imap.cc>    of "Sun, 29 Jan 2006 23:40:15 +0100." <v9y80yu08g.fsf@marauder.physik.uni-ulm.de> 
| X-Mailer: MH-E 7.4.2; nmh 1.0.4; XEmacs 21.4 (patch 18)
`----

MH-E bug?)

> If you've got the mail generated by report-xemacs-bug, it has the
> version information for all the installed packages.  In this case, it's
>
> (gnus ver: 1.87 upstream: 5.10.7)
>
> You could then track this back with the package ChangeLog.

(url-insert-file-contents "http://cvs.xemacs.org/viewcvs.cgi/*checkout*/XEmacs/packages/xemacs-packages/gnus/ChangeLog?rev=HEAD&content-type=text/plain")

Good.

> 2005-11-15  Norbert Koch  <viteno@xemacs.org>
>
> 	* Makefile (VERSION): XEmacs package 1.87 released.
>
> 2005-11-15  Steve Youngs  <steve@sxemacs.org>
>
> 	* Sync with upstream stable branch.
> 	Please see the ChangeLog.upstream files for details.
[...]
> Does that help?

Yes, I hope so.  Thanks guys.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

>> I would say gnus, since you get it with xemacs -vanilla (which means
>> that latin-unity isn't involved).

Reiner> I think some change from Aidan (added after the 5.10.6 release)
Reiner> loads latin-unity from Gnus:

I didn't have latin-unity installed, since it's not in the sumo.  I
tried installing latin-unity and latin-euro-standards (which latin-unity
depends on), but latin-euro-standards fails to load.  It says it can't
find find-charset.  Is this a MULE thing?  I'm running non-MULE XEmacs.

I also isolated the problem[1] a little further.  

    (mm-coding-system-p 'iso-8859-1) 

is returning nil.  This is because find-coding-system and
coding-system-p are both unbound, and mm-coding-system-list (the
function) is aliased to "ignore".

Where do I go from here?

thanks,
mike
--
[1] see http://list-archive.xemacs.org/xemacs-beta/200601/msg00345.html
    for details.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Stephen J. Turnbull]

>>>>> "Mike" == Mike Kupfer <kupfer@athyra.sfbay.sun.com> writes:

    Mike> Is this a MULE thing?  I'm running non-MULE XEmacs.

Yes, latin-unity depends on MULE.

    Mike> I also isolated the problem[1] a little further.

    Mike>     (mm-coding-system-p 'iso-8859-1)

    Mike> is returning nil.  This is because find-coding-system and
    Mike> coding-system-p are both unbound, and mm-coding-system-list
    Mike> (the function) is aliased to "ignore".

    Mike> Where do I go from here?

ding.  It's a Gnus issue (alternatively, they can say they don't
support no-MULE XEmacs any more).  It's possible that we should also
make an adjustment, but since they've got compatibility code in place,
they're going to be the best bet for a diagnosis.

There's an outside chance that configuring XEmacs --with-file-coding
will help.  (Check the output of .configure --help, the option names
have changed with the move to autoconf 2.59 in 21.5, and I don't
remember the 21.4 version for sure).  However, I don't think that will
actually know that 'iso-8859-1 is a coding system, and you'll end up
in the same place.



-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Sat, Feb 18 2006, Stephen J. Turnbull wrote:

>>>>>> "Mike" == Mike Kupfer <kupfer@athyra.sfbay.sun.com> writes:
>     Mike> I also isolated the problem[1] a little further.
>
>     Mike>     (mm-coding-system-p 'iso-8859-1)
>
>     Mike> is returning nil.  This is because find-coding-system and
>     Mike> coding-system-p are both unbound, and mm-coding-system-list
>     Mike> (the function) is aliased to "ignore".
>
>     Mike> Where do I go from here?
>
> ding.  It's a Gnus issue (alternatively, they can say they don't
> support no-MULE XEmacs any more).  It's possible that we should also
> make an adjustment, but since they've got compatibility code in
> place, they're going to be the best bet for a diagnosis.

I don't see any reason not to improve compatibility in `mm*.el' for
XEmacs (MULE or non-MULE).  But since most of the currently active
Gnus developers don't use XEmacs, I'm afraid this won't happen unless
you (i.e. XEmacs developers and users) contribute patches or concrete
proposals.

(I don't even have a non-MULE XEmacs available for cursory tests.)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

Reiner> I don't see any reason not to improve compatibility in `mm*.el'
Reiner> for XEmacs (MULE or non-MULE).  But since most of the currently
Reiner> active Gnus developers don't use XEmacs, I'm afraid this won't
Reiner> happen unless you (i.e. XEmacs developers and users) contribute
Reiner> patches or concrete proposals.

I'm willing to help as best I can, but I could use some clues, either
from the gnus or XEmacs folks.

mm-coding-system-p appears to be trying to figure out what coding system
interface to use, with the choices being find-coding-system,
coding-system-p, and mm-get-coding-system-list.  Are any of these
standard Emacs or XEmacs interfaces?  

The docstring for mm-coding-system-p says it returns "a coding system
object in XEmacs".  The XEmacs Lisp reference talks about coding
systems, under MULE.  Does that mean that coding system objects only
exist in MULE XEmacs?  What should (mm-coding-system-p 'iso-8859-1)
return in non-MULE XEmacs?

mike

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Sun, Feb 19 2006, Mike Kupfer wrote:

> mm-coding-system-p appears to be trying to figure out what coding system
> interface to use, 

[ This is the relevant code: ]

,----[ `mm-util.el' ]
| (defun mm-coding-system-p (cs)
|   "Return non-nil if CS is a symbol naming a coding system.
| In XEmacs, also return non-nil if CS is a coding system object.
| If CS is available, return CS itself in Emacs, and return a coding
| system object in XEmacs."
|   (if (fboundp 'find-coding-system)
|       (and cs (find-coding-system cs))
|     (if (fboundp 'coding-system-p)
| 	(when (coding-system-p cs)
| 	  cs)
|       ;; Is this branch ever actually useful?
|       (car (memq cs (mm-get-coding-system-list))))))
`----

So "Is this branch ever actually useful?" is used in non-MULE XEmacs.

> with the choices being find-coding-system, coding-system-p, and
> mm-get-coding-system-list.  Are any of these standard Emacs or
> XEmacs interfaces?

`coding-system-p' is a standard Emacs interface:

,----[ (info "(elisp)Lisp and Coding Systems") ]
|  -- Function: coding-system-p object
|      This function returns `t' if OBJECT is a coding system name or
|      `nil'.
`----

The function `find-coding-system' doesn't exist in Emacs 21/22.

In case it wasn't obvious: mm-coding-system-list (the function) is
aliased to "ignore" if (fbound 'coding-system-list) is nil.

Leaving this for the XEmacs folks...
> The docstring for mm-coding-system-p says it returns "a coding system
> object in XEmacs".  The XEmacs Lisp reference talks about coding
> systems, under MULE.  Does that mean that coding system objects only
> exist in MULE XEmacs?  What should (mm-coding-system-p 'iso-8859-1)
> return in non-MULE XEmacs?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Katsumi Yamaoka]

>>>>> In <v9irrb9bna.fsf@marauder.physik.uni-ulm.de> Reiner Steib wrote:

> Leaving this for the XEmacs folks...
>> The docstring for mm-coding-system-p says it returns "a coding system
>> object in XEmacs".  The XEmacs Lisp reference talks about coding
>> systems, under MULE.  Does that mean that coding system objects only
>> exist in MULE XEmacs?  What should (mm-coding-system-p 'iso-8859-1)
>> return in non-MULE XEmacs?

I have five XEmacsen installed.  Those are:

1. 21.4.19 with mule with file-coding
2. 21.4.19 with mule without file-coding
3. 21.4.19 without mule without file-coding
4. 21.5-b24 with mule with file-coding
5. 21.5-b24 without mule with file-coding

Only 3 doesn't have coding system functions including
`decode-coding-string', `encode-coding-region', etc., and mm-util
makes them all aliases to `identity', `ignore' or equivalents.
Even so, if I understand it correctly, that version of XEmacs
displays 8-bit data as Latin-1 characters.  For example:

(insert "\310")
È

(mm-decode-coding-string "\310" 'iso-8859-1)
"È"

(mm-decode-coding-string "\310" 'any-coding-system)
"È"

The last example suggests we can make `mm-coding-system-p' return
any value, that is, it doesn't have to be a coding system object.
Actually, the following form works:

(let ((mm-coding-system-list '(iso-8859-1)))
  (rfc2047-decode-string
   "=?ISO-8859-1?Q?Ignacio_Marambio_Cat=E1n?="))
"Ignacio Marambio Catán"

So, changing

(defvar mm-coding-system-list nil)

to

(defvar mm-coding-system-list '(iso-8859-1))

in mm-util.el will solve at least the problem that Mike Kupfer
first brought up.  Is there any coding systems which should be
added to the default value other than iso-8859-1 that XEmacs
without mule without file-coding supports?

I'm not sure it solves all problems with Gnus and that version
of XEmacs, though.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Stephen J. Turnbull]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

    Reiner> In case it wasn't obvious: mm-coding-system-list (the
    Reiner> function) is aliased to "ignore" if (fbound
    Reiner> 'coding-system-list) is nil.

So mm-coding-system-p passes nil back to Gnus.

    Reiner> Leaving this for the XEmacs folks...

    >> What should (mm-coding-system-p 'iso-8859-1) return in non-MULE
    >> XEmacs?

nil, according to Reiner's description of the design.  So Gnus is
receiving nil as a result from an internal Gnus function, and refusing
to handle that value, which eventually results in an error and buggy
display.

This is definitely not an XEmacs problem---it's all internal to Gnus.

So I guess Reiner is saying that no-MULE XEmacs is not supported by
Gnus any more.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Katsumi Yamaoka]

>>>>> In <b4mirrao4t0.fsf@jpl.org> Katsumi Yamaoka wrote:

> So, changing

> (defvar mm-coding-system-list nil)

> to

> (defvar mm-coding-system-list '(iso-8859-1))

> in mm-util.el will solve at least the problem that Mike Kupfer
> first brought up.

That's a wrong approach.  It should be something like:

(defvar mm-coding-system-list nil)
(defun mm-get-coding-system-list ()
  "Get the coding system list."
  (or mm-coding-system-list
      (setq mm-coding-system-list
	    (or (mm-coding-system-list) '(iso-8859-1)))))

> Is there any coding systems which should be added to the
> default value other than iso-8859-1 that XEmacs without mule
> without file-coding supports?

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Mon, Feb 20 2006, Stephen J. Turnbull wrote:

>>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>     Reiner> In case it wasn't obvious: mm-coding-system-list (the
>     Reiner> function) is aliased to "ignore" if (fbound
>     Reiner> 'coding-system-list) is nil.
>
> So mm-coding-system-p passes nil back to Gnus.

Yes, this is what the current mm-util.el code does in Mike's XEmacs.

>     Reiner> Leaving this for the XEmacs folks...
>
>     >> What should (mm-coding-system-p 'iso-8859-1) return in non-MULE
>     >> XEmacs?
>
> nil, according to Reiner's description of the design.  So Gnus is
> receiving nil as a result from an internal Gnus function, and refusing
> to handle that value, which eventually results in an error and buggy
> display.

Huh, I didn't describe the design, I only described the current code.
I don't know if the current code is correct for XEmacs or gives the
desired result there.

> This is definitely not an XEmacs problem---it's all internal to Gnus.
>
> So I guess Reiner is saying that no-MULE XEmacs is not supported by
> Gnus any more.

Stephen, I don't understand this finger pointing.  It doesn't bring us
any further.  If you look at Gnus' ChangeLog you won't find any
changes aiming to stop supporting no-MULE XEmacs.  There are only very
few XEmacs specific changes in that area, AFAICS.  Most of these were
patches from Aidan WRT latin-unity (I don't know if those are
responsible for Mike's problems or not).  If support of no-MULE XEmacs
has been broken, it wasn't done on purpose and as I wrote in my
previous mail, I'm all for fixing it if someone gives us an idea how
to fix it.

BTW, we are talking about the Gnus 5.10.x series which (compared to
5.10.1) should not contain any new features (only bugfixes).  We
didn't even officially drop support for Emacs 20.7 and XEmacs 21.1, see
<http://thread.gmane.org/v9y80ega32.fsf_-_@marauder.physik.uni-ulm.de>.
But I doubt that anyone verified that the current v5-10 branch works
on these Emacsen.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Katsumi Yamaoka]

At last I realized the problem is due to my change:

2005-10-19  Katsumi Yamaoka  <yamaoka@jpl.org>

	* rfc2047.el (rfc2047-allow-incomplete-encoded-text): New variable.
	(rfc2047-charset-to-coding-system): New function.
	(rfc2047-decode-encoded-words): New function.
	(rfc2047-decode-region): Use them.
	(rfc2047-decode-cte): Remove.
	(rfc2047-parse-and-decode): Remove.
	(rfc2047-decode): Remove.

Formerly `rfc2047-decode' didn't verify whether a given charset
is valid, while `rfc2047-charset-to-coding-system' (which merged
the feature of `rfc2047-decode' partially) does it now using
`mm-coding-system-p'.  However, it should be left to
`mm-charset-to-coding-system' even if it is incomplete[1] now.
Therefore, I've made a change in both the trunk and the v5-10
branch.

[1] The form (mm-charset-to-coding-system 'foo-bar-baz) returns
foo-bar-baz in non-Mule XEmacs 21.4.

The following diff is for the latest Gnus XEmacs package:

2006-02-20  Katsumi Yamaoka  <yamaoka@jpl.org>

	* rfc2047.el (rfc2047-charset-to-coding-system): Don't check the
	coding system which mm-charset-to-coding-system returns for a
	given charset is valid.

--- rfc2047.el~	2005-12-19 22:31:44 +0000
+++ rfc2047.el	2006-02-20 11:29:44 +0000
@@ -835,7 +835,7 @@
     (cond ((eq cs 'ascii)
 	   (setq cs (or (mm-charset-to-coding-system mail-parse-charset)
 			'raw-text)))
-	  ((setq cs (mm-coding-system-p cs)))
+	  ((mm-coding-system-p cs))
 	  ((and charset
 		(listp mail-parse-ignored-charsets)
 		(memq 'gnus-unknown mail-parse-ignored-charsets))

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Stephen J. Turnbull]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

    Reiner> Huh, I didn't describe the design,

OK.  Can you point me to a design document?

    >> This is definitely not an XEmacs problem---it's all internal to
    >> Gnus.

    >> So I guess Reiner is saying that no-MULE XEmacs is not
    >> supported by Gnus any more.

    Reiner> Stephen, I don't understand this finger pointing.

You described a situation that clearly is entirely a Gnus problem,
then said "it's up to the XEmacs guys."  You can't say you support
XEmacs and at the same time pass the buck that way.

BTW, there's nothing wrong with abandoning support for legacy code, or
telling the users that they have to support it themselves.  You've got
a finite amount of time, and maybe you've got something better to do
with that time than to deal with the legacy no-Mule configuration that
_your_ Emacs hasn't supported for about 4 years now.  I understand
that, and I'm sure Mike does too, even if we don't like it.

I'd just like to be able to tell Mike where he stands.

    Reiner> I'm all for fixing it if someone gives us an idea how to
    Reiner> fix it.

Fortunately, Katsumi Yamaoka, has something to say about the issue.
So it's a Gnus developer who's going to fix it after all.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

>>>>> "Katsumi" == Katsumi Yamaoka <yamaoka@jpl.org> writes:

Katsumi> The following diff is for the latest Gnus XEmacs package:

Thanks!  I'll give that a try tomorrow (today's a US holiday) and let
you know how it goes.

cheers,
mike

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Miles Bader]

Christ Stephen, stop being so damn pissy...

-miles
-- 
Come now, if we were really planning to harm you, would we be waiting here,
 beside the path, in the very darkest part of the forest?

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

> --- rfc2047.el~	2005-12-19 22:31:44 +0000
> +++ rfc2047.el	2006-02-20 11:29:44 +0000
> @@ -835,7 +835,7 @@
>      (cond ((eq cs 'ascii)
>  	   (setq cs (or (mm-charset-to-coding-system mail-parse-charset)
>  			'raw-text)))
> -	  ((setq cs (mm-coding-system-p cs)))
> +	  ((mm-coding-system-p cs))
>  	  ((and charset
>  		(listp mail-parse-ignored-charsets)
>  		(memq 'gnus-unknown mail-parse-ignored-charsets))

Yes, this works.

Thanks!

mike

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Aidan Kehoe]

 Ar an fichiú lá de mí Feabhra, scríobh Reiner Steib>: 

 > Stephen, I don't understand this finger pointing.  It doesn't bring us
 > any further.  If you look at Gnus' ChangeLog you won't find any
 > changes aiming to stop supporting no-MULE XEmacs.  There are only very
 > few XEmacs specific changes in that area, AFAICS.  Most of these were
 > patches from Aidan WRT latin-unity (I don't know if those are
 > responsible for Mike's problems or not).  

All my Gnus code that touches on latin-unity is wrapped in this (it’s in the
mm-xemacs-find-mime-charset-1 function):

(defmacro mm-xemacs-find-mime-charset (begin end)
  (when (featurep 'xemacs)
    `(and (featurep 'mule) (mm-xemacs-find-mime-charset-1 ,begin ,end))))

So on non-Mule systems, it won’t be touched, and there shouldn’t be
behavioural changes from the old code, which is as it should be.

On fr.lettres.langue.francaise, there are subject lines like this from a
participant using Gnus v5.10.7 on XEmacs 21.4.17 (from
http://groups.google.com/groups?selm=m2wte3cbiw.fsf@baba.ba ):

=?us-ascii?Q?=3D=3Fiso-8859-1=3Fq=3FRe=3A=5FEtymon=5F=5B=3DE9?=
 =?us-ascii?Q?tait=5F=3A=5FRe=3A=5Fquestion=5Fd'un=5Fd=3DE9butant=5Fen=5Fd?=
 =?us-ascii?Q?actylographie=5D=3F=3D?=

Which is broken, clearly (it’s a double encoding which shouldn’t be
happening). 

However, I can’t reproduce that double encoding with 21.4.17, the latest
XEmacs package Gnus (and xemacs -vanilla) , and nor can I reproduce the
described problem with RFC 2047-encoded user names not being decoded, they
work fine for me (even ISO-8859-15 encoded names are decoded, which is
probably broken.) If someone can come up with a more detailed recipe, I’ll
be happy to look into it.

 > If support of no-MULE XEmacs has been broken, it wasn't done on purpose
 > and as I wrote in my previous mail, I'm all for fixing it if someone
 > gives us an idea how to fix it.

-- 
In the beginning God created the heavens and the earth. And God was a
bug-eyed, hexagonal smurf with a head of electrified hair; and God said:
“Si, mi chiamano Mimi...”

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Reiner Steib]

On Sun, Apr 09 2006, Aidan Kehoe wrote:

>  Ar an fichiú lá de mí Feabhra, scríobh Reiner Steib>: 
>
>  > Stephen, I don't understand this finger pointing.  It doesn't bring us
>  > any further.  If you look at Gnus' ChangeLog you won't find any
>  > changes aiming to stop supporting no-MULE XEmacs.  There are only very
>  > few XEmacs specific changes in that area, AFAICS.  Most of these were
>  > patches from Aidan WRT latin-unity (I don't know if those are
>  > responsible for Mike's problems or not).  
>
> All my Gnus code that touches on latin-unity is wrapped in this (it’s in the
> mm-xemacs-find-mime-charset-1 function):
>
> (defmacro mm-xemacs-find-mime-charset (begin end)
>   (when (featurep 'xemacs)
>     `(and (featurep 'mule) (mm-xemacs-find-mime-charset-1 ,begin ,end))))
>
> So on non-Mule systems, it won’t be touched, and there shouldn’t be
> behavioural changes from the old code, which is as it should be.

Yes, it turned out that a change in `rfc2047.el' caused it, see
http://article.gmane.org/gmane.emacs.gnus.general/62033.

> On fr.lettres.langue.francaise, there are subject lines like this from a
> participant using Gnus v5.10.7 on XEmacs 21.4.17 (from
> http://groups.google.com/groups?selm=m2wte3cbiw.fsf@baba.ba ):
>
> =?us-ascii?Q?=3D=3Fiso-8859-1=3Fq=3FRe=3A=5FEtymon=5F=5B=3DE9?=
>  =?us-ascii?Q?tait=5F=3A=5FRe=3A=5Fquestion=5Fd'un=5Fd=3DE9butant=5Fen=5Fd?=
>  =?us-ascii?Q?actylographie=5D=3F=3D?=
>
> Which is broken, clearly 

Yes.

| User-Agent: Gnus/5.1007 (Gnus v5.10.7) XEmacs/21.4.17 (darwin)

Does this XEmacs have MULE?  Maybe we should add something to the
User-Agent in `gnus-extended-version' if (featurep 'mule) is nil.

Here's another example:

,----[ <news:mzez3i0p.fsf@nemo.im.schnuerpel.net> ]
| From: Sabine 'Sani' Schulz <heitschi.bumbeitschi.bumbum@albasani.net>
| Newsgroups: de.comm.software.newsreader
| Subject: Re: =?us-ascii?Q?=3D=3FISO-8859-15=3FQ=3FMesNews=5Ff=3DFCr=5FAnf?=
|  =?us-ascii?Q?=3DE4nger=5Fgeeignet=3D3F=3F=3D?=
| Date: Wed, 05 Apr 2006 22:05:26 +0200
| User-Agent: Gnus/5.1007 (Gnus v5.10.7) XEmacs/21.4.13 (windows-nt)
`----

> (it’s a double encoding which shouldn’t be happening).

Double encoding is necessary, e.g. if the user types the following
Subject:

| Subject: Ich hab' da mal 'ne Fräge: Was bedeuten denn diese Zeichen =?ISO-8859-15?Q?=F6?= ?

> However, I can’t reproduce that double encoding with 21.4.17, the latest
> XEmacs package Gnus (and xemacs -vanilla) , 

Did you try with a no-MULE XEmacs?  Could there be a difference
between an XEmacs without MULE on Windows and GNU/Linux?

> and nor can I reproduce the described problem with RFC 2047-encoded
> user names not being decoded, they work fine for me (even
> ISO-8859-15 encoded names are decoded, which is probably broken.) 

This has been fixed in Gnus CVS.  It should also be fixed in
xemacs-pkg-1_90.

BTW, will anyone sync the XEmacs Gnus package to Gnus 5.10.8 (released
today)?  Is the Position still Vacant?  See
<http://article.gmane.org/gmane.emacs.xemacs.beta/22341>.

> If someone can come up with a more detailed recipe, I’ll be happy to
> look into it.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Mike Kupfer]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

Reiner> BTW, will anyone sync the XEmacs Gnus package to Gnus 5.10.8
Reiner> (released today)?  Is the Position still Vacant?

I volunteered to take over; apparently, I'm the only person to step
forward.

I'm a little nervous about the time commitment, and I'm not a strong
Lisp programmer.  So if someone else wants to take the position, that's
fine with me.

The current situation is that Steve is putting together a cheat sheet to
help me get started.  There are a couple things I can do in the mean
time (e.g., pull down the XEmacs packages from CVS and try to build
them), but I haven't gotten to them yet.

mike

Re: [Bug: 21.4.18] gnus 5.10.7 not rendering Latin-1 headers

[Stephen J. Turnbull]

>>>>> "Mike" == Mike Kupfer <mike.kupfer@sun.com> writes:

    Mike> I'm a little nervous about the time commitment, and I'm not
    Mike> a strong Lisp programmer.  So if someone else wants to take
    Mike> the position, that's fine with me.

Thanks for taking this on!

Steve Y is the best to advise you about the time commitment for Gnus
maintenance, but you don't need to be a strong Lisp programmer.  As
package maintainer, mostly you need to

- keep abreast of changes in your package,

- of changes in the package infrastructure,

- try to integrate both into your package,

- notify the relevant teams of problems you have, and

- tell the Package Release Engineer when a release is ready.

Since you're not a Lisp programmer (yet? :), the other thing you can
do is LART us if there's a problem that's more an XEmacs problem than
a Gnus problem, but we seem to be ignoring it.  Since it's
unreasonable for you to take too much responsibility for those
yourself.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

Gnus: Autoloads for parse-time.el variables (was: [PATCH] package building problems.)

[Reiner Steib]

On Sat, May 12 2007, Aidan Kehoe wrote:

> Also included is a couple of autoloads necessary for M-x gnus to work
> correctly (on my machine). 
[...]
> --- xemacs-packages/gnus/lisp/parse-time.el	2006/03/16 04:18:06	1.5
> +++ xemacs-packages/gnus/lisp/parse-time.el	2007/05/12 21:19:35
> @@ -114,10 +114,12 @@
>  		list)))
>      (nreverse list)))
>
> +;;;###autoload
>  (defvar parse-time-months '(("jan" . 1) ("feb" . 2) ("mar" . 3)
>  			    ("apr" . 4) ("may" . 5) ("jun" . 6)
>  			    ("jul" . 7) ("aug" . 8) ("sep" . 9)
>  			    ("oct" . 10) ("nov" . 11) ("dec" . 12)))
> +;;;###autoload
>  (defvar parse-time-weekdays '(("sun" . 0) ("mon" . 1) ("tue" . 2)
>  			      ("wed" . 3) ("thu" . 4) ("fri" . 5) ("sat" . 6)))
>  (defvar parse-time-zoneinfo `(("z" 0) ("ut" 0) ("gmt" 0)

AFAICS, all the places where `parse-time-months' and
`parse-time-weekdays' are used, `parse-time.el' is already required
correctly at top-level or in the functions (in the v5-10 = stable
branch; also in Gnus 5.10.8 which should be in the pre-release package
tree of XEmacs, IIRC).  There's no need to add autoload, IMHO.

--8<---------------cut here---------------start------------->8---
-*- mode: grep; default-directory: ".../v5-10/lisp/" -*-
Grep started at Sun May 13 11:11:13

grep -nH -e 'parse-time-months\|parse-time-weekdays\|require..parse-time' *.el
message.el:4758:(eval-when-compile (require 'parse-time))
message.el:4762:  (require 'parse-time)
message.el:4772:					     parse-time-weekdays))))
message.el:4777:				      parse-time-months))))
nnimap.el:1392:(eval-when-compile (require 'parse-time))
nnimap.el:1395:  (require 'parse-time)
nnimap.el:1400:						 parse-time-months))))
nnultimate.el:43:(require 'parse-time)
nnultimate.el:187:				       parse-time-months))
nnultimate.el:190:				     parse-time-months)))
nnultimate.el:194:					parse-time-months))
parse-time.el:118:(defvar parse-time-months '(("jan" . 1) ("feb" . 2) ("mar" . 3)
parse-time.el:122:(defvar parse-time-weekdays '(("sun" . 0) ("mon" . 1) ("tue" . 2)
parse-time.el:132:  `(((6) parse-time-weekdays)
parse-time.el:134:    ((4) parse-time-months)
pop3.el:276:(eval-when-compile (defvar parse-time-months))
pop3.el:282:  (require 'parse-time)
pop3.el:294:				      parse-time-months))))

Grep finished (matches found) at Sun May 13 11:11:13
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Gnus: Autoloads for parse-time.el variables (was: [PATCH] package building problems.)

[Aidan Kehoe]

 Ar an triú lá déag de mí Bealtaine, scríobh Reiner Steib: 

 > AFAICS, all the places where `parse-time-months' and
 > `parse-time-weekdays' are used, `parse-time.el' is already required
 > correctly at top-level or in the functions (in the v5-10 = stable
 > branch; also in Gnus 5.10.8 which should be in the pre-release package
 > tree of XEmacs, IIRC).  There's no need to add autoload, IMHO.

Okay, looking at the current message.el, the problem I saw was a result of
the dependency tracking issues, and not a Gnus bug. Thanks for the
pointer. 

-- 
On the quay of the little Black Sea port, where the rescued pair came once
more into contact with civilization, Dobrinton was bitten by a dog which was
assumed to be mad, though it may only have been indiscriminating. (Saki)

Re: Gnus: Autoloads for parse-time.el variables (was: [PATCH] package building problems.)

[Mike Kupfer]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

Reiner> AFAICS, all the places where `parse-time-months' and
Reiner> `parse-time-weekdays' are used, `parse-time.el' is already
Reiner> required correctly at top-level or in the functions (in the
Reiner> v5-10 = stable branch; also in Gnus 5.10.8 which should be in
Reiner> the pre-release package tree of XEmacs, IIRC).  

Actually, the 5.10.8 package got promoted to stable a couple weeks (?)
ago.

Reiner> There's no need to add autoload, IMHO.

Agreed.

mike

Re: [Bug: 21.5-b28] XEmacs/gnus refuses to open newsgroup with identical messages

[Reiner Steib]

On Sat, Oct 06 2007, Stephen J. Turnbull wrote:

> Thomas Overgaard writes:
>  > One of the users of the newsgroup alt.os.linux.slackware has managed
>  > to setup his newsreader so that it post a duplicate of all his
>  > messages. When this happens Emacs/gnus simply refuses to open the
>  > group.

Please provide more information.  Error message?  Which author?
Message-IDs of "duplicate messages"?  BTW, I don't understand what you
mean with the term "duplicate" or "identical" message.  In Usenet,
articles cannot be identical because at least the Message-IDs need to
be unique, else the news server would refuse to accept the second
message.

>  > Previously I've seen this behavior if two messages just has identical
>  > >From and Date headers.

Could be a problem when sorting by date.  I can't reproduce the
problem when entering alt.os.linux.slackware (with ~1000) articles and
`gnus-thread-sort-functions' set to '(gnus-thread-sort-by-date).
(Using Emacs 22.)

>  > Heres The backtrace:
>
> Is there an error message?  If so, please post it.
>
> If not, then probably X?Emacs is doing what it was told to do, and the
> problem is in Gnus, which we can fix in XEmacs packages but would
> prefer to have a solution that is developed by a core Gnus worker.
>
> Have you reported this to ding@gnus.org

I can't find/recall such a report.  Cc-ing ding.

> and emacs-devel@gnu.org?

If Gnus fails in XEmacs, this is most probably irrelevant to
emacs-devel.

>  > (gnus ver: 1.9 upstream: 5.10.7)

This package is a old CVS snapshot from the stable branch of Gnus.
Isn't there a Gnus 5.10.8 XEmacs package available?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: [Bug: 21.5-b28] XEmacs/gnus refuses to open newsgroup with identical messages

[Thomas Overgaard]

On Saturday 06 October 2007 18:09, Reiner Steib wrote: 

Problem solved, see later.

> Please provide more information.  Error message?  Which author?

It was the infamous "Wrong type argument: arrayp," followed by the headers 
from one of the messages.

> Message-IDs of "duplicate messages"?  BTW, I don't understand what you
> mean with the term "duplicate" or "identical" message.  In Usenet,
> articles cannot be identical because at least the Message-IDs need to
> be unique, else the news server would refuse to accept the second
> message.
>
The author is Stanislaw Flatto <compaid@brownbear.com.au> and he has started 
to post his messages to two different servers simultaneously, and of cause 
the headers like Message-ID differ.
>
> Could be a problem when sorting by date.  I can't reproduce the
> problem when entering alt.os.linux.slackware (with ~1000) articles and
> `gnus-thread-sort-functions' set to '(gnus-thread-sort-by-date).

I've been using this setting for years:
(setq gnus-thread-sort-functions
      '(gnus-article-sort-by-number (not gnus-thread-sort-by-date)))

So far this has only been a problem two or three times before, but now it has 
become a real issue. So now I changed it to:
(setq gnus-thread-sort-functions
      '((not gnus-thread-sort-by-date)))

Problem solved, thanks for the pointer and sorry for the inconvenience.
-- 
Thomas Overgaard
Markledet 16
6950 Ringkjøbing

Thomas@Overgaard.mail.dk

gnus 5.10.8 for XEmacs (was [Bug: 21.5-b28] XEmacs/gnus refuses to open newsgroup with identical messages)

[Mike Kupfer]

>>>>> "Reiner" == Reiner Steib <reinersteib+gmane@imap.cc> writes:

Reiner> This package is a old CVS snapshot from the stable branch of
Reiner> Gnus.  Isn't there a Gnus 5.10.8 XEmacs package available?

Yes.  It's in the XEmacs stable package repository; pkg version 1.91.

mike