From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/35474 Path: main.gmane.org!not-for-mail From: Roland Mas Newsgroups: gmane.emacs.gnus.general Subject: Gnus, BBDB and MULE-UCS don't like one another Date: 26 Mar 2001 10:23:00 +0200 Sender: mas@echo.fr Message-ID: <87elvluefv.fsf@cachemir.echo-net.net> NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1035171212 3280 80.91.224.250 (21 Oct 2002 03:33:32 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 03:33:32 +0000 (UTC) Cc: srivasta@debian.org, kitame@northeye.org, himi@bird.scphys.kyoto-u.ac.jp, ding@gnus.org, bbdb-info@lists.sourceforge.net Return-Path: Original-Received: (qmail 10959 invoked by alias); 26 Mar 2001 08:23:02 -0000 Original-Received: (qmail 10954 invoked from network); 26 Mar 2001 08:23:01 -0000 Original-Received: from blackhole.x-echo.com (HELO cachemir.echo-net.net) (195.101.94.7) by gnus.org with SMTP; 26 Mar 2001 08:23:01 -0000 Original-Received: by cachemir.echo-net.net (Postfix, from userid 1000) id CC886482C; Mon, 26 Mar 2001 10:23:00 +0200 (CEST) Original-To: 88289@bugs.debian.org Mail-Copies-To: never User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7 Original-Lines: 146 Xref: main.gmane.org gmane.emacs.gnus.general:35474 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:35474 Hello people, This mail is addressed to the following people: - Manoj Srivastava (Debian Gnus maintainer); - Takuo Kitame (Debian BBDB and MULE-UCS maintainer); - Miyashita Hisashi, author of MULE-UCS; - the Gnus development mailing-list; - the BBDB development mailing-list; - the Debian bug tracking system report for the bug #88289 Please excuse the multiple addressees, but I think they all are concerned (or part of them are, in case of the mailing-lists). Justification is at the end. There is this story about a French guy (that would be me) who tries to learn Japanese, or at least not to forget it too fast now that he's out of school. The guy is a big Gnus fan, the guy loves Unicode (and needs it anyway), the guy uses BBDB a lot. And the guy uses Debian. So the guy installs the Debian packages for Gnus, BBDB and MULE-UCS (along with GNU Emacs 20.7). He reads and posts in fr.lettres.langue.japonaise, a French-speaking newsgroup about the Japanese language. French needs some non-ASCII characters, so does Japanese, UTF-8 is the encoding of choice, and all is well. Except... Except one day all ceases to be well: the UTF-8 messages are not decoded anymore. Which means the "small E with acute" character displays as "capital A with tilde" + "copyright sign" (which is its UTF-8 encoding), and the Japanese characters are completely unreadable. That is a Bad Situation, and we obviously want to go back toe the Good One. To make a long story short, I spent a few nights tracking the problem, and here's how it happens. Know that it happens when upgrading from Gnus 5.8.7 to 5.8.8, whatever (recent) versions of BBDB and MULE-UCS are installed. Let's say some user wants to start Gnus. emacs -f gnus is how I do it, but M-x gnus in an already running Emacs works the same. - Emacs starts and reads the startup.el file. On Debian, that startup file loads all *.elc? files in /etc/emacs20/, in their order of appearance. - Emacs loads 00debian-vars.elc (some variables and function definitions). - Then 20gnus-init.el (append the right dir to the load-path). - Then 50bbdb-init.el (require message, then set some variables). - Then 50mule-ucs-init.el (define the Unicode character sets and encodings). - Then Gnus starts. That's the outline (I'm not mentioning the non-relevant files). Then the user enters some group, and tries to read an UTF-8 encoded message. Some checks are done, amongst which one that checks that the charset is known. For optimisation puropses (I guess), the list of available charsets is cached in a variable, mm-coding-system-list. That variable is only accessed through the mm-get-coding-system-list function, that does all of the caching: the first time it is called, it copies the value of coding-system-list into mm-coding-system-list, and the following times it just returns the cached value. Now what happens? Well, it appears that under Gnus 5.8.7 the variable is nil at that moment, which means it gets computed right now and yields the correct list, whereas under Gnus 5.8.8 the value is already present, so the cached value is used. Unfortunately, in this last case, the cached value is wrong, in that it doesn't contain any reference to the encodings defined by MULE-UCS. Aha, and why does it not? Because the value was calculated too early, before MULE-UCS has initialised. Here comes the real story: - Emacs loads 00debian-vars.elc. No problem here. - Emacs loads 20gnus-init.el. Still okay. - Emacs load 50bbdb-init.el. Note that the MULE-UCS init step has not happened yet. - The 50bbdb-init.el has a (require 'message) clause, so message.el is loaded. - message.el requires 'nnheader. - nnheader.el requires 'mm-util, which defines the functions mm-get-coding-system-list and mm-coding-system-p, and the mm-charset-synonym-alist variable. - only *afterwards* is MULE-UCS initialised. It so happens that the mm-util.el in Gnus 5.8.7 defines the mm-charset-synonym-alist variable with a static content, whereas the one in Gnus 5.8.8 uses a backquote form that calls the mm-coding-system-p predicate, which in turn calls the mm-get-coding-system-list function. Which means, Gnus 5.8.8 initialises that mm-coding-system cache earlier than Gnus 5.8.7. Only it happens because BBDB triggers it indirectly. And unfortunately, B is before M in the latin alphabet, so the 50bbdb-init.el is lesser than 50mule-ucs-init.el. The final result is what looks like a bug. Hairy, eh? So, now come the reasons I sent this email to so many people. - Manoj Srivastava: I nagged you about this bug, it's not your fault, you can now sleep in peace. - Takuo Kitame: since you happen to maintain both the BBDB package and the MULE-UCS package for Debian, I'd suggest changing the init file names: simply renaming 50mule-ucs-init.el to 40mule-ucs-init.el fixes the problem here, and that's much less of a hack than the other solutions I have come with to fix that particular problem in Gnus. It basically does the right thing, since it defines the Unicode coding systems before doing something that uses the coding systems list. - Miyashita Hisashi and the BBDB folks: I think that maybe you could include a warning somewhere in the docs of your softwares, so that people using them would be aware of the implications of using BBDB and MULE-UCS. They just have to make sure MULE-UCS is initialised before BBDB is. - the Gnus development mailing-list: this might be useful to you and to some users... If they complain that Gnus broke UTF-8 reading you now could ask them if they use BBDB and MULE-UCS, and in what order these are initialized. - the Debian bug tracking system report for the bug #88289: well, that one seems to be either closable or reassignable to bdbb or (more probably) to mule-ucs. That's about it. I hope this will help people, although of course I hope for them they haven't spent too much time and energy into looking for that bug. If I may coin in a side note, I highly recommend the reading of "Dirk Gently's Holistic Detective Agency", by Douglas Adams. To quote the back cover: "[...] Dirk Gently, self-styled private investigator, sets out to prove the fundamental interconnectedness of all things by solving a mysterious murder, [...] unravelling a mysterious mystery and eating a lot of pizza." Although I personnally prefer sushi, that's rather close to the way I spent these last few nights. There's also one by Isaac Asimov where a succession of quite harmless acts end up as a crime, but I don't know the English title. Have a nice day, and send me your comments. Roland. -- Roland Mas 'And what would humans be without love?' RARE, said Death. -- in Sourcery (Terry Pratchett)