From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/66737 Path: news.gmane.org!not-for-mail From: Patrick Drechsler Newsgroups: gmane.emacs.gnus.general,gmane.emacs.devel Subject: Re: coding problem in article display with gnus 5.13 incl. in emacs cvs Date: Wed, 09 Apr 2008 10:02:13 +0200 Organization: none Message-ID: <87fxtvpibe.fsf@pdrechsler.de> References: <87myo494v0.fsf@pdrechsler.de> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1207728167 24532 80.91.229.12 (9 Apr 2008 08:02:47 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 9 Apr 2008 08:02:47 +0000 (UTC) Cc: emacs-devel@gnu.org To: ding@gnus.org Original-X-From: ding-owner+M15220@lists.math.uh.edu Wed Apr 09 10:03:20 2008 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1JjVH5-0008PY-7d for ding-account@gmane.org; Wed, 09 Apr 2008 10:03:19 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1JjVGK-0004rB-Kl; Wed, 09 Apr 2008 03:02:32 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JjVGI-0004qz-Uo for ding@lists.math.uh.edu; Wed, 09 Apr 2008 03:02:31 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1JjVGA-0005Xd-Ej for ding@lists.math.uh.edu; Wed, 09 Apr 2008 03:02:30 -0500 Original-Received: from main.gmane.org ([80.91.229.2] helo=ciao.gmane.org) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1JjVGK-0006kK-00 for ; Wed, 09 Apr 2008 10:02:32 +0200 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JjVG7-00085f-Db for ding@gnus.org; Wed, 09 Apr 2008 08:02:19 +0000 Original-Received: from dslb-088-065-101-223.pools.arcor-ip.net ([88.65.101.223]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 09 Apr 2008 08:02:19 +0000 Original-Received: from patrick by dslb-088-065-101-223.pools.arcor-ip.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 09 Apr 2008 08:02:19 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Followup-To: gmane.emacs.devel Original-Lines: 304 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: dslb-088-065-101-223.pools.arcor-ip.net Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEUAAACtra19fX3y8vKA gID///9AQEBawxxrAAABeUlEQVR4nHXSwXKDIBAA0G3a8Vw05JxY9E4MObeK3h1Cz2qt//8J3UWj ppIdBxmfu4sC2CcB070+PQHDuAcyhQPzwLVFuSlfqQK7eOFirfQ2ly7JC8a3Kip1sx6QLOaRB0wo IUg8gFUa2D2BIFVb+OYGhC23cB4aUDbaQt4AgifjCACt9HwgQfrjWZWDwNPjEwHbtBu44kPC5D9Y FknMgFf8aZFawDDGIuoTkAYndYczzd5hjqAYwXzQ/QCraB2401HvHyQhKMb2tbw0M7whuBNoIs4M W/Xh056fQzwLPF/g14FkrcLN4nKp9WLByIMImSwyycLjDDsOF93pVFdC6yofHkEjxDhWumtWsMdH edc7SOf2/QiYQiByUU0gSldKD4LyBLUbK+kCaoI+diNlxgOGmIFWhVDpe5RgqEiPRR6Buwx8PxXE G3CGzeMVmAnctUB5h3Hsthljp34Ne/wjbu4+cIovBVbqfH4vFdOkoI2SzEWUZdMMQ9k/5csXnnRd prMAAAAASUVORK5CYII= User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (gnu/linux) Cancel-Lock: sha1:t/cj+rbEMY6dw4oWVxIVPUu8FEs= X-Spam-Score: -2.6 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:66737 gmane.emacs.devel:94761 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Reiner Steib writes: > On Tue, Apr 08 2008, Patrick Drechsler wrote: > >> after updating GNU/Emacs CVS a few days ago I noticed that the >> article buffer sometimes does not display umlaut characters correctly >> anymore. > > I'm not aware of any changes in Gnus, thus most probably a change in > Emacs is the reason. (Cc-ing emacs-devel, keeping full citation.) > >> Symptoms: Either the article buffer displays (A) the character's >> control sequence (ie "ä" -> "\344") or (B) it skips the character it >> can not display. This did not happen in previous versions of Gnus >> v5.13 shipped with Emacs CVS. >> >> To reproduce (A): [...snipped...] > Does the problem only appear in Gcc-ed messages or in usual mail/news > messages as well? If the later, could you send such a message to > gmane-test@hawk.netfonds.no (gmane.test). --=-=-= Content-Transfer-Encoding: 8bit If I send a test message to gmane.test[1] using Thunderbird, and then view the message with Gnus, the characters are displayed correctly when visiting the group. On the other hand Gnus displays the message incorrectly in my "Sent" folder: ,---- | User-Agent: Thunderbird 1.5.0.14ubu (X11/20080306) | MIME-Version: 1.0 | Newsgroups: gmane.test | Subject: test | Content-Type: text/plain; charset=UTF-8; format=flowed | Content-Transfer-Encoding: 8bit | [...] | character: ä (4194276, #o17777744, #x3fffe4) | preferred charset: eight-bit (Raw bytes 0-255) | code point: 0xE4 | syntax: w which means: word | buffer code: #xE4 | file code: not encodable by coding system windows-1252-unix | display: no font available `---- Also other "normal" Emails are displayed incorrectly. Example: ,---- | User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) | MIME-Version: 1.0 | Content-Type: text/plain; charset=UTF-8; format=flowed | Content-Transfer-Encoding: 8bit | [...] | character: ä (4194276, #o17777744, #x3fffe4) | preferred charset: eight-bit (Raw bytes 0-255) | code point: 0xE4 | syntax: w which means: word | buffer code: #xE4 | file code: not encodable by coding system windows-1252-unix | display: no font available `---- --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit >> Placing the cursor on "\344" and `M-x describe-char' results in: >> >> ,---- >> | character: \344 (4194276, #o17777744, #x3fffe4) >> | preferred charset: eight-bit (Raw bytes 0-255) >> | code point: 0xE4 >> | syntax: w which means: word >> | buffer code: #xE4 >> | file code: not encodable by coding system undecided-unix >> | display: no font available >> | >> | Character code properties are not shown: customize what to show >> | >> | There are text properties here: >> | auto-composed t >> `---- >> >> Here is the output of `M-x describe-char' from the ä character in >> second sentence: >> >> ,---- >> | character: ä (228, #o344, #xe4) >> | preferred charset: iso-8859-1 (Latin-1 (ISO/IEC 8859-1)) >> | code point: 0xE4 >> | syntax: w which means: word >> | category: j:Japanese l:Latin >> | buffer code: #xC3 #xA4 >> | file code: #xC3 #xA4 (encoded by coding system utf-8-emacs-unix) >> | display: by this font (glyph code) >> | bitstream vera sans mono:pixelsize=13:foundry=bitstream:weight=regular:slant=roman:width=normal (#x6C) >> `---- >> >> (Don't ask me why the category indicates "j:Japanese") >> >> Concerning (B): >> >> On other occations I noticed that displayed umlauts are deleted or >> replaced by a space. >> >> Here is an example: >> >> ,---- >> | List-Id: Linux >> | To: patrickXXX@XXX.XX >> | From: Linux >> | Date: Mon, 07 Apr 2008 04:02:02 +0200 >> | Content-Type: multipart/alternative; boundary="-----b007da1ee42" >> | Content-Transfer-Encoding: 8bit >> | MIME-Version: 1.0 >> | Subject: heise online News Linux 07.04.2008 >> | Message-Id: >> | [...] >> | >> | * Nachrichten berblick zum Thema Linux der vergangenen sieben Tage * >> | * >> | Benchmark-Suite f r Linux >> | >> | Phoronix hat eine umfassende Benchmark-Suite f r Linux ver ffentlicht. >> | [...] >> `---- >> >> Can other people using Gnus (not "No Gnus") with current Gnu Emacs >> CVS confirm this behaviour? > > FTR: Gnus 5.13 is the same as "No Gnus", modulo delays in > synchronization. OK, thanks for correcting me. >> Going through the Gnus settings below, the only thing that catches my >> eye is "mm-charset-synonym-alist". Searching the Gnus manual for the >> terms "charset" results in section 3.20 Charsets. But I have trouble >> understanding it. > > `mm-charset-synonym-alist' should be irrelevant. OK. >> Can anybody give me a pointer which variable(s) need to be adopted? >> >> Previous versions of Gnus v5.13 included with Emacs CVS displayed the >> article buffer correctly. > > It would be very good if you could identify the change causing the > problem using bisection (specifying the date to cvs). I have never done this, but I'll give a try this weekend. How do I check out older cvs versions by date? This does not work: --8<---------------cut here---------------start------------->8--- cvs -z3 \ -D "2008-02-01 00:00:00" \ -d:pserver:anonymous@cvs.savannah.gnu.org:/cvsroot/emacs \ co emacs --8<---------------cut here---------------end--------------->8--- Thanks Patrick >> ,----[ Emacs version and settings: ] >> | In GNU Emacs 23.0.60.2 (i686-pc-linux-gnu, GTK+ Version 2.8.20) >> | of 2008-04-04 on golem >> | Windowing system distributor `The X.Org Foundation', version 11.0.70000000 >> | configured using `configure '--prefix=/home/patrick/prg/stow/emacs-devel' '--with-xft' '--with-x-toolkit=gtk'' >> | >> | Important settings: >> | value of $LC_ALL: nil >> | value of $LC_COLLATE: nil >> | value of $LC_CTYPE: nil >> | value of $LC_MESSAGES: nil >> | value of $LC_MONETARY: nil >> | value of $LC_NUMERIC: nil >> | value of $LC_TIME: nil >> | value of $LANG: de_DE.UTF-8 >> | value of $XMODIFIERS: nil >> | locale-coding-system: utf-8-unix >> | default-enable-multibyte-characters: t >> | >> | Major mode: Message >> | >> | Minor modes in effect: >> | gnus-message-citation-mode: t >> | mml-mode: t >> | delete-selection-mode: t >> | show-paren-mode: t >> | savehist-mode: t >> | pc-selection-mode: t >> | iswitchb-mode: t >> | display-time-mode: t >> | which-function-mode: t >> | shell-dirtrack-mode: t >> | tooltip-mode: t >> | mouse-wheel-mode: t >> | menu-bar-mode: t >> | file-name-shadow-mode: t >> | global-font-lock-mode: t >> | font-lock-mode: t >> | blink-cursor-mode: t >> | global-auto-composition-mode: t >> | auto-composition-mode: t >> | auto-compression-mode: t >> | column-number-mode: t >> | line-number-mode: t >> | transient-mark-mode: t >> | abbrev-mode: t >> `---- >> >> ,----[ Gnus settings ] >> | (setq gnus-default-nntp-server "") >> | (setq gnus-select-method >> | '(nntp "news.myserver.foo")) >> | (setq gnus-message-archive-group >> | '((if >> | (message-news-p) >> | "nnimap+mail.mymailserver.bla:INBOX.sent.news" "nnimap+mail.mymailserver.bla:INBOX.Sent"))) >> | (setq gnus-secondary-select-methods >> | '((nnml "private") >> | (nntp "news.gmane.org") >> | (nnimap "mail.mymailserver.bla" >> | (nnimap-list-pattern >> | ("*"))))) >> | (setq gnus-group-change-level-function 'gnus-topic-change-level) >> | (setq gnus-summary-expunge-below -999) >> | (setq gnus-summary-mode-hook >> | '(gnus-agent-mode)) >> | (setq gnus-exit-group-hook >> | '(gnus-uu-delete-work-dir gnus-uu-clean-up)) >> | (setq gnus-group-line-format "%M%S%p%P%5y:%B%(%G%)%l %O\n") >> | (setq gnus-group-catchup-group-hook >> | '(gnus-topic-update-topic)) >> | (setq gnus-exit-gnus-hook >> | '(mm-destroy-postponed-undisplay-list)) >> | (setq gnus-visible-headers "^From:\\|^Newsgroups:\\|^Subject:\\|^Date:\\|^Followup-To:\\|^Reply-To:\\|^Organization:\\|^Summary:\\|^Keywords:\\|^To:\\|^[BGF]?Cc:\\|^Posted-To:\\|^Mail-Copies-To:\\|^Mail-Followup-To:\\|^Apparently-To:\\|^Gnus-Warning:\\|^Resent-From:\\|^X-Sent:\\|^X-Newsreader:\\|^X-Mailer:\\|^User-Agent:\\|^X-User-Agent:\\|^X-Injected-Via-Gmane:\\|^Message-ID:") >> | (setq gnus-saved-headers "^From:\\|^Newsgroups:\\|^Subject:\\|^Date:\\|^Followup-To:\\|^Reply-To:\\|^Organization:\\|^Summary:\\|^Keywords:\\|^To:\\|^[BGF]?Cc:\\|^Posted-To:\\|^Mail-Copies-To:\\|^Mail-Followup-To:\\|^Apparently-To:\\|^Gnus-Warning:\\|^Resent-From:\\|^X-Sent:") >> | (setq gnus-article-prepare-hook >> | '(bbdb/gnus-pop-up-bbdb-buffer)) >> | (setq gnus-init-file "~/.emacs.d/.gnus") >> | (setq gnus-startup-hook >> | '(bbdb-insinuate-gnus)) >> | (setq gnus-setup-news-hook >> | '(gnus-agent-queue-setup gnus-fixup-nnimap-unread-after-getting-new-news)) >> | (setq gnus-save-newsrc-hook >> | '(bbdb-offer-save)) >> | (setq gnus-group-mode-hook >> | '(gnus-agent-mode gnus-topic-mode)) >> | (setq gnus-posting-styles >> | '((".*" >> | (organization "none") >> | (Face >> | '(gnus-convert-png-to-face "~/.emacs.d/faces/ant.png"))) >> | (".*gmane.comp.lang.r.general" >> | (signature-file "~/.emacs.d/mailsignature/sig-workcam-r") >> | (Face >> | '(gnus-convert-png-to-face "~/.emacs.d/faces/ant.png"))))) >> | (setq gnus-home-score-file "~/.emacs.d/total-score-file.SCORE") >> | (setq gnus-server-mode-hook >> | '(gnus-agent-mode)) >> | (setq mm-charset-synonym-alist >> | '((ibm866 . cp866) >> | (unicode . utf-16-le) >> | (ks_c_5601-1987 . cp949) >> | (windows-31j . cp932) >> | (iso8859-1 . iso-8859-1))) >> | (setq message-send-mail-function 'smtpmail-send-it) >> | (setq message-post-method >> | '(nntp "news")) >> | (setq message-setup-hook >> | '(bbdb-define-all-aliases bbdb-insinuate-message)) >> | (setq message-mode-hook >> | '(#[nil "\302\030\303 !)\207" >> | [gnus-article-copy gnus-setup-message-group nil gnus-configure-posting-styles] >> | 2] >> | #[nil "\302 \211\020\211\021\207" >> | [message-mailer message-newsreader gnus-extended-version] >> | 2])) >> | (setq message-header-setup-hook >> | '(gnus-inews-insert-archive-gcc gnus-inews-insert-gcc)) >> | (setq message-signature 'randomsig-signature) >> `---- Footnotes: [1] Archived-At: -- For animals, the entire universe has been neatly divided into things to (a) mate with, (b) eat, (c) run away from, and (d) rocks. (Terry Pratchett, Equal Rites) --=-=-=--