* Double Newlines @ 2006-07-25 7:15 David Abrahams 2006-07-25 11:42 ` Katsumi Yamaoka 0 siblings, 1 reply; 12+ messages in thread From: David Abrahams @ 2006-07-25 7:15 UTC (permalink / raw) Everything was working just fine for years, but now many of my outgoing messages are coming out with an extra blank line after each line I actually write. It only seems to happen when I reply to a message with a "Content-Transfer-Encoding: 8bit" header. Can anyone offer some small clue as to what I might do to fix the problem? Several earlier posts about this issue are at: http://news.gmane.org/find-root.php?message_id=%3cubqru8k1j.fsf%5f%2d%5f%40boost%2dconsulting.com%3e Incidentally, this appears not to be the first time I've experienced something like this http://news.gmane.org/find-root.php?message_id=%3c84bs4wt67d.fsf%40crybaby.uni%2dduisburg.de%3e but I can't for the life of me remember what the actual fix was. I currently have no customization of file-coding-system-alist. Thanks in advance, -- Dave Abrahams Boost Consulting www.boost-consulting.com ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-25 7:15 Double Newlines David Abrahams @ 2006-07-25 11:42 ` Katsumi Yamaoka 2006-07-25 14:29 ` David Abrahams 0 siblings, 1 reply; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-25 11:42 UTC (permalink / raw) Cc: ding >>>>> In <uhd16p32o.fsf@boost-consulting.com> David Abrahams wrote: > Everything was working just fine for years, but now many of my > outgoing messages are coming out with an extra blank line after each > line I actually write. It only seems to happen when I reply to a > message with a "Content-Transfer-Encoding: 8bit" header. > Can anyone offer some small clue as to what I might do to fix the problem? The following code saves in the ~/TESTMAIL file all mails you send. (setq message-send-mail-function (lambda nil (let ((coding-system-for-write 'binary)) (write-region (point-min) (point-max) "~/TESTMAIL")))) What do you see there? If newlines are doubled already, the cause might be in the Gnus' encoder. Otherwise, the program to send mails or the mail server might do something strange. Whether or no, I cannot imagine what does it, though. > Several earlier posts about this issue are at: > http://news.gmane.org/find-root.php?message_id=%3cubqru8k1j.fsf%5f%2d%5f%40boost%2dconsulting.com%3e > Incidentally, this appears not to be the first time I've experienced > something like this > http://news.gmane.org/find-root.php?message_id=%3c84bs4wt67d.fsf%40crybaby.uni%2dduisburg.de%3e > but I can't for the life of me remember what the actual fix was. I > currently have no customization of file-coding-system-alist. > Thanks in advance, > -- > Dave Abrahams > Boost Consulting > www.boost-consulting.com ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-25 11:42 ` Katsumi Yamaoka @ 2006-07-25 14:29 ` David Abrahams 2006-07-26 0:42 ` Katsumi Yamaoka 0 siblings, 1 reply; 12+ messages in thread From: David Abrahams @ 2006-07-25 14:29 UTC (permalink / raw) Cc: ding Katsumi Yamaoka <yamaoka@jpl.org> writes: >>>>>> In <uhd16p32o.fsf@boost-consulting.com> David Abrahams wrote: > >> Everything was working just fine for years, but now many of my >> outgoing messages are coming out with an extra blank line after each >> line I actually write. It only seems to happen when I reply to a >> message with a "Content-Transfer-Encoding: 8bit" header. > >> Can anyone offer some small clue as to what I might do to fix the problem? Yamaoka-san, Thank you *very* much for your reply... > The following code saves in the ~/TESTMAIL file all mails you > send. > > (setq message-send-mail-function > (lambda nil > (let ((coding-system-for-write 'binary)) > (write-region (point-min) (point-max) "~/TESTMAIL")))) > > What do you see there? This is a test=0D This is only a test=0D If this had been an actual message, we'd tell you.=0D > If newlines are doubled already, I don't know for sure how to interpret the above, but my first guess is that they're not (right?) > the cause might be in the Gnus' encoder. Otherwise, the program to > send mails or the mail server might do something strange. Whether > or no, I cannot imagine what does it, though. I'm running a local Exim on this machine to send the mail out. Maybe I should be looking at that. On the other hand, this particular instance of the phenomenon showed up in a newsgroup posting, not a mail message. I don't think NG postings are handled like regular mail, are they? If not, then they're not going through Exim and Gnus might still be involved. Does message-send-mail-function get used for NG postings, or just email? If it's just for email, then our TESTMAIL test doesn't tell us very much. -- Dave Abrahams Boost Consulting www.boost-consulting.com ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-25 14:29 ` David Abrahams @ 2006-07-26 0:42 ` Katsumi Yamaoka 2006-07-26 3:02 ` David Abrahams 2006-07-26 21:59 ` David Abrahams 0 siblings, 2 replies; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-26 0:42 UTC (permalink / raw) Cc: ding >>>>> In <ubqrdoizi.fsf@boost-consulting.com> David Abrahams wrote: >> (setq message-send-mail-function >> (lambda nil >> (let ((coding-system-for-write 'binary)) >> (write-region (point-min) (point-max) "~/TESTMAIL")))) >> >> What do you see there? > This is a test=0D > This is only a test=0D > If this had been an actual message, we'd tell you.=0D >> If newlines are doubled already, > I don't know for sure how to interpret the above, but my first guess > is that they're not (right?) That looks strange to me. `=0D' is the one that the CR character (0x0d) was encoded by quoted-printable. What does the following Lisp form return? (To do that, copy it to the *scratch* buffer and type `C-j' at the end of the last line.) [1] (let ((file (make-temp-file "testing")) (coding-system-for-write 'iso-8859-1) (coding-system-for-read 'binary)) (with-temp-file file (insert "foo\n")) (prog1 (with-temp-buffer (insert-file-contents file) (buffer-string)) (delete-file file))) If you see "foo^M " , your Emacs 22 has a bug. Since Kenichi Handa fixed the bug[2] recently[3], updating your Emacs 22 might fix the problem[4]. [1] It might probably be able to reproduce simply by the form: (encode-coding-string "foo\n" 'iso-8859-1) [2] Under the system in which the system_eol_type is CRLF (i.e., Windows), Emacs 22 encodes text by the coding system FOO as if it is FOO-dos. [3] (lisp/ChangeLog) 2006-07-20 Kenichi Handa <handa@m17n.org> * international/mule-cmds.el (select-safe-coding-system): Fix the way of deciding eol-type of the coding system. [4] I'm not quite sure that `=0D' is decoded into a newline. >> the cause might be in the Gnus' encoder. Otherwise, the program to >> send mails or the mail server might do something strange. Whether >> or no, I cannot imagine what does it, though. > I'm running a local Exim on this machine to send the mail out. Maybe > I should be looking at that. On the other hand, this particular > instance of the phenomenon showed up in a newsgroup posting, not a > mail message. I don't think NG postings are handled like regular > mail, are they? If not, then they're not going through Exim and Gnus > might still be involved. Does message-send-mail-function get used for > NG postings, or just email? If it's just for email, then our TESTMAIL > test doesn't tell us very much. If the problem is concerned with the charset encoding, it will happen with both mail and news. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 0:42 ` Katsumi Yamaoka @ 2006-07-26 3:02 ` David Abrahams 2006-07-26 11:59 ` Katsumi Yamaoka 2006-07-26 21:59 ` David Abrahams 1 sibling, 1 reply; 12+ messages in thread From: David Abrahams @ 2006-07-26 3:02 UTC (permalink / raw) Cc: ding Katsumi Yamaoka <yamaoka@jpl.org> writes: >>>>>> In <ubqrdoizi.fsf@boost-consulting.com> David Abrahams wrote: > >>> (setq message-send-mail-function >>> (lambda nil >>> (let ((coding-system-for-write 'binary)) >>> (write-region (point-min) (point-max) "~/TESTMAIL")))) >>> >>> What do you see there? > >> This is a test=0D >> This is only a test=0D >> If this had been an actual message, we'd tell you.=0D > >>> If newlines are doubled already, > >> I don't know for sure how to interpret the above, but my first guess >> is that they're not (right?) > > That looks strange to me. `=0D' is the one that the CR character > (0x0d) was encoded by quoted-printable. What does the following > Lisp form return? (To do that, copy it to the *scratch* buffer > and type `C-j' at the end of the last line.) [1] > > (let ((file (make-temp-file "testing")) > (coding-system-for-write 'iso-8859-1) > (coding-system-for-read 'binary)) > (with-temp-file file > (insert "foo\n")) > (prog1 > (with-temp-buffer > (insert-file-contents file) > (buffer-string)) > (delete-file file))) > > If you see > > "foo^M > " > , your Emacs 22 has a bug. That's what I see. > Since Kenichi Handa fixed the bug[2] > recently[3], updating your Emacs 22 might fix the problem[4]. Okay, that's my next step, thanks. > [1] It might probably be able to reproduce simply by the form: > > (encode-coding-string "foo\n" 'iso-8859-1) yep. > [2] Under the system in which the system_eol_type is CRLF (i.e., > Windows), Emacs 22 encodes text by the coding system FOO as if > it is FOO-dos. > > [3] (lisp/ChangeLog) > 2006-07-20 Kenichi Handa <handa@m17n.org> > > * international/mule-cmds.el (select-safe-coding-system): Fix the > way of deciding eol-type of the coding system. > > [4] I'm not quite sure that `=0D' is decoded into a newline. > >>> the cause might be in the Gnus' encoder. Otherwise, the program to >>> send mails or the mail server might do something strange. Whether >>> or no, I cannot imagine what does it, though. > >> I'm running a local Exim on this machine to send the mail out. Maybe >> I should be looking at that. On the other hand, this particular >> instance of the phenomenon showed up in a newsgroup posting, not a >> mail message. I don't think NG postings are handled like regular >> mail, are they? If not, then they're not going through Exim and Gnus >> might still be involved. Does message-send-mail-function get used for >> NG postings, or just email? If it's just for email, then our TESTMAIL >> test doesn't tell us very much. > > If the problem is concerned with the charset encoding, it will > happen with both mail and news. Thanks, I'll try rebuilding my emacs now. -- Dave Abrahams Boost Consulting www.boost-consulting.com ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 3:02 ` David Abrahams @ 2006-07-26 11:59 ` Katsumi Yamaoka 2006-07-27 9:28 ` Katsumi Yamaoka 0 siblings, 1 reply; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-26 11:59 UTC (permalink / raw) Cc: David Abrahams [-- Attachment #1: Type: text/plain, Size: 1645 bytes --] Hi, I don't know whether it is related to David Abrahams' problem, but I heard that the most recent Emacs 22 built on Windows behaves unlike the old ones. When saving a file in the case where `coding-system-for-write' is bound to the coding system `raw-text' for example, CRLF is used as the line break code. It is said that it happens as well when communicating with an external process. Because the default value of `nnmail-file-coding-system' and `nnmail-active-file-coding-system' is `raw-text', the nnml back end saves mail files, NOV files and the ACTIVE file as if the coding-system is set to `raw-text-dos'. Hideyuki SHIRAI said that LF was used for the line break code formerly. It might be generally a good idea to use CRLF when saving files on Windows, though. You might not seem that it causes a problem since Gnus reads those files using `raw-text' and CRLF is converted into LF again. However, OHASHI Akira reported that a problem occurs actually. He tried setting `gnus-nov-is-evil' to t and saw all summary lines in a nnml group appeared as: [ ?: nobody ] (none) I guess it always happens in nnmh groups. Could anyone who uses the most recent Emacs 22 built on Windows verify it? The cause is that the `nnheader-insert-head' function doesn't work properly. In that function, it is expected that `(insert-file-contents FILE nil BEG END)' returns `(FILE CHARS)', where CHARS is the number in which BEG is subtracted from END. However, it fails because of the CRLF-to-LF conversion. I don't use Emacs on Windows, so I hope someone follows it up. Here's a solution, though I haven't tested it yet. [-- Attachment #2: Type: application/emacs-lisp, Size: 992 bytes --] [-- Attachment #3: Type: text/plain, Size: 10 bytes --] Regards, ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 11:59 ` Katsumi Yamaoka @ 2006-07-27 9:28 ` Katsumi Yamaoka 2006-07-28 9:41 ` Katsumi Yamaoka 0 siblings, 1 reply; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-27 9:28 UTC (permalink / raw) Cc: OHASHI Akira, David Abrahams >>>>> In <b4m3bcotw52.fsf@jpl.org> Katsumi Yamaoka wrote: > However, OHASHI Akira reported that a problem occurs actually. > He tried setting `gnus-nov-is-evil' to t and saw all summary > lines in a nnml group appeared as: > [ ?: nobody ] (none) > I guess it always happens in nnmh groups. OHASHI-san confirmed it surely happens in nnmh groups with the recent Emacs 22 which runs on Windows. He also verified improving of the `nnheader-insert-head' function does the trick. > (defun nnheader-insert-head (file) I slightly modified and installed it to both the trunk and the v5-10 branch. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-27 9:28 ` Katsumi Yamaoka @ 2006-07-28 9:41 ` Katsumi Yamaoka 0 siblings, 0 replies; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-28 9:41 UTC (permalink / raw) >>>>> In <b4m8xmfflcp.fsf@jpl.org> Katsumi Yamaoka wrote: >> (defun nnheader-insert-head (file) > I slightly modified and installed it to both the trunk and the > v5-10 branch. I forgot to make it work with Mac as well as Windows. Fixed. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 0:42 ` Katsumi Yamaoka 2006-07-26 3:02 ` David Abrahams @ 2006-07-26 21:59 ` David Abrahams 2006-07-26 23:48 ` Katsumi Yamaoka 1 sibling, 1 reply; 12+ messages in thread From: David Abrahams @ 2006-07-26 21:59 UTC (permalink / raw) Cc: ding Katsumi Yamaoka <yamaoka@jpl.org> writes: >>>>>> In <ubqrdoizi.fsf@boost-consulting.com> David Abrahams wrote: > >>> (setq message-send-mail-function >>> (lambda nil >>> (let ((coding-system-for-write 'binary)) >>> (write-region (point-min) (point-max) "~/TESTMAIL")))) >>> >>> What do you see there? > >> This is a test=0D >> This is only a test=0D >> If this had been an actual message, we'd tell you.=0D > >>> If newlines are doubled already, > >> I don't know for sure how to interpret the above, but my first guess >> is that they're not (right?) > > That looks strange to me. `=0D' is the one that the CR character > (0x0d) was encoded by quoted-printable. Rebuilding emacs made that behavior go away (thank you!)... > What does the following > Lisp form return? (To do that, copy it to the *scratch* buffer > and type `C-j' at the end of the last line.) [1] > > (let ((file (make-temp-file "testing")) > (coding-system-for-write 'iso-8859-1) > (coding-system-for-read 'binary)) > (with-temp-file file > (insert "foo\n")) > (prog1 > (with-temp-buffer > (insert-file-contents file) > (buffer-string)) > (delete-file file))) > > If you see > > "foo^M > " > , your Emacs 22 has a bug. Since Kenichi Handa fixed the bug[2] > recently[3], updating your Emacs 22 might fix the problem[4]. ...but curiously, _that_ behavior remains! -- Dave Abrahams Boost Consulting www.boost-consulting.com ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 21:59 ` David Abrahams @ 2006-07-26 23:48 ` Katsumi Yamaoka 2006-07-27 18:02 ` Reiner Steib 0 siblings, 1 reply; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-26 23:48 UTC (permalink / raw) Cc: ding >>>>> In <u7j20qb6w.fsf@boost-consulting.com> David Abrahams wrote: > Rebuilding emacs made that behavior go away (thank you!)... Great. >> What does the following >> Lisp form return? (To do that, copy it to the *scratch* buffer >> and type `C-j' at the end of the last line.) [1] >> >> (let ((file (make-temp-file "testing")) >> (coding-system-for-write 'iso-8859-1) >> (coding-system-for-read 'binary)) >> (with-temp-file file >> (insert "foo\n")) >> (prog1 >> (with-temp-buffer >> (insert-file-contents file) >> (buffer-string)) >> (delete-file file))) >> >> If you see >> >> "foo^M >> " >> , your Emacs 22 has a bug. Since Kenichi Handa fixed the bug[2] >> recently[3], updating your Emacs 22 might fix the problem[4]. > ...but curiously, _that_ behavior remains! Yes, it does. Hideyuki SHIRAI who uses Windows told me yesterday that Kenichi Handa fixed only the following problem: (encode-coding-string "foo\n" 'iso-8859-1) "foo^M " Fixing this is enough to solve your problem, I think. OTOH, that the result of the above "testing" code contains ^M seems to be a spec of the most recent Emacs 22 which runs on Windows. I don't know whether it is good or bad, though. I posted a report about the possibility that it is harmful to Gnus[1]. [1] <b4m3bcotw52.fsf@jpl.org> or http://article.gmane.org/gmane.emacs.gnus.general/63502 ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-26 23:48 ` Katsumi Yamaoka @ 2006-07-27 18:02 ` Reiner Steib 2006-07-28 3:28 ` Katsumi Yamaoka 0 siblings, 1 reply; 12+ messages in thread From: Reiner Steib @ 2006-07-27 18:02 UTC (permalink / raw) On Thu, Jul 27 2006, Katsumi Yamaoka wrote: > Yes, it does. Hideyuki SHIRAI who uses Windows told me > yesterday that Kenichi Handa fixed only the following problem: > > (encode-coding-string "foo\n" 'iso-8859-1) > "foo^M > " > Fixing this is enough to solve your problem, I think. OTOH, > that the result of the above "testing" code contains ^M seems to > be a spec of the most recent Emacs 22 which runs on Windows. I > don't know whether it is good or bad, though. I posted a report > about the possibility that it is harmful to Gnus[1]. > > [1] <b4m3bcotw52.fsf@jpl.org> or > http://article.gmane.org/gmane.emacs.gnus.general/63502 I didn't follow this discussion in detail, but shouldn't this problem be reported to the Emacs developers? I don't think that Kenichi Handa uses Windows so he might not be aware of the problem. Is the changed behavior documented e.g. in etc/NEWS? Bye, Reiner. -- ,,, (o o) ---ooO-(_)-Ooo--- | PGP key available | http://rsteib.home.pages.de/ ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Double Newlines 2006-07-27 18:02 ` Reiner Steib @ 2006-07-28 3:28 ` Katsumi Yamaoka 0 siblings, 0 replies; 12+ messages in thread From: Katsumi Yamaoka @ 2006-07-28 3:28 UTC (permalink / raw) >>>>> In <v9ejw7udsj.fsf@marauder.physik.uni-ulm.de> >>>>> Reiner Steib wrote: > On Thu, Jul 27 2006, Katsumi Yamaoka wrote: [...] >> [1] <b4m3bcotw52.fsf@jpl.org> or >> http://article.gmane.org/gmane.emacs.gnus.general/63502 > I didn't follow this discussion in detail, but shouldn't this problem > be reported to the Emacs developers? I don't think that Kenichi Handa > uses Windows so he might not be aware of the problem. Is the changed > behavior documented e.g. in etc/NEWS? Once OHASHI Akira reported to the mule-ja mailing list the problem caused by the difference between the recent Emacs 22 on Windows and the old one, but Kenichi Handa hasn't responded yet so far. I'll repeat it again. Thanks. ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2006-07-28 9:41 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2006-07-25 7:15 Double Newlines David Abrahams 2006-07-25 11:42 ` Katsumi Yamaoka 2006-07-25 14:29 ` David Abrahams 2006-07-26 0:42 ` Katsumi Yamaoka 2006-07-26 3:02 ` David Abrahams 2006-07-26 11:59 ` Katsumi Yamaoka 2006-07-27 9:28 ` Katsumi Yamaoka 2006-07-28 9:41 ` Katsumi Yamaoka 2006-07-26 21:59 ` David Abrahams 2006-07-26 23:48 ` Katsumi Yamaoka 2006-07-27 18:02 ` Reiner Steib 2006-07-28 3:28 ` Katsumi Yamaoka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).