From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/47253 Path: main.gmane.org!not-for-mail From: TSUCHIYA Masatoshi Newsgroups: gmane.emacs.gnus.general Subject: Re: Continuation lines for persistent articles Date: Thu, 17 Oct 2002 22:19:46 +0900 Sender: owner-ding@hpc.uh.edu Message-ID: <8765w1td71.fsf@pine.kuee.kyoto-u.ac.jp> References: <87d6qelwjx.fsf@pine.kuee.kyoto-u.ac.jp> <87znti2qs6.fsf@crybaby.cs.uni-dortmund.de> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: main.gmane.org 1034865762 27714 80.91.224.249 (17 Oct 2002 14:42:42 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Thu, 17 Oct 2002 14:42:42 +0000 (UTC) Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 182BrE-0007Ch-00 for ; Thu, 17 Oct 2002 16:42:40 +0200 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 182BqV-0001w8-00; Thu, 17 Oct 2002 09:41:55 -0500 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Thu, 17 Oct 2002 09:42:35 -0500 (CDT) Original-Received: from epithumia.math.uh.edu (epithumia.math.uh.edu [129.7.128.2]) by sina.hpc.uh.edu (8.9.3/8.9.3) with ESMTP id JAA07938 for ; Thu, 17 Oct 2002 09:42:26 -0500 (CDT) Original-Received: (from tibbs@localhost) by epithumia.math.uh.edu (8.11.2/8.11.1) id g9HEfe426450 for ding@hpc.uh.edu; Thu, 17 Oct 2002 09:41:40 -0500 Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id IAA07753 for ; Thu, 17 Oct 2002 08:32:11 -0500 (CDT) Original-Received: (qmail 3785 invoked by alias); 17 Oct 2002 13:31:19 -0000 Original-Received: (qmail 3778 invoked from network); 17 Oct 2002 13:31:19 -0000 Original-Received: from quimby.gnus.org (80.91.224.244) by gnus.org with SMTP; 17 Oct 2002 13:31:19 -0000 Original-Received: from news by quimby.gnus.org with local (Exim 3.12 #1 (Debian)) id 182BPs-0002Xw-00 for ; Thu, 17 Oct 2002 16:14:24 +0200 Original-To: ding@gnus.org Original-Path: not-for-mail Original-Newsgroups: gnus.ding Original-Lines: 511 Original-NNTP-Posting-Host: pine.kuee.kyoto-u.ac.jp Original-X-Trace: quimby.gnus.org 1034864064 9064 130.54.208.130 (17 Oct 2002 14:14:24 GMT) Original-X-Complaints-To: usenet@quimby.gnus.org Original-NNTP-Posting-Date: 17 Oct 2002 14:14:24 GMT X-cite: xcite 1.46 User-Agent: Gnus/5.090008 (Oort Gnus v0.08) Emacs/21.2 (i386-debian-linux-gnu) Cancel-Lock: sha1:QIPmeIDQczoRoFj5VfGpqeDMLvM= Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:47253 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:47253 --=-=-= Content-Type: text/plain; charset=iso-2022-jp-2 Content-Transfer-Encoding: 8bit Hi, >> On Sun, 13 Oct 2002 19:27:05 +0200 >> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Gro.AN_johann) said as follows: >> I am worry that all functions which call nnheader-parse-head() must >> unfold continuation lines before calling it. If this observation >> is right, a better approach would be to make nnheader-parse-head() >> unfold them. >I think this is a good idea. I see. I think that related changes can be splited into 2 parts. The first part is installed to nnheader.el. This change introduces some new functions and makes nnheader-parse-head() unfold continuation lines. nnheader.el (nnheader-parse-naked-head): New function. (nnheader-parse-head): Use the above function, in order to handle continuation lines properly. (nnheader-remove-body): New function. (nnheader-remove-cr-followed-by-lf): New function. (nnheader-ms-strip-cr): Use the above function. I hope that this change can be installed to Gnus without ill effects. However, I suppose that it makes Gnus slow, because the new version of nnheader-parse-head() copies headers to temporary buffer in order to avoid side effects. The second part is required to reduce excessive header copy. I hope that these changes will make Gnus as fast as the older version. gnus-agent.el (gnus-agent-regenerate-group): Call `nnheader-remove-body'; Use `nnheader-parse-naked-head' instead of `nnheader-parse-head'. gnus-cache.el (gnus-cache-possibly-enter-article): Ditto. gnus-msg.el (gnus-inews-yank-articles): Do not unfold continuation lines by itself; Call `nnheader-parse-naked-head' instead of `nnheader-parse-head'. nndiary.el (nndiary-parse-head): Ditto. nnfolder.el (nnfolder-parse-head): Ditto. nnimap.el (nnimap-retrieve-headers-progress): Ditto. nnmaildir.el (nnmaildir--update-nov): Ditto. nnml.el (nnml-parse-head): Ditto. I checked changes of gnus-cache.el and nnml.el, but the other changes are not tested. Thanks to Mr. Yamaoka for his advices about these changes. --=-=-= Content-Type: text/x-patch Content-Disposition: inline Index: gnus-agent.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-agent.el,v retrieving revision 6.82 diff -u -r6.82 gnus-agent.el --- gnus-agent.el 2002/10/15 10:15:48 6.82 +++ gnus-agent.el 2002/10/17 06:57:36 @@ -2031,11 +2031,8 @@ (mm-with-unibyte-buffer (nnheader-insert-file-contents (concat dir (number-to-string (car arts)))) - (goto-char (point-min)) - (if (search-forward "\n\n" nil t) - (delete-region (point) (point-max)) - (goto-char (point-max))) - (setq header (nnheader-parse-head t))) + (nnheader-remove-body) + (setq header (nnheader-parse-naked-head))) (mail-header-set-number header (car arts)) (nnheader-insert-nov header) (setq changed t) Index: gnus-cache.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-cache.el,v retrieving revision 6.18 diff -u -r6.18 gnus-cache.el --- gnus-cache.el 2002/06/16 09:49:28 6.18 +++ gnus-cache.el 2002/10/17 06:57:36 @@ -179,7 +179,8 @@ (when (> (buffer-size) 0) (let ((coding-system-for-write gnus-cache-coding-system)) (gnus-write-buffer file)) - (setq headers (nnheader-parse-head t)) + (nnheader-remove-body) + (setq headers (nnheader-parse-naked-head)) (mail-header-set-number headers number) (gnus-cache-change-buffer group) (set-buffer (cdr gnus-cache-buffer)) Index: gnus-msg.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-msg.el,v retrieving revision 6.100 diff -u -r6.100 gnus-msg.el --- gnus-msg.el 2002/09/25 23:45:17 6.100 +++ gnus-msg.el 2002/10/17 06:57:37 @@ -719,8 +719,7 @@ (with-current-buffer gnus-article-copy (save-restriction (nnheader-narrow-to-headers) - (ietf-drums-unfold-fws) - (nnheader-parse-head t))))) + (nnheader-parse-naked-head))))) (message-yank-original) (setq beg (or beg (mark t)))) (when articles Index: nndiary.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nndiary.el,v retrieving revision 1.6 diff -u -r1.6 nndiary.el --- nndiary.el 2002/05/15 19:55:38 1.6 +++ nndiary.el 2002/10/17 06:57:37 @@ -1123,13 +1123,7 @@ (narrow-to-region (goto-char (point-min)) (if (search-forward "\n\n" nil t) (1- (point)) (point-max)))) - ;; Fold continuation lines. - (goto-char (point-min)) - (while (re-search-forward "\\(\r?\n[ \t]+\\)+" nil t) - (replace-match " " t t)) - ;; Remove any tabs; they are too confusing. - (subst-char-in-region (point-min) (point-max) ?\t ? ) - (let ((headers (nnheader-parse-head t))) + (let ((headers (nnheader-parse-naked-head))) (mail-header-set-chars headers chars) (mail-header-set-number headers number) headers)))) Index: nnfolder.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nnfolder.el,v retrieving revision 6.42 diff -u -r6.42 nnfolder.el --- nnfolder.el 2002/07/20 14:41:21 6.42 +++ nnfolder.el 2002/10/17 06:57:37 @@ -1150,13 +1150,7 @@ (if (search-forward "\n\n" e t) (setq e (1- (point))))) (with-temp-buffer (insert-buffer-substring buf b e) - ;; Fold continuation lines. - (goto-char (point-min)) - (while (re-search-forward "\\(\r?\n[ \t]+\\)+" nil t) - (replace-match " " t t)) - ;; Remove any tabs; they are too confusing. - (subst-char-in-region (point-min) (point-max) ?\t ? ) - (let ((headers (nnheader-parse-head t))) + (let ((headers (nnheader-parse-naked-head))) (mail-header-set-chars headers chars) (mail-header-set-number headers number) headers))))) Index: nnheader.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nnheader.el,v retrieving revision 6.28 diff -u -r6.28 nnheader.el --- nnheader.el 2002/08/20 09:12:50 6.28 +++ nnheader.el 2002/10/17 06:57:37 @@ -213,120 +213,135 @@ ;; Parsing headers and NOV lines. +(defsubst nnheader-remove-cr-followed-by-lf () + (goto-char (point-max)) + (while (search-backward "\r\n" nil t) + (delete-char 1))) + (defsubst nnheader-header-value () (skip-chars-forward " \t") (buffer-substring (point) (gnus-point-at-eol))) -(defun nnheader-parse-head (&optional naked) +(defun nnheader-parse-naked-head (&optional number) + ;; This function unfolds continuation lines in this buffer + ;; destructively. When this side effect is unwanted, use + ;; `nnheader-parse-head' instead of this function. (let ((case-fold-search t) - (cur (current-buffer)) (buffer-read-only nil) - in-reply-to lines p ref) - (goto-char (point-min)) - (when naked - (insert "\n")) - ;; Search to the beginning of the next header. Error messages - ;; do not begin with 2 or 3. + (cur (current-buffer)) + (p (point-min)) + in-reply-to lines ref) + (nnheader-remove-cr-followed-by-lf) + (ietf-drums-unfold-fws) + (subst-char-in-region (point-min) (point-max) ?\t ? ) + (goto-char p) + (insert "\n") (prog1 - (when (or naked (re-search-forward "^[23][0-9]+ " nil t)) - ;; This implementation of this function, with nine - ;; search-forwards instead of the one re-search-forward and - ;; a case (which basically was the old function) is actually - ;; about twice as fast, even though it looks messier. You - ;; can't have everything, I guess. Speed and elegance - ;; don't always go hand in hand. - (vector - ;; Number. - (if naked - (progn - (setq p (point-min)) - 0) - (prog1 - (read cur) - (end-of-line) - (setq p (point)) - (narrow-to-region (point) - (or (and (search-forward "\n.\n" nil t) - (- (point) 2)) - (point))))) - ;; Subject. - (progn - (goto-char p) - (if (search-forward "\nsubject:" nil t) - (nnheader-header-value) "(none)")) - ;; From. - (progn - (goto-char p) - (if (search-forward "\nfrom:" nil t) - (nnheader-header-value) "(nobody)")) - ;; Date. - (progn - (goto-char p) - (if (search-forward "\ndate:" nil t) - (nnheader-header-value) "")) - ;; Message-ID. - (progn - (goto-char p) - (if (search-forward "\nmessage-id:" nil t) - (buffer-substring - (1- (or (search-forward "<" (gnus-point-at-eol) t) - (point))) - (or (search-forward ">" (gnus-point-at-eol) t) (point))) - ;; If there was no message-id, we just fake one to make - ;; subsequent routines simpler. - (nnheader-generate-fake-message-id))) - ;; References. - (progn - (goto-char p) - (if (search-forward "\nreferences:" nil t) - (nnheader-header-value) - ;; Get the references from the in-reply-to header if there - ;; were no references and the in-reply-to header looks - ;; promising. - (if (and (search-forward "\nin-reply-to:" nil t) - (setq in-reply-to (nnheader-header-value)) - (string-match "<[^\n>]+>" in-reply-to)) - (let (ref2) - (setq ref (substring in-reply-to (match-beginning 0) - (match-end 0))) - (while (string-match "<[^\n>]+>" - in-reply-to (match-end 0)) - (setq ref2 (substring in-reply-to (match-beginning 0) - (match-end 0))) - (when (> (length ref2) (length ref)) - (setq ref ref2))) - ref) - nil))) - ;; Chars. - 0 - ;; Lines. - (progn - (goto-char p) - (if (search-forward "\nlines: " nil t) - (if (numberp (setq lines (read cur))) - lines 0) - 0)) - ;; Xref. - (progn - (goto-char p) - (and (search-forward "\nxref:" nil t) - (nnheader-header-value))) - - ;; Extra. - (when nnmail-extra-headers - (let ((extra nnmail-extra-headers) - out) - (while extra - (goto-char p) - (when (search-forward - (concat "\n" (symbol-name (car extra)) ":") nil t) - (push (cons (car extra) (nnheader-header-value)) - out)) - (pop extra)) - out)))) - (when naked - (goto-char (point-min)) - (delete-char 1))))) + ;; This implementation of this function, with nine + ;; search-forwards instead of the one re-search-forward and a + ;; case (which basically was the old function) is actually + ;; about twice as fast, even though it looks messier. You + ;; can't have everything, I guess. Speed and elegance don't + ;; always go hand in hand. + (vector + ;; Number. + (or number 0) + ;; Subject. + (progn + (goto-char p) + (if (search-forward "\nsubject:" nil t) + (nnheader-header-value) "(none)")) + ;; From. + (progn + (goto-char p) + (if (search-forward "\nfrom:" nil t) + (nnheader-header-value) "(nobody)")) + ;; Date. + (progn + (goto-char p) + (if (search-forward "\ndate:" nil t) + (nnheader-header-value) "")) + ;; Message-ID. + (progn + (goto-char p) + (if (search-forward "\nmessage-id:" nil t) + (buffer-substring + (1- (or (search-forward "<" (gnus-point-at-eol) t) + (point))) + (or (search-forward ">" (gnus-point-at-eol) t) (point))) + ;; If there was no message-id, we just fake one to make + ;; subsequent routines simpler. + (nnheader-generate-fake-message-id))) + ;; References. + (progn + (goto-char p) + (if (search-forward "\nreferences:" nil t) + (nnheader-header-value) + ;; Get the references from the in-reply-to header if + ;; there were no references and the in-reply-to header + ;; looks promising. + (if (and (search-forward "\nin-reply-to:" nil t) + (setq in-reply-to (nnheader-header-value)) + (string-match "<[^\n>]+>" in-reply-to)) + (let (ref2) + (setq ref (substring in-reply-to (match-beginning 0) + (match-end 0))) + (while (string-match "<[^\n>]+>" + in-reply-to (match-end 0)) + (setq ref2 (substring in-reply-to (match-beginning 0) + (match-end 0))) + (when (> (length ref2) (length ref)) + (setq ref ref2))) + ref) + nil))) + ;; Chars. + 0 + ;; Lines. + (progn + (goto-char p) + (if (search-forward "\nlines: " nil t) + (if (numberp (setq lines (read cur))) + lines 0) + 0)) + ;; Xref. + (progn + (goto-char p) + (and (search-forward "\nxref:" nil t) + (nnheader-header-value))) + ;; Extra. + (when nnmail-extra-headers + (let ((extra nnmail-extra-headers) + out) + (while extra + (goto-char p) + (when (search-forward + (concat "\n" (symbol-name (car extra)) ":") nil t) + (push (cons (car extra) (nnheader-header-value)) + out)) + (pop extra)) + out))) + (goto-char p) + (delete-char 1)))) + +(defun nnheader-parse-head (&optional naked) + (let ((cur (current-buffer)) num beg end) + (when (if naked + (setq num 0 + beg (point-min) + end (point-max)) + (goto-char (point-min)) + ;; Search to the beginning of the next header. Error + ;; messages do not begin with 2 or 3. + (when (re-search-forward "^[23][0-9]+ " nil t) + (end-of-line) + (setq num (read cur) + beg (point) + end (if (search-forward "\n.\n" nil t) + (- (point) 2) + (point))))) + (with-temp-buffer + (insert-buffer-substring cur beg end) + (nnheader-parse-naked-head num))))) (defmacro nnheader-nov-skip-field () '(search-forward "\t" eol 'move)) @@ -615,6 +630,13 @@ (point-max))) (goto-char (point-min))) +(defun nnheader-remove-body () + "Remove the body from an article in this current buffer." + (goto-char (point-min)) + (when (or (search-forward "\n\n" nil t) + (search-forward "\n\r\n" nil t)) + (delete-region (point) (point-max)))) + (defun nnheader-set-temp-buffer (name &optional noerase) "Set-buffer to an empty (possibly new) buffer called NAME with undo disabled." (set-buffer (get-buffer-create name)) @@ -834,9 +856,7 @@ (defun nnheader-ms-strip-cr () "Strip ^M from the end of all lines." (save-excursion - (goto-char (point-min)) - (while (re-search-forward "\r$" nil t) - (delete-backward-char 1)))) + (nnheader-remove-cr-followed-by-lf))) (defun nnheader-file-size (file) "Return the file size of FILE or 0." Index: nnimap.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nnimap.el,v retrieving revision 6.48 diff -u -r6.48 nnimap.el --- nnimap.el 2002/10/10 02:40:50 6.48 +++ nnimap.el 2002/10/17 06:57:38 @@ -528,10 +528,7 @@ (with-temp-buffer (buffer-disable-undo) (insert headers) - (nnheader-ms-strip-cr) - (nnheader-fold-continuation-lines) - (subst-char-in-region (point-min) (point-max) ?\t ? ) - (let ((head (nnheader-parse-head 'naked))) + (let ((head (nnheader-parse-naked-head))) (mail-header-set-number head uid) (mail-header-set-chars head chars) (mail-header-set-lines head lines) Index: nnmaildir.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nnmaildir.el,v retrieving revision 6.21 diff -u -r6.21 nnmaildir.el --- nnmaildir.el 2002/10/03 18:16:25 6.21 +++ nnmaildir.el 2002/10/17 06:57:38 @@ -371,8 +371,7 @@ (setq nov-mid 0)) (goto-char (point-min)) (delete-char 1) - (nnheader-fold-continuation-lines) - (setq nov (nnheader-parse-head 'naked) + (setq nov (nnheader-parse-naked-head) field (or (mail-header-lines nov) 0))) (unless (or (zerop field) (nnmaildir--param pgname 'distrust-Lines:)) (setq nov-mid field)) Index: nnml.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/nnml.el,v retrieving revision 6.40 diff -u -r6.40 nnml.el --- nnml.el 2002/05/13 15:29:09 6.40 +++ nnml.el 2002/10/17 06:57:38 @@ -701,16 +701,12 @@ (unless (zerop (buffer-size)) (narrow-to-region (goto-char (point-min)) - (if (re-search-forward "\n\r?\n" nil t) (1- (point)) (point-max)))) - ;; Fold continuation lines. - (goto-char (point-min)) - (while (re-search-forward "\\(\r?\n[ \t]+\\)+" nil t) - (replace-match " " t t)) - ;; Remove any tabs; they are too confusing. - (subst-char-in-region (point-min) (point-max) ?\t ? ) - ;; Remove any ^M's; they are too confusing. - (subst-char-in-region (point-min) (point-max) ?\r ? ) - (let ((headers (nnheader-parse-head t))) + (if (search-forward "\n\n" nil t) + (1- (point)) + (if (search-forward "\n\r\n" nil t) + (- (point) 2) + (point-max))))) + (let ((headers (nnheader-parse-naked-head))) (mail-header-set-chars headers chars) (mail-header-set-number headers number) headers)))) --=-=-= Content-Type: application/octet-stream Content-Disposition: inline -- TSUCHIYA Masatoshi --=-=-=--