From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/69279 Path: news.gmane.org!not-for-mail From: Dan Christensen Newsgroups: gmane.emacs.gnus.general Subject: Re: faster gnus-thread-latest-date Date: Thu, 10 Dec 2009 22:01:07 -0500 Message-ID: <87d42m2ozw.fsf@uwo.ca> References: <87my1uiien.fsf@uwo.ca> <87hbs2yw7f.fsf@uwo.ca> <87aaxt6zwy.fsf@lifelogs.com> <873a3lj89l.fsf@uwo.ca> <87y6lbfs42.fsf@uwo.ca> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1260500584 20421 80.91.229.12 (11 Dec 2009 03:03:04 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 11 Dec 2009 03:03:04 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M17684@lists.math.uh.edu Fri Dec 11 04:02:56 2009 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1NIvmR-00074e-9y for ding-account@gmane.org; Fri, 11 Dec 2009 04:02:55 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1NIvlW-0002Xk-I9; Thu, 10 Dec 2009 21:01:58 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1NIvlT-0002XS-Hh for ding@lists.math.uh.edu; Thu, 10 Dec 2009 21:01:55 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.69) (envelope-from ) id 1NIvlD-00059z-Q1 for ding@lists.math.uh.edu; Thu, 10 Dec 2009 21:01:55 -0600 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 3.36 #1 (Debian)) id 1NIvlD-0004qs-00 for ; Fri, 11 Dec 2009 04:01:39 +0100 Original-Received: from list by lo.gmane.org with local (Exim 4.50) id 1NIvlC-0006mD-94 for ding@gnus.org; Fri, 11 Dec 2009 04:01:38 +0100 Original-Received: from bas3-london14-1096778870.dsl.bell.ca ([65.95.132.118]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 11 Dec 2009 04:01:38 +0100 Original-Received: from jdc by bas3-london14-1096778870.dsl.bell.ca with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 11 Dec 2009 04:01:38 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 143 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: bas3-london14-1096778870.dsl.bell.ca User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.1.50 (gnu/linux) Cancel-Lock: sha1:L594H07L+0dyw2ZiHDzV8BcIY4Q= X-Spam-Score: -2.6 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:69279 Archived-At: --=-=-= Dan Christensen writes: > I'm now thinking that caching the parsed time in "time value" format (a > pair of integers) rather than a float might be more useful, since many > parts of Gnus need this information. It turns out that someone thought of this before, and implemented it as gnus-date-get-time in gnus-util, which caches the emacs time value as a text property on the date string that is being parsed! Clever idea: it works more generally than my method and doesn't require api changes to various functions. So all I had to do was call this function in a few places, and things are faster. For a group with about 6000 articles, the time for sorting with gnus-thread-sort-by-most-recent-date when from about 12 seconds to 1 second, and the time for generating the summary went from 8 seconds to 6 seconds. The patch below includes a ChangeLog. It also adds a message to gnus-sort-threads if the recursive sort fails and it falls back to the loop. That could be useful debugging information when timing these things. Please test and apply. Dan --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=latest-date.patch Index: ChangeLog =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/ChangeLog,v retrieving revision 7.2042 diff -u -r7.2042 ChangeLog --- ChangeLog 2 Dec 2009 12:10:00 -0000 7.2042 +++ ChangeLog 11 Dec 2009 02:53:52 -0000 @@ -1,3 +1,12 @@ +2009-12-10 Dan Christensen + + * gnus-util.el (gnus-user-date): Use gnus-date-get-time. + (gnus-dd-mmm): Use gnus-date-get-time. + * gnus-sum.el (gnus-thread-latest-date): Use gnus-date-get-time and + simplify logic. + (gnus-summary-limit-to-age): Use gnus-date-get-time. + (gnus-sort-threads): emit message if gnus-sort-threads-loop used. + 2009-12-02 Katsumi Yamaoka * lpath.el: Always bind default-file-name-coding-system for (S)XEmacs. Index: gnus-util.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-util.el,v retrieving revision 7.89 diff -u -r7.89 gnus-util.el --- gnus-util.el 9 Sep 2009 10:42:27 -0000 7.89 +++ gnus-util.el 11 Dec 2009 02:53:52 -0000 @@ -455,10 +455,10 @@ (defun gnus-user-date (messy-date) "Format the messy-date according to gnus-user-date-format-alist. -Returns \" ? \" if there's bad input or if an other error occurs. +Returns \" ? \" if there's bad input or if another error occurs. Input should look like this: \"Sun, 14 Oct 2001 13:34:39 +0200\"." (condition-case () - (let* ((messy-date (gnus-float-time (safe-date-to-time messy-date))) + (let* ((messy-date (gnus-float-time (gnus-date-get-time messy-date))) (now (gnus-float-time)) ;;If we don't find something suitable we'll use this one (my-format "%b %d '%y")) @@ -477,7 +477,7 @@ (defun gnus-dd-mmm (messy-date) "Return a string like DD-MMM from a big messy string." (condition-case () - (format-time-string "%d-%b" (safe-date-to-time messy-date)) + (format-time-string "%d-%b" (gnus-date-get-time messy-date)) (error " - "))) (defmacro gnus-date-get-time (date) Index: gnus-sum.el =================================================================== RCS file: /usr/local/cvsroot/gnus/lisp/gnus-sum.el,v retrieving revision 7.245 diff -u -r7.245 gnus-sum.el --- gnus-sum.el 30 Nov 2009 09:36:17 -0000 7.245 +++ gnus-sum.el 11 Dec 2009 02:53:58 -0000 @@ -4826,7 +4826,8 @@ ;; Even after binding max-lisp-eval-depth, the recursive ;; sorter might fail for very long threads. In that case, ;; try using a (less well-tested) non-recursive sorter. - (error (gnus-sort-threads-loop + (error (gnus-message 9 "Sorting threads with loop...") + (gnus-sort-threads-loop threads (gnus-make-sort-function gnus-thread-sort-functions)))) (gnus-message 8 "Sorting threads...done")))) @@ -4993,22 +4994,17 @@ "Sort threads such that the thread with the most recently dated article comes first." (> (gnus-thread-latest-date h1) (gnus-thread-latest-date h2))) +; Since this is called not only to sort the top-level threads, but +; also in recursive sorts to order the articles within a thread, each +; article will be processed many times. Thus it speeds things up +; quite a bit to use gnus-date-get-time, which caches the time value. (defun gnus-thread-latest-date (thread) "Return the highest article date in THREAD." - (let ((previous-time 0)) - (apply 'max - (mapcar - (lambda (header) - (setq previous-time - (condition-case () - (gnus-float-time (mail-header-parse-date - (mail-header-date header))) - (error previous-time)))) - (sort - (message-flatten-list thread) - (lambda (h1 h2) - (< (mail-header-number h1) - (mail-header-number h2)))))))) + (apply 'max + (mapcar '(lambda (header) (gnus-float-time + (gnus-date-get-time + (mail-header-date header)))) + (message-flatten-list thread)))) (defun gnus-thread-total-score-1 (root) ;; This function find the total score of the thread below ROOT. @@ -8232,9 +8228,7 @@ (when (and (vectorp (gnus-data-header d)) (setq date (mail-header-date (gnus-data-header d)))) (setq is-younger (time-less-p - (time-since (condition-case () - (date-to-time date) - (error '(0 0)))) + (time-since (gnus-date-get-time date)) cutoff)) (when (if younger-p is-younger --=-=-=--