From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/63925 Path: news.gmane.org!not-for-mail From: Alexandre Oliva Newsgroups: gmane.emacs.pretest.bugs,gmane.emacs.gnus.general Subject: Slow operations on buffers of tens of megabytes Date: Sun, 05 Nov 2006 03:37:06 -0200 Organization: Free Software Foundation Latin America Message-ID: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1162749489 8395 80.91.229.2 (5 Nov 2006 17:58:09 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 5 Nov 2006 17:58:09 +0000 (UTC) Original-X-From: emacs-pretest-bug-bounces+gebp-emacs-pretest-bug=gmane.org@gnu.org Sun Nov 05 18:58:02 2006 Return-path: Envelope-to: gebp-emacs-pretest-bug@gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1GgmFt-0001gm-IB for gebp-emacs-pretest-bug@gmane.org; Sun, 05 Nov 2006 18:58:01 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1GgmFt-000429-1S for gebp-emacs-pretest-bug@gmane.org; Sun, 05 Nov 2006 12:58:01 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1GgahI-0007Rg-Sb for emacs-pretest-bug@gnu.org; Sun, 05 Nov 2006 00:37:32 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1GgahH-0007MK-4F for emacs-pretest-bug@gnu.org; Sun, 05 Nov 2006 00:37:32 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1GgahH-0007MC-18 for emacs-pretest-bug@gnu.org; Sun, 05 Nov 2006 00:37:31 -0500 Original-Received: from [130.243.97.8] (helo=post.oru.se) by monty-python.gnu.org with esmtp (Exim 4.52) id 1GgahG-00074y-Q0 for emacs-pretest-bug@gnu.org; Sun, 05 Nov 2006 00:37:31 -0500 Original-Received: from brevduva.oru.se (brevduva.oru.se [130.243.97.7]) by post.oru.se (tlh) with ESMTP id kA55bSrI024206; Sun, 5 Nov 2006 06:37:28 +0100 (MET) Original-Received: from mendeleev.fsfeurope.org (PC109-200.oru.se [130.243.109.200]) by brevduva.oru.se (tlh) with ESMTP id kA55bH8Z005982; Sun, 5 Nov 2006 06:37:18 +0100 (MET) Original-Received: from freie.oliva.athome.lsd.ic.unicamp.br (unknown [201.12.75.88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mendeleev.fsfeurope.org (Postfix) with ESMTP id 030277BC7F2; Sun, 5 Nov 2006 05:34:13 +0000 (UTC) Original-Received: from free.oliva.athome.lsd.ic.unicamp.br (free.oliva.athome.lsd.ic.unicamp.br [172.31.160.1]) by freie.oliva.athome.lsd.ic.unicamp.br (8.13.8/8.13.8) with ESMTP id kA55b6ZJ007065; Sun, 5 Nov 2006 03:37:06 -0200 Original-Received: from free.oliva.athome.lsd.ic.unicamp.br (localhost.localdomain [127.0.0.1]) by free.oliva.athome.lsd.ic.unicamp.br (8.13.8/8.13.8) with ESMTP id kA55b7SU024310; Sun, 5 Nov 2006 03:37:07 -0200 Original-Received: (from aoliva@localhost) by free.oliva.athome.lsd.ic.unicamp.br (8.13.8/8.13.8/Submit) id kA55b6lv024309; Sun, 5 Nov 2006 03:37:06 -0200 Original-To: emacs-pretest-bug@gnu.org, ding@gnus.org User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.90 (gnu/linux) X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-1.6 (brevduva.oru.se [130.243.97.7]); Sun, 05 Nov 2006 06:37:18 +0100 (MET) X-oru-MailScanner: Found to be clean X-MailScanner-From: lxoliva@fsfla.org X-Mailman-Approved-At: Sun, 05 Nov 2006 12:57:26 -0500 X-BeenThere: emacs-pretest-bug@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Bug reports for CVS Emacs." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-pretest-bug-bounces+gebp-emacs-pretest-bug=gmane.org@gnu.org Errors-To: emacs-pretest-bug-bounces+gebp-emacs-pretest-bug=gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.pretest.bugs:14910 gmane.emacs.gnus.general:63925 Archived-At: Please describe exactly what actions triggered the bug and the precise symptoms of the bug: I use gnus to read my e-mail, and most of my messages are in nnfolder groups. This means lots and lots of messages are kept in a single mbox-like file. A few commonly-used groups have their e-mail stored in files/buffers with a few tens of megabytes. When I enter such a group, gnus goes over all unread or marked messages searching for regular expressions to score them, i.e., determine whether the messages should be highlighted, hidden, discarded, etc. XEmacs 21.5.27 with xemacs-sumo 20060510 (including gnus 5.10) enters such buffers very quickly, and only gets really slow for buffers that exceed hundreds of megabytes, with tens of thounsands of e-mails. In GNU Emacs 22.0.90.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars) of 2006-11-02 on free X server distributor `The X.Org Foundation', version 11.0.70101000 configured using `configure '--prefix=/home/aoliva/test/emacs-22.0.90' '--exec-prefix=/home/aoliva/test/emacs-22.0.90/H-x86_64-linux-gnu' 'CC=ccache gcc -fno-working-directory -m64'' using the built-in gnus 5.11, it takes minutes to enter groups with just a few hundred messages and just a few tens of megabytes in the underlying buffer/file, where XEmacs takes less than 10 seconds. Scoring of the messages closer to the beginning of the buffer is fast, but as we move to higher-numbered messages, that are closer to the end of such big files/buffers, gnus will only score 2-3 messages per minute, and that's what kills performance. I can't tell whether it is general big-buffer management that is causing such slow downs, or if it's regular expression searching, or some such, but this slow down is severely impacting my ability to switch back to GNU Emacs :-( I've historically switched back and forth as new major releases came up, even though I felt more at home at GNU Emacs. However, last time I used GNU Emacs for mail reading, I still used the nnml back end, instead of nnfolder, and nnml keeps each message in a separate file, so it was not affected by this problem. However, since that was much slower to enter groups on both Emacsen, because folders were fragmented, at some point in the last few months I switched to nnfolder, and that also had a great impact on my mail backup times :-) I hope this is enough information to figure out whom might help me understand what the problem is and how I might be able to overcome it, perhaps even helping fix it in GNU Emacs. Thanks in advance, Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: en_US.UTF-8 locale-coding-system: utf-8 default-enable-multibyte-characters: t -- Alexandre Oliva http://www.lsd.ic.unicamp.br/~oliva/ FSF Latin America Board Member http://www.fsfla.org/ Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org} Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}