From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/38726 Path: main.gmane.org!not-for-mail From: Dan Christensen Newsgroups: gmane.emacs.gnus.general Subject: Re: nnml/nnfolder marks faster Date: Wed, 12 Sep 2001 15:42:38 -0400 Sender: Dan Christensen Message-ID: <87lmjki4b5.fsf@uwo.ca> References: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1035174542 23973 80.91.224.250 (21 Oct 2002 04:29:02 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 04:29:02 +0000 (UTC) Return-Path: Return-Path: Original-Received: (qmail 19287 invoked from network); 12 Sep 2001 19:42:55 -0000 Original-Received: from pony.its.uwo.ca (129.100.2.63) by gnus.org with SMTP; 12 Sep 2001 19:42:55 -0000 Original-Received: from scratchy (ren.math.uwo.ca [129.100.75.76]) by pony.its.uwo.ca (8.10.2/8.10.2) with ESMTP id f8CJhMM19321 for ; Wed, 12 Sep 2001 15:43:25 -0400 (EDT) Original-Received: from jdc by scratchy with local (Exim 3.32 #1 (Debian)) id 15hFuA-0006dy-00; Wed, 12 Sep 2001 15:42:38 -0400 Original-To: ding@gnus.org In-Reply-To: (Simon Josefsson's message of "Sat, 08 Sep 2001 21:38:18 +0200") User-Agent: Gnus/5.090004 (Oort Gnus v0.04) Emacs/21.0.100 Original-Lines: 47 Xref: main.gmane.org gmane.emacs.gnus.general:38726 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:38726 I have a few groups that have really large .mrk files, and this is causing slow performance. I've noticed this more since the new .mrk code was introduced, and it is in particular noticeable when I *exit* a group with a large .mrk file. But I don't know if the new .mrk code is causing any problems, or whether it just makes them more visible. As an example, my nnfolder:debian.big group has about 1000 articles in it, and the .mrk file is somehow using about 840 bytes per article, which seems really excessive: -rw------- 1 jdc jdc 4231285 Sep 12 13:56 Gnus/Mail/debian.big -rw-r--r-- 1 jdc jdc 840761 Sep 12 14:46 Gnus/Mail/debian.big.mrk -rw------- 1 jdc jdc 310299 Sep 12 14:42 Gnus/Mail/debian.big.nov Here are a few excerpts from the .mrk file which look suspicious: There are screenfuls and screenfuls of "seen" marks that look like: (184755 . 184758) (184759 . 184762) which should be merged into a single long list I would think. Also, the score caching has stuff like: (score (185776 . -1) (185820 . -1) (185962 . -1) (185831 . -1) (185991 . -1) which isn't even stored in numerical order (how is lookup done??). I also wonder if this could be stored in a collapsed form using ranges (score ((12345 . 15000) . -1) etc. (in cases where there are consecutive articles with the same score.) Hmm, looking further, it is the score entries that are taking up 99% of the .mrk file, and most of them don't correspond to articles that exist in the group! Shouldn't score cache entries be removed when the article is deleted from the group (e.g. by the expiry mechanism)? I smell a bug. Dan -- Dan Christensen jdc+news@uwo.ca