From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/65325 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: Huge memory consumption on accessing large newsgroup Date: Mon, 01 Oct 2007 21:13:22 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: References: <87wsw4u21m.fsf@gmx.de> <87sl6rh886.fsf@gmx.de> <87hcldp4j1.fsf@srcf.ucam.org> <87odfjprux.fsf@enki.rimspace.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1191291330 28289 80.91.229.12 (2 Oct 2007 02:15:30 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Tue, 2 Oct 2007 02:15:30 +0000 (UTC) Cc: ding@gnus.org To: Daniel Pittman Original-X-From: ding-owner+M13837@lists.math.uh.edu Tue Oct 02 04:15:08 2007 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1IcXHm-000786-FI for ding-account@gmane.org; Tue, 02 Oct 2007 04:14:58 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1IcXGv-0007ET-T0; Mon, 01 Oct 2007 21:14:05 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1IcXGt-0007E6-T0 for ding@lists.math.uh.edu; Mon, 01 Oct 2007 21:14:03 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1IcXGn-0007AD-Lc for ding@lists.math.uh.edu; Mon, 01 Oct 2007 21:14:03 -0500 Original-Received: from blockstar.com ([170.224.69.95] helo=mail.blockstar.com) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1IcXGf-0004Wb-00 for ; Tue, 02 Oct 2007 04:13:49 +0200 Original-Received: from tzz (c-24-14-57-89.hsd1.il.comcast.net [24.14.57.89]) by mail.blockstar.com (Postfix) with ESMTP id EB5823F8114; Mon, 1 Oct 2007 19:29:34 -0700 (PDT) X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Followup-To: Daniel Pittman , ding@gnus.org In-Reply-To: <87odfjprux.fsf@enki.rimspace.net> (Daniel Pittman's message of "Mon, 01 Oct 2007 11:04:54 +1000") User-Agent: Gnus/5.110007 (No Gnus v0.7) Emacs/22.1.50 (darwin) X-Spam-Score: -2.5 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:65325 Archived-At: On Mon, 01 Oct 2007 11:04:54 +1000 Daniel Pittman wrote: DP> Ted Zlatanov writes: >> Consider using inversion lists. >> They don't require pairs of integers; each value represents a flip. >> They have other nice properties too, though it's been a while since I >> looked at them so I can't name them. DP> Mmm. The existing Gnus range code is pretty much as efficient, has DP> lower computational complexity for the operations Gnus uses and is DP> already existent and tested. We're talking about reducing memory consumption, not CPU usage. For CPU usage the current structure is pretty good. For memory usage, as you can see from the subject of the message, it's not ideal. We could use several data structures as needed to accomodate large data lists. Inversion lists are nice in some cases and not others; I think they are worth at least some consideration since they don't require a pair of numbers to express a range and thus may save some memory. I don't know the Emacs internals intimately, but I think (24 86 88 90) takes up less memory than ((24 . 86) (88 . 90)). Am I wrong? Note Gaute's original message was about compressing pairs of integers, not general Gnus data structures, so please consider my message in that context. DP> I don't think that you would see sufficient benefit from introducing the DP> additional data type to justify spending your time on it -- but since I DP> am not volunteering to do the work I can't tell you how to do it. ;) I am not sure if you are addressing me or Gaute. I made a suggestion, and an implementation is already in place (I mentioned Kim Storm implemented it). I'm willing to assist Gaute, but he has to be interested in my suggestion. I don't have the time for this as a solo project. Ted