From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/15197 Path: main.gmane.org!not-for-mail From: gsstark@mit.edu (Gregory S. Stark) Newsgroups: gmane.emacs.gnus.general Subject: Regular expression performance in emacs, number of cached regexps Date: 17 May 1998 13:49:54 -0400 Sender: owner-ding@hpc.uh.edu Message-ID: NNTP-Posting-Host: coloc-standby.netfonds.no X-Trace: main.gmane.org 1035154273 20787 80.91.224.250 (20 Oct 2002 22:51:13 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 20 Oct 2002 22:51:13 +0000 (UTC) Cc: ding@gnus.org Return-Path: Original-Received: from gwyn.tux.org (gwyn.tux.org [207.96.122.8]) by altair.xemacs.org (8.8.8/8.8.8) with ESMTP id KAA09512 for ; Sun, 17 May 1998 10:57:36 -0700 Original-Received: from sina.hpc.uh.edu (root@Sina.HPC.UH.EDU [129.7.3.5]) by gwyn.tux.org (8.8.8/8.8.8) with ESMTP id NAA32401 for ; Sun, 17 May 1998 13:55:13 -0400 Original-Received: from sina.hpc.uh.edu (lists@Sina.HPC.UH.EDU [129.7.3.5]) by sina.hpc.uh.edu (8.7.3/8.7.3) with ESMTP id MAH20927; Sun, 17 May 1998 12:55:03 -0500 (CDT) Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Sun, 17 May 1998 12:51:15 -0500 (CDT) Original-Received: from claymore.vcinet.com ([208.205.12.23]) by sina.hpc.uh.edu (8.7.3/8.7.3) with SMTP id MAA20914 for ; Sun, 17 May 1998 12:51:07 -0500 (CDT) Original-Received: (qmail 2946 invoked by uid 504); 17 May 1998 17:50:06 -0000 Original-Received: (qmail 2943 invoked from network); 17 May 1998 17:50:05 -0000 Original-Received: from bigbang.Generation.NET (205.205.118.5) by claymore.vcinet.com with SMTP; 17 May 1998 17:50:05 -0000 Original-Received: from portD41.Generation.NET.generation.net (gsstark@portC43.Generation.NET [205.205.118.199]) by bigbang.Generation.NET (8.8.7/8.8.7) with SMTP id NAA11252; Sun, 17 May 1998 13:49:56 -0400 (EDT) Original-To: gnu-emacs-bug@prep.ai.mit.edu Original-Cc: w3-beta@indiana.edu Original-Lines: 39 X-Mailer: Gnus v5.4.64/Emacs 19.34 Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:15197 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:15197 In profiling an elisp program I was working on I found that an inordinate quantity of string memory was being allocated in a function I thought I had optimized extensively. Using the source, I eventually found that the memory was allocated by the regular expression searching functions to cache a small constant number of compiled regular expressions. Unfortunately in my code, which was fairly short and simple, the behaviour was pathological. If I recall correctly, Emacs cached the five most recently used compiled regular expressions, and I was looping through about six or seven. So Emacs was repeatedly recompiling the same six or seven regular expressions and allocating memory only to be thrown away as garbage shortly later, causing excessive regular expression compilations and garbage collections. I've since optimized my package to use less than five regular expressions, but my concern is that several other large packages including W3 and Gnus use re-search-forward and looking-at liberally in code that is otherwise quite carefully optimized (more or less). Reducing the number of regular expressions these packages use in loops is likely to be quite hard. I suspect increasing the compile-time configuration constant governing the size of this cache will be a big speed and memory boost for these packages. greg In case people are curious, here is the code I use to profile memory usage in my packages. I've suggested something like this be integrated in elp.el, which I don't think would really be all that hard, but I don't think elp.el is under active development, which is a shame, it's a great tool. (defadvice the-function-to-profile (around profile-memory) (let (before after delta) (setq before (memory-use-counts)) ad-do-it (setq after (memory-use-counts)) (while before (push (- (pop after) (pop before)) delta)) (apply 'message "MEMORY USE cons:%d fl:%d vec:%d sym:%d str:%d misc:%d int:%d" (nreverse delta))))