From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by yquem.inria.fr (Postfix) with ESMTP id 2D998BC6C for ; Wed, 6 Feb 2008 13:04:07 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAAIYwqUfAXQInh2dsb2JhbACQMgEBAQgKKZ0F X-IronPort-AV: E=Sophos;i="4.25,312,1199660400"; d="scan'208";a="7703021" Received: from concorde.inria.fr ([192.93.2.39]) by mail1-smtp-roc.national.inria.fr with ESMTP; 06 Feb 2008 13:04:07 +0100 Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id m16C46rR019178 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Wed, 6 Feb 2008 13:04:06 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ao8CABsxqUfUVZgL/2dsb2JhbACteg X-IronPort-AV: E=Sophos;i="4.25,312,1199660400"; d="scan'208";a="8849416" Received: from hades.snarc.org ([212.85.152.11]) by mail3-smtp-sop.national.inria.fr with ESMTP; 06 Feb 2008 13:04:06 +0100 Received: by hades.snarc.org (Postfix, from userid 1000) id 118811B482; Wed, 6 Feb 2008 13:04:04 +0100 (CET) Date: Wed, 6 Feb 2008 13:04:03 +0100 To: Oliver Bandel Cc: caml-list@inria.fr Subject: Re: [Caml-list] Now it's faster (addendum to "Performance-question") Message-ID: <20080206120403.GA5335@snarc.org> References: <1202297628.47a99b1c7ec53@webmail.in-berlin.de> <1202298904.47a9a018998e4@webmail.in-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1202298904.47a9a018998e4@webmail.in-berlin.de> X-Warning: Email may contain unsmilyfied humor and/or satire. User-Agent: Mutt/1.5.16 (2007-06-11) From: tab@snarc.org (Vincent Hanquez) X-Miltered: at concorde with ID 47A9A236.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; 0100,:01 bandel:01 buffer:01 buffer:01 appending:01 shocking:98 55.:98 wrote:01 oliver:01 dataset:01 caml-list:01 functions:01 functions:01 data:02 data:02 On Wed, Feb 06, 2008 at 12:55:04PM +0100, Oliver Bandel wrote: > Hello, > > I should have changed the Subject to: "Shocking Performance!!!" > > but then possibly the spam-filter would become active ;-) > > > The performance dramatically increased now! > > I first had about 3min34 on my dataset. > After throwing out some of the "^"-using > functions, the time was about 1min55. > > Now, after I threw out the rest of that "^"-stuff > (which btw. made more of the catanations then > the first thrown out functions, but was not called > as often as trhe other functions) I'm under 20 seconds! > (17..18 seconds!) > > That's amazing! :-) well i'm pretty sure you could go down even further with your own implementation of a buffer library. the buffer library is actually pretty bad since it's actually just a simple string. each time the buffer need to grow, the string is reallocated and the previous one is copied to the new string. and you got the 16mb limit (max_string_length) on 32bit. if you implement a growing array of fixed sized string (4K for example), you just don't need to copy data each time your buffer need to grow. I suspect it might be even faster than the normal buffer in your case (lots of data appending), but depends on what you do with your buffer afterwards. -- Vincent Hanquez