From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id 42BB1BC57 for ; Thu, 16 Dec 2010 18:17:01 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEANLYCU1QRFuw/2dsb2JhbACkOHPCEIVKBA X-IronPort-AV: E=Sophos;i="4.59,356,1288566000"; d="scan'208";a="82933835" Received: from furbychan.cocan.org ([80.68.91.176]) by mail4-smtp-sop.national.inria.fr with ESMTP/TLS/AES256-SHA; 16 Dec 2010 18:16:20 +0100 Received: from rich by furbychan.cocan.org with local (Exim 4.69) (envelope-from ) id 1PTHRB-00044a-3U; Thu, 16 Dec 2010 17:16:17 +0000 Date: Thu, 16 Dec 2010 17:16:17 +0000 From: "Richard W.M. Jones" To: Joel Reymont Cc: caml-list@yquem.inria.fr Subject: Re: [Caml-list] appending data to a mmap-ed file Message-ID: <20101216171617.GB3647@annexia.org> References: <8D1E9D5B-188C-4AF3-980B-EF229BA98FB4@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8D1E9D5B-188C-4AF3-980B-EF229BA98FB4@gmail.com> User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam: no; 0.00; appending:01 appending:01 allocating:01 wrote:01 ints:01 caml-list:01 appended:02 data:02 data:02 append:02 structures:02 parse:02 string:02 library:03 library:03 On Thu, Dec 16, 2010 at 11:31:16AM +0000, Joel Reymont wrote: > I'm constantly appending to a file of stock quotes (ints, longs, > doubles, etc.). I have this file mapped into memory with mmap. > > What's the most efficient way to make newly appended data available > as part of the memory mapping? Unfortunately it's hard to reliably extend an mmap'd area. The reason is not that you can't do it, but that you might overrun another memory mapping after it, where that other mapping could be something important like your program or a shared library. The other mapping might not even be present at the time you initially map your file, but might appear as the result of an innocuous operation such as printing a string or allocating memory. Now you can, with a bunch of work, avoid this: parse /proc/self/maps, select a suitable base address for your mapping, move the mapping if it gets too large for the selected area or if another library is mapped in above it, etc. but this quickly gets very difficult. I would suggest a simpler way to solve your problem is simply to open the data file and append to it. If you need to reference the values, keep them in memory structures. Rich. -- Richard Jones Red Hat