Re: [Caml-list] memory mapped files with bigarray on windows

[Gerd Stolpmann <info@gerd-stolpmann.de>]

[-- Attachment #1: Type: text/plain, Size: 2914 bytes --]

Am Donnerstag, den 18.09.2014, 08:04 +0000 schrieb Bauer, Christoph:
> Dear All, 
> 
>  
> 
> my program maps  several blocks of a file into memory with
> Bigarray.Array1.map_file for writing.
> 
> Finally the file handle is closed with Unix.close.

You can close the file handle immediately after establishing the
mapping. The handle is only needed here as a reference of the file to
map, but it doesn't influence the way the file is written. In
particular, a close doesn't flush the contents to disk.

> On local files system everything works as expected.

Which is not guaranteed by POSIX, but normally works for all OS that
deeply integrate memory mapping. There is the msync() system call for
ensuring that all changes are finally reflected by the file.

>  But if the file is on a network drive,
> 
> the resulting file is after the close still  locked [1] and it
> contains just zeros.

Right, this is a known problem. Call msync(). I suggest it would be
easiest if msync() was part of the Bigarray module, but you can also
create your own wrapper.

Note that unmapping usually triggers the synchronization, but it doesn't
wait until the data is written (like calling msync() with the MS_ASYNC
flag).

> My workaround is, to call  Gc.full_major() before the close. Then all
> remaining bigarrays are
> 
> finalized (the finalizer calls caml_ba_unmap_file(), see
> bigarray_stubs.c).
> 

As noted, you cannot be sure that the data is really written at this
point. There is no way around msyncing the file.

Gerd

>  
> 
> Calling Gc.full_major () is somewhat dissatisfying. But I’m not sure
> how it could be solved more cleanly.
> 
> I would expect that Unix.close should unmap all remaining views of the
> file. But Unix.close knows nothing
> 
> about the bigarray mapping. 
> 
>  
> 
> Maybe a special Bigarray.unmap is the solution.
> 
>  
> 
> Does anyone has a better idea?
> 
>  
> 
> Thanks,
> 
> Christoph Bauer
> 
>  
> 
> [1] You can’t delete it in the windows explorer.
> 
>  
> 
>  
> 
> 
> -----------------
> Siemens Industry Software GmbH & Co. KG; Anschrift: Franz-Geuer-Str.
> 10, 50823 Köln;
> Kommanditgesellschaft: Sitz der Gesellschaft: Köln; Registergericht:
> Amtsgericht Köln, HRA 28227;
> Geschäftsführung und persönlich haftender Gesellschafter: Siemens
> Industry Software Management GmbH;
> Geschäftsführer: Urban August, Daniel Trebes; Sitz der Gesellschaft:
> Köln; Registergericht: Amtsgericht Köln, HRB 70858
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]