[Caml-list] portable truncate

[Caml-list] portable truncate

[Florent Monnier]

Hi,

I would like to know how to truncate a file under MS/Windows.
When I use any of the functions Unix.(LargeFile.)[f]truncate I get
these exceptions:

Exception: Invalid_argument "Unix.truncate not implemented".
Exception: Invalid_argument "Unix.LargeFile.ftruncate not implemented".

1) I thought that the module Unix only contains portable things for
Windows, so it's not true?
Or maybe this is only an issue with the protz's OCaml, not OCaml
itself that we can install or compile in another way ?

2) what other ways can I use to truncate my files in a portable way?
The files that I want to truncate are opened with Unix.openfile, and
so are of type Unix.file_descr.

I tryed to replicate these stubs without the macro that removes these
functions but I still get the exception from MyUnix.ftruncate: Failure
"ftruncate"

But if I use MyUnix.truncate instead (the input is then the filename,
not the Unix.file_descr as I would like), then it works.

So if I have to use this as a workaround I guess I have to close the
file_descr before and open it again after ? Or is it safe to do it at
the same time ?

I would still prefer a workarround that doesn't require to use
anything else than ocaml and its stdlib.


/* Xavier Leroy, projet Cristal, INRIA Rocquencourt
 Copyright 1996 Institut National de Recherche en Informatique et
 en Automatique.  All rights reserved.  This file is distributed
 under the terms of the GNU Library General Public License, with
 the special exception on linking described in the file LICENSE of OCaml.
*/
#include <sys/types.h>
#include <fail.h>
#include <mlvalues.h>
#include <io.h>
#include "unixsupport.h"
#include <unistd.h>

CAMLprim value myunix_ftruncate(value fd, value len)
{
  if (ftruncate(Int_val(fd), Long_val(len)) == -1)
    caml_failwith("ftruncate");
  return Val_unit;
}

CAMLprim value myunix_ftruncate_64(value fd, value len)
{
  if (ftruncate(Int_val(fd), Int64_val(len)) == -1)
    caml_failwith("ftruncate64");
  return Val_unit;
}

CAMLprim value myunix_truncate(value path, value len)
{
  if (truncate(String_val(path), Long_val(len)) == -1)
    caml_failwith("truncate");
  return Val_unit;
}

CAMLprim value myunix_truncate_64(value path, value len)
{
  if (truncate(String_val(path), Int64_val(len)) == -1)
    caml_failwith("truncate64");
  return Val_unit;
}


$ cat myUnix.ml

(* Xavier Leroy, projet Cristal, INRIA Rocquencourt
 Copyright 1996 Institut National de Recherche en Informatique et
 en Automatique.  All rights reserved.  This file is distributed
 under the terms of the GNU Library General Public License, with
 the special exception on linking described the file LICENSE of OCaml.
*)

type file_descr = Unix.file_descr

external ftruncate : file_descr -> int -> unit = "myunix_ftruncate"
external truncate : string -> int -> unit = "myunix_truncate"

module LargeFile = struct
  external ftruncate : file_descr -> int64 -> unit = "myunix_ftruncate_64"
  external truncate : string -> int64 -> unit = "myunix_truncate_64"
end


$ export CC='/bin/i686-w64-mingw32-gcc'
$ $CC -c -I c:/OCaml/lib/caml ftruncate.c
$ ocamlc -c myUnix.mli
$ ocamlc -c myUnix.ml
$ ocamlmklib -o myunix myUnix.cmo ftruncate.o


-- 
Cheers

Re: [Caml-list] portable truncate

[Mihamina Rakotomandimby]

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

On 2013-07-26 13:35, Florent Monnier wrote:
> I would like to know how to truncate a file under MS/Windows.
> When I use any of the functions Unix.(LargeFile.)[f]truncate I get
> these exceptions:
>
> Exception: Invalid_argument "Unix.truncate not implemented".
> Exception: Invalid_argument "Unix.LargeFile.ftruncate not implemented".
>
> 1) I thought that the module Unix only contains portable things for
> Windows, so it's not true?


Well, reading 
http://caml.inria.fr/pub/docs/manual-ocaml/libref/Unix.html I see:
Interface to the Unix system

It seems pretty clear to me...

-- 
RMA.


[-- Attachment #2: Type: text/html, Size: 1155 bytes --]

Re: [Caml-list] portable truncate

[Török Edwin]

On 07/26/2013 01:46 PM, Mihamina Rakotomandimby wrote:
> On 2013-07-26 13:35, Florent Monnier wrote:
>> I would like to know how to truncate a file under MS/Windows.
>> When I use any of the functions Unix.(LargeFile.)[f]truncate I get
>> these exceptions:
>>
>> Exception: Invalid_argument "Unix.truncate not implemented".
>> Exception: Invalid_argument "Unix.LargeFile.ftruncate not implemented".
>>
>> 1) I thought that the module Unix only contains portable things for
>> Windows, so it's not true?
> 
> 
> Well, reading http://caml.inria.fr/pub/docs/manual-ocaml/libref/Unix.html I see:
> Interface to the Unix system
> 
> It seems pretty clear to me...

There is a list of Unix functions that are not implemented on Windows here:
http://caml.inria.fr/pub/docs/manual-ocaml/manual036.html

--Edwin

Re: [Caml-list] portable truncate

[Daniel Bünzli]

Le vendredi, 26 juillet 2013 à 11:35, Florent Monnier a écrit :
> 1) I thought that the module Unix only contains portable things for
> Windows, so it's not true?

No it's not. What is not implemented/differs is documented here:

http://caml.inria.fr/pub/docs/manual-ocaml/manual036.html

Best,

Daniel

[Caml-list] Re: portable truncate

[Florent Monnier]

Hi,

2013/07/26, Florent Monnier wrote:
[...]
> 1) I thought that the module Unix only contains portable things for
> Windows, so it's not true?

This page indeed gives a list of unimplemented functions at the end:
[1] http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual036.html

but it depends which ocaml variants is used.

This page above says:
"The Cygwin port of OCaml fully implements all functions from the Unix module.
 The native Win32 ports implement a subset of them."

So this page talks about 2 ocaml variants:
- Cygwin ocaml,
- and native Win32 ocaml.

This other page, gives informations for 3 ocanl variants:
[2] http://caml.inria.fr/ocaml/portability.en.html

which are
- native Microsoft
- native MinGW
- and Cygwin

I should probably understand frm this that both
"native Microsoft" and "native MinGW" are native Win32 ocaml.

I made the error to think that native Win32 port only refers to ocaml
compiled with MSVC, while it seems that it also include protz's ocaml
that is a MinGW variant.

I'm still not sure though that compiling with MSVC or MinGW can be
considered equivalent.
And actually they seem not considering this page:
[2] http://caml.inria.fr/ocaml/portability.en.html

And considering the small test (called "MyUnix" below), it shows that
it seems that the function truncate() could work just fine.


[...]
> I tryed to replicate these stubs without the macro that removes these
> functions but I still get the exception from MyUnix.ftruncate: Failure
> "ftruncate"
>
> But if I use MyUnix.truncate instead (the input is then the filename,
> not the Unix.file_descr as I would like), then it works.
>
[...]
>
> /* Xavier Leroy, projet Cristal, INRIA Rocquencourt
>  Copyright 1996 Institut National de Recherche en Informatique et
>  en Automatique.  All rights reserved.  This file is distributed
>  under the terms of the GNU Library General Public License, with
>  the special exception on linking described in the file LICENSE of OCaml.
> */
> #include <sys/types.h>
> #include <fail.h>
> #include <mlvalues.h>
> #include <io.h>
> #include "unixsupport.h"
> #include <unistd.h>
>
> CAMLprim value myunix_ftruncate(value fd, value len)
> {
>   if (ftruncate(Int_val(fd), Long_val(len)) == -1)
>     caml_failwith("ftruncate");
>   return Val_unit;
> }
>
> CAMLprim value myunix_ftruncate_64(value fd, value len)
> {
>   if (ftruncate(Int_val(fd), Int64_val(len)) == -1)
>     caml_failwith("ftruncate64");
>   return Val_unit;
> }
>
> CAMLprim value myunix_truncate(value path, value len)
> {
>   if (truncate(String_val(path), Long_val(len)) == -1)
>     caml_failwith("truncate");
>   return Val_unit;
> }
>
> CAMLprim value myunix_truncate_64(value path, value len)
> {
>   if (truncate(String_val(path), Int64_val(len)) == -1)
>     caml_failwith("truncate64");
>   return Val_unit;
> }
>
>
> $ cat myUnix.ml
>
> (* Xavier Leroy, projet Cristal, INRIA Rocquencourt
>  Copyright 1996 Institut National de Recherche en Informatique et
>  en Automatique.  All rights reserved.  This file is distributed
>  under the terms of the GNU Library General Public License, with
>  the special exception on linking described the file LICENSE of OCaml.
> *)
>
> type file_descr = Unix.file_descr
>
> external ftruncate : file_descr -> int -> unit = "myunix_ftruncate"
> external truncate : string -> int -> unit = "myunix_truncate"
>
> module LargeFile = struct
>   external ftruncate : file_descr -> int64 -> unit = "myunix_ftruncate_64"
>   external truncate : string -> int64 -> unit = "myunix_truncate_64"
> end
>
>
> $ export CC='/bin/i686-w64-mingw32-gcc'
> $ $CC -c -I c:/OCaml/lib/caml ftruncate.c
> $ ocamlc -c myUnix.mli
> $ ocamlc -c myUnix.ml
> $ ocamlmklib -o myunix myUnix.cmo ftruncate.o
>

With this test, the function MyUnix.truncate DOES WORK.

I'm still in hope to find a workaround for ftruncate() that could work
with MinGW's toolchain.

-- 
Cheers

RE: [Caml-list] Re: portable truncate

[David Allsopp]

Florent Monnier wrote:
 
> 2013/07/26, Florent Monnier wrote:
> [...]
> > 1) I thought that the module Unix only contains portable things for
> > Windows, so it's not true?
> 
> This page indeed gives a list of unimplemented functions at the end:
> [1] http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual036.html
> 
> but it depends which ocaml variants is used.
> 
> This page above says:
> "The Cygwin port of OCaml fully implements all functions from the Unix
> module.
>  The native Win32 ports implement a subset of them."
> 
> So this page talks about 2 ocaml variants:
> - Cygwin ocaml,
> - and native Win32 ocaml.
> 
> This other page, gives informations for 3 ocanl variants:
> [2] http://caml.inria.fr/ocaml/portability.en.html

That page is slightly out of date as it doesn't refer to the 64 bit MinGW port :o)

> which are
> - native Microsoft
> - native MinGW
> - and Cygwin
> 
> I should probably understand frm this that both "native Microsoft" and
> "native MinGW" are native Win32 ocaml.

Correct

> I made the error to think that native Win32 port only refers to ocaml
> compiled with MSVC, while it seems that it also include protz's ocaml that
> is a MinGW variant.

Native in this context means "not Cygwin", indeed.

> I'm still not sure though that compiling with MSVC or MinGW can be
> considered equivalent.
> And actually they seem not considering this page:
> [2] http://caml.inria.fr/ocaml/portability.en.html

I've done some limited Googling, so may be barking up the wrong tree. I believe that in Microsoft C world, the function ftrunctate is called _chsize (http://msdn.microsoft.com/en-gb/library/aa246710(v=vs.60).aspx) in io.h and that MinGW defined a compatible ftruncate in unistd.h a long time ago. I expect that the reason for not implementing ftruncate in win32unix originally was either that whoever wrote it didn't know about _chsize or at the time it didn't work properly (e.g. on Windows 9x). The Windows API functions SetEndOfFile/SetFileValidData (which actually implement this) look comparatively new (Windows XP+), although Microsoft annoyingly altered the "Minimum supported" sections of MSDN a while ago so "minimum" no longer means "introduced".

Altering unix is relatively easy if you've compiled your own OCaml - edit files in otherlibs/win32unix and use make -f Makefile.nt [opt] to recompile. You can simply copy the replacement files into your ocaml lib directory while experimenting. Patches adding missing functionality to win32unix tend to be accepted quite quickly in my experience...


David

Re: [Caml-list] Re: portable truncate

[Adrien Nader]

On Fri, Jul 26, 2013, Florent Monnier wrote:
> Hi,
> 
> 2013/07/26, Florent Monnier wrote:
> [...]
> > 1) I thought that the module Unix only contains portable things for
> > Windows, so it's not true?
> 
> This page indeed gives a list of unimplemented functions at the end:
> [1] http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual036.html
> 
> but it depends which ocaml variants is used.
> 
> This page above says:
> "The Cygwin port of OCaml fully implements all functions from the Unix module.
>  The native Win32 ports implement a subset of them."
> 
> So this page talks about 2 ocaml variants:
> - Cygwin ocaml,
> - and native Win32 ocaml.
> 
> This other page, gives informations for 3 ocanl variants:
> [2] http://caml.inria.fr/ocaml/portability.en.html
> 
> which are
> - native Microsoft
> - native MinGW
> - and Cygwin
> 
> I should probably understand frm this that both
> "native Microsoft" and "native MinGW" are native Win32 ocaml.

Yes, they are.

> I made the error to think that native Win32 port only refers to ocaml
> compiled with MSVC, while it seems that it also include protz's ocaml
> that is a MinGW variant.
> 
> I'm still not sure though that compiling with MSVC or MinGW can be
> considered equivalent.
> And actually they seem not considering this page:
> [2] http://caml.inria.fr/ocaml/portability.en.html

The compiler differs but otherwise the differences are small. The libc
that is used on Windows is Microsoft's (actually called CRT); mingw*
provide some additions to that and apparently, ftruncate is provided.

In otherlibs/win32unix/unix.ml, you have:
  let ftruncate fd len = invalid_arg "Unix.ftruncate not implemented"

And in unistd.h, you have:

  #ifndef FTRUNCATE_DEFINED
  #define FTRUNCATE_DEFINED
  /* This is defined as a real library function to allow autoconf
     to verify its existence. */
  #if !defined(NO_OLDNAMES) || defined(_POSIX)
  int ftruncate(int, off32_t);
  int ftruncate64(int, off64_t);
  int truncate(const char *, off32_t);
  int truncate64(const char *, off64_t);
  #ifndef __CRT__NO_INLINE
  __CRT_INLINE int ftruncate(int __fd, off32_t __length)
  {
    return _chsize (__fd, __length);
  }
  #endif /* !__CRT__NO_INLINE */
  #else
  int ftruncate(int, _off_t);
  int ftruncate64(int, _off64_t);
  int truncate(const char *, _off_t);
  int truncate64(const char *, _off64_t);
  #ifndef __CRT__NO_INLINE
  __CRT_INLINE int ftruncate(int __fd, _off_t __length)
  {
    return _chsize (__fd, __length);
  }
  #endif /* !__CRT__NO_INLINE */
  #endif
  #endif /* FTRUNCATE_DEFINED */

Considering this, you should probably make a patch to add a file
win32unix/ftruncate.c which use the implementation above and replace the
implementation of ftruncate* in win32unix/unix.ml.
I don't know what is the purpose of the __CRT_NO_INLINE define; I've
already asked and will propagate the answer when I get it. It would be
good to have truncate*() too.

(and submit it to the bug tracker :-) )

-- 
Adrien Nader

Re: [Caml-list] Re: portable truncate

[Florent Monnier]

Hi David,

2013/07/27, David Allsopp wrote:
[...]
>> This page above says:
>> "The Cygwin port of OCaml fully implements all functions from the Unix
>> module.
>>  The native Win32 ports implement a subset of them."
>>
>> So this page talks about 2 ocaml variants:
>> - Cygwin ocaml,
>> - and native Win32 ocaml.
>>
>> This other page, gives informations for 3 ocanl variants:
>> [2] http://caml.inria.fr/ocaml/portability.en.html
>
> That page is slightly out of date as it doesn't refer to the 64 bit MinGW
> port :o)

We are then in a RTFM vs. WTFM situation :o)

We may then take this opportunity to import an up to date information
on ocaml.org


>> which are
>> - native Microsoft
>> - native MinGW
>> - and Cygwin
>>
>> I should probably understand frm this that both "native Microsoft" and
>> "native MinGW" are native Win32 ocaml.
>
> Correct
>
>> I made the error to think that native Win32 port only refers to ocaml
>> compiled with MSVC, while it seems that it also include protz's ocaml
>> that is a MinGW variant.
>
> Native in this context means "not Cygwin", indeed.

Except that MinGW comes with Cygwin, and so is also automatically
updated from its package management.

As a result it may not be that obvious for beginners and non-experts
if it's not written and explained explicitly.

Also on ocaml.org this ocaml variant is called:
  "Cygwin-based native Win32 port"

Which also doesn't help to know if mingw variant should be considered
as a cygwin variant or a win32 variant.

> I've done some limited Googling, so may be barking up the wrong tree. I
> believe that in Microsoft C world, the function ftrunctate is called _chsize
> (http://msdn.microsoft.com/en-gb/library/aa246710(v=vs.60).aspx) in io.h and
> that MinGW defined a compatible ftruncate in unistd.h a long time ago.

I did similar searches yesterday and already tried _chsize in MyUnix like this:
#define ftruncate _chsize
But it does not work.
The file is not truncated, and _chsize returns -1 which is the error code.


> I expect that the reason for not implementing ftruncate in win32unix
> originally was either that whoever wrote it didn't know about _chsize or at
> the time it didn't work properly (e.g. on Windows 9x). The Windows API
> functions SetEndOfFile/SetFileValidData (which actually implement this) look
> comparatively new (Windows XP+), although Microsoft annoyingly altered the
> "Minimum supported" sections of MSDN a while ago so "minimum" no longer
> means "introduced".

Anyway, MinGW has to use the OS, but is not a product from MS.
I believe that we can do a good use of this "feature".


> Altering unix is relatively easy if you've compiled your own OCaml - edit
> files in otherlibs/win32unix and use make -f Makefile.nt [opt] to recompile.
> You can simply copy the replacement files into your ocaml lib directory
> while experimenting. Patches adding missing functionality to win32unix tend
> to be accepted quite quickly in my experience...

I've made an attempt to compile the ocaml sources under Cygwin using
the MinGW toolchain, but I didn't succeeded.

I can still alter the unix module as I have shown in the previous
email with the module called MyUnix.

But if I want to share my code this won't help me much, I should bring
the fix upstream.
In case I can find a fix (I have good hope, but I'm not sure yet), do
you think I should use mantis, or just send a patch to protz ?

-- 
Regards

Re: [Caml-list] Re: portable truncate

[Adrien Nader]

On Sat, Jul 27, 2013, Adrien Nader wrote:
> Considering this, you should probably make a patch to add a file
> win32unix/ftruncate.c which use the implementation above and replace the
> implementation of ftruncate* in win32unix/unix.ml.
> I don't know what is the purpose of the __CRT_NO_INLINE define; I've
> already asked and will propagate the answer when I get it. It would be
> good to have truncate*() too.
> 
> (and submit it to the bug tracker :-) )

Actually, you should also check the return code and if an error has
occured, check errno to raise the corresponding exception.

As you can see, there is a variety of possible errors:
  http://msdn.microsoft.com/en-us/library/dk925tyb%28v=VS.80%29.aspx

-- 
Adrien Nader

Re: [Caml-list] Re: portable truncate

[Florent Monnier]

2013/07/27, Adrien Nader wrote:
> On Fri, Jul 26, 2013, Florent Monnier wrote:
[...]
>> I'm still not sure though that compiling with MSVC or MinGW can be
>> considered equivalent.
>> And actually they seem not considering this page:
>> [2] http://caml.inria.fr/ocaml/portability.en.html
>
> The compiler differs but otherwise the differences are small.

I have already understood that MinGW will produce a resulting binary
that equivalently makes use of MS' OS libs. But as far as I have
understood it also provide an additional layer which goal is to make
easier to compile something that comes from the Unix world. As a
result I expected to get a fully functional ocaml Unix module.


>                                                               The libc
> that is used on Windows is Microsoft's (actually called CRT); mingw*
> provide some additions to that and apparently, ftruncate is provided.

- truncate()  DOES work on my computer
- ftruncate() / _chsize()  DOESN'T


> In otherlibs/win32unix/unix.ml, you have:
>   let ftruncate fd len = invalid_arg "Unix.ftruncate not implemented"

I see,
I expected that a mingw variant would use "otherlibs/unix/"
So it seems I was wrong again on that point too.

> And in unistd.h, you have:
[...]
>   __CRT_INLINE int ftruncate(int __fd, off32_t __length)
>   {
>     return _chsize (__fd, __length);
>   }
[...]
> Considering this, you should probably make a patch to add a file
> win32unix/ftruncate.c which use the implementation above and replace the
> implementation of ftruncate* in win32unix/unix.ml.

Don't you think that "win32unix/" should remain as is for the MS
compiler, and that mingw variant should just switch to
"otherlibs/unix/" ?

> I don't know what is the purpose of the __CRT_NO_INLINE define; I've
> already asked and will propagate the answer when I get it. It would be
> good to have truncate*() too.
>
> (and submit it to the bug tracker :-) )

I do have only one test machine with Windows Seven Starter to make
tests. And I do not have the MS' compiler.

Are there other people arround living in Rennes (the town where I
live) that would accept to provide me access to other test machines
under MS/Windows ? Like for example Windows 8 ?
(So that I could do more serious tests.)

I would be interested to access Mac OS X too, also for portability works.
(Someone from the lug gave me a Mac in the past to make some
portability work, but this machine is too old now for this kind of
tasks.)

-- 
Regards

Re: [Caml-list] Re: portable truncate

[Adrien Nader]

On Sat, Jul 27, 2013, Florent Monnier wrote:
> 2013/07/27, Adrien Nader wrote:
> > On Fri, Jul 26, 2013, Florent Monnier wrote:
> [...]
> >> I'm still not sure though that compiling with MSVC or MinGW can be
> >> considered equivalent.
> >> And actually they seem not considering this page:
> >> [2] http://caml.inria.fr/ocaml/portability.en.html
> >
> > The compiler differs but otherwise the differences are small.
> 
> I have already understood that MinGW will produce a resulting binary
> that equivalently makes use of MS' OS libs. But as far as I have
> understood it also provide an additional layer which goal is to make
> easier to compile something that comes from the Unix world. As a
> result I expected to get a fully functional ocaml Unix module.

It's a very minimal layer. Microsoft CRT cannot be used for things like
printing properly long doubles or similar things. Its math
implementation is also lacking some C99 support (plus it's not that
fast). That's where the mingw CRT will provide you more things. For
others, it's the Microsoft one.

> >                                                               The libc
> > that is used on Windows is Microsoft's (actually called CRT); mingw*
> > provide some additions to that and apparently, ftruncate is provided.
> 
> - truncate()  DOES work on my computer
> - ftruncate() / _chsize()  DOESN'T

That's surprising and you should really try to see which error you get
throw errno. You can also try a tool like FileMonitor (or whatever it's
called nowadays) to see how it fails.

> > In otherlibs/win32unix/unix.ml, you have:
> >   let ftruncate fd len = invalid_arg "Unix.ftruncate not implemented"
> 
> I see,
> I expected that a mingw variant would use "otherlibs/unix/"
> So it seems I was wrong again on that point too.
> 
> > And in unistd.h, you have:
> [...]
> >   __CRT_INLINE int ftruncate(int __fd, off32_t __length)
> >   {
> >     return _chsize (__fd, __length);
> >   }
> [...]
> > Considering this, you should probably make a patch to add a file
> > win32unix/ftruncate.c which use the implementation above and replace the
> > implementation of ftruncate* in win32unix/unix.ml.
> 
> Don't you think that "win32unix/" should remain as is for the MS
> compiler, and that mingw variant should just switch to
> "otherlibs/unix/" ?

No. Mingw is not posix. Just think about fork(): it cannot exist on
Windows currently without resorting to the big layer that Cygwin
provides (along with its performance hit).

Btw, you can have a look at this entry in the Cygwin FAQ to better
understand what "cygwin" means:
  http://cygwin.com/faq.html#faq.api.fork

You can also take a look at the C files in otherlibs/win32unix: there's
quite a lot of work that is done to provide a matching API.

> > I don't know what is the purpose of the __CRT_NO_INLINE define; I've
> > already asked and will propagate the answer when I get it. It would be
> > good to have truncate*() too.
> >
> > (and submit it to the bug tracker :-) )
> 
> I do have only one test machine with Windows Seven Starter to make
> tests. And I do not have the MS' compiler.
> 
> Are there other people arround living in Rennes (the town where I
> live) that would accept to provide me access to other test machines
> under MS/Windows ? Like for example Windows 8 ?
> (So that I could do more serious tests.)

You can use the eval editions that are available on Microsoft's website.
They're time-limited (one month for some) and after that they will
shutdown after 1 hour but that's not an issue for testing, especially
for VMs. There are also ways to extend the original one-month period
that are documented on MSDN.

(you can also reinstall: qemu's cache=unsafe disk parameter makes
installation insanely fast; yypkg's mingw-builds should also make
installation of a build environment much faster)

-- 
Adrien Nader

Re: [Caml-list] portable truncate

[Florent Monnier]

2013/07/27, ygrek wrote:
> On Fri, 26 Jul 2013 12:35:03 +0200
> Florent Monnier wrote:
>
>> I would like to know how to truncate a file under MS/Windows.
>
> There is a patch sitting for ExtUnix implementing ftruncate.
> if you are willing to test it - it could be integrated into extunix
> and maybe further find it's way into upstream.

Sorry I don't know how to switch language to english in Cygwin.
On Linux I export LANG and/or LANGUAGE but here it doesn't do the trick.

$ sh configure
'C:\cygwin\bin\flexlink' n'est pas reconnu en tant que commande interne
ou externe, un programme ex▒cutable ou un fichier de commandes.
'C:\cygwin\bin\flexlink' n'est pas reconnu en tant que commande interne
ou externe, un programme ex▒cutable ou un fichier de commandes.
'C:\cygwin\bin\flexlink' n'est pas reconnu en tant que commande interne
ou externe, un programme ex▒cutable ou un fichier de commandes.
E: Field 'native_dynlink' is not set: Field 'flexdll_version' is not
set: Command 'C:\cygwin\bin\flexlink -help >
"C:\cygwin\tmp\oasis-374005.txt"' terminated with error code 1

$ which flexlink
/usr/bin/flexlink

$ ls c:/cygwin/bin/flexlink
c:/cygwin/bin/flexlink*

-- 
Regards

Re: [Caml-list] portable truncate

[Adrien Nader]

On Sat, Jul 27, 2013, Florent Monnier wrote:
> 2013/07/27, ygrek wrote:
> > On Fri, 26 Jul 2013 12:35:03 +0200
> > Florent Monnier wrote:
> >
> >> I would like to know how to truncate a file under MS/Windows.
> >
> > There is a patch sitting for ExtUnix implementing ftruncate.
> > if you are willing to test it - it could be integrated into extunix
> > and maybe further find it's way into upstream.
> 
> Sorry I don't know how to switch language to english in Cygwin.
> On Linux I export LANG and/or LANGUAGE but here it doesn't do the trick.
> 
> $ sh configure

First, you don't run configure for Windows:
  cp config/Makefile.mingw config/Makefile (or mingw64 if you want 64b)
  cp config/m-nt.h config/m.h
  cp config/s-nt.h config/s.h

  make -f Makefile.nt world.opt
  make -f Makefile.nt install

-- 
Adrien Nader

Re: [Caml-list] Re: portable truncate

[Florent Monnier]

2013/07/27, Adrien Nader wrote:
[...]
> It's a very minimal layer. Microsoft CRT cannot be used for things like
> printing properly long doubles or similar things. Its math
> implementation is also lacking some C99 support (plus it's not that
> fast). That's where the mingw CRT will provide you more things. For
> others, it's the Microsoft one.

mingw's additional lib has to be used if we're using thread for
example if I have understood correctly.

>> - truncate()  DOES work on my computer
>> - ftruncate() / _chsize()  DOESN'T
>
> That's surprising and you should really try to see which error you get
> throw errno. You can also try a tool like FileMonitor (or whatever it's
> called nowadays) to see how it fails.

I get EBADF

http://msdn.microsoft.com/en-us/library/dk925tyb%28v=VS.80%29.aspx

EBADF is if the specified file is read-only or the descriptor is invalid

the file has write access for everyone.
And the file descriptor is converted the same way than everywhere else.

Then I noticed that I copied the stub from otherlibs/unix/ not from
otherlibs/win32unix/
so the file descr is converted by Int_val() instead of Handle_val().

But now I still get : EBADF


>> Don't you think that "win32unix/" should remain as is for the MS
>> compiler, and that mingw variant should just switch to
>> "otherlibs/unix/" ?
>
> No. Mingw is not posix. Just think about fork(): it cannot exist on
> Windows currently without resorting to the big layer that Cygwin
> provides (along with its performance hit).
>
> Btw, you can have a look at this entry in the Cygwin FAQ to better
> understand what "cygwin" means:
>   http://cygwin.com/faq.html#faq.api.fork

Interesting, thanks a lot.

> You can also take a look at the C files in otherlibs/win32unix: there's
> quite a lot of work that is done to provide a matching API.

Thanks, it made me discovered about the Handle_val() convertion that
only exists in win32unix.


> You can use the eval editions that are available on Microsoft's website.
> They're time-limited (one month for some) and after that they will
> shutdown after 1 hour but that's not an issue for testing, especially
> for VMs. There are also ways to extend the original one-month period
> that are documented on MSDN.
>
> (you can also reinstall: qemu's cache=unsafe disk parameter makes
> installation insanely fast; yypkg's mingw-builds should also make
> installation of a build environment much faster)

Thanks a lot Adrien,
your answers are very interesting!

-- 
Regards

Re: [Caml-list] portable truncate

[Florent Monnier]

>> > There is a patch sitting for ExtUnix implementing ftruncate.
>> > if you are willing to test it - it could be integrated into extunix
>> > and maybe further find it's way into upstream.
>>
>> Sorry I don't know how to switch language to english in Cygwin.
>> On Linux I export LANG and/or LANGUAGE but here it doesn't do the trick.
>>
>> $ sh configure
>
> First, you don't run configure for Windows:
>   cp config/Makefile.mingw config/Makefile (or mingw64 if you want 64b)

Sorry Adrien, but ygrek was talking about extunix, another lib than
what is in otherlibs of the ocaml sources ;)

--

Re: [Caml-list] Re: portable truncate

[Adrien Nader]

On Sat, Jul 27, 2013, Florent Monnier wrote:
> 2013/07/27, Adrien Nader wrote:
> [...]
> > It's a very minimal layer. Microsoft CRT cannot be used for things like
> > printing properly long doubles or similar things. Its math
> > implementation is also lacking some C99 support (plus it's not that
> > fast). That's where the mingw CRT will provide you more things. For
> > others, it's the Microsoft one.
> 
> mingw's additional lib has to be used if we're using thread for
> example if I have understood correctly.

What makes you think so?

A quick look seems to indicate the opposite. There is no specific thread
support provided by mingw*.
There are pthreads-win32 and winpthreads but I don't see the build
system use them on Windows; I see calls to CreateThead() (typical win32
stuff) however for instance.

> >> - truncate()  DOES work on my computer
> >> - ftruncate() / _chsize()  DOESN'T
> >
> > That's surprising and you should really try to see which error you get
> > throw errno. You can also try a tool like FileMonitor (or whatever it's
> > called nowadays) to see how it fails.
> 
> I get EBADF
> 
> http://msdn.microsoft.com/en-us/library/dk925tyb%28v=VS.80%29.aspx
> 
> EBADF is if the specified file is read-only or the descriptor is invalid
> 
> the file has write access for everyone.
> And the file descriptor is converted the same way than everywhere else.
> 
> Then I noticed that I copied the stub from otherlibs/unix/ not from
> otherlibs/win32unix/
> so the file descr is converted by Int_val() instead of Handle_val().
> 
> But now I still get : EBADF

And you've opened the file as read-write too?
If so, you should maybe try Process Monitor to make sure everything was
done as expected:
  http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

-- 
Adrien Nader

Re: [Caml-list] Re: portable truncate

[Florent Monnier]

2013/7/27, Adrien Nader wrote:
> On Sat, Jul 27, 2013, Florent Monnier wrote:
[...]
>> mingw's additional lib has to be used if we're using thread for
>> example if I have understood correctly.
>
> What makes you think so?

The sentence below that is on the website of the project:

"It does depend on a number of DLLs provided by Microsoft themselves,
as components of the operating system; most notable among these is
MSVCRT.DLL, the Microsoft C runtime library. Additionally, threaded
applications must ship with a freely distributable thread support DLL,
provided as part of MinGW itself"

> A quick look seems to indicate the opposite. There is no specific thread
> support provided by mingw*.
> There are pthreads-win32 and winpthreads but I don't see the build
> system use them on Windows; I see calls to CreateThead() (typical win32
> stuff) however for instance.
>
>> >> - truncate()  DOES work on my computer
>> >> - ftruncate() / _chsize()  DOESN'T
>> >
>> > That's surprising and you should really try to see which error you get
>> > throw errno. You can also try a tool like FileMonitor (or whatever it's
>> > called nowadays) to see how it fails.
>>
>> I get EBADF
>>
>> http://msdn.microsoft.com/en-us/library/dk925tyb%28v=VS.80%29.aspx
>>
>> EBADF is if the specified file is read-only or the descriptor is invalid
>>
>> the file has write access for everyone.
>> And the file descriptor is converted the same way than everywhere else.
>>
>> Then I noticed that I copied the stub from otherlibs/unix/ not from
>> otherlibs/win32unix/
>> so the file descr is converted by Int_val() instead of Handle_val().
>>
>> But now I still get : EBADF
>
> And you've opened the file as read-write too?

let open_flags = [Unix.O_RDWR]
O_RDWR  (* Open for reading and writing *)


> If so, you should maybe try Process Monitor to make sure everything was
> done as expected:
>   http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

Thanks Adrien, I'll have a look at this tomorrow.

-- 
Regards

Re: [Caml-list] Re: portable truncate

[Adrien Nader]

On Sat, Jul 27, 2013, Florent Monnier wrote:
> 2013/7/27, Adrien Nader wrote:
> > On Sat, Jul 27, 2013, Florent Monnier wrote:
> [...]
> >> mingw's additional lib has to be used if we're using thread for
> >> example if I have understood correctly.
> >
> > What makes you think so?
> 
> The sentence below that is on the website of the project:
> 
> "It does depend on a number of DLLs provided by Microsoft themselves,
> as components of the operating system; most notable among these is
> MSVCRT.DLL, the Microsoft C runtime library. Additionally, threaded
> applications must ship with a freely distributable thread support DLL,
> provided as part of MinGW itself"

I'd be on it being outdated.

The "pthreads" libraries on Windows are meant to ease porting of
applications to Windows and I wouldn't be surprised that pthreads-win32
had first been used for that. Then when thread support was finally
implemented without requiring pthreads-win32, updating the documentation
was of course forgotten. :-) 

That just an hypothesis but it seems likely enough for me.

-- 
Adrien Nader

Re: [Caml-list] Re: portable truncate

[Török Edwin]

On 07/27/2013 10:53 PM, Adrien Nader wrote:
> On Sat, Jul 27, 2013, Florent Monnier wrote:
>> 2013/7/27, Adrien Nader wrote:
>>> On Sat, Jul 27, 2013, Florent Monnier wrote:
>> [...]
>>>> mingw's additional lib has to be used if we're using thread for
>>>> example if I have understood correctly.
>>>
>>> What makes you think so?
>>
>> The sentence below that is on the website of the project:
>>
>> "It does depend on a number of DLLs provided by Microsoft themselves,
>> as components of the operating system; most notable among these is
>> MSVCRT.DLL, the Microsoft C runtime library. Additionally, threaded
>> applications must ship with a freely distributable thread support DLL,
>> provided as part of MinGW itself"
> 
> I'd be on it being outdated.
> 
> The "pthreads" libraries on Windows are meant to ease porting of
> applications to Windows and I wouldn't be surprised that pthreads-win32
> had first been used for that. Then when thread support was finally
> implemented without requiring pthreads-win32, updating the documentation
> was of course forgotten. :-) 
> 
> That just an hypothesis but it seems likely enough for me.
> 

There was a time when some MinGW-compiled applications depended on mingw10.dll that had to setup/cleanup something related to exceptions. 
According to [1] in mingw-w64 this is no longer required because their runtime includes the required support:

I haven't used MinGW (or in fact Windows) since quite a long time so I don't know what the situation is today, but since mingw-w64 can build both 32 and 64-bit applications this is probably not an issue anymore.

http://sourceforge.net/apps/trac/mingw-w64/wiki/Feature%20list

--Edwin