[Caml-list] [ANN] Bos 0.1.0

[Caml-list] [ANN] Bos 0.1.0

[Daniel Bünzli]

Hello, 

I'd like to announce the first release of bos:

```
Bos provides support for basic and robust interaction with the operating system in OCaml. It has functions to access the process environment, parse command line arguments, interact with the file system and run command line programs.

Bos works equally well on POSIX and Windows operating systems.

Bos depends on Rresult, Astring, Fmt, Fpath, Logs and the OCaml Unix library. 
It is distributed under the ISC license.
```

Homepage: http://erratique.ch/software/bos
API docs: http://erratique.ch/software/bos/doc/

Bos can be seen as improved Sys module for programs and scripts that have light OS interaction requirements. Inspired by the excellent work of scsh [0], it seems this path has been pursued more than once in OCaml for example with cash [1] or shcaml [2]. 

Bos however has a slightly different take on this. It does not try to recover the shell's terseness or processing model but rather tries to overcome the shell's brittleness in face of errors, error reporting and insane quoting conventions. The difference between a script and a program is an artificial one and bos tries to encourage you to write programs that do not fail obscurely and evolve gracefully from a quick, small one (a.k.a "script") to a complex one (a.k.a. "program") while keeping good usability for the end user who is the one who eventually gets processing failures in the face.

This first version number is intentionally low, as I don't see bos as fully finished at the moment. Some interfaces could be tweaked and other added in the future (feedback and discussion on the issue tracker is welcome). I did however already write a few programs/scripts that make use of most of the features provided by the library, so the low number should not be interpreted as "alpha" quality software. What is provided here should work as described and if it doesn't, scream on the issue tracker.

Best, 

Daniel


[0] https://scsh.net/
[1] http://pauillac.inria.fr/cash/
[2] http://users.eecs.northwestern.edu/~jesse/code/shcaml/

Re: [Caml-list] [ANN] Bos 0.1.0

[Martin DeMello]

[-- Attachment #1: Type: text/plain, Size: 2678 bytes --]

Looks very nice! Do you think this would be a good route towards getting
ocamlbuild working seamlessly on windows?

martin

On Mon, May 23, 2016 at 2:01 PM, Daniel Bünzli <daniel.buenzli@erratique.ch>
wrote:

> Hello,
>
> I'd like to announce the first release of bos:
>
> ```
> Bos provides support for basic and robust interaction with the operating
> system in OCaml. It has functions to access the process environment, parse
> command line arguments, interact with the file system and run command line
> programs.
>
> Bos works equally well on POSIX and Windows operating systems.
>
> Bos depends on Rresult, Astring, Fmt, Fpath, Logs and the OCaml Unix
> library.
> It is distributed under the ISC license.
> ```
>
> Homepage: http://erratique.ch/software/bos
> API docs: http://erratique.ch/software/bos/doc/
>
> Bos can be seen as improved Sys module for programs and scripts that have
> light OS interaction requirements. Inspired by the excellent work of scsh
> [0], it seems this path has been pursued more than once in OCaml for
> example with cash [1] or shcaml [2].
>
> Bos however has a slightly different take on this. It does not try to
> recover the shell's terseness or processing model but rather tries to
> overcome the shell's brittleness in face of errors, error reporting and
> insane quoting conventions. The difference between a script and a program
> is an artificial one and bos tries to encourage you to write programs that
> do not fail obscurely and evolve gracefully from a quick, small one (a.k.a
> "script") to a complex one (a.k.a. "program") while keeping good usability
> for the end user who is the one who eventually gets processing failures in
> the face.
>
> This first version number is intentionally low, as I don't see bos as
> fully finished at the moment. Some interfaces could be tweaked and other
> added in the future (feedback and discussion on the issue tracker is
> welcome). I did however already write a few programs/scripts that make use
> of most of the features provided by the library, so the low number should
> not be interpreted as "alpha" quality software. What is provided here
> should work as described and if it doesn't, scream on the issue tracker.
>
> Best,
>
> Daniel
>
>
> [0] https://scsh.net/
> [1] http://pauillac.inria.fr/cash/
> [2] http://users.eecs.northwestern.edu/~jesse/code/shcaml/
>
>
>
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3817 bytes --]

Re: [Caml-list] [ANN] Bos 0.1.0

[Daniel Bünzli]

Le mardi, 24 mai 2016 à 00:17, Martin DeMello a écrit :
> Do you think this would be a good route towards getting ocamlbuild working seamlessly on windows?

Well bos does not do any kind of magic to work well on windows except avoiding what needs to be avoided, namely fork(2). Contrary to what it's name suggests the Unix library is not a bad OS abstraction layer and works quite well on windows; even more so with the work David Allsopp put into 4.03.  

At the moment, but it may change in the future, bos is only a more lighter and simpler way to work with the Unix library correctly. Unix programming can be tricky and quite verbose, e.g. to setup program pipelines. Also note that bos' simpler interface may be not be suitable for all programs, i.e. bos doesn't replace Unix.

IIRC (Gabriel may want to comment) to get ocamlbuild working seamlessly on windows one needs to get rid of fork, a few pointless shellouts and make people stop use Sh command specs in their plugins. So I don't think bos necessarily helps here, using Unix conservatively in ocamlbuild should do.

Best,  

Daniel

Re: [Caml-list] [ANN] Bos 0.1.0

[Gabriel Scherer]

There are some parts of the ocamlbuild internals that have their own
implementations of file path handling and quoting/escaping, and that
are a bit too arcane to my taste. (I tend to not change them because
they mostly work well, but they come back to bite us once in a while.)
When I got the announcement for Bos, I indeed thought that it could be
a nice idea to have a look. (I probably won't do it myself in the
short term, my development time is very constrained right now; plus
the fact that I cannot test on Windows myself makes me wary of
changing those parts of the codebase.)

On Mon, May 23, 2016 at 6:43 PM, Daniel Bünzli
<daniel.buenzli@erratique.ch> wrote:
> Le mardi, 24 mai 2016 à 00:17, Martin DeMello a écrit :
>> Do you think this would be a good route towards getting ocamlbuild working seamlessly on windows?
>
> Well bos does not do any kind of magic to work well on windows except avoiding what needs to be avoided, namely fork(2). Contrary to what it's name suggests the Unix library is not a bad OS abstraction layer and works quite well on windows; even more so with the work David Allsopp put into 4.03.
>
> At the moment, but it may change in the future, bos is only a more lighter and simpler way to work with the Unix library correctly. Unix programming can be tricky and quite verbose, e.g. to setup program pipelines. Also note that bos' simpler interface may be not be suitable for all programs, i.e. bos doesn't replace Unix.
>
> IIRC (Gabriel may want to comment) to get ocamlbuild working seamlessly on windows one needs to get rid of fork, a few pointless shellouts and make people stop use Sh command specs in their plugins. So I don't think bos necessarily helps here, using Unix conservatively in ocamlbuild should do.
>
> Best,
>
> Daniel
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] [ANN] Bos 0.1.0

[Adrien Nader]

It's not really fork but Unix.open_process*.

The signatures and documentation sum up the issue:
  val open_process_in : string -> Pervasives.in_channel
  [...] The command is interpreted by the shell /bin/sh (cf. system).

As such, the process command-line needs to be properly quoted. I'm not
going to attempt to get the whole logic right so I'll only give keywords
for things that can be involved: /bin/sh, $SHELL, quoting for posix
shells, quoting for windows command-line (entirely different and 200%
ad-hoc over the course of 20 years), cmd.exe, %COMSPEC% (and probably
more things).

I see the use of a single string as the main issue because it makes
supporting the quoting for windows process invocations very difficult
(the entry point on windows gets a single string with holds every
process argument: argv in main() is actually the result of parsing...).

When spawning with a string array or string list, one isn't usually
concerned about shell escapes: only special characters such as !, ':' or
'&' might matter but the last really do not make sense in this context..

This makes it possible to quote each argument separately before merging
everything (with quoting) and call the process (NB: there are also
_exec* functions in the Microsoft CRT that could help).
I consider moving away from single-string arguments a prerequisite for
proper and sane handling of this. One annoyance for ocamlbuild is that
it exposes that same API to its plugins.

Support for wide-character functions is another change needed for better
Windows support in the OCaml ecosystem. If this has to be done with API
changes or new APIs, maybe it will be the right time to also remove
support (or not offer them at all in the new APIs) for functions that
use a single string to spawn functions.

Back to the topic, as Gabrial said, there is a number of useful
functions in ocamlbuild (see My_Unix for instance iirc) which work
fairly well and are therefore risky changes (that's actually pretty
funny logic). Nevertheless, I don't think they should stay there.

-- 
Adrien Nader

Re: [Caml-list] [ANN] Bos 0.1.0

[SP]

On Mon, May 23, 2016 at 11:01:45PM +0200, Daniel Bünzli wrote:
>I'd like to announce the first release of bos:
>Bos provides support for basic and robust interaction with the operating system in OCaml. It has functions to access the process environment, parse command line arguments, interact with the file system and run command line programs.

Thanks for your contributions Daniel.

What does this mean for Cmdliner? Does it overlap?

[Cmdliner]: http://erratique.ch/software/cmdliner

-- 
	SP

Re: [Caml-list] [ANN] Bos 0.1.0

[Daniel Bünzli]

Le mardi, 24 mai 2016 à 14:57, SP a écrit :
> What does this mean for Cmdliner? Does it overlap?

The answer is in the documentation preamble of the OS.Arg module:  

http://erratique.ch/software/bos/doc/Bos.OS.Arg.html

Best,  

Daniel

Re: [Caml-list] [ANN] Bos 0.1.0

[SP]

On Tue, May 24, 2016 at 03:14:49PM +0200, Daniel Bünzli wrote:
>The answer is in the documentation preamble of the OS.Arg module:
>http://erratique.ch/software/bos/doc/Bos.OS.Arg.html

Thanks

-- 
	SP