Scripting in ocaml

Scripting in ocaml

[Denis Bueno]

I've been writing bash scripts to perform various build- and
development-related tasks, and I don't enjoy it. I won't bore you with
detailed reasons why. The upshot is that I'd like to script in OCaml.

I have considered writing a few camlp4 extensions to make it easier to
write scripts:

  1) create a syntax which grabs environment variables:

    e.g. $FOO would grab the value of the environment variable FOO

  2) some sort of more convenient process interaction, e.g., for piping.

(1) seems pretty straightforward, though I haven't found the time to
implement it yet. (2) is not as clear to me, but I'll think about it
and probably look at scsh.

I googled a bit but couldn't find anything related to this. Has
anything done, or started doing, anything like this?

-Denis

Re: [Caml-list] Scripting in ocaml

[Erik de Castro Lopo]

Denis Bueno wrote:

> I've been writing bash scripts to perform various build- and
> development-related tasks, and I don't enjoy it. I won't bore you with
> detailed reasons why. The upshot is that I'd like to script in OCaml.

Makes a lot of sense. I used to do scripting style tasks in Python
but nowadays I prefer to use Ocaml.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"We are shut up in schools and college recitation rooms for ten or
fifteen years, and come out at last with a belly full of words and
do not know a thing." --  Ralph Waldo Emerson

Re: [Caml-list] Scripting in ocaml

[skaller]

On Thu, 2006-12-21 at 15:34 +1100, Erik de Castro Lopo wrote:
> Denis Bueno wrote:
> 
> > I've been writing bash scripts to perform various build- and
> > development-related tasks, and I don't enjoy it. I won't bore you with
> > detailed reasons why. The upshot is that I'd like to script in OCaml.
> 
> Makes a lot of sense. I used to do scripting style tasks in Python
> but nowadays I prefer to use Ocaml.

As one of the authors of several major pieces of Python,
my comment is that whilst it provides great convenience
and good code structure .. dynamic typing just doesn't
scale.

The big problem using Ocaml (bytecode) for scripting
is probably the ugly dynamic loading support, and 
for long running systems, the lack of unloading support.
(plus the syntax which isn't really well suited to small
scale scripting applications).

As an alternative you might consider Neko and/or NekoML.

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Scripting in ocaml

[Till Varoquaux]

You might want to have a look at cash:
http://pauillac.inria.fr/cash/
It aims at being an ocaml equivalent of scsh. I don't know if it is
still maintained.

chamo (one of cameleon's [http://pauillac.inria.fr/cash/] component)
is also ocaml scriptable. They included an enhanced toplevel. You
could reuse part of there work .

Till
P.S. sorry john for spamming you...

On 12/21/06, skaller <skaller@users.sourceforge.net> wrote:
> On Thu, 2006-12-21 at 15:34 +1100, Erik de Castro Lopo wrote:
> > Denis Bueno wrote:
> >
> > > I've been writing bash scripts to perform various build- and
> > > development-related tasks, and I don't enjoy it. I won't bore you with
> > > detailed reasons why. The upshot is that I'd like to script in OCaml.
> >
> > Makes a lot of sense. I used to do scripting style tasks in Python
> > but nowadays I prefer to use Ocaml.
>
> As one of the authors of several major pieces of Python,
> my comment is that whilst it provides great convenience
> and good code structure .. dynamic typing just doesn't
> scale.
>
> The big problem using Ocaml (bytecode) for scripting
> is probably the ugly dynamic loading support, and
> for long running systems, the lack of unloading support.
> (plus the syntax which isn't really well suited to small
> scale scripting applications).
>
> As an alternative you might consider Neko and/or NekoML.
>
> --
> John Skaller <skaller at users dot sf dot net>
> Felix, successor to C++: http://felix.sf.net
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 06:22:36PM +1100, skaller wrote:
> 
> As one of the authors of several major pieces of Python,
> my comment is that whilst it provides great convenience
> and good code structure .. dynamic typing just doesn't
> scale.
> 
> The big problem using Ocaml (bytecode) for scripting
> is probably the ugly dynamic loading support, and 
> for long running systems, the lack of unloading support.
> (plus the syntax which isn't really well suited to small
> scale scripting applications).

Actually, I find that static typing is the biggest problem I have with
administration script writing in OCaml.  It can be great for bigger
projects where dynamic typing doesn't always scale (as you mentioned),
but for something under thirty lines of code (to pick a number out of
thin air) the syntax requirements imposed by static typing and the
semantic gymnastics that must sometimes be done (string_of_int, et
cetera) can get in the way a bit.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
Ben Franklin: "As we enjoy great Advantages from the Inventions of
others we should be glad of an Opportunity to serve others by any
Invention of ours, and this we should do freely and generously."

Re: [Caml-list] Scripting in ocaml

[skaller]

On Thu, 2006-12-21 at 02:18 -0700, Chad Perrin wrote:
> On Thu, Dec 21, 2006 at 06:22:36PM +1100, skaller wrote:
 
> > The big problem using Ocaml (bytecode) for scripting
> > is probably the ugly dynamic loading support, and 
> > for long running systems, the lack of unloading support.
> > (plus the syntax which isn't really well suited to small
> > scale scripting applications).
> 
> Actually, I find that static typing is the biggest problem I have with
> administration script writing in OCaml.  It can be great for bigger
> projects where dynamic typing doesn't always scale (as you mentioned),
> but for something under thirty lines of code (to pick a number out of
> thin air) the syntax requirements imposed by static typing and the
> semantic gymnastics that must sometimes be done (string_of_int, et
> cetera) can get in the way a bit.

If you examine your
scripting code I'm going to bet you find that most variables
actually have a single statically known type!

What I find with useful with Python code is stuff like:

(a) exec/eval of loaded strings into executable code

(b) scope control, turning dictionaries into scopes
to modify the behaviour of script

This is indeed dynamics .. and requires dynamic typing,
but it isn't really that variables have a changing type,
it seems more about the ability of a system to extend
itself dynamically.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 09:29:15PM +1100, skaller wrote:
> On Thu, 2006-12-21 at 02:18 -0700, Chad Perrin wrote:
> > On Thu, Dec 21, 2006 at 06:22:36PM +1100, skaller wrote:
>  
> > > The big problem using Ocaml (bytecode) for scripting
> > > is probably the ugly dynamic loading support, and 
> > > for long running systems, the lack of unloading support.
> > > (plus the syntax which isn't really well suited to small
> > > scale scripting applications).
> > 
> > Actually, I find that static typing is the biggest problem I have with
> > administration script writing in OCaml.  It can be great for bigger
> > projects where dynamic typing doesn't always scale (as you mentioned),
> > but for something under thirty lines of code (to pick a number out of
> > thin air) the syntax requirements imposed by static typing and the
> > semantic gymnastics that must sometimes be done (string_of_int, et
> > cetera) can get in the way a bit.
> 
> If you examine your
> scripting code I'm going to bet you find that most variables
> actually have a single statically known type!

Actually, I typically don't even use variables in small OCaml scripts.
I occasionally use "let n =", but don't find need to do the reference
thing very often when the body of code is kept small.

The most common problem I have with static typing getting in the way is
having to decide on either multiple print statements or constructing a
string from several different data types to use a single print
statement.  Where something like that takes one short line of code in
the dynamically typed languages I'm used to using, it can be half my
code when doing something similar in an OCaml script.

Plus, y'know, list handling is easier for me in Perl or UCBLogo, with
the former providing better general purpose text processing capability
and the latter having an attractive prefix notation functional syntax
(among other benefits).  Neither compiles to persistent executable
native binaries, though, which (combined with its high level syntax and
several other means of running OCaml code) is really my main reason for
liking OCaml.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
print substr("Just another Perl hacker", 0, -2);

Re: [Caml-list] Scripting in ocaml

[Serge Aleynikov]

skaller wrote:
> On Thu, 2006-12-21 at 15:34 +1100, Erik de Castro Lopo wrote:
>> Denis Bueno wrote:
>>
>>> I've been writing bash scripts to perform various build- and
>>> development-related tasks, and I don't enjoy it. I won't bore you with
>>> detailed reasons why. The upshot is that I'd like to script in OCaml.
>> Makes a lot of sense. I used to do scripting style tasks in Python
>> but nowadays I prefer to use Ocaml.
> 
> As one of the authors of several major pieces of Python,
> my comment is that whilst it provides great convenience
> and good code structure .. dynamic typing just doesn't
> scale.

Could you please illustrate your point by more concrete reasoning. 
Erlang uses dynamic typing (or rather strict typing) and scales very well.

> The big problem using Ocaml (bytecode) for scripting
> is probably the ugly dynamic loading support, and 
> for long running systems, the lack of unloading support.
> (plus the syntax which isn't really well suited to small
> scale scripting applications).

Why is dynamic loading ugly?  In Erlang dynamic loading allows to do 
hot-swappable code-reloading which is a very neat feature for 
long-running systems.

> As an alternative you might consider Neko and/or NekoML.
> 

Regards,

Serge

Re: [Caml-list] Scripting in ocaml

[skaller]

On Thu, 2006-12-21 at 08:30 -0500, Serge Aleynikov wrote:
> skaller wrote:

> > As one of the authors of several major pieces of Python,
> > my comment is that whilst it provides great convenience
> > and good code structure .. dynamic typing just doesn't
> > scale.
> 
> Could you please illustrate your point by more concrete reasoning. 

It isn't a matter of reasoning but experience. Many people believe
static typing is good because it improves the likelihood a program
is correct, perhaps by making it easier for both the human and
machine to reason about the code -- and in particular catch
errors which would otherwise require testing, and even then
the actual location of a bug might be hard to determine.

I guess the arguments for static typing and scalability are
well known. So the reasoning part is clear: we infer larger 
programs without dynamic typing are harder to get right.

That has indeed been my experience. For small programs I can
see all at once on the screen, dynamic typing is ok. For larger
programs, the type system seems to help by abstracting the
code, so even the larger bulk of code can be scanned and
remembered as a whole, if not in detail. 

I guess static typing allows some kind of divide-and-conquer
approach.

> Erlang uses dynamic typing (or rather strict typing) and scales very well.

Perhaps that is because it is purely functional?

> > The big problem using Ocaml (bytecode) for scripting
> > is probably the ugly dynamic loading support, and 
> > for long running systems, the lack of unloading support.
> > (plus the syntax which isn't really well suited to small
> > scale scripting applications).
> 
> Why is dynamic loading ugly? 

Sorry, I wasn't clear. I meant to refer to dynamic loading in Ocaml,
not in general. Exactly why that is I don't know. Perhaps the
interaction of the static typing and dynamic loading is not
easy because it is not well understood how to make it so,
in Ocaml, and perhaps in general (without losing type safety)


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Scripting in ocaml

[Serge Aleynikov]

skaller wrote:
> On Thu, 2006-12-21 at 08:30 -0500, Serge Aleynikov wrote:
>> skaller wrote:
> 
>>> As one of the authors of several major pieces of Python,
>>> my comment is that whilst it provides great convenience
>>> and good code structure .. dynamic typing just doesn't
>>> scale.
>> Could you please illustrate your point by more concrete reasoning. 
> 
> It isn't a matter of reasoning but experience. Many people believe
> static typing is good because it improves the likelihood a program
> is correct, perhaps by making it easier for both the human and
> machine to reason about the code -- and in particular catch
> errors which would otherwise require testing, and even then
> the actual location of a bug might be hard to determine.

OCaml has static typing but doesn't provide an actual location of code 
in exceptions.  Is it because it was hard or intentional?

> I guess the arguments for static typing and scalability are
> well known. So the reasoning part is clear: we infer larger 
> programs without dynamic typing are harder to get right.

I agree with your statements applicable to imperative/OO dynamicly typed 
languages.  However if a dynamic language is functional, this makes it 
possible to provide run-time static analysis to identify type related 
bugs and unreachable code.  An example of this is the Dyalizer tool for 
Erlang.

In this case running such a tool after compilation reveals the same 
problems that a compile-level static checker would do, but allows for 
run-time code reloading, more safe term serialization/deserialization, 
and safer cross-language integration.  Needless to say, this benefits 
comes at price of efficiency, but there are many cases when these 
features out-weight a reasonable performance loss that can be regained 
by parallelizing computations.

> Sorry, I wasn't clear. I meant to refer to dynamic loading in Ocaml,
> not in general. Exactly why that is I don't know. Perhaps the
> interaction of the static typing and dynamic loading is not
> easy because it is not well understood how to make it so,
> in Ocaml, and perhaps in general (without losing type safety)

I believe that introducing strict typing in the language might help in 
bridging static typing and dynamic loading.  Strict typing would allow 
to do run-time type verification and type-specific guard checks. 
Though, this would no longer be Ocaml as we know it today.  ;-)

Serge

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 09:59:15AM -0500, Serge Aleynikov wrote:
> 
> I believe that introducing strict typing in the language might help in 
> bridging static typing and dynamic loading.  Strict typing would allow 
> to do run-time type verification and type-specific guard checks. 
> Though, this would no longer be Ocaml as we know it today.  ;-)

I think you lost me.  What do you mean by "strict typing" such that
OCaml doesn't do it?  From what I've seen, OCaml is both statically and
strongly typed.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"A script is what you give the actors.  A program
is what you give the audience." - Larry Wall

Re: [Caml-list] Scripting in ocaml

[Daniel Bünzli]

Le 21 déc. 06 à 21:25, Chad Perrin a écrit :

> I think you lost me.  What do you mean by "strict typing" such that
> OCaml doesn't do it?  From what I've seen, OCaml is both statically  
> and
> strongly typed.

What do you mean by strong typing ?

Daniel

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 09:41:01PM +0100, Daniel Bünzli wrote:
> Le 21 déc. 06 à 21:25, Chad Perrin a écrit :
> 
> >I think you lost me.  What do you mean by "strict typing" such that
> >OCaml doesn't do it?  From what I've seen, OCaml is both statically  
> >and
> >strongly typed.
> 
> What do you mean by strong typing ?

I mean that it doesn't allow you to go around doing in-place type
changes willy-nilly the way something like C does.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"It's just incredible that a trillion-synapse computer could actually
spend Saturday afternoon watching a football game." - Marvin Minsky

strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Daniel Bünzli]

Le 21 déc. 06 à 23:16, Chad Perrin a écrit :

> I mean that it doesn't allow you to go around doing in-place type
> changes willy-nilly the way something like C does.

(Well in fact you can with Obj.magic, but that's not my point)

The problem is that this weak/strong terminology is hopelessly  
confused (see [1],[2]). Since there is no clear unique definition of  
strong/weak typing I think this terminology should be avoided. I tend  
to favour the definitions you can find in the introduction of this  
book [3] which are imho less confusing.

Basically the author distinguishes on one hand between statically and  
dynamically typed languages, and on the other hand, between safe and  
unsafe languages.

Static and dynamic type checking refers to whether type checks are  
respectively performed at compilation or run time.

Safety is broadly defined as follows :

"A safe language is one that protects its own abstractions. Every  
high-level language provides abstractions of machine services. Safety  
refers to the language's ability to guarantee the integrity of these  
abstractions and of higher-level abstractions introduced by the  
programmer using the definitional facilities of the language"

Later he gives the following chart

        |Statically checked       | Dynamically checked
-------------------------------------------------
safe   | ML, Haskell, Java, etc. | Lisp, Scheme, Perl, Postscript, etc
unsafe | C, C++, etc.            |

Subsequently he adds :

"Language safety is seldom absolute. Safe languages often offer  
programmers "escape hatches", such as foreign function calls to code  
written in other, possibly unsafe, languages. Indeed such escape  
hataches are sometimes provided in a controlled from within the  
language itself--Obj.magic in Ocaml, ... "

These are just definitions. But it is hard to argue when words do not  
have a common meaning between arguers. I just think these definitions  
make it simpler to have a common understanding of what we are talking  
about.

Best,

Daniel

[1] <http://en.wikipedia.org/wiki/Strong_typing>
[2] <http://www.artima.com/intv/strongweak.html>
[3] <http://www.cis.upenn.edu/~bcpierce/tapl/>

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Tom]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

>
>
> Later he gives the following chart
>
>         |Statically checked       | Dynamically checked
> -------------------------------------------------
> safe   | ML, Haskell, Java, etc. | Lisp, Scheme, Perl, Postscript, etc
> unsafe | C, C++, etc.            |
>
>
But this chart is not expressive enough... I believe that the properties
implied by "weak/strong" refer to the ability (or the disability) of the
compiler/runtime (or rather semantics of the language) to change types at
will (actually, whenever this seems useful, in cases such as "string" + 7 or
"9" - "3").

This category would include C and C++ (implicit conversions of numbers) and
certainly dynamically checked languages such as php, javascript, (probably
also) Ruby, Python, ...

I believe that these languages need to be distinguished.

- Tom

[-- Attachment #2: Type: text/html, Size: 1188 bytes --]

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Daniel Bünzli]

Le 22 déc. 06 à 17:51, Tom a écrit :

> But this chart is not expressive enough...

Its aim is not to describe anything you can say about programming  
languages. It describes the particular notion that Chad was implying  
by "strong" typing.

> I believe that the properties implied by "weak/strong" refer to the  
> ability (or the disability) of the compiler/runtime (or rather  
> semantics of the language) to change types at will (actually,  
> whenever this seems useful, in cases such as "string" + 7 or "9" -  
> "3").

That's _one_ of the bullet points listed in [1] as a _possible_  
meaning for weak/strong typing. Note that your reaction makes my  
point : do not use the notion of weak/strong typing it always mean  
something different to other persons and hence it doesn't mean anything.

A language that has the property you describe can simply be said to  
have implicit type conversions.

Best,

Daniel

[1] <http://en.wikipedia.org/wiki/Strong_typing>

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[skaller]

On Fri, 2006-12-22 at 17:51 +0100, Tom wrote:
>         
>         Later he gives the following chart
>         
>                 |Statically checked       | Dynamically checked 
>         -------------------------------------------------
>         safe   | ML, Haskell, Java, etc. | Lisp, Scheme, Perl,
>         Postscript, etc
>         unsafe | C, C++, etc.            |
>         
> 
> But this chart is not expressive enough...

It is also unclear what you mean by 'unsafe'.

Ocaml is not safe:

	let a = Array.create 0 0 in
	let y = a.[99] in (* WOOPS *)

The fact that an exception is thrown may or may not
make the language safe depending on whether or not
you INTEND to trigger an exception. The best you can
say is that if you don't catch it, its a bug.

Otherwise, you have to read the comments to know if
the out of bounds access was deliberate .. something
compilers can't do very easily. 

In fact C arrays are safer precisely because the behaviour
is not defined. That way you KNOW an out of bound access is
an error, so reasoning about the code .. and optimising it ..
is easier.

Furthermore you can always enable array bounds checking
with an instrumentation switch or tool like Purify.

Well array bounds probably aren't contentious: most people
would say an access violation was a bug, even in Ocaml.
But you can't be so sure with

	try Some (Hashtbl.find table key) 
	with Not_found -> None

That doesn't look like a bug, if the key isn't found.

There's a difference in intent .. but the language
fails to express it. Exceptions and specified dynamic
checks are 'evil' :)

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Daniel Bünzli]

Le 22 déc. 06 à 19:16, skaller a écrit :

> It is also unclear what you mean by 'unsafe'.
>
> Ocaml is not safe:
>
> 	let a = Array.create 0 0 in
> 	let y = a.[99] in (* WOOPS *)
>
> The fact that an exception is thrown may or may not
> make the language safe depending on whether or not
> you INTEND to trigger an exception. The best you can
> say is that if you don't catch it, its a bug.

I'm sorry I don't follow you. \x13This would be an example of safety \x13 
according to the definition I gave. Ocaml protects its array  
abstraction -- which allows to look at a bounded piece of memory as a  
finite array of values of a certain type --\x13 by raising \x13an exception  
if you try to access out of the bounds of the memory instead of  
silently \x13running the program with the arbitrary data it got by  
reading out of bounds.

Daniel

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[skaller]

On Fri, 2006-12-22 at 19:47 +0100, Daniel Bünzli wrote:
> Le 22 déc. 06 à 19:16, skaller a écrit :
> 
> > It is also unclear what you mean by 'unsafe'.
> >
> > Ocaml is not safe:
> >
> > 	let a = Array.create 0 0 in
> > 	let y = a.[99] in (* WOOPS *)
> >
> > The fact that an exception is thrown may or may not
> > make the language safe depending on whether or not
> > you INTEND to trigger an exception. The best you can
> > say is that if you don't catch it, its a bug.
> 
> I'm sorry I don't follow you. \x13This would be an example of safety \x13 
> according to the definition I gave. Ocaml protects its array  
> abstraction -- which allows to look at a bounded piece of memory as a  
> finite array of values of a certain type --\x13 by raising \x13an exception  
> if you try to access out of the bounds of the memory instead of  
> silently \x13running the program with the arbitrary data it got by  
> reading out of bounds.

This is not my understanding of what safe means.

Your program is safe? Ok, so would you use it to
control a nuclear reactor? Do you really think anyone
cares if the reactor blows, whether the program
core dumped, failed to core dump, or threw an exception?

to me safe means 'cannot fail'. But perhaps i misunderstand:
it would be interesting to see another definition.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[David Brown]

skaller wrote:

> This is not my understanding of what safe means.

> Your program is safe? Ok, so would you use it to
> control a nuclear reactor? Do you really think anyone
> cares if the reactor blows, whether the program
> core dumped, failed to core dump, or threw an exception?
>
> to me safe means 'cannot fail'. But perhaps i misunderstand:
> it would be interesting to see another definition.

This is certainly a good definition of "safety", but not one commonly
used when referring to type systems.  It also has problems with
undecidability, and trying to enforce it generally results in
programming languages or language subsets that are so restrictive that
general purpose programming becomes very tedius, or just impossible.
Even systems that put a lot of effort into this kind of safety (such
as SPARKAda) don't claim that they "cannot fail", but instead refer to
it as high-integrity.

Most references use phrases like "type safety", although this seems to
get different definitions depending on the user.  Although specific
instances in a program, the compiler might be able to statically
determine if an array bounds is going to be violated, this can't be
done in the general case.

So, I'm not sure how defining "safe" so high that it can't be
implemented is useful to anyone.  Those interested in this topic
should definitely read up on the Ariane 5 rocket's first launch.  For
efficiency considerations the developers disabled some of the runtime
constraint checks on data conversions, which resulted in this failure.

Dave

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Fri, Dec 22, 2006 at 12:03:09PM -0800, David Brown wrote:
> skaller wrote:
> 
> > This is not my understanding of what safe means.
> 
> > Your program is safe? Ok, so would you use it to
> > control a nuclear reactor? Do you really think anyone
> > cares if the reactor blows, whether the program
> > core dumped, failed to core dump, or threw an exception?
> >
> > to me safe means 'cannot fail'. But perhaps i misunderstand:
> > it would be interesting to see another definition.
> 
> This is certainly a good definition of "safety", but not one commonly
> used when referring to type systems.  It also has problems with
> undecidability, and trying to enforce it generally results in
> programming languages or language subsets that are so restrictive that
> general purpose programming becomes very tedius, or just impossible.
> Even systems that put a lot of effort into this kind of safety (such
> as SPARKAda) don't claim that they "cannot fail", but instead refer to
> it as high-integrity.
> 
> Most references use phrases like "type safety", although this seems to
> get different definitions depending on the user.  Although specific
> instances in a program, the compiler might be able to statically
> determine if an array bounds is going to be violated, this can't be
> done in the general case.

This all seems rather simple to me:

It's type-safe if you cannot get type errors without subverting the type
system.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
This sig for rent:  a Signify v1.14 production from http://www.debian.org/

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[skaller]

On Fri, 2006-12-22 at 13:17 -0700, Chad Perrin wrote:

> This all seems rather simple to me:
> 
> It's type-safe if you cannot get type errors without subverting the type
> system.

You are missing something. There's no such thing as a 'type error'
at run time in a statically typed language. At run time object values
are just bit strings, they don't HAVE any types -- and even that
is an abstraction (really you just have registers and machine
code .. not even really bit strings).

Python has a static type system. Did you know? Everything has
type 'object'.

So in fact Python is quite type safe, and statically so.
Big deal. The type system isn't expressive enough for that
to mean anything!

Similarly, Ocaml is NOT type safe because array bounds can
be violated at run time. This is because the type system
is not expressive enough to capture the array bound as
part of the type.

So saying a language is type-safe is meaningless unless
you also say how expressive the type system is: either
way the program isn't safe, and the choice of typing is
often dictated by what can be statically checked, rather
than what NEEDS to be checked. So 'type safe' is a fairly
vague concept in this sense.

It really comes down to "I think Ocaml is a better programming
language for my purpose than C++". 

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Sat, Dec 23, 2006 at 02:48:23PM +1100, skaller wrote:
> On Fri, 2006-12-22 at 13:17 -0700, Chad Perrin wrote:
> 
> > This all seems rather simple to me:
> > 
> > It's type-safe if you cannot get type errors without subverting the type
> > system.
> 
> You are missing something. There's no such thing as a 'type error'
> at run time in a statically typed language. At run time object values
> are just bit strings, they don't HAVE any types -- and even that
> is an abstraction (really you just have registers and machine
> code .. not even really bit strings).

That's not quite true.  A "type error" is what happens when an operation
is performed on a value of a type different from the operation's type.
For instance, integer addition on a float or string, or concatenation on
an integer or float, is a type error in most languages.  Statically
typed languages are commonly thought to avoid such type errors at
runtime almost tautologically, but play with casts and pointers in some
dangerous ways in C/C++ and you may quickly learn that is not the case.


> 
> Python has a static type system. Did you know? Everything has
> type 'object'.

That's a pretty weird interpretation of the Python type system -- and
it's categorically false, because there are many things in Python that
are not objects.  You might be able to make such a claim for Ruby, even
though I would consider even that to be a dubious claim due to the other
type-related behavior of the language.  In fact, objects have types.
That's sorta the point of objects, from a certain perspective: defining
types to extend your type system in a manner that makes it possible to
achieve certain benefits in coding such as reusability and modularity.


> 
> So in fact Python is quite type safe, and statically so.
> Big deal. The type system isn't expressive enough for that
> to mean anything!
> 
> Similarly, Ocaml is NOT type safe because array bounds can
> be violated at run time. This is because the type system
> is not expressive enough to capture the array bound as
> part of the type.

I don't know OCaml quite well enough to say this with authority, but at
a guess I'd say that's not a type error.  It's an error, but not a type
error, because it does not involve applying an operation to a value of a
given type (or similarly mismatch types).  It's more akin to a buffer
overrun than a type error.

Also, this directly contradicts your own earlier definition of "type
safety", because OCaml is pretty clearly a statically typed language.


> 
> So saying a language is type-safe is meaningless unless
> you also say how expressive the type system is: either
> way the program isn't safe, and the choice of typing is
> often dictated by what can be statically checked, rather
> than what NEEDS to be checked. So 'type safe' is a fairly
> vague concept in this sense.
> 
> It really comes down to "I think Ocaml is a better programming
> language for my purpose than C++".

So much for language theory.  I guess it's all balderdash, then.

By the way, I wrote a longish email (I didn't think it was *THAT* long,
but apparently it was) earlier today that never made it through to the
list.  I can only guess that it must be in moderation because of its
length.  It gets into some depth on the subject of defining type system
characteristics, and might be worth a read if it ever appears.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"A script is what you give the actors.  A program
is what you give the audience." - Larry Wall

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Sat, Dec 23, 2006 at 06:42:02AM +1100, skaller wrote:
> 
> Your program is safe? Ok, so would you use it to
> control a nuclear reactor? Do you really think anyone
> cares if the reactor blows, whether the program
> core dumped, failed to core dump, or threw an exception?

That's health-safety, or life-safety, or something like that.  It's not
the same as type-safety.


> 
> to me safe means 'cannot fail'. But perhaps i misunderstand:
> it would be interesting to see another definition.

You're right, in a sense: safety means "cannot fail".  In the case of
type-safety, however, all that means is that the type system "cannot
fail".  The caveat is that, of course, if you evade the type system in
some way, its type-safety becomes to some extent immaterial.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
print substr("Just another Perl hacker", 0, -2);

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Daniel Bünzli]

Le 22 déc. 06 à 20:42, skaller a écrit :

> This is not my understanding of what safe means.

I gave a definition and showed how the definition applies to ocaml,  
if you don't want to use that term for that then call it 'zorglub',  
that suits me, it is just a property. So ocaml is a zorglub  
programming language.


Le 22 déc. 06 à 21:14, Chad Perrin a écrit :

> In point of fact, there is a clear definition.

And that is, unsurprisingly, your definition. Can you show me an  
authoritative source to support your claims ?

Anyway, this terminology is as spoiled as this discussion so it is  
worth not using it.

Daniel

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Sat, Dec 23, 2006 at 01:58:12PM +0100, Daniel Bünzli wrote:
> Le 22 déc. 06 à 21:14, Chad Perrin a écrit :
> 
> >In point of fact, there is a clear definition.
> 
> And that is, unsurprisingly, your definition. Can you show me an  
> authoritative source to support your claims ?
> 
> Anyway, this terminology is as spoiled as this discussion so it is  
> worth not using it.

Yeah, terminology sucks.  Let's abandon all terminology.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"The first rule of magic is simple. Don't waste your time waving your
hands and hopping when a rock or a club will do." - McCloctnick the Lucid

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Tom]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

>
>
> > And that is, unsurprisingly, your definition. Can you show me an
> > authoritative source to support your claims ?
> >
> > Anyway, this terminology is as spoiled as this discussion so it is
> > worth not using it.
>
> Yeah, terminology sucks.  Let's abandon all terminology.


Why so? Why not rather define it, so that it is definite and absolute once
and for all. Then we become that authoritative source... :P

- Tom

[-- Attachment #2: Type: text/html, Size: 652 bytes --]

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Sat, Dec 23, 2006 at 10:50:35PM +0100, Tom wrote:
> >
> >
> >> And that is, unsurprisingly, your definition. Can you show me an
> >> authoritative source to support your claims ?
> >>
> >> Anyway, this terminology is as spoiled as this discussion so it is
> >> worth not using it.
> >
> >Yeah, terminology sucks.  Let's abandon all terminology.
> 
> 
> Why so? Why not rather define it, so that it is definite and absolute once
> and for all. Then we become that authoritative source... :P

My statement was meant to be sarcastic.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"There comes a time in the history of any project when it becomes necessary
to shoot the engineers and begin production." - MacUser, November 1990

Re: strong/weak typing terminology (was Re: [Caml-list] Scripting in ocaml)

[Chad Perrin]

On Fri, Dec 22, 2006 at 01:21:07PM +0100, Daniel Bünzli wrote:
> Le 21 déc. 06 à 23:16, Chad Perrin a écrit :
> 
> >I mean that it doesn't allow you to go around doing in-place type
> >changes willy-nilly the way something like C does.
> 
> (Well in fact you can with Obj.magic, but that's not my point)
> 
> The problem is that this weak/strong terminology is hopelessly  
> confused (see [1],[2]). Since there is no clear unique definition of  
> strong/weak typing I think this terminology should be avoided. I tend  
> to favour the definitions you can find in the introduction of this  
> book [3] which are imho less confusing.

In point of fact, there is a clear definition.  The problem is that
people who don't pay much attention to it end up using the terms
"strongly typed" and "weakly typed" in a (ahem) weakly typed manner.
Thus, it is the colloquial definition that is almost meaningless due to
widespread conflict in its use, while there is a formal definition that
endures somewhere behind all the noise.

> 
> Basically the author distinguishes on one hand between statically and  
> dynamically typed languages, and on the other hand, between safe and  
> unsafe languages.
> 
> Static and dynamic type checking refers to whether type checks are  
> respectively performed at compilation or run time.

That's absolutely correct, and fits the formal definitions.


> 
> Safety is broadly defined as follows :
> 
> "A safe language is one that protects its own abstractions. Every  
> high-level language provides abstractions of machine services. Safety  
> refers to the language's ability to guarantee the integrity of these  
> abstractions and of higher-level abstractions introduced by the  
> programmer using the definitional facilities of the language"

That's an almost useless representation of the definition,
unfortunately, because the terms used in it are vague in application and
the structure of the definition as presented is a bit too complex.
While he may well know what he's talking about, his prose reads a bit
like poorly written code -- which is to say that it's almost unreadable
in any meaningful sense.


> 
> Later he gives the following chart
> 
>        |Statically checked       | Dynamically checked
> -------------------------------------------------
> safe   | ML, Haskell, Java, etc. | Lisp, Scheme, Perl, Postscript, etc
> unsafe | C, C++, etc.            |
> 
> Subsequently he adds :
> 
> "Language safety is seldom absolute. Safe languages often offer  
> programmers "escape hatches", such as foreign function calls to code  
> written in other, possibly unsafe, languages. Indeed such escape  
> hataches are sometimes provided in a controlled from within the  
> language itself--Obj.magic in Ocaml, ... "

He comes dangerously close to touching on strong/weak typing in that
statement.

Here's a summary of type system characteristics as I learned about them,
and confirmed by experience, research, reasoning over the years:

type-safety: A type-safe language is one whose core type system does not
allow for type errors at runtime.  Such a language is almost impossible
while still being a useful general-purpose language, but many languages
come close enough to warrant the term "type-safe" for common usage.  By
contrast, a type-unsafe language in its purest meaning is one that does
nothing to prevent type errors at all.

statically typed: A statically typed language is one that determines
types at compile time.  By contrast, a dynamically typed language is one
that determines types at runtime, generally based on context.

strongly typed: A strongly typed language is one that does not allow the
programmer to simply dodge the type system and where any instantiated or
applied datum has a given type that cannot be changed, explicitly or
implicitly.  By contrast, a weakly typed language provides casts, bit
manipulation, and other in-place conversions or evasions to subvert
types.

duck typing: This is a more recent term than others related to typing,
and arose with the Ruby community.  It essentially refers to a language
that allows for a type to be unset until the datum to be typed has a
typable value, at which point the type is "obvious" and set in stone.
The most common implementation of this uses dynamic typing, but there
are static type systems that could be called "duck typed", such as OCaml
and Objective C -- though in OCaml's case, it is achieved through type
inference, and in Objective C it is limited to object types and is
determined by behavior rather than type inference.

That's how I've come to understand type system characteristics, and it's
an internally consistent set of definitions that provides meaningful
descriptive terms and agrees with the statements of all the experts who
actually seem to care about the respective terms' meanings.  Perhaps
more to the point, it fits the computer science framework of language
theory that informs the axioms on which the best programming theory
books available (at least to judge by those I've seen).

That makes OCaml type-safe, statically typed, strongly typed, and
arguably duck typed (depending on how you use it).  It also makes Ruby
duck typed, type-safe, dynamically typed, and strongly typed; it makes
Perl type-safe, dynamically typed, and strongly typed; it makes C
statically typed, weakly typed, and kind of half-assed type-safe if
you're careful; it makes C++ less type-safe, weakly typed (though
slightly less so than C), and statically typed, as well as allowing you
to fake duck typing via templates and the like; it makes Objective C
statically typed, exactly as half-assed type-safe as C, half-assed
strongly typed (more so than C or C++), and duck typed insofar as its
object system is concerned; it makes PHP weakly typed, dynamically
typed, and type-unsafe; it makes Java fairly type-safe, weakly typed,
and statically typed; and it makes Pascal statically typed, weakly
typed, and mostly type-safe.

At least, that's what comes to mind off the top of my head.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"Real ugliness is not harsh-looking syntax, but having to
build programs out of the wrong concepts." - Paul Graham

Re: [Caml-list] Scripting in ocaml

[Serge Aleynikov]

Chad Perrin wrote:
 > On Thu, Dec 21, 2006 at 09:59:15AM -0500, Serge Aleynikov wrote:
 >> I believe that introducing strict typing in the language might help in
 >> bridging static typing and dynamic loading.  Strict typing would allow
 >> to do run-time type verification and type-specific guard checks.
 >> Though, this would no longer be Ocaml as we know it today.  ;-)
 >
 > I think you lost me.  What do you mean by "strict typing" such that
 > OCaml doesn't do it?  From what I've seen, OCaml is both statically and
 > strongly typed.
 >

What I meant by "strict typing" was performing type checks *at runtime*, 
i.e. that there are no unsafe operations.

Say, as an example, doing type checks of function arguments at runtime:

let f = fun i when is_integer (i) -> i
           | x when is_float   (x) -> int_of_float x;;

(raising Bad_match of some sort if neither one of the two guards pass)

Direct application of this could be for making the Marshal/Unmarshal 
modules more safe when marshaling data over files/sockets between 
applications written in heterogeneous languages.

Serge


-- 
Serge Aleynikov
Routing R&D, IDT Telecom
Tel: +1 (973) 438-3436
Fax: +1 (973) 438-1464

Re: [Caml-list] Scripting in ocaml

[Philippe Wang]

Serge Aleynikov a écrit :

> What I meant by "strict typing" was performing type checks *at runtime*, 
> i.e. that there are no unsafe operations.
> 
> Say, as an example, doing type checks of function arguments at runtime:
> 
> let f = fun i when is_integer (i) -> i
>           | x when is_float   (x) -> int_of_float x;;
> 
> (raising Bad_match of some sort if neither one of the two guards pass)
> 
> Direct application of this could be for making the Marshal/Unmarshal 
> modules more safe when marshaling data over files/sockets between 
> applications written in heterogeneous languages.

If you want to do that, use Lisp...
Or use sum types...
type t = Int of int | Float of float | ...

OCaml forgets types at runtime !
This means that you can't know without a huge cost (cf. SafeUnmarshal 
costs), because what you can do in O(1) is to know whether a value is an 
int or a pointer...

If OCaml is particularly efficient, it's probably mostly because of that!
...

Cheers,

--
Philippe Wang
   mail(at)philippewang.info

Re: [Caml-list] Scripting in ocaml

[Serge Aleynikov]

Philippe Wang wrote:
>> What I meant by "strict typing" was performing type checks *at 
>> runtime*, i.e. that there are no unsafe operations.
>>
>> Say, as an example, doing type checks of function arguments at runtime:
>>
>> let f = fun i when is_integer (i) -> i
>>           | x when is_float   (x) -> int_of_float x;;
>>
>> (raising Bad_match of some sort if neither one of the two guards pass)
>>
>> Direct application of this could be for making the Marshal/Unmarshal 
>> modules more safe when marshaling data over files/sockets between 
>> applications written in heterogeneous languages.
> 
> If you want to do that, use Lisp...

Indeed. I am also quite happy with having that feature in Erlang.  ;-)

> Or use sum types...
> type t = Int of int | Float of float | ...
> 
> OCaml forgets types at runtime !
> This means that you can't know without a huge cost (cf. SafeUnmarshal 
> costs), because what you can do in O(1) is to know whether a value is an 
> int or a pointer...

There doesn't seem to be a large overhead for knowing if a value is a 
closure, string, float or float array either (they have dedicated tag 
values accessed with a single dereferencing).  With having the compiler 
reserve some tag values for other basic types that could optionally be 
inspected at run-time, it perhaps wouldn't penalize efficiency very badly.

> If OCaml is particularly efficient, it's probably mostly because of that!
> ...

This thread began with John's statement that "interaction of the static 
typing and dynamic loading was not easy".  Any dynamic type check by all 
means would reduce efficiency of a statically typed language.  But maybe 
it's possible to do dynamic type checking optional and make it available 
if explicitly requested.  At least if would make dynamic code loading 
less complicated.

BR,
Serge


-- 
Serge Aleynikov
Routing R&D, IDT Telecom
Tel: +1 (973) 438-3436
Fax: +1 (973) 438-1464

Re: [Caml-list] Scripting in ocaml

[Jon Harrop]

On Thursday 21 December 2006 21:27, Philippe Wang wrote:
> If you want to do that, use Lisp...

Lisp is too slow.

> Or use sum types...
> type t = Int of int | Float of float | ...

Manual boxing is too verbose.

> OCaml forgets types at runtime!

Some type related information is certainly retained, e.g. to unbox float 
arrays.

> This means that you can't know without a huge cost (cf. SafeUnmarshal
> costs), because what you can do in O(1) is to know whether a value is an
> int or a pointer...

I'd like to quantify this cost. I've read papers and heard work stating that 
carrying run-time type information can be cheap but I see evidence that might 
point to the contrary, e.g. F# is significantly slower than OCaml but it has 
concurrent GC that was designed for a non-FPL.

In F#, you have run-time type information. Amongst other things, this allows 
you to dispatch to more efficient type-specialised functions. For example, 
you can write functions over polymorphic arrays and dispatch to optimised 
BLAS versions for float arrays when the input happens to be a float array.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 04:11:00PM -0500, Serge Aleynikov wrote:
> Chad Perrin wrote:
> > On Thu, Dec 21, 2006 at 09:59:15AM -0500, Serge Aleynikov wrote:
> >> I believe that introducing strict typing in the language might help in
> >> bridging static typing and dynamic loading.  Strict typing would allow
> >> to do run-time type verification and type-specific guard checks.
> >> Though, this would no longer be Ocaml as we know it today.  ;-)
> >
> > I think you lost me.  What do you mean by "strict typing" such that
> > OCaml doesn't do it?  From what I've seen, OCaml is both statically and
> > strongly typed.
> >
> 
> What I meant by "strict typing" was performing type checks *at runtime*, 
> i.e. that there are no unsafe operations.

So, basically, by "strict typing" you mean something like both
compile-time and runtime type checking?  If it was *only* runtime type
checking, you'd just be using a dynamic type system (the opposite of
static typing, basically).

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"The first rule of magic is simple. Don't waste your time waving your
hands and hopping when a rock or a club will do." - McCloctnick the Lucid

Re: [Caml-list] Scripting in ocaml

[Jon Harrop]

On Thursday 21 December 2006 22:19, Chad Perrin wrote:
> So, basically, by "strict typing" you mean something like both
> compile-time and runtime type checking?  If it was *only* runtime type
> checking, you'd just be using a dynamic type system (the opposite of
> static typing, basically).

As many Lisp compilers try to do type inference and check types at run-time 
(giving warnings), Lisp is not dynamically typed according to your argument. 
Moreover, this is a property of the implementation and not of the language.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Fri, Dec 22, 2006 at 12:37:32PM +0000, Jon Harrop wrote:
> On Thursday 21 December 2006 22:19, Chad Perrin wrote:
> > So, basically, by "strict typing" you mean something like both
> > compile-time and runtime type checking?  If it was *only* runtime type
> > checking, you'd just be using a dynamic type system (the opposite of
> > static typing, basically).
> 
> As many Lisp compilers try to do type inference and check types at run-time 
> (giving warnings), Lisp is not dynamically typed according to your argument. 
> Moreover, this is a property of the implementation and not of the language.

By that argument, you might as well say that all type system
characteristics are properties of the implementation, and not the
language.  Ultimately, anything you do in designing a language must be
reflected in the implementation.

That doesn't mean it's "just an implementation detail", though.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
"The first rule of magic is simple. Don't waste your time waving your
hands and hopping when a rock or a club will do." - McCloctnick the Lucid

Re: [Caml-list] Scripting in ocaml

[skaller]

On Thu, 2006-12-21 at 16:11 -0500, Serge Aleynikov wrote:
> Chad Perrin wrote:

> What I meant by "strict typing" was performing type checks *at runtime*, 
> i.e. that there are no unsafe operations.
> 
> Say, as an example, doing type checks of function arguments at runtime:
> 
> let f = fun i when is_integer (i) -> i
>            | x when is_float   (x) -> int_of_float x;;
> 
> (raising Bad_match of some sort if neither one of the two guards pass)
> 

Just BTW .. it is very bad to raise exceptions on type errors.
The program should be aborted.

The reason is that raising such exceptions also allows for
catching them, which means doing a type error is no longer
unsafe and no longer a bug, but a legitimate technique.
This in turn defeats most static type analysis you might do.

For example this destroys the ability to analyse Python
statically for the purpose of optimising it.

It is *essential* that the language description not
mandate raising exceptions on type errors, but rather
specify the action is undefined .. even if the implementation
raises an exception, the language specification must NOT
require that. This prevents programmers actually relying on it
and allows a static analyser to optimise the code on the
assumption it is well typed.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Scripting in ocaml

[Jon Harrop]

On Friday 22 December 2006 02:51, skaller wrote:
> Just BTW .. it is very bad to raise exceptions on type errors.

For what definition of "type"?

> The reason is that raising such exceptions also allows for
> catching them, which means doing a type error is no longer
> unsafe and no longer a bug, but a legitimate technique.
> This in turn defeats most static type analysis you might do.

Absolutely. But the ability to do run-time dispatch based upon type is an 
advantage of dynamic typing, so it is something that you do not want to lose.

> For example this destroys the ability to analyse Python
> statically for the purpose of optimising it.

Yes. An optimising Python compiler will only be adopted/useful if it can 
evaluate any Python. Note that this could mean reverting to interpreted 
bytecode when the program is inherently dynamically typed.

> It is *essential* that the language description not
> mandate raising exceptions on type errors, but rather
> specify the action is undefined .. even if the implementation
> raises an exception, the language specification must NOT
> require that. This prevents programmers actually relying on it
> and allows a static analyser to optimise the code on the
> assumption it is well typed.

You can raise exceptions from unexpectedly typed code whilst keeping the 
advantages of static checking and performance in F#, for example. This gives 
you the advantages of both worlds: performance/reliability when leveraging 
static typing and brevity/generality when leveraging dynamic typing.

For example, I recently benchmarked C++, F#, OCaml and Python for computing 
discrete wavelet transforms. F# (on 32-bit WXP) was slightly faster than 
OCaml (on 64-bit Debian), so it can have very competitive performance:

http://groups.google.co.uk/group/comp.lang.python/msg/0229d2c6484ea491?hl=en&
http://groups.google.co.uk/group/comp.lang.python/msg/daf7bbb2bd7e99f3?hl=en&

Yet F# retains run-time type information so you can use a generic print 
function (print_any) on any type, have your dynamic code loading and so on. 
The best of both worlds.

On a related note, F# supports operator overloading, which greatly simplifies 
many mathematical expressions at the cost of requiring more type annotations.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Scripting in ocaml

[Jon Harrop]

On Thursday 21 December 2006 21:11, Serge Aleynikov wrote:
> What I meant by "strict typing" was performing type checks *at runtime*,
> i.e. that there are no unsafe operations.

I'd call run-time type checking "dynamic typing" and type checking such that 
there are no unsafe operations "strong typing". So OCaml is strongly, 
statically typed.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Scripting in ocaml

[Jon Harrop]

On Thursday 21 December 2006 14:59, Serge Aleynikov wrote:
> OCaml has static typing but doesn't provide an actual location of code
> in exceptions.  Is it because it was hard or intentional?

F# uses .NET exceptions. They do provide such information but they are so slow 
that you must painstakingly avoid any use of exceptions in performance 
critical code, boxing and unrolling manually if necessary. This is in stark 
contrast to OCaml's lightning fast exceptions that are used as the basis of 
optimisations.

> > I guess the arguments for static typing and scalability are
> > well known. So the reasoning part is clear: we infer larger
> > programs without dynamic typing are harder to get right.
>
> I agree with your statements applicable to imperative/OO dynamicly typed
> languages.  However if a dynamic language is functional, this makes it
> possible to provide run-time static analysis to identify type related
> bugs and unreachable code.  An example of this is the Dyalizer tool for
> Erlang.

What do you mean by "run-time static analysis"? Surely if something is 
analysed at run-time then it cannot be static by definition?

> In this case running such a tool after compilation reveals the same
> problems that a compile-level static checker would do, but allows for
> run-time code reloading, more safe term serialization/deserialization,
> and safer cross-language integration.  Needless to say, this benefits
> comes at price of efficiency, but there are many cases when these
> features out-weight a reasonable performance loss that can be regained
> by parallelizing computations.

Then it might be worthwhile for F#, which supports concurrency.

> > Sorry, I wasn't clear. I meant to refer to dynamic loading in Ocaml,
> > not in general. Exactly why that is I don't know. Perhaps the
> > interaction of the static typing and dynamic loading is not
> > easy because it is not well understood how to make it so,
> > in Ocaml, and perhaps in general (without losing type safety)
>
> I believe that introducing strict typing in the language might help in
> bridging static typing and dynamic loading.  Strict typing would allow
> to do run-time type verification and type-specific guard checks.
> Though, this would no longer be Ocaml as we know it today.  ;-)

You could add run-time type information to OCaml and check it at boundaries, 
like marshalling. However, you'd want to optimise it away whenever possible, 
if not for performance reasons then to conserve memory.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Scripting in ocaml

[Serge Aleynikov]

Jon Harrop wrote:
> On Thursday 21 December 2006 14:59, Serge Aleynikov wrote:
>> OCaml has static typing but doesn't provide an actual location of code
>> in exceptions.  Is it because it was hard or intentional?
> 
> F# uses .NET exceptions. They do provide such information but they are so slow 
> that you must painstakingly avoid any use of exceptions in performance 
> critical code, boxing and unrolling manually if necessary. This is in stark 
> contrast to OCaml's lightning fast exceptions that are used as the basis of 
> optimisations.

I wonder if you or anyone on the list have done any benchmarking of 
F#/Mono vs. OCaml on Linux/Solaris.  If so, could you please provide 
some relevant links?

> What do you mean by "run-time static analysis"? Surely if something is 
> analysed at run-time then it cannot be static by definition?

Sorry, I meant post-compilation static analysis.  The kind of analysis
that allows to traverse bytecode abstract syntax tree and through type
inference make conclusions about type errors.  For example see the
"Identified discrepancies" section of the Dialyzer (the tool that does 
post-compilation analysis):
http://www.die.net/doc/linux/man/man3/dialyzer.3.html

Such a tool can give power to a dynamically typed language comparable to
a static type-checker embedded in the compiler, while still allowing for
having run-time safety features in presence of code reloading, 
cross-program message passing, marshaling, etc.

Serge

Re: [Caml-list] Scripting in ocaml

[skaller]

On Sat, 2006-12-23 at 18:50 +0000, Jon Harrop wrote:

> What do you mean by "run-time static analysis"? Surely if something is 
> analysed at run-time then it cannot be static by definition?

Run time control flow analysis analysing invariant code blocks,
so you could say it is 'run time static analysis' because
it is done at run time, but analysing stuff which is 'static' :)

A JIT interpreter might do such stuff.

I guess this what was intended?

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Scripting in ocaml

[Richard Jones]

On Wed, Dec 20, 2006 at 10:41:20PM -0500, Denis Bueno wrote:
> I've been writing bash scripts to perform various build- and
> development-related tasks, and I don't enjoy it. I won't bore you with
> detailed reasons why. The upshot is that I'd like to script in OCaml.
> 
> I have considered writing a few camlp4 extensions to make it easier to
> write scripts:
> 
>  1) create a syntax which grabs environment variables:
> 
>    e.g. $FOO would grab the value of the environment variable FOO
> 
>  2) some sort of more convenient process interaction, e.g., for piping.

I think it's a great idea - I'd love to push OCaml for scripting.
However I hope your camlp4-fu is up to snuff.  You'd want, as you say,
a syntax for pipelines and file redirection, but more importantly
you'd want a very simple syntax for running commands.  So you can
write some unholy OCaml/sh combination like:

  let nr_files = int_of_string ` ls | wc -l `

Actually, even Perl isn't a very usable alternative for shell
scripting because of the amount of code you have to write just to fork
off a command and capture the output.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Internet Marketing and AdWords courses - http://merjis.com/courses - NEW!
Merjis blog - http://blog.merjis.com - NEW!

Re: [Caml-list] Scripting in ocaml

[Chad Perrin]

On Thu, Dec 21, 2006 at 02:59:54PM +0000, Richard Jones wrote:
> 
> Actually, even Perl isn't a very usable alternative for shell
> scripting because of the amount of code you have to write just to fork
> off a command and capture the output.

That's not my experience.  Running an external command from Perl is
easy, and there are a number of ways to do it, to suit pretty much any
purpose.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
unix virus: If you're using a unixlike OS, please forward
this to 20 others and erase your system partition.

Re: [Caml-list] Scripting in ocaml

[Martin Jambon]

On Thu, 21 Dec 2006, Richard Jones wrote:

> On Wed, Dec 20, 2006 at 10:41:20PM -0500, Denis Bueno wrote:
> > I've been writing bash scripts to perform various build- and
> > development-related tasks, and I don't enjoy it. I won't bore you with
> > detailed reasons why. The upshot is that I'd like to script in OCaml.
> >
> > I have considered writing a few camlp4 extensions to make it easier to
> > write scripts:
> >
> >  1) create a syntax which grabs environment variables:
> >
> >    e.g. $FOO would grab the value of the environment variable FOO
> >
> >  2) some sort of more convenient process interaction, e.g., for piping.
>
> I think it's a great idea - I'd love to push OCaml for scripting.
> However I hope your camlp4-fu is up to snuff.  You'd want, as you say,
> a syntax for pipelines and file redirection, but more importantly
> you'd want a very simple syntax for running commands.  So you can
> write some unholy OCaml/sh combination like:
>
>   let nr_files = int_of_string ` ls | wc -l `

It's something that I'd love to have too. An implementation of a
simple subset of sh would be nice. The programming features would be
handled by ocaml, so we need a way to use ocaml variables as
shell variables (of type string, string list or command) in addition to
environment variables.

Should camlp4's quotations be used for this, or should the special syntax
be handled by another preprocessor? I don't know.
A quotation looks like << ls | wc -l >> or <:cmd< ls | wc -l >>.

If something like this already exists, please let us know.
If not, I'd be glad to help design the thing.



Martin

--
Martin Jambon, PhD
http://martin.jambon.free.fr

Re: [Caml-list] Scripting in ocaml

[Aleksey Nogin]

On 20.12.2006 19:41, Denis Bueno wrote:

> I've been writing bash scripts to perform various build- and
> development-related tasks, and I don't enjoy it. I won't bore you with
> detailed reasons why. The upshot is that I'd like to script in OCaml.
[...]
> I googled a bit but couldn't find anything related to this. Has
> anything done, or started doing, anything like this?

Not quite scripting in OCaml, but related - the OMake build system comes
with its own shell interpreter - osh. The language is not OCaml, but
it's a functional language that was _specifically_ designed as a
scripting language, so I would argue that writing scripts in osh is more
convenient that scripting in OCaml (although, of course, for somebody
already familiar with OCaml, learning osh might be a bit harder that
learning some hypothetical scripting extension of OCaml).

Note that if the goal is specifically "scripts to perform various build-
and development-related tasks" as you've mentioned, then I would
definitely suggest looking at OMake and osh - there the scripting
language is the same as the build specification language and you can
inline osh scriplets directly into "make-style" build rules of OMake.

See http://omake.metaprl.org/ for more information.

Aleksey

Re: [Caml-list] Scripting in ocaml

[Ian Zimmerman]

On 12/26/06, Aleksey Nogin <nogin@metaprl.org> wrote:

> Not quite scripting in OCaml, but related - the OMake build system comes
> with its own shell interpreter - osh. The language is not OCaml, but
> it's a functional language that was _specifically_ designed as a
> scripting language, so I would argue that writing scripts in osh is more
> convenient that scripting in OCaml (although, of course, for somebody
> already familiar with OCaml, learning osh might be a bit harder that
> learning some hypothetical scripting extension of OCaml).
>
> Note that if the goal is specifically "scripts to perform various build-
> and development-related tasks" as you've mentioned, then I would
> definitely suggest looking at OMake and osh - there the scripting
> language is the same as the build specification language and you can
> inline osh scriplets directly into "make-style" build rules of OMake.

Does it handle building in general, or just OCaml-based projects?  For example,
can it deduce header dependencies for a C file - possibly with a
plugin, like cons
or scons?  I couldn't use one OMake for building OCaml projects and
make for others.

Ian

Re: [Caml-list] Scripting in ocaml

[Aleksey Nogin]

On 26.12.2006 15:36, Ian Zimmerman wrote:

> On 12/26/06, Aleksey Nogin <nogin@metaprl.org> wrote:
> 
>> Not quite scripting in OCaml, but related - the OMake build system comes
>> with its own shell interpreter - osh. The language is not OCaml, but
>> it's a functional language that was _specifically_ designed as a
>> scripting language, so I would argue that writing scripts in osh is more
>> convenient that scripting in OCaml (although, of course, for somebody
>> already familiar with OCaml, learning osh might be a bit harder that
>> learning some hypothetical scripting extension of OCaml).
>>
>> Note that if the goal is specifically "scripts to perform various build-
>> and development-related tasks" as you've mentioned, then I would
>> definitely suggest looking at OMake and osh - there the scripting
>> language is the same as the build specification language and you can
>> inline osh scriplets directly into "make-style" build rules of OMake.
> 
> Does it handle building in general, or just OCaml-based projects?  For
> example, can it deduce header dependencies for a C file - possibly with a
> plugin, like cons or scons? 

Yes, definitely. OMake is meant to be language-agnostic. It comes with
standard libraries for OCaml (including support for preprocessors, for
ocamlfind and experimental support for the Menhir parser-generator), C,
C++ and LaTeX, including appropriate dependency scanning rules for there
languages and support for projects that have a mixture of these
languages. It should be fairly straightforward to add support for other
languages as well (for example, recently Dirk Heinrichs had written an
experimental Ada module).

To rehash from http://omake.metaprl.org/, OMake is designed for
scalability and portability and its features include:

- Support for projects spanning several directories or directory
hierarchies.

- Fast, reliable, automated, scriptable dependency analysis using MD5
digests, with full support for incremental builds.

- Fully scriptable, includes a library that providing support for
standard tasks in C, C++, OCaml, and LaTeX projects, or a mixture thereof.

For small projects, a configuration file may be as simple as a single line

   .DEFAULT: $(CProgram prog, foo bar baz)

which states that the program "prog" is built from the files foo.c,
bar.c, and baz.c. This one line will also invoke the default standard
library scripts for discovering implicit dependencies in C files (such
as dependencies on included header files).

- Full native support for rules that build several files at once.

- Portability: omake provides a uniform interface on Linux/Unix
(including 64-bit architectures), Win32, Cygwin, Mac OS X, and other
platforms that are supported by OCaml.

- Built-in functions that provide the most common features of programs
like grep, sed, and awk. These are especially useful on Win32.

- Active filesystem monitoring, where the build automatically restarts
whenever you modify a source file. This can be very useful during the
edit/compile cycle.

- A built-in command-interpreter osh that can be used interactively.

Aleksey

Re: [Caml-list] Scripting in ocaml

[Richard Jones]

This is the syntax, right?

http://omake.metaprl.org/omake-shell.html#chapter:shell

It looks remarkably shell-like.  For those (like me) too lazy to
download the source, can you give us an idea of how this is
implemented?  Is it an alternate syntax for OCaml or an interpreter
written using ocamllex, etc.?

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Internet Marketing and AdWords courses - http://merjis.com/courses - NEW!
Merjis blog - http://blog.merjis.com - NEW!

Re: [Caml-list] Scripting in ocaml

[Aleksey Nogin]

On 27.12.2006 10:39, Richard Jones wrote:

> This is the syntax, right?
> 
> http://omake.metaprl.org/omake-shell.html#chapter:shell
> 
> It looks remarkably shell-like.

The above link only covers the "proper shell" parts of the OMake/osh
language (i.e. parts related to calling external commands).

Some features of the general language are outlined in
http://omake.metaprl.org/omake-language.html#chapter:language

> For those (like me) too lazy to
> download the source, can you give us an idea of how this is
> implemented?  Is it an alternate syntax for OCaml or an interpreter
> written using ocamllex, etc.?
> 
The OMake/osh language is fairly different from OCaml. The language was
designed specifically to work nicely in build specifications and shell
scriplets,  which is a quite different set of constraints than the ones
OCaml is designed for. The result is a functional language with dynamic
typing and dynamic scoping.

OMake is implemented in OCaml (with a bit of C -
fam/gamin/kqueue/inotify bindings, Windows-specific code, etc). It can
be characterized as an interpreter (although before a file is executed,
there is a small compilation-like translation step) implemented using
ocamllex, ocamlyacc, etc. The parsed+translated version of each file is
cached to avoid the penalty of having to do it on every run.

Aleksey