caml-list - the Caml user's mailing list
 help / color / mirror / Atom feed
* Ask for a more efficient way to deallocate memory (full version)
@ 2007-12-09 21:39 Fabrice.Pardo
  2007-12-09 21:55 ` [Caml-list] " Olivier Andrieu
                   ` (3 more replies)
  0 siblings, 4 replies; 19+ messages in thread
From: Fabrice.Pardo @ 2007-12-09 21:39 UTC (permalink / raw)
  To: caml-list

Hello, and sorry for my previous unfinished message.

As the function Unix.opendir returns a value,
we can expect that Unix.closedir will be automatically
called when the dir_handle value is no more referenced.
That's not the case, and calling opendir too many times as in

for i = 1 to 2000 do
  let d = Unix.opendir "/tmp" in ()
done;;

  raises
Exception: Unix.Unix_error (Unix.EMFILE, "opendir", "/tmp").

Replacing Unix.opendir by safe_opendir:

let safe_opendir path =
  let _ = Gc.major ()
  and d = Unix.opendir path
  in Gc.finalise Unix.closedir d

seems a solution, but the efficiency is very low due to high
cost of garbage collector calling function.

My question is how to write a code as efficient
as it would be possible using a reference-counted language.

Thanks.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 21:39 Ask for a more efficient way to deallocate memory (full version) Fabrice.Pardo
@ 2007-12-09 21:55 ` Olivier Andrieu
  2007-12-10 11:25   ` Fabrice Pardo
  2007-12-09 21:57 ` Oliver Bandel
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 19+ messages in thread
From: Olivier Andrieu @ 2007-12-09 21:55 UTC (permalink / raw)
  To: Fabrice.Pardo; +Cc: caml-list

On Dec 9, 2007 10:39 PM,  <Fabrice.Pardo@lpn.cnrs.fr> wrote:
> Hello, and sorry for my previous unfinished message.
>
> As the function Unix.opendir returns a value,
> we can expect that Unix.closedir will be automatically
> called when the dir_handle value is no more referenced.

Well, not really: a GC manages memory, not OS resources like file descriptors.
I wouldn't recommend relying on the GC to close those kind of resources.

Write your code like this:

let with_dir fname f =
  let d = Unix.opendir f in
  try let r = f d in Unix.closedir d ; r
  with exn -> Unix.closedir d ; raise exn

once the "f" function returns, you directory handle is closed.

-- 
  Olivier


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 21:39 Ask for a more efficient way to deallocate memory (full version) Fabrice.Pardo
  2007-12-09 21:55 ` [Caml-list] " Olivier Andrieu
@ 2007-12-09 21:57 ` Oliver Bandel
  2007-12-09 22:12 ` Jon Harrop
  2007-12-09 22:16 ` Oliver Bandel
  3 siblings, 0 replies; 19+ messages in thread
From: Oliver Bandel @ 2007-12-09 21:57 UTC (permalink / raw)
  To: caml-list

Hello,


Zitat von Fabrice.Pardo@lpn.cnrs.fr:

> Hello, and sorry for my previous unfinished message.
>
> As the function Unix.opendir returns a value,
> we can expect that Unix.closedir will be automatically
> called when the dir_handle value is no more referenced.
> That's not the case,

Yes, and that's a operating system issue, not OCaml's.
There btw. a long while ago was already a discussion on that.

opendir does only give back a handle, and that does not need much
memory. So it is not a problem for that you should use the Gc-module.

the error you got says: there are not more filedescriptors available to
you, and the system gives you an error because of that and OCaml then
throsws you an exception.

You have to open and to close by yourself.

There are many possiblities to do that work.
You can code it directly in a function.
So, you could open the directory, read the conetnst to alist,
then close the directory and give back the dircontens (list
of entries) as a return value.

The calling function then can collect the items.
Or give a list of dierctories to the function and
let it fo all opendir/readdir/closedir things,
collecting all dir-contents.

So, this all is possible without Gc-module, and using the
Gc-module shows me that you don't understand the rpoblem at hand: it is
not the garbage collector that must be used to throw away allocated
data, it is the Operating-System, which needs the files to be closed.

As it is not the Garbage Collector, it makes no sense to use
Gc-functions here.

Just open the dir, read the contents and then close the dir.
That's all.




> and calling opendir too many times as in
>
> for i = 1 to 2000 do
>   let d = Unix.opendir "/tmp" in ()
> done;;

This makles no sense.
Why to open directories and not use them?

Do you go to the bakery, buy 2000 breads and then
put them in your garage, not eating one of them?


[...]
>
> Replacing Unix.opendir by safe_opendir:
>
> let safe_opendir path =
>   let _ = Gc.major ()
>   and d = Unix.opendir path
>   in Gc.finalise Unix.closedir d
>
> seems a solution, but the efficiency is very low due to high
> cost of garbage collector calling function.

So, don't use it.


>
> My question is how to write a code as efficient
> as it would be possible using a reference-counted language.
[...]

So, it seems you think OCaml is very inefficient and so you try
to make coding "as efficient as possible" before you write any line of
code. This makes no sense. OCaml is astouningly efficient,
and you should better start to write your code and only if you
come to a limit, where you really think you have to optimize,
then you should.

So, do you really want to open some-thousand directories?
Then you can write a function, that gets a list of directories as
argument and then in a loop
  opens  --- reads (and daves data) --- closes


No Gc-stuff needed.

Ciao,
   Oliver


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 21:39 Ask for a more efficient way to deallocate memory (full version) Fabrice.Pardo
  2007-12-09 21:55 ` [Caml-list] " Olivier Andrieu
  2007-12-09 21:57 ` Oliver Bandel
@ 2007-12-09 22:12 ` Jon Harrop
  2007-12-09 22:34   ` Oliver Bandel
  2007-12-09 22:16 ` Oliver Bandel
  3 siblings, 1 reply; 19+ messages in thread
From: Jon Harrop @ 2007-12-09 22:12 UTC (permalink / raw)
  To: caml-list

On Sunday 09 December 2007 21:39, Fabrice.Pardo@lpn.cnrs.fr wrote:
> Hello, and sorry for my previous unfinished message.
>
> As the function Unix.opendir returns a value,
> we can expect that Unix.closedir will be automatically
> called when the dir_handle value is no more referenced.

You cannot expect that.

> ...
> My question is how to write a code as efficient
> as it would be possible using a reference-counted language.

RAII wrapped in a higher-order function as Olivier described is the idiomatic 
solution in OCaml.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 21:39 Ask for a more efficient way to deallocate memory (full version) Fabrice.Pardo
                   ` (2 preceding siblings ...)
  2007-12-09 22:12 ` Jon Harrop
@ 2007-12-09 22:16 ` Oliver Bandel
  3 siblings, 0 replies; 19+ messages in thread
From: Oliver Bandel @ 2007-12-09 22:16 UTC (permalink / raw)
  To: caml-list

Zitat von Fabrice.Pardo@lpn.cnrs.fr:

[...]
>
> My question is how to write a code as efficient
> as it would be possible using a reference-counted language.
[...]

I don't now, from which languages you come, but possibly Python or Perl.

In a discussion on using data-structures (e.g. lists) compared to do it
"with iterators", I talked witha Python-programmer. He insisted to use
iterators, because that's so much more effifient.
Yes, when using Python you possibly should use it, because it's not as
efficient like OCaml is.

I prefer to have the data in a data structure,
so when changing something I easily can insert
a map-function to recalculate the results, without
ugly hacking.
And always I was astouned on OCaml's efficiency.

But when I started with OCaml I also had same
ideas in my head: I couldn't believe that it's true
how efficient OCaml is. But you have to do very
heavy things to come to the limits. I doubt that you are at
that point. :-)


Ciao,
   Oliver


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 22:12 ` Jon Harrop
@ 2007-12-09 22:34   ` Oliver Bandel
  0 siblings, 0 replies; 19+ messages in thread
From: Oliver Bandel @ 2007-12-09 22:34 UTC (permalink / raw)
  To: caml-list

Zitat von Jon Harrop <jon@ffconsultancy.com>:

> On Sunday 09 December 2007 21:39, Fabrice.Pardo@lpn.cnrs.fr wrote:
> > Hello, and sorry for my previous unfinished message.
> >
> > As the function Unix.opendir returns a value,
> > we can expect that Unix.closedir will be automatically
> > called when the dir_handle value is no more referenced.
>
> You cannot expect that.
>
> > ...
> > My question is how to write a code as efficient
> > as it would be possible using a reference-counted language.
>
> RAII wrapped in a higher-order function as Olivier described is the
> idiomatic
> solution in OCaml.

1st: it has a little bug ;-)
2nd: why using a higher-level function here?

I would directly code the reading of the dir,
because the typical application of opendir would
be to read the contents of the dir.

But possibly you have some more ideas on how to use
the contents of a dir, different than reading ot's contents? ;-)

Or perhaps it's a matter of taste, what one preferes.

Higher-order functions are a fine thing;
I would use them, if necessary, but not
obessively every time and everywhere.

Ciao,
   Oliver

P.S.: One possibly also could use CPS for that open-read-close stuff.
      But I'm not sure if this would make sense.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-09 21:55 ` [Caml-list] " Olivier Andrieu
@ 2007-12-10 11:25   ` Fabrice Pardo
  2007-12-10 12:03     ` Michaël Le Barbier
  2007-12-10 16:33     ` Oliver Bandel
  0 siblings, 2 replies; 19+ messages in thread
From: Fabrice Pardo @ 2007-12-10 11:25 UTC (permalink / raw)
  To: caml-list

Olivier Andrieu wrote:
>
> Well, not really: a GC manages memory, not OS resources like file descriptors.
> I wouldn't recommend relying on the GC to close those kind of resources.
>
> Write your code like this:
>
> let with_dir fname f =
>   let d = Unix.opendir f in
>   try let r = f d in Unix.closedir d ; r
>   with exn -> Unix.closedir d ; raise exn
>
> once the "f" function returns, you directory handle is closed.
>
>   
Thanks, this solution is clear and compact.

I think that all functions allocating resources out of the OCaml heap
should be documented as it, warning the programmer that
he should not be lazy, relying on automatic deallocation of resources.
The fact that Unix.opendir should be better documented
in that sense is marginal for me. It was only an example
of external C-allocated resource.

When using these kind of external C functions,
OCaml seems then less comfortable to the programmer
than reference counted languages.
The price to pay is not too high as functional code permit to encapsulate
the problem as shown by Olivier.

I have a new question now:
What is the drawback if we keep hidden
unsecure external functions (allocating out of the heap
resources, as Unix.opendir),
only publishing secure functions as "with_dir" ?
Of course, I mean other drawback than changing the API.

-- 
Fabrice


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 11:25   ` Fabrice Pardo
@ 2007-12-10 12:03     ` Michaël Le Barbier
  2007-12-10 16:33     ` Oliver Bandel
  1 sibling, 0 replies; 19+ messages in thread
From: Michaël Le Barbier @ 2007-12-10 12:03 UTC (permalink / raw)
  To: Fabrice Pardo; +Cc: caml-list

Fabrice Pardo <Fabrice.Pardo@Lpn.cnrs.fr> writes:

> I have a new question now:
> What is the drawback if we keep hidden
> unsecure external functions (allocating out of the heap
> resources, as Unix.opendir),
> only publishing secure functions as "with_dir" ?
> Of course, I mean other drawback than changing the API.

A drawback is that careful management of these ressources is no more
possible. If the choices made by module designer to ensure security do
not fit well your problem, you are left with no option other than
writing poorly behaving applications.

In contrast, given a module that delegates ressource management to its
users, it is fairly trivial to wrap modules's functionnalitites into a
module that automatizes ressource management in a way that will fit
many uses.

One would write an `EasyLazyUnix', `CookedUnix' or `UnixChest' module
with automatic deallocation of ressources.
-- 
Cordialement,
Michaël LB


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 11:25   ` Fabrice Pardo
  2007-12-10 12:03     ` Michaël Le Barbier
@ 2007-12-10 16:33     ` Oliver Bandel
  2007-12-10 20:27       ` Richard Jones
  1 sibling, 1 reply; 19+ messages in thread
From: Oliver Bandel @ 2007-12-10 16:33 UTC (permalink / raw)
  To: caml-list

Zitat von Fabrice Pardo <Fabrice.Pardo@Lpn.cnrs.fr>:

[...]
> When using these kind of external C functions,
> OCaml seems then less comfortable to the programmer
> than reference counted languages.

I doubt that reference-count is the reason here.
perl also uses reference count, but Filehandles
and Dirhandles have to be closed with close / closedir.

So, a language that uses reference counting does not necessarily
handle file-/dir-handles.

There are different things you mix up.

BTW: The Unix-module it's name is "Unix"-module,
and it is not part of the so-called OCaml standard-lib.

And because it is used to access the Unix-API (hence it's name),
it should behave like a OCaml-binding to the Unix-API.
And this means: you have a opendir, a ´readdir and a closedir
function. And they behave like the Unix-API, as mentioned above.

When you come from C, use Unix-functions, and then go to
OCaml, you have the aequivalents there too.

If you want "safer" functions, write them (or use already existing
wrapper-libs, if they are there). But the need to have access to the
Unix-API is there, because it's system-near programming, and too much
High-level programmign around it, is not what one needs, when using
the system's API.

The discussion I mentioned in one of my ealrier mails
has brought in some arguments to the reason, why it is good,
not to make all the deallocation-things automatically.
I think one reason is overhead. But one reason more was
to have a separation of system-level and non-system level, but on that
point I'm quite unsure to get the arguments right from my memory.

If I have some time, I can look for the thread.
Possibly someone else has the link at hand.



[...]
>
> I have a new question now:
> What is the drawback if we keep hidden
> unsecure external functions

Unsecure?
No coredump, so it's secure.

If you need more and more and more RAM in your application and your
applications ays to you: "Sorry, all memory ehausted!",
is this an unsecure application?

Unsecure for me would be if the program throws an exception
that can't be caught ans is from a different level of programming.
If OCaml gives you a Unix-exception, when you wasted all your
filedescriptors, this IMHO is not unsecure. Unsecure in this case would
be, if you get a segmentation fault/ coredump, instead a clean
OCaml exception.



> (allocating out of the heap
> resources, as Unix.opendir),
> only publishing secure functions as "with_dir" ?

The Unix-module has the reason, that it gives you access to the system's
API. And the systems API is the systems API is the systems API.
If you need access to it, use Unix-module. If you use a higher-level
library, and do not have access to the system itself, you get more and
more bloat. I would think, this is one of the problems,
Java-programmers have... at least I remember people who had problems
with their super-super highleve network-API (including automatically
encrypted network connections) and nothing worked because of that
bloat.

What a good cure would it have been, to open a socket. ;-)


Ciao,
   Oliver

P.S.: Imagine a programming language that handles all the things you
want to have in the way you look for it... then people would say: Well,
what a bad language, that does not give me clear access to the system!
=> Then one possibly would have a problem to write system-near drivers,
where you need direct, clear access to the system, without many layers
between your language and the system!

Each thing has two (or more) sides. For me the things that OCaml offers
are the right things. OCaml is not perfect, but very very good.
And as I have done some Unix-programming, I find it not only ok, I find
it necessary to have clear access to the system, as I have also in C.
And I don't want to switch back to C, only because I want to have access
to the Unix-API! (And there is no limitation in writing coe around that
unix-module.

So: one can add wrapping code around, but you can't scratch code off, if
it is always there and bloats your libs!

It's not like in color-systems: you can use additive and subtractive
ways of mixing colors, but normally you only can add code, not throw it
away if it's built-in!


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 16:33     ` Oliver Bandel
@ 2007-12-10 20:27       ` Richard Jones
  2007-12-10 21:05         ` Oliver Bandel
  2007-12-10 21:15         ` Gordon Henriksen
  0 siblings, 2 replies; 19+ messages in thread
From: Richard Jones @ 2007-12-10 20:27 UTC (permalink / raw)
  To: Oliver Bandel; +Cc: caml-list

On Mon, Dec 10, 2007 at 05:33:21PM +0100, Oliver Bandel wrote:
> Zitat von Fabrice Pardo <Fabrice.Pardo@Lpn.cnrs.fr>:
> 
> [...]
> > When using these kind of external C functions,
> > OCaml seems then less comfortable to the programmer
> > than reference counted languages.
> 
> I doubt that reference-count is the reason here.
> perl also uses reference count, but Filehandles
> and Dirhandles have to be closed with close / closedir.

This isn't true.  In Perl file handles are closed at the end of a
scope if they are no longer used.  In other words a Perl-equivalent to
this loop will never use more than a single file descriptor:

  while (true) {
    open "foo"
  }

Even in a GC'd language which could finalize the file handle you could
never be sure when the GC is going to be called so it could use an
indefinite number of file descriptors.

There is a really good paper on this subject -- how ref counting and
garbage collection are orthogonal language concepts -- but I can't
find it right now.

Rich.

-- 
Richard Jones
Red Hat


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 20:27       ` Richard Jones
@ 2007-12-10 21:05         ` Oliver Bandel
  2007-12-10 21:15         ` Gordon Henriksen
  1 sibling, 0 replies; 19+ messages in thread
From: Oliver Bandel @ 2007-12-10 21:05 UTC (permalink / raw)
  To: caml-list

Zitat von Richard Jones <rich@annexia.org>:

> On Mon, Dec 10, 2007 at 05:33:21PM +0100, Oliver Bandel wrote:
> > Zitat von Fabrice Pardo <Fabrice.Pardo@Lpn.cnrs.fr>:
> >
> > [...]
> > > When using these kind of external C functions,
> > > OCaml seems then less comfortable to the programmer
> > > than reference counted languages.
> >
> > I doubt that reference-count is the reason here.
> > perl also uses reference count, but Filehandles
> > and Dirhandles have to be closed with close / closedir.
>
> This isn't true.  In Perl file handles are closed at the end of a
> scope if they are no longer used.  In other words a Perl-equivalent
> to
> this loop will never use more than a single file descriptor:
>
>   while (true) {
>     open "foo"
>   }


Oh, so I was wrong here. :(

Perl handles Filehandles in a special way (not as other
things in perl), and I thought it also handles Filehandles
different, in respect to reference counting.

Well, some details I've forgotten during the time I rather
used OCaml instead of Perl ;-(

Thanks for the correction.


[...]
> There is a really good paper on this subject -- how ref counting and
> garbage collection are orthogonal language concepts
[...]

That was, what I wanted to express, but my example
was wrong.

Ciao,
   Oliver


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 20:27       ` Richard Jones
  2007-12-10 21:05         ` Oliver Bandel
@ 2007-12-10 21:15         ` Gordon Henriksen
  2007-12-10 22:13           ` Oliver Bandel
  2007-12-10 23:24           ` Jon Harrop
  1 sibling, 2 replies; 19+ messages in thread
From: Gordon Henriksen @ 2007-12-10 21:15 UTC (permalink / raw)
  To: caml-list

[-- Attachment #1: Type: text/plain, Size: 2606 bytes --]

On Dec 10, 2007, at 15:27, Richard Jones wrote:

> On Mon, Dec 10, 2007 at 05:33:21PM +0100, Oliver Bandel wrote:
>
>> Zitat von Fabrice Pardo <Fabrice.Pardo@Lpn.cnrs.fr>:
>>
>>> When using these kind of external C functions, OCaml seems then  
>>> less comfortable to the programmer than reference counted languages.
>>
>> I doubt that reference-count is the reason here. perl also uses  
>> reference count, but Filehandles and Dirhandles have to be closed  
>> with close / closedir.
>
> This isn't true.  In Perl file handles are closed at the end of a  
> scope if they are no longer used.  In other words a Perl-equivalent  
> to this loop will never use more than a single file descriptor:
>
>  while (true) {
>    open "foo"
>  }
>
> Even in a GC'd language which could finalize the file handle you  
> could never be sure when the GC is going to be called so it could  
> use an indefinite number of file descriptors.
>
> There is a really good paper on this subject -- how ref counting and  
> garbage collection are orthogonal language concepts -- but I can't  
> find it right now.


True. To address the original point, however:

This is a significant problem that crops up frequently in language/ 
runtime designs, especially when migrating to static, garbage  
collected runtimes from scripting runtimes. parrot tackled this  
problem; they call it timely collection and went far out of their way  
to implement it, performing miniature collections when a resource  
variable goes out of scope. This is rather expensive.

The practical reality is that, short of re-engineering memory  
management, developers migrating from C++ (RTTI) and scripting  
languages may need to retrain themselves to recognize that:

finalization does not provide a time bound on destruction
garbage collection manages memory and no other resources

C# provides a convenient using construct to ease the pain, but Java  
code is littered with finally blocks for this reason.

Ocaml is hampered in that its try block has no native finally  
construct, which requires repetition in the normal flow-of-control and  
exceptional cases. Implementing reusable using constructs with filters  
and closures for each type of resource is the best option I've seen,  
although a generic finally construct is also possible also using  
closures:

let finally expr cleanup =
   try
     let result = expr () in
     cleanup ();
     result
   with x ->
     cleanup ();
     raise x

Unfortunately, this is not at all attractive at the call site.

— Gordon


[-- Attachment #2: Type: text/html, Size: 10331 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 21:15         ` Gordon Henriksen
@ 2007-12-10 22:13           ` Oliver Bandel
  2007-12-10 22:59             ` Jon Harrop
  2007-12-10 23:24           ` Jon Harrop
  1 sibling, 1 reply; 19+ messages in thread
From: Oliver Bandel @ 2007-12-10 22:13 UTC (permalink / raw)
  To: caml-list

Zitat von Gordon Henriksen <gordonhenriksen@mac.com>:

[...]
> Ocaml is hampered in that its try block has no native finally
> construct, which requires repetition in the normal flow-of-control
> and
> exceptional cases.
[...]


I have not looked at it in detail and did not used it,
but there is an implementation of a language extension by Martin Jambon,
that offers a try...finally.

Because I don't know how it's implemented, I don't know if this adds a
performance lack. But possibly it would make sense to look at it closer
to explore it's advantages and disadvantages...
...but I've no experience in OCaml extension, so I hope someone here can
say some wprds about it.


Look here:

   http://martin.jambon.free.fr/ocaml.html

Look for "TryFinally".


Ciao,
   Oliver


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 22:13           ` Oliver Bandel
@ 2007-12-10 22:59             ` Jon Harrop
  2007-12-10 23:29               ` Jon Harrop
                                 ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Jon Harrop @ 2007-12-10 22:59 UTC (permalink / raw)
  To: caml-list

On Monday 10 December 2007 22:13, Oliver Bandel wrote:
> Because I don't know how it's implemented, I don't know if this adds a
> performance lack. But possibly it would make sense to look at it closer
> to explore it's advantages and disadvantages...
> ...but I've no experience in OCaml extension, so I hope someone here can
> say some wprds about it.

OCaml has a great macro system called camlp4 that lets you add new syntactic 
constructs and rewrite OCaml code at will. So Martin's code adds a new 
syntactic construct and automatically injects the code for the compiler that 
we must currently write by hand. Consequently, its performance will be 
superb: as good as hand-written OCaml code.

However, using any camlp4 macros requires using the camlp4 replacement for the 
front-end of the compiler. That uses a different parsing technology (top-down 
recursive descent LL rather than bottom-up LALR) so the error messages from 
the compiler are completely different.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 21:15         ` Gordon Henriksen
  2007-12-10 22:13           ` Oliver Bandel
@ 2007-12-10 23:24           ` Jon Harrop
  1 sibling, 0 replies; 19+ messages in thread
From: Jon Harrop @ 2007-12-10 23:24 UTC (permalink / raw)
  To: caml-list

On Monday 10 December 2007 21:15, Gordon Henriksen wrote:
> let finally expr cleanup =
>    try
>      let result = expr () in
>      cleanup ();
>      result
>    with x ->
>      cleanup ();
>      raise x

I was asking Pierre Weis about getting "try finally" into OCaml recently. I 
think the best work-around is to use a combinator, i.e.:

  let try_finally x f g =
    ...

where "x" gets passed to "f" and "g", so you typically do:

  try_finally
    (open_in file)
    input_line
    close_in

which looks nicer. However, I still think this construct (and others) should 
be added to the language.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 22:59             ` Jon Harrop
@ 2007-12-10 23:29               ` Jon Harrop
  2007-12-11  2:03               ` Yitzhak Mandelbaum
  2007-12-15 21:33               ` Oliver Bandel
  2 siblings, 0 replies; 19+ messages in thread
From: Jon Harrop @ 2007-12-10 23:29 UTC (permalink / raw)
  To: caml-list

On Monday 10 December 2007 22:59, Jon Harrop wrote:
> On Monday 10 December 2007 22:13, Oliver Bandel wrote:
> > Because I don't know how it's implemented, I don't know if this adds a
> > performance lack. But possibly it would make sense to look at it closer
> > to explore it's advantages and disadvantages...
> > ...but I've no experience in OCaml extension, so I hope someone here can
> > say some wprds about it.
>
> OCaml has a great macro system called camlp4 that lets you add new
> syntactic constructs and rewrite OCaml code at will. So Martin's code adds
> a new syntactic construct and automatically injects the code for the
> compiler that we must currently write by hand. Consequently, its
> performance will be superb: as good as hand-written OCaml code.
>
> However, using any camlp4 macros requires using the camlp4 replacement for
> the front-end of the compiler. That uses a different parsing technology
> (top-down recursive descent LL rather than bottom-up LALR) so the error
> messages from the compiler are completely different.

Incidentally, the single biggest disadvantage of adding syntactic constructs 
like this using macros is that your code will no longer autoindent properly.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 22:59             ` Jon Harrop
  2007-12-10 23:29               ` Jon Harrop
@ 2007-12-11  2:03               ` Yitzhak Mandelbaum
  2007-12-15 21:33               ` Oliver Bandel
  2 siblings, 0 replies; 19+ messages in thread
From: Yitzhak Mandelbaum @ 2007-12-11  2:03 UTC (permalink / raw)
  To: caml-list

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]


On Dec 10, 2007, at 5:59 PM, Jon Harrop wrote:

>
> However, using any camlp4 macros requires using the camlp4  
> replacement for the
> front-end of the compiler. That uses a different parsing technology  
> (top-down
> recursive descent LL rather than bottom-up LALR) so the error  
> messages from
> the compiler are completely different.


Just to clarify : the error messages from the *parser* are completely  
different. Everything after the parser is the same with or without  
camlp4. Most importantly, you don't have to learn new type-checker  
messages.

Yitzhak

--------------------------------------------------
Yitzhak Mandelbaum
AT&T Labs - Research

http://www.research.att.com/~yitzhak



[-- Attachment #2: Type: text/html, Size: 3755 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-10 22:59             ` Jon Harrop
  2007-12-10 23:29               ` Jon Harrop
  2007-12-11  2:03               ` Yitzhak Mandelbaum
@ 2007-12-15 21:33               ` Oliver Bandel
  2007-12-16 15:14                 ` Jon Harrop
  2 siblings, 1 reply; 19+ messages in thread
From: Oliver Bandel @ 2007-12-15 21:33 UTC (permalink / raw)
  To: caml-list

Zitat von Jon Harrop <jon@ffconsultancy.com>:

> On Monday 10 December 2007 22:13, Oliver Bandel wrote:
> > Because I don't know how it's implemented, I don't know if this
> adds a
> > performance lack. But possibly it would make sense to look at it
> closer
> > to explore it's advantages and disadvantages...
> > ...but I've no experience in OCaml extension, so I hope someone
> here can
> > say some wprds about it.
>
> OCaml has a great macro system called camlp4 that lets you add new
> syntactic
> constructs and rewrite OCaml code at will.

Yes, so far I know it.


> So Martin's code adds a
> new
> syntactic construct and automatically injects the code for the
> compiler that
> we must currently write by hand. Consequently, its performance will
> be
> superb: as good as hand-written OCaml code.

So, is it just a kind of pre-compilation?
The core-OCaml compiler will be the same?

Is the TryFinally-code slight enough, so that
it's behaving like handwritten OCaml? Did you look
at the TryFinally in detail?


>
> However, using any camlp4 macros requires using the camlp4
> replacement for the
> front-end of the compiler.

I have not looked at the OCaml-compiler in detail,
but from what you wrote I would assume, there
are at least two parts.



> That uses a different parsing technology
> (top-down
> recursive descent LL rather than bottom-up LALR) so the error
> messages from
> the compiler are completely different.

Oh, well.... that's new to me.

I had not needed camlp4 so far. And I don't know if it makes sense to
look at it now, because there will be (or already is?) a replacement
(from 3.10 on?).

I hope the camlp4-tutorial will be updated to the camlp4-replacement.
Possibly later I might use it. Then it would be good to have
introductional material.

Ciao,
   Oliver


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Caml-list] Ask for a more efficient way to deallocate memory (full version)
  2007-12-15 21:33               ` Oliver Bandel
@ 2007-12-16 15:14                 ` Jon Harrop
  0 siblings, 0 replies; 19+ messages in thread
From: Jon Harrop @ 2007-12-16 15:14 UTC (permalink / raw)
  To: caml-list

On Saturday 15 December 2007 21:33, Oliver Bandel wrote:
> Zitat von Jon Harrop <jon@ffconsultancy.com>:
> > So Martin's code adds a
> > new
> > syntactic construct and automatically injects the code for the
> > compiler that
> > we must currently write by hand. Consequently, its performance will
> > be
> > superb: as good as hand-written OCaml code.
>
> So, is it just a kind of pre-compilation?

You could say that, yes. It is pre-compiling code written in your own syntax 
into ordinary OCaml code.

> The core-OCaml compiler will be the same?

Yes.

> Is the TryFinally-code slight enough, so that
> it's behaving like handwritten OCaml?

Exactly, yes. If you're worried about performance then there are cases where 
you can actually use camlp4 macros to improve performance.

> Did you look at the TryFinally in detail?

The article I wrote for the OCaml Journal actually walks through the creation 
of a camlp4 macro that augments OCaml's syntax with a try..finally construct.

> > However, using any camlp4 macros requires using the camlp4
> > replacement for the
> > front-end of the compiler.
>
> I have not looked at the OCaml-compiler in detail,
> but from what you wrote I would assume, there
> are at least two parts.

You don't really see that there are two parts to the compiler. The only thing 
that intrudes on the programmer (besides different error messages) is the new 
compilation line.

> I had not needed camlp4 so far. And I don't know if it makes sense to
> look at it now, because there will be (or already is?) a replacement
> (from 3.10 on?).

I've only been talking about the new implementation in OCaml 3.10 and beyond. 
There was a previous implementation as well (now renamed camlp5).

> I hope the camlp4-tutorial will be updated to the camlp4-replacement.
> Possibly later I might use it. Then it would be good to have
> introductional material.

Yes. There is certainly a shortage of information on this subject. Eventually 
the camlp5 will be phased out but I can't see that happening until its new 
replacement camlp4 is properly documented.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e


^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2007-12-16 15:57 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-12-09 21:39 Ask for a more efficient way to deallocate memory (full version) Fabrice.Pardo
2007-12-09 21:55 ` [Caml-list] " Olivier Andrieu
2007-12-10 11:25   ` Fabrice Pardo
2007-12-10 12:03     ` Michaël Le Barbier
2007-12-10 16:33     ` Oliver Bandel
2007-12-10 20:27       ` Richard Jones
2007-12-10 21:05         ` Oliver Bandel
2007-12-10 21:15         ` Gordon Henriksen
2007-12-10 22:13           ` Oliver Bandel
2007-12-10 22:59             ` Jon Harrop
2007-12-10 23:29               ` Jon Harrop
2007-12-11  2:03               ` Yitzhak Mandelbaum
2007-12-15 21:33               ` Oliver Bandel
2007-12-16 15:14                 ` Jon Harrop
2007-12-10 23:24           ` Jon Harrop
2007-12-09 21:57 ` Oliver Bandel
2007-12-09 22:12 ` Jon Harrop
2007-12-09 22:34   ` Oliver Bandel
2007-12-09 22:16 ` Oliver Bandel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).