[Caml-list] [ANN] The Missing Library

[Caml-list] [ANN] The Missing Library

[John Goerzen]

Hello,

Some of you may remember my complaints about missing functions in the
standard library.  To help address those, I have started work on my own
OCaml library to augment the standard functions.  You can obtain it,
along with online API docs covering every function, at: 

  gopher://quux.org/1/devel/missinglib
  
  or

  http://quux.org/devel/missinglib

[Debian users: I have uploaded it to sid, but will take a few days to
appear.]

Some excerpts from the README:

Missinglib is a collection of OCaml-related utilities.  The following
modules are are provided:

ConfigParser            System for parsing configuration files
Hashtbloper             Hash table convenience operators
Hashtblutil             Hash table utilities
Lexingutil              Lexing-related utilities
Listutil                List-manipulation utilities
Slice                   Underlying API for Slice operators
Sliceoper               Flexible subparts of arrays, lists, and strings
Streamutil              Stream parser utilities
Strutil                 String-related utilities

The entire library has no prerequisites save the OCaml standard library and
findlib and is designed to install without complexity on a variety of
systems.  It could also easily be embedded within your own source trees
so that users need not have it installed beforehand.

----------

I would greatly appreciate constructive criticism on any aspect of the
package, especially the build system.  I have tried to make it possible
to "make; make install" on just about any platform, regardless of
availability of ocamlopt.  It took some hoop-jumping, though, so
suggestions are welcome :-)

Some basic info on the modules present: The ConfigParser module can read
and write sectioned .INI-style files and is mostly compatible with
Python's ConfigParser module.  Sliceoper defines some more powerful ways
of indexing arrays, lists, and strings (some of these concepts were
discussed on this list).  Strutil provides functions like strip, lstrip,
and rstrip (removes whitespace from either end, beginning, or end, of a
string).  Listutil provides a "replace" that is analogous to
Hashtbl.replace, but for association lists; and "sub" that is similar to
String.sub or Array.sub.  Hashtbloper defines some more useful ways of
working with hash tables, such as:

  hash /> 5 

   is the same as: 

  Hashtbl.find hash 5 

    let sections = Hashtbl.create 5;;
    let options = Hashtbl.create 5;;
    Hashtbl.replace options "option1" "value1";;
    Hashtbl.replace sections "section1" options;;
    sections /> "section1" /> "option1";;
                 returns "value1" 

   let newhash = hash // ("key", "value");; 

      (Copies hash, adds the pair to the copy, and returns it -- similar
       in concept to :: for lists)

BTW, is this list the right place for a message like this?

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Kenneth Knowles]

I assume you are aware the two ExtLib libraries that are also "missing" standard
library functionality?  I can't help but suggest some cooperation between your
three efforts to make one very well designed extended standard library (under a
new name that is more "positive," and not conflicting with another
project)

Kenn


On Fri, Apr 23, 2004 at 01:51:48PM -0500, John Goerzen wrote:
> Hello,
> 
> Some of you may remember my complaints about missing functions in the
> standard library.  To help address those, I have started work on my own
> OCaml library to augment the standard functions.  You can obtain it,
> along with online API docs covering every function, at: 
> 
>   gopher://quux.org/1/devel/missinglib
>   
>   or
> 
>   http://quux.org/devel/missinglib
> 
> [Debian users: I have uploaded it to sid, but will take a few days to
> appear.]
> 
> Some excerpts from the README:
> 
> Missinglib is a collection of OCaml-related utilities.  The following
> modules are are provided:
> 
> ConfigParser            System for parsing configuration files
> Hashtbloper             Hash table convenience operators
> Hashtblutil             Hash table utilities
> Lexingutil              Lexing-related utilities
> Listutil                List-manipulation utilities
> Slice                   Underlying API for Slice operators
> Sliceoper               Flexible subparts of arrays, lists, and strings
> Streamutil              Stream parser utilities
> Strutil                 String-related utilities
> 
> The entire library has no prerequisites save the OCaml standard library and
> findlib and is designed to install without complexity on a variety of
> systems.  It could also easily be embedded within your own source trees
> so that users need not have it installed beforehand.
> 
> ----------
> 
> I would greatly appreciate constructive criticism on any aspect of the
> package, especially the build system.  I have tried to make it possible
> to "make; make install" on just about any platform, regardless of
> availability of ocamlopt.  It took some hoop-jumping, though, so
> suggestions are welcome :-)
> 
> Some basic info on the modules present: The ConfigParser module can read
> and write sectioned .INI-style files and is mostly compatible with
> Python's ConfigParser module.  Sliceoper defines some more powerful ways
> of indexing arrays, lists, and strings (some of these concepts were
> discussed on this list).  Strutil provides functions like strip, lstrip,
> and rstrip (removes whitespace from either end, beginning, or end, of a
> string).  Listutil provides a "replace" that is analogous to
> Hashtbl.replace, but for association lists; and "sub" that is similar to
> String.sub or Array.sub.  Hashtbloper defines some more useful ways of
> working with hash tables, such as:
> 
>   hash /> 5 
> 
>    is the same as: 
> 
>   Hashtbl.find hash 5 
> 
>     let sections = Hashtbl.create 5;;
>     let options = Hashtbl.create 5;;
>     Hashtbl.replace options "option1" "value1";;
>     Hashtbl.replace sections "section1" options;;
>     sections /> "section1" /> "option1";;
>                  returns "value1" 
> 
>    let newhash = hash // ("key", "value");; 
> 
>       (Copies hash, adds the pair to the copy, and returns it -- similar
>        in concept to :: for lists)
> 
> BTW, is this list the right place for a message like this?
> 
> -- John
> 
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Alexander V. Voinov]

Hi All,

Kenneth Knowles wrote:

>I assume you are aware the two ExtLib libraries that are also "missing" standard
>library functionality?  I can't help but suggest some cooperation between your
>three efforts to make one very well designed extended standard library (under a
>new name that is more "positive," and not conflicting with another
>project)
>  
>
I'd second this. For about a year now, I've been using one of these 
extlibs, one which implements tail-recursive versions of the list 
operations, and never looked back. Recently I had to install another 
extlib because a package I need depends on it. This does not, of course, 
increase maintanability and all other -bilities you may imagine.

Thank you!

Alexander

>Kenn
>
>
>On Fri, Apr 23, 2004 at 01:51:48PM -0500, John Goerzen wrote:
>  
>
>>Hello,
>>
>>Some of you may remember my complaints about missing functions in the
>>standard library.  To help address those, I have started work on my own
>>OCaml library to augment the standard functions.  You can obtain it,
>>along with online API docs covering every function, at: 
>>
>>  gopher://quux.org/1/devel/missinglib
>>  
>>  or
>>
>>  http://quux.org/devel/missinglib
>>
>>[Debian users: I have uploaded it to sid, but will take a few days to
>>appear.]
>>
>>Some excerpts from the README:
>>
>>Missinglib is a collection of OCaml-related utilities.  The following
>>modules are are provided:
>>
>>ConfigParser            System for parsing configuration files
>>Hashtbloper             Hash table convenience operators
>>Hashtblutil             Hash table utilities
>>Lexingutil              Lexing-related utilities
>>Listutil                List-manipulation utilities
>>Slice                   Underlying API for Slice operators
>>Sliceoper               Flexible subparts of arrays, lists, and strings
>>Streamutil              Stream parser utilities
>>Strutil                 String-related utilities
>>
>>The entire library has no prerequisites save the OCaml standard library and
>>findlib and is designed to install without complexity on a variety of
>>systems.  It could also easily be embedded within your own source trees
>>so that users need not have it installed beforehand.
>>
>>----------
>>
>>I would greatly appreciate constructive criticism on any aspect of the
>>package, especially the build system.  I have tried to make it possible
>>to "make; make install" on just about any platform, regardless of
>>availability of ocamlopt.  It took some hoop-jumping, though, so
>>suggestions are welcome :-)
>>
>>Some basic info on the modules present: The ConfigParser module can read
>>and write sectioned .INI-style files and is mostly compatible with
>>Python's ConfigParser module.  Sliceoper defines some more powerful ways
>>of indexing arrays, lists, and strings (some of these concepts were
>>discussed on this list).  Strutil provides functions like strip, lstrip,
>>and rstrip (removes whitespace from either end, beginning, or end, of a
>>string).  Listutil provides a "replace" that is analogous to
>>Hashtbl.replace, but for association lists; and "sub" that is similar to
>>String.sub or Array.sub.  Hashtbloper defines some more useful ways of
>>working with hash tables, such as:
>>
>>  hash /> 5 
>>
>>   is the same as: 
>>
>>  Hashtbl.find hash 5 
>>
>>    let sections = Hashtbl.create 5;;
>>    let options = Hashtbl.create 5;;
>>    Hashtbl.replace options "option1" "value1";;
>>    Hashtbl.replace sections "section1" options;;
>>    sections /> "section1" /> "option1";;
>>                 returns "value1" 
>>
>>   let newhash = hash // ("key", "value");; 
>>
>>      (Copies hash, adds the pair to the copy, and returns it -- similar
>>       in concept to :: for lists)
>>
>>BTW, is this list the right place for a message like this?
>>
>>-- John
>>
>>-------------------
>>To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
>>Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
>>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>    
>>
>
>-------------------
>To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
>Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>
>  
>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 01:09:51PM -0700, Alexander V. Voinov wrote:
> I'd second this. For about a year now, I've been using one of these 
> extlibs, one which implements tail-recursive versions of the list 
> operations, and never looked back. Recently I had to install another 
> extlib because a package I need depends on it. This does not, of course, 
> increase maintanability and all other -bilities you may imagine.

Well said.  I don't think that the right answer is for the two Extlibs
to merge; one just needs to rename (and I think it should be the C one).  
There is value in having a library that is implemented in pure OCaml,
too.

Note, though, that I didn't release something named "Extlib" :-)

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 12:52:06PM -0700, Kenneth Knowles wrote:
> 
> I assume you are aware the two ExtLib libraries that are also "missing" standard
> library functionality?  I can't help but suggest some cooperation between your
> three efforts to make one very well designed extended standard library (under a
> new name that is more "positive," and not conflicting with another
> project)

Yes, I am.  One contains C components, which is not what I'm trying to
do here.  It's a nice library, and one I'd like to see in Debian, but
unfortunately since it shares the same name as an existing one, it
can't.  You might consider this a public call for one of you to rename
your library, BTW.

My library does not conflict with anything else.  I am giong to look at
working with the extlib.sf.net folks, though.  At the surface, I'm
wanting to provide larger modules (such as the ConfigParser already
present, but perhaps also an IMAP client module, etc.) that seem not
quite what either ExtLib is all about.

But the fact that these libraries exist is, at its heart, a symptom of
the problems with the OCaml standard library.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Maxence Guesdon]

Dans son style inimitable, John Goerzen écrivait:

> But the fact that these libraries exist is, at its heart, a symptom of
> the problems with the OCaml standard library.

I'm getting bored of this song. What about this one :
"The number of extlibs is a symptom of the problems with the
community of OCaml users" ?

-- 
Maxence Guesdon

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 10:36:11PM +0200, Maxence Guesdon wrote:
> Dans son style inimitable, John Goerzen écrivait:
> 
> > But the fact that these libraries exist is, at its heart, a symptom of
> > the problems with the OCaml standard library.
> 
> I'm getting bored of this song. What about this one :
> "The number of extlibs is a symptom of the problems with the
> community of OCaml users" ?

You view it as a problem that some in the OCaml community would like to
see a more featureful and easy to use standard library?  Why?

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Maxence Guesdon]

Dans son style inimitable, John Goerzen écrivait:

> On Fri, Apr 23, 2004 at 10:36:11PM +0200, Maxence Guesdon wrote:
> > Dans son style inimitable, John Goerzen écrivait:
> > 
> > > But the fact that these libraries exist is, at its heart, a symptom of
> > > the problems with the OCaml standard library.
> > 
> > I'm getting bored of this song. What about this one :
> > "The number of extlibs is a symptom of the problems with the
> > community of OCaml users" ?
> 
> You view it as a problem that some in the OCaml community would like to
> see a more featureful and easy to use standard library?  Why?

This has already been discussed last month, see this thread
http://caml.inria.fr/archives/200404/msg00045.html

- Maxence

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Maxence Guesdon]

> > You view it as a problem that some in the OCaml community would like to
> > see a more featureful and easy to use standard library?  Why?
> 
> This has already been discussed last month, see this thread
> http://caml.inria.fr/archives/200404/msg00045.html

Sorry, the real start of the whole discussion is here:
http://caml.inria.fr/archives/200403/msg00091.html

- Maxence

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> > > You view it as a problem that some in the OCaml community would like
to
> > > see a more featureful and easy to use standard library?  Why?
> >
> > This has already been discussed last month, see this thread
> > http://caml.inria.fr/archives/200404/msg00045.html
>
> Sorry, the real start of the whole discussion is here:
> http://caml.inria.fr/archives/200403/msg00091.html
>
> - Maxence

That was indeed quite a nice thread, flammy and trolly one but actually
quite interesting :-)

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 11:18:31PM +0200, Maxence Guesdon wrote:
> 
> > > You view it as a problem that some in the OCaml community would like to
> > > see a more featureful and easy to use standard library?  Why?
> > 
> > This has already been discussed last month, see this thread
> > http://caml.inria.fr/archives/200404/msg00045.html
> 
> Sorry, the real start of the whole discussion is here:
> http://caml.inria.fr/archives/200403/msg00091.html

Got it.  We seem to be e-mailing past each other a bit here.

Anyway, I've been reading that thread and haven't really seen anyone
defending the standard library yet.  I found this message particularly
compelling:

http://caml.inria.fr/archives/200403/msg00166.html

And Xavier's reply:

http://caml.inria.fr/archives/200403/msg00171.html

Which, if I may boil several pages down into a few characters, seems to
say "We know it sucks, but don't have time to fix it quickly."  That
does not translate, in my mind, to "your complaints are invalid", which
is what you seem to be saying.  So I still don't understand where you're
coming from.

All of which, though, is somewhat beside the point.  How are we going to
*fix* it?

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Maxence Guesdon]

Dans son style inimitable, John Goerzen écrivait:

> > Sorry, the real start of the whole discussion is here:
> > http://caml.inria.fr/archives/200403/msg00091.html
> 
> Got it.  We seem to be e-mailing past each other a bit here.
> 
> Anyway, I've been reading that thread and haven't really seen anyone
> defending the standard library yet.  I found this message particularly
> compelling:
> 
> http://caml.inria.fr/archives/200403/msg00166.html
> 
> And Xavier's reply:
> 
> http://caml.inria.fr/archives/200403/msg00171.html
> 
> Which, if I may boil several pages down into a few characters, seems to
> say "We know it sucks, but don't have time to fix it quickly."  That
> does not translate, in my mind, to "your complaints are invalid", which
> is what you seem to be saying.  So I still don't understand where you're
> coming from.

I'm not saying "your complaints are invalid", i'm just (personnally) 
tired of always reading the same discussions again and again.

> All of which, though, is somewhat beside the point.  How are we going to
> *fix* it?

Why not start by merging your lib with Extlib ?

- m

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Matthieu BRUCHER]

Maxence Guesdon wrote:

>Dans son style inimitable, John Goerzen écrivait:
>
>  
>
>>>Sorry, the real start of the whole discussion is here:
>>>http://caml.inria.fr/archives/200403/msg00091.html
>>>      
>>>
>>Got it.  We seem to be e-mailing past each other a bit here.
>>
>>Anyway, I've been reading that thread and haven't really seen anyone
>>defending the standard library yet.  I found this message particularly
>>compelling:
>>
>>http://caml.inria.fr/archives/200403/msg00166.html
>>
>>And Xavier's reply:
>>
>>http://caml.inria.fr/archives/200403/msg00171.html
>>
>>Which, if I may boil several pages down into a few characters, seems to
>>say "We know it sucks, but don't have time to fix it quickly."  That
>>does not translate, in my mind, to "your complaints are invalid", which
>>is what you seem to be saying.  So I still don't understand where you're
>>coming from.
>>    
>>
>
>I'm not saying "your complaints are invalid", i'm just (personnally) 
>tired of always reading the same discussions again and again.
>
>  
>
>>All of which, though, is somewhat beside the point.  How are we going to
>>*fix* it?
>>    
>>
>
>Why not start by merging your lib with Extlib ?
>
>- m
>  
>

I'm not a well-known or simply good programmer, just my few cents, but 
if the library is that buggy, it is perhaps more useful to refactor some 
part of it ?

--
Matthieu BRUCHER
Etudiant en 3ème année à SUPELEC
Option Information, Signaux, Mesures

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Maxence Guesdon]

Dans son style inimitable, Matthieu BRUCHER écrivait:

> >Why not start by merging your lib with Extlib ?
> >
> >- m
> >  
> >
> 
> I'm not a well-known or simply good programmer, just my few cents, but 
> if the library is that buggy, it is perhaps more useful to refactor some > part of it ?

The question is not about the "buggyness" but rather the completeness
of the core library.

- m

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 11:12:04PM +0200, Maxence Guesdon wrote:
> Dans son style inimitable, John Goerzen écrivait:
> > You view it as a problem that some in the OCaml community would like to
> > see a more featureful and easy to use standard library?  Why?
> 
> This has already been discussed last month, see this thread
> http://caml.inria.fr/archives/200404/msg00045.html

I checked that message, and several followups down the road, but it
appears to be a version control debate followed by some other people
that seemed to mostly agree with me wrt the Unix module and getting code
into the standard library.  Can you provide a more specific reference
that develops your point of view more clearly?

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 04:10:03PM -0500, John Goerzen wrote:
> On Fri, Apr 23, 2004 at 10:36:11PM +0200, Maxence Guesdon wrote:
> > Dans son style inimitable, John Goerzen écrivait:
> > 
> > > But the fact that these libraries exist is, at its heart, a symptom of
> > > the problems with the OCaml standard library.
> > 
> > I'm getting bored of this song. What about this one :
> > "The number of extlibs is a symptom of the problems with the
> > community of OCaml users" ?
> 
> You view it as a problem that some in the OCaml community would like to
> see a more featureful and easy to use standard library?  Why?

Let me expand on this a bit.  It is clear that people are unsatisfied
with the quality of the standard library and that others have been able
to provide useful features (*cough* IPv6 and other basic Unix calls) far
sooner than Inria has.  Why are you oppsed to this?

There are some things that will work fine with a system such as GODI
once it gets a little more mature.  My ConfigParser module, for
instance, has no need to be in the standard library (OTOH, there's no
need for it to be outside it either, but I don't care either way on that
if GODI continues improving.)

But what about the C Extlib's expanded support for sockets and IPv6?  I
can either use its support, or the standard Unix module.  If I choose
the standard Unix module, I have compatibility with other OCaml code
written by others, but I lack IPv6.  If I choose the C Extlib module,
all my I/O is tied to that module. 

A plethora of mutually-incompatible modules that duplicate and extend
standard library features is in nobody's interest.  The result will be
an irrelevant standard library and a fragmented development community.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Alain.Frisch]

On Fri, 23 Apr 2004, John Goerzen wrote:

> Let me expand on this a bit.  It is clear that people are unsatisfied
> with the quality of the standard library and that others have been able
> to provide useful features (*cough* IPv6 and other basic Unix calls) far
> sooner than Inria has.  Why are you oppsed to this?

Please note that Unix is *not* part of the OCaml standard library. It is
just a library included in the standard distribution.

Maybe the situation would more clear if:
1] the standard library was called the bootstrap library;
2] other libraries were removed from the standard distribution (ok, maybe
   except dynlink, bigarray, threads which require some cooperaton with
   the compiler, AFAIK).

The relevant message in the thread Maxence makes reference to is probably:

  http://caml.inria.fr/archives/200403/msg00171.html


OCaml has a slow release cycle and the ocaml team wants to keep control
over the standard distribution. So, having most of the libraries outside
of this distribution is a good thing, provided there is a good package
manager (either system specific or OCaml oriented). This allows the
community to gain control over the libraries and have quicker release
cycles.


> There are some things that will work fine with a system such as GODI
> once it gets a little more mature.  My ConfigParser module, for
> instance, has no need to be in the standard library (OTOH, there's no
> need for it to be outside it either, but I don't care either way on that
> if GODI continues improving.)

GODI is a technical solution.

In order to develop something like a replacement for the standard library,
there need to be some cooperation between members of the community. From
what I have seen, the extlib projet (this one:
http://sourceforge.net/projects/ocaml-lib/) has set up an informal
peer-review system, which is good. People interested in extending the
stdlib might want to join their effort (I'm personally quite happy
with the stdlib, I don't care writing three-liners functions several
times; I'd better see more libraries for higher level stuff such as
network protocols).

> But what about the C Extlib's expanded support for sockets and IPv6?

Seems that IPv6 support is already in the ocaml cvs...


-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Sat, Apr 24, 2004 at 12:04:08AM +0200, Alain.Frisch@ens.fr wrote:
> On Fri, 23 Apr 2004, John Goerzen wrote:
> 
> > Let me expand on this a bit.  It is clear that people are unsatisfied
> > with the quality of the standard library and that others have been able
> > to provide useful features (*cough* IPv6 and other basic Unix calls) far
> > sooner than Inria has.  Why are you oppsed to this?
> 
> Please note that Unix is *not* part of the OCaml standard library. It is
> just a library included in the standard distribution.

That is interesting to note but, does it actually mean something?  (This
is not a rhetorical question; I really don't know.)  I always assumed
the libraries listed separately in the docs were just that way because I
had to explicitly list unix.cmxa or str.cmxa to link with them...

> Maybe the situation would more clear if:
> 1] the standard library was called the bootstrap library;
> 2] other libraries were removed from the standard distribution (ok, maybe
>    except dynlink, bigarray, threads which require some cooperaton with
>    the compiler, AFAIK).

Yes.  That could make a lot of things easier.  As I've pointed out in
some of my other messages, the problem with Unix in particular is that
forking it will result in a painful compatibility choice for developers.
If Unix could be split off the core and allowed to evolve more rapidly,
that seems only a good thing to me.

> The relevant message in the thread Maxence makes reference to is probably:
> 
>   http://caml.inria.fr/archives/200403/msg00171.html
> OCaml has a slow release cycle and the ocaml team wants to keep control
> over the standard distribution. So, having most of the libraries outside
> of this distribution is a good thing, provided there is a good package
> manager (either system specific or OCaml oriented). This allows the
> community to gain control over the libraries and have quicker release
> cycles.

I agree completely.  I read that message, and Xavier writes "the
preferred way to contribute to Caml is through independent libraries and
tools, not by aiming at getting your stuff in the core OCaml
distribution."  That fine if it adds something that does not already
exist in the core (database APIs, PCRE, configuration parsers to name a
few examples.)  But I don't think the suggestion really works for things
like Unix that are already there.

> peer-review system, which is good. People interested in extending the
> stdlib might want to join their effort (I'm personally quite happy
> with the stdlib, I don't care writing three-liners functions several

I guess one thing I need to do is split my activities into "extends
stdlib" and "separate module".

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Alain.Frisch]

On Fri, 23 Apr 2004, John Goerzen wrote:

> That is interesting to note but, does it actually mean something?  (This
> is not a rhetorical question; I really don't know.)  I always assumed
> the libraries listed separately in the docs were just that way because I
> had to explicitly list unix.cmxa or str.cmxa to link with them...

I guess that one of the reason for not having these libraries inside the
stdlib is to avoid useless dependencies to C units for the core system (so
that to port ocamlc to a new architecture, you have a minimal amount of
non-portable code to consider). Also, some of these libraries are really
system specific, or require extra C libraries. Better have a really
standard and portable stdlib.

> > Maybe the situation would more clear if:
> > 1] the standard library was called the bootstrap library;
> > 2] other libraries were removed from the standard distribution (ok, maybe
> >    except dynlink, bigarray, threads which require some cooperaton with
> >    the compiler, AFAIK).
>
> Yes.  That could make a lot of things easier.  As I've pointed out in
> some of my other messages, the problem with Unix in particular is that
> forking it will result in a painful compatibility choice for developers.

Extending libraries that require C support is tricky, because you have to
make sure that the OCaml distribution will still compile smoothly on all
supported platforms (do you know exactly which functions are specific to
your OS, and how to write reliable test code for the configure script ?).


Could you elaborate on the compatibility point, since it seems to be at
core of your unhapiness ?

You can have a module extending Unix and working with the same abstract
data types (like UnixExtras in his Shawn's ExtLib), e.g. Unix.file_descr.
If there is a problem, it is a maintainance problem for the developer of
the extension, but the implementation of Unix.file_descr is not likely to
change so often.

Also, you can decide to have a complete re-implementation of the
functionnalities in Unix. You are free from the burden of keeping
compatibility, so you can choose a better organization (Unix is way too
monolithic, in my opinion), etc...  The same project could use Unix and
the new library. If you don't specify the type equality MyNewUnix.fd ==
Unix.file_descr (and don't give coercion functions between these types),
you won't be able to pass your file descriptors to other libraries which
expect Unix.file_descr. Are they many libraries with a dependency to
Unix.file_descr in there interface out there ?  The common situation is
that another library use Unix internally; in this case there in no
problem.


Btw, the compatibility choice already appears with the stdlib, because
there are several incompatible ways to do the same thing in OCaml. As an
example, take the *printf function. If you write a library with an
abstract type t and want to provide a printer, you have to choose between
(or give all of):

print: (out_channel -> t -> unit) -> t -> unit
  (* for use with Printf.(f?)printf *)

print: (unit -> t -> string) -> t -> string
  (* for use with (Printf|Format).sprintf *)

print: (Buffer.t -> t -> unit) -> t -> unit
  (* for use with Printf.bprintf *)

print: (Format.formatter -> t-> unit) -> t -> unit
  (* for use with Format.fprintf *)

...


> I agree completely.  I read that message, and Xavier writes "the
> preferred way to contribute to Caml is through independent libraries and
> tools, not by aiming at getting your stuff in the core OCaml
> distribution."  That fine if it adds something that does not already
> exist in the core (database APIs, PCRE, configuration parsers to name a
> few examples.)  But I don't think the suggestion really works for things
> like Unix that are already there.

You could say that PCRE is an incompatible rewrite of Str. Good example.
Many people use it instead of Str even if it is not in the standard
distribution. Similarly, LablGTK seems more widely used than LablTk.

If a group of people is willing to put some effort into the development of
a clean Posix library (well-designed, well-tested under many systems,
well-documented, and well-maintained), it might win over Unix. I can see
no obstacle to that, neither technical nor political.

-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

[...]
> Btw, the compatibility choice already appears with the stdlib, because
> there are several incompatible ways to do the same thing in OCaml. As an
> example, take the *printf function. If you write a library with an
> abstract type t and want to provide a printer, you have to choose between
> (or give all of):
>
> print: (out_channel -> t -> unit) -> t -> unit
>   (* for use with Printf.(f?)printf *)
>
> print: (unit -> t -> string) -> t -> string
>   (* for use with (Printf|Format).sprintf *)
>
> print: (Buffer.t -> t -> unit) -> t -> unit
>   (* for use with Printf.bprintf *)
>
> print: (Format.formatter -> t-> unit) -> t -> unit
>   (* for use with Format.fprintf *)
>
> ...

There is a beginning of answer in the ExtLib : it's the new IO module that
is enabling high-level abstracts inputs/ouputs. http://ocaml-lib.sf.net as
usual :-)

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: [Caml-list] [ANN] The Missing Library
Date: Sat, 24 Apr 2004 11:28:19 +0200

> There is a beginning of answer in the ExtLib : it's the new IO module that
> is enabling high-level abstracts inputs/ouputs. http://ocaml-lib.sf.net as
> usual :-)

While we are at it...

Several libraries (Cryptokit, ocamlnet, and Camomile, as far as I
know) have their own I/O modules.  Could we unify these interfaces?
IO module of SF Extlib might be a contender, but its channels are
abstract types, which I think problematic.

>From Camomile perspective, I want channels

1) Polymorphic, so that channels can handle Unicode characters
   directly.

2) Based on Objects, so that each library can make an extension of
   channels while remain compatible with the common interface.

3) Light weight.

Using objects has the extra plus that what we really have to in this
case is only using the same method name.  We do not need to introduce
a dependency to the common I/O library.

Just for the reference, I put Camomile OOChannel interface in the
last.  I'd like to hear the opinion of library developpers.

(** Generic input/output channel *)
class type ['a] obj_input_channel = 
  object 
    method close : unit 

	(** If the channel is already terminated, raise End_of_file *)
    method get : 'a 
  end

class type ['a] obj_output_channel = 
  object
    method close : unit
    method flush : unit
    method put : 'a -> unit
  end

class ['a] channel_of_stream : 'a Stream.t -> ['a] obj_input_channel

val stream_of_channel : 'a #obj_input_channel -> 'a Stream.t

class ['a, 'b] filter_in : 
    ('b obj_output_channel -> 'a obj_output_channel) ->
      'a obj_input_channel -> 
	['b] obj_input_channel

class ['a] unget : 'a #obj_input_channel ->
    object
      inherit ['a] obj_input_channel
      method unget : 'a -> unit
    end

class type char_input_channel =
  object
    inherit [char] obj_input_channel
    method input : string -> int -> int -> int
    method really_input : string -> int -> int -> unit
  end

class type char_output_channel =
  object
    inherit [char] obj_output_channel
    method output : string -> int -> int -> unit
  end

class char_input_channel_of : char obj_input_channel -> char_input_channel
class char_output_channel_of : char obj_output_channel -> char_output_channel

class of_in_channel : Pervasives.in_channel -> char_input_channel
class of_out_channel : Pervasives.out_channel -> char_output_channel

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)

[Gerd Stolpmann]

On Son, 2004-04-25 at 10:56, Yamagata Yoriyuki wrote:
> From: "Nicolas Cannasse" <warplayer@free.fr>
> Subject: Re: [Caml-list] [ANN] The Missing Library
> Date: Sat, 24 Apr 2004 11:28:19 +0200
> 
> > There is a beginning of answer in the ExtLib : it's the new IO module that
> > is enabling high-level abstracts inputs/ouputs. http://ocaml-lib.sf.net as
> > usual :-)
> 
> While we are at it...
> 
> Several libraries (Cryptokit, ocamlnet, and Camomile, as far as I
> know) have their own I/O modules.  Could we unify these interfaces?
> IO module of SF Extlib might be a contender, but its channels are
> abstract types, which I think problematic.
> 
> From Camomile perspective, I want channels
> 
> 1) Polymorphic, so that channels can handle Unicode characters
>    directly.
> 
> 2) Based on Objects, so that each library can make an extension of
>    channels while remain compatible with the common interface.
> 
> 3) Light weight.
> 
> Using objects has the extra plus that what we really have to in this
> case is only using the same method name.  We do not need to introduce
> a dependency to the common I/O library.

As you see there are different approaches to tackle similar problems.
Extlib's IO module mainly abstracts over the input and output types, but
it is not extensible. ocamlnet's and camomile's object channels are both
class types, and thus extensible by design. They differ, however, in
what they see as their atoms, i.e. smallest entities read from and
written to a channel, for ocamlnet atoms are strings, for camomile atoms
are characters (char or UChar.t), reflecting a different view what the
libraries regard as important features.

I could imagine ocamlnet and camomile share the same signatures if
camomile would use some kind of polymorphic strings instead.
String-based I/O is much faster than character-based I/O, so camomile
would even profit from this change. However, this unification requires
that we define the algebraic properties of strings and string buffers,
which is not as easy as it sounds.

What we could do at least: Use the same method names for the same
operation, and provide adapters so users can wrap ocamlnet channels as
camomile channels and vice versa. (BTW, the same should be done for our
Unicode strings.)

Gerd

> Just for the reference, I put Camomile OOChannel interface in the
> last.  I'd like to hear the opinion of library developpers.
> 
> (** Generic input/output channel *)
> class type ['a] obj_input_channel = 
>   object 
>     method close : unit 
> 
> 	(** If the channel is already terminated, raise End_of_file *)
>     method get : 'a 
>   end
> 
> class type ['a] obj_output_channel = 
>   object
>     method close : unit
>     method flush : unit
>     method put : 'a -> unit
>   end
> 
> class ['a] channel_of_stream : 'a Stream.t -> ['a] obj_input_channel
> 
> val stream_of_channel : 'a #obj_input_channel -> 'a Stream.t
> 
> class ['a, 'b] filter_in : 
>     ('b obj_output_channel -> 'a obj_output_channel) ->
>       'a obj_input_channel -> 
> 	['b] obj_input_channel
> 
> class ['a] unget : 'a #obj_input_channel ->
>     object
>       inherit ['a] obj_input_channel
>       method unget : 'a -> unit
>     end
> 
> class type char_input_channel =
>   object
>     inherit [char] obj_input_channel
>     method input : string -> int -> int -> int
>     method really_input : string -> int -> int -> unit
>   end
> 
> class type char_output_channel =
>   object
>     inherit [char] obj_output_channel
>     method output : string -> int -> int -> unit
>   end
> 
> class char_input_channel_of : char obj_input_channel -> char_input_channel
> class char_output_channel_of : char obj_output_channel -> char_output_channel
> 
> class of_in_channel : Pervasives.in_channel -> char_input_channel
> class of_out_channel : Pervasives.out_channel -> char_output_channel
> 
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: Gerd Stolpmann <info@gerd-stolpmann.de>
Subject: Re: Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)
Date: Sun, 25 Apr 2004 13:54:01 +0200

> They differ, however, in
> what they see as their atoms, i.e. smallest entities read from and
> written to a channel, for ocamlnet atoms are strings, for camomile atoms
> are characters (char or UChar.t), reflecting a different view what the
> libraries regard as important features.
> 
> I could imagine ocamlnet and camomile share the same signatures if
> camomile would use some kind of polymorphic strings instead.
> String-based I/O is much faster than character-based I/O, so camomile
> would even profit from this change. However, this unification requires
> that we define the algebraic properties of strings and string buffers,
> which is not as easy as it sounds.

When I did a compatison, the speed of Camomile code converter is in
the same order of iconv (EUC-JP -> UTF8 2-times slower, UTF8 -> EUC-JP
50% faster).  I doubt that char-based I/O is significantly slower,
unless operation is very simple.  String-based I/O has to manage
buffer strings, which causes extra cost, and anyways the major cost
comes from elsewhere.  (For code converters, the major cost is caused
by table lookup.)

That said, I plan to add string-based I/O for character channels,
partially to interpolate C fucitons.  So, for character channels,
Camomile would be compatible ocamlnet.

--
Yamagata Yoriyuki


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: Common IO structure

[Gerd Stolpmann]

On Mon, 2004-04-26 at 16:53, Yamagata Yoriyuki wrote:
> From: Gerd Stolpmann <info@gerd-stolpmann.de>
> Subject: Re: Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)
> Date: Sun, 25 Apr 2004 13:54:01 +0200
> 
> > They differ, however, in
> > what they see as their atoms, i.e. smallest entities read from and
> > written to a channel, for ocamlnet atoms are strings, for camomile atoms
> > are characters (char or UChar.t), reflecting a different view what the
> > libraries regard as important features.
> > 
> > I could imagine ocamlnet and camomile share the same signatures if
> > camomile would use some kind of polymorphic strings instead.
> > String-based I/O is much faster than character-based I/O, so camomile
> > would even profit from this change. However, this unification requires
> > that we define the algebraic properties of strings and string buffers,
> > which is not as easy as it sounds.
> 
> When I did a compatison, the speed of Camomile code converter is in
> the same order of iconv (EUC-JP -> UTF8 2-times slower, UTF8 -> EUC-JP
> 50% faster).  I doubt that char-based I/O is significantly slower,
> unless operation is very simple.  String-based I/O has to manage
> buffer strings, which causes extra cost, and anyways the major cost
> comes from elsewhere.  (For code converters, the major cost is caused
> by table lookup.)

I really believe this, when you are doing charset conversions. Note that
ocamlnet's channels are used for other things, too, e.g. email parsing,
and char-by-char operation is simply inacceptable for that.

So I fear one cannot generally say that char-by-char I/O is the common
case.

> That said, I plan to add string-based I/O for character channels,
> partially to interpolate C fucitons.  So, for character channels,
> Camomile would be compatible ocamlnet.

Good news.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)

[Nicolas Cannasse]

> > There is a beginning of answer in the ExtLib : it's the new IO module
that
> > is enabling high-level abstracts inputs/ouputs. http://ocaml-lib.sf.net
as
> > usual :-)
>
> While we are at it...
>
> Several libraries (Cryptokit, ocamlnet, and Camomile, as far as I
> know) have their own I/O modules.  Could we unify these interfaces?
> IO module of SF Extlib might be a contender, but its channels are
> abstract types, which I think problematic.
[...]

I see your point, tell me if I'm wrong :
You would like to add ExtLib IO's support to Camomile, without actually the
need for the user to install ExtLib in order to compile your library. You're
true about that, and that's a good idea. I'll try to think about adding some
OO wrappers on ExtLib IO, as well as conversion functions from/to ExtLib
input/outputs objects and records before next release.

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: Common IO structure (was Re: [Caml-list] [ANN] The Missing Library)
Date: Sun, 25 Apr 2004 21:42:52 +0200

> I see your point, tell me if I'm wrong :
> You would like to add ExtLib IO's support to Camomile, without actually the
> need for the user to install ExtLib in order to compile your library. You're
> true about that, and that's a good idea. I'll try to think about adding some
> OO wrappers on ExtLib IO, as well as conversion functions from/to ExtLib
> input/outputs objects and records before next release.

What I want is more than that.  I want that we agree some minimal set
of methods for IO channles, and agree to accept such an IO channel as
an argument for our library functions, or provide a converter to a
native IO channel of the library.

For a polymorphic channel, minimal class types would be

(for input)
['a] object get : unit -> 'a end 
(raise End_of_file when there is no more element to read.)

(for output)
['a] object 
     put : 'a -> unit 
     flush : unit -> unit 
     close : unit -> unit
end

for a character channel,
(for input)
object input : string -> int -> int -> int end

([c#input s pos len] fills s from pos with less than [len] characters,
and returns the number of characters really filled.)

(for output)
object 
       output : string -> int -> int -> unit
       flush : unit -> unit
       close : unit -> unit
end
([c#output s pos len] outputs [len] characters from the position
[pos])

I'm interested in your opinion (and of all caml list participants).
--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Jacques GARRIGUE]

From: Yamagata Yoriyuki <yoriyuki@mbg.ocn.ne.jp>

> What I want is more than that.  I want that we agree some minimal set
> of methods for IO channles, and agree to accept such an IO channel as
> an argument for our library functions, or provide a converter to a
> native IO channel of the library.
[...]
> (for output)
> object 
>        output : string -> int -> int -> unit
>        flush : unit -> unit
>        close : unit -> unit
> end
> ([c#output s pos len] outputs [len] characters from the position
> [pos])

All this seems reasonable.
Note that Format also uses [spaces] and [newline], but there are
reasonable defaults for them.

This also emphasizes one of the advantages of objects: since their
types can be compared for equality, several libraries can use the same
type without requiring a common header (that is, if everybody agrees
on the interface, as you suggest).

     Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

> > What I want is more than that.  I want that we agree some minimal set
> > of methods for IO channles, and agree to accept such an IO channel as
> > an argument for our library functions, or provide a converter to a
> > native IO channel of the library.
[...]
> All this seems reasonable.
> Note that Format also uses [spaces] and [newline], but there are
> reasonable defaults for them.
>
> This also emphasizes one of the advantages of objects: since their
> types can be compared for equality, several libraries can use the same
> type without requiring a common header (that is, if everybody agrees
> on the interface, as you suggest).

Is there any chance of getting the same behavior with records ?
Records are currently module-bounded, if Ocaml was enabling structural
comparison (even without subtyping) it would be very useful.

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Jacques GARRIGUE]

From: "Nicolas Cannasse" <warplayer@free.fr>

> > This also emphasizes one of the advantages of objects: since their
> > types can be compared for equality, several libraries can use the same
> > type without requiring a common header (that is, if everybody agrees
> > on the interface, as you suggest).
> 
> Is there any chance of getting the same behavior with records ?
> Records are currently module-bounded, if Ocaml was enabling structural
> comparison (even without subtyping) it would be very useful.

With the current records?
There are some obstacles, like the fact records allow polymorphic
recursion, makeing impossible to check structural equality.
Or the fact that the order of members is relevant, meaning that you
would only get a weak equality anyway.

New polymorphic records?
I would rather try to make objects easier to use, as they are
certainly more powerful. All the current discussion on IO suggests
that you cannot only define one minimal interface, but actually need a
hierarchy of interfaces.

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

> > I see your point, tell me if I'm wrong :
> > You would like to add ExtLib IO's support to Camomile, without actually
the
> > need for the user to install ExtLib in order to compile your library.
You're
> > true about that, and that's a good idea. I'll try to think about adding
some
> > OO wrappers on ExtLib IO, as well as conversion functions from/to ExtLib
> > input/outputs objects and records before next release.
>
> What I want is more than that.  I want that we agree some minimal set
> of methods for IO channles, and agree to accept such an IO channel as
> an argument for our library functions, or provide a converter to a
> native IO channel of the library.
[...]
> I'm interested in your opinion (and of all caml list participants).

Did you had a look at my OO wrappers proposal for IO module (posted on
ExtLib mailling list) ?

Some infos for people who doesn't know about ExtLib IO module :
ExtLib IO library is bi-polymorphic for performances reasons : it as two
polymorphic types parameters. A file input for example is a (char,string)
IO.input so there is two things we can write : chars and strings. Outputs
have a third parameter that is "what is returned by the close function" : a
file output is a (char,string,unit) IO.ouput , and a buffer output is a
(char,string,string) IO.output (the contents of the buffer is returned when
the ouput is closed).

here's how are created IOs :

val create_in :
  read:(unit -> 'a) ->
  nread:(int -> 'b) ->
  pos:(unit -> int) ->
  available:(unit -> int option) -> close:(unit -> unit) -> ('a, 'b) input

val create_out :
  write:('a -> unit) ->
  nwrite:('b -> unit) ->
  pos:(unit -> int) ->
  flush:(unit -> unit) -> close:(unit -> 'c) -> ('a, 'b, 'c) output

the "minimal set of methods" is 5 methods for both.
The OO wrappers I proposed are doing exactly what you want , they're
converting from and to IO input/outputs. Could you explain what part of the
problem they're not answering ?

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[skaller]

On Tue, 2004-04-27 at 00:23, Nicolas Cannasse wrote:

> Some infos for people who doesn't know about ExtLib IO module :
> ExtLib IO library is bi-polymorphic for performances reasons : it as two
> polymorphic types parameters. A file input for example is a (char,string)
> IO.input so there is two things we can write : chars and strings. Outputs
> have a third parameter that is "what is returned by the close function" : a
> file output is a (char,string,unit) IO.ouput , and a buffer output is a
> (char,string,string) IO.output (the contents of the buffer is returned when
> the ouput is closed).

That's pretty nice concept.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Mon, 26 Apr 2004 16:23:09 +0200

> val create_in :
>   read:(unit -> 'a) ->
>   nread:(int -> 'b) ->
>   pos:(unit -> int) ->
>   available:(unit -> int option) -> close:(unit -> unit) -> ('a, 'b) input
> 
> val create_out :
>   write:('a -> unit) ->
>   nwrite:('b -> unit) ->
>   pos:(unit -> int) ->
>   flush:(unit -> unit) -> close:(unit -> 'c) -> ('a, 'b, 'c) output
> 
> the "minimal set of methods" is 5 methods for both.
> The OO wrappers I proposed are doing exactly what you want , they're
> converting from and to IO input/outputs. Could you explain what part of the
> problem they're not answering ?

You miss my point.  What I propose is having agreement over I/O
channels.  So, having OO wrappers only solves the half of our
problems.  Another half is whether or not developpers accept
them.

And from my point of view, your proposal has some problems.  For one
thing, it is not compatible to the already existing I/O channels in
other libraries than Extlib.  Camomile uses get and put for your read
and write, and ocamlnet and cryptokit uses input and output (IIRC) for
your nread and nwrite.  Another problem is that it is not minimal
enough.  For character converters, it is impossible to predict how
many characters will be available, for example.  And requiring "pos",
"nread", "nwrite" seems arbitrary for me.  They are somtimes useful
and improvement, but not necessary.  

Since I want that these interfaces are accepted as the common
standard, I wanted that the requirement is absolutely minimal.  My
proposal in the previous mail is inspired by the view that channels
are co-inductive types defined by its constructer and consumer.
Without those methods, they are not channels any more.

I'm interested in your opinion.

--
Yamagata Yoriyuki


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

> You miss my point.  What I propose is having agreement over I/O
> channels.  So, having OO wrappers only solves the half of our
> problems.  Another half is whether or not developpers accept
> them.
>
> And from my point of view, your proposal has some problems.  For one
> thing, it is not compatible to the already existing I/O channels in
> other libraries than Extlib.  Camomile uses get and put for your read
> and write, and ocamlnet and cryptokit uses input and output (IIRC) for
> your nread and nwrite.

So ? That's exactly what we're talking about it there : making a choice. And
that include naming of course. I don't say that the name we choosed for
ExtLib IO are better, it's just that "reading" and "writing" on an IO seems
natural to me.

> Another problem is that it is not minimal
> enough.  For character converters, it is impossible to predict how
> many characters will be available, for example.  And requiring "pos",
> "nread", "nwrite" seems arbitrary for me.  They are somtimes useful
> and improvement, but not necessary.

That's true, I agree with you but on the last point : they are necessary in
order to get good performances. Concerning "available", it returns None if
no data available. "pos" might throw an exception as well when unavailable
(looks like pos and available should have same behavior here). And
nread/nwrite can simply call n times read/write. That means that any library
can put default implementation for additional "not minimal" constructs :
they will behave poorly (writing a string char by char) but will interface
well with other IO that are supporting them correctly. If implementing
efficently nread/nwrite require additionnal effort, then let's implement a
default behavior and implement it better later. Having theses functions make
room for future improvements, which is not done with minimal IO.

> Since I want that these interfaces are accepted as the common
> standard, I wanted that the requirement is absolutely minimal.  My
> proposal in the previous mail is inspired by the view that channels
> are co-inductive types defined by its constructer and consumer.
> Without those methods, they are not channels any more.
>
> I'm interested in your opinion.

Mapping several IO ( a zlib compression + a base64 encoding + a unicode
reader ) and using them all together to read and write chars will definitly
slow down. Buffering is our friend, and require some more constructs.

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Gerd Stolpmann]

On Mon, 2004-04-26 at 21:28, Nicolas Cannasse wrote:
> > And from my point of view, your proposal has some problems.  For one
> > thing, it is not compatible to the already existing I/O channels in
> > other libraries than Extlib.  Camomile uses get and put for your read
> > and write, and ocamlnet and cryptokit uses input and output (IIRC) for
> > your nread and nwrite.
> 
> So ? That's exactly what we're talking about it there : making a choice. And
> that include naming of course. I don't say that the name we choosed for
> ExtLib IO are better, it's just that "reading" and "writing" on an IO seems
> natural to me.

That sounds like a paraphrase for "better" without using this word. I
would like to hear real arguments for why certain names should be used.
For example, one reason can be that there is already a user base.

I think names are just names, and there are usually several ways of
referring to things. However, when several independent libraries
_choose_ to name their methods in a coherent way, I would say this is
very intelligent.

> > Another problem is that it is not minimal
> > enough.  For character converters, it is impossible to predict how
> > many characters will be available, for example.  And requiring "pos",
> > "nread", "nwrite" seems arbitrary for me.  They are somtimes useful
> > and improvement, but not necessary.
> 
> That's true, I agree with you but on the last point : they are necessary in
> order to get good performances. Concerning "available", it returns None if
> no data available. "pos" might throw an exception as well when unavailable
> (looks like pos and available should have same behavior here). And
> nread/nwrite can simply call n times read/write. That means that any library
> can put default implementation for additional "not minimal" constructs :
> they will behave poorly (writing a string char by char) but will interface
> well with other IO that are supporting them correctly. If implementing
> efficently nread/nwrite require additionnal effort, then let's implement a
> default behavior and implement it better later. Having theses functions make
> room for future improvements, which is not done with minimal IO.

Guess what? ocamlnet implements all that in a convincing way. Read its
introduction to OO wrappers for I/O:

http://ocamlnet.sourceforge.net/intro/netchannels.html

The signature of Netchannels as reference:

http://cvs.sourceforge.net/viewcvs.py/ocamlnet/ocamlnet/src/netstring/netchannels.mli?rev=1.12&view=auto

Of course, we should not discuss all that, but concentrate on a
reasonable level of abstraction, no matter whether this is a low-level
or high-level abstraction for a certain library.

I would suggest to adopt the names "input", "output", "close_in",
"close_out" of the standard library, as users are already familiar with
them, and this functionality is already quite powerful. Of course, this
is only reasonable for byte channels, not for Unicode channels.

I think we should not meet on the level of character-by-character I/O,
although byte channels and Unicode channels could then share the same
method names. The reason is simple: Users of byte channels don't want to
do char-by-char I/O while users of Unicode channels can accept that. I'm
speaking of the _users_ intentionally, not of the implementors, as the
users will decide which class interface will be successful and which
not. I think the way to go is an adaptor class that translates between
byte strings and Unicode characters, and does the necessary conversions.

Of course, sharing the same method name is possible, in ocamlnet we have
e.g. output_char where camomile has put_char. So the question is whether
this is worth the effort.

As I am the main developer of ocamlnet, I can say that I am willing to
change method names, or define additional methods, when the community
agrees on a certain standard. This greatly improves the interoperability
of the libraries, and is worth the pain resulting from the numerous
follow-up changes in dependent libraries and programs.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Mon, Apr 26, 2004 at 10:56:59PM +0200, Gerd Stolpmann wrote:
> I would suggest to adopt the names "input", "output", "close_in",
> "close_out" of the standard library, as users are already familiar with
> them, and this functionality is already quite powerful. Of course, this
> is only reasonable for byte channels, not for Unicode channels.

Don't those names seem incompatible with read/write files and
bidirectional channels such as network sockets?

(Ok, so close_in and close_out could be mapped to shutdown(2) in the
latter case, but I very much suspect that would NOT be what a programmer
woudl suspect...)

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Gerd Stolpmann]

On Mon, 2004-04-26 at 23:14, John Goerzen wrote:
> On Mon, Apr 26, 2004 at 10:56:59PM +0200, Gerd Stolpmann wrote:
> > I would suggest to adopt the names "input", "output", "close_in",
> > "close_out" of the standard library, as users are already familiar with
> > them, and this functionality is already quite powerful. Of course, this
> > is only reasonable for byte channels, not for Unicode channels.
> 
> Don't those names seem incompatible with read/write files and
> bidirectional channels such as network sockets?

Why?

> (Ok, so close_in and close_out could be mapped to shutdown(2) in the
> latter case, but I very much suspect that would NOT be what a programmer
> woudl suspect...)

That depends on what the object represents. In Unix, shutdown(2) refers
to the connection whereas close(2) refers to the descriptor. That means,
one can close one half of a connection, but not one half of a
descriptor. I think this is a deficiency of the Unix API (which has
historical reasons, bidirectional pipes were added late).

As OO channels are not limited to file descriptors, I don't see why we
should model them strictly after the Unix API. Separating close_in and
close_out has the advantage that input and output behaviour can be
really independent, and it is simpler to inherit input and output
behaviour from different superclasses. When mapping to Unix, one can
implement the rule that the descriptor is closed when both directions
are closed.

With only one "close" for both directions, one cannot model
bidirectional channels without additional methods like "shutdown".

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

Gerd Stolpmann wrote:
> I would like to hear real arguments for why certain names should be used.
> For example, one reason can be that there is already a user base.

Many people are familiar with the C functions 'read' and 'write' in 
Unix.  Perhaps for this reason, Perl, Python and Java also use the names 
'read' and 'write'.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: Gerd Stolpmann <info@gerd-stolpmann.de>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Mon, 26 Apr 2004 22:56:59 +0200

> Of course, sharing the same method name is possible, in ocamlnet we have
> e.g. output_char where camomile has put_char. So the question is whether
> this is worth the effort.

Camomile uses "put", not "put_char", because channels are polymorphic.
If ocamlnet channels have input/output for strings, but have
output_char for one Unicode character, then I would say output_char is
different from Camomile "put", because Camomile "put" is supposed to
output one atom (for character channles, atom is char, not a Unicode
character.)

Since I am convinced by Gerd's argument for close_in/close_out, I
updates my proposal as
(for input)
['a] object 
     get : unit -> 'a 
     close_in : unit
end 
(raise End_of_file when there is no more element to read.)

(for output)
['a] object 
     put : 'a -> unit 
     flush : unit -> unit 
     close_out : unit -> unit
end

for a character channel,
(for input)
object 
       input : string -> int -> int -> int 
       close_in : unit
end

([c#input s pos len] fills s from pos with less than [len] characters,
and returns the number of characters really filled.)

(for output)
object 
       output : string -> int -> int -> unit
       flush : unit -> unit
       close_out : unit -> unit
end
([c#output s pos len] outputs [len] characters from the position
[pos])

But the distinction of put/input, get/output may be confusing.  Hmmm.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Gerd Stolpmann]

On Die, 2004-04-27 at 18:00, Yamagata Yoriyuki wrote:
> From: Gerd Stolpmann <info@gerd-stolpmann.de>
> Subject: Re: [Caml-list] Re: Common IO structure
> Date: Mon, 26 Apr 2004 22:56:59 +0200
> 
> > Of course, sharing the same method name is possible, in ocamlnet we have
> > e.g. output_char where camomile has put_char. So the question is whether
> > this is worth the effort.
> 
> Camomile uses "put", not "put_char", because channels are polymorphic.

Oh sorry, I just remembered it wrong.

> If ocamlnet channels have input/output for strings, but have
> output_char for one Unicode character, then I would say output_char is
> different from Camomile "put", because Camomile "put" is supposed to
> output one atom (for character channles, atom is char, not a Unicode
> character.)

Yes, in principle, they are different.

> Since I am convinced by Gerd's argument for close_in/close_out, I
> updates my proposal as
> (for input)
> ['a] object 
>      get : unit -> 'a 
>      close_in : unit
> end 
> (raise End_of_file when there is no more element to read.)
> 
> (for output)
> ['a] object 
>      put : 'a -> unit 
>      flush : unit -> unit 
>      close_out : unit -> unit
> end
> 
> for a character channel,
> (for input)
> object 
>        input : string -> int -> int -> int 
>        close_in : unit
> end
> 
> ([c#input s pos len] fills s from pos with less than [len] characters,
> and returns the number of characters really filled.)
> 
> (for output)
> object 
>        output : string -> int -> int -> unit
>        flush : unit -> unit
>        close_out : unit -> unit
> end
> ([c#output s pos len] outputs [len] characters from the position
> [pos])
> 
> But the distinction of put/input, get/output may be confusing.  Hmmm.

I am sure we are very close to a solution here. 

As another name for get/put we also have receive/send. The Event module
of the stdlib uses them.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

> > > And from my point of view, your proposal has some problems.  For one
> > > thing, it is not compatible to the already existing I/O channels in
> > > other libraries than Extlib.  Camomile uses get and put for your read
> > > and write, and ocamlnet and cryptokit uses input and output (IIRC) for
> > > your nread and nwrite.
> >
> > So ? That's exactly what we're talking about it there : making a choice.
And
> > that include naming of course. I don't say that the name we choosed for
> > ExtLib IO are better, it's just that "reading" and "writing" on an IO
seems
> > natural to me.
>
> That sounds like a paraphrase for "better" without using this word. I
> would like to hear real arguments for why certain names should be used.
> For example, one reason can be that there is already a user base.
>
> I think names are just names, and there are usually several ways of
> referring to things. However, when several independent libraries
> _choose_ to name their methods in a coherent way, I would say this is
> very intelligent.

I agree with you.
Let's agree on something consistent : naming is a non-issue.

> > > Another problem is that it is not minimal
> > > enough.  For character converters, it is impossible to predict how
> > > many characters will be available, for example.  And requiring "pos",
> > > "nread", "nwrite" seems arbitrary for me.  They are somtimes useful
> > > and improvement, but not necessary.
> >
> > That's true, I agree with you but on the last point : they are necessary
in
> > order to get good performances. Concerning "available", it returns None
if
> > no data available. "pos" might throw an exception as well when
unavailable
> > (looks like pos and available should have same behavior here). And
> > nread/nwrite can simply call n times read/write. That means that any
library
> > can put default implementation for additional "not minimal" constructs :
> > they will behave poorly (writing a string char by char) but will
interface
> > well with other IO that are supporting them correctly. If implementing
> > efficently nread/nwrite require additionnal effort, then let's implement
a
> > default behavior and implement it better later. Having theses functions
make
> > room for future improvements, which is not done with minimal IO.
>
> Guess what? ocamlnet implements all that in a convincing way. Read its
> introduction to OO wrappers for I/O:
>
> http://ocamlnet.sourceforge.net/intro/netchannels.html

We're actually quite near to agree on what should be the minimal
requirements.
Looks like you included pos_in / pos_out into the "fundamentals methods" ,
would you agree to drop theses ?

In short, I think we all want different things :
- Yamagata Yoriyuki want IO to be on a char basis (and that makes sense for
Unicode)
- you would prefer having buffered channels (and that make sense for network
protocols, parsing, ...)
- I propose that we have two way of accessing the channel, that can be
buffered or unbuffered, or others. I think this is enough general to cover a
lot of different usage, and introduce some interesting polymorphism.
I would like to get your opinion on that.

Best Regards,
Nicolas Cannasse


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Gerd Stolpmann]

On Die, 2004-04-27 at 21:08, Nicolas Cannasse wrote:
> >
> > http://ocamlnet.sourceforge.net/intro/netchannels.html
> 
> We're actually quite near to agree on what should be the minimal
> requirements.
> Looks like you included pos_in / pos_out into the "fundamentals methods" ,
> would you agree to drop theses ?

I would not regard them as fundamental. Actually, they are derived, as
they only count the characters flowing through input/output. (They have
nothing to do with absolute file positions.)

Of course, I don't remove them from ocamlnet, as they are very useful
for that context. It is no problem to coerce them away, and to create
them when needed by inheriting them from a virtual class. That won't
complicate interoperability very much.

> In short, I think we all want different things :
> - Yamagata Yoriyuki want IO to be on a char basis (and that makes sense for
> Unicode)
> - you would prefer having buffered channels (and that make sense for network
> protocols, parsing, ...)

They can also be unbuffered, it is not specified. The nice thing about
ocamlnet's channels is that you can add buffers when you need them,
without changing the signature.

Of course, the focus are buffered channels.

> - I propose that we have two way of accessing the channel, that can be
> buffered or unbuffered, or others. I think this is enough general to cover a
> lot of different usage, and introduce some interesting polymorphism.
> I would like to get your opinion on that.

I guess you mean this one from a previous mail:

class ['a,'b] input = object
    method read : 'a
    method nread :  int -> 'b
    method close_in : unit
end

class ['a,'b,'c] output = object
    method write : 'a
    method nwrite : 'b
    method close_out : 'c
end

I doubt this is very practical. Consider you want to write Unicode
characters into a file (I think a common example). The file as such is a
byte stream, but you want an additional Unicode interface that converts
implicitly to, say UTF-8. With your idea of generalised channel, the
only way to do this is to build layers, something like

let file_as_byte_channel = new file_out_channel "name" in
let file_as_uni_channel = new convert_utf8_to_byte file_as_byte_channel

Call file_as_byte_channel # write to output a byte, and call
file_as_uni_channel # write to output a Unicode character. You don't
have a single object that can do both, however. Even worse: If you want
to use both interfaces alternately, you have to be careful to flush
buffers at the right time (in the case there are buffers).

I think it is better to have two methods, one for the polymorphic case,
and one for strings. The latter plays a special role, simply because all
I/O finally is string I/O.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

[...]
> I guess you mean this one from a previous mail:
>
> class ['a,'b] input = object
>     method read : 'a
>     method nread :  int -> 'b
>     method close_in : unit
> end
>
> class ['a,'b,'c] output = object
>     method write : 'a
>     method nwrite : 'b
>     method close_out : 'c
> end
>
> I doubt this is very practical. Consider you want to write Unicode
> characters into a file (I think a common example). The file as such is a
> byte stream, but you want an additional Unicode interface that converts
> implicitly to, say UTF-8. With your idea of generalised channel, the
> only way to do this is to build layers, something like
>
> let file_as_byte_channel = new file_out_channel "name" in
> let file_as_uni_channel = new convert_utf8_to_byte file_as_byte_channel
>
> Call file_as_byte_channel # write to output a byte, and call
> file_as_uni_channel # write to output a Unicode character. You don't
> have a single object that can do both, however. Even worse: If you want
> to use both interfaces alternately, you have to be careful to flush
> buffers at the right time (in the case there are buffers).

I don't know so much about UTF-8, but does it accept normal bytes ? For
example, ANSI chars are converted to identity by UTF-8, aren't they ? So to
write text only you only need to keep the second instance. Of course if
you're dealing with a channel that can write both binary data (without any
conversion) and text data (with UTF-8) the best is to write an adaptator
that will enable you to do the both, and flush the buffers for you. But
we're already out of the common example you're describing.

Layered IO are powerful. This is how it works in Java : you create for
example a ZipOutputStream with an existing OutputStream. Of course it's
possible that the ZipOutputStream have its own internal buffer, so you have
to be very careful to flush it before writing something directly to the
underlying OutputStream.

Pseudo code :
    use my ouputstream
        create a new zip outputstream wrapped on my outputstream
        write contents to it
        flush
    continue using my outputstream

That's same for UTF-8/binary streams.
More interesting, for example in Java you have a CRC32OutputStream, you can
put in at any layer you want, and at any time extract the CRC32 calculated
from all the data that went through it.

> I think it is better to have two methods, one for the polymorphic case,
> and one for strings. The latter plays a special role, simply because all
> I/O finally is string I/O.

I don't agree with this.
Most of IO are (char,string) IO that's true but some are not, I already show
example that were (bool,(int * int)) output.
Of course, you can always define the following :

class type ['a,'b] abstract_input = object
        method read : 'a
        method read_buf : 'b
        ....
end

class type ['a] input = ['a,string] abstract_input

So having one more polymorphic parameter is not so troublesome, and can
actually help.

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Tue, 27 Apr 2004 21:08:18 +0200

> - Yamagata Yoriyuki want IO to be on a char basis (and that makes sense for
> Unicode)
> - you would prefer having buffered channels (and that make sense for network
> protocols, parsing, ...)
> - I propose that we have two way of accessing the channel, that can be
> buffered or unbuffered, or others. I think this is enough general to cover a
> lot of different usage, and introduce some interesting polymorphism.
> I would like to get your opinion on that.

I agree buffered I/O for byte-char I/O.  I prefer

object  ... input : string -> int -> int -> int ... end
object  ... output : string -> int -> int -> unit ... end

than your nread/nwrite though. 

I am against buffered I/O for polymorphic channels, because it would
not be easy to come up with a standard for buffer types.  All
arguments for buffered I/O raised in the list are so far about
byte-character I/O (including UTF-8 channels.)

Di-polymorphic channels are interesting, but unless we have a
standard for buffer types, it would not be useful for the standard.
It will be easy to write a mapping from uni-polymorphic channels to
Di-polymorphic channels and vice verse, so IO system of Extlib does
not need to change.  In the future, when Extlib IO is widely used, we
could regard Extlib IO as the standard.

Since we do not have even common Unicode character type, we can not
discuss standardization of Unicode channels.  (one thing at a time!)
Please see my all arguments about Unicode channels as an example of
polymorphic channels.

I still believe my proposal in the previous mail 
  http://caml.inria.fr/archives/200404/msg00716.html 
is reasonable, except for the method names.

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Mon, 26 Apr 2004 21:28:53 +0200

> > You miss my point.  What I propose is having agreement over I/O
> > channels.  So, having OO wrappers only solves the half of our
> > problems.  Another half is whether or not developpers accept
> > them.
> >
> > And from my point of view, your proposal has some problems.  For one
> > thing, it is not compatible to the already existing I/O channels in
> > other libraries than Extlib.  Camomile uses get and put for your read
> > and write, and ocamlnet and cryptokit uses input and output (IIRC) for
> > your nread and nwrite.
> 
> So ? That's exactly what we're talking about it there : making a choice. And
> that include naming of course. I don't say that the name we choosed for
> ExtLib IO are better, it's just that "reading" and "writing" on an IO seems
> natural to me.

get/put are used in Camomie already, and input/output are used in
ocamlnet.  OO wrappers of Extlib IO are only recent addition, so
changing Extlib is more natural.  In this case, what Extlib should do is
just changing OO wrappers.  I do not think you need to change IO
module itself.

> 
> > Another problem is that it is not minimal
> > enough.  For character converters, it is impossible to predict how
> > many characters will be available, for example.  And requiring "pos",
> > "nread", "nwrite" seems arbitrary for me.  They are somtimes useful
> > and improvement, but not necessary.
> 
> That's true, I agree with you but on the last point : they are necessary in
> order to get good performances. Concerning "available", it returns None if
> no data available. "pos" might throw an exception as well when unavailable
> (looks like pos and available should have same behavior here).

My philosophy is to make the type informative as far as possible.  If
some method does not work, I would rather remove the method, and
notify this fact to a user in the compile time (not in the runtime).  A
user, or the library developer can provide wrappers if necessary.

Philoshophy aside, I do not see how pos and available improve
performance.  pos certainly decreases performance.  Anyways disks are
much slower than CPU, so arguing small performance benefit is
nonsense.  Since there are many possible "improvements" (seek, unget,
length, destination addresses), it would be better to stick the
algebraically minimal specification.

Note that I do not oppose extension as such.  I oppose making them as
the standard.

> And nread/nwrite can simply call n times read/write. That means that
> any library can put default implementation for additional "not
> minimal" constructs : they will behave poorly (writing a string char
> by char) but will interface well with other IO that are supporting
> them correctly. If implementing efficently nread/nwrite require
> additionnal effort, then let's implement a default behavior and
> implement it better later. Having theses functions make room for
> future improvements, which is not done with minimal IO.

Choosing a type of buffers is not trivial (except char, in this case I
propose using stirng, of courst).  For exmaple, what is for a Unicode
channel?  UTF8/UTF16/UTF32 strings, array, DynArray.t, all have their
own advantage.  And if someone uses UTF8, another uses UTF16 and so
on, then there is not much point of having standard.

Of course we would make "nread/nwrite" use a default buffer type
(maybe list, as you do Extlib), but I doubt that such "filler" methods
do any good.

For extensibility, it is guaranteed when we agreed to use object.  As
I stated above, I do not oppose using extension of channel types.
What I propose is just to make channels in each library an extension
of "standard" channel objects, and to make standard channel work as an
argument to the library API as far as possible.  (or provide a
converter)

> 
> > Since I want that these interfaces are accepted as the common
> > standard, I wanted that the requirement is absolutely minimal.  My
> > proposal in the previous mail is inspired by the view that channels
> > are co-inductive types defined by its constructer and consumer.
> > Without those methods, they are not channels any more.
> >
> > I'm interested in your opinion.
> 
> Mapping several IO ( a zlib compression + a base64 encoding + a unicode
> reader ) and using them all together to read and write chars will definitly
> slow down. Buffering is our friend, and require some more constructs.

For characters, I have proposed bufferd IO.

--
Yamagata Yoriyuki


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

> > So ? That's exactly what we're talking about it there : making a choice.
And
> > that include naming of course. I don't say that the name we choosed for
> > ExtLib IO are better, it's just that "reading" and "writing" on an IO
seems
> > natural to me.
>
> get/put are used in Camomie already, and input/output are used in
> ocamlnet.  OO wrappers of Extlib IO are only recent addition, so
> changing Extlib is more natural.  In this case, what Extlib should do is
> just changing OO wrappers.  I do not think you need to change IO
> module itself.

As someone told, read/write concepts are used in most of other languages
(including Java, C, and many others). I agree ExtLib's IO are recent and
should not dictate how other libraries methods should be named. But since
we're standardizing things, Camomile , Ocamlnet and Extlib will have all to
rewrite some code in order to be intercompatible. Since we need to do that,
let's make a choice not based on what's already written, but on what's is
best for the end user. A slashdot poll would help here :)
But don't worry, if you and Gerd agree on some naming, ExtLib IO will
follow. I just want that the read/write naming be taken care as much as
other naming possibilities.

> > > Another problem is that it is not minimal
> > > enough.  For character converters, it is impossible to predict how
> > > many characters will be available, for example.  And requiring "pos",
> > > "nread", "nwrite" seems arbitrary for me.  They are somtimes useful
> > > and improvement, but not necessary.
> >
> > That's true, I agree with you but on the last point : they are necessary
in
> > order to get good performances. Concerning "available", it returns None
if
> > no data available. "pos" might throw an exception as well when
unavailable
> > (looks like pos and available should have same behavior here).
>
> My philosophy is to make the type informative as far as possible.  If
> some method does not work, I would rather remove the method, and
> notify this fact to a user in the compile time (not in the runtime).  A
> user, or the library developer can provide wrappers if necessary.
>
> Philoshophy aside, I do not see how pos and available improve
> performance.  pos certainly decreases performance.  Anyways disks are
> much slower than CPU, so arguing small performance benefit is
> nonsense.  Since there are many possible "improvements" (seek, unget,
> length, destination addresses), it would be better to stick the
> algebraically minimal specification.
>
> Note that I do not oppose extension as such.  I oppose making them as
> the standard.


I agree on dropping pos and available from the standard. ExtLib will deal
consistenly without them.


> > And nread/nwrite can simply call n times read/write. That means that
> > any library can put default implementation for additional "not
> > minimal" constructs : they will behave poorly (writing a string char
> > by char) but will interface well with other IO that are supporting
> > them correctly. If implementing efficently nread/nwrite require
> > additionnal effort, then let's implement a default behavior and
> > implement it better later. Having theses functions make room for
> > future improvements, which is not done with minimal IO.
>
> Choosing a type of buffers is not trivial (except char, in this case I
> propose using stirng, of courst).  For exmaple, what is for a Unicode
> channel?  UTF8/UTF16/UTF32 strings, array, DynArray.t, all have their
> own advantage.  And if someone uses UTF8, another uses UTF16 and so
> on, then there is not much point of having standard.
>
> Of course we would make "nread/nwrite" use a default buffer type
> (maybe list, as you do Extlib), but I doubt that such "filler" methods
> do any good.

They'll maybe not - in the Unicode case, but they'll definilty help for
other IO.
Concerning the channels, I'm against having 4 classes instead of 2. If the
user need to write both chars and strings, he will need to carry two objects
instead of one.
My proposal is based on the following :

class input = object
     method read : char
     method nread  : int -> string
     method close_in : unit
end

class output = object
     method write : char
     method nwrite : string
     method close_out : unit
end

and since it's so easy, let's add some polymorphism to get more general and
more powerful IO objects :

class ['a,'b] input = object
    method read : 'a
    method nread :  int -> 'b
    method close_in : unit
end

class ['a,'b,'c] output = object
    method write : 'a
    method nwrite : 'b
    method close_out : 'c
end

Having bi-polymorphism over an IO is really powerful : you can handle
buffered read/write like this or polymorphic writes (two ways of reading,
two ways of writing).
An example from the ExtLib :

val input_bits : (char,'a) input -> (bool,int) input
val output_bits : (char,'a,'b) output -> (bool,(int * int),'b) output

enable you to read bit-by-bit over a channel :

let ch = input_bits .... in
let b = IO.read ch in (* read a bit as a boolean *)
let n = IO.nread ch 5 in (* read 5 bits as an integer *)
...

let ch = output_bits ... in
IO.write ch true; (* write a bit *)
IO.write ch false;
IO.nwrite ch (5,31); (* write 31 using 5 bits *)

It's not only an extension : putting that in the core interface enable a
wide kind of IO.
I'm interested in your thinking about that.

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: "Nicolas Cannasse" <warplayer@free.fr>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Tue, 27 Apr 2004 18:17:32 +0200

> As someone told, read/write concepts are used in most of other languages
> (including Java, C, and many others).

read/write of Unix are block-wise operations.  They are not suited for
names for element-wise operations.  get/put come from
get_char/put_char of C.  Maybe would get_element/put_element be
better?

> They'll maybe not - in the Unicode case, but they'll definilty help for
> other IO.

Unicode IO is one of most important IO, second of character IO.
(Maybe most important in the future.)  And I doubt the benefit of
bufferd IO, as I stated in the previous mail.  Unless operation is
very simple, and atoms are very small, (that is, character IO) extra
cost of element-wise IO is not important.  Note that channels can
internally process data by chunks.  Only difference is that copy
between the internal buffer and an external buffer occurs in bulk. or
element-wise.

I'm interested in an emprical evidence, though.

> If the user need to write both chars and strings, he will need to
> carry two objects instead of one.

No.  The guy will use a single object having a signature like this.

object
 method	get : char
 method input : string -> int -> int -> int
 method close_in : unit
end

It confirms both of class type I proposed for input.

> class input = object
>      method read : char
>      method nread  : int -> string
>      method close_in : unit
> end
> 
> class output = object
>      method write : char
>      method nwrite : string
>      method close_out : unit
> end

The types of nread and nwrite differ substantially from input/output
in ocamlnet, which means major rewrite of ocamlnet.  And I am not sure
that allocating a new string for each input offers the performance
benefit.

I'm interested in (potential) users of IO libraries.  Could someone
comment on IO system of Jave, Perl, Python, for example?

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

Yamagata Yoriyuki wrote:
> I doubt the benefit of
> bufferd IO, as I stated in the previous mail.  Unless operation is
> very simple, and atoms are very small, (that is, character IO) extra
> cost of element-wise IO is not important.

But there are times when you want to read one byte or character at a 
time, and in those cases, buffering saves the overhead of a function or 
method call per byte/char.

Buffering is also useful w you have to process a very large amount of 
data, and cannot keep it all in memory at once.

> I'm interested in (potential) users of IO libraries.  Could someone
> comment on IO system of Jave, Perl, Python, for example?

In Java there are two I/O libraries, the original one (java.io)[1] and 
the new one (java.nio)[2].  The old one has the virtue of being easy to 
understand and use, and flexible enough for many situations.  The basic 
InputStream and OutputStream classes deal only with bytes, have 
Unix-like 'read' and 'write' methods, and do no buffering.  There are 
derived classes such as FileInputStream and SocketInputStream.  The API 
allows you to add functionality to a stream by using wrappers.  For 
example, to add buffering to any InputStream, you wrap it in a 
BufferedInputStream (which is a class derived from InputStream).  To 
marshal Java objects to a byte stream, you wrap an OutputStream in an 
ObjectOutputStream, and pass objects to the ObjectOutputStream.

Classes derived from Reader and Writer deal with characters, and can be 
wrapped around streams to perform conversions between bytes and 
characters.  For example, to read bytes and convert them to characters, 
you wrap an InputStream in an InputStreamReader, which has a constructor 
that says which encoding to read, and 'read' methods that return 
(Unicode) characters.  Another example of a Reader is LineNumberReader, 
which counts lines in its input.

This is all fine as far as it goes, but it turns out to be cumbersome, 
and in some cases impossible, to implement certain things efficiently 
using this API.  The java.nio API solves these problems, but it is much 
more complicated to use.

For example, suppose you have to read a large amount of text from a 
network connection, convert it to another encoding, and save it in a 
file.  There's too much text to store all of it in memory at once, and 
you're dealing with a lot of network requests at the same time, so in 
any case you want to minimise the amount of memory used by each request. 
  You'd like to be able to read about 4K at a time, convert the bytes to 
the target encoding, and write them to the file.  You could make a 4K 
byte array and use it as a buffer, but what if the input encoding is 
UTF-8?  You might get an incomplete character at the end of the buffer; 
if the UTF-8 decoder is expecting a complete string, it will choke.

The solution in java.nio is to have two different kinds of buffer 
classes: ByteBuffer and CharBuffer.  You can fill up a ByteBuffer, and 
use a Decoder to convert the bytes to Unicode characters; the Decoder 
will read as many complete characters as it can, and put them in a 
CharBuffer.  You then 'compact' the ByteBuffer, which moves any 
remaining bytes to the beginning of the buffer, and start again. 
(Similarly, you can use an Encoder to convert the characters to bytes in 
the target encoding, filling up a ByteBuffer which you can then write to 
an output channel.)

Some of other useful things java.nio provides are:

* 'Direct' byte buffers.  'Given a direct byte buffer, the Java virtual 
machine will make a best effort to perform native I/O operations 
directly upon it. That is, it will attempt to avoid copying the buffer's 
content to (or from) an intermediate buffer before (or after) each 
invocation of one of the underlying operating system's native I/O 
operations.'

* Buffers that correspond to a memory-mapped region of a file.  This can 
be useful for dealing with huge files; it takes advantage of the 
operating system's support for memory-mapped files, where available.

* 'Scattering' channels.  'A scattering read operation reads, in a 
single invocation, a sequence of bytes into one or more of a given 
sequence of buffers. Scattering reads are often useful when implementing 
network protocols or file formats that, for example, group data into 
segments consisting of one or more fixed-length headers followed by a 
variable-length body.  Similar gathering write operations are defined in 
the GatheringByteChannel interface.'

My own view is that the flexibility and efficiency permitted by java.nio 
are valuable, but that its complexity is a problem.  The behaviour of 
the buffer classes[3] is tricky to understand and therefore error-prone.

Ben

[1] http://java.sun.com/j2se/1.4.2/docs/api/java/io/package-summary.html

[2] http://java.sun.com/j2se/1.4.2/docs/api/java/nio/package-summary.html

[3] http://java.sun.com/j2se/1.4.2/docs/api/java/nio/Buffer.html

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Wed, Apr 28, 2004 at 12:35:26AM +0100, Benjamin Geer wrote:
> In Java there are two I/O libraries, the original one (java.io)[1] and 
> the new one (java.nio)[2].  The old one has the virtue of being easy to 
> understand and use, and flexible enough for many situations.  The basic 

Uh, no.  I don't have the API reference in front of me, but I seem to
recall somewhere around a dozen or so predefined classes for doing
I/O...  all sorts of StreamReaders, etc, etc.  Please do not model
anything after this horribly bloated API.

Python is simple.  One standard for everything.  You get read(),
write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
for you), seek(), etc.  This can apply to files, strings, sockets,
pipes, whatever.  Before we can start fussing about unicode
abstractions, I think we need to have a uniform I/O layer.  Already we
have two competing ones (Pervasives vs. Unix) that don't exactly play
along well, and we have no way to emulate a channel with a Buffer (which
is a quite useful thing, one which I wish I had many times for OUnit
test cases).  Others have other I/O layers for sockets, etc. too.

Once you have a standard base to use, it makes more sense to build
Unicode or other logical readers atop that standard.  But make it lean
and flat; just tall and fat like Java.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

On Tue, Apr 27, 2004 at 10:44:15PM -0500, John Goerzen wrote:
> On Wed, Apr 28, 2004 at 12:35:26AM +0100, Benjamin Geer wrote:
> > In Java there are two I/O libraries, the original one (java.io)[1] and 
> > the new one (java.nio)[2].  The old one has the virtue of being easy to 
> > understand and use, and flexible enough for many situations.  The basic 
> 
> Uh, no.  I don't have the API reference in front of me, but I seem to
> recall somewhere around a dozen or so predefined classes for doing
> I/O...  all sorts of StreamReaders, etc, etc.  Please do not model
> anything after this horribly bloated API.

Agreed!  The java.io.* stuff is a great example of what NOT to do.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Perl4Caml lets you use any Perl library in your type-safe Objective
CAML programs. http://www.merjis.com/developers/perl4caml/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

John Goerzen wrote:
> On Wed, Apr 28, 2004 at 12:35:26AM +0100, Benjamin Geer wrote:
> 
>>In Java there are two I/O libraries, the original one (java.io)[1] and 
>>the new one (java.nio)[2].  The old one has the virtue of being easy to 
>>understand and use, and flexible enough for many situations.  The basic 
> 
> Uh, no.  I don't have the API reference in front of me,

I provided links to it in the very message you're replying to.

> but I seem to
> recall somewhere around a dozen or so predefined classes for doing
> I/O...

I've been using java.io every day for several years, and I find those 
classes simple and intuitive, particularly the layered approach of using 
wrappers to add functionality to stream objects, as Nicholas Cannasse 
points out in another message.

However, I agree that there are too many classes in java.nio; I'm pretty 
sure something simpler can be done in Caml using its more powerful 
polymorphism.

> Python is simple.  One standard for everything.  You get read(),
> write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
> for you), seek(), etc.  This can apply to files, strings, sockets,
> pipes, whatever.  Before we can start fussing about unicode
> abstractions, I think we need to have a uniform I/O layer.

OK, but then you can leave out readline(), readlines() and xreadlines(), 
because they don't make any sense unless you've already dealt with 
character encodings.

Then, before you can divide text into lines, you also need to know which 
newline character(s) to use.  This needs to be configurable 
programmatically rather than guessed based on the platform the program 
is running on; some protocols require you to use \r\n regardless of the 
platform.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Wed, Apr 28, 2004 at 10:30:13PM +0100, Benjamin Geer wrote:
> >>the new one (java.nio)[2].  The old one has the virtue of being easy to 
> >>understand and use, and flexible enough for many situations.  The basic 
> >
> >Uh, no.  I don't have the API reference in front of me,
> 
> I provided links to it in the very message you're replying to.
> 
> >but I seem to
> >recall somewhere around a dozen or so predefined classes for doing
> >I/O...
> 
> I've been using java.io every day for several years, and I find those 
> classes simple and intuitive, particularly the layered approach of using 
> wrappers to add functionality to stream objects, as Nicholas Cannasse 
> points out in another message.

I'm looking at java.io right now.  I count no less than 10 interfaces
and 50 classes.  Let's say that I want to open up a file for read/write
access and be able to seek around in it.  Looking at the class list, I
don't know if I want BufferedInputStream, BufferedOutputStream,
BufferedReader, BufferedWriter, CharArrayReader, CharArrayWriter,
DataInputStream, DataOutputStream, File, FileDescriptor,
FileInputStream, FileOutputStream, FileReader, FileWriter, InputStream,
InputStreamReader, OutputStream, OutputStreamWriter, RandomAccessFile,
Reader, or Writer.  Really, I literally *do not know how to open a
simple file*.  I would not call that intuitive.  Even OCaml, for all its
faults, makes that easier (and that's after I've already found the
function I need in Unix!)

> >Python is simple.  One standard for everything.  You get read(),
> >write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
> >for you), seek(), etc.  This can apply to files, strings, sockets,
> >pipes, whatever.  Before we can start fussing about unicode
> >abstractions, I think we need to have a uniform I/O layer.
> 
> OK, but then you can leave out readline(), readlines() and xreadlines(), 
> because they don't make any sense unless you've already dealt with 
> character encodings.

No, they can simply be implemented in terms of read().

> Then, before you can divide text into lines, you also need to know which 
> newline character(s) to use.  This needs to be configurable 
> programmatically rather than guessed based on the platform the program 
> is running on; some protocols require you to use \r\n regardless of the 
> platform.

That's pretty easy as a class variable, not to mention that \r\n or \n
line-endings can be automatically handled in a reliable way if used
uniformly throughout a file.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

Indeed.

People are really missing the point that an API is a user interface
for _programmers_.  The most important thing is that it be easy, short
and simple for programmers to do their job - ie. write _programs_.
'Java.io' and all its miriad of ill-conceived classes which require
you to remember, recall and rewite tons of crap is a great example of
a terrible _programming_ UI.

So a good way to start, and indeed the way I started when thinking
about mod_caml/ocamldbi, is:

* What programs do people need to write?

* What are they going to have to _type_ to write those programs?

Let's get what you need to _type_ as short as possible, and then you
probably have a good UI that _programmers_ can use.

</annoyed mode="off">

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
'There is a joke about American engineers and French engineers. The
American team brings a prototype to the French team. The French team's
response is: "Well, it works fine in practice; but how will it hold up
in theory?"'

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

Richard Jones wrote:
> * What programs do people need to write?

Most of what I do involves financial messages, which tend to be small, 
and come in many different encodings and formats.  So what I usually 
need to do is read until EOF (from a file, a socket or whatever), 
convert the data to a Unicode string, and run it through some kind of 
parser (e.g. an XML parser).  Going the other way, I have a Unicode 
string, and I just want to convert it to bytes and write it to a file or 
socket.  That covers 90% of what I do with I/O.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

On Thu, Apr 29, 2004 at 12:51:19PM +0100, Benjamin Geer wrote:
> Richard Jones wrote:
> >* What programs do people need to write?
> 
> Most of what I do involves financial messages, which tend to be small, 
> and come in many different encodings and formats.  So what I usually 
> need to do is read until EOF (from a file, a socket or whatever), 
> convert the data to a Unicode string, and run it through some kind of 
> parser (e.g. an XML parser).  Going the other way, I have a Unicode 
> string, and I just want to convert it to bytes and write it to a file or 
> socket.  That covers 90% of what I do with I/O.

Actually me too.  90% of my file IO requirement is to slurp a whole
file into a string, which suggests that any proposed IO interface
which doesn't allow you to do that in a single line is going to be a
non-starter:

let content = slurp_filename filename in ...

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Perl4Caml lets you use any Perl library in your type-safe Objective
CAML programs. http://www.merjis.com/developers/perl4caml/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

Richard Jones wrote:
> Actually me too.  90% of my file IO requirement is to slurp a whole
> file into a string

Sometimes I also do get very large messages, so it's nice if, when 
converting from, say, Arabic EBCDIC encoding to Unicode, I don't have to 
read the entire message as EBCDIC bytes, then convert the whole thing to 
Unicode at once.  It uses much less memory if I can read, say, 1K of 
EBCDIC bytes, convert them to Unicode characters, append them to a 
string buffer, and repeat until done.

Another thing that I get fairly often (though less often) is a file 
containing many messages separated by some delimiter.  In this case I 
want to read through the file, converting bytes to Unicode characters as 
I go along, and looking for the delimiter until I have a whole message 
in a string.  I don't want to read the whole file at once, because it 
might be far too big.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: John Goerzen <jgoerzen@complete.org>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Wed, 28 Apr 2004 16:44:42 -0500

> > >Python is simple.  One standard for everything.  You get read(),
> > >write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
> > >for you), seek(), etc.  This can apply to files, strings, sockets,
> > >pipes, whatever.  Before we can start fussing about unicode
> > >abstractions, I think we need to have a uniform I/O layer.
> > 
> > OK, but then you can leave out readline(), readlines() and xreadlines(), 
> > because they don't make any sense unless you've already dealt with 
> > character encodings.
> 
> No, they can simply be implemented in terms of read().

It will break when UTF-16/UTF-32 are used.  The line separator should
be handled after code conversion.  At least that is the idea of
Unicode standard.  (But Since Unicode standard is challenged by
reality in every aspect, maybe nobody cares.)

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 07:27:46PM +0900, Yamagata Yoriyuki wrote:
> > > >Python is simple.  One standard for everything.  You get read(),
> > > >write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
> > > >for you), seek(), etc.  This can apply to files, strings, sockets,
> > > >pipes, whatever.  Before we can start fussing about unicode
> > > >abstractions, I think we need to have a uniform I/O layer.
> > > 
> > > OK, but then you can leave out readline(), readlines() and xreadlines(), 
> > > because they don't make any sense unless you've already dealt with 
> > > character encodings.
> > 
> > No, they can simply be implemented in terms of read().
> 
> It will break when UTF-16/UTF-32 are used.  The line separator should
> be handled after code conversion.  At least that is the idea of
> Unicode standard.  (But Since Unicode standard is challenged by
> reality in every aspect, maybe nobody cares.)

You are missing the point.  read() could handle the code conversion.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: John Goerzen <jgoerzen@complete.org>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Thu, 29 Apr 2004 08:03:35 -0500

> 
> On Thu, Apr 29, 2004 at 07:27:46PM +0900, Yamagata Yoriyuki wrote:
> > > > >Python is simple.  One standard for everything.  You get read(),
> > > > >write(), readline(), readlines(), xreadlines() (hello Extlib, this one's
> > > > >for you), seek(), etc.  This can apply to files, strings, sockets,
> > > > >pipes, whatever.  Before we can start fussing about unicode
> > > > >abstractions, I think we need to have a uniform I/O layer.
> > > > 
> > > > OK, but then you can leave out readline(), readlines() and xreadlines(), 
> > > > because they don't make any sense unless you've already dealt with 
> > > > character encodings.
> > > 
> > > No, they can simply be implemented in terms of read().
> > 
> > It will break when UTF-16/UTF-32 are used.  The line separator should
> > be handled after code conversion.  At least that is the idea of
> > Unicode standard.  (But Since Unicode standard is challenged by
> > reality in every aspect, maybe nobody cares.)
> 
> You are missing the point.  read() could handle the code conversion.

No, what I wanted to say is that the line separator should be handled
in the Unicode level, not the byte-character level.  Your design
assumes read() always returns new line characters as in ASCII.  This
would not hold when read() returns UTF-16/UTF-32.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 10:40:36PM +0900, Yamagata Yoriyuki wrote:
> > > > > OK, but then you can leave out readline(), readlines() and xreadlines(), 
> > > > > because they don't make any sense unless you've already dealt with 
> > > > > character encodings.
> > > > 
> > > > No, they can simply be implemented in terms of read().
> > > 
> > > It will break when UTF-16/UTF-32 are used.  The line separator should
> > > be handled after code conversion.  At least that is the idea of
> > > Unicode standard.  (But Since Unicode standard is challenged by
> > > reality in every aspect, maybe nobody cares.)
> > 
> > You are missing the point.  read() could handle the code conversion.
> 
> No, what I wanted to say is that the line separator should be handled
> in the Unicode level, not the byte-character level.  Your design
> assumes read() always returns new line characters as in ASCII.  This
> would not hold when read() returns UTF-16/UTF-32.

I don't see why that is the case.  If read() returns UTF-16 data,
readlines() works with it, and would of course be scanning it for a
UTF-16 EOL character or string.  I don't see where that's the problem.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Yamagata Yoriyuki]

From: John Goerzen <jgoerzen@complete.org>
Subject: Re: [Caml-list] Re: Common IO structure
Date: Thu, 29 Apr 2004 09:02:40 -0500

> On Thu, Apr 29, 2004 at 10:40:36PM +0900, Yamagata Yoriyuki wrote:
> > > > > > OK, but then you can leave out readline(), readlines() and xreadlines(), 
> > > > > > because they don't make any sense unless you've already dealt with 
> > > > > > character encodings.
> > > > > 
> > > > > No, they can simply be implemented in terms of read().
> > > > 
> > > > It will break when UTF-16/UTF-32 are used.  The line separator should
> > > > be handled after code conversion.  At least that is the idea of
> > > > Unicode standard.  (But Since Unicode standard is challenged by
> > > > reality in every aspect, maybe nobody cares.)
> > > 
> > > You are missing the point.  read() could handle the code conversion.
> > 
> > No, what I wanted to say is that the line separator should be handled
> > in the Unicode level, not the byte-character level.  Your design
> > assumes read() always returns new line characters as in ASCII.  This
> > would not hold when read() returns UTF-16/UTF-32.
> 
> I don't see why that is the case.  If read() returns UTF-16 data,
> readlines() works with it, and would of course be scanning it for a
> UTF-16 EOL character or string.  I don't see where that's the problem.

Encoding could be stateful, so there would be no single representation
of EOL. (*)  Ok, this is very unlikely case currently, but I think there
is an interesting encoding for Unicode which is fully stateful.  So,
readlines() needs to fully aware of the encoding.

My proposal is mainly for sharing common channel types among
libraries, so that a user can pass a channel from a libraries to
anonther withoug writing a glue code.  Since parsing endline, or
loading the whole file into the string mainly occurs in the endpoint
of IO, I do not think standardizing them are necessary for this
purpose.

I do not think standardizing the endpoint API is important, because I
think that in the end, we will use only one library as the endpoint of
IO.

(*) IIRC, RFC defines the endianness of UTF-16 is swapped in the
middle of the stream, when "BOM" 0xfffe appears.

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[james woodyatt]

On 29 Apr 2004, at 08:31, Yamagata Yoriyuki wrote:
> Encoding could be stateful, so there would be no single representation
> of EOL. (*)  Ok, this is very unlikely case currently, but I think 
> there
> is an interesting encoding for Unicode which is fully stateful.  So,
> readlines() needs to fully aware of the encoding.

This transcoding I/O channel under discussion is required to contain 
internal state for other reasons.  With non-blocking I/O, an underlying 
transport may present only those octets that are ready for reading, 
which may leave a codepoint incomplete at the end of the currently 
received octets.  Even without non-blocking I/O, a read can be 
interrupted by a system signal event and still return less than the 
number of octets requested.  It is not sufficient to defer signal 
processing until after the read completes— sometimes (but not always), 
a signal explicitly means to abort reading immediately.

> My proposal is mainly for sharing common channel types among
> libraries, so that a user can pass a channel from a libraries to
> anonther withoug writing a glue code.  Since parsing endline, or
> loading the whole file into the string mainly occurs in the endpoint
> of IO, I do not think standardizing them are necessary for this
> purpose.
>
> I do not think standardizing the endpoint API is important, because I
> think that in the end, we will use only one library as the endpoint of
> IO.

Most of us.  Some of us have other concerns that I don't see anyone 
else trying to address.  At some point, probably soon, I will be 
writing a wrapper around OpenSSL.  I need non-blocking I/O.  I need to 
parse XML documents of unbounded length, which means using a SAX-like 
parser (I have that now).  I need to be able to parse an arbitrary 
number of XML documents simultaneously.  In potentially any of the 
legal Unicode transfer encodings.  And I need to be responsive to 
events in near real-time.

I have the "control inversion" nightmare from hell.  That's why I have 
forced myself to learn functional programming techniques.

An I/O library that I can use is simply not going to be something that 
can satisfy Richard's requirement that he be able to slurp a whole file 
into an application data structure with a single line of code.  So I'm 
writing my own.  Richard will be appalled by how it works.

So I'm watching this discussion with a certain bemused detachment: I 
wonder what new and improved API will be coming from this that I will 
still find inadequate for my tasks.

> (*) IIRC, RFC defines the endianness of UTF-16 is swapped in the
> middle of the stream, when "BOM" 0xfffe appears.

This is quite true.  Happens all the time, too.


-- 
j h woodyatt <jhw@wetware.com>
markets are only free to the people who own them.
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

james woodyatt wrote:
> I need non-blocking I/O.

Have you had a look at Gerd Stolpmann's Equeue module?

http://www.ocaml-programming.de/packages/documentation/equeue/

Ben


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[james woodyatt]

On 29 Apr 2004, at 16:53, Benjamin Geer wrote:
> james woodyatt wrote:
>> I need non-blocking I/O.
>
> Have you had a look at Gerd Stolpmann's Equeue module?

Oh yes.  I did.  I wrote something different.  I haven't released it.


-- 
j h woodyatt <jhw@wetware.com>
that's my village calling... no doubt, they want their idiot back.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

John Goerzen wrote:
> I'm looking at java.io right now.  I count no less than 10 interfaces
> and 50 classes.  Let's say that I want to open up a file for read/write
> access and be able to seek around in it.  Looking at the class list, I
> don't know if I want BufferedInputStream, BufferedOutputStream,
> BufferedReader, BufferedWriter, CharArrayReader, CharArrayWriter,
> DataInputStream, DataOutputStream, File, FileDescriptor,
> FileInputStream, FileOutputStream, FileReader, FileWriter, InputStream,
> InputStreamReader, OutputStream, OutputStreamWriter, RandomAccessFile,
> Reader, or Writer.  Really, I literally *do not know how to open a
> simple file*.  I would not call that intuitive.

You actually have to *read* the documentation, not just glance at the 
class names. :)  That's to be expected with a powerful API.  Once you 
understand the key concepts governing the design of the API, it makes 
sense, it and becomes intuitive to select the classes you need.  I tried 
to point out these concepts in the message you replied to.

To read a file containing UTF-8 text, one line at a time:

BufferedReader in =
     new BufferedReader
     (new InputStreamReader
      (new FileInputStream(filename), "UTF8"));

while (true)
{
     String line = in.readLine();

     if (line == null)
     {
         break;
     }

     System.out.println(line);
}

This illustrates the main design concept I was talking about. 
InputStream is an abstract class; different implementations either know 
how to get input from a particular source (like FileInputStream), or are 
meant to be used as wrappers around another InputStream to add 
functionality (like buffering).  All the classes whose names end in 
'Stream' deal with bytes only; the ones whose names end in 'Reader' or 
'Writer' deal with characters.  See?  It's easy once you know the pattern.

To open a file for read/write access and be able to seek around in it:

RandomAccessFile file = new RandomAccessFile(filename, "rw");

The methods in RandomAccessFile are pretty self-explanatory.

Ben

>>OK, but then you can leave out readline(), readlines() and xreadlines(), 
>>because they don't make any sense unless you've already dealt with 
>>character encodings.
> 
> No, they can simply be implemented in terms of read().

A line is a chunk of text, not of bytes.  I don't think it makes sense 
to deal with text unless you know what encoding it's in.

>>Then, before you can divide text into lines, you also need to know which 
>>newline character(s) to use.  This needs to be configurable 
>>programmatically
> 
> That's pretty easy as a class variable

I was only pointing out that neither Python (as far as I can tell) nor 
Java do this.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

On Thu, Apr 29, 2004 at 12:23:03PM +0100, Benjamin Geer wrote:
> John Goerzen wrote:
> >I'm looking at java.io right now.  I count no less than 10 interfaces
> >and 50 classes.  Let's say that I want to open up a file for read/write
> >access and be able to seek around in it.  Looking at the class list, I
> >don't know if I want BufferedInputStream, BufferedOutputStream,
> >BufferedReader, BufferedWriter, CharArrayReader, CharArrayWriter,
> >DataInputStream, DataOutputStream, File, FileDescriptor,
> >FileInputStream, FileOutputStream, FileReader, FileWriter, InputStream,
> >InputStreamReader, OutputStream, OutputStreamWriter, RandomAccessFile,
> >Reader, or Writer.  Really, I literally *do not know how to open a
> >simple file*.  I would not call that intuitive.
> 
> You actually have to *read* the documentation, not just glance at the 
> class names. :)  That's to be expected with a powerful API.  Once you 
> understand the key concepts governing the design of the API, it makes 
> sense, it and becomes intuitive to select the classes you need.  I tried 
> to point out these concepts in the message you replied to.

You shouldn't need to read a boatload of documentation just to read a
file, even one encoded in UTF-8.  Especially one encoded in UTF-8,
since in the future most files on Unix will be encoded that way.

Think of an API as like a user interface.  It's a UI for programmers
to use.

Good user interface design does *not* require you to read manuals to
find out how to use it (excepting very special cases like airplanes,
surgical equipment, etc.).  If you designed a website or a computer
program that required you to read a manual before you could use it, no
one would ever use it.  Simple fact.  Don't make an API which needs
you to read manuals to do a trivial operation like slurping in a UTF-8
file.

Here's how you read in and parse a CSV file using my OCaml CSV library:

  let csv = Csv.load csvfile in

('csv' is a list of list of strings).  I deliberately chose to make
the common case this simple because it's the common case and people
shouldn't have to remember much to use it.

> To read a file containing UTF-8 text, one line at a time:
> 
> BufferedReader in =
>     new BufferedReader
>     (new InputStreamReader
>      (new FileInputStream(filename), "UTF8"));

This example really reflects all that's wrong in Java.  Thankfully I
don't have to do Java programming any more - I got out of that job as
soon as I could.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://www.YouUnlimited.co.uk/ - management courses

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

Richard Jones wrote:
> Good user interface design does *not* require you to read manuals to
> find out how to use it

I don't expect programming APIs to be as obvious as user interfaces.  If 
I can work out how to use an API without reading a tutorial or HOWTO, 
that's great, but if not, I don't see it as a flaw in the API.

There is a tradeoff between power and flexibility on the one hand, and 
obviousness and simplicity on the other.

> I deliberately chose to make
> the common case this simple because it's the common case and people
> shouldn't have to remember much to use it.

Java does that, too.  If your default encoding is UTF-8, and you want to 
read a UTF-8 file, you can write this instead:

BufferedReader in = new BufferedReader(new FileReader(filename));

If you don't need buffering, you can write:

FileReader in = new FileReader(filename);

and read characters one at a time or in blocks, instead of line-by-line.

I like the fact that the API makes a distinction between bytes and 
characters; I can deal with I/O as raw bytes, or I can use an optional 
layer that translates bytes into characters.  If I don't need that 
layer, I don't have to pay for the overhead.  Likewise, I can add 
buffering only if I need it.  Best of all, if

 > Here's how you read in and parse a CSV file using my OCaml
 > CSV library:
 >
 >   let csv = Csv.load csvfile in

This seems to take too much for granted.  How do I specify the encoding 
and the line endings?  Is all that part of the Csv library, meaning that 
the Csv library has yet another mechanism for handling encodings?  What 
if my file is encrypted or compressed?  How do I make Csv.load use my 
decryption or decompression function?  Do you see what I'm getting at?

This is why I'm pleading for a layered API, so that character encoding, 
buffering, compression, encryption, and any other optional processing 
can be handled by optional layers that everyone can use, whether they're 
dealing with network protocols or just reading files from a disk. 
Moreover, I'm arguing that there should be some stream abstraction that 
can be passed around to things like Csv.load, and that different 
implementations of this abstraction should be free to implement 
decoding, decompression, and so on.  Does that make sense to you?

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 04:10:46PM +0100, Benjamin Geer wrote:
> This is why I'm pleading for a layered API, so that character encoding, 
> buffering, compression, encryption, and any other optional processing 

But you do not need a Java-esque API to do that.  All you need is a
standardized File object.  You could instantiate one of these by opening
a file.  Or perhaps by passing an existing object to the initializer for
a gzip decompressor or a Unicode processor.  ExtLib in CVS has one
approach to this.  I'd prefer to use OCaml's object system myself.

Perhaps I'll write up a proof-of-concept for missinglib...

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

John Goerzen wrote:
> On Thu, Apr 29, 2004 at 04:10:46PM +0100, Benjamin Geer wrote:
> 
>>This is why I'm pleading for a layered API, so that character encoding, 
>>buffering, compression, encryption, and any other optional processing 
> 
> But you do not need a Java-esque API to do that.  All you need is a
> standardized File object.  You could instantiate one of these by opening
> a file.  Or perhaps by passing an existing object to the initializer for
> a gzip decompressor or a Unicode processor.

The key for me is that I need to be able to chain processing steps 
together, so that I can, for example, decompress gzip format and convert 
the result to Unicode, a few bytes at a time.  This suggests to me that 
the gzip compressor and the Unicode processor should themselves be 
implementations of the standard File object, so I can wrap a gzip 
decompressor around an underlying data source, then wrap the Unicode 
decoder around the gzip decompressor.  The advantage of this approach is 
that the Unicode decoder doesn't know it's dealing with a gzip 
decompressor; it only knows it's dealing with something it can read 
bytes from.  I can then easily remove the decompression step if needed. 
  And that brings us back to a Java-like approach.  If you can think of 
a better way of accomplishing this, I'd love to see it.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

On Thu, Apr 29, 2004 at 04:46:49PM +0100, Benjamin Geer wrote:
> The key for me is that I need to be able to chain processing steps 
> together, so that I can, for example, decompress gzip format and convert 
> the result to Unicode, a few bytes at a time.  This suggests to me that 
> the gzip compressor and the Unicode processor should themselves be 
> implementations of the standard File object, so I can wrap a gzip 
> decompressor around an underlying data source, then wrap the Unicode 
> decoder around the gzip decompressor.  The advantage of this approach is 
> that the Unicode decoder doesn't know it's dealing with a gzip 
> decompressor; it only knows it's dealing with something it can read 
> bytes from.  I can then easily remove the decompression step if needed. 

I entirely agree that this is needed.  My focus however was on making
it simple to do the common things, and possible to do the rare, hard
stuff.  So the API designer should start by writing example programs
in the non-existant API.  Here are some rather fuzzy ideas of mine:

open IO

(* Read a whole file. *)
let content = slurp filename;;

(* Get a list of lines from a compressed file. *)
let lines = open filename >> unzip >> slurp_lines;;

(* Call function f line-by-line on a UTF-16 encoded file. *)
open filename >> utf16_decode >> slurp_lines >> (List.iter f);;

There are some obvious problems (eg. how are files closed? is '>>' a
reserved operator already?) but it's nice to think about what an easy
to use API might look like first *before* thinking about the
implementation.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Learning Objective CAML for C, C++, Perl and Java programmers:
http://www.merjis.com/richj/computers/ocaml/tutorial/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 04:46:49PM +0100, Benjamin Geer wrote:
> John Goerzen wrote:
> >On Thu, Apr 29, 2004 at 04:10:46PM +0100, Benjamin Geer wrote:
> >But you do not need a Java-esque API to do that.  All you need is a
> >standardized File object.  You could instantiate one of these by opening
> >a file.  Or perhaps by passing an existing object to the initializer for
> >a gzip decompressor or a Unicode processor.
> 
> The key for me is that I need to be able to chain processing steps 
> together, so that I can, for example, decompress gzip format and convert 
> the result to Unicode, a few bytes at a time.  This suggests to me that 
> the gzip compressor and the Unicode processor should themselves be 
> implementations of the standard File object, so I can wrap a gzip 
> decompressor around an underlying data source, then wrap the Unicode 
> decoder around the gzip decompressor.  The advantage of this approach is 
> that the Unicode decoder doesn't know it's dealing with a gzip 
> decompressor; it only knows it's dealing with something it can read 
> bytes from.  I can then easily remove the decompression step if needed. 
>  And that brings us back to a Java-like approach.  If you can think of 
> a better way of accomplishing this, I'd love to see it.

What you have proposed here is exactly what I am proposing and what
Python does.  It appears we are somehow in complete agreement about what
should happen.  I guess the disagreement is whether it is like Java.  I
maintain it is not, since there is a single File object that is used for
everything -- both files themselves and various converters.  But hey, if
you write this and say it's like Java, I'll be happy anyway.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

John Goerzen wrote:
> What you have proposed here is exactly what I am proposing and what
> Python does.  It appears we are somehow in complete agreement about what
> should happen.  I guess the disagreement is whether it is like Java.  I
> maintain it is not, since there is a single File object that is used for
> everything -- both files themselves and various converters.

A basic difference between Python and Java, perhaps, is that Java makes 
a strict distinction between bytes and characters.  In Java, characters 
are always Unicode characters.  Since the most basic kind of stream 
(what you're calling a 'file object') can't possibly be a converter, its 
interface can't return characters.  Therefore, in Java, at some point 
you need a different interface, what you could call a 'file object that 
knows about characters', which Java calls Reader.

However, in Caml we can surely do better, as the 'Common IO Structure' 
thread shows; the basic file/channel/stream type can be polymorphic. 
Now if only Nicholas Cannasse, Gerd Stolpmann and Yamagata Yoriyuki can 
only agree on the type parameters... :)

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Brian Hurt]

Again, sorry for the delay in responding...

On Thu, 29 Apr 2004, Richard Jones wrote:

> Think of an API as like a user interface.  It's a UI for programmers
> to use.

This is a dangerous analogy, I think.  There are several differences.  
First, I damned well *expect* programmers to be willing to RTFM.  

Second, and more importantly, take care that in making the API easy to use
for the common case you don't make it impossible to use in the odd case.  
Being able to extend and reuse the library in unexpected ways and in new
situations is more important to me as making it easy to use in the common
case.  Because I inevitably end up in the odd case trying to make the 
library do something it wasn't designed to do, and discovering you can't 
do that.  This is annoying enough in programs- it's enough to make me dump 
languages in APIs.

And this is one case where we can have our cake and eat it too.  My 
opinion is that the base library should be as flexible as possible- and 
then we can provide wrapper classes/functions for the common cases.  This 
increases the number of classes and functions, however...

The Java.io library has it's drawbacks, I'll freely admit.  But the core 
idea- which is the same core idea as the Unix command line, I comment- is 
the best I've ever seen for doing I/O.

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 12:23:03PM +0100, Benjamin Geer wrote:
> >InputStreamReader, OutputStream, OutputStreamWriter, RandomAccessFile,
> >Reader, or Writer.  Really, I literally *do not know how to open a
> >simple file*.  I would not call that intuitive.
> 
> You actually have to *read* the documentation, not just glance at the 
> class names. :)  That's to be expected with a powerful API.  Once you 

We were talking about being intuitive here.  I'd have to read maybe a
dozen different class descriptions, have to understand the differences
between them, and cross-reference back and forth between them to figure
out how to get the object I want.  Or pay for a Java book that describes
these relationships itself.

> understand the key concepts governing the design of the API, it makes 
> sense, it and becomes intuitive to select the classes you need.  I tried 
> to point out these concepts in the message you replied to.

Even when I lived and breathed Java every day for a year, its I/O API
was not intuitive.  Actually, most of its APIs were not intuitive.
Everybody I worked with had the Javadoc for the API bookmarked and
referred to it constantly.

In contrast, the API in Python, Perl, or even C is very easy to use.

Here's one of the problems.  Java's API makes it complex to do simple
things without simplifying complex things.  Recall my example -- being
able to open a file read/write and seeking around in it?  In C, I'd do:

   int file = open(filename, O_RDWR) or FILE * file = fopen(filename, "r+")

Perl, it is:
 
   open(FH, "+<", $filename)

Or, in Python:

   file = open(filename, "r+")

If I don't know my language's code for file modes, I have one simple
place to look.

Now, none of these examples do UTF-8 or other conversions.  That's fine.
I usually don't need that.  In fact, I dare say that any kind of
conversion like that is by far the minority case.

Java requires me to wade through and think about all of these things
plus the way the file will eventually be used (do I want an array of
bytes, an array of chars, strings, etc?) right when I open it.  That's
bad form.  Make the open a generic call, and let people build upon the
file object from there.  This is how C and Python work.  (Perl is a
little wacko with its open call, but it works that way too, mostly.)

> To read a file containing UTF-8 text, one line at a time:
> 
> BufferedReader in =
>     new BufferedReader
>     (new InputStreamReader
>      (new FileInputStream(filename), "UTF8"));
> 
> while (true)
> {
>     String line = in.readLine();
> 
>     if (line == null)
>     {
>         break;
>     }
> 
>     System.out.println(line);
> }

But the scary part is that this is about how hard it is to read a file
of ASCII text, one line at a time.  Whereas, with Python, I'd do:

for line in open(filename, "r").xreadlines():
    print line

See what I mean about intuitive?

But what about UTF-8 in Python?

import codecs
file = codecs.open(filename, "r", "UTF-8")
for line in file.xreadlines():
    print line

By all means, if we are going to emulate a design from another language,
let us emulate this one.  It is far cleaner and sensible.  For more
info, see file:/usr/share/doc/python2.3/html/lib/module-codecs.html.

Essentially, the codecs.open call opens a file handle and returns a
StreamReader object that has the file handle passed in to it.  But
here's the key: this StreamReader object is itself a "file-like object"
in Python parlance.  That means you can use it everywhere you could have
used a standard file object (assuming the code is capable of handling
Unicode strings, which it usually is.)  So you still have the helpful
abstraction of Java without all the mess.

And before you say, "See, Java has a StreamReader too!", note that
codecs defines *4* classes: StreamWriter, StreamReader,
StreamReaderWriter, and StreamRecoder.  I can handle that.

> functionality (like buffering).  All the classes whose names end in 
> 'Stream' deal with bytes only; the ones whose names end in 'Reader' or 
> 'Writer' deal with characters.  See?  It's easy once you know the pattern.

But the point is, this distinction is at the wrong place.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Thu, Apr 29, 2004 at 08:23:16AM -0500, John Goerzen wrote:
> And before you say, "See, Java has a StreamReader too!", note that
> codecs defines *4* classes: StreamWriter, StreamReader,
> StreamReaderWriter, and StreamRecoder.  I can handle that.

And I should add that the only time I even care about these 4 classes is
when I need to do some sort of charset handling or conversion.

If I'm just reading binary or ASCII text, the entire issue is completely
irrelevant to me.  As it should be.

And yet if I want to do charset conversions (the example I posted could
be extended to, for instance, convert Latin-2 to UTF-8 on the fly with
only a few more characters), it's still easy.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Benjamin Geer]

John Goerzen wrote:
> We were talking about being intuitive here.  I'd have to read maybe a
> dozen different class descriptions, have to understand the differences
> between them, and cross-reference back and forth between them to figure
> out how to get the object I want.  Or pay for a Java book that describes
> these relationships itself.

No, you just need a little tutorial; Sun has a good free one online:

http://java.sun.com/docs/books/tutorial/essential/io/index.html

> Everybody I worked with had the Javadoc for the API bookmarked and
> referred to it constantly.

I've always done that in all programming languages.  I have more 
important things to remember than all the little details of all the APIs 
I use.

> Here's one of the problems.  Java's API makes it complex to do simple
> things without simplifying complex things.  Recall my example -- being
> able to open a file read/write and seeking around in it?  In C, I'd do:
> 
>    int file = open(filename, O_RDWR) or FILE * file = fopen(filename, "r+")
> 
> Perl, it is:
>  
>    open(FH, "+<", $filename)
> 
> Or, in Python:
> 
>    file = open(filename, "r+")

How are any of those simpler than the Java example I gave you:

RandomAccessFile file = new RandomAccessFile(filename, "rw");

> Java requires me to wade through and think about all of these things
> plus the way the file will eventually be used (do I want an array of
> bytes, an array of chars, strings, etc?) right when I open it.

If you want to open a file for reading, you have this in Java:

InputStream in = new FileInputStream(filename);

That's all; you're ready to read bytes.  If you want to convert bytes 
into characters, you can make that decision later:

Reader inReader = new InputStreamReader(in);

That will use your system's default character encoding.  Since this is a 
common case, Java provides a convenience class as a shortcut (no 
InputStream needed):

Reader inReader = new FileReader(filename);

On the other hand, suppose you're reading data that's compressed in Zip 
format?  Given any InputStream (from a socket, a file, or whatever), you 
can wrap it in a ZipInputStream, which knows how to decompress a Zip file:

ZipInputStream zIn = new ZipInputStream(in);

Wasn't that easy?

> for line in open(filename, "r").xreadlines():
>     print line
> 
> See what I mean about intuitive?

I don't find it intuitive for every 'file-like object' (i.e. stream or 
channel) to know about character encodings and assume that it can 
meaningfully chop its data up into lines.  In many cases (e.g. when 
dealing with image or audio files) that assumption will be false.  It 
seems more intuitive to me to have character encodings (and compression, 
encryption, and all other transformations of bytes) in separate, 
optional layers.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Nicolas Cannasse]

[...]
> > I'm interested in (potential) users of IO libraries.  Could someone
> > comment on IO system of Jave, Perl, Python, for example?
>
> In Java there are two I/O libraries, the original one (java.io)[1] and
[...]

For information, here's the java.io.InputStream documentation. All input
streams need to implement theses functions :

 int available()
 void close()
 void mark(int readlimit)
 boolean markSupported()
 abstract  int read()
 int read(byte[] b)
 int read(byte[] b, int off, int len)
 void reset()
 long skip(long n)

IMHO, there is too much, and seekable stream should already be a special
kind of input streams.

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[skaller]

On Wed, 2004-04-28 at 02:58, Yamagata Yoriyuki wrote:

> I'm interested in an emprical evidence, though.

You don't need it. It is clear that there are
common (99%) of all cases where UTF-8 representation
of ISO10646 is the same as ASCII, and 90% of the
rest using Latin-1 which converts very very fast.

In these common cases the overhead of non-inlined
function calls to convert characters could be very serious.

Perhaps it isn't and perhaps it is. Who knows?
Providing bulk conversions seems a prudent way to
hedge your bets. It makes the interface richer,
but there is a universal default for the bulk
operations, so no burden is imposed on the implementor.

To add to the argument in favour of bulk conversions:
in principle, doing *any* conversions on I/O is a bad
idea. The order of priority is:

	1. single point codecs
	2. string codecs
	3. IO codecs

Doesn't really make sense to have (1) and (3) and not (2).
	

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[John Goerzen]

On Wed, Apr 28, 2004 at 01:58:00AM +0900, Yamagata Yoriyuki wrote:
> From: "Nicolas Cannasse" <warplayer@free.fr>
> Subject: Re: [Caml-list] Re: Common IO structure
> Date: Tue, 27 Apr 2004 18:17:32 +0200
> 
> > As someone told, read/write concepts are used in most of other languages
> > (including Java, C, and many others).
> 
> read/write of Unix are block-wise operations.  They are not suited for

They can also be used for character-at-a-time operations.

> names for element-wise operations.  get/put come from
> get_char/put_char of C.  Maybe would get_element/put_element be
> better?

But anyway, why dismiss the block-wise operations?  They're still
useful.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Common IO structure

[Richard Jones]

On Wed, Apr 28, 2004 at 01:58:00AM +0900, Yamagata Yoriyuki wrote:
> I'm interested in an emprical evidence, though.

I can't claim empirical evidence, but I do know that UTF-8
input/output is increasingly important to me.  We've moved to a model
where all the websites we're doing (even for UK customers) are UTF-8
throughout.  PostgreSQL is UNICODE, webserver serves UTF-8 pages, all
strings manipulated inside the Caml code are really UTF-8 strings
(although we rarely deal with them as anything other than pure bytes).
The only time we convert to other encodings is for handling stupid
mail systems like Hotmail which cannot deal with UTF-8.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
"One serious obstacle to the adoption of good programming languages is
the notion that everything has to be sacrificed for speed. In computer
languages as in life, speed kills." -- Mike Vanier

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Oliver Bandel]

On Fri, Apr 23, 2004 at 11:26:40PM -0500, John Goerzen wrote:
> On Sat, Apr 24, 2004 at 12:04:08AM +0200, Alain.Frisch@ens.fr wrote:
> > On Fri, 23 Apr 2004, John Goerzen wrote:
> > 
> > > Let me expand on this a bit.  It is clear that people are unsatisfied
> > > with the quality of the standard library and that others have been able
> > > to provide useful features (*cough* IPv6 and other basic Unix calls) far
> > > sooner than Inria has.  Why are you oppsed to this?
> > 
> > Please note that Unix is *not* part of the OCaml standard library. It is
> > just a library included in the standard distribution.
> 
> That is interesting to note but, does it actually mean something?  (This
> is not a rhetorical question; I really don't know.)  I always assumed
> the libraries listed separately in the docs were just that way because I
> had to explicitly list unix.cmxa or str.cmxa to link with them...

Yes, that is really intersting tolook at it this way.

> 
> > Maybe the situation would more clear if:
> > 1] the standard library was called the bootstrap library;
> > 2] other libraries were removed from the standard distribution (ok, maybe
> >    except dynlink, bigarray, threads which require some cooperaton with
> >    the compiler, AFAIK).
> 
> Yes.  That could make a lot of things easier.  As I've pointed out in
> some of my other messages, the problem with Unix in particular is that
> forking it will result in a painful compatibility choice for developers.
> If Unix could be split off the core and allowed to evolve more rapidly,
> that seems only a good thing to me.

It could also mean that - if Extlib or something else - is mature
and ok for the OCaml-core-team then it could be added here too.

Why not developping external libraries in fast cycles and
- from time to time, when reviewed by the OCaml-team -
adding such libraries in a slower cycle into the Ocaml
distribution?

Similar to Linux's unstable-stable distinction, where one
might rather use the stable versions when working in an
production environment and the unstable when one wants
to try the newest feateures... it could be done with OCaml
in a similar way: Using the newest external libs, when
you want, or using the (lower update frequencies) not-so-external
libs, that come along with the distribution, but it will not
be updated as often as the versions/releases of the "external"
lib that was not released with the OCaml distribution?

So "external" libraries could come along with the OCaml-distribution,
but will not updated as often as the same libraries on the
public cvs.

This would be not "free access cvs VERSUS INRIA-Ocaml-distribution",
it would be "free assess cvs ALONG WITH INRIA-Ocaml-distribution".



Ciao,
   Oliver

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Henri DF]

ok this thread is way too long already, and (as maxence pointed out), is 
basically the same old rehash of the same old thing, but i will still add
2c...

> On Fri, Apr 23, 2004 at 04:10:03PM -0500, John Goerzen wrote:
> Let me expand on this a bit.  It is clear that people are unsatisfied
> with the quality of the standard library and that others have been able

i would rephrase that as "that *some* [1] people are unsatisfied, where 
*some* might be a very small number". 

and we shouldn't forget that the boring, satisfied users obviously (such 
as yours truly) do not have much to complain about and so might well be
the silent majority on this list..

[1] where 'some' might be a very small number


henri
> to provide useful features (*cough* IPv6 and other basic Unix calls) far
> sooner than Inria has.  Why are you oppsed to this?
> 
> There are some things that will work fine with a system such as GODI
> once it gets a little more mature.  My ConfigParser module, for
> instance, has no need to be in the standard library (OTOH, there's no
> need for it to be outside it either, but I don't care either way on that
> if GODI continues improving.)
> 
> But what about the C Extlib's expanded support for sockets and IPv6?  I
> can either use its support, or the standard Unix module.  If I choose
> the standard Unix module, I have compatibility with other OCaml code
> written by others, but I lack IPv6.  If I choose the C Extlib module,
> all my I/O is tied to that module. 
> 
> A plethora of mutually-incompatible modules that duplicate and extend
> standard library features is in nobody's interest.  The result will be
> an irrelevant standard library and a fragmented development community.
> 
> -- John
> 
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Shawn Wagner]

On Fri, Apr 23, 2004 at 04:33:25PM -0500, John Goerzen wrote:
> But what about the C Extlib's expanded support for sockets and IPv6?  I
> can either use its support, or the standard Unix module.  If I choose
> the standard Unix module, I have compatibility with other OCaml code
> written by others, but I lack IPv6.  If I choose the C Extlib module,
> all my I/O is tied to that module.

Since you mention C, I assume you're referring to my Extlib, which doesn't
have any IPv6 support, or socket functions (Aside from providing the
sendfile syscall on a couple of OSes).

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[james woodyatt]

On 23 Apr 2004, at 14:33, John Goerzen wrote:
>
> A plethora of mutually-incompatible modules that duplicate and extend
> standard library features is in nobody's interest.  The result will be
> an irrelevant standard library and a fragmented development community.

Yeah— that was such a huge roadblock to the popularity of the C 
language.

As one of those guys who maintains a library that supplements the Ocaml 
standard library with additional stuff, e.g. IPv6 sockets, functional 
deques, and continuation monads— I have to disagree.  (And no, I'm not 
at all bummed that nobody cares about my puny library.  If you like 
Extlib, then keep using it.)

A vibrant community of people outside INRIA developing independent 
[even mutually-incompatible] extensions to the distribution libraries 
is in *everybody's* interest.  It serves as proof of the success of 
Projet Cristal that the community of users of its language is growing.  
Contention among providers of foundation library code is to be 
expected— even celebrated.  Better news still would be an open market, 
holding a plethora of mutually incompatible *closed-source* library 
modules.  That would be the *real* indication that INRIA has gotten its 
euros worth in research spending.

Vive la fragmentation!


-- 
j h woodyatt <jhw@wetware.com>
markets are only free to the people who own them.
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

RE: [Caml-list] [ANN] The Missing Library

[Brandon J. Van Every]

james woodyatt
>
> Contention among providers of foundation library code is to be
> expected— even celebrated.  Better news still would be an
> open market,
> holding a plethora of mutually incompatible *closed-source* library
> modules.  That would be the *real* indication that INRIA has
> gotten its
> euros worth in research spending.
>
> Vive la fragmentation!

Yeah, but C does have a standard library, so let's not overplay the
value of fragmentation.  Let's ask instead: is there any OCaml library
that right now, should uncontroversially be part of a standard?  If not,
a collection of 2 or 3 candidate libraries that should be groomed for
standard?  I liked Ben's comment about how the Linux people handle it,
this notion of an interim period of deciding what's adviseable to keep.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

"We live in a world of very bright people building
crappy software with total shit for tools and process."
                                - Ed Mckenzie
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.643 / Virus Database: 411 - Release Date: 3/25/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Benjamin Geer]

james woodyatt wrote:
> On 23 Apr 2004, at 14:33, John Goerzen wrote:
>> A plethora of mutually-incompatible modules that duplicate and extend
>> standard library features is in nobody's interest.  The result will be
>> an irrelevant standard library and a fragmented development community.
> 
> Yeah— that was such a huge roadblock to the popularity of the C language.

The fact that people often heroically manage to make nearly impossible 
situations livable is not a reason for creating such situations on purpose.

> A vibrant community of people outside INRIA developing independent [even 
> mutually-incompatible] extensions to the distribution libraries is in 
> *everybody's* interest.

Are you joking?  Have you ever tried to write a program using several 
mutually incompatible libraries?

Before C++ had STL, everyone wrote their own string class.  Surely you 
can imagine the resulting contortions when libraries with different 
string classes had to be used together.  If an application programmer 
constantly has to translate between different libraries' conceptions of 
string, I/O channel or Internet address, the result is not simply 
endless annoyance (it is really a pain to write and use that sort of 
glue code constantly); more importantly, it makes applications 
unreadable and unmaintainable.

I work in a domain where integration of many applications and libraries 
is essential.  With all its faults, at least Java has standard String, 
InputStream and InetAddress classes.  Life without these things would be 
a nightmare.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Sun, 2004-04-25 at 21:50, Benjamin Geer wrote:
> Are you joking?  Have you ever tried to write a program using several 
> mutually incompatible libraries?
> 
> Before C++ had STL, everyone wrote their own string class.  Surely you 
> can imagine the resulting contortions when libraries with different 
> string classes had to be used together.  

I was an active member of the C++ committee at that time.

There was a set of classes in the Working Draft. 
A string class, IOstreams, some other things.
All independently designed and voted on.

It was a mess. A Standard string class. Better than no 
Standard. Then Alex Stepanov gave a brief talk on a library
being developing at HP, based on an older Ada library.

The ideas were new (to the C++ committee). But there was
a coherent design, and stated design *principles* based
on mathematics, built by researchers.

Almost the *whole* Standard C++ library was overturned
at the next meeting in a single vote in favour of STL,
and what wasn't thrown out completely was modified to
conform to the container-iterator-algorithm framework.

Changes in the C++ language itself were then almost
exclusively driven by the requirements of STL,
partial specialisations in particular.

The committee actually did a very good job firming
up the details, discarding non-essential components,
and writing detailed specifications -- although
they made a few very serious mistakes, developing
allocators being the worst disaster (they should have
been thrown out completely).

The result is, in my opinion, the best CS library
EVER built. Its just a pity the C++ language doesn't
have what it takes to drive it (lexical scoping 
like ML has).

My conclusion from this process is: a loose community
is no more capable of designing a coherent library
than a high level programming language. A small high
power research team is needed for that.

But a small research team is just as incapable of
industrialising a library: filling out the details,
following the principles layed down, enriching
and refining, testing and debating. That requires
a larger community.

A brief note on the theory behind STL: we don't 
understand it. I have an inkling. STL does two things
that you can't do in Ocaml.

(1) It provides functorial polymorphism.
Iterators generalise over data structures
themselves, not merely their value type.

(2) Iterators provide control inversion.
Compare the functional 'iter' and 'map'
like HOF's of Ocaml, where the user 
supplies a callback, with the control inverted 
algorithms of STL, which use iterators to 
demand the data.

I'm not criticizing Ocaml or it's library here,
rather I'm making a point about process.
The Ocaml library is a loose collection
of things, like the C++ library before STL came along.
Turning the Ocaml library over to the community before
the reseachers have a coherent design will be a disaster.

Holding onto it afterwards will be too.

IMHO: the Ocaml team is doing the right thing.
The Standard library is being kept small, being
upgraded slowly, whilst the main emphasis is
where it needs to be to support a new, coherent,
library design.

The type system.

I think it's time to say thanks for the good work!
Keep it up! .. and to ask: how can we help better?

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Martin Berger]

> The result is, in my opinion, the best CS library
> EVER built. Its just a pity the C++ language doesn't
> have what it takes to drive it (lexical scoping 
> like ML has).

i agree with this. what i wonder is: why not do the STL for
ocaml? of course ocaml's typing system is not (yet) up
expressive enough to express/enforce all relevant concepts,
and may never be, but so what? neither is c++, but the
STL is highly successful.

it should not be too hard to come up with an ocaml-like syntax
(an extension of ocaml) to specify STL-like data structures and
algorithms and compile them down to ocaml. i think that would be
very useful.

what do you think?

martin

maybe this text is of interest:

http://www.osl.iu.edu/publications/pubs/2003/comparing_generic_programming03.pdf

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-04-26 at 22:08, Martin Berger wrote:
> > The result is, in my opinion, the best CS library
> > EVER built. Its just a pity the C++ language doesn't
> > have what it takes to drive it (lexical scoping 
> > like ML has).
> 
> i agree with this. what i wonder is: why not do the STL for
> ocaml? of course ocaml's typing system is not (yet) up
> expressive enough to express/enforce all relevant concepts,
> and may never be, but so what? neither is c++, but the
> STL is highly successful.

ExtLib (the sf one at ocaml-lib.sf.net) does have some
STL like concepts. See below for more comments ..

> maybe this text is of interest:
> 
> http://www.osl.iu.edu/publications/pubs/2003/comparing_generic_programming03.pdf

Yeah, I've read that. There is some interesting
analysis, but the basis is flawed. The authors approached
the subject with a pre-conceived idea, that happens
to be invalid. 

They're basically hooked on 'generic' meaning
what templates do: bind in a macro like manner
by name. This isn't proper polymorphism.

It *is* what provides the functorially polymorphic
behaviour. Aka 'polyadic'. But there are better ways.

Err .. guests arriving .. more later :D

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

	let rec print printer deref next x = 
		printer (deref x);
		print printer deref next (next x)
	;;

	let pi x = print_endline (string_of_int x) in
	print pi List.hd List.tl [1;2;3;4;5]
	;;


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Brian Hurt]

On Mon, 26 Apr 2004, Martin Berger wrote:

> 
> > The result is, in my opinion, the best CS library
> > EVER built. Its just a pity the C++ language doesn't
> > have what it takes to drive it (lexical scoping 
> > like ML has).
> 
> i agree with this. what i wonder is: why not do the STL for
> ocaml? of course ocaml's typing system is not (yet) up
> expressive enough to express/enforce all relevant concepts,
> and may never be, but so what? neither is c++, but the
> STL is highly successful.

I'm wondering what concepts Ocaml can't express/enforce?  The only thing I 
can think of that you can do in C++ templates that I can't do in Ocaml is 
code generation- I have to use pcaml for that :-).

Well, one caveat- Ocaml doesn't allow for operator overloading.  But it 
*does* allow for defining new operators.  

It's been too long since I've used the STL- what data structures and 
algorithms does it provide?  I'm betting that Ocaml, or Ocaml + ExtLib, 
already provides most if not all of them.

Maybe we should take this to the extlib mailing list...

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Wednesday 28 April 2004 5:31 am, Brian Hurt wrote:
> It's been too long since I've used the STL- what data structures and
> algorithms does it provide?

The main thing is that it is iterator-centric, so you pass iterators around 
instead of containers. For example, to represent a subarray without having to 
copy it. These iterators are classified according to their abilities (e.g. 
trivial, forward, random-access etc.). I think these classifications are 
theoretical - a C++ compiler can't enforce them so if a program misuses an 
iterator then trying to compile it will often produce 10 pages of 
gobbledygook errors pointing into the standard library.

This is quite useful for 1D containers, which the STL provides (list, slist, 
vector, deque), but it doesn't lend itself well to many useful containers 
(trees, matrices, higher-dimensional arrays) which exhibit more interesting 
connectivities than a bidirectional iterator can represent and which do not 
exhibit a clear notion of "sub".

So you get things like an STL set, implemented using red-black trees, which is 
actually an ordered set and which lets you insert and delete elements, 
iterate through them, all the usual stuff. They chose to use semi-inclusive 
bound (e.g. "[a,b)") and, consequently, you typically had an "end" iterator 
which pointed beyond the end of the container (equivalent to the NULL pointer 
in a C list, in raw terms).

I often found myself using a "typedef" to give my current favourite STL 
container a new name which a bunch of my code could then use. When I felt 
that another container type was trendy enough to switch to, I'd just change 
the "typedef" line. That's actually quite useful for trying out different 
data structures for performance etc.

The STL also provided data structures (e.g. valarray, complex) which were 
intended to be used for high-performance numerics. But this was horribly 
misguided as, basically, you can't do decent optimisation on those without 
them being built-in types. The Blitz++ library went some way to addressing 
this by (ab)using the (Turing complete!) template type system to perform 
partial specialisations and AST optimisations on expressions. But then 
everyone realised that was just too nasty, and it died a death.

I'm not saying that these things can't be done in ocaml, just that you can do 
them easily in C++ using the STL. I'd be very interested to hear ocaml 
equivalents though! If you want to know more about the STL then I suggest you 
refer back to Stepanov's ramblings, rather than looking at the "standard" 
which was, unfortunately, bastardised by a committee...

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 15:13, Jon Harrop wrote:
> On Wednesday 28 April 2004 5:31 am, Brian Hurt wrote:
> > It's been too long since I've used the STL- what data structures and
> > algorithms does it provide?
> 
> The main thing is that it is iterator-centric, 

> This is quite useful for 1D containers, which the STL provides (list, slist, 
> vector, deque), but it doesn't lend itself well to many useful containers 
> (trees, matrices, higher-dimensional arrays) which exhibit more interesting 
> connectivities than a bidirectional iterator can represent and which do not 
> exhibit a clear notion of "sub".

That isn't true. Computers can only do things in sequence
(well, ignoring fledgling parallelism). 

Using iterators, your higher dimensional problem is factored
into two halves: the sequential algorithm, and the sequence
generator.

There aint no other way (short of parallelism). 
Iterators simply enforce a control inversion boundary:
the sequence generator is callback driven, the algorithm
its driver: iterators mediate the control relation.

Exactly how you define your iterator is up to you,
but clearly the specification factors your problem
as I described.

Obvious examples include say 'depth first traversal'
of a tree. However you process a tree, it can be
done with an iterator.

Perhaps it is necessary to generalise the concept of
'next': in STL you can +/- int for random iterators,
but there is no reason next can't be generalised to
take an arbitrary motion indicator.

Clearly that reflects on the general structure of the
container, so there are higher classes of iterators
than just forward and random. The key concept is
a 'current location' and 'movement instruction'.

Sounds like Logo got it right with Turtle graphics :D

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Wednesday 28 April 2004 9:37 am, skaller wrote:
> On Wed, 2004-04-28 at 15:13, Jon Harrop wrote:
> > ...
> > more interesting connectivities than a bidirectional iterator can
> > represent and which do not exhibit a clear notion of "sub".
>
> That isn't true. Computers can only do things in sequence
> (well, ignoring fledgling parallelism).
[snip]

I think you've taken the STL concept of an iterator and generalised it so much 
that it now covers all computation. :-)

This notion of an "iterator" would have to be specialised to a specific 
sequence-generating algorithm. That algorithm could be arbitrarily 
complicated. Sounds like a higher-order function to me. What aspect of this 
can ocaml not express/enforce?

In fact, it's just a thought, but if nobody has implemented the hoisting of 
array bounds checking in ocamlopt yet, could this be circumvented by 
implementing "iterator" pairs which can be "moved" such that the range of 
elements that they represent is always valid. Then you could have iter, map 
etc. working on the range, knowing that they could use unsafe_get because it 
was guaranteed to be valid?

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 19:18, Jon Harrop wrote:
> On Wednesday 28 April 2004 9:37 am, skaller wrote:
> > On Wed, 2004-04-28 at 15:13, Jon Harrop wrote:
> > > ...
> > > more interesting connectivities than a bidirectional iterator can
> > > represent and which do not exhibit a clear notion of "sub".
> >
> > That isn't true. Computers can only do things in sequence
> > (well, ignoring fledgling parallelism).
> [snip]
> 
> I think you've taken the STL concept of an iterator and generalised it so much 
> that it now covers all computation. :-)

All computation by sequential machines.

> This notion of an "iterator" would have to be specialised to a specific 
> sequence-generating algorithm. That algorithm could be arbitrarily 
> complicated. Sounds like a higher-order function to me. What aspect of this 
> can ocaml not express/enforce?

Control inversion. In Ocaml you can write:

List.iter (fun x -> print_endline x) ["Hello"; "World"];;

but this isn't an iterator solution because the client
algorithm is a callback. Callbacks suck :)

Using an iterator we write:

while ( p != e) do print_endline (get p); incr p done;

This is quite different because the client algorithm
is *reading* the data, not being called with it.

The List.iter style HOF is very limited in utility
in a functional language: there is no state.

In Ocaml you can add state of course. But it's a mess.
The saving grace of the HOF is that for simple problems,
the termination is assured, and, the statelessness
of the callback is actually an important property 
for reasoning about semantics.

The control inverted solution is clearly much better
for complex problems because it allow the stack
to maintain some state.

to say another way: an iterator IS just a control inverted
HOF, indeed, the (abstraction of) the List.iter HOF.

A third view: all computation is an engineering combination
of the List.iter kind of iteration and the 'iterators'
kind of iteration. Something about 'bisimulation' and
inductive and coinductive types and categorical duality
principle is here but I don't know exactly what. 

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Wed, Apr 28, 2004 at 09:24:43PM +1000, skaller wrote:
> On Wed, 2004-04-28 at 19:18, Jon Harrop wrote:
> Control inversion. In Ocaml you can write:
> 
> List.iter (fun x -> print_endline x) ["Hello"; "World"];;

[ snip ]

> while ( p != e) do print_endline (get p); incr p done;
 [ snip ]
> 
> The List.iter style HOF is very limited in utility
> in a functional language: there is no state.

> In Ocaml you can add state of course. But it's a mess.

I don't find it all that problematic.  For one, you can just maintain
the state in the scope above the callback.  The callback has access to
all those variables, of course.  And you could always just pass whatever
object or record you like in to the callback.  Neither is all that
complex to me.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 01:18, John Goerzen wrote:
> On Wed, Apr 28, 2004 at 09:24:43PM +1000, skaller wrote:
> > On Wed, 2004-04-28 at 19:18, Jon Harrop wrote:
> > Control inversion. In Ocaml you can write:
> > 
> > List.iter (fun x -> print_endline x) ["Hello"; "World"];;
> 
> [ snip ]
> 
> > while ( p != e) do print_endline (get p); incr p done;
>  [ snip ]
> > 
> > The List.iter style HOF is very limited in utility
> > in a functional language: there is no state.
> 
> > In Ocaml you can add state of course. But it's a mess.
> 
> I don't find it all that problematic.  

Compare:

let count = ref 0 in
List.iter
(fun x -> 
  if !start = 0 then 
  (start := 1; print "["; print x; )
  else (print ";"; print x)
)
lst; 
if (!start <> 0) then print "]"

and please don't tell me that isn't a total mess compared with

proc print_list(){
  read x;
  if x == None goto empty;
  print "["
next:>
  print x;
  read x;
  if x == None goto endoff;
  print ";"; 
  goto next:
endoff:> 
  print "]"
empty:
}

Notice there are NO FLAGS. The entire state
is represented by the program counter.
In a more complex problem, the entire
state can be local. You can even write
a recursive procedure .. that builds a list  ..
or even a tree .. do you want modify 
the external state like that, building
a data structure to *emulate* a stack
when you can actually have a real one?

By the way this is valid, executable Felix.
And of course it works behind the scenes
by emulating a stack :D

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Thu, Apr 29, 2004 at 02:28:35AM +1000, skaller wrote:
> On Thu, 2004-04-29 at 01:18, John Goerzen wrote:
> > > In Ocaml you can add state of course. But it's a mess.
> > 
> > I don't find it all that problematic.  
> 
> Compare:
> 
> let count = ref 0 in
> List.iter
> (fun x -> 
>   if !start = 0 then 
>   (start := 1; print "["; print x; )
>   else (print ";"; print x)
> )
> lst; 
> if (!start <> 0) then print "]"
> 
> and please don't tell me that isn't a total mess compared with

Of course, your example ignores nicer features available for
this sort of thing.  For instance:

let print_list l =
  let printfunc count item =
    print_string (if count = 0 then "[" else ";");
    print_string item; count + 1 in
  if (List.fold_left printfunc 0 l) > 0 then print_string "]";;

Actually, I think that's more readable than your iterative example
below.

-- John

> proc print_list(){
>   read x;
>   if x == None goto empty;
>   print "["
> next:>
>   print x;
>   read x;
>   if x == None goto endoff;
>   print ";"; 
>   goto next:
> endoff:> 
>   print "]"
> empty:
> }
> 
> Notice there are NO FLAGS. The entire state
> is represented by the program counter.
> In a more complex problem, the entire
> state can be local. You can even write
> a recursive procedure .. that builds a list  ..
> or even a tree .. do you want modify 
> the external state like that, building
> a data structure to *emulate* a stack
> when you can actually have a real one?
> 
> By the way this is valid, executable Felix.
> And of course it works behind the scenes
> by emulating a stack :D
> 
> -- 
> John Skaller, mailto:skaller@users.sf.net
> voice: 061-2-9660-0850, 
> snail: PO BOX 401 Glebe NSW 2037 Australia
> Checkout the Felix programming language http://felix.sf.net
> 
> 
> 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 04:02, John Goerzen wrote:
> On Thu, Apr 29, 2004 at 02:28:35AM +1000, skaller wrote:
> > On Thu, 2004-04-29 at 01:18, John Goerzen wrote:
> > > > In Ocaml you can add state of course. But it's a mess.
> > > 
> > > I don't find it all that problematic.  
> > 
> > Compare:
> > 
> > let count = ref 0 in
> > List.iter
> > (fun x -> 
> >   if !start = 0 then 
> >   (start := 1; print "["; print x; )
> >   else (print ";"; print x)
> > )
> > lst; 
> > if (!start <> 0) then print "]"
> > 
> > and please don't tell me that isn't a total mess compared with
> 
> Of course, your example ignores nicer features available for
> this sort of thing. 

Indeed. If I may be so bold to observe .. you've control
inverted. You're now driving the iteration. You've
got rid of the HOF List.iter and implemented your own
special one.

The point of the code I displayed was that it
control inverts *without* getting rid of the 
driver loop.

Also .. a small bug in your translation: it prints "[]"
for an empty list. The original prints "".

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Andreas Rossberg]

skaller wrote:
> 
> Indeed. If I may be so bold to observe .. you've control
> inverted. You're now driving the iteration. You've
> got rid of the HOF List.iter and implemented your own
> special one.

No, he has not. He is using fold to drive the iteration, which is just a 
more general variant of iter, that allows (1) threading some additional 
state and (2) incrementally constructing a result, although John's code 
is only making use of (1).

> The point of the code I displayed was that it
> control inverts *without* getting rid of the 
> driver loop.

I'm not sure I understand.

> Also .. a small bug in your translation: it prints "[]"
> for an empty list. The original prints "".

No, John's version does the same.

Cheers,

   - Andreas

-- 
Andreas Rossberg, rossberg@ps.uni-sb.de

Let's get rid of those possible thingies!  -- TB

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Thu, Apr 29, 2004 at 10:54:09AM +1000, skaller wrote:
> > > if (!start <> 0) then print "]"
> > > 
> > > and please don't tell me that isn't a total mess compared with
> > 
> > Of course, your example ignores nicer features available for
> > this sort of thing. 
> 
> Indeed. If I may be so bold to observe .. you've control
> inverted. You're now driving the iteration. You've
> got rid of the HOF List.iter and implemented your own
> special one.

Isn't fold_left an iterator too, just one that works with slightly
different semantics?

Of course, your example could be simplified to:

let print_list l =
  let count = ref 0 in
  let printfunc item = 
    print_string (if !count = 0 then "[" else ";");
    print_string item; count := !count + 1 in
  List.iter printfunc l;
  if !count > 0 then print_string "]";;

> The point of the code I displayed was that it
> control inverts *without* getting rid of the 
> driver loop.
> 
> Also .. a small bug in your translation: it prints "[]"
> for an empty list. The original prints "".

No.  Again, my code was:

let print_list l =
  let printfunc count item =
    print_string (if count = 0 then "[" else ";");
    print_string item; count + 1 in
  if (List.fold_left printfunc 0 l) > 0 then print_string "]";;

Try it.  If the list is empty, printfunc is never called since there is
nothing to iterate over.  The return value from fold_left is zero since
the count was never incremented, so the if statement evalutes to false
and "]" is never called.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Wednesday 28 April 2004 5:28 pm, skaller wrote:
[snip]
> let count = ref 0 in
> List.iter
> (fun x ->
>   if !start = 0 then
>   (start := 1; print "["; print x; )
>   else (print ";"; print x)
> )
> lst;
> if (!start <> 0) then print "]"

You mean:

let print_list string_of l = print_string ("["^(String.concat "; " (List.map 
string_of l))^"]")

Personally I'd factor out the making of and printing of the string. If you 
want a more "direct" conversion then just write a "list_mapi" function and 
test "i" each time.

> and please don't tell me that isn't a total mess compared with
> ...

Oh my God, you're not seriously advocating a page of gotos over that are you?

> Notice there are NO FLAGS. The entire state
> is represented by the program counter.
> In a more complex problem, the entire
> state can be local. You can even write
> a recursive procedure .. that builds a list  ..
> or even a tree .. do you want modify
> the external state like that, building
> a data structure to *emulate* a stack
> when you can actually have a real one?

I think you can easily have it either way in ocaml too. For the imperative 
style you can use map or iter with a function which has side-effects on some 
more global state. For the functional approach you use fold and pass the 
state on each time.

For example, I wrote this piece of code yesterday:

let rec eval bindings t = match t with
...
| Sequence l ->
    let f (bindings, l) t =
      let (bindings, t) = eval bindings t in (bindings, t::l) in
    let (bindings, l) = List.fold_left f (bindings, []) l in
    (bindings, Sequence (List.rev l))
| ...

It loops through a sequence of commands, executing each one in turn which may 
affect the current state "bindings" and the new state is passed on to the 
next command.

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 04:42, Jon Harrop wrote:

> Oh my God, you're not seriously advocating a page of gotos over that are you?

Certainly: its a state machine, that's how the work,
you can't get around that.

However the goto's aren't of consequence here.
I you wanted you could use a for loop, or
whatever .. 

> I think you can easily have it either way in ocaml too. For the imperative 
> style you can use map or iter with a function which has side-effects on some 
> more global state. For the functional approach you use fold and pass the 
> state on each time.

You are missing the point. Neither solution control inverts.
So they're both weak. In the control inverted solution
you aren't not called with some state, you do the calling
so you can use for example let/bindings and recursion
to maintain state "functionally": you cannot do that
with the callback driven model, you're forced to use
mutable state.

In practice not all complex problems admit
a clean solution using functional states,
you may need some variables too. But at least
you can reduce the need to use it if you have
control inversion built into your language.

Sure like to see a monadic/lazy solution to
the example using streams ..


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Thursday 29 April 2004 2:03 am, skaller wrote:
> You are missing the point.

I sure am. ;-)

> Neither solution control inverts.

Does that mean that my function is being called by map, rather than having my 
function explicitly fetch the next element?

So, is this "control inverted":

let print string_of l =	
  let rec helper l = match l with
    [] -> ""
  | h::[] -> "; "^(string_of h)^"]"
  | h::t -> "; "^(string_of h)^(helper t) in
  print_string (match l with
    [] -> ""
  | h::t -> "["^(string_of h)^(helper t))

> So they're both weak.

By this, do you mean that "control inverted" functions are more generic?

If the above function is control inverted, in what way is it better than my 
original?

> In the control inverted solution
> you aren't not called with some state,

Is that the same as "you are called with some state"?

> you do the calling
> so you can use for example let/bindings and recursion
> to maintain state "functionally": you cannot do that
> with the callback driven model, you're forced to use
> mutable state.

Can you give me a simple example where you are forced to use mutable state?

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 11:56, Jon Harrop wrote:
> On Thursday 29 April 2004 2:03 am, skaller wrote:
> > You are missing the point.
> 
> I sure am. ;-)
> 
> > Neither solution control inverts.
> 
> Does that mean that my function is being called by map, rather than having my 
> function explicitly fetch the next element?

Yes.

> > So they're both weak.
> 
> By this, do you mean that "control inverted" functions are more generic?

The control inverted (thread-of-control) solution allows
to use the stack in the traditional way. For example
recursion.  So it can solve at least context free
problems.

A callback with finite state object is clearly limited
to finite state machine: can only solve regular
problems.

Of course you can build more complex state object in both
cases to generalise. The point of the algorithmic
solution is that at least some of that state
can be expressed naturally with lexical scoping
and control flow.

I suggest you consider some C program which processes
files .. and rewrite it now please to be a callback

	void accept_character(char x)
	{
	}

and you will see what I mean. This is untenable.

What does the OS do for you? It control inverts.
Most OS swap stacks to do this. This is also
one reason to use threads, unrelated to asynchronicity.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 11:56, Jon Harrop wrote:

> 
> Can you give me a simple example where you are forced to use mutable state?

Implement List.rev by defining "revit" below:

List.iter revit lst

Note: you have also implemented something more.
You have also implemented: Array.to_rev_list

Array.iter revit arr

Your revit function is polyadic with respect
to the input data structure, the XXXX.iter
function is used to factor that out.

This property is also shared by the
'read/pull/control inverted/thread-like/
continuation passing' solution.

In fact, the two solutions are dual.
They do the same job.

For "full inversion" you need to also
invert the HOF. For example here is
the traditional master List.iter:

let iter f = function 
| h::t -> f h; iter f t 
| [] ->()

This is the desirable form. This is an algorithm
that writes its output: it is the caller.

The inverse is a supplier of list elements:

let make_supplier lst =
  let lst = ref lst in
  let get () = function
  | h :: t -> lst :=t; h
  | [] -> raise Not_found
in get

This is hard to write but allows the client
routine to get() elements.

Traditionally, algorithms had to be factored
so one of the cooperating pair of routines
had to be unnatural: one had to be master,
and the other slave.

This sux. You want BOTH to be masters.
You can clearly do that with two threads and
a monitor object (more precisely, coroutines).

Felix control inverts arbitrary procedural code
mechanically: that is, you write code that reads
data and it is translated into a callback.

Wouldn't you rather program an *active* agent
for your complex GUI widgets that a passive callback
driven one?

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Thursday 29 April 2004 4:00 am, skaller wrote:
> On Thu, 2004-04-29 at 11:56, Jon Harrop wrote:
> > Can you give me a simple example where you are forced to use mutable
> > state?
> ...
> The inverse is a supplier of list elements:
>
> let make_supplier lst =
>   let lst = ref lst in
>   let get () = function
>
>   | h :: t -> lst :=t; h
>   | [] -> raise Not_found

But can't this be implemented without mutable state as:

let make_supplier lst =
  let rec get lst () = match lst with [] -> raise Not_found | h::t -> (h, fun 
() -> get t ()) in
  get lst

so it returns the next element and a function to "supply" the rest.

> ...
> let iter f = function
> | h::t -> f h; iter f t
> | [] ->()
>
> This is the desirable form.

Why is that desirable? To me, it looks as though this form is necessarily more 
verbose and, therefore, undesirable.

> Traditionally, algorithms had to be factored
> so one of the cooperating pair of routines
> had to be unnatural: one had to be master,
> and the other slave.

Is mutual recursion not an example of having two "master" functions.

> Felix control inverts arbitrary procedural code
> mechanically: that is, you write code that reads
> data and it is translated into a callback.

What is your favourite example of this code morphing being useful?

> Wouldn't you rather program an *active* agent
> for your complex GUI widgets that a passive callback
> driven one?

I don't know. I just brush my hands over the keyboard and programs come 
out. :-)

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 15:04, Jon Harrop wrote:
> On Thursday 29 April 2004 4:00 am, skaller wrote:
> > On Thu, 2004-04-29 at 11:56, Jon Harrop wrote:
> > > Can you give me a simple example where you are forced to use mutable
> > > state?
> > ...
> > The inverse is a supplier of list elements:
> >
> > let make_supplier lst =
> >   let lst = ref lst in
> >   let get () = function
> >
> >   | h :: t -> lst :=t; h
> >   | [] -> raise Not_found
> 
> But can't this be implemented without mutable state as:
> 
> let make_supplier lst =
>   let rec get lst () = match lst with [] -> raise Not_found | h::t -> (h, fun 
> () -> get t ()) in
>   get lst
> 
> so it returns the next element and a function to "supply" the rest.

CPS transform?


> I don't know. I just brush my hands over the keyboard and programs come 
> out. :-)

teach me your technique! When I do that, i just get bugs leaping out!

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[james woodyatt]

On 28 Apr 2004, at 18:03, skaller wrote:
>
> Sure like to see a monadic/lazy solution to
> the example using streams ..

Let's see... a contrived example... you have a sequence of strings, and 
you'd like to process it into a sequence of different strings with a 
state machine.  What would that look like using my Cf library?

Consider this demonstration:

     (*---
       This is a trivial demonstration of sequences, flows and monadic 
programming
       with the Pagoda Cf library.  It shows how to write a flow that 
reads a
       sequence of strings and writes the sequence of strings with 
duplicates elided,
       using a state-continuation monad, parameterized with the state of 
the set of
       strings already seen in the input.  The flow resulting from 
evaluating the monad
       is then used in a function that operates on lists of strings.
      ---*)
     open Cf_scmonad.Op

     module String_set = Cf_rbtree.Create(String)

     (* val memoize: (string, string) Cf_flow.t *)
     let memoize =
         let rec loop () =
             Cf_flow.readSC >>= fun s ->
             Cf_scmonad.load >>= fun u ->
             if String_set.member s u then
                 loop ()
             else
                 Cf_flow.writeSC s >>= fun () ->
                 let u = String_set.put s u in
                 Cf_scmonad.store u >>= fun () ->
                 loop ()
         in
         Cf_flow.evalSC (loop ()) String_set.null

     (* val uniq: string list -> string list *)
     let uniq s =
         let z = Cf_seq.of_list s in
         let z = Cf_flow.commute memoize z in
         Cf_seq.to_list z

     let test () =
         let s1 = [ "Hello"; "World!"; "Hello"; "AGAIN!" ] in
         let s2 = [ "Hello"; "World!"; "AGAIN!" ] in
         let s2' = uniq s1 in
         if s2 <> s2' then failwith "Error in uniq!"

     ;;test ()
     (*--- End of demonstration ---*)

This is a toy, and it will obviously not produce the most efficient 
program in the world.  To solve this particular problem, in actuality, 
I probably wouldn't use the monad.  I'd just write a quick loop on the 
sequence, and I'd use a mutable hash table for state.

However, the code above begins to illustrate how to build much more 
complicated state machines.  In a real case study, e.g. my 
implementation of RFC 3080, the "loop" function will often be a 
collection of recursive state-event handling functions.


-- 
j h woodyatt <jhw@wetware.com>
markets are only free to the people who own them.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Andreas Rossberg]

skaller wrote:
> 
> Sure like to see a monadic/lazy solution to
> the example using streams ..

Admittedly, I'm too lazy ( :-) ) to give an example right now, but the 
idea behind the lazy solution is very simple: for iteration, your 
collection provides a function that delivers a list of all items. You 
can then iterate over it as you would normally do for lists.

However, the important point is: the list is constructed lazily (in 
OCaml, by using the type I showed in my previous posting, or something 
similar). That is, only when you try to match the next item, i.e. the 
tail of the stream (in OCaml, by forcing it) it is actually computed. 
Thereby, the client takes control over the iteration.

Note that lazy functional languages can often perform optimizations like 
deforestation to avoid actual construction of the intermediate list.

Cheers,

   - Andreas

-- 
Andreas Rossberg, rossberg@ps.uni-sb.de

Let's get rid of those possible thingies!  -- TB

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[james woodyatt]

On 28 Apr 2004, at 04:24, skaller wrote:
>
> to say another way: an iterator IS just a control inverted
> HOF, indeed, the (abstraction of) the List.iter HOF.

Also not precisely true, since iterators are typically lazy evaluations 
of the underlying container.  It would be more precise to say that they 
are an abstraction of the FStream module in the CamlP4 distribution.  
(I have a similar module in my Cf library, called Cf_seq, which I think 
offers more functionality, so that's what I use.)  It's possible to do 
a whole helluva lot with lazily evaluated functional streams and 
monadic programming.

With Ocaml, monadic programming is a little more convoluted than with 
Haskell, but it's not at all out of the bounds of reality.  I do it all 
the time, and I don't miss the C++ STL (or Stepanov's original work) 
even a little bit.  In fact, I suspect there are things I would 
consider trivial to do with monads and functional streams that the C++ 
STL just falls down and dies when you try to torture it into allowing.


-- 
j h woodyatt <jhw@wetware.com>
markets are only free to the people who own them.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 03:07, james woodyatt wrote:
> On 28 Apr 2004, at 04:24, skaller wrote:
> >
> > to say another way: an iterator IS just a control inverted
> > HOF, indeed, the (abstraction of) the List.iter HOF.
> 
> Also not precisely true,

[]

> With Ocaml, monadic programming is a little more convoluted than with 
> Haskell, but it's not at all out of the bounds of reality.  I do it all 
> the time, 

If you find a simple enough example I'd love to see it.
I have quite a lot of 'multi-pass' phases in my Felix compiler
where i build temporary data structures in between.

I'd love to glue some of these together to eliminate building
whole data structures. For example I have phases:

	tokeniser -> rewrite token stream -> parser ->
	macro expansion and constant folding ->
	desugaring (syntactic term rewriting) ->
	lifting (separate declarations from executable code)

which can almost certainly be glued together, if only
I knew how (without rewriting the algorithms completely).


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Marcin 'Qrczak' Kowalczyk]

W liście z czw, 29-04-2004, godz. 03:31 +1000, skaller napisał:

> I have quite a lot of 'multi-pass' phases in my Felix compiler
> where i build temporary data structures in between.
> 
> I'd love to glue some of these together to eliminate building
> whole data structures.

I'm not sure it's worth the trouble. There are two issues of compiler
phases: organization and efficiency.

Concerning the organization, my opinion is that we do want to separate
it into several small phases, so each pass becomes easier to manage.
Trying to do too much at once sometimes requires to recompute the same
thing several times, because there is no convenient intermediate form
to store it for reuse, and it's especially bad if source and target
languages require very different code structure - then it's very hard
to get right at all.

And for the efficiency, today's computers are usually capable of fitting
two successive phases of the whole module in memory. GCC recently
switched to optimizing a module at a time rather than a function at a
time as previously. Also, lazy structures with some smart inversion of
control flow typically introduce an overhead of creating many closures
and calling many functions through pointers.

>  For example I have phases:
> 
> 	tokeniser -> rewrite token stream -> parser ->
> 	macro expansion and constant folding ->
> 	desugaring (syntactic term rewriting) ->
> 	lifting (separate declarations from executable code)

In my compiler of my language (implemented in itself) I have the
following phases. They came out of necessity - I wasn't able to manage
more complex transformations at once.

Code representation                | Compiler phase
-----------------------------------+-----------------------------------
Source string                      |
                                   | Lexer
List of tokens (strict)            |
                                   | Parser
Source syntax tree                 |
                                   | Expander which translates syntactic
                                   | sugar and resolves names; macros
                                   | will be here when implemented;
                                   | an interface file is written and
                                   | interfaces of used modules are read
More abstract tree, capturing the  |
essence of the semantics; smart    |
constructors compute the set of    |
free local variables of function   |
nodes                              |
                                   | Various optimizations will be here:
                                   | inlining, lambda-lifting of some
                                   | functions, type analysis (the
                                   | language is dynamically typed);
                                   | this phase is not implemented yet
The same representation of abstract|
tree as before                     |
                                   | Planner which lifts functions and 
                                   | constant data to the toplevel
                                   | and determines the sequence of
                                   | operations to perform inside each
                                   | function
A set of global definitions,       |
including function bodies expressed|
as almost flat sequences of        |
statements (modulo conditionals)   |
                                   | C coder which splits functions into
                                   | fragments between calls (needed
                                   | because of tail calls), allocates
                                   | virtual registers, stack frame
                                   | slots & temporary C variables,
                                   | determines where the stack frame
                                   | is active and generates C code
C abstract syntax tree             |
                                   | Pretty printer for a subset of
                                   | C syntax
The C output string                |

Phases are executed strictly in order, with no overlap. Each
representation of code uses different types with different structure,
except that information about a name (11 fields) is shared between
the "abstract" and "sequential" representation, and various atomic
data are shared between many phases (e.g. literal values, source
locations).

The slowest part is currently the pretty printer.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-05-03 at 10:02, Marcin 'Qrczak' Kowalczyk wrote:
> W liście z czw, 29-04-2004, godz. 03:31 +1000, skaller napisał:
> 
> > I have quite a lot of 'multi-pass' phases in my Felix compiler
> > where i build temporary data structures in between.
> > 
> > I'd love to glue some of these together to eliminate building
> > whole data structures.
> 
> I'm not sure it's worth the trouble.

Neither am I, even given an easy way to do it.

>  There are two issues of compiler
> phases: organization and efficiency.
> 
> Concerning the organization, my opinion is that we do want to separate
> it into several small phases, so each pass becomes easier to manage.

How small though?

> Trying to do too much at once sometimes requires to recompute the same
> thing several times, because there is no convenient intermediate form
> to store it for reuse,

Oh, this is a perfect description of my horrible lookup algorithm!

Originally, it was decoupled, but it didn't work.
I've had to glue the whole thing together in a single operation
because everything is so mutually recursive.

For example: typing a function return value requires
binding all the return statement expressions, which
requires lookup, which requires binding all the
function names in the expression, which requires
overloading which requires binding the signatures
of all the candidate functions.. one of which could
be the original function you're trying to find
the type of, and if it's that one you need to find
its return types .. woops! That was the original problem :)

i would like to augment the system so calls of the form:

	f x y z

allow overloading on all arguments: at present you can
only overload on the first one. Seems easy .. just
extended application of the unification algorithm.
Only it requires determining the return type of f,
and not just the parameter type. The parameter type
must be specified .. the return type does not.

So I'd have to recursively calculate the return type.
which is already problematic (see above) .. which might
create yet another cycle if doing that called the overload
resolution routine again with the same arguments ..

I'm scared to even try it [There are 4 distinct entry points
to the algorithm, and 4 distinct 'exclusion' lists marking
them to prevent infinite recursion .. its never clear to me 
which list to reset and which one to push a new fixpoint onto,
nor is it clear what to do when a cycle is detected..

>  Also, lazy structures with some smart inversion of
> control flow typically introduce an overhead of creating many closures
> and calling many functions through pointers.

At one stage I rewrote the 'token filter' part of Vyper using
Ocaml streams. About 3 times slower I think.

>                                    | C coder which splits functions into
>                                    | fragments between calls (needed
>                                    | because of tail calls), 

Interested in how you handle tail calls in C.
In theory this is easy, using goto. Unfortunately
it requires generating a single massive function,
and gcc is too much of a toy compiler to really trust with that.

> The slowest part is currently the pretty printer.

The slowest part of Felix is the lookup algorithm
because of the really heavy (exponential time) recursion
and lack of proper memoisation.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Marcin 'Qrczak' Kowalczyk]

Hoping that they will not kill us for off-topic...

> > Concerning the organization, my opinion is that we do want to separate
> > it into several small phases, so each pass becomes easier to manage.
> 
> How small though?

Of course extremes are bad; not *too* small.

Actions which are mutually recursive should be done in one phase, unless
the language requires very deep phase loops.

(The primary example is C++ where evaluation of constant expressions
can influence parsing of further parts, with name resolution and type
checking and template instantiation sitting between so they are also all
mutually recursive. I have no idea how to sanely organize a C++
compiler.)

> > Trying to do too much at once sometimes requires to recompute the same
> > thing several times, because there is no convenient intermediate form
> > to store it for reuse,
> 
> Oh, this is a perfect description of my horrible lookup algorithm!
> 
> Originally, it was decoupled, but it didn't work.
> I've had to glue the whole thing together in a single operation
> because everything is so mutually recursive.

Hmm. This probably depends on the language. Mine doesn't have anything
which requires far "phase lookahead". It's dynamically typed and there
is no compile-time overloading, only dynamic dispatch.

The only troublesome part wrt. the order was possible recursion among
definitions in a block. All definitions are potentially mutually
recursive, and using a name before its definition has been executed, in
any other way than attaching it to a closure, is an error, by necessity
sometimes detected only at runtime.

I needed to know which names will be introduced before processing
the contents of the definitions. This was done in the same phase as
understanding arguments of "builtin macros", which were all parsed like
normal applications. So I needed to recognize macro names and analyze
their arguments twice: first to predict which names will be introduced,
then to compile the definitions.

The duplication was getting ugly. I considered moving the recognition
of builtin macros to a separate phase, done before name resolution,
but this would make user-defined macros impossible in future, because
recognition of what is a macro should be done after name resolution.

So my solution was to analyze each sequence of definitions in two
mini-phases, with an explicit representation between them. The first
mini-phase checks the syntax of arguments of builtin macros, creates
objects representing properties of introduced names, updates the
environment which maps source names to these name objects, and outputs
the sequence of definitions represented in different types. Builtin
macros have separate kinds of nodes and bound names already use name
objects, but all subexpressions are still unanalyzed, in their source
form. The second mini-phase doesn't change the environment and can
finally descend into expressions.

> Interested in how you handle tail calls in C.

The portable variant uses the well-known trampoline style, where each C
function returns a pointer to the next function to jump into. The stack
is managed explicitly. But for x86 I process the assembler output of gcc
and convert code which returns an address (marked with a comment in asm)
with a jump. This increased performance by about 30%.

If the compiler is ever used on other architectures, someone who knows
other assemblers might extend the mangler to handle them. I only know
the x86 assembly.

This idea was not mine, GHC does a similar thing. But its mangler does
many horrible things, like putting some data structures adjacent to code
so they can be found at a known negative offset from the code pointer,
and removing prologues and epilogues. Mine does less things, it only
converts ret (and possibly loading of a return value) to a jump.

It does require some work, for example gcc before version 3.4.0 liked
to merge code of multiple exit points from a function. Often merged were
code paths after loading different results. So I have to duplicate these
code paths, so that each branch can have its own return value at the end
(turning "movl $constant_label,%eax" and "ret" into "jmp constant_label"
is particularly good for the CPU, but sometimes the same "ret"
corresponds to multiple labels from merged exit paths). It happens that
gcc-3.4.0 doesn't merge so much.

The mangler is 94 lines of Perl (not counting comments and empty lines).

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-05-03 at 18:58, Marcin 'Qrczak' Kowalczyk wrote:

> (The primary example is C++ where evaluation of constant expressions
> can influence parsing of further parts, with name resolution and type
> checking and template instantiation sitting between so they are also all
> mutually recursive. I have no idea how to sanely organize a C++
> compiler.)

Neither did any of the C++ compiler writers for at least 15 years.

> Hmm. This probably depends on the language. Mine doesn't have anything
> which requires far "phase lookahead". 

Felix is statically typed, everything is recursive,
recursive types are supported, it has generics,
overloading by selecting the most specialised 
function (like C++). Whilst a function parameter
must be annotated with a type, the return type
doesn't need to be given (usually:). 

Also 'typeof(e)' is allowed.. including for
function parameter types ...

To solve this algebraically I think I'd need
meta-sums: the type of an overload 

	f: int -> 'a
	f: long -> 'b

would be something like:

	int of 'a | long of 'b

so that

	f x

would be typed:

	typematch typeof(x) with 
	| int 'a -> 'a
	| long 'b -> ;b


I actually support such type terms as well as lambda abstractions.
>From the std library:

  typedef fun dom(t:TYPE):TYPE = 
    typematch t with
    | ?a -> _ => a
    endmatch
  ;

Note that dom isn't a total function, and so this isn't
entirely parametric polymophism: the construction clearly
allows for partial specialisations .. :)

However the overload resolution doesn't use this stuff
(I'm not sure how to do all the reductions).

Anyhow there is a central 'bottleneck', lookup,
where all the nesting and recursion is unravelled.
Thereafter everything is flat and fully bound,
and much easier to work with..

.. data flow analysis will be a breeze by comparison :D

> The only troublesome part wrt. the order was possible recursion among
> definitions in a block. All definitions are potentially mutually
> recursive,

Same in Felix

>  and using a name before its definition has been executed, in
> any other way than attaching it to a closure, is an error, by necessity
> sometimes detected only at runtime.

In Felix this can't be a problem for functions, only for
variables. Unlike Ocaml, 

	fun f ...

declares a class, whilst

	let f x = ..

in Ocaml constructs a value. Ocaml's f is a closure immediately,
Felix's is just a C++ class. The class gets instantiated
on use, so direct calls are never a problem. Neither is intermodule
recursion. uninitialised variables are possible though: 
I just blame the programmer :D

> The duplication was getting ugly. 

I know the feeling, except in my case 'multiplication' would
be more precise than mere 'duplicaton' :)

> So my solution was to analyze each sequence of definitions in two
> mini-phases, with an explicit representation between them. 

Right. Hence your comment in a previous email: you've got
an intermediate representation (ugly) but it's localised
enough to be sensible in some way, for example linear.

> > Interested in how you handle tail calls in C.
> 
> The portable variant uses the well-known trampoline style, where each C
> function returns a pointer to the next function to jump into. The stack
> is managed explicitly. 

That's what Felix does for procedures (except I'm using heap
store for stack frames). I can now convert tail procedure calls
into gotos. But I do no such thing for functions, they just
run on the stack (even though their stack frames are heap allocated,
the return addresses are not).

> But for x86 I process the assembler output of gcc
> and convert code which returns an address (marked with a comment in asm)
> with a jump. This increased performance by about 30%.

Ah. Inline asm tricks  .. been thinking of that. 

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Marcin 'Qrczak' Kowalczyk]

W liście z pon, 03-05-2004, godz. 20:58 +1000, skaller napisał:

> Felix is statically typed, everything is recursive,
> recursive types are supported, it has generics,
> overloading by selecting the most specialised 
> function (like C++).

I left static typing of my language for possible future research,
because designing a good static type system is extremely hard.

Type systems from the functional side typically don't support open world
assumption wrt. the set of subtypes of a type. They can't express the OO
way of adding subtypes in various parts of the program, without a
centralized place where all such extensions are gathered and tied
together into one type, and with the ability to dynamically check
whether the given object has the given subtype of the declared type.
In other words, you can't emulate dynamic typing in them.

Well, SML and OCaml do have one extensible type, exn, but using it for
other purposes than exceptions feels like a hack. Trying to extend
functions which were previously defined for only some cases is worse -
you either make a reference to a function, incrementally enrich it,
and in effect check cases linearly, or do some non-portable tricks of
putting an exception constructor in a hash table. It's all ugly.

They poorly support generic programming in the sense of working
generically on pure data terms of various types. For example
implementing the natural equality and ordering for algebraic types in
both OCaml and Haskell relies on builtin compiler magic. OCaml has
builtin hashing and serialization, Haskell has builtin conversion to and
from text - but if you wanted to implement hashing and serialization in
Haskell, or textual printing and parsing of arbitrary values in OCaml,
you have do to it by hand for different types.

OCaml doesn't have Haskell-like type classes, polymorphic recursion nor
type variables instantiated with higher order kinds. Haskell doesn't
have parametrized modules, polymorphic variants nor an OO subsystem.
Neither language allows to share record field labels between types.
Neither language allows to implement these features without modifying
the compiler. And their implementations of type checking is so complex
that I'm afraid a few people in the world understand it enough to be
able to change it.

Type systems from the OO side do support extending the set of subtypes,
and thus can emulate dynamic typing. But they don't support the other
dimension of extensibility: adding an interface to an old type. They
only recently learned about parametric polymorphism, without which
they plainly sucked. (They = Java and C#; Eiffel did have parametric
polymorphism earlier, but its type system has major holes.) And they
lead to a verbose syntax because of lack of type inference. They used
to be simple, but parametric polymorphism adds a significant complexity.

Compared to that, dynamic typing is extremely easy and flexible. Yes,
it doesn't detect so many errors in compile time; I miss it when using
a dynamically typed language. And it leads to slightly slower code. But
it's so simple! Both conceptually and in the implementation.

I will try to design a static type system for my language in future.
Preferably such that after removing type declarations, the program will
work in the dynamically typed variant, which will not go away. The
language is lexically scoped including globals, it avoids type puns,
e.g. it distinguishes lists from tuples, conditionals require booleans,
the tail of a list must be a list, variables are immutable by default,
there are no uninitialized variables - so it should be quite close to
what a static type system expects, but I still think it would be very
hard to design a static type system which is not overly restrictive.

> > and using a name before its definition has been executed, in
> > any other way than attaching it to a closure, is an error, by necessity
> > sometimes detected only at runtime.
> 
> In Felix this can't be a problem for functions, only for
> variables. Unlike Ocaml, 
> 
> 	fun f ...
> 
> declares a class, whilst
> 
> 	let f x = ..
> 
> in Ocaml constructs a value.

I don't understand.

The problem in my language is that here:
   let f x {y + 1};
   let z = f 10;
   let y = z + 1;
there is no single place which should be detected as an error. Well,
the problem is solved: execution of f will cause a runtime excepion, and
the compiler reports statically only obvious cases where the use of a
variable is outside an expression set to be executed later.

> But I do no such thing for functions, they just run on the stack
> (even though their stack frames are heap allocated, the return
> addresses are not).

I don't like imposing arbitrary resource limits without a good reason.
The stack of my implementation is resized when needed, so there is no
reason to avoid deep recursion if it only replaced a deep stack with a
long heap-allocated list. For example Map on lists is implemented in the
traditional non-tail-recursive way, and it works for long lists too.

A small stack size would be a problem for threads.

Since arguments and the return address are passed in global C variables,
there is no need to allocate a stack frame in many short functions which
don't contain non-tail calls, and in others it's often allocated only in
those execution paths which need it. So the fact that allocation of a
stack frame performs an overflow check should not be a big worry.

I think there are two major performance problems (but just guessing):

1. Since the virtual registers are in global C variables, there is lot
   of moving data to and from them, even in simple functions. Putting
   some global variables in machine registers, possible in GCC, is
   problematic: on x86 there are so few registers that gcc sometimes
   fails to generate code when %esi or %edi are taken, and of course it
   generates worse code if fewer registers are available.

   I tried to put some variables in physical registers, the improvement
   was not big. Perhaps I will try again now, the code generator has
   evolved a bit since then.

2. Ordinary code uses dynamic dispatch a lot, e.g. for all arithmetic,
   and currently the only way to avoid that is to use inline C code
   (similar to inline asm in C). Even some type inference would not
   help much because of bignums (did I say that I hate arbitrary
   limitations? I can't just not have bignums!). Even if I implement
   inlining, not much could be inlined from a dispatched function.
   Performing the dispatch statically when possible would be very
   hard...

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> > Felix is statically typed, everything is recursive,
> > recursive types are supported, it has generics,
> > overloading by selecting the most specialised
> > function (like C++).
>
> I left static typing of my language for possible future research,
> because designing a good static type system is extremely hard.
>
> Type systems from the functional side typically don't support open world
> assumption wrt. the set of subtypes of a type. They can't express the OO
> way of adding subtypes in various parts of the program, without a
> centralized place where all such extensions are gathered and tied
> together into one type, and with the ability to dynamically check
> whether the given object has the given subtype of the declared type.
> In other words, you can't emulate dynamic typing in them.

You're true, decentralized OO subtyping is more flexible because types are
"opened" for further additions, while ocaml types are "closed" and fully
defined and centralized (that's not the case for polymorphic variants of
course).
I have been working recently on a OO static type system with global type
inference and structural subtyping. The typer works pretty well, although
not yet optimized. I found some interesting properties : redefining an
inherited method is possible, but actually safely breaks implicit subtyping
acquired from inheritence..

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[brogoff]

On Mon, 3 May 2004, Marcin 'Qrczak' Kowalczyk wrote:
> OCaml doesn't have Haskell-like type classes, polymorphic recursion nor

I think OCaml does support polymorphic recursion, in at least two nonideal
ways, first through explicit polymorphism of record fields/polymorphic
methods  and secondly through recursive modules. You disagree?

Other than that, I mostly agree (OK, I'd prefer a better solution than
implicit stack resizing, like holes or promises, or whatever, for non tail
rec list functions), but I'm not ready to ditch OCaml for a dynamically typed
language.

-- Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Marcin 'Qrczak' Kowalczyk]

W liście z pon, 03-05-2004, godz. 07:24 -0700, brogoff@speakeasy.net
napisał:

> I think OCaml does support polymorphic recursion, in at least two nonideal
> ways, first through explicit polymorphism of record fields/polymorphic
> methods  and secondly through recursive modules. You disagree?

Ok, now it does. It didn't have polymorphic fields in records and
recursive modules not so long ago. And it looks uglier than in Haskell,
where the only thing you need to do is to provide an explicit type
signature of the function.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-05-03 at 22:40, Marcin 'Qrczak' Kowalczyk wrote:

> > In Felix this can't be a problem for functions, only for
> > variables. Unlike Ocaml, 
> > 
> > 	fun f ...
> > 
> > declares a class, whilst
> > 
> > 	let f x = ..
> > 
> > in Ocaml constructs a value.
> 
> I don't understand.

In Felix,

	fun f(x:int) = { return x; }

declares a class:

	struct f {
		int apply(int x) { return x; }
	};

So *direct* calls are like:

	f().apply(1)

where the closure is created on the fly.
The Ocaml:

	let f x = x

defines f as a variable already containing
the closure .. so you can't call f until
that variable is actually initialised.

> The problem in my language is that here:
>    let f x {y + 1};
>    let z = f 10;
>    let y = z + 1;
> there is no single place which should be detected as an error.

Same in Felix:

include "std";
fun f (x:int):int= { return y + 1; }
val z = f 10;
val y = z + 1;

print z; endl; 
print y; endl;

[skaller@pelican] ~>flx xxx
1
2

Well, no run time error: y just happens to
be zero initially :)

> 2. Ordinary code uses dynamic dispatch a lot, e.g. for all arithmetic,
>    and currently the only way to avoid that is to use inline C code
>    (similar to inline asm in C). Even some type inference would not
>    help much because of bignums (did I say that I hate arbitrary
>    limitations? I can't just not have bignums!).

Felix uses gmp. However it has all the C int types as well.
Oh .. none of them are built in, not even bool.
You *have* to 'inline C' the lot.

Did I say I hate arbitrary limitations like needing
to have builtin types? <g>


>  Even if I implement
>    inlining, not much could be inlined from a dispatched function.
>    Performing the dispatch statically when possible would be very
>    hard...

You'd be surprised. When I wrote Vyper, I did a lot
of analysis to try to make things static.
For Python, that proved almost impossible.

With a little care you can probably 
do very much better with just a little type inference.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Marcin 'Qrczak' Kowalczyk]

W liście z wto, 04-05-2004, godz. 01:08 +1000, skaller napisał:

> Well, no run time error: y just happens to
> be zero initially :)

Ah. My language doesn't let the programmer observe the value of
a variable before proper initialization. Variables are immutable
by default. I don't like undetected errors which are possible to detect,
and I claim using a variable before initialization is most definitely an
error.

What are values with no "natural" default initialized to? For example
functions. In OCaml's syntax:

let f x = x + 1
let g x = x - 1
let h() = v 0
let v = if h() > 0 then g else f

> Felix uses gmp. However it has all the C int types as well.

I have one integer type on the language level, which is tagged in
the lowest bits or uses GMP if it's bigger. The difference of the
representation is fully transparent for the programmer, except in
inline C. There are C functions for converting between this hybrid
representation and various C types provided.

> Oh .. none of them are built in, not even bool.
> You *have* to 'inline C' the lot.

My bool type is defined in the language, without inline C :-)

> You'd be surprised. When I wrote Vyper, I did a lot
> of analysis to try to make things static.
> For Python, that proved almost impossible.
> 
> With a little care you can probably 
> do very much better with just a little type inference.

You mean that Python's specific features make it hard? I imagine the
main problem is that in Python almost everything is mutable, including
global function bindings.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: Tail-calls in C code (was: [ANN] The Missing Library)

[Wolfgang Lux]

Marcin 'Qrczak' Kowalczyk wrote:

> The portable variant uses the well-known trampoline style, where each C
> function returns a pointer to the next function to jump into. The stack
> is managed explicitly. But for x86 I process the assembler output of 
> gcc
> and convert code which returns an address (marked with a comment in 
> asm)
> with a jump. This increased performance by about 30%.
>
> If the compiler is ever used on other architectures, someone who knows
> other assemblers might extend the mangler to handle them. I only know
> the x86 assembly.

You can find a somewhat more portable way to achieve a similar effect
using Gnu C in order to emit ``portable assembler'' code instead of
post-processing the compiled assembler code here:
   Fergus Henderson and Thomas Conway and Zoltan Somogyi,
   Compiling logic programs to {C} using {GNU C} as a portable assembler,
   Proc. ILPS '95 Postconference Workshop on Sequential Implementation
   Technologies for Logic Programming, pp. 1-15.
   http://www.cs.mu.oz.au/mercury/information/papers/mercury_to_c.ps.gz

I'm using that in my Curry compiler as well and it works nicely on 
Linux,
Solaris, and other OSes as long as you do not want to compile position
independent code. Unfortunately, it doesn't work on Mac OS X for exactly
that reason.

Wolfgang

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Tail-calls in C code (was: [ANN] The Missing Library)

[skaller]

On Mon, 2004-05-03 at 21:32, Wolfgang Lux wrote:

>    http://www.cs.mu.oz.au/mercury/information/papers/mercury_to_c.ps.gz

Curiously the basic continuation passing
model is given as (casts elided):

	while(fp) fp = (*fp)();

Here's the eqivalent Felix driver loop:

	while(p) p = p->resume();

where the continuation is a class object,
rather than a function pointer. 

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Tail-calls in C code (was: [ANN] The Missing Library)

[skaller]

On Mon, 2004-05-03 at 21:32, Wolfgang Lux wrote:
>    http://www.cs.mu.oz.au/mercury/information/papers/mercury_to_c.ps.gz
> 
> I'm using that in my Curry compiler as well and it works nicely on 
> Linux,
> Solaris, and other OSes as long as you do not want to compile position
> independent code. Unfortunately, it doesn't work on Mac OS X for exactly
> that reason.

It also won't work with Felix at all because none of the
GNU extensions are available in C++ :(

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Tail-calls in C code (was: [ANN] The Missing Library)

[skaller]

On Mon, 2004-05-03 at 22:38, skaller wrote:
> On Mon, 2004-05-03 at 21:32, Wolfgang Lux wrote:
> >    http://www.cs.mu.oz.au/mercury/information/papers/mercury_to_c.ps.gz
> > 

> It also won't work with Felix at all because none of the
> GNU extensions are available in C++ :(

Ouch. Sorry I'm wrong: my version of g++ handles
addressing labels just fine .. 

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Tail-calls in C code (was: [ANN] The Missing Library)

[Marcin 'Qrczak' Kowalczyk]

W liście z pon, 03-05-2004, godz. 13:32 +0200, Wolfgang Lux napisał:

>    Compiling logic programs to {C} using {GNU C} as a portable assembler,

> as long as you do not want to compile position independent code.

So it's not that portable :-)

Well, I'm worried about violating GCC rules so badly, when jumping to
a label from a different function. What if the functions have different
stack frame sizes?

Are you sure it works e.g. when one function uses a variable length
array, and thus uses %ebp even though -fomit-frame-pointer is in effect
and other functions don't use %ebp? Who is responsible for adjusting
%esp along the jumps?

What if one function has put callee-saved registers on the stack and
happily uses them for local variables, where the other doesn't, and
jumping from the second function to the first and back clobbers %ebx
which is not properly restored later?

Anyway, my scheme in theory works with -fPIC but it generates horrible
code, on x86 at least. The pointer to global variables is computed in
each function separately, and since virtual registers are in global
variables... Normal C functions don't use so many globals so -fPIC
doesn't hurt them that much.

I guess that normal C code doesn't have to recompute the pointer to
globals in all cases when jumping between static functions. But gcc
doesn't know that the address of the function being taken will not be
really returned but will be the target of a jump, so it must prepare
the code to be called from unexpected places.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak@knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Wed, Apr 28, 2004 at 06:13:19AM +0100, Jon Harrop wrote:
> On Wednesday 28 April 2004 5:31 am, Brian Hurt wrote:
> > It's been too long since I've used the STL- what data structures and
> > algorithms does it provide?
> 
> The main thing is that it is iterator-centric, so you pass iterators around 
> instead of containers. For example, to represent a subarray without having to 
> copy it. These iterators are classified according to their abilities (e.g. 

That's not necessarily a good thing.  For one, it leads to complexity.
What if you modify the parent array?  Would the iterator show the new or
original values?  What if you truncate the parent?  Would the iterator
produce fewer results than it originally promised?  And of course, you
can't so nicely use iterators in match clauses like you can lists.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Wednesday 28 April 2004 4:15 pm, John Goerzen wrote:
> On Wed, Apr 28, 2004 at 06:13:19AM +0100, Jon Harrop wrote:
> > The main thing is that it is iterator-centric, so you pass iterators
> > around instead of containers. For example, to represent a subarray
> > without having to copy it. These iterators are classified according to
> > their abilities (e.g.
>
> That's not necessarily a good thing.  For one, it leads to complexity.

Absolutely, incidental complexity at that.

> What if you modify the parent array?  Would the iterator show the new or
> original values?

New values. The iterator presents a view of the original data. For example, 
you might have a function which normalises a vector which is passed to it as 
a pair of iterators. You could then use this function to normalise a column 
vector in a matrix by passing it "stride" iterators which transparently 
return a single element from each row (i.e. a column) instead of adjacent 
elements.

> What if you truncate the parent?  Would the iterator
> produce fewer results than it originally promised?

No, your program would crash. Possibly corrupting the file system. ;-)

> And of course, you
> can't so nicely use iterators in match clauses like you can lists.

Yes, exactly.

I think the "control driven" STL propaganda is just marketing hype. IMHO, such 
approaches to algorithmics are only justified as an inadequate work-around 
for the absence of higher-order functions. Look for "bind2nd" in the STL, for 
example.

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Brian Hurt]

Sorry for the delay on the response- I'm way behind on my email...

On Wed, 28 Apr 2004, Jon Harrop wrote:

> On Wednesday 28 April 2004 5:31 am, Brian Hurt wrote:
> > It's been too long since I've used the STL- what data structures and
> > algorithms does it provide?
> 
> The main thing is that it is iterator-centric, so you pass iterators around 
> instead of containers. 

Just for the record, you *can* do this just fine in Ocaml.  Ext-lib is 
already doing this.  This isn't a limitation of the language, it's a 
feature lack of the core library.

> These iterators are classified according to their abilities (e.g. 
> trivial, forward, random-access etc.). 

I don't think having lots of different types of iterators is all that 
usefull.  Once you get beyond a simple linear walk through the data 
structure, the nature of the data structure becomes important.

> I'm not saying that these things can't be done in ocaml, just that you
> can do them easily in C++ using the STL. I'd be very interested to hear
> ocaml equivalents though! If you want to know more about the STL then I
> suggest you refer back to Stepanov's ramblings, rather than looking at
> the "standard"  which was, unfortunately, bastardised by a committee...

Much of what you listed is either a) already implemented in the standard
library, or b) implemented in Ext-Lib.  The only stuff I noticed which we
lacked were the numerical stuff- 2D matricies/numerical algorithms and
complex numbers.  As you note, complex really needs to be a builtin.  And, 
IMHO, numerical algos really need to be a whole library by themselves- see 
LAPACK as an example.

--
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Sat, 2004-05-01 at 01:58, Brian Hurt wrote:

> Just for the record, you *can* do this just fine in Ocaml.  Ext-lib is 
> already doing this.  This isn't a limitation of the language, it's a 
> feature lack of the core library.

This isn't quite correct. Extlib enums are not the same
as iterators. However it is true that an important part
of the iterator concept is captured in Extlib.

basic C++ iterators have 3 functions: dereference, advance,
and compare-equal with strict invalidation conditions.

More advanced iterators support total order comparisons,
and allow copies to be held (as well as possibly reverse movement
and random access)

Extlib enums use a quite different concept. In particular
C++ iterators have very strict interpretation of validity
and copyability which Extlib does not specify.

Also enums do not support positional comparisons, or subranging
using a pair of iterator.

In addition C++ iterators support insertion, using an 
iterator to find a position in a container, and numerous
other properties all of which are directly related to their
origins conceptually as pointers into arrays.


> I don't think having lots of different types of iterators is all that 
> usefull.  Once you get beyond a simple linear walk through the data 
> structure, the nature of the data structure becomes important.

They're vital. The kinding of iterators as mentioned above
is fundamental. You may be right that, for example, bidirectional
or random iterators are less useful, but the distinctions
between forward, input, and output iterators are crucial,
and the lack of that distinction in Extlib effectively
breaks the library. I won't touch it until this is fixed.
Indeed, there is work in the C++ committee to FURTHER
classify iterators, particularly in respect of
mutability of the container and buffering of the
dereference operation.

Basically: a forward iterator is a position in a container
which lives in memory. Provided you don't modify the
container, iterators remain valid. Iteration does
not consume the container in any way. 

Input iterators can be copied but only one is valid
after advancing (the one you advanced). They are
used to operate on streams where advancing requires
a destructive operation.

In type theoretic terms, something like:
forward iterators work on inductive data types, 
input iterators on coinductive data types.

The difference is utterly fundamental.

Extlib tries to present a common interface for
input and forward enumerations. 

This is inconsistent. What needs changing may
well ONLY be documentation, I'm not sure.
Enums, like iterators, are intrinsically unsafe
to use. The conditions under which the 
interface will yield stated results must be
specified *precisely and pedantically*.

And it is NOT easy to do so. Witness STL.

To give an example: suppose you have two
stream handles and apply a double Extlib.Enum.iter2
on them. By specification, that causes a force
operation.

It seems clear, right? 

It isn't. What happens if its the SAME enumeration?

In C++ STL this is handled because there is
a STRICT interpretation of an algorithm like:

  while(p!=e) cout << *p++ << *q++;

Here ISO C rules make it clear that the next
two elements of the input sequence get
printed in an arbitrary order, then repeat.

Extlib iter makes no guarrantees at all.
It might well crash immediately because it
forces one of the handles completely first,
so the second one is already exhausted
because its the same iteration : this is
sure to be an issue if the enumeration
is a generator function.

Note in the C++ example it is trivial to
*enforce* an ordering:

  while(p!=e) { cout << *p++; cout  << *q++; }

Now we know the output of a single generator
will be strictly ordered.

Note that this assumes we have a handle!
If it isn't a handle kind of iterator,
then p++ immediately invalidates q,
and so q++ is undefined.

So specifying the semantics of the above
algorithm for an STL iterator is 
VERY HARD! It definitely requires a sophisticated
classification scheme for iterators.

There's no way around this: there is a compromise
between a complex set of iterator abstractions,
and simply accepting 'undefined behaviour' as
a specification for an algorithm.

Extlib can easily err on the side of 'undefined'.
But it must be pedantic about what IS defined
and what is not or there is no way to trust
it to integrate memory containers and streams.

Last I looked a symptom of the design fault
is found in the 'fast_count' function.
Whilst  that exists, the library is necessarily
flawed. Arguments about 'its useful' do not hold
any water: it has to be justified in terms
of a specified abstraction or thrown out.

At least part of the problem here is that
Ocaml is primarily a functional language,
and functional languages can't handle
stateful programming easily. Iterators are
intrinsically stateful: thats the whole point
of them.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 14:31, Brian Hurt wrote:
> On Mon, 26 Apr 2004, Martin Berger wrote:

> I'm wondering what concepts Ocaml can't express/enforce? 

Iterators. 

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Martin Berger]

>>I'm wondering what concepts Ocaml can't express/enforce? 
> 
> 
> Iterators. 

it's been a long time since i used C++, but i seem to remember that
complexity guarantees are also part of the STL specification: something
like "to go from element x to element x+n" using a linear iterator takes
at most O(n) steps".

martin

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 18:42, Martin Berger wrote:
> >>I'm wondering what concepts Ocaml can't express/enforce? 
> > 
> > 
> > Iterators. 
> 
> it's been a long time since i used C++, but i seem to remember that
> complexity guarantees are also part of the STL specification: something
> like "to go from element x to element x+n" using a linear iterator takes
> at most O(n) steps".

Yes, that's true of C++ Standard Library iterators.
However, in the more abstract context, I'd say that important
idea is a design guide, not a requirement.

Clearly a single fixed iteration technique for a given
data structure is not enough for all problems, any more than
a fixed set of algorithms.

In particular if you consider more general tree navigation,
you can certainly do anything with iterators, but clearly
the movement instructions may have to be generalised.
You may need 'goto_parent' as well as 'goto_next_child' 
for example..

The key thing characterising an iterator appears to be
that it is some state representing a position in a data 
structure.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: [ANN] The Missing Library

[Shivkumar Chandrasekaran]

This might be off-topic, but wasn't there some work on a purely 
functional approach to this (?) problem? Looking back in my own ocaml 
code in the sparse matrix module in CamlFloat, I see that I used Gerard 
Huet's ideas on "contextual lists" (I think that is what it was 
called), which seems to have some bearing on this discussion.

--shiv--

PS:

I found some references:

  G. Huet. Linear Contexts and the Sharing Functor: Techniques for 
Symbolic Computation. Workshop on Thirty Five Years of Automath, 
Heriot-Watt University, Edinburgh, April 2002. Slides available as  ps. 
Final version in Thirty Five Years of Automating Mathematics, ed. 
Fairouz Kamareddine, Kluwer 2003. Available electronically as  pdf or 
ps.

at

http://pauillac.inria.fr/~huet/bib.html

On Apr 28, 2004, at 4:38 AM, skaller wrote:

> The key thing characterising an iterator appears to be
> that it is some state representing a position in a data
> structure.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Yaron M. Minsky]

On Wed, 2004-04-28 at 04:24, skaller wrote:
> On Wed, 2004-04-28 at 14:31, Brian Hurt wrote:
> > On Mon, 26 Apr 2004, Martin Berger wrote:
> 
> > I'm wondering what concepts Ocaml can't express/enforce? 
> 
> Iterators. 

Why can't you do this kind of in ocaml?  Returning something like a
"next" function would seem to give you a very basic (but usable)
iterator.  Which part of the iterator abstraction can't you do?

y

-- 
|--------/            Yaron M. Minsky              \--------|
|--------\ http://www.cs.cornell.edu/home/yminsky/ /--------|

Open PGP --- KeyID B1FFD916
Fingerprint: 5BF6 83E1 0CE3 1043 95D8 F8D5 9F12 B3A9 B1FF D916


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 21:31, Yaron M. Minsky wrote:
> On Wed, 2004-04-28 at 04:24, skaller wrote:
> > On Wed, 2004-04-28 at 14:31, Brian Hurt wrote:
> > > On Mon, 26 Apr 2004, Martin Berger wrote:
> > 
> > > I'm wondering what concepts Ocaml can't express/enforce? 
> > 
> > Iterators. 
> 
> Why can't you do this kind of in ocaml?  Returning something like a
> "next" function would seem to give you a very basic (but usable)
> iterator.  Which part of the iterator abstraction can't you do?

I suggest you try it. I don't know how to answer the question.
I think the answer is 'C++ templates provide functorial
polymorphism (polyadicity), Ocaml has no such expressive power.'

HOF's like 'map' and 'fold' make sense
for polynomial data structures -- lists and trees etc.
But you have to write 'List.map' or 'Array.map'. There
is no single 'map' which works for all data structures
in Ocaml.

There is in C++, and its a one line idiom:

// STL copy
for(; i!=e; ++i) *o++ = *i++;

Fold is as easy to define (called Accumulate in STL).

These definitions are polyadic. They work for ALL
data structures -- provided they have iterators
of course :D

A polyadic 'map' and 'fold' for ML can be defined
for polynomial data types (lists, trees etc).
I mean, you can easily code it yourself when you 
need it.

Ocaml can't yet automate this: C++ templates have
allowed this for some time. The C++ technology
isn't sound though. It works, usually, but you
can't be sure.

I'm no theorist, although I worked on a compiler
that implemented functorial polymorphism.
Perhaps real theorist can explain better what I'm
talking about.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> > Why can't you do this kind of in ocaml?  Returning something like a
> > "next" function would seem to give you a very basic (but usable)
> > iterator.  Which part of the iterator abstraction can't you do?
>
> I suggest you try it. I don't know how to answer the question.
> I think the answer is 'C++ templates provide functorial
> polymorphism (polyadicity), Ocaml has no such expressive power.'
>
> HOF's like 'map' and 'fold' make sense
> for polynomial data structures -- lists and trees etc.
> But you have to write 'List.map' or 'Array.map'. There
> is no single 'map' which works for all data structures
> in Ocaml.

There is ExtLib Enum.map that's pretty good at doing that.

> There is in C++, and its a one line idiom:
>
> // STL copy
> for(; i!=e; ++i) *o++ = *i++;
>
> Fold is as easy to define (called Accumulate in STL).
>
> These definitions are polyadic. They work for ALL
> data structures -- provided they have iterators
> of course :D

That's exactly the same as ExtLib enums : define an enumerator for your data
structure, and you can apply a full range of algorithms on it. I don't
really understand the difference with  STL.

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 22:36, Nicolas Cannasse wrote:

> There is ExtLib Enum.map that's pretty good at doing that.

Yes indeed. Extlib enums are very promising.

> That's exactly the same as ExtLib enums : define an enumerator for your data
> structure, and you can apply a full range of algorithms on it. I don't
> really understand the difference with  STL.

I think there are a couple of differences.

First, Extlib enums are weaker than STL iterators abstractly.
An enum is a single object representing the *tail* of an enumeration.
An STL iterator represents a *position*. Thus for example
two STL iterators represent the tail of an enumeration.
This is much more powerful (and also less safe dynamically).
STL iterators can also be used to denote insertion points,
result of a search, etc.

Secondly, the semantics of STL iterators are more refined
and better specified. There are several classes of STL
iterators (input, output, forward, bidirectional, ..).
Also the invalidation conditions are dealt with
in detail. STL iterators also have minimum laziness
of one value..the laziness of enums is not quite
so well defined... recall I had some problems
with the specifications and still do.

However, given an enumeration, I agree there is 
some kind of polyadic behaviour represented.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> On Wed, 2004-04-28 at 22:36, Nicolas Cannasse wrote:
>
> > There is ExtLib Enum.map that's pretty good at doing that.
>
> Yes indeed. Extlib enums are very promising.
>
> > That's exactly the same as ExtLib enums : define an enumerator for your
data
> > structure, and you can apply a full range of algorithms on it. I don't
> > really understand the difference with  STL.
>
> I think there are a couple of differences.
>
> First, Extlib enums are weaker than STL iterators abstractly.
> An enum is a single object representing the *tail* of an enumeration.
> An STL iterator represents a *position*. Thus for example
> two STL iterators represent the tail of an enumeration.
> This is much more powerful (and also less safe dynamically).
> STL iterators can also be used to denote insertion points,
> result of a search, etc.

Such comparison  tail = end , is also well fitted for the C++ world, where
in general NULL is the end of everything, and operator overloading helps
here a lot. Just one question : what is the excepted behavior of equality
between two iterators coming from two different data structures - both
containing the same type of elements ? What about equality between two
iterators over two different dynamic arrays ? Is the first iterator
index-bounded by the second array (the one it's not itering on) ?

> Secondly, the semantics of STL iterators are more refined
> and better specified. There are several classes of STL
> iterators (input, output, forward, bidirectional, ..).
> Also the invalidation conditions are dealt with
> in detail. STL iterators also have minimum laziness
> of one value..the laziness of enums is not quite
> so well defined... recall I had some problems
> with the specifications and still do.

Please state clearly your problems on the ExtLib mailling list, I will be
happy to think about theses :)
Enums are still young so there not yet specialized enums that can  remove
elements while itering or give random access. Maybe in the future...

Best Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 00:02, Nicolas Cannasse wrote:
> > On Wed, 2004-04-28 at 22:36, Nicolas Cannasse wrote:

> Such comparison  tail = end , is also well fitted for the C++ world, where
> in general NULL is the end of everything, and operator overloading helps
> here a lot. Just one question : what is the excepted behavior of equality
> between two iterators coming from two different data structures 

They have distinct types so: compile time error.
[I assume you mean for example a list and an array]

>  What about equality between two
> iterators over two different dynamic arrays ? 

I think from memory, equality must work. 
However, less (<) operator is undefined.

> Please state clearly your problems on the ExtLib mailling list, I will be
> happy to think about theses :)

I already did quite extensively.. :)

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jean-Christophe Filliatre]

skaller writes:
 > > > > I'm wondering what concepts Ocaml can't express/enforce? 
 > > > 
 > > > Iterators. 
 > > 
 > > Why can't you do this kind of in ocaml?  Returning something like a
 > > "next" function would seem to give you a very basic (but usable)
 > > iterator.  Which part of the iterator abstraction can't you do?
 > 
 > I suggest you try it. I don't know how to answer the question.

Perhaps I missed  something in this thread but it  is not so difficult
to  write iterators  in  ocaml (of  course it  can  not be  done in  a
systematic way --- may be that was John's point).

In ocamlgraph we provide depth-first and breadth-first graph traversal
both as HOF and as iterators. Iterators have the following signature:

======================================================================
  type iterator
  val start : G.t -> iterator
  val step : iterator -> iterator
  val get : iterator -> G.V.t
======================================================================

(where G.t is  the type of graphs).  It was a lot of  fun coding these
itearators,  and very  useful  to write  a  backtracking algorithm  to
colors graphs.  It was particularly easy as  iterators are implemented
as persistent  data structures:  thus there is  no `undo' part  in the
backtracking code.

I guess one cannot write  a backtracking algorithm using the usual HOF
fold function.

If one is interested the code is here: 
http://www.lri.fr/~filliatr/ocamlgraph/

-- 
Jean-Christophe Filliâtre

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Wed, 2004-04-28 at 23:15, Jean-Christophe Filliatre wrote:
> skaller writes:

> In ocamlgraph we provide depth-first and breadth-first graph traversal
> both as HOF and as iterators. Iterators have the following signature:
> 
> ======================================================================
>   type iterator
>   val start : G.t -> iterator
>   val step : iterator -> iterator
>   val get : iterator -> G.V.t

Perhaps the iterator is actually the triple:

	(iterator, step, get)

Ignoring the lack of termination, you might
think to write a function now:

let rec print p (it, step, get) =
  p (get it);
  print p (step it, step, get)

where:
  p:'a -> unit
  it:'b
  step: 'b->'b
  get: 'b -> 'a

val print : 'a -> 'b * ('b -> 'b) * ('b -> 'c) -> 'd = <fun>

[The 'd will be unit if I trapped termination with an exception]

So perhaps it can be done .. although not for Hashtbl.
Hastbl.iter is control inverse of what is required.
Should work fine for 'Set' using 'choose' function.
Works for List.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jean-Christophe Filliatre]

skaller writes:
 > 
 > > In ocamlgraph we provide depth-first and breadth-first graph traversal
 > > both as HOF and as iterators. Iterators have the following signature:
 > > 
 > > ======================================================================
 > >   type iterator
 > >   val start : G.t -> iterator
 > >   val step : iterator -> iterator
 > >   val get : iterator -> G.V.t
 > 
 > Perhaps the iterator is actually the triple:
 > 
 > 	(iterator, step, get)

In  the   particular  case  of  ocamlgraph,  the   iterators  are  not
implemented as functions triples (though they could be). Instead it is
basically a stack for the DFS and  a queue for the DFS (plus the graph
itself and a  set of not yet visited nodes to  ensure the traversal of
all connected components).

-- 
Jean-Christophe

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 00:40, Jean-Christophe Filliatre wrote:
> skaller writes:
>  > 
>  > Perhaps the iterator is actually the triple:
>  > 
>  > 	(iterator, step, get)
> 
> In  the   particular  case  of  ocamlgraph,  the   iterators  are  not
> implemented as functions triples (though they could be).

I meant that 'mathematically' the actual iterator is a triple.
You need to specify more than the 'iterator value' to specify
an iterator: you need its methods too.

Similar to 'cartesian product' is not a value, but a value
and a pair of projection functions.

Categorically .. the value is an irrelavance: only the
functions *actually* matter .. so perhaps an iterator
*really* is just a pair:

	step:unit->unit
	get: unit->'a

which is what Andreas Rossburg said earlier, except he
combined 'step' into 'get' and added a termination condition:

	get: unit -> 'a option

Example is the well known function 'get_token()' used
by parsers. Actually, this is an input iterator.

Step/get is a forward iterator. Input iterators have no
interesting semantics, they don't have enough structure.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Andreas Rossberg]

skaller wrote:
>>
>>Why can't you do this kind of in ocaml?  Returning something like a
>>"next" function would seem to give you a very basic (but usable)
>>iterator.  Which part of the iterator abstraction can't you do?
> 
> I suggest you try it. I don't know how to answer the question.
> I think the answer is 'C++ templates provide functorial
> polymorphism (polyadicity), Ocaml has no such expressive power.'

It has, but only on the module level.

Anyway, I don't think it is needed for generic iteration. The most 
important aspects of iterators easily can be captures by either HOFs or 
by lazy streams. For example, a forward iterator just might be an 
arbitrary function of type

   unit -> 'a option

Repeated calls advance the iterator, at the end it returns None.

More functionally, with streams, an iterator is a value of type 'a 
stream, probably definable in OCaml as follows:

   type 'a stream = 'a stream' lazy_t
   and 'a stream' = Nil | Cons of 'a * 'a stream

Essentially, you iterate over (parts of) arbitrary structure by turning 
them into lazy lists. Using laziness to move from a push to a pull model 
(which is what your control inversion means, IIUC) is rather a standard 
FP technique.

The main difference wrt C++ style iterators is that you need to give the 
end of the iteration in advance (if you want some particular range). I 
don't think you lose something important by that limitation.

	 - Andreas

-- 
Andreas Rossberg, rossberg@ps.uni-sb.de

Let's get rid of those possible thingies!  -- TB

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: [ANN] The Missing Library

[Shivkumar Chandrasekaran]

Again, this might be off-topic, but the purely functional language 
Clean (as of version 2.0 at least) supposedly can do this via ``generic 
programming''. It is not clear though how useful it is.

--shiv--


On Apr 28, 2004, at 5:09 AM, skaller wrote:

> I suggest you try it. I don't know how to answer the question.
> I think the answer is 'C++ templates provide functorial
> polymorphism (polyadicity), Ocaml has no such expressive power.'
>
> HOF's like 'map' and 'fold' make sense
> for polynomial data structures -- lists and trees etc.
> But you have to write 'List.map' or 'Array.map'. There
> is no single 'map' which works for all data structures
> in Ocaml.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: [ANN] The Missing Library

[skaller]

On Thu, 2004-04-29 at 02:10, Shivkumar Chandrasekaran wrote:
> Again, this might be off-topic, but the purely functional language 
> Clean (as of version 2.0 at least) supposedly can do this via ``generic 
> programming''. It is not clear though how useful it is.

I'm not sure what Clean does. However functorial polymorphism
is clearly vitally important and extremely powerful.

One example which works in FISh 1, which is an array
language, is that you can write an algorithm
which works for a rank n problem.
I have *seen* a heat flow algorithm which can be
tested on a 2D object, and the algorithm then
applied to a 3D object -- without changing anything.

Just try to do that in C or Fortran or Ocaml.
You can't. In C for example, a typical operation
on a 2D object is a double loop:

	for(i= 
	for(j=

but for 3D you need:

	for(i=
	for(j=
	for(k=

that is, you have no choice implementing generalised
tensor mathematics than to rewrite the program
for every value of n, the rank.

Yet the (tensor) maths is identical and independent
of the rank. I guess there are numerous other
science problems where you do a calc for a certain
set of generalised coordinates .. and then need
to add more coordinates and recalculate ..
only you have to rewrite the program every time.

For more general data stuctures of the form

type 'a x = X of int * 'a | Y of 'a * 'a | Empty 

etc, it is clear how to write a map function.
But each such data structure has a distinct map
function with a distinct type.

Actually, if you have fold, you can define map
for all such data stuctures: map is just a special
case of fold where the output value is a similarly
shaped data structure with different value type.

So clearly .. with functorial polymorphism
you get a LOT more resuable code than at the moment.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: [ANN] The Missing Library

[Shivkumar Chandrasekaran]

On Apr 28, 2004, at 10:14 AM, skaller wrote:

> Just try to do that in C or Fortran or Ocaml.
> You can't. In C for example, a typical operation
> on a 2D object is a double loop:
>
> 	for(i=
> 	for(j=
>
> but for 3D you need:
>
> 	for(i=
> 	for(j=
> 	for(k=
>
> that is, you have no choice implementing generalised
> tensor mathematics than to rewrite the program
> for every value of n, the rank.
>
> Yet the (tensor) maths is identical and independent
> of the rank. I guess there are numerous other
> science problems where you do a calc for a certain
> set of generalised coordinates .. and then need
> to add more coordinates and recalculate ..
> only you have to rewrite the program every time.

Not sure exactly where this is going, but I do have ocaml code where n 
is  just a parameter, and the code works across n-dimensional arrays. 
Maybe you have something more general in mind since I am not familiar 
with FISh1.


>
> For more general data stuctures of the form
>
> type 'a x = X of int * 'a | Y of 'a * 'a | Empty
>
> etc, it is clear how to write a map function.
> But each such data structure has a distinct map
> function with a distinct type.

Clean 2.0 can automatically handle your case (I believe). It can 
produce the different map's, each with a different type, automatically 
from one generic definition. Check it out!

--shiv--

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: [ANN] The Missing Library

[Jon Harrop]

On Wednesday 28 April 2004 6:14 pm, skaller wrote:
> I have *seen* a heat flow algorithm which can be
> tested on a 2D object, and the algorithm then
> applied to a 3D object -- without changing anything.

I designed and wrote C++ code like this for my PhD in Physics. It was 
partially specialised over the dimensionality of the Euclidean space and over 
the tupleness of particle interactions (i.e. n=2 for pair potential etc.). I 
used C++ templates to perform the partial specialisation. Short of finding 
and having to work around a bunch of bugs in gcc, the resulting code was 
reliable and considerably more efficient than unspecialised code.

> Just try to do that in C or Fortran or Ocaml.

I bet MetaCaml would do a much, much better job than C++. And I bet it 
wouldn't segfault the compiler. ;-)

> You can't.

That isn't true, of course. They're all Turing complete so, if the worst comes 
to the worst, you can write a Fish compiler in C...

> In C for example, a typical operation
> on a 2D object is a double loop:
>
> 	for(i=
> 	for(j=
>
> but for 3D you need:
>
> 	for(i=
> 	for(j=
> 	for(k=

The first step in writing such code correctly is to realise that this is a bad 
way of looking at the problem. You don't want nested loops, you just want a 
function to propagate computation, whether it be to the next element or to 
the first element on the next row. Incidentally, there is no performance loss 
in making this part of the generalisation.

> that is, you have no choice implementing generalised
> tensor mathematics than to rewrite the program
> for every value of n, the rank.

That is only true if you approach the problem wrongly. If you approach the 
problem correctly then you can do everything that you are asking for.

> Yet the (tensor) maths is identical and independent
> of the rank. I guess there are numerous other
> science problems where you do a calc for a certain
> set of generalised coordinates .. and then need
> to add more coordinates and recalculate ..
> only you have to rewrite the program every time.

Yes, the ability to write such code is imperative (ho, ho, ho) for scientific 
code. I found it very useful to debug my algorithms in 2D before using them 
for production stuff in 3D.

> For more general data stuctures of the form
>
> type 'a x = X of int * 'a | Y of 'a * 'a | Empty
>
> etc, it is clear how to write a map function.
> But each such data structure has a distinct map
> function with a distinct type.

Yes, for efficiency you want to use a tuple (2-tuple for 2D, 3-tuple for 3D 
etc.) but for generality you want to use a list. Partial specialisation gives 
you the best of both worlds.

> So clearly .. with functorial polymorphism
> you get a LOT more resuable code than at the moment.

Not if you do it properly.

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Benjamin Geer]

james woodyatt wrote:
> On 23 Apr 2004, at 14:33, John Goerzen wrote:
>> A plethora of mutually-incompatible modules that duplicate and extend
>> standard library features is in nobody's interest.  The result will be
>> an irrelevant standard library and a fragmented development community.
> 
> Yeah— that was such a huge roadblock to the popularity of the C language.

The difficulty of writing portable C or C++ code, given the differences 
between libraries available on different platforms, has been one of the 
main reasons for the massive industry shift towards Java in recent years.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Sun, 2004-04-25 at 22:20, Benjamin Geer wrote:
> james woodyatt wrote:
> > On 23 Apr 2004, at 14:33, John Goerzen wrote:
> >> A plethora of mutually-incompatible modules that duplicate and extend
> >> standard library features is in nobody's interest.  The result will be
> >> an irrelevant standard library and a fragmented development community.
> > 
> > Yeah— that was such a huge roadblock to the popularity of the C language.
> 
> The difficulty of writing portable C or C++ code, given the differences 
> between libraries available on different platforms, has been one of the 
> main reasons for the massive industry shift towards Java in recent years.

Portable C++ is easy to write. The main reason for Java's
success, technically, is as simple as "garbage collection".
The main library reason was the standard GUI.

Portable graphics is impossible without a standard,
Java had it. And as we all know memory management in
C is a nightmare. C++ is vastly superior with
constructors/destructors but garbage collection is
hard to beat for easy memory management :D

However, the 'main' reason for Java's success wasn't
technical. I mean, that woeful crap still doesn't 
have any kind of serious polymorphism. C++ is vastly
superior as a language to Java.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Benjamin Geer]

skaller wrote:
> Portable C++ is easy to write.

I'm thinking of the differences between the Unix and the Win32 APIs. 
That's one kind of library incompatibility that has caused programmers 
no end of pain.  (And don't forget that Microsoft still has its own C++ 
string class.)  That's the sort of pain I think we can avoid in the 
OCaml world by putting in place a reasonable community structure.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-04-26 at 01:07, Benjamin Geer wrote:
> skaller wrote:
> > Portable C++ is easy to write.
> 
> I'm thinking of the differences between the Unix and the Win32 APIs. 
> That's one kind of library incompatibility that has caused programmers 
> no end of pain.  

I agree, but that isn't the fault of C or C++, anymore than
the collection of common libraries that are available 
and are hard to interface: the same problem will *always*
arise in any general purpose programming language which
is popular enough to inspire a large number of development efforts.

>  That's the sort of pain I think we can avoid in the 
> OCaml world by putting in place a reasonable community structure.

I don't see how we can avoid this pain.
We still have to deal with interfacing to existing libraries.

We can, perhaps, ameliorate some of the pain by a coordinated
effort at standardisation, but we are also going to get
extra pain that is not felt by C/C++ programmers, because
we actually have to create bindings for C/C++ libraries
to use them: C/C++ programmers do not.

This is one reason I think an Ocaml based wrapper generator
would be a good idea: the community can gain control over
the generator and make it work much better than a foreign
tool like SWIG.

Of course this isn't  complete solution, and it has no
real impact on designing a standard, fairly rich, algorithms
and data structures library.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Basile STARYNKEVITCH]

On Fri, Apr 23, 2004 at 04:10:03PM -0500, John Goerzen wrote:
> On Fri, Apr 23, 2004 at 10:36:11PM +0200, Maxence Guesdon wrote:
> > Dans son style inimitable, John Goerzen écrivait:
> > 
> > > But the fact that these libraries exist is, at its heart, a symptom of
> > > the problems with the OCaml standard library.

I disagree with the above sentence.

> > I'm getting bored of this song. What about this one :
> > "The number of extlibs is a symptom of the problems with the
> > community of OCaml users" ?


> You view it as a problem that some in the OCaml community would like to
> see a more featureful and easy to use standard library?  Why?

I agree with Maxence. Even if I currently work (on a temporary
position ending on 15th september 2004) at INRIA in Cristal, I will
give my personal opinion here, which probably is not exactly that of
the OCaml project (ie Cristal) at INRIA.

I think that the standard library is actually rather perhaps too big,
than too small!

People should really use external libraries, and it is not the job of
INRIA to develop all of them. It is the job of the community to make
and enhance additional libraries. The community works quite well,
given the big amount of good libraries (advertised in the hump or
otherwise, e.g. freshmeat or sourceforge). And more and more external
contributions appear every week!

It is a pity that 2 libraries named ExtLib exist, and it is certainly
not the fault of OCaml team (I tend to believe that one of the ExtLib
named library is nearly dormant, and I invite the author of this
library to rename it).

The goal of INRIA (as defined by official documents) is to make useful
reasearch in computer science (If you are interested by the documents,
you can read them in French on INRIA's web site). As a French taxpayer
(and there are not enough taxes spent in France on reasearch - there
is globally under-funding of research in the whole European Union,
especially when compared to the US or to Japan), I don't think that it
is the goal of INRIA to code lots of external libraries, or even to
manage the naming of many packages, which is a very labor-consuming
social task not really related to research. And more importantly,
these tasks are not defined to be done at INRIA by its official
policy.

INRIA is not a big commercial company like Sun. So the goals of INRIA
w.r.t. Ocaml are not similar to the goals of Sun w.r.t. Java. And the
means (e.g. funding) devoted to Ocaml is much smaller than those
devoted to Java (or probably Python or Perl).

Again, the raison d'être of INRIA is research, not [mainly]
development of software libraries. And INRIA's performance is not
measured (for the french government, ie for the entity giving the
money to INRIA) in number of lines of OCaml source code in libraries!
Again, all these official documents are public (sometimes in French),
including (IIRC) the evaluation of INRIA's work.

Everyone can contribute to Ocaml by developping libraries, and
advertising them (e.g. thru the Hump). If there is a name clash, it
should be socially resolved, and it is not a research goal to solve it
(this is why it is not INRIA's job to solve such conflicts, and
solving such "managing" or "social" problems is extremely time
consuming, hence very costly).

In my opinion, the standard library should be kept small, and should
contain only the functions commonly needed to the compiler and to all
the Ocaml software produced by INRIA in relation to the OCaml language
(e.g. compilers and similar tools). The standard library should
certainly not becomes the union of all useful OCaml modules (from the
Hump - which is more a Bazar, while the core Ocaml software, including
the standard library, is a Cathedral).

For example, when (2 years ago) I worked on Poesia (at CEA in that
time), I had to code an interface to uname, and I just coded it,
without asking anybody.

People should not expect every OCaml problem to be solved at INRIA. It
is not reasonable (and not the mission of INRIA anyway!), but people
should continue to contribute to OCaml thru useful libraries, and
should use and enhance these external contributions.


Regarding standard libraries, they should be kept small, because they
have to be learnt with the language. I believe that one of Common Lisp
problems (and perhaps source of failure) is the huge size of its
standard library, which is intimidating.

Again, everything above is my own opinion, not those of INRIA (where I
happen to work currently) - I am not paid to talk for INRIA! But I
think that there is too much criticism w.r.t. to the work on Ocaml
done at INRIA. "C'est la rançon du succès" (I leave this last sentence
in French).


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/ 
email: basile<at>starynkevitch<dot>net 
aliases: basile<at>tunes<dot>org = bstarynk<at>nerim<dot>net
8, rue de la Faïencerie, 92340 Bourg La Reine, France

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Sat, Apr 24, 2004 at 12:08:15AM +0200, Basile STARYNKEVITCH wrote:
> On Fri, Apr 23, 2004 at 04:10:03PM -0500, John Goerzen wrote:
> > > I'm getting bored of this song. What about this one :
> > > "The number of extlibs is a symptom of the problems with the
> > > community of OCaml users" ?
> 
> > You view it as a problem that some in the OCaml community would like to
> > see a more featureful and easy to use standard library?  Why?
> 

[ snip ]

> I think that the standard library is actually rather perhaps too big,
> than too small!

If you are advocating splitting bits of the standard library into a
centrally-maintained community resource, that makes a lot of sense to
me.  If you are repeating the argument that "feh, task x can be
trivially done with less than 10 lines of code!", then I disagree
completely.  Which is it? :-)

> It is a pity that 2 libraries named ExtLib exist, and it is certainly
> not the fault of OCaml team (I tend to believe that one of the ExtLib
> named library is nearly dormant, and I invite the author of this
> library to rename it).

I agree.

> especially when compared to the US or to Japan), I don't think that it
> is the goal of INRIA to code lots of external libraries, or even to
> manage the naming of many packages, which is a very labor-consuming

I have never asked that they do that, nor do I expect them to.  However,
we have a somewhat unfortunate situation.  INRIA does, for whatever
reason, maintain a large OCaml library -- probably the largest in
existance and certainly the most widely distributed.  This same library
is, in the opinion of some, lacking in features which INRIA has for
plenty of understandable reasons (lack of time, etc.) not implemented.

Now we are stuck.  If I were to write "John's unix" or "John's str",
they would be incompatible with the standard unix or str.  Code using
modules written for one could not just automatically use modules written
for another.  (I assume there are some exceptions here, but probably not
many.)

So I think there are three good ways to move forward:

1. INRIA devotes more resources to the library;

2. INRIA becomes much more active with accepting patches from people
   in the community and communicates better with the community;

3. INRIA splits off Unix (and perhaps others such as dbm, labltk, etc)
   for the community to maintain.  INRIA could, at their option,
   bundle snapshots of these community efforts with various OCaml
   releases to ensure no loss of functionality with a standard install.

> INRIA is not a big commercial company like Sun. So the goals of INRIA
> w.r.t. Ocaml are not similar to the goals of Sun w.r.t. Java. And the

I'm not sure that is a bad thing :-)

> Everyone can contribute to Ocaml by developping libraries, and
> advertising them (e.g. thru the Hump). If there is a name clash, it

What what do you say to the compatibility problems I've raised?

To be sure, there are yet technical obstacles, such as difficult
installation and build systems, but from the recent discussion on those,
I am accepting that they will be resolved.

> should be socially resolved, and it is not a research goal to solve it
> (this is why it is not INRIA's job to solve such conflicts, and
> solving such "managing" or "social" problems is extremely time
> consuming, hence very costly).

I agree again.  Let the community do its thing.  But if they are to do
that, you must let them by not insisting on bundling dated software with
the standard distribution.

> In my opinion, the standard library should be kept small, and should
> contain only the functions commonly needed to the compiler and to all
> the Ocaml software produced by INRIA in relation to the OCaml language
> (e.g. compilers and similar tools). The standard library should
> certainly not becomes the union of all useful OCaml modules (from the
> Hump - which is more a Bazar, while the core Ocaml software, including
> the standard library, is a Cathedral).

I have no problem with that CPAN-like approach.  Personally, I prefer
Python's approach, but CPAN works too :-)  Of course, there's nothing
stopping INRIA from taking compatibly-licensed stuff from the community
and bundling it with OCaml.

> Regarding standard libraries, they should be kept small, because they
> have to be learnt with the language. I believe that one of Common Lisp
> problems (and perhaps source of failure) is the huge size of its
> standard library, which is intimidating.

I'm not sure about CL, but I can heartily echo that sentiment with Java.
Regardless of the final outcome of the library, it should be
well-planned.

> Again, everything above is my own opinion, not those of INRIA (where I
> happen to work currently) - I am not paid to talk for INRIA! But I
> think that there is too much criticism w.r.t. to the work on Ocaml
> done at INRIA. "C'est la rançon du succès" (I leave this last sentence
> in French).

Please don't take my messages that way; I do not mean them like that at
all.  I understand completely how INRIA has constraints like everyone
else and am not complaining that they don't put more resources to it.

I do think the state of affairs is bad.  I think we need to fix it.  And
I think that INRIA has to be involved in that fix, even if their
involvement ends with saying "Here, have at it; this is no longer part
of OCaml" or "Here, have at it; we'll sync with your CVS when we make a
release" or something along those lines.  I'm not asking INRIA to solve
all the world's problems.  Just to decide whether or not they want to
:-)

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Oliver Bandel]

On Sat, Apr 24, 2004 at 12:08:15AM +0200, Basile STARYNKEVITCH wrote:
[...] 
> Regarding standard libraries, they should be kept small, because they
> have to be learnt with the language. I believe that one of Common Lisp
> problems (and perhaps source of failure) is the huge size of its
> standard library, which is intimidating.

Yes. This is one point: Without good books it seems unpossible to
master Lisp because of its library size.

But I don't agree that the OCaml-livbraries are too big.
But I also accept the INRIA Cathedral in respect to the
Core-OCaml. IMHO OCaml is an excellent language.

And I'm not interested in incompatibel or bloated
Libraries. So it's good that INRIA/OCaml-core-developers
do not accept every thing that is talked about here.

But some little enhancements in convenience could be made
by some little more code in the libraries.

For example: The first time I explored OCaml was with
OCaml 3.01. I printed out the complete Reference manual
and sometimes, when I had the time, I learned to program
with it. Otherwise, when I had no time to do so,
I paused with OCaml.

And one day I used the Hashtbl-module, and recently had
the need to go through all bindings and collect data (in
a functional way).
But: there was no Hashtbl.fold in the manual.
Well the task would have been possible inmperatively with
Hashtbl.iter, but I looked for a Hashtbl.fold.
Well, and I was happxy, that at that time I asked for it,
it already was implemented in the nbewer versions of
OCaml. So IMHO there is progression on needed things
in the standard lib, and it is well done.

And when I ask for a Hashtbl.keys now, this may be accepted
by the OCaml-team, or it may not be accepted.
There is discussion and maybe they think: OK, good idea,
or they trhink: Well, nice idea, but not for the standard lib.

If the latter, I can use Hashtb.fold and programming it by myself,
or may use other Libraries.

But at least the discussion about new functions should be
possible.

BTW: Is there a list, where people can "throw in" their
     ideas/suggestions to OCaml?
     Or is all done by this mailing list? Wouldn't it
     be nice to have a separated list, or a webpage,
     where suggestions could be listed without further
     discussion, only that it is possible to gain an overview
     on what people want to have?
     I think it's goof that the OCaml-core developers will have
     the last word to say about it, but coming up with ideas
     should also be allowed.



> 
> Again, everything above is my own opinion, not those of INRIA (where I
> happen to work currently) - I am not paid to talk for INRIA! But I
> think that there is too much criticism w.r.t. to the work on Ocaml
> done at INRIA. "C'est la rançon du succès" (I leave this last sentence
> in French).

I don't want to see my mails as criticism to OCaml-team/INRIA.
They have done a great work and I'm glad to have the possibility
to program with OCaml.

I only tell here some ideas and suggestions and what I want to
have there, but this is not intended as a criticism to
the OCaml/INRIA people!


Ciao,
   Oliver

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Sat, 2004-04-24 at 20:10, Oliver Bandel wrote:

> 
> BTW: Is there a list, where people can "throw in" their
>      ideas/suggestions to OCaml?

Yes, there is a browser based bug tracker with
an option to flag an input 'feature-request'.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Kenneth Knowles]

On Fri, Apr 23, 2004 at 03:23:42PM -0500, John Goerzen wrote:
> But the fact that these libraries exist is, at its heart, a symptom of
> the problems with the OCaml standard library.

I'm just encouraging the dissenters to cooperate to more thoroughly demonstrate
their case, and get a well-design library.

I do suggest splitting domain-independent stuff like extended list and hashtable
functions from specialized stuff like IMAP and config file parsers.

Kenn

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 01:54:51PM -0700, Kenneth Knowles wrote:
> On Fri, Apr 23, 2004 at 03:23:42PM -0500, John Goerzen wrote:
> > But the fact that these libraries exist is, at its heart, a symptom of
> > the problems with the OCaml standard library.
> 
> I'm just encouraging the dissenters to cooperate to more thoroughly demonstrate
> their case, and get a well-design library.
> 
> I do suggest splitting domain-independent stuff like extended list and hashtable
> functions from specialized stuff like IMAP and config file parsers.

That does make some sense; the former could possibly go into Extlib
without much difficulty.  They also represent less new work, so I'd have
less of a complaint about my work ending up under the LGPL instead of
the GPL.

I'll talk to them about it.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Brian Hurt]

On Fri, 23 Apr 2004, Kenneth Knowles wrote:

> On Fri, Apr 23, 2004 at 03:23:42PM -0500, John Goerzen wrote:
> > But the fact that these libraries exist is, at its heart, a symptom of
> > the problems with the OCaml standard library.
> 
> I'm just encouraging the dissenters to cooperate to more thoroughly demonstrate
> their case, and get a well-design library.

A large part of the problem is one of mindshare.  The fact that extlib is
out there isn't immediately obvious to someone just comming into Ocaml.  
In fact, it's rather hard to find.  So, someone new comes in, discovers
that such-and-such a function or functionality is missing from the
standard library, doesn't see a standard way to get said added to the 
standard library, and decides to help out the community by starting a 
parallel standard library with looser submission rules.  Lather, rinse, 
repeat.

> I do suggest splitting domain-independent stuff like extended list and
> hashtable functions from specialized stuff like IMAP and config file
> parsers.

Personally, I see three levels of library functionality:

1) Core libraries, those libraries which are so core, so important that
it's hard to impossible to write programs without them.  Examples would be
Pervasives (duh), List, Array, etc.  This list may be expanded to include
those libraries needed to compile/run the compiler.  This level would be
maintained by the INRIA folks.

2) Standard Environment Libraries.  These would be the libraries a program
could count on to be installed in normal environments.  Examples would
include GUI libraries, various network protocols, an XML parser, etc.  
This would be a single, monolithic, project under a single license, but
one maintained by the community as a whole.

3) A CPAN-like tool.  This is for libraries that are not commonly used 
enough for inclusion into the standard environment (ex. PSQueue), or 
libraries with licensing issues.  Most programs shouldn't need any OPAN 
libraries, and no project should depend upon more than 1-2 of them.  The 
key here is a simple, common, install procedure for libraries.

Note that particular features/functionality can migrate between the 
levels.  The INRIA folks could decide that, for example, Hashtbl should 
get kicked out and moved down to the standard environment.  Or they could 
decide to move XML parsing into the core libraries.  The standard 
environment could decide to start including encryption libraries from 
OPAN, or they could kick PSQueue out into OPAN because not enough people 
are using it.

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[skaller]

On Mon, 2004-04-26 at 01:43, Brian Hurt wrote:
> On Fri, 23 Apr 2004, Kenneth Knowles wrote:

> Personally, I see three levels of library functionality:
> 
> 1) Core libraries, 

> 2) Standard Environment Libraries.  

> 3) A CPAN-like tool. 

You missed the most interesting one :

(4) Algorithms and datastructures

Both (2) and (4) are 'extensions' of the minimalist
functionality of (1).


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Brian Hurt]

On 26 Apr 2004, skaller wrote:

> On Mon, 2004-04-26 at 01:43, Brian Hurt wrote:
> > On Fri, 23 Apr 2004, Kenneth Knowles wrote:
> 
> > Personally, I see three levels of library functionality:
> > 
> > 1) Core libraries, 
> 
> > 2) Standard Environment Libraries.  
> 
> > 3) A CPAN-like tool. 
> 
> You missed the most interesting one :
> 
> (4) Algorithms and datastructures
> 
> Both (2) and (4) are 'extensions' of the minimalist
> functionality of (1).
> 

Depending upon the algorithm and data structure, it could be in any one of 
the three.  The list data structure and sort really belong in the core 
libraries.  The matrix data structure and matrix invert really (IMHO) 
belong in the standard environment.  Priority search queues and polynomial 
factorization really belong in OPAN.

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Florian Hars]

John Goerzen wrote:
> At the surface, I'm
> wanting to provide larger modules (such as the ConfigParser already
> present, but perhaps also an IMAP client module, etc.) that seem not
> quite what either ExtLib is all about.

An IMAP module  might find reception there:
http://sourceforge.net/projects/ocamlnet

Yours, Florian.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Eric C. Cooper]

On Fri, Apr 23, 2004 at 01:51:48PM -0500, John Goerzen wrote:
> [Debian users: I have uploaded it to sid, but will take a few days to
> appear.]

I am a bit surprised by this.  What is the process by which OCaml
packages are added to Debian?  Are there any checks and balances?
Without in any way passing judgment on your particular effort, it
seems that a Debian developer with an OCaml agenda might have a
conflict of interest here, especially considering the scarce resource
that is the global OCaml module namespace.

-- 
Eric C. Cooper          e c c @ c m u . e d u

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 04:41:15PM -0400, Eric C. Cooper wrote:
> On Fri, Apr 23, 2004 at 01:51:48PM -0500, John Goerzen wrote:
> > [Debian users: I have uploaded it to sid, but will take a few days to
> > appear.]
> 
> I am a bit surprised by this.  What is the process by which OCaml
> packages are added to Debian?  Are there any checks and balances?
> Without in any way passing judgment on your particular effort, it
> seems that a Debian developer with an OCaml agenda might have a
> conflict of interest here, especially considering the scarce resource
> that is the global OCaml module namespace.

There are no more restrictions on OCaml packages in Debian than for any
other language.  The namespace issue is not really unique to OCaml;
pretty much every other language execpt Java[1] would share it, too.
Though the use of findlib packages helps somewhat.  Part of what's good
about Debian is that if a developer finds a package useful and is able
to maintain it up to the standards set by Debian, there's very little
reason to keep it out.  The general process of adding a package to
Debian involves approval by FTP masters, though one cannot expect them
to know the namespace issues of every language.

Having said that, there is an active community of OCaml maintainers in
Debian that would pretty quickly notice any problem, and something going
into sid doesn't automatically mean that it will ever be released.  If
something truly did conflict with another package, it'd likely get a bug
report pretty quick.

This remains a reason that the C Extlib has not been packaged, since it
would conflict with the pure OCaml Extlib.  I even e-mailed the author
of the C package, and he refused to rename it.  I could upload that, and
the FTP masters might possibly even accept it, but it would be
uninstallable on any system that uses the pure OCaml Extlib.  That would
be sure to generate some bug reports that could lead to the removal of
the new package (and would certainly prevent it from being released).

-- John

[1] because everyone has to use com.foo.example.blah.blah

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Shawn Wagner]

On Fri, Apr 23, 2004 at 04:16:28PM -0500, John Goerzen wrote:

> 
> This remains a reason that the C Extlib has not been packaged, since it
> would conflict with the pure OCaml Extlib.  I even e-mailed the author
> of the C package, and he refused to rename it.  I could upload that, and
> the FTP masters might possibly even accept it, but it would be
> uninstallable on any system that uses the pure OCaml Extlib.  That would
> be sure to generate some bug reports that could lead to the removal of
> the new package (and would certainly prevent it from being released).
> 

I don't understand why my Extlib, which has, as far as I know, been around
in one form or another for a lot longer than the other one, and is far more
actively maintained and updated, is the one some people think should be
renamed.

Even if one or the other was renamed, you still couldn't use them together,
because they have some modules with the same names. And both have conflicts
with other libraries you might want to use (Camomile, for example). Insert
standard whine about needing namespace support.

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Kenneth Knowles]

On Fri, Apr 23, 2004 at 03:28:29PM -0700, Shawn Wagner wrote:
> Insert
> standard whine about needing namespace support.

Couldn't you just put your modules in a container module via -pack?
Why have namespaces in addition to modules?

I *would* like to see a simple change to allow dots in module names if possible
(I haven't thought much about it)... This is how Haskell makes deep namespaces
easier, but I don't think they have the same "open" as O'Caml.

Kenn

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Shawn Wagner]

On Fri, Apr 23, 2004 at 03:37:24PM -0700, Kenneth Knowles wrote:
> On Fri, Apr 23, 2004 at 03:28:29PM -0700, Shawn Wagner wrote:
> > Insert
> > standard whine about needing namespace support.
> 
> Couldn't you just put your modules in a container module via -pack?

I haven't seen any libraries by anyone that uses -pack, but it seems
cumbersome (Compile a bunch of .ml files to .cmo, compile them to one with
pack, then that to a .cma? And what about a .mli for describing the packed
.cmo?) Also the ocamlopt version doesn't seem to be very portable, which is
a point against it.

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] ocamlopt -pack portability

[John Carr]

> Also the ocamlopt version [of -pack] doesn't seem to be very portable,
> which is a point against it.

ocamlopt -pack needs GNU binutils.  That is no problem for Linux
users but a significant barrier for many others.

The names of symbols in OCaml object files begin with the name of the
module they belong to.  Unlike Java compilers, ocamlopt does not know
the name of the container module at compile time.  When modules are
packed ocamlopt calls objcopy --redefine-sym to change the names of
symbols to match the new fully qualified module name.  For example, if
you pack a module named Hashtbl into a module named Stdlib, the symbol
Hashtbl must be renamed to Stdlib__Hashtbl.

I usually run Solaris, which does not come with binutils.  I consider
binutils to be too large a package to justify installing for such a
minor feature so I wrote a C program to do the symbol renaming.  (Send
me email if you want it.  It uses libelf and I have only tested it on
Solaris.  You also need a minor change to Asmpackager.extract_symbols
to work with Solaris nm.)

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlopt -pack portability

[Oliver Bandel]

On Fri, Apr 23, 2004 at 09:38:36PM -0400, John Carr wrote:
[...]
> I usually run Solaris, which does not come with binutils.

What's about binutils re-written in Ocaml?

> I consider
> binutils to be too large a package to justify installing for such a
> minor feature so I wrote a C program to do the symbol renaming.  (Send
> me email if you want it.  It uses libelf and I have only tested it on
> Solaris.  You also need a minor change to Asmpackager.extract_symbols
> to work with Solaris nm.)

Why a C-Program and not an Ocaml-program?



Ciao,
   Oliver

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlopt -pack portability

[John Carr]

> > I consider
> > binutils to be too large a package to justify installing for such a
> > minor feature so I wrote a C program to do the symbol renaming.  (Send
> > me email if you want it.  It uses libelf and I have only tested it on
> > Solaris.  You also need a minor change to Asmpackager.extract_symbols
> > to work with Solaris nm.)
> 
> Why a C-Program and not an Ocaml-program?

Solaris has a C library to manipulate ELF files and I knew how to use
it.  Writing a similar program in ocaml would have taken more time.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 03:37:24PM -0700, Kenneth Knowles wrote:
> On Fri, Apr 23, 2004 at 03:28:29PM -0700, Shawn Wagner wrote:
> > Insert
> > standard whine about needing namespace support.
> 
> Couldn't you just put your modules in a container module via -pack?
> Why have namespaces in addition to modules?

I believe one argument against this is that .cm(x)a allows only the part
of the library actually needed to be linked in.  Could be a problem with
large libs bloating the size of apps.


-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Yamagata Yoriyuki]

From: Shawn Wagner <shawnw@speakeasy.org>
Subject: Re: [Caml-list] [ANN] The Missing Library
Date: Fri, 23 Apr 2004 15:28:29 -0700

> I don't understand why my Extlib, which has, as far as I know, been around
> in one form or another for a lot longer than the other one, and is far more
> actively maintained and updated, is the one some people think should be
> renamed.

SF Extlib is actively developped, though not released often.

Nicolas, Could you plan the next release for the near future?
Personally, I am waiting that SF Extlib installer becomes aware of
findlib.  Then, I will make Camomile dependent to SF Extlib (since
there are users using findlib), and maybe make a package for GODI.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> From: Shawn Wagner <shawnw@speakeasy.org>
> Subject: Re: [Caml-list] [ANN] The Missing Library
> Date: Fri, 23 Apr 2004 15:28:29 -0700
>
> > I don't understand why my Extlib, which has, as far as I know, been
around
> > in one form or another for a lot longer than the other one, and is far
more
> > actively maintained and updated, is the one some people think should be
> > renamed.
>
> SF Extlib is actively developped, though not released often.

I confirm that. Most of the users are synchronized with ExtLib CVS which is
far more up-to-date than the current release.
One of the main difference I see between the 2 ExtLibs :
- SF.net ExtLib is part of a community process, it accepts inputs and
contributions from everyone. Although I'm the main contributor and somehow
pushing the project up, I'm not alone and every addon is first openly
discussed on the mailling list.
- Shawn ExtLib is personnal project , so I would actually except Shawn to
rename its library, without caring about the "first-released" argument.

> Nicolas, Could you plan the next release for the near future?
> Personally, I am waiting that SF Extlib installer becomes aware of
> findlib.  Then, I will make Camomile dependent to SF Extlib (since
> there are users using findlib), and maybe make a package for GODI.

I have been actually thinking this for a while. The release will come soon.

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Shawn Wagner]

On Sat, Apr 24, 2004 at 11:19:08AM +0200, Nicolas Cannasse wrote:
> > From: Shawn Wagner <shawnw@speakeasy.org>
> > Subject: Re: [Caml-list] [ANN] The Missing Library
> > Date: Fri, 23 Apr 2004 15:28:29 -0700
> >
> > > I don't understand why my Extlib, which has, as far as I know, been
> around
> > > in one form or another for a lot longer than the other one, and is far
> more
> > > actively maintained and updated, is the one some people think should be
> > > renamed.
> >
> > SF Extlib is actively developped, though not released often.
> 
> I confirm that. Most of the users are synchronized with ExtLib CVS which is
> far more up-to-date than the current release.
> One of the main difference I see between the 2 ExtLibs :
> - SF.net ExtLib is part of a community process, it accepts inputs and
> contributions from everyone. Although I'm the main contributor and somehow
> pushing the project up, I'm not alone and every addon is first openly
> discussed on the mailling list.
> - Shawn ExtLib is personnal project , so I would actually except Shawn to
> rename its library, without caring about the "first-released" argument.
> 

Um. I'll take code, suggestions, and bug reports and fixes from anyone...
I'm pretty sure that's mentioned in the README.

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Alain.Frisch]

On Sat, 24 Apr 2004, Shawn Wagner wrote:

> > - SF.net ExtLib is part of a community process, it accepts inputs and
> > contributions from everyone. Although I'm the main contributor and somehow
> > pushing the project up, I'm not alone and every addon is first openly
> > discussed on the mailling list.
> > - Shawn ExtLib is personnal project , so I would actually except Shawn to
> > rename its library, without caring about the "first-released" argument.
> >
>
> Um. I'll take code, suggestions, and bug reports and fixes from anyone...
> I'm pretty sure that's mentioned in the README.

If Shawn is not willing to change the name of his library, and if it is
true that this library was publicly available and announced on the Humps,
the Caml list db, etc... before the other one, it is clear that it was a
mistake to re-use the name and it should be fixed as soon as possible.
Nobody has any authority to tell the extlib.sf.net developers what to do,
but if their project pretends to target the whole community and aims at
getting widely accepted, they'd better comply with basic social rules,
IMHO.


My 0.02 EUR.


  Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Nicolas Cannasse]

> On Sat, 24 Apr 2004, Shawn Wagner wrote:
>
> > > - SF.net ExtLib is part of a community process, it accepts inputs and
> > > contributions from everyone. Although I'm the main contributor and
somehow
> > > pushing the project up, I'm not alone and every addon is first openly
> > > discussed on the mailling list.
> > > - Shawn ExtLib is personnal project , so I would actually except Shawn
to
> > > rename its library, without caring about the "first-released"
argument.
> > >
> >
> > Um. I'll take code, suggestions, and bug reports and fixes from
anyone...
> > I'm pretty sure that's mentioned in the README.
>
> If Shawn is not willing to change the name of his library, and if it is
> true that this library was publicly available and announced on the Humps,
> the Caml list db, etc... before the other one, it is clear that it was a
> mistake to re-use the name and it should be fixed as soon as possible.
> Nobody has any authority to tell the extlib.sf.net developers what to do,
> but if their project pretends to target the whole community and aims at
> getting widely accepted, they'd better comply with basic social rules,
> IMHO.
>
>
> My 0.02 EUR.
>
>
>   Alain

You're true about that, and other ExtLib people of course checked that there
was so other library with the same name when we started ( I did ). As this
time, Shawn's ExtLib was not registered not announced anywhere. However it
did a release on 6 Jul 2003, only on Gerd's "OCaml Link Database" and was
not featured on the mailing list. As this time we (sf.net extlib people)
were actively working at releasing the first version and several topics on
the caml list had already been announced what we were doing and how it will
be called :-)

That's for my 0.03 Eur :-)

Regards,
Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Florian Hars]

Alain.Frisch@ens.fr wrote:
> if it is
> true that this library was publicly available and announced on the Humps,
> the Caml list db, etc... before the other one, it is clear that it was a
> mistake to re-use the name and it should be fixed as soon as possible.

Shawn's extlib did not exist in early April 2003:
http://web.archive.org/web/20030404000203/http://raevnos.pennmush.org/code/ocaml.html
In fact, Shawn himself claims a release date of April 23, 2003 on
http://raevnos.pennmush.org/code/ocaml.html
(The link db has June 6, 2003 for the initial release:
http://www.npc.de/ocaml/linkdb/list.cgi?type=author&frames=false )

By Shawn's release date, SF extlib had had it's first commits and had been 
mentioned on the mailing list for almost exactly one month:
http://cvs.sourceforge.net/viewcvs.py/ocaml-lib/extlib-dev/extLib.ml
http://caml.inria.fr/archives/200303/msg00323.html
and had been discussed in a public space under that name for another month:
http://sourceforge.net/mailarchive/forum.php?thread_id=1703091&forum_id=29880

Yours, Florian.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Yamagata Yoriyuki]

From: Shawn Wagner <shawnw@speakeasy.org>
Subject: Re: [Caml-list] [ANN] The Missing Library
Date: Fri, 23 Apr 2004 15:28:29 -0700

> And both have conflicts with other libraries you might want to use
> (Camomile, for example). Insert standard whine about needing
> namespace support.

>From 0.5.0, Camomile builds two fravors of the libraries
(camomile.cm[x]a, and camomileunpack.cm[x]a).  In camomile.cm[x]a, all
modules are put under Camomile module by -pack.  If you have a name
space problem, use this one.

Unfortunately, this makes Camomile load some unnecessary data for
initialization, but I judge that the namespace issue is more
important.

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[John Goerzen]

On Fri, Apr 23, 2004 at 03:28:29PM -0700, Shawn Wagner wrote:
> On Fri, Apr 23, 2004 at 04:16:28PM -0500, John Goerzen wrote:
> I don't understand why my Extlib, which has, as far as I know, been around
> in one form or another for a lot longer than the other one, and is far more
> actively maintained and updated, is the one some people think should be
> renamed.

To me, the simple reason is: the other is more popular and is likely to
remain so.

That does not mean yours is inferior, but it has a different audience.

That said, if it's such a big problem, you *both* could change names.

In your case, the sole reason I have never used your library is because
it is named Extlib.  If it were not, it would have been an installable
.deb by now, and I'd use it.  In at least this one case, the name is
preventing the library from becoming more popular.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] [ANN] The Missing Library

[Jon Harrop]

On Saturday 24 April 2004 5:51 am, John Goerzen wrote:
> On Fri, Apr 23, 2004 at 03:28:29PM -0700, Shawn Wagner wrote:
> > ...
> That said, if it's such a big problem, you *both* could change names.

Yes, you could call yours "ExtLib2", for example. ;-)

Cheers,
Jon.

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Proposal: community standard library project (was: Re: [Caml-list] [ANN] The Missing Library)

[Benjamin Geer]

Shawn Wagner wrote:
> Even if one or the other was renamed, you still couldn't use them together,
> because they have some modules with the same names. And both have conflicts
> with other libraries you might want to use (Camomile, for example). Insert
> standard whine about needing namespace support.

Here were are with the exactly the sort of problems for which a 
community structure is needed.

I'm envisaging a new 'community standard library' project which would 
supplement (and perhaps partly replace) INRIA's standard library, and in 
which:

   * Libraries would be categorised according to function, e.g.
     data structures, Unicode, I/O, network protocols, etc.

   * No duplicate or incompatible functionality would be allowed
     in each functional area.  People would have to cooperate
     to make their stuff work together.

   * Minimum standards of portability, documentation, etc. would be
     checked.

   * Everything would be released via GODI.

In order to resolve the sorts of technical and social conflicts Shawn 
mentions above, I think this project would need a structure for making 
decisions, perhaps something like the one I proposed:

http://caml.inria.fr/archives/200403/msg00227.html

Whether that particular structure is used or not, I think the main thing 
is that there should be an efficient process for avoiding duplication 
and incompatibility, for maintaining minimum standards, and for 
resolving conflicts.

Many existing Caml libraries could be folded into this new project, 
after some work to make them compatible with each other and to remove 
duplication.

The result would be a consistent and increasingly complete community-run 
'library distribution' without the fragmentation that we are starting to 
see now.

Is there much interest in such a project?  If so, I think the members of 
this list easily have the technical and intellectual resources to set it 
up.  I would be glad to put some work into getting it started.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] RE: Proposal: community standard library project

[Brandon J. Van Every]

Benjamin Geer wrote:
>
>    * No duplicate or incompatible functionality would be allowed
>      in each functional area.  People would have to cooperate
>      to make their stuff work together.

"No incompatible functionality" is easy to agree upon.

I don't easily swallow "no duplicate functionality," however.  Libraries
have to prove their worth in the field somehow.  I would rather see
various libraries compete by free market mechanism, until for a given
set of tasks, one is clearly 'de facto' better than another.  i.e. is
better maintained, is revved more frequently or usefully, etc.  If you
make some kind of a priori announcement on what the 'standard
functionality' is going to be, that's pretty much historical accident of
which committee people were awake and functioning at the time.  I would
rather see 2 different GUI libraries battle it out in the real world.
Or 2 different wrappers for the same GUI library battle it out.  I don't
trust design-by-committee.

That said, I don't think *gratuitous* duplication is adviseable.  I
don't want 2 different libraries with different words for exactly the
same functions.

It all depends on the granularity at which you define the word
'duplicate'.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

"We live in a world of very bright people building
crappy software with total shit for tools and process."
                                - Ed Mckenzie
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.643 / Virus Database: 411 - Release Date: 3/25/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] RE: Proposal: community standard library project

[Benjamin Geer]

Brandon J. Van Every wrote:
> I don't easily swallow "no duplicate functionality," however.  Libraries
> have to prove their worth in the field somehow.

I think the Linux kernel developers have a good way of handling this: if 
there are two competing libraries that do the same thing, they both stay 
outside the project, as patches, until there is a consensus about which 
is better; then the maintainers include one of the libraries in the 
project, and the other one takes a bow and honourably leaves the 
stage.[1]  The maintainers' job is to ensure that this process is 
neither too long nor too short.  It's OK to have two libraries that do 
the same thing for a short time, in order to choose the best approach; 
but in the long term, duplication leads to wasted effort.

Therefore, both libraries should have a chance to prove themselves; but 
if after a certain amount of time there is no clear consensus about 
which is better, and it is not feasible to combine the best aspects of 
both, the maintainers should use their own judgement and pick one of 
them, which is what Linus does.  It is better to resolve conflicts and 
move on than to let them persist endlessly.

Ben

[1] For a good example of this, see: 
http://www-124.ibm.com/pthreads/docs/announcement

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

RE: [Caml-list] RE: Proposal: community standard library project

[Brandon J. Van Every]

Benjamin Geer
> Brandon J. Van Every wrote:
> > I don't easily swallow "no duplicate functionality,"
> > however.  Libraries
> > have to prove their worth in the field somehow.
>
> I think the Linux kernel developers have a good way of
> handling this: if
> there are two competing libraries that do the same thing,
> they both stay
> outside the project, as patches, until there is a consensus
> about which
> is better; then the maintainers include one of the libraries in the
> project, and the other one takes a bow and honourably leaves the
> stage.[1]  The maintainers' job is to ensure that this process is
> neither too long nor too short.

Sounds good to me.  If only the United Nations worked with such
punctuality and magnanimity, we might have more agreement about wars
such as Iraq.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.643 / Virus Database: 411 - Release Date: 3/25/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Re: Proposal: community standard library project

[Jacques GARRIGUE]

From: Benjamin Geer <ben@socialtools.net>

> I'm envisaging a new 'community standard library' project which would 
> supplement (and perhaps partly replace) INRIA's standard library, and in 
> which:
> 
>    * Libraries would be categorised according to function, e.g.
>      data structures, Unicode, I/O, network protocols, etc.
> 
>    * No duplicate or incompatible functionality would be allowed
>      in each functional area.  People would have to cooperate
>      to make their stuff work together.
> 
>    * Minimum standards of portability, documentation, etc. would be
>      checked.
> 
>    * Everything would be released via GODI.

Writing down this makes clearer part of the problem with ocaml and/or
its community: your second point would be very hard to satisfy.

Why? Because ocaml gives you many ways to define an API for any
functionality. And does not try to decide which is better. So you can
only end up with duplicate libraries, according to personal tastes.
Not only there are plenty of duplicate libraries around, but I'm sure
that many ocaml programmers prefer to create their own private library
rather than using some available one.

You can be functional or imperative, use optional arguments or not,
use objects or modules, etc...
Even interfacing a C library can be done in many different ways...

This does not mean that people should not cooperate, just that you
cannot force them to.

The 1st point is reasonable. This is already what the humps are
doing, but a system allowing requests for new functionalities would be
nice too.
The 3rd point, I would rather see it as having more
meta-documentation, ie information on libraries without having to
download and install: requirements, extensiveness, quality of the
documentation, user reviews...
The last point is clear enough.

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

RE: [Caml-list] Re: Proposal: community standard library project

[Brandon J. Van Every]

Jacques GARRIGUE wrote:
>
> but I'm sure
> that many ocaml programmers prefer to create their own private library
> rather than using some available one.

But that is, frankly, not relevant.  Those of us who want something
standard and off-the-shelf don't care about someone doing their own
thing, and vice versa.  A standards effort doesn't exist for the lone
maverick who's intent on doing things his own way.

> This does not mean that people should not cooperate, just that you
> cannot force them to.

The people who are interested in a standard, who see that as a benefit,
should cooperate.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.668 / Virus Database: 430 - Release Date: 4/24/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[Martin Jambon]

On Mon, 26 Apr 2004, Jacques GARRIGUE wrote:

> Why? Because ocaml gives you many ways to define an API for any
> functionality. And does not try to decide which is better. So you can
> only end up with duplicate libraries, according to personal tastes.
> Not only there are plenty of duplicate libraries around, but I'm sure
> that many ocaml programmers prefer to create their own private library
> rather than using some available one.
>
> You can be functional or imperative, use optional arguments or not,
> use objects or modules, etc...
> Even interfacing a C library can be done in many different ways...

So, we can have maybe one "standard OO library" or a
"standard ???-style library".
Users like to follow only one simple and precise paradigm.
Users will not tolerate exceptions to The Paradigm.

(Citation des Inconnus : « la devise de la maison : ne jamais prendre les
gens pour des cons, mais ne pas oublier qu'ils le sont. »)

OCaml is great because developers may choose between
several programming styles, but I think that the interface of a library
must explicitely follow only one style. This is true for any library,
standard or not.
It might sometimes require Caml-styleA to Caml-styleB wrappers...


Martin

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[Benjamin Geer]

Jacques GARRIGUE wrote:
> ocaml gives you many ways to define an API for any
> functionality. And does not try to decide which is better. So you can
> only end up with duplicate libraries, according to personal tastes.

I don't mind whether a standard library uses classes or functors, or 
both; what I care about is that different parts of it should be able to 
interact easily with each other.  Therefore I'm delighted to see the 
discussion currently going on about integrating I/O channels across 
ExtLib, Camomile and ocamlnet.

It seems to me that different styles are suited to different situations. 
  For example, I think most people, regardless of their preferred 
programming style, would agree that you made the right choice by using 
an object-oriented style for LablGTK.  Someone who wants a Caml 
interface to GTK will almost certainly be happy to adapt to the style 
you chose.

Moreover, the same issue exists (though perhaps to a lesser extent) in 
many popular languages.  C++, for example, supports object-oriented and 
generic styles.  Lisp supports functional and object-oriented styles. 
Python supports imperative, object-oriented and functional styles.

I'm sure that, with sufficient goodwill, discussions like the ones going 
on now can lead to solutions that most people will be satisfied with.

> The 3rd point, I would rather see it as having more
> meta-documentation, ie information on libraries without having to
> download and install: requirements, extensiveness, quality of the
> documentation, user reviews...

Perhaps eventually GODI could support that kind of information.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[Gerd Stolpmann]

On Mon, 2004-04-26 at 20:25, Benjamin Geer wrote:
> > The 3rd point, I would rather see it as having more
> > meta-documentation, ie information on libraries without having to
> > download and install: requirements, extensiveness, quality of the
> > documentation, user reviews...
> 
> Perhaps eventually GODI could support that kind of information.

There was a short, and quite technical thread on godi-list:

https://gps.dynxs.de/pipermail/godi-list/2004-April/000053.html

Summarised, it could be made working that one can only install the
documentation part of the packages, without the packages as such, and
without fulfilling all the requirements. Then, it is quite easy to set
up a documentation server.

However, don't expect this happen soon, too few people are working on
GODI currently, and it hard to find time to add new packaging features.
The developers currently concentrate on the core, on new packages, and
on simplifying the packaging method. Btw, I'll report about the progress
of GODI the next days.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[skaller]

On Tue, 2004-04-27 at 05:37, Gerd Stolpmann wrote:

> Btw, I'll report about the progress
> of GODI the next days.

Pls let us know when ability to download from CVS is likely
to happen, for Ocaml itself (since I use the CVS version),
and also for packages which choose to expose their CVS versions.

This would be a major plus I think, since so many packages
(including Ocaml) are in flux, and bleeding edge developers
can't really use GODI properly unless they can specify CVS
for key libraries. The most important probably being one's
OWN library under development. This makes new libraries
available early, and updates available instantly,
without any need to 'release' a new version to GODI.

Oh and of course GODI itself can upgrade itself from
CVS .. the total result will not be particularly
stable, but it will allow a significant amount
of *early* cooperation between developers.

Side note: I am close to *abandoning* tarball releases
for the development sources of Felix and simply
providing a single Makefile with  'download_from_CVS' 
and 'Upgrade_from_CVS' targets... saves messing around
uploading tarballs all the time .. I'll still use
tarballs for stable releases though.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[John Goerzen]

On Tue, Apr 27, 2004 at 06:24:50AM +1000, skaller wrote:
> On Tue, 2004-04-27 at 05:37, Gerd Stolpmann wrote:
> 
> > Btw, I'll report about the progress
> > of GODI the next days.
> 
> Pls let us know when ability to download from CVS is likely
> to happen, for Ocaml itself (since I use the CVS version),
> and also for packages which choose to expose their CVS versions.

Let's not limit to CVS, please.  CVS is but one version control system.
Subversion and Arch are both popular alternatives here.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

RE: [Caml-list] Re: Proposal: community standard library project

[Brandon J. Van Every]

John Goerzen wrote:
>
> Let's not limit to CVS, please.  CVS is but one version
> control system.
> Subversion and Arch are both popular alternatives here.

Fine... realize Arch is UNIX only, there will be no Windows version in
any forseeable future.  Also there are various Windows version control
systems that cost money.  One version control system vs. any / every
version control system is really a different scope of project.

Some developers I've talked to think Subversion is the future in open
source land though, for whatever reason.  I haven't done any actual CVS
work to know why Subversion might be better.

Downloading from CVS is easy enough, provided that you're not asked to
go through irritating security protocols.  I haven't figured out how to
grab GNU Make off of the Savannah website because they have some
security protocol activated.  In contrast, grabbing The Nebula Device
from Sourceforge is easy.  If you want to deploy stuff to people who are
interested in getting "real work" done, make sure it's easy to just get
and go.  If you are trying to convert people to your language, people
don't have time for elaborate CVS configuration rituals.

For simple downloading, the reliability of the repository site is more
important than the version control system employed.  Sourceforge often
has problems with anonymous CVS.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.668 / Virus Database: 430 - Release Date: 4/24/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[skaller]

On Tue, 2004-04-27 at 06:39, John Goerzen wrote:

> Let's not limit to CVS, please.  CVS is but one version control system.
> Subversion and Arch are both popular alternatives here.

The emphasis on CVS here is because it is used for 
Ocaml itself, for GODI itself, and also is the standard 
system on Sourceforge. That covers a lot of projects.

There may well be other systems, and perhaps GODI
can handle them at some stage. But right now
NOT having CVS access is a complete show stopper
for those people trying to collaborate developing
several libraries and synchronising them: for example
the ExtLib, Ocaml itself, the other Extlib, Camomile.

To factor these projects, stored in multiple places,
requires a fast turnaround for change propagation,
much faster than can be obtained from tarball releases.

So if you want a community structure mediated by GODI
in which to participate you're going to have to use
CVS for a while, if only to participate in upgrading
GODI to also handle Subversion and Arch or whatever :D

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[Alain.Frisch]

On 27 Apr 2004, skaller wrote:

> On Tue, 2004-04-27 at 06:39, John Goerzen wrote:
>
> > Let's not limit to CVS, please.  CVS is but one version control system.
> > Subversion and Arch are both popular alternatives here.
>
> The emphasis on CVS here is because it is used for
> Ocaml itself, for GODI itself, and also is the standard
> system on Sourceforge. That covers a lot of projects.

GODI uses Subversion, not CVS.

FWIW, there is an apps-cduce-cvs package for GODI; I will upload quite
often fresh snapshots from the CDuce CVS. The same model can be used for
other applications and libraries. Automatic checkout of CVS/subversion/...
is a step further, but I wouldn't say it is a priority for now.

-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[Gerd Stolpmann]

Am Die, 2004-04-27 um 11.06 schrieb skaller:
> On Tue, 2004-04-27 at 06:39, John Goerzen wrote:
> 
> > Let's not limit to CVS, please.  CVS is but one version control system.
> > Subversion and Arch are both popular alternatives here.
> 
> The emphasis on CVS here is because it is used for 
> Ocaml itself, for GODI itself, and also is the standard 
> system on Sourceforge. That covers a lot of projects.

As Alain points out, GODI prefers Subversion for its own configuration
management.

> There may well be other systems, and perhaps GODI
> can handle them at some stage. 

This is quite difficult. The simple part is to download with cvs, but
then? As cvs software changes frequently, but the package makes
assumptions about the software, it is uncertain whether the build and/or
packaging will succeed. I mean, in general, without knowing the
software.

> But right now
> NOT having CVS access is a complete show stopper
> for those people trying to collaborate developing
> several libraries and synchronising them: for example
> the ExtLib, Ocaml itself, the other Extlib, Camomile.

For the OCaml CVS version the plan is to keep it completely outside of
GODI, and to just make fake packages to fulfill the dependencies
(usually packages programmed in OCaml have a dependency on the ocaml
package), so one can create the rest of GODI without using its ocaml
packages.

> To factor these projects, stored in multiple places,
> requires a fast turnaround for change propagation,
> much faster than can be obtained from tarball releases.

I don't understand that. If fast turnaround is really important, isn't
it better to have a self-made script that controls the build process?
E.g. checks out everything needed, and compiles one part after the
other?

I think you want something that cannot be (easily) done with GODI. This
is a quite conservative project in the sense that a GODI package also
ensures some qualities, i.e. it can be built, one can delete it, one can
make a binary package from it, etc.

> So if you want a community structure mediated by GODI
> in which to participate you're going to have to use
> CVS for a while, if only to participate in upgrading
> GODI to also handle Subversion and Arch or whatever :D

I don't believe that the majority of the community is interested in CVS
versions of any software. The advanced people are. And even these will
profit from GODI, because even if you use one or the other CVS version,
this is not true for the majority of libraries.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Re: Proposal: community standard library project

[skaller]

On Tue, 2004-04-27 at 21:29, Gerd Stolpmann wrote:

> As Alain points out, GODI prefers Subversion for its own configuration
> management.

My mistake, an assumption. Sorry.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] CVS labeling (was Re: Proposal: community standard library project)

[Brandon J. Van Every]

Gerd Stolpmann wrote:
>
> This is quite difficult. The simple part is to download with cvs, but
> then? As cvs software changes frequently, but the package makes
> assumptions about the software, it is uncertain whether the
> build and/or
> packaging will succeed. I mean, in general, without knowing the
> software.

Well, for CVS in any old state, you must assume failure.

You could collect statistics on how often a given project's CVS
repository will actually build, and how often it works even if it
builds, but the latter is a permanent grey area.  What's a showstopper
bug?  It's a qualitative judgement... I suppose you could have people
vote on how bad a given CVS package is, but that presumes enough use and
enough voting for it to be relevant.

Does CVS have a labeling capability? i.e. mark all files in the
repository at a certain point, so that files can easily be recalled at
that point later?  If it's easy to do, then you might cajole open source
project admins into actually labeling their CVS repositories at points
of relative stability.  This still depends on project discipline of
course.  But definitely less discipline than making official stable
release packages.  Open source projects tend to drift on forever with
stuff getting added into CVS.  Making official release packages is a
chore that most people don't like.  Also people have trouble deciding
when the official release should happen and meeting the goals for it.
Volunteer labor and all of that.

A label also doesn't mean 'stable for everybody'.  It might work fine on
Linux but not Windows or Mac.  So ideally, a label would also state what
platforms it builds on.  This information could be wrong or not
submitted, however.  So, then you'd need to roll back to the next
earlier version.  Wet, lather, rinse, repeat until the darned thing
builds, or until it has gone too far back for the user to be interested
anymore.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

"We live in a world of very bright people building
crappy software with total shit for tools and process."
                                - Ed Mckenzie
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.668 / Virus Database: 430 - Release Date: 4/24/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] CVS labeling (was Re: Proposal: community standard library project)

[John Goerzen]

On Tue, Apr 27, 2004 at 11:13:50AM -0700, Brandon J. Van Every wrote:
> You could collect statistics on how often a given project's CVS
> repository will actually build, and how often it works even if it
> builds, but the latter is a permanent grey area.  What's a showstopper

Of course, there are several erroneous assumptions (not necessarily
yours) that are going into this conversation:

1. That checking out HEAD from CVS will produce the latest development
   version of the software, or a version more recent than the latest
   "release";

2. That HEAD is where development occurs;

3. That HEAD is more volatile than a "release".

cvs does support tagging and branching and I'd think it not too unusual
to have a "main" branch and a development branch, and make new commits
on the main only when devel is ready to release.

> Does CVS have a labeling capability? i.e. mark all files in the
> repository at a certain point, so that files can easily be recalled at
> that point later?  If it's easy to do, then you might cajole open source

It does, and is easy.

-- John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners