[Caml-list] PostgreSQL-OCaml 1.0.1

[Caml-list] PostgreSQL-OCaml 1.0.1

[Markus Mottl]

Hi,

I am pleased to announce the first release of PostgreSQL-OCaml, a very
complete library for accessing the PostgreSQL-database. This library
builds on Alain Frisch's previous interface (postgres-20010808).

The code has been cleaned up, a few not so important additions have been
made, a few functions, methods and types have been renamed (as well as
the library) and may expect or return different types of parameters. The
interface is very well-documented now, and the library can be easily
installed and used with findlib. It should be fairly straightforward to
adapt existing code that uses the old version.

You can download the library here:

  http://www.oefai.at/~markus/home/ocaml_sources.html#postgresql-ocaml

Enjoy!

Best regards,
Markus Mottl

-- 
Markus Mottl          http://www.oefai.at/~markus          markus@oefai.at

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Stefano Zacchiroli]

On Wed, Jan 28, 2004 at 07:32:18PM +0100, Markus Mottl wrote:
> complete library for accessing the PostgreSQL-database. This library
> builds on Alain Frisch's previous interface (postgres-20010808).

This sentence doesn't clarify an issue: is this library supposed to
supersede Alain's bindings or to be a fork?

TIA,
Cheers.

-- 
^Stefano Zacchiroli - Computer Science PhD student @ Uny Bologna, Italy$
^zack@{cs.unibo.it,debian.org,bononia.it} -- http://www.bononia.it/zack$
^Frequentando il mio maestro mi ero reso conto [.] che la logica poteva$
^servire a molto a condizione di entrarci dentro e poi di uscirne -Adso$

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Markus Mottl]

On Wed, 28 Jan 2004, Stefano Zacchiroli wrote:
> On Wed, Jan 28, 2004 at 07:32:18PM +0100, Markus Mottl wrote:
> > complete library for accessing the PostgreSQL-database. This library
> > builds on Alain Frisch's previous interface (postgres-20010808).
> 
> This sentence doesn't clarify an issue: is this library supposed to
> supersede Alain's bindings or to be a fork?

I actually haven't asked Alain yet what he thinks about it :-)

Since I currently need PostgreSQL-bindings, I just took his version and
adapted it to my taste. The interface was hardly documented (its most
serious shortcoming), which usually prevents libraries from widespread
use so I also did that.

Alain hasn't changed anything in three years so I suppose this library
would otherwise have been unmaintained anyway?

There is hardly anything left to add to this library so there are no
foreseeable major changes. The new additions and cleanups also do not
make such a big difference. If Alain wants to add or change anything,
he will still recognize the code and can do so easily. I'd say it's safe
to use the new version...

Regards,
Markus

-- 
Markus Mottl          http://www.oefai.at/~markus          markus@oefai.at

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Sven Luther]

On Thu, Jan 29, 2004 at 12:21:31AM +0100, Markus Mottl wrote:
> On Wed, 28 Jan 2004, Stefano Zacchiroli wrote:
> > On Wed, Jan 28, 2004 at 07:32:18PM +0100, Markus Mottl wrote:
> > > complete library for accessing the PostgreSQL-database. This library
> > > builds on Alain Frisch's previous interface (postgres-20010808).
> > 
> > This sentence doesn't clarify an issue: is this library supposed to
> > supersede Alain's bindings or to be a fork?
> 
> I actually haven't asked Alain yet what he thinks about it :-)
> 
> Since I currently need PostgreSQL-bindings, I just took his version and
> adapted it to my taste. The interface was hardly documented (its most
> serious shortcoming), which usually prevents libraries from widespread
> use so I also did that.
> 
> Alain hasn't changed anything in three years so I suppose this library
> would otherwise have been unmaintained anyway?
> 
> There is hardly anything left to add to this library so there are no
> foreseeable major changes. The new additions and cleanups also do not
> make such a big difference. If Alain wants to add or change anything,
> he will still recognize the code and can do so easily. I'd say it's safe
> to use the new version...

BTW, I wonder if the alioth ocaml project could also be a place for
hosting upstream stuff like this one ? For information, alioth is a
sourceforge like infrastructure for hosting debian related projects.

Friendly,

Sven Luther

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Alain.Frisch]

On Thu, 29 Jan 2004, Markus Mottl wrote:

> Alain hasn't changed anything in three years so I suppose this library
> would otherwise have been unmaintained anyway?

Ahem. The last release is dated 2004-01-20 ;-)  (added support for
escaping strings).

The library is used by a non-negligible number of people, including by
some companies and for other libraries (e.g. mod_caml, Chartpak, DBForge,
...). And according to some mails I got, the users are quite satisfied
with the current library. Moreover, it has been packaged for some Linux
distributions.

It's a good thing to add documentation (even though the users would need
to read the libpq doc anyway). Actually, this is the only feature request
for Postgres that I have not honoured. But changing the interface
(changing names, types, and removing all the non-OO low-level interface)
would force existing users to rewrite there code for no benefit (AFAICT).
So I'm not going to stop maintaining the existing library, even though I'd
be glad to give maintenance to you since I'm not using the library, and
even though I feel it's somewhat a waste of resources for the community
to maintain and package two libraries.

I would have appreciated to know about your intention to "fork" the
library before the public annoucement so that we could have agreed on a
policy. Well anyway, not a big deal.



-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

>From my point of view (mod_caml depends on the postgres support) it
would be nice to have just one library to work with.  Perhaps combine
the best features of both into a single release?

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
NET::FTPSERVER is a full-featured, secure, configurable, database-backed
FTP server written in Perl: http://www.annexia.org/freeware/netftpserver/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Markus Mottl]

On Thu, 29 Jan 2004, Alain.Frisch@ens.fr wrote:
> Ahem. The last release is dated 2004-01-20 ;-)  (added support for
> escaping strings).

Uh, oh, this must have been some kind of quantum entanglement making us
start adding changes the same day ;-)

> The library is used by a non-negligible number of people, including
> by some companies and for other libraries (e.g. mod_caml, Chartpak,
> DBForge, ...). And according to some mails I got, the users are quite
> satisfied with the current library. Moreover, it has been packaged
> for some Linux distributions.

Yes, your library was a very good starting point. I think the new version
is slightly cleaner and more regular though. I have just released a new
version (1.2.2) that makes some incompatible changes for better support
of default arguments, because, as it seems, you want to maintain the
initial interface, anyway. Thus, there is not so much need for me to
keep things as compatible as possible.

> It's a good thing to add documentation (even though the users
> would need to read the libpq doc anyway).

True, but having some short docs in the interface saves time in many
cases.

> Actually, this is the only feature request for Postgres that I have
> not honoured. But changing the interface (changing names, types,
> and removing all the non-OO low-level interface) would force existing
> users to rewrite there code for no benefit (AFAICT).

Understandable.

> So I'm not going to stop maintaining the existing library, even though
> I'd be glad to give maintenance to you since I'm not using the library,
> and even though I feel it's somewhat a waste of resources for the
> community to maintain and package two libraries.

How about making my recent release the new major version? Most libraries
have to introduce incompatibilities at some point of time when they
evolve anyway. I still remember changing the interface of Pcre to make
use of default arguments, a totally incompatible move. Users dropped
the old version instantly without complaining.

> I would have appreciated to know about your intention to "fork" the
> library before the public annoucement so that we could have agreed on
> a policy. Well anyway, not a big deal.

Sorry about that! I actually hadn't even intended to rewrite it, this,
ahem, just happened! I wanted to use your library for storing online
business transactions and therefore took a very close look at all of
the code. Since I'm a pedant by nature, I started rewriting some bits
here and there and suddenly it came to me that I could as well make a
new release :-)

Next time I'll contact original authors a bit earlier. It was really a
stupid coincidence that we started out adding changes more or less the
same day after several years.

On Thu, 29 Jan 2004, Richard Jones wrote:
> From my point of view (mod_caml depends on the postgres support) it
> would be nice to have just one library to work with.  Perhaps combine
> the best features of both into a single release?

Even though the interfaces are not fully type compatible, it is very
easy to adapt existing code. The OCaml-compiler + supported editors
will allow you to jump from effected line to line very quickly,
and changes usually require hardly any effort. The two versions are
otherwise feature-compatible and also pretty feature-complete wrt. what
PostgreSQL offers.

Best regards,
Markus

-- 
Markus Mottl          http://www.oefai.at/~markus          markus@oefai.at

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Josh Burdick]

Richard Jones wrote:

>>From my point of view (mod_caml depends on the postgres support) it
>would be nice to have just one library to work with.  Perhaps combine
>the best features of both into a single release?
>  
>
   If there's going to be a "new improved version", I think maybe it 
should work with multiple databases.  ODBC, JDBC, and DBI all attempt 
this.  There could be a signature (perhaps called "DBI" to appeal to 
Perl people , and structs Postgres, MySQL, Oracle, etc., which implement 
this.
   Different databases have varying levels of standards-conformance, of 
course, and there's all sorts of variation between databases, so I 
wouldn't expect one to be able to take one OCaml program written with 
this interface, and blithely switch databases, especially for 
complicated stuff.  But you should be able to write simple "select * 
from customers"-type queries, against any database, without totally 
re-working your OCaml code.
   Database independence seems like a good goal to move towards.  If 
there's going to be potential interface changes anyways, now might be a 
good time to plan a more general interface.
   (I used the old Postgres module, and it worked well.)

   Josh Burdick
   jburdick@gradient.cis.upenn.edu

>Rich.
>
>  
>


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Shawn Wagner]

On Fri, Jan 30, 2004 at 12:50:41AM -0500, Josh Burdick wrote:

>   If there's going to be a "new improved version", I think maybe it 
> should work with multiple databases.  ODBC, JDBC, and DBI all attempt 
> this.  There could be a signature (perhaps called "DBI" to appeal to 
> Perl people , and structs Postgres, MySQL, Oracle, etc., which implement 
> this.

I think there are a couple of projects to do this. I maintain the MySQL
bindings for ocaml, and I'm willing to support any such database-independant
interface to SQL in it.

>   Different databases have varying levels of standards-conformance, of 
> course, and there's all sorts of variation between databases, so I 
> wouldn't expect one to be able to take one OCaml program written with 
> this interface, and blithely switch databases, especially for 
> complicated stuff.  But you should be able to write simple "select * 
> from customers"-type queries, against any database, without totally 
> re-working your OCaml code.

I've had vague ideas of adding functions that build the query strings for
you, so you can do things like

let querystring = Sql.select ~table:"addresses"
~columns:["street";"city";"state";"zipcode"] ~where:"name='so and so'"
and then feed that to whatever database library you like: MySQL, Postgres,
Oracle, whatever.

If I actually used SQL these days, I'd probably have done it already. Anyone
interested in this sort of thing?

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Vitaly Lugovsky]

On Fri, 30 Jan 2004, Josh Burdick wrote:

>    If there's going to be a "new improved version", I think
> maybe it should work with multiple databases.  ODBC, JDBC, and
> DBI all attempt this.

 It was already discussed here. The conclusion was: BAD IDEA. No
way to work efficiently with different DBs using the same
approach. Btw., I don't have any problems providing support for
MySQL, PostgreSQL and Oracle from one OCaml application - but the
abstraction is on a higher level then JDBC-like interfaces
provides.

> "select * from customers"-type queries, against any database,
> without totally re-working your OCaml code.

 No way. Too much differences in the way how DBs returns the
query results (e.g., consider the postgresql asyncronous
messages). More layers converting values would just give much
more overhead, without any significiant advantages.


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Vitaly Lugovsky wrote:
>>   If there's going to be a "new improved version", I think
>>maybe it should work with multiple databases.  ODBC, JDBC, and
>>DBI all attempt this.
> 
>  It was already discussed here. The conclusion was: BAD IDEA. No
> way to work efficiently with different DBs using the same
> approach.

In the company I work for (a large financial software vendor), the 
unanimous answer would be 'we don't care if it's less efficient; nothing 
else is acceptable.'  Our customers insist on being able to use our 
products with whatever database they prefer (and certainly our 
competitors' products can do this).  We simply cannot afford to rewrite 
and maintain all our database-related code for every one of those 
databases.  For us (and, I think, for most software vendors, certainly 
all the ones I've worked for) the additional abstraction is well worth a 
slight loss of efficiency.  It is quite efficient enough for us.  The 
lack of a standard database API is one of the things that, 
unfortunately, would make it very difficult for me to convince my boss 
to let me use Caml.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

If people think that the mod_caml DBI approach is the right one (and
it may not be, although it certainly Works For Us), then I could spin
this off into a separate package to make it easier to install.

This is dbi.mli:

http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup

This is dbi_postgres.mli, which is the driver we use all the time in
production at Merjis, hence the most stable / developed:

http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi_postgres.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup

This is the driver for Perl DBDs:

http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi_perl.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup

The interface is designed to make it quick to access databases, in the
sense that you write the minimum code necessary.  It's not type safe,
in the sense that if you write garbage SQL, it won't be detected until
runtime.  Alex Baretta is, I believe, pursuing an alternative approach
using Camlp4 which is supposed to be type safe.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://www.YouUnlimited.co.uk/ - management courses

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Gerd Stolpmann]

On Fre, 2004-01-30 at 11:47, Richard Jones wrote:
> If people think that the mod_caml DBI approach is the right one (and
> it may not be, although it certainly Works For Us), then I could spin
> this off into a separate package to make it easier to install.
> 
> This is dbi.mli:
> 
> http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup
> 
> This is dbi_postgres.mli, which is the driver we use all the time in
> production at Merjis, hence the most stable / developed:
> 
> http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi_postgres.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup
> 
> This is the driver for Perl DBDs:
> 
> http://savannah.nongnu.org/cgi-bin/viewcvs/modcaml/modcaml/dbi_perl.mli?rev=HEAD&content-type=text/vnd.viewcvs-markup
> 
> The interface is designed to make it quick to access databases, in the
> sense that you write the minimum code necessary.  It's not type safe,
> in the sense that if you write garbage SQL, it won't be detected until
> runtime.  Alex Baretta is, I believe, pursuing an alternative approach
> using Camlp4 which is supposed to be type safe.

Hi,

I think we really need this type of interface. I have some experience
with Perl DBI, and although our company did have the need to support
several database systems in the same product, this made things a lot
easier, simply because you don't have to learn a new API when you switch
to a another database system.

So I would it really appreciate if this interface were released as
independent package.

Of course, I have also some technical remarks:

- I don't understand why there are several ways to represent NULL.
  For example, what is the difference between `Null, 
  `TimestampOption None, and `Timestamp ts where ts.ts_is_null=true?

- Using host,port,user,password to identify and authenticate a connection
  works in most cases, but is not very wise for the long-term stability
  of the interface. For example, it is already possible to authenticate
  with X509 certificates, which cannot be expressed with this scheme.
  I would suggest to only have two strings, one called "system_id",
  the other called "authenticator". In general, these strings are
  interpreted by the driver, but there are conventions, e.g.
  "host=XXX,port=XXX" for the system_id, and "user=XXX,password=XXX"
  or "user=XXX,x509privatekey=XXX,x509passphrase=XXX" for the authenticator.

- There is no way to pass options to the driver when one uses the Factory.
  I think it would be a good idea to have a string for driver options,
  again in the format "key=value,...".

- There should be method for escaping strings (so one can generate SQL
  statements)

- One exception should be special: That transactions are aborted because
  of conflicts (think of Postgres' MVCC) or deadlocks and should be repeated. 
  I think the other possible errors can be subsumed in one Sql_error exception
  as it is currently done.

- There should be a possibility to coerce a generic connection object
  back to the driver's class. This can be done by adding a method
  driver_handle that just raises a driver-specific exception
  returning the handle. This way it is possible to switch to low-level
  code at any time, e.g. for optimizations for certain DBMS.

- Following the coding guidelines for O'Caml, methods with side-effects
  should have a functional type, e.g. finish : unit->unit instead
  of just finish:unit. This has also the advantage of making it 
  simpler to later add optional arguments to methods.

- There was the question how to handle BLOBs. I worked with DBMS where
  TEXT is a BLOB type. I think there should be a BLOB class like

  class type blob = object
    method length : int64
    method get_chunk : pos:int64 -> len:int -> string
    method write_file : string -> unit
  end

  and there should be methods (of the connection) creating such BLOBs:
  
  method create_blob_from_string : string -> blob
  method create_blob_from_file : string -> blob

  I think this is the minimal BLOB interface we should support. More
  advanced BLOB interfaces can be implemented in the drivers.

I hope we get a really good generic DBMS layer soon.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

I pretty much agree with all of Gerd's comments.  I'll accelerate the
process of splitting out Dbi from mod_caml so that it can be used and
developed separately.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Learning Objective CAML for C, C++, Perl and Java programmers:
http://www.merjis.com/richj/computers/ocaml/tutorial/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Markus Mottl]

On Fri, 30 Jan 2004, Benjamin Geer wrote:
> In the company I work for (a large financial software vendor), the
> unanimous answer would be 'we don't care if it's less efficient;
> nothing else is acceptable.'  Our customers insist on being able to
> use our products with whatever database they prefer (and certainly
> our competitors' products can do this).  We simply cannot afford to
> rewrite and maintain all our database-related code for every one of
> those databases.  For us (and, I think, for most software vendors,
> certainly all the ones I've worked for) the additional abstraction is
> well worth a slight loss of efficiency.  It is quite efficient enough
> for us.  The lack of a standard database API is one of the things that,
> unfortunately, would make it very difficult for me to convince my boss
> to let me use Caml.

Having followed the discussion for a while now, I'd say that it's
probably best to keep the development of drivers for specific databases
and the abstract layer separated. If anybody needs top performance,
they can always use the low-level library, but, as is obvious, some
people really need more abstract layers. Sometimes flexibility is better
than efficiency.

Developing a good abstract library for accessing databases is a
challenging task, much more difficult than writing a concrete one for
some specific database. I think people shouldn't go overboard with
abstraction here, and should rather try to develop a unified interface
for a given set of widely used databases, e.g. PostgreSQL, MySQL, Oracle.

Regards,
Markus

-- 
Markus Mottl          http://www.oefai.at/~markus          markus@oefai.at

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Vitaly Lugovsky]

On Fri, 30 Jan 2004, Benjamin Geer wrote:

> >  It was already discussed here. The conclusion was: BAD IDEA. No
> > way to work efficiently with different DBs using the same
> > approach.
>
> In the company I work for (a large financial software vendor),
> the unanimous answer would be 'we don't care if it's less
> efficient; nothing else is acceptable.' Our customers insist on
> being able to use our products with whatever database they
> prefer (and certainly our competitors' products can do this).

 And what's a problem? Write a portable layer, which provides
YOUR application-oriented abstractions. Writing database-related
sub-layer would be easy enough yet, but you will not pay the
price of the efficiency.

> We simply cannot afford to rewrite and maintain all our
> database-related code for every one of those databases.

 It can't be so large. Fine-grained layer always contains no more
then a dozen of low-level entities.

>  For us (and, I think, for most software vendors, certainly all
> the ones I've worked for) the additional abstraction is well
> worth a slight loss of efficiency.  It is quite efficient
> enough for us.

 I've heared a lot of evil words from the industry programmers
about the "good enough" efficiency of the JDBC. And, they're even
have no alternatives, poor ones...

>  The lack of a standard database API is one of the things that,
> unfortunately, would make it very difficult for me to convince
> my boss to let me use Caml.

 You may provide one, your own, which is limited to the standard
JDBC-like abstraction level, it would be easy to support, if you
don't want to write a higher level abstractions (don't forget -
you'd pay much more for the problems with SQL incompatibility -
so there is no damn need in the JDBC-level layers at all). But
why the maintainers of that DB bindings, like me, would follow
someones "standard" limitations?


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Vitaly Lugovsky wrote:
>  And what's a problem? Write a portable layer, which provides
> YOUR application-oriented abstractions.

More code means a more expensive, more time-consuming project.  Managers 
see that as a bad thing.  JDBC has many flaws, but it has enabled us to 
complete projects in a small fraction of the time that we would have 
needed if we had had to write our own portable database abstraction 
layer, using the low-level interface to each database.

Moreover, the absence of a *standard* database abstraction layer makes 
it impossible for people to write standard libraries that perform 
database access.

>>We simply cannot afford to rewrite and maintain all our
>>database-related code for every one of those databases.
> 
>  It can't be so large. Fine-grained layer always contains no more
> then a dozen of low-level entities.

One of our applications contains millions of lines of code and uses 
thousands of database tables.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Alex Baretta]

Benjamin Geer wrote:
> Vitaly Lugovsky wrote:
> 
>>>   If there's going to be a "new improved version", I think
>>> maybe it should work with multiple databases.  ODBC, JDBC, and
>>> DBI all attempt this.
>>
>>
>>  It was already discussed here. The conclusion was: BAD IDEA. No
>> way to work efficiently with different DBs using the same
>> approach.
> 
> 
> In the company I work for (a large financial software vendor), the 
> unanimous answer would be 'we don't care if it's less efficient; nothing 
> else is acceptable.'  Our customers insist on being able to use our 
> products with whatever database they prefer (and certainly our 
>...


This is true at a different abstraction level. Native drivers are needed 
  to be able to create high-performance db specific applications. On top 
of these it is possible to build multi-tier db connectivity solutions.

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Alex Baretta wrote:
> Native drivers are needed 
>  to be able to create high-performance db specific applications. On top 
> of these it is possible to build multi-tier db connectivity solutions.

That seems like a good approach to me.

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Alain.Frisch]

On Fri, 30 Jan 2004, Josh Burdick wrote:

>    If there's going to be a "new improved version", I think maybe it
> should work with multiple databases.  ODBC, JDBC, and DBI all attempt
> this.  There could be a signature (perhaps called "DBI" to appeal to
> Perl people , and structs Postgres, MySQL, Oracle, etc., which implement
> this.

IMHO, such a standard interface should be kept separated from
engine-specific bindings.  The idea is to have specific bindings that
follow closely the "official"  C API for a given engine, which allows the
programmer to refer easily to the documentation of the C interface and to
access all the advanced and non standard features of his database backend.

Then you can build on top of such bindings higher levels of abstraction in
order to expose a common subset with a unified interface. This issue has
been raised several times on this mailing list, and no consensus has been
reached on a common interface (AFAIK).


-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Alain.Frisch]

On Wed, 28 Jan 2004, Stefano Zacchiroli wrote:

> On Wed, Jan 28, 2004 at 07:32:18PM +0100, Markus Mottl wrote:
> > complete library for accessing the PostgreSQL-database. This library
> > builds on Alain Frisch's previous interface (postgres-20010808).
>
> This sentence doesn't clarify an issue: is this library supposed to
> supersede Alain's bindings or to be a fork?


Markus and I discussed the issue.

I'll continue to maintain my library, but it is very stable (no bug
reported in 3 years), so in practice, it only means that the package will
remain available. However, it is considered deprecated: new users should
better use Markus' revision.  Existing users can continue with my version
if they are happy with it, or switch to the new interface (the transition
should be easy; OTOH it will mainly bring an aesthetic improvement).

For distribution package maintainers, I think the best thing is to keep my
library available and provide Markus' one under another name.


--  Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

[Repost, because apparently the spam filter ate the last version of
this message]

mod_caml contains a Dbi layer which does all that.  For example, see a
random piece of code which I wrote yesterday attached to the end of
this message.

You can even use Perl DBDs with it, although I mainly use it to access
PostgreSQL through Alaine's Postgres module.

[Attachment moved to http://www.annexia.org/tmp/dbi-ex.txt ]

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
MAKE+ is a sane replacement for GNU autoconf/automake. One script compiles,
RPMs, pkgs etc. Linux, BSD, Solaris. http://www.annexia.org/freeware/makeplus/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Richard Jones wrote:
> mod_caml contains a Dbi layer which does all that.  For example, see a
> random piece of code which I wrote yesterday attached to the end of
> this message.

Looks about right to me.  How does it handle errors?  (Does it always 
throw the same exception, regardless of the type of error?  One of the 
things that's annoying about JDBC is that it doesn't distinguish between 
invalid SQL, lost database connectons, deadlocks, etc.)  Can it handle 
BLOBs and CLOBs?

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

On Fri, Jan 30, 2004 at 12:15:21PM +0000, Benjamin Geer wrote:
> Richard Jones wrote:
> >mod_caml contains a Dbi layer which does all that.  For example, see a
> >random piece of code which I wrote yesterday attached to the end of
> >this message.
> 
> Looks about right to me.  How does it handle errors?  (Does it always 
> throw the same exception, regardless of the type of error?  One of the 
> things that's annoying about JDBC is that it doesn't distinguish between 
> invalid SQL, lost database connectons, deadlocks, etc.)

It throws Sql_error when it can determine an error.  I'm not very
familiar with JDBC, except that I know a lot of people don't like it.
mod_caml's Dbi is much more closely related to Perl DBI, as you might
have guessed.

> Can it handle 
> BLOBs and CLOBs?

Since we use PostgreSQL for everything at Merjis, we use the 'text'
and 'byte' types which are much more useful and flexible than BLOBs.
So there's no specific support, although it wouldn't be too hard to
add it.  I've no idea how Perl DBI handles BLOBs either, since all the
databases I've ever used with it have had proper unlimited length text
types, so there's no need for them.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
NET::FTPSERVER is a full-featured, secure, configurable, database-backed
FTP server written in Perl: http://www.annexia.org/freeware/netftpserver/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

Here's a few more examples I've culled from our code.  We're doing
some (I think) neat stuff here, eg. using sth#map and sth#iter
to quickly iterate over the results from queries:

http://www.annexia.org/tmp/dbi-ex2.txt

... combined with pre-prepared statements:

http://www.annexia.org/tmp/dbi-ex3.txt

And we have some neat code that lets you tie a variable to a
particular field in the database, a kind of (primitive) persistence:

http://www.annexia.org/tmp/dbi-ex4.txt

We use this to tie Gtk widgets to database fields, using the
transaction capabilities of the database to implement Save and
Rollback operations in the GUI.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
If I have not seen as far as others, it is because I have been
standing in the footprints of giants.  -- from Usenet

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Richard Jones wrote:
> It throws Sql_error when it can determine an error.  I'm not very
> familiar with JDBC, except that I know a lot of people don't like it.
> mod_caml's Dbi is much more closely related to Perl DBI, as you might
> have guessed.

If the exception string just contains whatever error message the 
database produced, the application will have to parse the string to find 
out what went wrong.  And since different databases produce different 
error messages, this removes some of the benefit of having an DBI layer 
in the first place.  So it would be nice to have something like this:

type sql_error_type = Invalid_sql | Lost_connection | Deadlock | Other ;;

exception SQL_error of sql_error_type * string ;;

It might be a good idea to have a look at the error codes returned by 
Oracle, DB2 and SQL Server, and try to come up with some reasonable 
vendor-neutral codes for the DBI layer.

>>Can it handle 
>>BLOBs and CLOBs?
> 
> Since we use PostgreSQL for everything at Merjis, we use the 'text'
> and 'byte' types which are much more useful and flexible than BLOBs.
> So there's no specific support, although it wouldn't be too hard to
> add it.  I've no idea how Perl DBI handles BLOBs either,

A friend of mine who has been working with Perl DBI and Microsoft SQL 
Server says it doesn't seem to handle them at all.

> since all the
> databases I've ever used with it have had proper unlimited length text
> types, so there's no need for them.

PostgreSQL and MySQL are great databases, but all the banks we work with 
are very attached to either Oracle, DB2 or SQL Server, so it's essential 
for us to support those databases.  (Asking a bank to switch to a 
different database vendor is like asking the Pope to convert to another 
religion...)

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Benjamin Geer]

Richard Jones wrote:
> It throws Sql_error when it can determine an error.

Another thing I forgot to bring up about error handling.  If an error 
occurs, you generally want to roll back the transaction.  If you're 
using a connection pool, you want to make sure you close the connection 
when you're done with it, regardless of whether the transaction was 
committed or rolled back.  Committing a transaction, rolling back a 
transaction, and closing a connection are all operations that could also 
throw exceptions.  Writing out all those try-with expressions every time 
becomes very verbose and error-prone.  I think this is the sort of 
problem that functional programming is really well suited to.  For 
example, you could have a standard function like this:

let do_f_with_error_handling f conn = ...

It would begin a transaction on the specified connection, run f, handle 
any exceptions by rolling back the transaction (or else commit it if 
there were no errors), then close the connection (also handling any 
errors).  It would return the value returned by f, or if it caught an 
exception, it would throw that exception (after cleaning up).

(This is one of the annoying things about JDBC; I tend to write 
pseudo-functional classes to do this sort of thing in Java, but in Caml 
it would be much more straightforward.)

Ben

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PostgreSQL-OCaml 1.0.1

[Richard Jones]

On Fri, Jan 30, 2004 at 12:56:43PM +0000, Benjamin Geer wrote:
> Richard Jones wrote:
> >It throws Sql_error when it can determine an error.
> 
> Another thing I forgot to bring up about error handling.  If an error 
> occurs, you generally want to roll back the transaction.  If you're 

You might want to have a look at the code to see if it does what you
want.

In practice, writing mainly CGI scripts and standalone scripts as we
do, this is never a problem for us.  Our code is structured like this:

  let dbh = (* open connection, or get connection from pool *)

  (* do work with the connection, never checking for any exceptions *)

  dbh#commit

In the case of a standalone script, the script dies (exits) when an
error occurs, printing the SQL error message.  The database then
automatically rolls back the connection.  This is just the behaviour
of PostgreSQL - if a connection is closed in the middle of a
transaction then the database gets rolled back.

In the case of CGI scripts, the Dbi Pool code automatically performs a
dbh#rollback just before taking back a connection and placing it back
into the pool.  So if an error did occur before the commit, then the
database gets rolled back.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://www.YouUnlimited.co.uk/ - management courses

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners