Re: Portability of applications written in OCAML: C stuff

Re: Portability of applications written in OCAML: C stuff

[Juergen Pfitzenmaier]

Max Skaller wrote on Wed, 23 Feb 2000 10:43:46 +1100
>         1. DO NOT USE A FILE CALLED 'config.h'
....
>        do NOT use generic names on the assumption your code will
>        be built with your Makefile in a separate directory.

I have seen some big projects (> 100 MB code) and several smaller ones. In each of
them people had to fight with the configuration. They all settled for what seems to
be the least pain: built and distribute (!) in small blocks; keep each block in its
own directory; use the same general frame for each block if possible (yes: use the
name config.h over and over again).
Even if its leading away from OCaml: I think that your real problem is with "make"
and its way of building project. Look for "configuration management" and "make replacement"
on the net.

pfitzen

Re: Portability of applications written in OCAML: C stuff

[skaller]

Juergen Pfitzenmaier wrote:
> 
> Max Skaller wrote on Wed, 23 Feb 2000 10:43:46 +1100
> >         1. DO NOT USE A FILE CALLED 'config.h'
> ....
> >        do NOT use generic names on the assumption your code will
> >        be built with your Makefile in a separate directory.
> 
> I have seen some big projects (> 100 MB code) and several smaller ones. In each of
> them people had to fight with the configuration. They all settled for what seems to
> be the least pain: built and distribute (!) in small blocks; keep each block in its
> own directory; use the same general frame for each block if possible (yes: use the
> name config.h over and over again).
> Even if its leading away from OCaml: I think that your real problem is with "make"
> and its way of building project. Look for "configuration management" and "make replacement"
> on the net.

no thanks. you see, I have a vastly superior system.
It is called interscript. (http://interscript.sourceforge.net).
Interscript is a literate programming tool written in Python.
Interscript 'script' is Python too. This replaces _completely_
all configuration control, takes responsibility not only for
compiling target software -- and of course, documenting it --
but also testing and installing it.

Unlike like 'make', 'autoconf' and various lame shell scripts,
Python is a full scale general purpose programming language.

Interscript, being experimental, does not yet provide the uniform
language independent (and that includes human languages as well as
programming
languages) build platform I'd like it to, and it is a bit slow
generating
large documents.

For this reason, I an writing a fast, fully featured, Python compatible
interpreter call Vyper (http://Vyper.sourceforge.net), which is designed
to support later building of a compiler. Vyper is written in Ocaml.

The current version of Vyper is not literate programmed, and it won't be
until Vyper executes interscript satisfactorily. [One of the main hang
ups
is finalising Python class instance objects -- which means executing
their __del__ methods, in the correct order(ocaml 2.999 finalisers work,
but finalise the objects in the wrong order at present).

The combination of Vyper and Interscript will be a rather amazingly
powerful
tool -- compared with make, autoconf, and shell script.

Interscript programs are distributed as sets of text files. Rarely are
subdirectories needed, mainly because an interscript source is a level
of abstraction above a standard source code source file. [Interscript
itself,
for example, has a tree like directory structure of Python packages,
but the original sources typically contain a whole directory contents
in a few files (interscript is 'verging' on needing subdirectories for
the source).]

In a sense, interscript sources are 'self extracting archives'.
And unlike other methods of packaging 'free' or other software sources,
you _always_ get documentation (even if it is only a program listing :-)

As to autoconf -- it may well make sense to run it _once_ to build
a configuration library interscript can use to control the building
of software. But typically, the ability of interscript to _generate_
code provides a vastly superior method of configuration to hackery
like C macros, conditional compilation, and other weird C'isms
(that won't work with other languages).

So my aim is to eventually distribute Vyper as interscript source
(as interscript itself is distributed, together with a prebuild
version for bootstrapping). And that means I need to get rid of 
subpackage makes, autoconfs, and other things, so as to unify
the distribution mechanism.

So you see, you're quite right:

> Even if its leading away from OCaml: I think that your real problem is with "make"
> and its way of building project. 

but

>Look for "configuration management" and "make replacement" on the net.

I'be been programming for quite some time, and I know how the ludicrous
make
and configuration systems people use on systems, including Unix and
Windows,
work -- or should I say, _fail_ to work. This is well known. Why else
would Redhat, for example, have created rpm? (that doesn't work so well
either ..)

-- 
John (Max) Skaller, mailto:skaller@maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
checkout Vyper http://Vyper.sourceforge.net
download Interscript http://Interscript.sourceforge.net

Re: Portability of applications written in OCAML: C stuff

[Juergen Pfitzenmaier]

Sorry, I forgot that URL last time. Here they are.
   http://www.baldmt.com/cons/
   ftp://ftp.cs.colorado.edu/pub/distribs/odin/

pfitzen

Re: Portability of applications written in OCAML: C stuff

[skaller]

Alan Schmitt wrote:

> Now if you would be so kind as to explain what better solution you
> have compared to makefiles, I'm all ears.

See my previous email. I have no end of problems installing
things (in general) as source. Ocaml is a pleasant exception,
it is very easy to build.

BTW: please think about configuration management and installation for
a second: it is well known to be a nightmare on all systems.
Specialists are employed by most companies with more than few boxes
just to keep them running and maintain the software configuration.
I think it's obvious on reflection that this is a really difficult
problem, and there really has to be something a lot better than
make and friends.

-- 
John (Max) Skaller, mailto:skaller@maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850
checkout Vyper http://Vyper.sourceforge.net
download Interscript http://Interscript.sourceforge.net

Re: Portability of applications written in OCAML: C stuff

[Juergen Pfitzenmaier]

Alan Schmitt wrote on Fri, 25 Feb 2000 08:37:43 +0000
>  Now if you would be so kind as to explain what better solution you
>  have compared to makefiles, I'm all ears.
I don't know what others have to say, but take a look at tools like odin and cons.

pfitzen

Re: Portability of applications written in OCAML: C stuff

[STARYNKEVITCH Basile]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 1028 bytes --]

>>>>> "Max" == Max Skaller <maxs@in.ot.com.au> writes:

    Max> Gerd Stolpmann wrote:
    >> We all know that Windows is not a serious operating system
    >> because it lacks support for many tasks we all EXPECT from an
    >> operating system.

    Max> 	? I do not understand. I run NT at home, and it
    Max> supports a LOT of functionality I expect from an operating
    Max> system that my Linux box does not.

Although I do prefer Linux to Microsoft systems I really think that OS
flamewars definitely DO NOT belong to the Ocaml list.

N.B. Any opinions expressed here are only mine, and not of my organization.
N.B. Les opinions exprimees ici me sont personnelles et n engagent pas le CEA.

---------------------------------------------------------------------
Basile STARYNKEVITCH   ----  Commissariat à l Energie Atomique 
DTA/LETI/DEIN/SLA * CEA/Saclay b.528 (p111f) * 91191 GIF/YVETTE CEDEX * France
phone: 1,69.08.60.55; fax: 1.69.08.83.95 home: 1,46.65.45.53
email: Basile point Starynkevitch at cea point fr 

Re: Portability of applications written in OCAML: C stuff

[Alan Schmitt]

>> In particular, when you say "must work under windows", do you mean out
>> of the box window, or one with some tools installed (ocaml, python,
>> cygwin thus bash, make ...)
>
>	The current requirements are for Python and Ocaml installed.
>Plus you need the same C compiler used to build Python and Ocaml.
>[You also need GTK]

I have never compiled ocaml under windows, but I thought there was
some kind of nmake to do it.

And some day, it should be possible to compile it using cygwin, which
is slightly cheaper than some m$ compilers ;-) and which makes
available many nice command line tools.

>> True, but since there is a nice mechanism to deal with this (ie
>> Makefile and directories), is this really worth it ?
>
>	What? Nice mechanism? Are you joking or ignorant?
>

I must say I am ignorant. I've only compiled software under Linux, and
installed it under Linux and Windows.

The only way to get really rid of software under windows is to
reinstall the whole system from scratch (the garbage collecting of
dlls is not that good ;-). Most of the time, when I uninstall
something that decided to go live in common directories, uninstall
fails. Under Linux, I think makefiles work well, and it is possible to
specify an install directory during the configuration process (and
when I say that, I mean there will be nothing that will go in another
directory, as it can happen for some other os).

Now if you would be so kind as to explain what better solution you
have compared to makefiles, I'm all ears.

Alan Schmitt
--
The hacker: someone who figured things out and made something cool happen.

Re: Portability of applications written in OCAML: C stuff

[Max Skaller]

Gerd Stolpmann wrote:

> We all know that Windows is not a serious operating system because it lacks
> support for many tasks we all EXPECT from an operating system. 

	? I do not understand. I run NT at home, and it supports
a LOT of functionality I expect from an operating system that my
Linux box does not.

	It runs games and office products like Word and IE. 
It has a clean, snappy, GUI. It has a reasonable scheduler, and a 
superior method of program construction (DLLs).


-- 
John (Max) Skaller at OTT [Open Telecommications Ltd]
mailto:maxs@in.ot.com.au      -- at work
mailto:skaller@maxtal.com.au  -- at home

Re: Portability of applications written in OCAML: C stuff

[Max Skaller]

Alan Schmitt wrote:
> 
> ..
> >In particular, while my product is currently Unix only, the build
> >process
> >must work under Windows too, which rules out make, autoconf, and other
> >such silliness. :-)
> 
> I gather this rules out using a compiler too ... ;-)


	I wish. The client must configure commands to compile
and link C. This cannot be avoided.

> In particular, when you say "must work under windows", do you mean out
> of the box window, or one with some tools installed (ocaml, python,
> cygwin thus bash, make ...)

	The current requirements are for Python and Ocaml installed.
Plus you need the same C compiler used to build Python and Ocaml.
[You also need GTK]

> I have many "config.h" in many source distributions in my /usr/src
> directory. I have never had such a problem. 

> True, but since there is a nice mechanism to deal with this (ie
> Makefile and directories), is this really worth it ?

	What? Nice mechanism? Are you joking or ignorant?

-- 
John (Max) Skaller at OTT [Open Telecommications Ltd]
mailto:maxs@in.ot.com.au      -- at work
mailto:skaller@maxtal.com.au  -- at home

Re: Portability of applications written in OCAML: C stuff

[Gerd Stolpmann]

On Thu, 24 Feb 2000, Max Skaller wrote:
>This is not acceptable. My rules are: after the client untars the
>distribution of
>my product, they may have to edit part of the build script, 'maker'
>which is 
>written in Python. Then they say 'python maker' and the build has to
>work
>without any other intervention. Make and autoconf are not permitted: any
>work
>these do must be done by ME in the 'maker' script IN PYTHON.
>
>In particular, while my product is currently Unix only, the build
>process
>must work under Windows too, which rules out make, autoconf, and other
>such silliness. :-)

We all know that Windows is not a serious operating system because it lacks
support for many tasks we all EXPECT from an operating system. This is one of
the reasons why I do not offer Windows support for my components. You have only
two possibilities: (A) reinventing the wheel by writing tools yourself; (B) not
offering special Windows support.

Please don't blame us that it is difficult to implement (A).

I would say that your build process is ... mhhh ... strange. If I did it
with only limited means, I would still distinguish between source directories,
and a (temporary) installation directory, say:

lib: the installation directory
source1, source2, ...: source directories

Simply compile the sources one after another (in a fixed order), and install
the results in lib. While compiling the sources, include -I ../lib such that
the previous results can be used in the next compilation step. This is simple
enough to be implemented even without Makefiles and autoconf.

>> "config.h" is generated by "configure" automatically and is actually part
>> of the C-library "pcre", which I always copy verbatim from its author.
>> Unless you also copy this "configure"-script out of its assigned directory
>> into your source dir, it won't do any harm to your files.
>
>	pcre cannot build without config.h, and neither can mlgtk.
>So there is a conflict, since both require a file called 'config.h'.

As far as I know you cannot choose the name of "config.h"; this is hard-wired
in autoconf.

>> That's what directories are for!
>
>	I do not wish to build and mess around with subpackages
>in directories, which amount to a small part of my overall package:
>pcre, for example, corresponds to a single library module with a few
>functions in it. There are hundreds of functions to build, and I want
>the sources all in the SAME directory.

Your argumentation is ridiculous. I often have directories with only very few
files if there good reasons to create them. In general I would argue that
directories reduce the mess.

>	When ocaml supports packages with subdirectory structures natively,
>THEN I will require the client install third party packages, rather than
>distributing them as an integral part of my source.

Why do you need a "native" package implementation? I think there are good
reasons to have one (e.g. a uniform installation procedure), but in your case
you can use ANY package mechanism (I have one), simply distribute it with your
sources.

>> I agree on this: a standard way for adding libraries, etc., would probably
>> boost productivity...
>
>	Enormously -- but I will be happy to wait until it can be done
>right, since it is VERY hard to 'fix' a broken design here. {The python 
>model is OK, but not perfect}

What does "right" mean? It depends on your expectations. For example, I
think a package mechanism should not include any build functionality (some kind
of "make") because there are already good solutions (even for Windows (cygwin)).

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 100             
64293 Darmstadt     EMail:   Gerd.Stolpmann@darmstadt.netsurf.de (privat)
Germany                     
----------------------------------------------------------------------------

Re: Portability of applications written in OCAML: C stuff

[Alan Schmitt]

..
>In particular, while my product is currently Unix only, the build
>process
>must work under Windows too, which rules out make, autoconf, and other
>such silliness. :-)

I gather this rules out using a compiler too ... ;-)
In particular, when you say "must work under windows", do you mean out
of the box window, or one with some tools installed (ocaml, python,
cygwin thus bash, make ...)

>	pcre cannot build without config.h, and neither can mlgtk.
>So there is a conflict, since both require a file called 'config.h'.

I have many "config.h" in many source distributions in my /usr/src
directory. I have never had such a problem. Actually, most of the
time, to build an application you don't need to build the
library. You actually require that it is there (and the nice thing is
you can make your ./configure check for it). Librairies and include
files are installed in some canonical places which can be looked up,
or non-canonical which can be explicitely given to the installation
script.

>> >       PLEASE use filenames that are specialised to your package,
>> >       do NOT use generic names on the assumption your code will
>> >       be built with your Makefile in a separate directory.
>> 
>> This is impossible. 
>
>	It is not impossible to try.

True, but since there is a nice mechanism to deal with this (ie
Makefile and directories), is this really worth it ?
Do you think we should adopt the java convention for naming ? This
would be a pain ...

Alan Schmitt
--
The hacker: someone who figured things out and made something cool happen.

Re: Portability of applications written in OCAML: C stuff

[Sven LUTHER]

On Thu, Feb 24, 2000 at 01:55:57PM +1100, Max Skaller wrote:
> Markus Mottl wrote:
> > 
> > > The way these packages are distributed is a 'build in separate
> > > directory and install' model. This is NOT acceptable for my
> > > package. I require all components to be built and usable
> > > WITHOUT installing them (with one exception: the standard ocaml (and
> > > python)
> > > distributions).
> > 
> > Hm, I don't know about "mlgtk", but what concerns the PCRE, you need not

About mlgtk, ...

you can use it without installing it anywhere, look at the examples for
guidance on how to do it (mostly you haveto set the right -I and -L options,
so ocaml will find the libraries).

Friendly,

Sven LUTHER

Re: Portability of applications written in OCAML: C stuff

[Max Skaller]

Markus Mottl wrote:
> 
> > The way these packages are distributed is a 'build in separate
> > directory and install' model. This is NOT acceptable for my
> > package. I require all components to be built and usable
> > WITHOUT installing them (with one exception: the standard ocaml (and
> > python)
> > distributions).
> 
> Hm, I don't know about "mlgtk", but what concerns the PCRE, you need not
> install it anywhere to use it. The only thing you have to do is place
> directory "{whatever}/pcre_ocaml/pcre-OCaml" on the include- and the
> library path when you compile/link your project.


	This model doesn't really work for me. Consider that typically
this requires

	1) run .configure
	2) run make

This is not acceptable. My rules are: after the client untars the
distribution of
my product, they may have to edit part of the build script, 'maker'
which is 
written in Python. Then they say 'python maker' and the build has to
work
without any other intervention. Make and autoconf are not permitted: any
work
these do must be done by ME in the 'maker' script IN PYTHON.

In particular, while my product is currently Unix only, the build
process
must work under Windows too, which rules out make, autoconf, and other
such silliness. :-)

> "config.h" is generated by "configure" automatically and is actually part
> of the C-library "pcre", which I always copy verbatim from its author.
> Unless you also copy this "configure"-script out of its assigned directory
> into your source dir, it won't do any harm to your files.

	pcre cannot build without config.h, and neither can mlgtk.
So there is a conflict, since both require a file called 'config.h'.

> >       PLEASE use filenames that are specialised to your package,
> >       do NOT use generic names on the assumption your code will
> >       be built with your Makefile in a separate directory.
> 
> This is impossible. 

	It is not impossible to try.

> How do you want to guarantee that files of other people
> don't get overwritten if you mix some distribution with theirs? 

	I guarrantee it, since I am combining the sources.

> That's what directories are for!

	I do not wish to build and mess around with subpackages
in directories, which amount to a small part of my overall package:
pcre, for example, corresponds to a single library module with a few
functions in it. There are hundreds of functions to build, and I want
the sources all in the SAME directory.

	When ocaml supports packages with subdirectory structures natively,
THEN I will require the client install third party packages, rather than
distributing them as an integral part of my source.
 
> I agree on this: a standard way for adding libraries, etc., would probably
> boost productivity...

	Enormously -- but I will be happy to wait until it can be done
right, since it is VERY hard to 'fix' a broken design here. {The python 
model is OK, but not perfect}

-- 
John (Max) Skaller at OTT [Open Telecommications Ltd]
mailto:maxs@in.ot.com.au      -- at work
mailto:skaller@maxtal.com.au  -- at home

Re: Portability of applications written in OCAML: C stuff

[Markus Mottl]

> The way these packages are distributed is a 'build in separate
> directory and install' model. This is NOT acceptable for my
> package. I require all components to be built and usable
> WITHOUT installing them (with one exception: the standard ocaml (and
> python)
> distributions). 

Hm, I don't know about "mlgtk", but what concerns the PCRE, you need not
install it anywhere to use it. The only thing you have to do is place
directory "{whatever}/pcre_ocaml/pcre-OCaml" on the include- and the
library path when you compile/link your project.

In case you also use my "OcamlMakefile", this should be fairly easy, e.g.
add something like:

  INCDIRS = ../../pcre-OCaml
  LIBDIRS = ../../pcre-OCaml
  LIBS    = pcre

> [In other words I must distribute IN MY PACKAGE everything
> which is not stock standard in the official ocaml distribution,
> and a single command must build the whole system]

I don't see any problem here - just "entering" into the directory of the
"third-party" distribution, "making" and including the directory with the
generated libraries should work fine.

> 	1. DO NOT USE A FILE CALLED 'config.h'
> 	  
> 	Such a file will clobber someone elses config.h
> 	Pleae call these files 'pcre_config.h' or 'gdk_config.h'
> 	or whatever.

"config.h" is generated by "configure" automatically and is actually part
of the C-library "pcre", which I always copy verbatim from its author.
Unless you also copy this "configure"-script out of its assigned directory
into your source dir, it won't do any harm to your files.

> 	PLEASE use filenames that are specialised to your package,
> 	do NOT use generic names on the assumption your code will
> 	be built with your Makefile in a separate directory.

This is impossible. How do you want to guarantee that files of other people
don't get overwritten if you mix some distribution with theirs? That's what
directories are for!

> 	2. Do NOT require you package be 'main'.
> 	Only ONE main is allowed, and it is MINE.
> 	Do NOT supply 'main' in ANY C libraries.
> 	(you can supply a sample 'main.c', but is must be that: a sample)

I always prefix C-identifiers with very unique names so as to avoid
name-clashes. I also don't see any "main"-routine in the underlying
PCRE-library other than in example files.

> Some of these problems will be alieviated when Ocaml gets a proper
> mechanism to add components to the system. This is non-trivial,
> but it is handled much better in Python than Ocaml at present.

I agree on this: a standard way for adding libraries, etc., would probably
boost productivity...

Regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

Re: Portability of applications written in OCAML: C stuff

[Max Skaller]

I have a problem using some third party\x03C codes with ocaml,
this includes pcre (defintely) and possibly mlgtk.
Some work needs to be done figuring out the right way to do this.
Here's the problem:

I am distributing an ocaml/C tool which uses third party
components like mlgtk and pcre which require C code.

The way these packages are distributed is a 'build in separate
directory and install' model. This is NOT acceptable for my
package. I require all components to be built and usable
WITHOUT installing them (with one exception: the standard ocaml (and
python)
distributions). 

What I do is copy relevant source files
from distributed packages, and, if necessary, edit them,
then I build them as part of my system, directly from these
sources.

[In other words I must distribute IN MY PACKAGE everything
which is not stock standard in the official ocaml distribution,
and a single command must build the whole system]

This is a pain. I would like to ask suppliers of third party
packages please

	1. DO NOT USE A FILE CALLED 'config.h'
	  
	Such a file will clobber someone elses config.h
	Pleae call these files 'pcre_config.h' or 'gdk_config.h'
	or whatever.

	PLEASE use filenames that are specialised to your package,
	do NOT use generic names on the assumption your code will
	be built with your Makefile in a separate directory.

	2. Do NOT require you package be 'main'.
	Only ONE main is allowed, and it is MINE.
	Do NOT supply 'main' in ANY C libraries.
	(you can supply a sample 'main.c', but is must be that: a sample)

	[This also seems to affect 'numerix']


Some of these problems will be alieviated when Ocaml gets a proper
mechanism to add components to the system. This is non-trivial,
but it is handled much better in Python than Ocaml at present.

I have made quite extensive additions to mlgtk, but I cannot
supply a 'patch' to the original authors because I have been
forced to rename files, and make edits which break that packages
build model.

-- 
John (Max) Skaller at OTT [Open Telecommications Ltd]
mailto:maxs@in.ot.com.au      -- at work
mailto:skaller@maxtal.com.au  -- at home