[Caml-list] Building a custom compiler environment

[Caml-list] Building a custom compiler environment

[Jonathan Roewen]

Hi,

I'm trying to compile a custom 3.09.0 environment, using the system
ocaml compilers (some 3.08 version).

Is it not possible to compile the tools and things with 3.08? I get
the following error:

ocamlc -warn-error A -I utils -I parsing -I typing -I bytecomp -I
driver -I toplevel -c typing/typecore.ml
File "typing/typecore.ml", line 748, characters 12-29:
Unbound value Printf.sub_format
make[1]: *** [typing/typecore.cmo] Error 2

I've had a look in a 3.08 branch, and this function doesn't appear to
exist; however, I see it is present in the 3.09 source tree.

I suppose the only option is to install 3.09 as system ocaml tools?

Kindest Regards,

Jonathan

Re: [Caml-list] Building a custom compiler environment

[Jonathan Roewen]

Well, I upgraded tools to 3.09.0, so that's one problem out of the way
(though slightly annoying .. ah well).

Here's my latest error :-)

make[1]: Entering directory `/home/jonathan/dst/kernel'
../ocaml/bin/ocamlc -output-obj -o bytekernel.o asm.mli keyboard.ml
VFS.ml PCI.ml busManager.ml bytekernel.ml
Error while linking /home/jonathan/dst/ocaml/lib/ocaml/stdlib.cma(Pervasives):
Reference to undefined global `End_of_file'
make[1]: *** [bytekernel.o] Error 2

Obviously, I've removed some code that defines this somewhere (hope I
don't have to make too many changes to my custom environment). Any
hints?

Jonathan

Re: [Caml-list] Building a custom compiler environment

[Gerd Stolpmann]

On Sun, 2005-11-13 at 17:05 +1300, Jonathan Roewen wrote:
> Well, I upgraded tools to 3.09.0, so that's one problem out of the way
> (though slightly annoying .. ah well).
> 
> Here's my latest error :-)
> 
> make[1]: Entering directory `/home/jonathan/dst/kernel'
> ../ocaml/bin/ocamlc -output-obj -o bytekernel.o asm.mli keyboard.ml
> VFS.ml PCI.ml busManager.ml bytekernel.ml
> Error while linking /home/jonathan/dst/ocaml/lib/ocaml/stdlib.cma(Pervasives):
> Reference to undefined global `End_of_file'
> make[1]: *** [bytekernel.o] Error 2
> 
> Obviously, I've removed some code that defines this somewhere (hope I
> don't have to make too many changes to my custom environment). Any
> hints?

No, End_of_file is a predefined exception. Actually, you must not define
it, the compiler does it for you.

Look into typing/predef.ml to see all the predefined types and values.
(In byterun/fail.h there is a second, independent source of predefined
exceptions.)

Just a hypothesis: You do not have a reference to End_of_file at all,
and it is expunged from the global table because of this. Later the
compiler tries to add the code for the predefined exceptions, and cannot
find End_of_file any longer. - This may be total nonsense, however.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Telefon: 06151/153855                  Telefax: 06151/997714
------------------------------------------------------------

Re: [Caml-list] Building a custom compiler environment

[Jonathan Roewen]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

> No, End_of_file is a predefined exception. Actually, you must not define
> it, the compiler does it for you.

That's what I thought; so I can't figure out why it's missing.

Could it be how I compiled ocamlc?

I removed all the bootstrap stuff, and used system ocaml 3.09 compiler
tools to build the custom ocaml environment.

I'll attach the modified makefile, so it's clear what I've done.

And, if this is the case, then what are the correct steps to building
ocamlc without requiring a bootstrap cycle?

Jonathan

[-- Attachment #2: Makefile --]
[-- Type: text/plain, Size: 9866 bytes --]

#########################################################################
#                                                                       #
#                            Objective Caml                             #
#                                                                       #
#            Xavier Leroy, projet Cristal, INRIA Rocquencourt           #
#                                                                       #
#   Copyright 1999 Institut National de Recherche en Informatique et    #
#   en Automatique.  All rights reserved.  This file is distributed     #
#   under the terms of the Q Public License version 1.0.                #
#                                                                       #
#########################################################################

# $Id: Makefile,v 1.199 2005/09/24 16:20:36 xleroy Exp $

# The main Makefile

include config/Makefile
include stdlib/StdlibModules

CAMLC=ocamlc
COMPFLAGS=-warn-error A $(INCLUDES)
LINKFLAGS=

CAMLYACC=ocamlyacc
YACCFLAGS=-v
CAMLLEX=ocamllex
CAMLDEP=ocamldep
DEPFLAGS=$(INCLUDES)
CAMLRUN=ocamlrun
SHELL=/bin/sh
MKDIR=mkdir -p

INCLUDES=-I utils -I parsing -I typing -I bytecomp -I driver \
         -I toplevel

UTILS=utils/misc.cmo utils/tbl.cmo utils/config.cmo \
  utils/clflags.cmo utils/terminfo.cmo utils/ccomp.cmo utils/warnings.cmo \
  utils/consistbl.cmo

PARSING=parsing/linenum.cmo parsing/location.cmo parsing/longident.cmo \
  parsing/syntaxerr.cmo parsing/parser.cmo \
  parsing/lexer.cmo parsing/parse.cmo parsing/printast.cmo

TYPING=typing/unused_var.cmo typing/ident.cmo typing/path.cmo \
  typing/primitive.cmo typing/types.cmo \
  typing/btype.cmo typing/oprint.cmo \
  typing/subst.cmo typing/predef.cmo \
  typing/datarepr.cmo typing/env.cmo \
  typing/typedtree.cmo typing/ctype.cmo \
  typing/printtyp.cmo typing/includeclass.cmo \
  typing/mtype.cmo typing/includecore.cmo \
  typing/includemod.cmo typing/parmatch.cmo \
  typing/typetexp.cmo typing/stypes.cmo typing/typecore.cmo \
  typing/typedecl.cmo typing/typeclass.cmo \
  typing/typemod.cmo

COMP=bytecomp/lambda.cmo bytecomp/printlambda.cmo \
  bytecomp/typeopt.cmo bytecomp/switch.cmo bytecomp/matching.cmo \
  bytecomp/translobj.cmo bytecomp/translcore.cmo \
  bytecomp/translclass.cmo bytecomp/translmod.cmo \
  bytecomp/simplif.cmo bytecomp/runtimedef.cmo

BYTECOMP=bytecomp/meta.cmo bytecomp/instruct.cmo bytecomp/bytegen.cmo \
  bytecomp/printinstr.cmo bytecomp/opcodes.cmo bytecomp/emitcode.cmo \
  bytecomp/bytesections.cmo bytecomp/dll.cmo bytecomp/symtable.cmo \
  bytecomp/bytelink.cmo bytecomp/bytelibrarian.cmo bytecomp/bytepackager.cmo

DRIVER=driver/pparse.cmo driver/errors.cmo driver/compile.cmo \
  driver/main_args.cmo driver/main.cmo

TOPLEVEL=driver/pparse.cmo driver/errors.cmo driver/compile.cmo \
  toplevel/genprintval.cmo toplevel/toploop.cmo \
  toplevel/trace.cmo toplevel/topdirs.cmo toplevel/topmain.cmo

TOPLEVELLIB=toplevel/toplevellib.cma
TOPLEVELSTART=toplevel/topstart.cmo

COMPOBJS=$(UTILS) $(PARSING) $(TYPING) $(COMP) $(BYTECOMP) $(DRIVER)

TOPLIB=$(UTILS) $(PARSING) $(TYPING) $(COMP) $(BYTECOMP) $(TOPLEVEL)

TOPOBJS=$(TOPLEVELLIB) $(TOPLEVELSTART)

EXPUNGEOBJS=utils/misc.cmo utils/tbl.cmo \
  utils/config.cmo utils/clflags.cmo \
  typing/ident.cmo typing/path.cmo typing/types.cmo typing/btype.cmo \
  typing/predef.cmo bytecomp/runtimedef.cmo bytecomp/bytesections.cmo \
  bytecomp/dll.cmo bytecomp/meta.cmo bytecomp/symtable.cmo toplevel/expunge.cmo

PERVASIVES=$(STDLIB_MODULES) outcometree topdirs toploop

all: runtime ocamlc ocamltools library otherlibraries

LIBFILES=stdlib.cma std_exit.cmo *.cmi camlheader

# Installation
install: FORCE
	if test -d $(BINDIR); then : ; else $(MKDIR) $(BINDIR); fi
	if test -d $(LIBDIR); then : ; else $(MKDIR) $(LIBDIR); fi
	if test -d $(STUBLIBDIR); then : ; else $(MKDIR) $(STUBLIBDIR); fi
	cd $(LIBDIR); rm -f dllbigarray.so dlllabltk.so dllnums.so \
          dllthreads.so dllunix.so dllgraphics.so dllmldbm.so dllstr.so \
          dlltkanim.so
	cd byterun; $(MAKE) install
	cp ocamlc $(BINDIR)/ocamlc$(EXE)
	cd stdlib; $(MAKE) install
	cp toplevel/toplevellib.cma $(LIBDIR)/toplevellib.cma
	#cp expunge $(LIBDIR)/expunge$(EXE)
	cp typing/outcometree.cmi typing/outcometree.mli $(LIBDIR)
	#cp toplevel/topstart.cmo $(LIBDIR)
	cp toplevel/toploop.cmi toplevel/topdirs.cmi toplevel/topmain.cmi $(LIBDIR)
	cd tools; $(MAKE) install
	for i in $(OTHERLIBRARIES); do \
          (cd otherlibs/$$i; $(MAKE) install) || exit $$?; \
        done
	cp config/Makefile $(LIBDIR)/Makefile.config

clean:: partialclean

# The compiler

ocamlc: $(COMPOBJS)
	$(CAMLC) $(LINKFLAGS) -o ocamlc $(COMPOBJS)

partialclean::
	rm -f ocamlc

# The toplevel

toplevel/toplevellib.cma: $(TOPLIB)
	$(CAMLC) -a -o $@ $(TOPLIB)

partialclean::
	rm -f toplevel/toplevellib.cma

# The configuration file

utils/config.ml: utils/config.mlp config/Makefile
	@rm -f utils/config.ml
	sed -e 's|%%LIBDIR%%|$(LIBDIR)|' \
            -e 's|%%BYTERUN%%|$(BINDIR)/ocamlrun|' \
            -e 's|%%CCOMPTYPE%%|cc|' \
            -e 's|%%BYTECC%%|$(BYTECC) $(BYTECCCOMPOPTS) $(SHAREDCCCOMPOPTS)|' \
            -e 's|%%BYTELINK%%|$(BYTECC) $(BYTECCLINKOPTS)|' \
            -e 's|%%NATIVECC%%|$(NATIVECC) $(NATIVECCCOMPOPTS)|' \
            -e 's|%%NATIVELINK%%|$(NATIVECC) $(NATIVECCLINKOPTS)|' \
            -e 's|%%PARTIALLD%%|ld -r $(NATIVECCLINKOPTS)|' \
            -e 's|%%PACKLD%%|ld -r $(NATIVECCLINKOPTS)|' \
            -e 's|%%BYTECCLIBS%%|$(BYTECCLIBS)|' \
            -e 's|%%NATIVECCLIBS%%|$(NATIVECCLIBS)|' \
            -e 's|%%RANLIBCMD%%|$(RANLIBCMD)|' \
            -e 's|%%CC_PROFILE%%|$(CC_PROFILE)|' \
            -e 's|%%ARCH%%|$(ARCH)|' \
            -e 's|%%MODEL%%|$(MODEL)|' \
            -e 's|%%SYSTEM%%|$(SYSTEM)|' \
            -e 's|%%EXT_OBJ%%|.o|' \
            -e 's|%%EXT_ASM%%|.s|' \
            -e 's|%%EXT_LIB%%|.a|' \
            -e 's|%%EXT_DLL%%|.so|' \
            -e 's|%%SYSTHREAD_SUPPORT%%|$(SYSTHREAD_SUPPORT)|' \
            utils/config.mlp > utils/config.ml
	@chmod -w utils/config.ml

partialclean::
	rm -f utils/config.ml

beforedepend:: utils/config.ml

# The parser

parsing/parser.mli parsing/parser.ml: parsing/parser.mly
	$(CAMLYACC) $(YACCFLAGS) parsing/parser.mly

partialclean::
	rm -f parsing/parser.mli parsing/parser.ml parsing/parser.output

beforedepend:: parsing/parser.mli parsing/parser.ml

# The lexer

parsing/lexer.ml: parsing/lexer.mll
	$(CAMLLEX) parsing/lexer.mll

partialclean::
	rm -f parsing/lexer.ml

beforedepend:: parsing/lexer.ml

# The auxiliary lexer for counting line numbers

parsing/linenum.ml: parsing/linenum.mll
	$(CAMLLEX) parsing/linenum.mll

partialclean::
	rm -f parsing/linenum.ml

beforedepend:: parsing/linenum.ml

# The numeric opcodes

bytecomp/opcodes.ml: byterun/instruct.h
	sed -n -e '/^enum/p' -e 's/,//g' -e '/^  /p' byterun/instruct.h | \
        awk -f tools/make-opcodes > bytecomp/opcodes.ml

partialclean::
	rm -f bytecomp/opcodes.ml

beforedepend:: bytecomp/opcodes.ml

# The predefined exceptions and primitives

byterun/primitives:
	cd byterun; $(MAKE) primitives

bytecomp/runtimedef.ml: byterun/primitives byterun/fail.h
	(echo 'let builtin_exceptions = [|'; \
	 sed -n -e 's|.*/\* \("[A-Za-z_]*"\) \*/$$|  \1;|p' byterun/fail.h | \
	 sed -e '$$s/;$$//'; \
         echo '|]'; \
         echo 'let builtin_primitives = [|'; \
         sed -e 's/.*/  "&";/' -e '$$s/;$$//' byterun/primitives; \
	 echo '|]') > bytecomp/runtimedef.ml

partialclean::
	rm -f bytecomp/runtimedef.ml

beforedepend:: bytecomp/runtimedef.ml

#do we need tools/cvt_emit??
tools/cvt_emit: tools/cvt_emit.mll
	cd tools; \
	$(MAKE) CAMLC="../$(CAMLRUN) ../boot/ocamlc -I ../stdlib" cvt_emit

# The "expunge" utility

expunge: $(EXPUNGEOBJS)
	$(CAMLC) $(LINKFLAGS) -o expunge $(EXPUNGEOBJS)

partialclean::
	rm -f expunge

# The runtime system for the bytecode compiler

runtime:
	cd byterun; $(MAKE) all
	if test -f stdlib/libcamlrun.a; then :; else \
          ln -s ../byterun/libcamlrun.a stdlib/libcamlrun.a; fi
clean::
	cd byterun; $(MAKE) clean
	rm -f stdlib/libcamlrun.a
	rm -f stdlib/caml
alldepend::
	cd byterun; $(MAKE) depend

# The library

library: ocamlc
	cd stdlib; $(MAKE) all
partialclean::
	cd stdlib; $(MAKE) clean
alldepend::
	cd stdlib; $(MAKE) depend

# Tools

ocamltools: ocamlc
	cd tools; $(MAKE) all
partialclean::
	cd tools; $(MAKE) clean
alldepend::
	cd tools; $(MAKE) depend

# The extra libraries

otherlibraries:
	for i in $(OTHERLIBRARIES); do \
          (cd otherlibs/$$i; $(MAKE) RUNTIME=$(RUNTIME) all) || exit $$?; \
        done
partialclean::
	for i in $(OTHERLIBRARIES); do \
          (cd otherlibs/$$i; $(MAKE) partialclean); \
        done
clean::
	for i in $(OTHERLIBRARIES); do (cd otherlibs/$$i; $(MAKE) clean); done
alldepend::
	for i in $(OTHERLIBRARIES); do (cd otherlibs/$$i; $(MAKE) depend); done

# Check that the stack limit is reasonable.

checkstack:
	@if $(BYTECC) -o tools/checkstack tools/checkstack.c; \
	  then tools/checkstack; \
	  else :; \
	fi
	@rm -f tools/checkstack

# Default rules

.SUFFIXES: .ml .mli .cmo .cmi .cmx

.ml.cmo:
	$(CAMLC) $(COMPFLAGS) -c $<

.mli.cmi:
	$(CAMLC) $(COMPFLAGS) -c $<

.ml.cmx:
	$(CAMLOPT) $(COMPFLAGS) -c $<

partialclean::
	rm -f utils/*.cm[iox] utils/*.[so] utils/*~
	rm -f parsing/*.cm[iox] parsing/*.[so] parsing/*~
	rm -f typing/*.cm[iox] typing/*.[so] typing/*~
	rm -f bytecomp/*.cm[iox] bytecomp/*.[so] bytecomp/*~
	rm -f asmcomp/*.cm[iox] asmcomp/*.[so] asmcomp/*~
	rm -f driver/*.cm[iox] driver/*.[so] driver/*~
	rm -f toplevel/*.cm[iox] toplevel/*.[so] toplevel/*~
	rm -f tools/*.cm[iox] tools/*.[so] tools/*~
	rm -f *~

depend: beforedepend
	(for d in utils parsing typing bytecomp asmcomp driver toplevel; \
	 do $(CAMLDEP) $(DEPFLAGS) $$d/*.mli $$d/*.ml; \
	 done) > .depend

alldepend:: depend

FORCE:

include .depend

Re: [Caml-list] Building a custom compiler environment

[Jonathan Roewen]

I also need to ask: why are the primitives built INTO the compiler? Is
there any sound explanation for this?

One reason I could think of is to force the linker to keep those
symbols in the final output file (when using -output-obj). But if
they're not used, then why do they need to be present? For Dynlink to
work correctly?

This seems to be the -one- thing to make building my custom compiler
environment not work without requiring a customised ocamlc (with less
primitives builtin).

Jonathan

Re: [Caml-list] Building a custom compiler environment

[skaller]

On Tue, 2005-11-15 at 11:36 +1300, Jonathan Roewen wrote:
> I also need to ask: why are the primitives built INTO the compiler? Is
> there any sound explanation for this?
> 
> One reason I could think of is to force the linker to keep those
> symbols in the final output file (when using -output-obj). But if
> they're not used, then why do they need to be present? For Dynlink to
> work correctly?
> 
> This seems to be the -one- thing to make building my custom compiler
> environment not work without requiring a customised ocamlc (with less
> primitives builtin).

Perhaps: as a workaround make a dummy object file with the required
symbols in it and link it in .. since you're not using them,
it doesn't matter how you define them .. :)

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net