From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id BFBF17EFCB for ; Tue, 18 Feb 2014 19:50:33 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of adrien@notk.org) identity=pra; client-ip=91.121.71.147; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="adrien@notk.org"; x-sender="adrien@notk.org"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of adrien@notk.org designates 91.121.71.147 as permitted sender) identity=mailfrom; client-ip=91.121.71.147; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="adrien@notk.org"; x-sender="adrien@notk.org"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@nautica.notk.org) identity=helo; client-ip=91.121.71.147; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="adrien@notk.org"; x-sender="postmaster@nautica.notk.org"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhELADyqA1NbeUeT/2dsb2JhbABZgwaEEYogs1AWdIJPQ0AIAgUTDgIRBRgdAUGHbps0jxWiABeBKYx2EQGDBg8xNYEUBJRDg2gBkiODLjuBNQ X-IPAS-Result: AhELADyqA1NbeUeT/2dsb2JhbABZgwaEEYogs1AWdIJPQ0AIAgUTDgIRBRgdAUGHbps0jxWiABeBKYx2EQGDBg8xNYEUBJRDg2gBkiODLjuBNQ X-IronPort-AV: E=Sophos;i="4.97,502,1389740400"; d="scan'208";a="59118156" Received: from nautica.notk.org ([91.121.71.147]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/ADH-AES256-SHA; 18 Feb 2014 19:50:33 +0100 Received: by nautica.notk.org (Postfix, from userid 1003) id 1BA67C01D; Tue, 18 Feb 2014 19:50:32 +0100 (CET) Date: Tue, 18 Feb 2014 19:50:32 +0100 From: Adrien Nader To: caml-list@inria.fr Message-ID: <20140218185032.GA20593@notk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [Caml-list] [RFC] Remaining changes for cross-compilation support in OCaml Hi, Roughly one year ago, late Wojciech Meyer and I started working on the integration of a set of patches aimed at enabling the OCaml compiler to be used for cross-compilation. Some of the patches so far have caused issues to some people. The goal of this RFC and email thread is to raise awareness of the upcoming work and its implications and also to start getting feedback as early as possible. The patches should be handled through the github interface which Gabriel Scherer mentionned a few weeks ago. ******************************** Note ******************************** The words "target", "host" and "build" below have the meaning that the GNU project gives them when it comes to compilation. When compiling an application or library: - build: the system the compilation process runs on - host: the system that will run the binaries that are built - target: meaningless When compiling a cross-compiler: - build: the system the compilation process runs on - host: the system that will run the binaries that are built (i.e. the cross-compiler) - target: the system for which the cross-compiler builds binaries *************************** Immediate Goals *************************** - build "${target}-ocamlc", "${target}-ocamlopt", ..., i.e. a cross-compiler for ${target} - typical cross-compilation of projects is expected to go through ocamlfind; in practice, oasis-based projects can be cross-compiled with no change **************************** Further Goals **************************** - cross-compile the compiler: build a native Windows toolchain on Linux - canadian cross support: ${build}, ${target}, ${host} are all different, i.e. on Linux, build a toolchain that runs on Windows 64b but targets Windows 32b or a different architecture/OS like ARM Linux. *************************** Current status *************************** - the compiler makes few assumptions about the target; this is very good. - build system is fairly large, has been in flux because of the separation of labltk and camlp4, supports both GNU make and BSD's makes (i.e. it uses no extension beyong POSIX), some things are split between "Makefile" and "Makefile.nt" (and for the ".nt" variant, only GNU make is supported and these files do use GNU make extensions). - most of the patches I ended up with have been upstreamed; a couple hairy ones are left and one in particular caused issues when bootstrapping. ****************************** Approach ****************************** *** A cross-compiler requires a native toolchain of the same version *** A working _native_ compiler toolchain of the _exact_ same version as the cross-toolchain is needed, both when building the cross-toolchain and when using it; the main reason was camlp4 but I believe it is still a valid restriction *** Give up the restriction to POSIX for makefiles *** Either remove compatibility with BSD makes, or move GNU-make-isms to "GNUMakefile", BSD-makes-isms to "BSDMakefile" and common data to "Makefile.common" with the GNU/BSDMakefile files being the "entrypoints". The manual page for GNU make states (and mk is similar but with BSDMakefile iirc): "If no -f option is present, make will look for the makefiles GNUmakefile, makefile, and Makefile, in that order." However, separating the code which benefits from the extension requires that: - the amount of non-shared code (i.e. the size of GNUmakefile and BSDMakefile files) is small and at least the data can be shared; this is meant for a few specific functions that are currently shelled-out because they are too difficult to write in POSIX make - most BSDs, if not all, can be supported without workarounds, kludges or additional hurdles caused by their fragmentation (for instance, OpenBSD's pmake is 10-years late compared to NetBSD's and has several bugs that are long solved in NetBSD) *** Use configure-defined variables instead of "ocaml{c,opt}"... *** The creation of the cross-compiler will rely on a native toolchain on ${build}. As such, ./configure will locate that toolchain and store its paths. This forbids any call like "./ocamlopt"; everything must rely on the paths found by configure. *** Use Int64.t in the compiler instead of Nativeint *** Currently, OCaml uses Nativeint to handle all the integers in the code that is being compiled. The "native" is for ${build}, not for ${target}. With a 64b host and 32b target, OCaml will output its values over 64 bits in the assembly and ld will then complain and truncate them. Move to Int64.t instead and delay the conversion to the right bitness until code emiting. The following types change: Cmm.expression, Cmm.data_item, Arch.specific_operation, Mach.operation. After that it's mostly a matter of fixing the type errors (around 50); since I do not have all the platforms that OCaml supports, I need to devise a way to compile the files for the other architectures too as a quick test. (types to change: Cmm.{expression,data_item}, Arch.specific_operation, Mach.operation) *** For bytecode, assume C libraries contain the required primitives *** When doing linking for bytecode with -custom, OCaml will dlopen() the corresponding library and dlsym() the primitive as an early check for their availability at runtime. Quite obviously, this fails for cross-compilation and the only solution I can think of is to disable the check completely. I have already written the corresponding patch; it adds 4 trivial lines and changes one. *** Devil is in the details *** While I wish the list above was exhaustive, I know for sure that it isn't, partly because some of the changes needed are implementation details more than general approaches. Do not hesitate to ask for clarification on anything you might wonder. -- Adrien Nader