[Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Matthieu Dubuget]

Hello,

I'm trying to setup a system to test it on my code base.

Currently, I have installed
- cygwin + mingw64-i686-gcc-core
- a patched flexdll (basically: makes use of i686-w64-mingw32-gcc 
instead of gcc -mno-cygwin)
- compiled and installed ocaml
> $ ocamlc -config
> version: 3.12.1
> standard_library_default: C:/cygwin/home/matt/ocamlmgw/lib
> standard_library: C:/cygwin/home/matt/ocamlmgw/lib
> standard_runtime: ocamlrun
> ccomp_type: cc
> bytecomp_c_compiler: i686-w64-mingw32-gcc -O -mms-bitfields -Wall
> -Wno-unused
> bytecomp_c_libraries: -lws2_32
> native_c_compiler: i686-w64-mingw32-gcc -O -mms-bitfields -Wall
> -Wno-unused
> native_c_libraries: -lws2_32
> native_pack_linker: i686-w64-mingw32-ld -r -o
> ranlib: i686-w64-mingw32-ranlib
> cc_profile:
> architecture: i386
> model: default
> system: mingw
> asm: i686-w64-mingw32-as
> ext_obj: .o
> ext_asm: .s
> ext_lib: .a
> ext_dll: .dll
> os_type: Win32
> default_executable_name: camlprog.exe
> systhread_supported: true

- compiled and installed findlib, menhir, ounit, ocamlcsv, frontc, 
xmllight, camomile, batteries

I have a problem with camlzip. In fact, the problem is not really 
ocaml-related… I do not know how to compile zlib with mingw64-i686-gcc, 
and use it to compile camlzip?

Does anybody has some idea to help?

Thanks

Matt

Re: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Adrien]

Hi,

On 08/08/2011, Matthieu Dubuget <matthieu.dubuget@gmail.com> wrote:
> I have a problem with camlzip. In fact, the problem is not really
> ocaml-related… I do not know how to compile zlib with mingw64-i686-gcc,
> and use it to compile camlzip?

If you want windows libraries *quickly*, tap into opensuse's packages.

Fridrich Strba is making an awesome work about that. Many libraries,
and all working well.
Since you're getting windows libraries, you don't have to care if
they're from some linux distribution or another one as long you remain
consistent. The only issue you might be facing is whether you can run
their i686-w64-mingw32-gcc binary on your current distribution.

You can find the packages ("236" currently) at:
  https://build.opensuse.org/project/list?searchtext=windows%3Amingw&excludefilter=home%3A

Debian also has working binaries for windows but they use the
mingw.org and their own prefix (i586-msvc or something like that,
which is completely not standard and which doesn't really have a
reason to exist</rant>). Fedora has some too but I think that OpenSuse
has the biggest set of libraries.

As a note about the "i686-w64-mingw32" triplet, and its
"x86_64-w64-mingw32" friend, which are both weird and annoying to
type.
The trailing "32" should disappear and the "w64" wasn't really chosen
by the mingw-w64 project but by binutils as far as I know (let's hope
it changes).
Also, even though it's called "mingw-w64", it's possible to compile
32bit and 64bit applications for windows. Both are working well and
equally supported.


Hope this helps,

Adrien Nader

RE: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[David Allsopp]

Matthieu Dubuget wrote:
> Hello,
> 
> I'm trying to setup a system to test it on my code base.
> 
> Currently, I have installed
> - cygwin + mingw64-i686-gcc-core
> - a patched flexdll (basically: makes use of i686-w64-mingw32-gcc instead
> of gcc -mno-cygwin)
> - compiled and installed ocaml
> > $ ocamlc -config
> > version: 3.12.1
> > standard_library_default: C:/cygwin/home/matt/ocamlmgw/lib
> > standard_library: C:/cygwin/home/matt/ocamlmgw/lib
> > standard_runtime: ocamlrun
> > ccomp_type: cc
> > bytecomp_c_compiler: i686-w64-mingw32-gcc -O -mms-bitfields -Wall
> > -Wno-unused
> > bytecomp_c_libraries: -lws2_32
> > native_c_compiler: i686-w64-mingw32-gcc -O -mms-bitfields -Wall
> > -Wno-unused
> > native_c_libraries: -lws2_32
> > native_pack_linker: i686-w64-mingw32-ld -r -o
> > ranlib: i686-w64-mingw32-ranlib
> > cc_profile:
> > architecture: i386
> > model: default
> > system: mingw
> > asm: i686-w64-mingw32-as
> > ext_obj: .o
> > ext_asm: .s
> > ext_lib: .a
> > ext_dll: .dll
> > os_type: Win32
> > default_executable_name: camlprog.exe
> > systhread_supported: true

Good to know that 64-bit MinGW is working - that said, I thought the MinGW port was broken in 3.12.1 or is that not affected with the 64-bit compiler? Could you share details on how you configured OCaml?

> - compiled and installed findlib, menhir, ounit, ocamlcsv, frontc,
> xmllight, camomile, batteries
> 
> I have a problem with camlzip. In fact, the problem is not really ocaml-
> related... I do not know how to compile zlib with mingw64-i686-gcc, and use
> it to compile camlzip?

I use the following two commands to compile zlib 1.2.5 (with shared lib support) with normal 32-bit mingw:

sed -i -e "s/MODE=0/MODE=1/" -e "s/)gcc/\0 -mno-cygwin/" win32/Makefile.gcc
make BINARY_PATH=C:/Dev/OCaml/bin LIBRARY_PATH=C:/Dev/OCaml/lib INCLUDE_PATH=C:/Dev/OCaml/lib -f win32/Makefile.gcc all install

Presumably a few tweaks at the top of Makefile.gcc should allow you to compile a 64-bit version?

For camlzip 1.04 I add the following META file (borrowed from BSD ports, IIRC):
---
name="zip"
version="1.04"
description="reading and writing ZIP, JAR and GZIP files"
requires="unix"
archive(byte)="zip.cma"
archive(native)="zip.cmxa"
linkopts = ""
---

You need to edit its Makefile - change ZLIB_LIB to -lzdll if you want to link against zlib1.dll (-lz will give you static linking). You need to change ZLIB_LIBDIR to be where you installed your custom zlib or the resulting binary will be linked against Cygwin's zlib (for me that's C:/Dev/OCaml/lib, therefore). I then build with make all allopt and install with ocamlfind install camlzip *.dll *.a zip.{cmxa,cm[aix],mli} {zlib,gzip}.{cm[ix],mli} META

The examples work and definitely don't depend on cygz.dll

HTH,


David

Re: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Matthieu Dubuget]

Date: 10/08/2011 10:25
From: David Allsopp <dra-news@metastack.com> wrote:
> Good to know that 64-bit MinGW is working - that said, I thought the MinGW port was broken in 3.12.1 or is that not affected with the 64-bit compiler? Could you share details on how you configured OCaml?

This is not 64 bits. i686-w64-mingw32 is 32 bits. But for this, I 
adapted some patches borrowed from bug #5179, that may interest you.

Here is the way I follow in order to install everything.
(I have scripts for this, but they are not generic…)

First, install cygwin, with something like this:

> @echo off
>
> rem "miniInstallPack/" is supposed to be on the user's desktop
>
> rem If you put "depot_cygwin/" in "miniInstallPack/" from a previous
> rem cygwin install, it will be used to save some bandwidth
> IF NOT EXIST depot_cygwin GOTO :NOCYGWINDEPOTDIR
>
> rem But only if you don't already have a c:\depot_cygwin
> IF EXIST c:\depot_cygwin GOTO :NOCYGWINDEPOTDIR
> XCOPY /E /Q /Y depot_cygwin c:\depot_cygwin\
> :NOCYGWINDEPOTDIR
>
> setup.exe -s ftp://mirror.switch.ch/mirror/cygwin -R c:\cygwin -q -l 
> c:\depot_cygwin -n -C Base -P 
> wget,unzip,binutils,diffutils,file,gdbm,make,ncurses,patchutils,m4,time,patch,autoconf,automake,mingw64-i686-gcc-core,vim
> c:
> chdir c:\cygwin\bin
>
> rem création du home
> bash --login -i -c "exit"
>
> bash --login -c "echo 'export TMP=/tmp' >> .bashrc"
> bash --login -c "echo 'export TEMP=/tmp' >> .bashrc"
> bash --login -c "cp -a `cygpath -d -D -m`/miniInstallPack ."
> rem the following script is to use kitty
> bash --login -c "cd miniInstallPack && bash 12_kittycyg.sh"
> bash --login -c "[ ! -e 'miniInstallPack/download' ] && mkdir 
> miniInstallPack/download"
> bash --login -c "[ ! -e 'miniInstallPack/extracted' ] && mkdir 
> miniInstallPack/extracted"
>
> echo
> echo Now, you'll have to log-in
> echo cd miniInstallPack
> echo and run 00_install_suite.sh
> echo

00_install_suite.sh is something like that:

> #!/bin/bash
> . installation_configuration.sh
>
> rm -rf `cygpath -d -D -m`/miniInstallPack
> rm -rf depot_cygwin
>
> . ./02_flexlink.sh
> . ./03_tcl.sh
> . ./04_ocaml.sh
> . ./05_findlib.sh
> . ./06_menhir.sh
> . ./11_emacs.sh
> . ./20_caml_mode.sh
> . ./19_tuareg.sh
> . ./15_bitstring_improved.sh
> . ./14_ounit.sh
> . ./21_ocamlcsv.sh
> . ./25_frontc.sh
> . ./22_magit.sh
> . ./23_xmllight.sh
> . ./08_camomile.sh
> . ./09_batteries.sh
> . ./24_zlib.sh
> . ./24_camlzip.sh
> ...

where installation_configuration.sh contains some functions and variables:

> #!/bin/bash
>
> here=`pwd`
> prefix=/home/matt
> appsprefix=${prefix}/apps
> prefixwin=C:/cygwin${prefix}
> appsprefixwin=${prefixwin}/apps
> downloaddir=${here}/download
> extractdir=${here}/extracted
> ocaml=ocaml-3.12.1
>
> emacs=emacs-23.3
> emacsarchive=${emacs}-bin-i386.zip
> emacsdownload=http://ftp.gnu.org/pub/gnu/emacs/windows/${emacsarchive}
> emacsdestdir=${appsprefix}/emacs
>
> ocamldir=ocamlmgw
> ocamldirwin=${prefixwin}/${ocamldir}
> ocamllib=${prefix}/${ocamldir}/lib
> bindir=${prefix}/${ocamldir}/bin
> tkroot=c:/tcl
> gtkdestdir=${appsprefix}/GTK
>
> # download_file file url
> download_file (){
> if [[ -e ${downloaddir}/$1 ]]
> then
> echo "'$1' already there."
> else
> echo "'$1' is not there: downloading..."
> echo
> wget --no-check-certificate -O ${downloaddir}/$1 $2
> echo
>
> if [[ -e ${downloaddir}/$1 ]]
> then
> echo "DOWNLOAD FINISHED -> OK"
> else
> echo "FAILED. exiting"
> exit 1
> fi
> fi
> }
> # extract_archive archiv dest dir
> extract_archive (){
> if [[ -e $2/$3 && -d $2/$3 ]]
> then
> echo "'$3' already extracted"
> else
> echo "Extracting '$3'..."
> tar -xaC $2 -f $1
> fi
> }
>
> # extract_archive_zip archive destdir
> extract_archive_zip (){
> if [[ -e "$2" && -d "$2" ]]
> then
> echo "'$2' already extracted"
> else
> echo "Extracting '$2'..."
> unzip -d "$2" "$1"
> fi
> }
>
> # extract_archive_zip_overwrite archive destdir
> extract_archive_zip_overwrite (){
> unzip -q -o -d "$2" "$1"
> }
>
> affiche_install (){
> echo
> echo -----------------------
> echo $1
> echo -----------------------
> echo
> }
>
> prepend_path (){
> export PATH="$1:$PATH"
> }
>
> append_path (){
> export PATH="$PATH:$1"
> }

Flexlink is a patched version (see bug #5179)

03_tcl.sh:
> #!/bin/bash
> affiche_install "TCL (Activetcl8.5.9.2.294317-win32-ix86-threaded)"
> activetcl=ActiveTcl8.5.9.2.294317-win32-ix86-threaded.exe
> activetcldownload=http://downloads.activestate.com/ActiveTcl/releases/8.5.9.2/${activetcl}
>
> download_file ${activetcl} ${activetcldownload}
> chmod +x ${downloaddir}/${activetcl}
> ${downloaddir}/${activetcl} --directory ${tkroot}

04_ocaml.sh:
> #!/bin/bash
> cd ${here}
> affiche_install "OCaml"
> ocamlarchive=${ocaml}.tar.bz2
> ocamldownload=http://caml.inria.fr/pub/distrib/ocaml-3.12/${ocamlarchive}
>
> # download
> download_file ${ocamlarchive} ${ocamldownload}
>
> #Extract
> extract_archive ${downloaddir}/${ocamlarchive} ${extractdir} ${ocaml}
>
> #Patch (for cygwin mingw cross compilation use)
> cd ${extractdir}/${ocaml} && patch -p1 -i ../../ocaml-i686-mingw64-patch
>
> #Configure
> cp ${extractdir}/${ocaml}/config/s-nt.h ${extractdir}/${ocaml}/config/s.h
> cp ${extractdir}/${ocaml}/config/m-nt.h ${extractdir}/${ocaml}/config/m.h
> cp ${extractdir}/${ocaml}/config/Makefile.mingw 
> ${extractdir}/${ocaml}/config/Makefile
>
> #We want to install in a different place
> cmd1="s|PREFIX=C:/ocamlmgw|PREFIX=$ocamldirwin|"
> sed -i.bak -e "$cmd1" ${extractdir}/${ocaml}/config/Makefile
>
> cd ${extractdir}/${ocaml} && \
> make -f Makefile.nt world && \
> make -f Makefile.nt opt && \
> make -f Makefile.nt opt.opt && \
> make -f Makefile.nt install
>
> echo "export PATH=\$PATH:${bindir}" >> ${HOME}/.bashrc
> append_path "${bindir}"
> cd ${here}

05_findlib.sh
> #!/bin/bash
> cd ${here}
> affiche_install "Findlib"
>
> findlib=findlib-1.2.7
> findlibarchive=${findlib}.tar.gz
> findlibdownload=http://download.camlcity.org/download/${findlibarchive}
>
> download_file ${findlibarchive} ${findlibdownload}
> extract_archive ${downloaddir}/${findlibarchive} ${extractdir} ${findlib}
>
> #If configure is not patched, a trailing \r\n remains in camlp4's META
> #file, which breaks it's use with ocamlfind
> cd ${extractdir}/${findlib} && \
> sed -i -e "s/camlp4 -where/camlp4 -where | tr -d '\\\\015'/" configure 
> && \
> ./configure -cygpath && \
> make all && make opt && make install
> cd ${here}


...

If somebody wants them, I can share those recipes.

Salutations

Matt

Re: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Adrien]

On 10/08/2011, Matthieu Dubuget <matthieu.dubuget@gmail.com> wrote:
>
> Date: 10/08/2011 10:25
> From: David Allsopp <dra-news@metastack.com> wrote:
>> Good to know that 64-bit MinGW is working - that said, I thought the MinGW
>> port was broken in 3.12.1 or is that not affected with the 64-bit
>> compiler? Could you share details on how you configured OCaml?
>
> This is not 64 bits. i686-w64-mingw32 is 32 bits. But for this, I
> adapted some patches borrowed from bug #5179, that may interest you.

To clarify the situation a bit, in the triplet, "w64" refers to the
mingw-w64 project. mingw.org uses "pc" instead (and you can configure
any build to use that one too, but the behaviour of your toolchain
will change). It could also be McDonalds were McDonald's doing
anything with GCC.

For "mingw32", the issue is that many configure scripts (and other
elements) check for "mingw32" when they should actually check for
"mingw*". Noone is to blame, it used to be perfectly acceptable but
it'll take some time to change (it's hopefully improving).


By the way, ocaml's configure scripts have at least one problem:
windows uses the LLP64 model while linux and many others use the LP64
model [1]. The endianess test uses a variable of type long and assumes
it is the same size as a pointer which is not the case on windows 64.
On C99, you can use (u)intptr_t and types from stddef.h (size_t,
ptrdiff_t, ...) instead. These types can also be used with GCC and
probably others even when in non-C99 mode.

[1] http://en.wikipedia.org/wiki/LLP64#Specific_C-language_data_models

If I remember correctly, you're still supposed to use the Makefile.nt
(make -f Makefile.nt) files with minor changes instead of calling
configure when using mingw and you're outside of cygwin which means
the issue won't always pop up but it's quite important to have this in
mind when porting to windows 64 (it's a common issue).

--

Adrien Nader

RE: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[David Allsopp]

Adrien Nader wrote:
> On 10/08/2011, Matthieu Dubuget <matthieu.dubuget@gmail.com> wrote:
> >
> > Date: 10/08/2011 10:25
> > From: David Allsopp <dra-news@metastack.com> wrote:
> >> Good to know that 64-bit MinGW is working - that said, I thought the
> >> MinGW port was broken in 3.12.1 or is that not affected with the
> >> 64-bit compiler? Could you share details on how you configured OCaml?
> >
> > This is not 64 bits. i686-w64-mingw32 is 32 bits. But for this, I
> > adapted some patches borrowed from bug #5179, that may interest you.
 
<snip>

> By the way, ocaml's configure scripts have at least one problem:
> windows uses the LLP64 model while linux and many others use the LP64
> model [1]. The endianess test uses a variable of type long and assumes it
> is the same size as a pointer which is not the case on windows 64.
> On C99, you can use (u)intptr_t and types from stddef.h (size_t,
> ptrdiff_t, ...) instead. These types can also be used with GCC and
> probably others even when in non-C99 mode.
> 
> [1] http://en.wikipedia.org/wiki/LLP64#Specific_C-language_data_models
> 
> If I remember correctly, you're still supposed to use the Makefile.nt
> (make -f Makefile.nt) files with minor changes instead of calling
> configure when using mingw and you're outside of cygwin which means the
> issue won't always pop up but it's quite important to have this in mind
> when porting to windows 64 (it's a common issue).

That's correct - on Windows, you only run configure if you're targeting native Cygwin. There are alternate Makefiles in config/ and you edit and rename the one you need for the other ports.


David

RE: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[David Allsopp]

Matthieu Dubuget wrote:
> From: David Allsopp <dra-news@metastack.com> wrote:
> > Good to know that 64-bit MinGW is working - that said, I thought the
> MinGW port was broken in 3.12.1 or is that not affected with the 64-bit
> compiler? Could you share details on how you configured OCaml?
> 
> This is not 64 bits. i686-w64-mingw32 is 32 bits.

Ah, my bad - I mistakenly thought the mingw32 was the project and w64 the architecture! 

I'm probably being low, but is the main benefit of using mingw-w64-mingw32 over the version shipped with Cygwin (-mno-cygwin) simply that the latter isn't working with 3.12.1 or is there another reason for using it?

> > #If configure is not patched, a trailing \r\n remains in camlp4's META
> > #file, which breaks it's use with ocamlfind cd
> > ${extractdir}/${findlib} && \ sed -i -e "s/camlp4 -where/camlp4 -where
> > | tr -d '\\\\015'/" configure && \ ./configure -cygpath && \ make all
> > && make opt && make install cd ${here}

I thought that one had been fixed (but clearly not, looking at my 1.2.7 META file for camlp4) - worth sending upstream to Gerd.

Going back to your original question - did those instructions work for zlib?


David 

Re: [Caml-list] Trying to setup mingw OCaml 3.12.1 + fresh cygwin

[Adrien]

On 12/08/2011, David Allsopp <dra-news@metastack.com> wrote:
> Matthieu Dubuget wrote:
>> From: David Allsopp <dra-news@metastack.com> wrote:
>> > Good to know that 64-bit MinGW is working - that said, I thought the
>> MinGW port was broken in 3.12.1 or is that not affected with the 64-bit
>> compiler? Could you share details on how you configured OCaml?
>>
>> This is not 64 bits. i686-w64-mingw32 is 32 bits.
>
> Ah, my bad - I mistakenly thought the mingw32 was the project and w64 the
> architecture!
>
> I'm probably being low, but is the main benefit of using mingw-w64-mingw32
> over the version shipped with Cygwin (-mno-cygwin) simply that the latter
> isn't working with 3.12.1 or is there another reason for using it?

Well, first and as far as I know, cygwin has had a package for the
*-w64-mingw32 for some time now. They are maintained by someone active
within the mingw-w64 project.

The reason to use one or another is to know which tools you are going
to use. Compiling from cygwin to windows without cygwin (what you call
-mno-cygwin) is actually cross-compilation!

Using mingw* from cmd.exe means you don't have a shell interpreter or
most of the posix tools.

Using it from msys (which is an ancient fork from cygwin which
requires a fork from gcc-2.96) means you have a mutant environment
which is trying to both please posix and windows. I'd call msys
"mostly unreliable crap". It's pretty light, it looks and feels native
but it's a weird mix and _you_ should absolutely never link against
the msys dll.

Cygwin simply lets you use more tools and have them working better
usually. Cygwin is also way slower (I still remember simple
./configure scripts taking 15 minutes to run, and networking has a
higher latency too).

Of course, you can also cross-compile from linux or any other posixy OS.


All of them have advantages and issues. One issue of msys is having
even more headaches when doing stuff on windows by the way.


As for -mno-cygwin, I saw mentions of it on IRC quite recently and
found out that it was on its way out. Actually that it was already
removed. See:
  "PING: Deprecation of -mno-cygwin"
  http://cygwin.com/ml/cygwin/2009-03/threads.html#00766

It has been deprecated since 2009 and your GCC documentation should
actually reflect that. I think it was only removed very recently which
means it won't disappear until the next release at least but it's
probably already subbject to bitrot.

As far as I know (haven't looked at that), the right way is simply to
cross-compile. Instead of passing -mno-cygwin as a CFLAG, use
--host=x86_64-w64-mingw32 when calling configure.

--

Adrien Nader