[Caml-list] Application crashing when build with cross-compiler

[Caml-list] Application crashing when build with cross-compiler

[Stefan Hellermann]

[-- Attachment #1: Type: text/plain, Size: 5043 bytes --]

Hi!

I need help fixing a crash of the OCaml application unison. It only crashes
on my 32 bit embedded mips device, for which I compiled unison on an
OCaml-4.02.3 cross-compiler with -custom (NATIVE=false UISTYLE=text).
Unison crashes when syncing files and a file changed on both hosts in
different ways. unison asks then for an action, and crashes if I hit l or
x. This is reproducable even with two local folders on the embedded device.
I cannot reproduce this on my x86_64 linux host, so it's not a simple
unison bug. It's maybe related to the cross-compiler or maybe the
cross-compiler uncovers a bug in unison.

To Reproduce:
root@test:/tmp# mkdir unison1 unison2
root@test:/tmp# echo a > unison1/test.txt
root@test:/tmp# echo b > unison2/test.txt
root@test:/tmp# unison unison1 unison2
Contacting server...
Looking for changes
Warning: No archive files were found for these roots, whose canonical names
are:
        /tmp/unison1
        /tmp/unison2
This can happen either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.

Update detection may take a while on this run if the replicas are
large.

Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.

If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.

Donations to the Unison project are gratefully accepted:
http://www.cis.upenn.edu/~bcpierce/unison

Press return to continue.[<spc>]                       <= hit space
Reconciling changes

unison1        unison2
file     <-?-> file       test.txt  [] ?
Commands:
  f                     follow unison's recommendation (if any)
  I                     ignore this path permanently
  E                     permanently ignore files with this extension
  N                     permanently ignore paths ending with this name
  m                     merge the versions
  d                     show differences
  x                     show details
  L                     list all suggested changes tersely
  l                     list all suggested changes with details
  p or b                go back to previous item
  g                     proceed immediately to propagating changes
  q                     exit unison without propagating any changes
  /                     skip
  > or .                propagate from left to right
  < or ,                propagate from right to left
file     <-?-> file       test.txt  [] L                         <= Hitting
L is ok
file     <-?-> file       test.txt

file     <-?-> file       test.txt  [] l                         <= Hitting
l crashes
file     <-?-> file       test.txt
  Invalid argument: String.blit / Bytes.blit_string
Raised at file "pervasives.ml", line 31, characters 25-45
Called from file "buffer.ml", line 94, characters 2-47
Called from file "camlinternalFormat.ml", line 1732, characters 32-46
Called from file "camlinternalFormat.ml", line 1732, characters 32-46
Called from file "camlinternalFormat.ml", line 1732, characters 32-46
Called from file "printf.ml", line 33, characters 4-22
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uicommon.ml", line 170, characters 22-43
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uicommon.ml", line 240, characters 17-48
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 188, characters 17-50
Called from file "list.ml", line 73, characters 12-15
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 340, characters 21-184
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 545, characters 35-57
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 718, characters 6-47
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 788, characters 6-90
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 810, characters 19-66
Called from file
"/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
uitext.ml", line 870, characters 21-43



Unison sources can be found here:
https://webdav.seas.upenn.edu/viewvc/unison/branches/2.48/src/

Hopefully someone can help me debug this. I'm not an OCaml developer, I'm
trying to port OCaml and unison to embedded devices.

Stefan

[-- Attachment #2: Type: text/html, Size: 7002 bytes --]

Re: [Caml-list] <DKIM> Application crashing when build with cross-compiler

[Stefan Hellermann]

[-- Attachment #1: Type: text/plain, Size: 6161 bytes --]

I tried to reproduce this on a x86_32 cross-compiled target, but failed. I
tried to create exactly the same environment for x86_32, same libc, same
compiler, same kernel. Only difference to compiling for my target is
endianess, mips is big endian and x86_32 is little endian.

Should I try to descripe how to setup Qemu for an mips target and build
OCaml and unison with a cross-compiler?

Is there anything else I can try? Can someone point me to a few test
scripts which I could run on the mips target and see if something fails?

Stefan

2015-10-13 14:49 GMT+02:00 Pierre Chambart <pierre.chambart@ocamlpro.com>:

> Does that fail only on your device, or does it also fail on x86_64 to
> x86_32 cross-compilation ? If it is the case, could you provide a complete
> sequence of what is needed to reproduce the failure ?
> --
> Pierre
>
> On 13/10/2015 13:50, Stefan Hellermann wrote:
>
> Hi!
>
> I need help fixing a crash of the OCaml application unison. It only
> crashes on my 32 bit embedded mips device, for which I compiled unison on
> an OCaml-4.02.3 cross-compiler with -custom (NATIVE=false UISTYLE=text).
> Unison crashes when syncing files and a file changed on both hosts in
> different ways. unison asks then for an action, and crashes if I hit l or
> x. This is reproducable even with two local folders on the embedded device.
> I cannot reproduce this on my x86_64 linux host, so it's not a simple
> unison bug. It's maybe related to the cross-compiler or maybe the
> cross-compiler uncovers a bug in unison.
>
> To Reproduce:
> root@test:/tmp# mkdir unison1 unison2
> root@test:/tmp# echo a > unison1/test.txt
> root@test:/tmp# echo b > unison2/test.txt
> root@test:/tmp# unison unison1 unison2
> Contacting server...
> Looking for changes
> Warning: No archive files were found for these roots, whose canonical
> names are:
>         /tmp/unison1
>         /tmp/unison2
> This can happen either
> because this is the first time you have synchronized these roots,
> or because you have upgraded Unison to a new version with a different
> archive format.
>
> Update detection may take a while on this run if the replicas are
> large.
>
> Unison will assume that the 'last synchronized state' of both replicas
> was completely empty.  This means that any files that are different
> will be reported as conflicts, and any files that exist only on one
> replica will be judged as new and propagated to the other replica.
> If the two replicas are identical, then no changes will be reported.
>
> If you see this message repeatedly, it may be because one of your machines
> is getting its address from DHCP, which is causing its host name to change
> between synchronizations.  See the documentation for the
> UNISONLOCALHOSTNAME
> environment variable for advice on how to correct this.
>
> Donations to the Unison project are gratefully accepted:
> http://www.cis.upenn.edu/~bcpierce/unison
>
> Press return to continue.[<spc>]                       <= hit space
> Reconciling changes
>
> unison1        unison2
> file     <-?-> file       test.txt  [] ?
> Commands:
>   f                     follow unison's recommendation (if any)
>   I                     ignore this path permanently
>   E                     permanently ignore files with this extension
>   N                     permanently ignore paths ending with this name
>   m                     merge the versions
>   d                     show differences
>   x                     show details
>   L                     list all suggested changes tersely
>   l                     list all suggested changes with details
>   p or b                go back to previous item
>   g                     proceed immediately to propagating changes
>   q                     exit unison without propagating any changes
>   /                     skip
>   > or .                propagate from left to right
>   < or ,                propagate from right to left
> file     <-?-> file       test.txt  [] L                         <=
> Hitting L is ok
> file     <-?-> file       test.txt
>
> file     <-?-> file       test.txt  [] l                         <=
> Hitting l crashes
> file     <-?-> file       test.txt
>   Invalid argument: String.blit / Bytes.blit_string
> Raised at file "pervasives.ml", line 31, characters 25-45
> Called from file "buffer.ml", line 94, characters 2-47
> Called from file "camlinternalFormat.ml", line 1732, characters 32-46
> Called from file "camlinternalFormat.ml", line 1732, characters 32-46
> Called from file "camlinternalFormat.ml", line 1732, characters 32-46
> Called from file "printf.ml", line 33, characters 4-22
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uicommon.ml", line 170, characters 22-43
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uicommon.ml", line 240, characters 17-48
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 188, characters 17-50
> Called from file "list.ml", line 73, characters 12-15
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 340, characters 21-184
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 545, characters 35-57
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 718, characters 6-47
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 788, characters 6-90
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 810, characters 19-66
> Called from file
> "/home/stefan/openwrt/build_dir/target-x86_64_musl-1.1.11/unison-2.48.3/
> uitext.ml", line 870, characters 21-43
>
>
>
> Unison sources can be found here:
> https://webdav.seas.upenn.edu/viewvc/unison/branches/2.48/src/
>
> Hopefully someone can help me debug this. I'm not an OCaml developer, I'm
> trying to port OCaml and unison to embedded devices.
>
> Stefan
>
>
>

[-- Attachment #2: Type: text/html, Size: 10720 bytes --]

Re: [Caml-list] <DKIM> Application crashing when build with cross-compiler

[Adrien Nader]

On Wed, Oct 14, 2015, Stefan Hellermann wrote:
> I tried to reproduce this on a x86_32 cross-compiled target, but failed. I
> tried to create exactly the same environment for x86_32, same libc, same
> compiler, same kernel. Only difference to compiling for my target is
> endianess, mips is big endian and x86_32 is little endian.
> 
> Should I try to descripe how to setup Qemu for an mips target and build
> OCaml and unison with a cross-compiler?
> 
> Is there anything else I can try? Can someone point me to a few test
> scripts which I could run on the mips target and see if something fails?

Hmmmm, endianess is indeed likely to cause troubles.

Do the cross-build from x86_64 to ix86  (or the other way round) and
look for warnings from as (need the end of the build iirc). It will
point you to lines that result from assumptions about word size and,
quite likely, endianness. I have no idea if they will be the ones that
cause you troubles but they're worth checking I believe.

Unfortunately I don't know the code emission well enough to be more
precise about its requirements.

As for the qemu approach, it's something not to neglect because it's
easy to setup and gives results.

-- 
Adrien Nader

Re: [Caml-list] <DKIM> Application crashing when build with cross-compiler

[Xavier Leroy]

On 14/10/15 14:29, Stefan Hellermann wrote:
> I tried to reproduce this on a x86_32 cross-compiled target, but failed. I
> tried to create exactly the same environment for x86_32, same libc, same
> compiler, same kernel. Only difference to compiling for my target is
> endianess, mips is big endian and x86_32 is little endian.

Well, MIPS can also run in little-endian ("mipsel" in Debian speak),
but I'm assuming you run a big-endian MIPS kernel.

I see Debian has a Unison package for mips and other big-endian
platforms, which I presume works, so it's not an endianness dependency
in Unison itself nor in the OCaml libraries that Unison depends on.

> Is there anything else I can try?

You could build the OCaml runtime in debug mode:

cd /path/to/ocaml/sources/byterun && make ocamlrund

Then use it to run Unison:

/path/to/ocaml/sources/byterun/ocamlrund unison file1 file2

ocamlrund is compiled without C optimizations and with a lot of
internal checks, so it could help find possible problems with your
cross C compiler or target C library.

> Can someone point me to a few test scripts
> which I could run on the mips target and see if something fails?

There's always the testsuite/ part of the OCaml source distribution,
which you could run either on the target or on the host through qemu.

cd /path/to/ocaml/sources/testsuite && make BYTECODE_ONLY=true all

Adrien Nader adds:

> Hmmmm, endianess is indeed likely to cause troubles.

I'm skeptical.  Wrong endianness in config/m.h would cause ocamlrun to
crash immediately.

> Do the cross-build from x86_64 to ix86  (or the other way round) and
> look for warnings from as (need the end of the build iirc). 

The original poster is using bytecode, as native-code generation is
not available for MIPS anyway.

- Xavier Leroy

Re: [Caml-list] <DKIM> Application crashing when build with cross-compiler

[Adrien Nader]

On Wed, Oct 14, 2015, Xavier Leroy wrote:
> > Hmmmm, endianess is indeed likely to cause troubles.
> 
> I'm skeptical.  Wrong endianness in config/m.h would cause ocamlrun to
> crash immediately.
> 
> > Do the cross-build from x86_64 to ix86  (or the other way round) and
> > look for warnings from as (need the end of the build iirc). 
> 
> The original poster is using bytecode, as native-code generation is
> not available for MIPS anyway.

Right, my bad! I had forgotten about that and had based my answer on
native code gen.

-- 
Adrien Nader