[rust-dev] Problems cross-compiling to ARM9

[rust-dev] Problems cross-compiling to ARM9

[Tomi Pieviläinen]

Hi,

I'm trying to get cross compilation to Sitara AM1808 (Lego Mindstorms
EV3) working, but haven't been able to get working binaries.

First I simply tried compiling with `rustc
--target=arm-unknown-linux-gnueabi`, but got errors with missing std.

So next I compiled rust from source with `configure
--target=arm-unknown-linux-gnueabi --host=i686-unknown-linux-gnu`.
Then I could create a binary without errors with `env
LD_LIBRARY_PATH=$HOME/local/rust-cross/lib
PATH=$HOME/local/rust-cross/bin:$PATH local/rust-cross/bin/rustc -C
linker=/usr/bin/arm-linux-gnueabi-gcc
--target=arm-unknown-linux-gnueabi hello.rs`, but trying to run the
binary on the device just gives me "Illegal instruction".

For the record, checking the type of the helloworld with file gives

    hello: ELF 32-bit LSB  shared object, ARM, EABI5 version 1 (SYSV),
    dynamically linked (uses shared libs), for GNU/Linux 2.6.32,
    BuildID[sha1]=f48dc778edb4e4818915f45e4e6e985cf4e7eb59, not stripped

while working bash from the device is

    bash: ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV),
    dynamically linked (uses shared libs), for GNU/Linux 2.6.32,
    BuildID[sha1]=8ab870f6f5cbbb621b85763f557b41832a3f3181, stripped

I also tried setting target-cpu=arm9 and target-feature=+v5te, but
that didn't affect anything.

Next I tried using an external assembler by telling rustc -C
no-integrated-as. That gave a bunch of errors::

    error: linking with `/usr/bin/arm-linux-gnueabi-gcc` failed: exit code: 1
    note: /usr/bin/arm-linux-gnueabi-gcc '-c' '-o' 'hello.o' 'hello.s'
    note: hello.s: Assembler messages:
    hello.s:576: Error: selected processor does not support ARM mode `blx r2'
    hello.s:692: Error: selected processor does not support ARM mode `blx r2'
    hello.s:785: Error: selected processor does not support ARM mode `blx r2'
    hello.s:855: Error: selected processor does not support ARM mode `blx r2'
    hello.s:1280: Rd and Rm should be different in mul
    hello.s:2505: Rd and Rm should be different in mul
    hello.s:2663: Error: selected processor does not support ARM mode `clz r0,r0'
    hello.s:2730: Rd and Rm should be different in mul
    hello.s:2958: Error: selected processor does not support ARM mode `blx r2'
    hello.s:3074: Error: selected processor does not support ARM mode `blx r2'
    hello.s:3167: Error: selected processor does not support ARM mode `blx r2'
    hello.s:3237: Error: selected processor does not support ARM mode `blx r2'
    hello.s:3662: Rd and Rm should be different in mul
    hello.s:3979: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4058: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4146: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4220: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4306: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4380: Error: selected processor does not support ARM mode `blx r2'
    hello.s:4450: Error: selected processor does not support ARM mode `blx r2'
    hello.s:5135: Rd and Rm should be different in mul

    error: aborting due to previous error

As my last idea I tried emitting bc, ir and asm from rustc and using
llc and arm-linux-gnueabi-as to make binaries, but llc from ir gave

    llc: hello.ll:88:140: error: expected value token
    call void
    @_ZN2os4args20h5a44d7ab05eef5ebjtjE(%"struct.collections::vec::Vec<[collections::string::String]>[#6]"*
    noalias nocapture sret dereferenceable(12) %1)

, bc

    llc: hello.bc: error: Invalid value

and the assembler

    hello.s: Assembler messages:
    hello.s:1279: Rd and Rm should be different in mul
    hello.s:2504: Rd and Rm should be different in mul
    hello.s:2729: Rd and Rm should be different in mul
    hello.s:3661: Rd and Rm should be different in mul
    hello.s:5134: Rd and Rm should be different in mul

similar to using the no-integrated-as earlier.

I'm now fully out of ideas what could be going wrong and how to fix it.
Any suggestions would be very much appreciated.

-- 
Tomi Pieviläinen, +358 400 487 504
A: Because it disrupts the natural way of thinking.
Q: Why is top posting frowned upon?

Re: [rust-dev] Problems cross-compiling to ARM9

[Riad S. Wahby]

Tomi Pieviläinen <tomi.pievilainen@iki.fi> wrote:
> --target=arm-unknown-linux-gnueabi hello.rs`, but trying to run the
> binary on the device just gives me "Illegal instruction".

In the past when I've seen this error, it's because your C cross
toolchain is built for a slightly wrong architecture. Can you verify
that C programs cross compiled with your cross-gcc work correctly?

-=rsw

Re: [rust-dev] Problems cross-compiling to ARM9

[Tomi Pieviläinen]

> In the past when I've seen this error, it's because your C cross
> toolchain is built for a slightly wrong architecture. Can you verify
> that C programs cross compiled with your cross-gcc work correctly?

Christmas is over, back to work. C works fine just by calling
arm-linux-gnueabi-gcc. I hadn't tried an empty rust program, but that
also stops with the same "illegal instructions". The other hints are a
bit over my head right now, as I haven't done much anything this close
to metal. I'll continue studying this and report on whatever I find.

I'm still wondering if rustc is somehow creating "bad" IR, since LLVM
doesn't like that.

-- 
Tomi Pieviläinen, +358 400 487 504
A: Because it disrupts the natural way of thinking.
Q: Why is top posting frowned upon?

Re: [rust-dev] Problems cross-compiling to ARM9

[Tomi Pieviläinen]

[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]

So far my best guess is that the problem is the Assmebly line

mrc     15, 0, r4, cr13, cr0, {3}

in main, which means that it's trying to read from coprocessor 15,
register 13, part 3. According to
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0360e/CACEAIHG.html
that's a thread ID that's read only for user processes. But that's for
ARM11, and the corresponding doc for ARM9 (926EJ-S, specifically)
http://infocenter.arm.com/help/topic/com.arm.doc.ddi0198e/I1002240.html
doesn't list thread registers at all. So I'm guessing it doesn't
exist, and trying to read from it naturally fails.

A friend noticed that
https://github.com/rust-lang/rust/blob/master/src/librustc_back/target/arm_unknown_linux_gnueabi.rs
specifies ARMv6 as the target, but either retargetting from command
line or recompiling with that file patched for v5te didn't help; the
resulting ASM is the same in that part.

I think the problem might be
https://github.com/rust-lang/rust/blob/master/src/rt/arch/arm/record_sp.S
which assumes a more modern architecture. So perhaps this means that
rust has stricter requirements than C gnueabi?

Anyone have any idea if that's a larger problem, or simply something
nobody has written the small handcoded ASMs needed for ARMv5 or v4? If
latter, I might be able to wrap my head around this.

-- 
Tomi Pieviläinen, +358 400 487 504
A: Because it disrupts the natural way of thinking.
Q: Why is top posting frowned upon?

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [rust-dev] Problems cross-compiling to ARM9

[Valerii Hiora]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

Hi Tomi,

> Anyone have any idea if that's a larger problem, or simply something 
> nobody has written the small handcoded ASMs needed for ARMv5 or v4?
> If latter, I might be able to wrap my head around this.

  The problem you've got is related to segmented stack support. It need
fix on 2 levels:

- Rust - can be relatively easy fixed by providing (or patching) a
target and marking it as a target which doesn't support segmented
stacks, see example [1]

   Once it works you can play a bit to provide a correct implementation
in record_sp.S and morestack.S

- LLVM - as I remember some time ago LLVM generated a function prologue
which uses the same instruction for any ARM device, may be that was
patched in upstream, may be not. You can also ask Vladimir Pouzanov and
zinc.rs [2] team, AFAIK they had the similar problem too and definitely
have a workaround.

[1]
https://github.com/rust-lang/rust/blob/master/src/librustc_back/target/arm_apple_ios.rs#L33
[2]  https://zinc.rs/

-- 

  Valerii


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [rust-dev] Problems cross-compiling to ARM9

[Tomi Pieviläinen]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

> - Rust - can be relatively easy fixed by providing (or patching) a
> target and marking it as a target which doesn't support segmented
> stacks, see example [1]

> - LLVM - as I remember some time ago LLVM generated a function
> prologue which uses the same instruction for any ARM device

The prologue problem was fixed by adding "morestack: false," to the
linux ARM target specification, similar to how it is in the iOS
target.

>    Once it works you can play a bit to provide a correct
>    implementation in record_sp.S and morestack.S

My device does not really benefit from the segmented stack, since it
has only 64MB RAM, so I simply modified libstd/sys/common/stack.rs to
define stubs for all ARMs, not just iOS. Not that I would have the
skills to do it anyway.

With these two modifications I got helloworld working! Thank you for
the segmented stack on iOS hint. I'll now start programming what I
originally wanted, and will report if I find any other problems.

-- 
Tomi Pieviläinen, +358 400 487 504
A: Because it disrupts the natural way of thinking.
Q: Why is top posting frowned upon?

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [rust-dev] Problems cross-compiling to ARM9

[Vladimir Pouzanov]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

FWIW, we're just disabling segmented stack in zinc for now exactly because
of the same problem. I have a patch for llvm but I didn't really push it
upstream hard enough.

On Tue, Dec 30, 2014 at 11:23 AM, Valerii Hiora <valerii.hiora@gmail.com>
wrote:

> Hi Tomi,
>
> > Anyone have any idea if that's a larger problem, or simply something
> > nobody has written the small handcoded ASMs needed for ARMv5 or v4?
> > If latter, I might be able to wrap my head around this.
>
>   The problem you've got is related to segmented stack support. It need
> fix on 2 levels:
>
> - Rust - can be relatively easy fixed by providing (or patching) a
> target and marking it as a target which doesn't support segmented
> stacks, see example [1]
>
>    Once it works you can play a bit to provide a correct implementation
> in record_sp.S and morestack.S
>
> - LLVM - as I remember some time ago LLVM generated a function prologue
> which uses the same instruction for any ARM device, may be that was
> patched in upstream, may be not. You can also ask Vladimir Pouzanov and
> zinc.rs [2] team, AFAIK they had the similar problem too and definitely
> have a workaround.
>
> [1]
>
> https://github.com/rust-lang/rust/blob/master/src/librustc_back/target/arm_apple_ios.rs#L33
> [2]  https://zinc.rs/
>
> --
>
>   Valerii
>
>
> _______________________________________________
> Rust-dev mailing list
> Rust-dev@mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>
>


-- 
Sincerely,
Vladimir "Farcaller" Pouzanov
http://farcaller.net/

[-- Attachment #2: Type: text/html, Size: 2494 bytes --]

Re: [rust-dev] Problems cross-compiling to ARM9

[Valerii Hiora]

[-- Attachment #1: Type: text/plain, Size: 416 bytes --]

Hi Tomi,

> The prologue problem was fixed by adding "morestack: false," to the 
> linux ARM target specification, similar to how it is in the iOS 
> target.

  Right, my bad, that should be enough.

> My device does not really benefit from the segmented stack, since it 
> has only 64MB RAM

  Segmented stacks in Rust for a long time are used only for catching a
stack overflow.

-- 

  Valerii


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]