[9fans] Go: CGO and Plan 9

[9fans] Go: CGO and Plan 9

[lucio]

I need some help getting my mind around what should be a simple issue:
how to connect Go with Plan 9's "native" toolchain.

To be more specific, I have a port of the OpenLDAP library for Plan 9
that works quite adequately and I would like to access it from GO. I
guess this is true of any similar library ported using APE.  It seems
to me that because Go can connect to native C libraries and because in
Plan 9 the calling conventions are in fact identical, there ought to
be an easy way to do it, but I don't realy understand what's involved.

I suspect that the Go builder would have to be enhanced to deal with
this, while at the same time I wonder if the C compiler and assembler
should be used in a Go APE-like environment.  Whatever, I need help
thinking this through, can anyone assist?

++L

PS: I'd like the answer to be portable outside of Plan 9 (in p9p, to
be exact), but that isn't essential.

Re: [9fans] Go: CGO and Plan 9

[Pavel Zholkover]

CGO will generate C code stubs that require gcc to compile.
The gcc generated executable (_cgo_.o in the example bellow) is then
parsed for it's dwarf debug info
(I think) to get import the names and dso's of imported symbols.
Later, when you build an executable using the cgo'd package; 6l will
generate a dynamically linked executable making
the dynamic linker load/mmap the needed libraries (regular go
executables are statically linked, and there is their great value
in my opinion).

I think that if you really want LDAP, you'd either implement it
natively in Go, or massage the plan9 ported version of OpenLDAP
to build using the Go C compilers as part of a Go ldap.a library.

Attached is a simple cgo build example with verbose output from the go tool.
/tmp/test$ cat cgo_atoi.go
package cgo_atoi

// #include <stdlib.h>
import "C"

func Atoi(s string) int {
n, _ := C.atoi(C.CString(s))
        return int(n)
}

/tmp/test$ go build -v -x -work
WORK=/tmp/go-build967270508
_/tmp/test
mkdir -p $WORK/_/tmp/test/_obj/
cd /tmp/test
/usr/local/go/pkg/tool/linux_amd64/cgo -objdir $WORK/_/tmp/test/_obj/
-- -I $WORK/_/tmp/test/_obj/ cgo_atoi.go
/usr/local/go/pkg/tool/linux_amd64/6c -FVw -I $WORK/_/tmp/test/_obj/
-I /usr/local/go/pkg/linux_amd64 -o $WORK/_/tmp/test/_obj/_cgo_defun.6
-DGOOS_linux -DGOARCH_amd64 $WORK/_/tmp/test/_obj/_cgo_defun.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/_/tmp/test/_obj/ -o
$WORK/_/tmp/test/_obj/_cgo_main.o -c $WORK/_/tmp/test/_obj/_cgo_main.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/_/tmp/test/_obj/ -o
$WORK/_/tmp/test/_obj/_cgo_export.o -c
$WORK/_/tmp/test/_obj/_cgo_export.c
gcc -I . -g -O2 -fPIC -m64 -pthread -I $WORK/_/tmp/test/_obj/ -o
$WORK/_/tmp/test/_obj/cgo_atoi.cgo2.o -c
$WORK/_/tmp/test/_obj/cgo_atoi.cgo2.c
gcc -I . -g -O2 -fPIC -m64 -pthread -o $WORK/_/tmp/test/_obj/_cgo_.o
$WORK/_/tmp/test/_obj/_cgo_main.o $WORK/_/tmp/test/_obj/_cgo_export.o
$WORK/_/tmp/test/_obj/cgo_atoi.cgo2.o
/usr/local/go/pkg/tool/linux_amd64/cgo -objdir $WORK/_/tmp/test/_obj/
-dynimport $WORK/_/tmp/test/_obj/_cgo_.o -dynout
$WORK/_/tmp/test/_obj/_cgo_import.c
/usr/local/go/pkg/tool/linux_amd64/6c -FVw -I $WORK/_/tmp/test/_obj/
-I /usr/local/go/pkg/linux_amd64 -o
$WORK/_/tmp/test/_obj/_cgo_import.6 -DGOOS_linux -DGOARCH_amd64
$WORK/_/tmp/test/_obj/_cgo_import.c
/usr/local/go/pkg/tool/linux_amd64/6g -o $WORK/_/tmp/test/_obj/_go_.6
-p _/tmp/test -D _/tmp/test -I $WORK
$WORK/_/tmp/test/_obj/_cgo_gotypes.go
$WORK/_/tmp/test/_obj/cgo_atoi.cgo1.go
/usr/local/go/pkg/tool/linux_amd64/pack grcP $WORK $WORK/_/tmp/test.a
$WORK/_/tmp/test/_obj/_go_.6 $WORK/_/tmp/test/_obj/_cgo_import.6
$WORK/_/tmp/test/_obj/_cgo_defun.6 $WORK/_/tmp/test/_obj/_cgo_export.o
$WORK/_/tmp/test/_obj/cgo_atoi.cgo2.o

- Pavel

On Wed, Jan 30, 2013 at 6:41 AM,  <lucio@proxima.alt.za> wrote:
> I need some help getting my mind around what should be a simple issue:
> how to connect Go with Plan 9's "native" toolchain.
>
> To be more specific, I have a port of the OpenLDAP library for Plan 9
> that works quite adequately and I would like to access it from GO. I
> guess this is true of any similar library ported using APE.  It seems
> to me that because Go can connect to native C libraries and because in
> Plan 9 the calling conventions are in fact identical, there ought to
> be an easy way to do it, but I don't realy understand what's involved.
>
> I suspect that the Go builder would have to be enhanced to deal with
> this, while at the same time I wonder if the C compiler and assembler
> should be used in a Go APE-like environment.  Whatever, I need help
> thinking this through, can anyone assist?
>
> ++L
>
> PS: I'd like the answer to be portable outside of Plan 9 (in p9p, to
> be exact), but that isn't essential.
>
>

Re: [9fans] Go: CGO and Plan 9

[Aram Hăvărneanu]

?c and ?g use the same calling convention, and segmented stacks are
implemented by the linker. You don't need cgo in order to call C code
from Go. Why do you want cgo?

Look at the various C files in the Go standard library, especially
inside runtime. Those are compiled with the Plan 9 compiler and Go
calls it just fine.

-- 
Aram Hăvărneanu

Re: [9fans] Go: CGO and Plan 9

[Anthony Martin]

Aram Hăvărneanu <aram.h@mgk.ro> once said:
> ?c and ?g use the same calling convention, and segmented stacks are
> implemented by the linker. You don't need cgo in order to call C code
> from Go. Why do you want cgo?

This isn't true. Most of the Plan 9 compilers will pass the
first function argument in a register (not including 8c) and
return the result in a register (including 8c).

The C compilers in the Go toolchain pass everything on the
stack and return the result in a register.

The Go compilers pass everything on the stack including the
output parameters. No registers are involved in the calling
convention.

Another difference is in C is variadic functions in the face
of segmented stacks. The C compilers in the Go toolchain will
not compile a variadic function without a NOSPLIT annotation.
And even then, only if the resulting stack frame is small.

> Look at the various C files in the Go standard library, especially
> inside runtime. Those are compiled with the Plan 9 compiler and Go
> calls it just fine.

They're also careful not to validate the above assumptions
and they do not include any outside header files.

The right way to solve this problem is to modify cgo. I just
haven't had the time.

Cheers,
  Anthony

Re: [9fans] Go: CGO and Plan 9

[lucio]

> ?c and ?g use the same calling convention, and segmented stacks are
> implemented by the linker. You don't need cgo in order to call C code
> from Go. Why do you want cgo?
>
> Look at the various C files in the Go standard library, especially
> inside runtime. Those are compiled with the Plan 9 compiler and Go
> calls it just fine.

Thanks for the reply (you and Pavel Zholkover, so far).  As I
mentioned, it's not that I "want cgo", it's that I need to get my mind
around the issue and it is just too complicated.  It would be a
beneficial side effect if I can get OpenLDAP (client utilities)
working at the same time, but the moment I have a mental image of what
is actually going on, getting down to tacks will be easy enough.

So what I'm asking is for help in understanding how what is already
hard to understand in the GCC context where it is "documented" can in
fact be mirrored in the Plan 9 context, where it ought to be "native".
There are just way too many mirrors in this hall!  And my mind isn't
trained to separate them out.

Sorry to cast my problems at 9fans, I've tried to make sense of it
all, but I haven't been able to separate out the details, my brain
just insists in trying to tie everything together at once.  With a bit
of luck, contributions from here will reveal something I've been
missing.

In passing, dhog's GCC port to Plan 9 clarified a little the issue of
different call conventions, but I think that's adding to my confusion,
now.

++L

Re: [9fans] Go: CGO and Plan 9

[lucio]

> The C compilers in the Go toolchain pass everything on the
> stack and return the result in a register.
>
> The Go compilers pass everything on the stack including the
> output parameters. No registers are involved in the calling
> convention.

That isn't a big deal, in my case, because the language handled by the
C compilers in the Go toolchain is the same as Plan 9 native C, so,
within reason, I would expect the Go toolchain to compile the OpenLDAP
programs successfully, it doesn't matter if they are not consistent
with the Plan 9 native toolchain.

I suppose this raises the first concrete question: how do I get the
"go" tool to compile the OpenLDAP libraries (and possibly programs) in
some sensible fashion?  It would be distressing to have to increase
the autoconf hell at this point.  A partial answer to this would be
preferable to a blunt "You should not be wanting to do this".

Again, thanks for the contributions, the fog isn't visibly lifting
yet, but the feeling's there :-)

++L

PS: Note that even corrections of crossed wires have been extremely
helpful.

PPS: Eventually, there needs to be ONE Go + C + ASM toolchain.  It may
take years and a lot of TLC to produce, but any other outcome, with
the possible exception that something will obsolete each of the above
languages, is a frightening prospect in the long term.  Let's keep
that in mind and not contribute intentionally and unnecessarily to the
divergence.