libcap/contrib/bug216610

Linking psx and C code without cgo

Overview

In some embedded situations, there is a desire to compile Go binaries to include some C code, but not libc etc. For a long time, I had assumed this was not possible, since using cgo requires libc and libpthread linkage.

This embedded compilation need was referenced in a bug filed against the "psx" package. The bug-filer was seeking an alternative to CGO_ENABLED=1 compilation requiring the cgo variant of psx build. However, the go "runtime" package will always panic() if you try this because it needs libpthread and [g]libc to work.

In researching that bug report, however, I have learned there is a trick to combining a non-CGO built binary with compiled C code. I learned about it from a brief reference in the Go Programming Language Wiki.

This present directory evolved from my attempt to understand and hopefully resolve what was going on as reported in that bug into an example of this trick. I was unable to resolve the problem as reported because of the aformentioned panic() in the Go runtime. However, I was able to demonstrate embedding C code in a Go binary without use of cgo. In such a binary, the Go-native version of "psx" is thus achievable. This is what the example in this present directory demonstrates.

Caveat Emptor: this example is very fragile. The Go team only supports cgo linking against C. That being said, I'd certainly like to receive bug fixes, etc for this directory if you find you need to evolve it to make it work for your use case.

Content

In this example we have:

• Some C code for the functions fib_init() and fib_next() that combine to implement a compute engine to determine Fibonacci Numbers. The source for this is in the sub directory c/fib.c.

• Some Go code, in the directory go/fibber that uses this C compiled compute kernel.

• c/gcc.sh which is a wrapper for gcc that adjusts the compilation to be digestible by Go's (internal) linker (the one that gets invoked when compiling CGO_ENABLED=0. Using gcc directly instead of this wrapper generates an incomplete binary - which miscomputes the expected answers. See the discussion below for what seems to be going on.

• A top level Makefile to build it all.

Building and running the built binary

Set things up with:

$ git clone git://git.kernel.org/pub/scm/libs/libcap/libcap.git
$ cd libcap
$ make all
$ cd contrib/bug216610
$ make clean all

When you run ./go/fib it should generate the following output:

$ ./go/fib
psx syscall result: PID=<nnnnn>
fib: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...
$

Where <nnnnn> is the PID of the program at runtime and will be different each time the program is invoked.

Discussion

The Fibonacci detail of what is going on is mostly uninteresting. The reason for developing this example was to explore the build issues in the reported Bug 216610. Ultimately, this example offers an alternative path to building a nocgo program that links to compute kernel of C code.

The reason we have added the c/gcc.sh wrapper for gcc is that we've found the Go linker has a hard time digesting the cross-sectional %rip based data addressing that various optimization modes of gcc like to use. Specifically, in the x86_64/amd64 architecture, if a R_X86_64_PC32 relocation entry made in a .text section refers to an .rodata.cst8 section in a generated .syso file, the Go linker seems to replace this reference with a 0 offset to (%rip). What our wrapper script does is rewrite the generated assembly to store these data references to the .text section. The Go linker has no problem with this same section relative addressing and is able to link the resulting objects without problems.

If you want to cross compile, we have support for 32-bit arm compilation: what is needed for the Raspberry PI. To get this support, try:

$ make clean all arms
$ cd go
$ GOARCH=arm CGO_ENABLED=0 go build

The generated fib binary runs on a 32-bit Raspberry Pi.

Future thoughts

At present, this example only works on Linux with x86_64 and arm build architectures. (In go-speak that is linux_amd64 and linux_arm). This is because I have only provided some bridging assembly for Go to C calling conventions for those architecture targets: ./go/fibber/fibs_linux_amd64.s and ./go/fibber/fibs_linux_arm.s. The non-native, make arms, cross compilation requires the docker command to be available.

I intend to implement an arm64 build, when I have a system on which to test it.

Note The Fedora system on which I've been developing this has some SELINUX impediment to naively using the docker -v ... bind mount option. I need the :z suffix for bind mounting. I don't know how common an issue this is. On Fedora, building the arm variants of the .syso file can be performed as follows:

$ docker run --rm -v $PWD/c:/shared:z -h debian -u $(id -u) -it expt shared/build.sh

Reporting bugs

Please report issues or offer improvements to this example via the Fully Capable libcap website.