[9fans] OS X threads + dynamic linking

["Russ Cox" <rsc@swtch.com>]

[The only claim this has to being on topic is that
I need an answer to make libthread work on OS X x86.
That and it has to do with dynamic linking.  Sorry.]

In the gcc program below, the function runonstack uses
some carefully crafted inline assembly to run the function fn
on the given stack (the pointer passed in is the top of the stack).
Because of the use of assembly, it is an x86-only program.

It works great on Linux and on FreeBSD.  On OS X, though,
the function running on the alternate stack dies when it
tries to call printf.  A gdb session running the program
is shown after the program.

It's dying in the dynamic linker trying to resolve printf.
If I call printf in main before calling runonstack, then printf
no longer fails.  However, if I then call exit(0) inside hello,
then that fails, because exit isn't resolved yet.

Thus, it appears that somehow the x86 OS X library routines
cannot handle being called from an alternate stack.
This just cannot be true.  I thought everyone had already
made that mistake and moved on.

If anyone can either confirm that this program has no hope
of working on OS X (that would be unfortunate) or can tell
me what I need to do to make it work (that would be better!),
I'd greatly appreciate it.

As of right now, it appears that the plan9port CVS tree
builds just fine on x86 OS X except that all the threaded
programs crash due to this problem.

Thanks for any help.
Russ



#include <stdio.h>
#include <stdlib.h>

char stack[1048576];
char *state;

void
hello(void)
{
	state = "calling printf";
	printf("hello, world\n");
	state = "done with printf";
}

void
runonstack(void (*fn)(void), char *stack)
{
	state = "in assembly";
	asm(
		"pushal\n"
		"movl 8(%ebp), %ebx\n"	/* ebx = fn */
		"movl 12(%ebp), %eax\n"	/* ecx = stack */
		"xchgl %esp, %eax\n"
		"pushl %eax\n"
		"call *%ebx\n"
		"popl %esp\n"
		"popal\n"
	);
	state = "out of assembly";
}

int
main(int argc, char **argv)
{
	runonstack(hello, stack+sizeof stack);
}

---

$ gdb a.out
GNU gdb 6.1-20040303 (Apple version gdb-437) (Fri Jan 13 18:45:48 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

(gdb) run
Starting program: /Users/rsc/a.out
Reading symbols for shared libraries . done

Program received signal EXC_BAD_INSTRUCTION, Illegal instruction/operand.
0x8fe136e4 in __dyld_stub_binding_helper_interface ()
(gdb) print (char*)state
$1 = 0x1fa8 "calling printf"
(gdb) quit
The program is running.  Exit anyway? (y or n) y
$