[9fans] problem with building from source on vx32 solved

[9fans] problem with building from source on vx32 solved

[ron minnich]

I'd been having a problem once I built from source on vx32.

lots of commands would hang until I hit return.

ratrace showed this:

rminnich@xcpu2:~$ more /tmp/problem
ratrace -c /bin/date
24577 date Pread 0x19f6 0
0ffffee0/"." 8 0 = 1 "" 0x11cef69ae0b06c68 0x11cef69c0a58d900
24577 date Close 0x1a30 0 = 0 "" 0x11cef69c0b601f70 0x11cef69c0b607948
24577 date Open 0x1a89 0000702c/"/dev/bintime" 00000020 = 0 ""
0x11cef69c0c2119c8 0x11c
ef69c0c5911e8

Note the weird read from fd 0. Makes no sense.

To shorten a long story, I tracked it to this:
pid = _tos->pid
in libc/9sys/nsec.c, which was added last july 23.

pid is 0 in _tos->pid on 9vx.

A simple change:
pid = getpid();

fixes it. Is it essential for performance to have _tos->pid? It would
seem so. If it's really that necessary, then maybe getpid() could just
cache the pid.

Or I'll try to fix 9vx.

For now, I'm going with getpid(), and sysfromiso will too.

thanks

ron

Re: [9fans] problem with building from source on vx32 solved

[erik quanstrom]

> A simple change:
> pid = getpid();

shouldn't you just fix it so _tos->pid is set?

also, the comments in the new nsec.c don't
match the code.  the code looks something like this

	do{
		...
		tries = 0;
	}while(tries++ == 0);	/* retry once */

of course either this should be

	/* just retry for ever */
	for(;;){
		...
	}
or
	/* retry once */
	for(tries = 0; tries < 2; tries++)

also, there's nothing stoping the race between
- two procs allocating the same element of the pid array
- two procs opening the global fd twice

both cases have the potential to leak an unlimited amount
of file descriptors.  i've added a proposed change.

- erik

----
#include <u.h>
#include <libc.h>
#include <tos.h>

static uvlong order = 0x0001020304050607ULL;

static void
be2vlong(vlong *to, uchar *f)
{
	uchar *t, *o;
	int i;

	t = (uchar*)to;
	o = (uchar*)&order;
	for(i = 0; i < sizeof order; i++)
		t[o[i]] = f[i];
}

static Lock nseclk;
static Lock fdlk;
static int fd = -1;
static struct {
	int	pid;
	int	fd;
} fds[64];

vlong
nsec(void)
{
	uchar b[8];
	vlong t;
	int pid, i, *f, tries;

	/*
	 * Threaded programs may have multiple procs
	 * with different fd tables, so we may need to open
	 * /dev/bintime on a per-pid basis
	 */
	pid = _tos->pid;
	f = nil;
	for(i = 0; i < nelem(fds); i++)
		if(fds[i].pid == pid){
			f = &fds[i].fd;
			break;
		}
	if(i == nelem(fds)){
		lock(&nseclk);
		for(i = 0; i < nelem(fds); i++)
			if(fds[i].pid == 0){
				fds[i].pid = pid;
				fds[i].fd = -1;
				f = &fds[i].fd;
				break;
			}
		unlock(&nseclk);
	}
	if(f == nil)
		f = &fd;
	for(tries = 0; tries < 2; tries++){
		if(*f < 0){
			if(f == &fd)
				lock(&fdlk);
			if((*f = open("/dev/bintime", OREAD|OCEXEC)) < 0)
				break;
			fd = *f;
			if(f == &fd)
				unlock(&fdlk);
		}
		if(pread(*f, b, sizeof b, 0) == sizeof b){
			be2vlong(&t, b);
			return t;
		}
		close(*f);
		*f = -1;
	}
	if(i < nelem(fds))
		fds[i].pid = 0;	/* unlocked release */
	return 0;
}

Re: [9fans] problem with building from source on vx32 solved

[ron minnich]

On Tue, Aug 31, 2010 at 5:54 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>> A simple change:
>> pid = getpid();
>
> shouldn't you just fix it so _tos->pid is set?

yep. I just have to find the time.

>
> also, the comments in the new nsec.c don't
> match the code.  the code looks something like this
>
>        do{
>                ...
>                tries = 0;
>        }while(tries++ == 0);   /* retry once */
>
> of course either this should be
>
>        /* just retry for ever */
>        for(;;){
>                ...
>        }
> or
>        /* retry once */
>        for(tries = 0; tries < 2; tries++)
>
> also, there's nothing stoping the race between
> - two procs allocating the same element of the pid array
> - two procs opening the global fd twice



personal opinion: the difficulty of getting this code right, and its
complexity, tell me something is wrong with the interface. Look at the
trouble we're going through to get a 64-bit number out of the kernel.
Look at how hard it is to get right.

Two ways out:
1. "reserved" fds. an fd that is always open, can not be closed, and
will return important values, e.g. reserved fd 32766 could always be a
connection to bintime
2. a system call to get bintime.

others? Either way I don't see the current approach as making a lot of sense.

ron

Re: [9fans] problem with building from source on vx32 solved

[erik quanstrom]

> personal opinion: the difficulty of getting this code right, and its
> complexity, tell me something is wrong with the interface. Look at the
> trouble we're going through to get a 64-bit number out of the kernel.
> Look at how hard it is to get right.
>
> Two ways out:
> 1. "reserved" fds. an fd that is always open, can not be closed, and
> will return important values, e.g. reserved fd 32766 could always be a
> connection to bintime
> 2. a system call to get bintime.

both of these seem like trouble, and could cause reduced functionality.
i can bind nsec from someplace else if i want to, now.

> others? Either way I don't see the current approach as making a lot of sense.

why not put nsecfd in _tos, too?  sincechildren
not sharing fd tables must have a different _tos,
they can change the fd.

the remaining byte ordering perhaps should be
promoted (with its twin) to 9sys.  there's an
implementation of both that accept any interger
size in sdodin.c in the contrib version of sd.

- erik

Re: [9fans] problem with building from source on vx32 solved

[ron minnich]

OK, the problem with _tos on 9vx is making some sense.

gcc and 8c disagree on the size of the Tos struct. It's declared the
same way in 9vx as in
Plan 9, but 8c thinks it is 56 bytes and gcc thinks it is 88.

The reason is that 9vx is compiled on a 64-bit machine in this case,
so we are running 32-bit binaries on a 64-bit kernel, i.e. it is
heterogeneous. Amusing :-)

Guess I'll try 9vx as a 32-bit binary and see what shakes.

ron

Re: [9fans] problem with building from source on vx32 solved

[erik quanstrom]

On Thu Sep  2 02:38:12 EDT 2010, rminnich@gmail.com wrote:
> OK, the problem with _tos on 9vx is making some sense.
>
> gcc and 8c disagree on the size of the Tos struct. It's declared the
> same way in 9vx as in
> Plan 9, but 8c thinks it is 56 bytes and gcc thinks it is 88.
>
> The reason is that 9vx is compiled on a 64-bit machine in this case,
> so we are running 32-bit binaries on a 64-bit kernel, i.e. it is
> heterogeneous. Amusing :-)
>
> Guess I'll try 9vx as a 32-bit binary and see what shakes.

since 56%4==0, can't you correct the size mismatch by using
32-bit types to prevent 64-bit 9vx from doubling the size?

- erik

Re: [9fans] problem with building from source on vx32 solved

[ron minnich]

On Thu, Sep 2, 2010 at 4:57 AM, erik quanstrom <quanstro@quanstro.net> wrote:

> since 56%4==0, can't you correct the size mismatch by using
> 32-bit types to prevent 64-bit 9vx from doubling the size?

yep, that's what I'm doing now. I just ran out of time to finish it.

ron

Re: [9fans] problem with building from source on vx32 solved

[ron minnich]

On Thu, Sep 2, 2010 at 8:05 AM, ron minnich <rminnich@gmail.com> wrote:
> On Thu, Sep 2, 2010 at 4:57 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>
>> since 56%4==0, can't you correct the size mismatch by using
>> 32-bit types to prevent 64-bit 9vx from doubling the size?
>
> yep, that's what I'm doing now. I just ran out of time to finish it.

https://rminnich@bitbucket.org/rminnich/vx32

it's done and available. This tree also includes the hack in sched.c
that avoids the exploding 9vx problem. But we need to get a real fix.



ron

Re: [9fans] problem with building from source on vx32 solved

[Charles Forsyth]

>that avoids the exploding 9vx problem. But we need to get a real fix.

i'm sure i posted the real fix to the list (essentially issuing CLD at specific points).

Re: [9fans] problem with building from source on vx32 solved

[ron minnich]

On Fri, Sep 3, 2010 at 1:21 AM, Charles Forsyth <forsyth@terzarima.net> wrote:
>>that avoids the exploding 9vx problem. But we need to get a real fix.
>
> i'm sure i posted the real fix to the list (essentially issuing CLD at specific points).
>
>

I missed it. If you can point me to it I'll give it a try.

ron

Re: [9fans] problem with building from source on vx32 solved

[Philippe Anel]

  You'll also have to include Charles's CLD fix. Regarding sched.c, what
do you call a real fix ?
I mean, this hack just works isn't it ?

Phil;

> it's done and available. This tree also includes the hack in sched.c
> that avoids the exploding 9vx problem. But we need to get a real fix.
>
>
>
> ron
>
>