[9fans] Go and 21-bit runes (and a bit of Go status)

[9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

Just a heads-up to 9fans that the Go build for Plan 9 (386 or ARM) now
expects the underlying platform to be updated with the 21-bit runes
fixes from Bell Labs, the pertinent submission has been accepted and
processed.

Being up to date may not improve the build as much as one may wish,
but not being up to date is guaranteed to be a problem.

Anthony Martin is also in the process of getting another important
patch to the Go distribution approved, while there may be delays
figuring out what to do about the SSE2 extension to the Intel 386
architecture.

Regarding the latter, Plan 9 does not allow floating point
instructions to be executed within note handling, but erring on the
side of caution also forbids instructions such as MOVOU (don't ask me)
which is part of the SSE(2?) extension, but hardly qualifies as a
floating point instruction.

I have yet to see the type of suggestion for Go, but specially for
Plan 9, that resolves all future conflicts on this score: it is a
difficult bit of architectural design on both sides.  I, for one,
would be interested in plausible suggestions.  The 64-bit people may
need them even more, or maybe not at all.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]

> Regarding the latter, Plan 9 does not allow floating point
> instructions to be executed within note handling, but erring on the
> side of caution also forbids instructions such as MOVOU (don't ask me)
> which is part of the SSE(2?) extension, but hardly qualifies as a
> floating point instruction.

movou (movdqu in the manual) is a sse2 data movement instruction.
not all sse2 instructions require that sse be turned on (pause, for example),
but movou uses at least one xmm register so is clearly using the sse
unit, thus requiring that it be turned on.

the go runtime memmove uses movou for memmoves between 33 and 128
bytes.  i only see a 10 cycle difference for these cases on my atom machine,
(maximum 13%), so we're not missing out on much here by not using sse.

the real win, or loss for the plan 9 memmove, is in the short memmoves.
but this is a µbenchmark, and it would be more convincing with a real
world test.

- erik

harness; 8.memmovetest
memmove
1	92.42578 cycles/op
2	81.28125 cycles/op
4	56.47266 cycles/op
8	58.32422 cycles/op
16	62.28516 cycles/op
32	70.26563 cycles/op
64	86.32031 cycles/op
128	118.3125 cycles/op
512	323.5078 cycles/op
1024	587.1094 cycles/op
4096	2119.242 cycles/op
131072	133058.5 cycles/op

rt·memmove
1	20.60156 cycles/op
2	20.34375 cycles/op
4	24.46875 cycles/op
8	22.42969 cycles/op
16	27.45703 cycles/op
32	52.82813 cycles/op
64	79.19531 cycles/op
128	129.1289 cycles/op
512	314.4492 cycles/op
1024	569.9648 cycles/op
4096	2132.297 cycles/op
131072	135378.3 cycles/op

[-- Attachment #2: memmovetest.c --]
[-- Type: text/plain, Size: 1526 bytes --]

#include <u.h>
#include <libc.h>

typedef struct Movtab Movtab;
typedef void* (*Movfn)(void*, void*, ulong);

	u32int	runtimecpuid_edx = 0x4000000;
extern	void*	runtimememmove(void*, void*, ulong);

struct Movtab {
	Movfn	f;
	char	*name;
};

uvlong	hz;
int	sztab[] = {1, 2, 4, 8, 16, 32, 64, 128, 512, 1024, 4096, 128*1024, };
uchar	buf0[128*1024];
uchar	buf1[128*1024];
Movtab	movtab[] = {memmove, "memmove",  runtimememmove, "rt·memmove", };
//Movfn	movtab[] = {memmove, runtimememmove};

uvlong
gethz(void)
{
	char buf[1024], *f[5];
	int n, fd;

	fd = open("/dev/time", OREAD);
	if(fd == -1)
		sysfatal("%s: open /dev/time: %r", argv0);
	n = pread(fd, buf, sizeof buf-1, 0);
	if(n <= 0)
		sysfatal("%s: read /dev/time: %r", argv0);

	buf[n] = 0;
	n = tokenize(buf, f, nelem(f));
	if(n < 4)
		sysfatal("%s: /dev/time: unexpected fmt", argv0);

	return strtoull(f[3], 0, 0);
}

void
inner(Movfn f, ulong sz)
{
	int i;

	for(i = 0; i < 1024; i++)
		f(buf1, buf0, sz);
}

void
main(int argc, char **argv)
{
	int i, j;
	uvlong t[2], c[nelem(movtab)][nelem(sztab)];
//	double dhz;
	Movfn f;

	ARGBEGIN{
	}ARGEND

	hz = gethz();
//	dhz = hz;

	for(i = 0; i < 2; i++){
		print("%s\n", movtab[i].name);
		f = movtab[i].f;
		for(j = 0; j < nelem(sztab); j++){
			cycles(t + 0);
			inner(f, sztab[j]);
			cycles(t + 1);
			c[i][j] = t[1] - t[0];

			print("%d	%g cycles/op\n", sztab[j], c[i][j]/1024.);
			sleep(0);
		}
		print("\n");
	}

	exits("");
}

[-- Attachment #3.1: Type: text/plain, Size: 331 bytes --]

from postmaster@kw:
The following attachment had content that we can't
prove to be harmless.  To avoid possible automatic
execution, we changed the content headers.
The original header was:

	Content-Disposition: attachment; filename=memmove_386.s
	Content-Type: text/plain; charset="UTF-8"
	Content-Transfer-Encoding: 8bit

[-- Attachment #3.2: memmove_386.s.suspect --]
[-- Type: application/octet-stream, Size: 4116 bytes --]

// Inferno's libkern/memmove-386.s
// http://code.google.com/p/inferno-os/source/browse/libkern/memmove-386.s
//
//         Copyright © 1994-1999 Lucent Technologies Inc.  All rights reserved.
//         Revisions Copyright © 2000-2007 Vita Nuova Holdings Limited (www.vitanuova.com).  All rights reserved.
//         Portions Copyright 2009 The Go Authors. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
#define MOVOU	MOVDQU

TEXT runtimememmove(SB), $0
	MOVL	to+0(FP), DI
	MOVL	fr+4(FP), SI
	MOVL	n+8(FP), BX

	// REP instructions have a high startup cost, so we handle small sizes
	// with some straightline code.  The REP MOVSL instruction is really fast
	// for large sizes.  The cutover is approximately 1K.  We implement up to
	// 128 because that is the maximum SSE register load (loading all data
	// into registers lets us ignore copy direction).
tail:
	TESTL	BX, BX
	JEQ	move_0
	CMPL	BX, $2
	JBE	move_1or2
	CMPL	BX, $4
	JBE	move_3or4
	CMPL	BX, $8
	JBE	move_5through8
	CMPL	BX, $16
	JBE	move_9through16
	TESTL	$0x4000000, runtimecpuid_edx(SB) // check for sse2
	JEQ	nosse2
	CMPL	BX, $32
	JBE	move_17through32
	CMPL	BX, $64
	JBE	move_33through64
	CMPL	BX, $128
	JBE	move_65through128
	// TODO: use branch table and BSR to make this just a single dispatch

nosse2:
/*
 * check and set for backwards
 */
	CMPL	SI, DI
	JLS	back

/*
 * forward copy loop
 */
forward:
	MOVL	BX, CX
	SHRL	$2, CX
	ANDL	$3, BX

	REP;	MOVSL
	JMP	tail
/*
 * check overlap
 */
back:
	MOVL	SI, CX
	ADDL	BX, CX
	CMPL	CX, DI
	JLS	forward
/*
 * whole thing backwards has
 * adjusted addresses
 */

	ADDL	BX, DI
	ADDL	BX, SI
	STD

/*
 * copy
 */
	MOVL	BX, CX
	SHRL	$2, CX
	ANDL	$3, BX

	SUBL	$4, DI
	SUBL	$4, SI
	REP;	MOVSL

	CLD
	ADDL	$4, DI
	ADDL	$4, SI
	SUBL	BX, DI
	SUBL	BX, SI
	JMP	tail

move_1or2:
	MOVB	(SI), AX
	MOVB	-1(SI)(BX*1), CX
	MOVB	AX, (DI)
	MOVB	CX, -1(DI)(BX*1)
move_0:
	RET
move_3or4:
	MOVW	(SI), AX
	MOVW	-2(SI)(BX*1), CX
	MOVW	AX, (DI)
	MOVW	CX, -2(DI)(BX*1)
	RET
move_5through8:
	MOVL	(SI), AX
	MOVL	-4(SI)(BX*1), CX
	MOVL	AX, (DI)
	MOVL	CX, -4(DI)(BX*1)
	RET
move_9through16:
	MOVL	(SI), AX
	MOVL	4(SI), CX
	MOVL	-8(SI)(BX*1), DX
	MOVL	-4(SI)(BX*1), BP
	MOVL	AX, (DI)
	MOVL	CX, 4(DI)
	MOVL	DX, -8(DI)(BX*1)
	MOVL	BP, -4(DI)(BX*1)
	RET
move_17through32:
	MOVOU	(SI), X0
	MOVOU	-16(SI)(BX*1), X1
	MOVOU	X0, (DI)
	MOVOU	X1, -16(DI)(BX*1)
	RET
move_33through64:
	MOVOU	(SI), X0
	MOVOU	16(SI), X1
	MOVOU	-32(SI)(BX*1), X2
	MOVOU	-16(SI)(BX*1), X3
	MOVOU	X0, (DI)
	MOVOU	X1, 16(DI)
	MOVOU	X2, -32(DI)(BX*1)
	MOVOU	X3, -16(DI)(BX*1)
	RET
move_65through128:
	MOVOU	(SI), X0
	MOVOU	16(SI), X1
	MOVOU	32(SI), X2
	MOVOU	48(SI), X3
	MOVOU	-64(SI)(BX*1), X4
	MOVOU	-48(SI)(BX*1), X5
	MOVOU	-32(SI)(BX*1), X6
	MOVOU	-16(SI)(BX*1), X7
	MOVOU	X0, (DI)
	MOVOU	X1, 16(DI)
	MOVOU	X2, 32(DI)
	MOVOU	X3, 48(DI)
	MOVOU	X4, -64(DI)(BX*1)
	MOVOU	X5, -48(DI)(BX*1)
	MOVOU	X6, -32(DI)(BX*1)
	MOVOU	X7, -16(DI)(BX*1)
	RET

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[cinap_lenrek]

> Regarding the latter, Plan 9 does not allow floating point
> instructions to be executed within note handling, but erring on the
> side of caution also forbids instructions such as MOVOU (don't ask me)
> which is part of the SSE(2?) extension, but hardly qualifies as a
> floating point instruction.

The reason for FP being forbidden in note handler is that the kernel only saves
the general purpose (Ureg) registers of the interrupted/notified process
context. The fp or xmm registers are *not* saved and a note handler modifying
those (thru fp instructions or sse instructions) would trash these registers
for the program interrupted by the note.

you could save the ureg, and jump out of the note handler with notejmp(),
save the fp/sse registers yourself and then do the handling of the note
outside of the note context. (this is how signals are implemented in ape).

or we change the kernel to save the fp registers in notify() as well,
pushing them on the user stack and restoring them on noted() just like
the Ureg.

or GO could just stop using *OMG-OPTIMIZED* SSE memmove() in the note
handler.

--
cinap

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> or GO could just stop using *OMG-OPTIMIZED* SSE memmove() in the note
> handler.

But it would not stop users from doing so, so at minimum we'd have to
detect the abuse and report it, rather than crash.

Saving the entire register space would be expensive for all
well-behaved processes and avoiding the micro-optimisation in
memmove() (a Plan 9-specific option) would be my recommendation.  But
dragons do lurk and will need to be slain.

Incidentally, do the FP registers need to be saved for the sake of
MOVOUs?  Or should I ask whether MOVOUs clobber registers not saved
before note handling?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> movou (movdqu in the manual) is a sse2 data movement instruction.
> not all sse2 instructions require that sse be turned on (pause, for example),
> but movou uses at least one xmm register so is clearly using the sse
> unit, thus requiring that it be turned on.

I see Erik answers my question: xmm registers may be clobbered.  I
suppose they could be saved in the Go runtime, if absolutely
essential?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[cinap_lenrek]

the saving isnt the problem. the kernel already flushes the fp registers
to the process fpsave area on notify. its just that we do *not* copy
the registers to the user stack, but save them in the process fpsave
area.

as theres just just one fpsave area in the process, and not one for
notes and one for normal code, note handler is forbitten to use fp again.

its not for the sake of movou. its for the sake of the process interrupted
by the note.

say, you have a programm that gets just interrupted by note while in
that omgoptimized sse memmove() where it just loaded some chunks into
XMM0 register, and then the note fires.

then the note handler does memmove itself modifying XMM0 itself loading
it with something completely different. then note handler finishes
continuing the original programm, then XMM0 would contain the garbage
from the note handler! it would look for the program like if registers
randomly change under it!

--
cinap

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> then the note handler does memmove itself modifying XMM0 itself loading
> it with something completely different. then note handler finishes
> continuing the original programm, then XMM0 would contain the garbage
> from the note handler! it would look for the program like if registers
> randomly change under it!

True enough.  If memmove() were the only problem, solving it would be
easy.  One option: drop MOVOU altogether; another option: save xmm8
(following from what you said).  But it's the whole FP edifice that's
relevant in the bigger picture: it may be bad practice, but what if I
want to compute the next iteration for pi in a note handler?  How is
the Go runtime going to stop me, or at least make sure I am aware that
I should not be doing it rather than give me an incorrect answer that
I then use to fire a ballistic missile at the wrong target?

(I concede that I have not thought about this much - feel free to
think you have to explain this to an idiot.)

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Kurt H Maier]

On Sun, Jun 02, 2013 at 05:54:14PM +0200, lucio@proxima.alt.za wrote:
> I should not be doing it rather than give me an incorrect answer that
> I then use to fire a ballistic missile at the wrong target?


I knew Google was up to something.

khm

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> I knew Google was up to something.

Google?  Who's Google?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

cinap_lenrek@gmx.de once said:
> or GO could just stop using *OMG-OPTIMIZED* SSE memmove()
> in the note handler.

This is exactly what I did in my patch. This was just
a regression. Someone changed memmove a few weeks ago.

Nothing to see here.

  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> I see Erik answers my question: xmm registers may be clobbered.  I
> suppose they could be saved in the Go runtime, if absolutely
> essential?

no, they can not.  saving registers is something that is done on
context switch by the scheduler, and the go runtime is not
involved in context switching; this is a user-level transparent thing.

there are things that could be done.  but before getting radical in a
hurry, is there any place other than runtime·memmove() that
would use sse in a note handler?

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> there are things that could be done.  but before getting radical in a
> hurry, is there any place other than runtime·memmove() that
> would use sse in a note handler?

I presumed, perhaps incorrectly, that users are allowed to write their
own signal handlers and are not prohibited from using floating point
instructions in them.

++L

[9fans] More Go status

[lucio]

I have applied Anthony's CL 9796043 together with some tweaks to
pkg/runtime/sys_plan9_386.s which I will pass on to Anthony as soon as
I can; this has made it possible to complete the first set of run.rc
tests without the major incidents I used to see.  Some tests still
fail, but I wasn't expecting miracles: I believe Anthony (and maybe
others) are still working on changes I am not familiar with.

Still, I think I can report some progress, specially towards being
able to run a Go builder for the Plan 9/386 platform.  What we are
going to do about builders for the various offshoots I can't tell.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Jun  3 01:39:33 EDT 2013, lucio@proxima.alt.za wrote:
> > there are things that could be done.  but before getting radical in a
> > hurry, is there any place other than runtime·memmove() that
> > would use sse in a note handler?
> 
> I presumed, perhaps incorrectly, that users are allowed to write their
> own signal handlers and are not prohibited from using floating point
> instructions in them.

there are no signals in plan 9.  i assume that you mean note handlers.

note handlers may be user set but may not do floating point.
see notify(2) for documentation.

as cinap pointed out, this is because the fp save area for the process
is (potentially) busy when the note is delivered.  think of it this way
here up = user process.  this is the kernel convention.  some details
of this are slightly incorrect for clarity.

user			kernel
generates fault
			saves up registers,
			saves up floating point, sets fp state to FPillegal
				the save area is already used.
starts note handler
generates fp fault
			saves up registers,
			finds that we're in FPillegal state, and process
			becomes Broken

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> starts note handler
> generates fp fault
> 			saves up registers,
> 			finds that we're in FPillegal state, and process
> 			becomes Broken

That conflicts with Go's intent, I've no idea how one addresses it.
Ideally, the kernel should trigger a condition that the Go runtime can
deal with.  I can see why it is unlikely.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Jun  3 12:35:40 EDT 2013, lucio@proxima.alt.za wrote:
> > starts note handler
> > generates fp fault
> > 			saves up registers,
> > 			finds that we're in FPillegal state, and process
> > 			becomes Broken
>
> That conflicts with Go's intent, I've no idea how one addresses it.
> Ideally, the kernel should trigger a condition that the Go runtime can
> deal with.  I can see why it is unlikely.

if by intent, you mean that go is using xmm registers as if they were
general purpose registers, then the solution is to stop doing that.
and there's such a patch already.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> if by intent, you mean that go is using xmm registers as if they were
> general purpose registers, then the solution is to stop doing that.
> and there's such a patch already.

No, Go's intent is to minimise runtime surprises.  It is possible to
define signal (calling them notes does not change their nature)
handlers and nothing in the Go specifications compels the user not to
use floating point instructions in such handlers.  It would also not
be possible to enforce such restrictions in known implementations of
Go and that creates a conflict.  Unless I'm straying into areas I am
ignorant of, something needs to be done for Go and Plan 9 to coexist
amicably, the current conditions are prone to Go-incompatible
behaviour.

Solving the memmove() problem just puts the crisis off for a while.
Once Go is being used on Plan 9 and especially if a user expects
portability from more common platforms, this particular
incompatibility is likely to hurt.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Jun  3 13:05:07 EDT 2013, lucio@proxima.alt.za wrote:
> > if by intent, you mean that go is using xmm registers as if they were
> > general purpose registers, then the solution is to stop doing that.
> > and there's such a patch already.
>
> No, Go's intent is to minimise runtime surprises.  It is possible to
> define signal (calling them notes does not change their nature)
> handlers and nothing in the Go specifications compels the user not to
> use floating point instructions in such handlers.  It would also not
> be possible to enforce such restrictions in known implementations of

signals are not compatable with notes.  i don't think this
can be truely portable code anyway.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bakul Shah]

On Jun 3, 2013, at 10:07 AM, erik quanstrom <quanstro@quanstro.net> wrote:

> On Mon Jun  3 13:05:07 EDT 2013, lucio@proxima.alt.za wrote:
>>> if by intent, you mean that go is using xmm registers as if they were
>>> general purpose registers, then the solution is to stop doing that.
>>> and there's such a patch already.
>> 
>> No, Go's intent is to minimise runtime surprises.  It is possible to
>> define signal (calling them notes does not change their nature)
>> handlers and nothing in the Go specifications compels the user not to
>> use floating point instructions in such handlers.  It would also not
>> be possible to enforce such restrictions in known implementations of
> 
> signals are not compatable with notes.  i don't think this
> can be truely portable code anyway.

Not compatible but signals have similar restrictions. A signal may be delivered at any time where any state maintained in usercode may be inconsistent. In particular use of any non-reentrant function can cause trouble. Used to be, you don't use floating pt. in signal handlers as that would require the kernel to save more state, slowing down signal delivery, or it could cause another trap where the kernel can do the lazy saving trick. Most mallocs are non reentrant as well and you shouldn't use malloc in a handler. All in all a very restricted environment. 

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

On 3 June 2013 18:04, <lucio@proxima.alt.za> wrote:

> No, Go's intent is to minimise runtime surprises.


It's not a runtime surprise to Go programmers, since no ordinary Go code
runs in that note handler.
There's a rather elaborate implementation to convert notes (or signals)
into something acceptable
to the rest of the Go runtime. It does as little as it can.

[-- Attachment #2: Type: text/html, Size: 748 bytes --]

[9fans] SSE in a note handler

[Steve Simon]

Linuxemu runs all its linux api emulation inside a note handler,
now it does no specific SSE code (its the linux SSE code that was
troublesome here), but if there are any sse library functions in
plan9 (like memmove) then linuxemu may run them from a note handler.

Just an opinion from a different side.

-Steve

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

Hi Anthony,

is this the patch set you're referring to?
https://codereview.appspot.com/download/issue9796043_58001.diff

if so, is there a diff set for go1.2?

Thanks,
-Skip



On Sun, Jun 2, 2013 at 12:37 PM, Anthony Martin <ality@pbrane.org> wrote:

> cinap_lenrek@gmx.de once said:
> > or GO could just stop using *OMG-OPTIMIZED* SSE memmove()
> > in the note handler.
>
> This is exactly what I did in my patch. This was just
> a regression. Someone changed memmove a few weeks ago.
>
> Nothing to see here.
>
>   Anthony
>
>

[-- Attachment #2: Type: text/html, Size: 1177 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

Skip Tavakkolian <skip.tavakkolian@gmail.com> once said:
> is this the patch set you're referring to?
> https://codereview.appspot.com/download/issue9796043_58001.diff
>
> if so, is there a diff set for go1.2?

Hi Skip,

There were a few problems with that CL that I'm just now
starting to iron out so I extracted out the memmove fix
into a separate CL.

The patch sets that you'll need to build Go 1.2 on Plan 9 are:

    cmd/8g: work around 64-bit bug in 8c for Plan 9
    https://codereview.appspot.com/14521056

    build: do not use the host's libbio on Plan 9
    https://codereview.appspot.com/14604047

    runtime: do not use memmove in the Plan 9 signal handler
    https://codereview.appspot.com/34640045

The Go tree is still in a code freeze but I'll have those
CLs submitted as soon as it reopens.

Also, we have three months (until the Go 1.3 code freeze) to
get the Plan 9 port passing all tests on the build dashboard
or it will have to move outside the main repository:

    https://code.google.com/p/go-wiki/wiki/PortingPolicy

If you want to help with this effort, please join the go-plan9
mailing list (https://groups.google.com/d/forum/go-plan9).

Cheers,
  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> The Go tree is still in a code freeze but I'll have those
> CLs submitted as soon as it reopens.
>
> Also, we have three months (until the Go 1.3 code freeze) to
> get the Plan 9 port passing all tests on the build dashboard
> or it will have to move outside the main repository:

there is no demotivation in open source as powerful as a threat.

is the threat standing?  that is, if the plan 9 port is broken again
when 1.5 rolls around in just a few more months, does the plan 9
port get booted then, too?

personally, i think a preemtive strike is in order.  a lot of time
has been spent chasing basic broken-with-merge issues, the
rather superfluous libbio macros, the self-inflicted memmove
problem, &c.  those are just recent issues.  given the rate of go
development, i would think this is likely to continue.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> is the threat standing?  that is, if the plan 9 port is broken again
> when 1.5 rolls around in just a few more months, does the plan 9
> port get booted then, too?

The threat is real: Plan 9 is a burden for the developers and lack of
feedback is a valid cause for dismissal.  Of the two-prong threat,
only lack of builders is continuous: that the build may break and be
neglected for a while is not as serious as knowing that no one is
paying attention or, worse, that no one even knows that there is a
problem.

But the port is also in trouble.  I dread the need to reconcile
Gorka's work on ARM with the 1.2 release, I don't really know where to
start.  And as far as 386 and amd64 goes, I don't even know or grasp
what ails the Bell Labs release (and I agree with you that the most
recent adjustments are feeble at best), nevermind where we're standing
with 9atom, 9front and the few other versions out in the wild.

The solution is not with "open source" but with rolling up one's
sleeves and figuring out how to converge as much as possible the
different offerings.  I don't think Go needs to be thrown away, I think
it is a motivating force itself, but in this particular case we need
some leadership to guide short-term development in a better direction.

Listing the outstanding issues, technical or political, would be my
starting point, but I was not involved in most of the (not so) recent
in-fighting and I don't know how that ought to be resolved.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

> I don't think Go needs to be thrown away, I think it is a motivating
> force itself,

Why?

khm

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 10:01:48 EST 2013, lucio@proxima.alt.za wrote:
> > is the threat standing?  that is, if the plan 9 port is broken again
> > when 1.5 rolls around in just a few more months, does the plan 9
> > port get booted then, too?
>
> The threat is real: Plan 9 is a burden for the developers and lack of

perhaps we're seperated by a common language.  "standing" in the sense
that for each new release does the same condition hold: if it does not
pass all the tests, it is evicted from the main line.

> The solution is not with "open source" but with rolling up one's
> sleeves and figuring out how to converge as much as possible the

given that most of the time is spent bailing water out of the boat,
and fixing things that weren't previously broken, it seems to me that
like anything it's a two-way street.

anyway, what's the argument for not just forking?

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

It would be very hard to replicate what Go can do for Plan 9 with something
else.  There is a large and growing collection of packages that make it
possible to deal with the dizzying number of protocols and APIs that are
today's WWW.  Another advantage of Go is that, like Limbo, it enables the
young enthusiastic 9fans to contribute meaningful work in a shorter time
that would otherwise be required (e.g. mastering C, etc.). I think, in the
long run, the pain of the teething problems are worth the effort.

This current situation is not insurmountable;  it seems that we have enough
people who are interested and a handful who are contributors that we can
make this happen with some coordination.



On Mon, Dec 2, 2013 at 6:33 AM, erik quanstrom <quanstro@quanstro.net>wrote:

> > The Go tree is still in a code freeze but I'll have those
> > CLs submitted as soon as it reopens.
> >
> > Also, we have three months (until the Go 1.3 code freeze) to
> > get the Plan 9 port passing all tests on the build dashboard
> > or it will have to move outside the main repository:
>
> there is no demotivation in open source as powerful as a threat.
>
> is the threat standing?  that is, if the plan 9 port is broken again
> when 1.5 rolls around in just a few more months, does the plan 9
> port get booted then, too?
>
> personally, i think a preemtive strike is in order.  a lot of time
> has been spent chasing basic broken-with-merge issues, the
> rather superfluous libbio macros, the self-inflicted memmove
> problem, &c.  those are just recent issues.  given the rate of go
> development, i would think this is likely to continue.
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 2167 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> Quoting lucio@proxima.alt.za:
>
>> I don't think Go needs to be thrown away, I think it is a motivating
>> force itself,
>
> Why?
>
It's my opinion.  Do you have a problem with that?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> anyway, what's the argument for not just forking?

I like Go's portability across platforms.  Having a version for Plan 9
that is inconsistent in this respect is exactly the opposite of what I
want.  Nothing stops anyone from forking, but losing the code review
by a community of experts will definitely bother me enormously.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> This current situation is not insurmountable;  it seems that we have enough
> people who are interested and a handful who are contributors that we can
> make this happen with some coordination.

We will almost certainly need more focus from Bell Labs or at least
less reluctance.  More than ever, they hold the fate of Plan 9 in
their hands and it may be necessary to snatch it from their grasp.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeff Sickel]

Contributions would be great, but we’re swimming upstream to spawn and die.

The libbio changes are a good example.  More than a month before the release
candidates started rolling out there was one patch (https://codereview.appspot.com/14604047/)
to try and reconcile the changes.  I also added a patch that let us keep our
version of libbio (https://codereview.appspot.com/15750047/) and still pick
up the required four new functions that happen to be replicated elsewhere.
Neither patch is ideal, and neither has made progress in getting rolled into
the release.

It takes time, more than effort, to keep up with the various Go developer
lists where changes actually take place.  The case of libbio changes happened
at a time when no one in the Plan 9 community was really looking at the
threads until it was too late to comment.  Now we’re caught trying to
keep up again.  So either we start posting codereview emails to the 9fans
list, or we find a better way to collaborate and make Plan 9 a first tier
target for Go.



On Dec 2, 2013, at 10:10 AM, Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:

> It would be very hard to replicate what Go can do for Plan 9 with something else.  There is a large and growing collection of packages that make it possible to deal with the dizzying number of protocols and APIs that are today's WWW.  Another advantage of Go is that, like Limbo, it enables the young enthusiastic 9fans to contribute meaningful work in a shorter time that would otherwise be required (e.g. mastering C, etc.). I think, in the long run, the pain of the teething problems are worth the effort.
> 
> This current situation is not insurmountable;  it seems that we have enough people who are interested and a handful who are contributors that we can make this happen with some coordination.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> It takes time, more than effort, to keep up with the various Go developer
> lists where changes actually take place.  The case of libbio changes happened
> at a time when no one in the Plan 9 community was really looking at the
> threads until it was too late to comment.  Now we’re caught trying to
> keep up again.  So either we start posting codereview emails to the 9fans
> list, or we find a better way to collaborate and make Plan 9 a first tier
> target for Go.

I know that David and I are monitoring golang-dev pretty closely.  By
the same token, I suspect that Gorka and Nemo don't (I don't know this
for a fact, I'm speculating).  What would help immensely would be if
we could advertise each of the willing contributors' own interest in
the development so that we can fill gaps when they arise.

For example, I have no idea where Anthony Martin is heading with the
notes code, but I am aware that he is probably ahead of the pack in
that direction.  Should I spot something in golang-dev that I
understand may affect coding Plan 9's note handling for Go, I would
like to be able to alert him.  Not knowing how closely he himself
monitors golang-dev, or how much he would appreciate being nagged,
right now I err on the side of caution.

It will take time to build a team of contributors that is familiar
with everyone else's role, but that is a good reason to get cracking
now, rather than later.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> I know that David and I are monitoring golang-dev pretty closely.  By
> the same token, I suspect that Gorka and Nemo don't (I don't know this
> for a fact, I'm speculating).  What would help immensely would be if
> we could advertise each of the willing contributors' own interest in
> the development so that we can fill gaps when they arise.
>
> For example, I have no idea where Anthony Martin is heading with the
> notes code, but I am aware that he is probably ahead of the pack in
> that direction.  Should I spot something in golang-dev that I
> understand may affect coding Plan 9's note handling for Go, I would
> like to be able to alert him.  Not knowing how closely he himself
> monitors golang-dev, or how much he would appreciate being nagged,
> right now I err on the side of caution.
>
> It will take time to build a team of contributors that is familiar
> with everyone else's role, but that is a good reason to get cracking
> now, rather than later.

all this is moot unless we can get plan 9 integrated into go's automatic
build system.  is this doable?  i have resources to make this happen if
it is.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 12:22:09 EST 2013, lucio@proxima.alt.za wrote:
> > anyway, what's the argument for not just forking?
>
> I like Go's portability across platforms.  Having a version for Plan 9
> that is inconsistent in this respect is exactly the opposite of what I
> want.  Nothing stops anyone from forking, but losing the code review
> by a community of experts will definitely bother me enormously.

i thought the language had become more ossified.  as time goes on
this is less and less of a concern.  i see more performance changes than
anything these days.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

>> Quoting lucio@proxima.alt.za:
>>
>>> I don't think Go needs to be thrown away, I think it is a motivating
>>> force itself,
>>
>> Why?
>>
> It's my opinion.  Do you have a problem with that?
>

Why do you hold this opinion?  While your defensiveness is hilarious, it's a
simple matter of curiosity.  I'm trying to understand the fervor
behind another
language-cum-fashion-accessory.  Specifically:  how will Go enable good things
that, say, python has not?  Which good things?  Are all of them web based?

I'm not asking this idly.  I have resources available for projects like this,
but I'm tired of wasting them on projects that go nowhere once the developers'
attention span shifts to whatever hip new thing is promoted by $COMPANY.


khm

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Please explain this assertion (e.g. TSEMACQUIRE contradicts it).



On Mon, Dec 2, 2013 at 9:25 AM, <lucio@proxima.alt.za> wrote:

> > This current situation is not insurmountable;  it seems that we have
> enough
> > people who are interested and a handful who are contributors that we can
> > make this happen with some coordination.
>
> We will almost certainly need more focus from Bell Labs or at least
> less reluctance.  More than ever, they hold the fate of Plan 9 in
> their hands and it may be necessary to snatch it from their grasp.
>
> ++L
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1006 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 340 bytes --]

I believe one or two 9fans have provided systems for this purpose.


On Mon, Dec 2, 2013 at 10:33 AM, erik quanstrom <quanstro@labs.coraid.com>wrote:
>
>
> all this is moot unless we can get plan 9 integrated into go's automatic
> build system.  is this doable?  i have resources to make this happen if
> it is.
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 738 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 14:17:04 EST 2013, skip.tavakkolian@gmail.com wrote:

> I believe one or two 9fans have provided systems for this purpose.
>
>
> On Mon, Dec 2, 2013 at 10:33 AM, erik quanstrom <quanstro@labs.coraid.com>wrote:
> >
> >
> > all this is moot unless we can get plan 9 integrated into go's automatic
> > build system.  is this doable?  i have resources to make this happen if
> > it is.

if so, something's gone very wrong.  usually the purpose of automatic
build machines it to insure that things build and hopefully pass a few basic
tests before the code can be checked in.  i'd assumed that this was not working
since clearly the plan 9 386 port is broken on a regular basis.

if this can't be sorted out, then tracking mainline go is going to be quite
hard, maybe even impractical.  it's so much harder to get things fixed after
code is merged than before.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Christopher Nielsen]

On Mon, Dec 2, 2013 at 10:33 AM, erik quanstrom
<quanstro@labs.coraid.com> wrote:
> all this is moot unless we can get plan 9 integrated into go's automatic
> build system.  is this doable?  i have resources to make this happen if
> it is.
>
> - erik

It's definitely doable. I have a builder key that I've provided to
David du Colombier. My Plan 9 install needs a little love, but it's
almost ready for full-time dev. I have some spare cycles to work on
the port, too. I feel strongly that we can get the port in shape by
the March 1 deadline.

--
Christopher Nielsen
"They who can give up essential liberty for temporary safety, deserve
neither liberty nor safety." --Benjamin Franklin
"The tree of liberty must be refreshed from time to time with the
blood of patriots & tyrants." --Thomas Jefferson

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

let me preface my original statement with the following legalese weasel
clause: "at some point in the past, i was let to believe that..."



On Mon, Dec 2, 2013 at 11:26 AM, erik quanstrom <quanstro@labs.coraid.com>wrote:

> On Mon Dec  2 14:17:04 EST 2013, skip.tavakkolian@gmail.com wrote:
>
> > I believe one or two 9fans have provided systems for this purpose.
> >
> >
> > On Mon, Dec 2, 2013 at 10:33 AM, erik quanstrom <
> quanstro@labs.coraid.com>wrote:
> > >
> > >
> > > all this is moot unless we can get plan 9 integrated into go's
> automatic
> > > build system.  is this doable?  i have resources to make this happen if
> > > it is.
>
> if so, something's gone very wrong.  usually the purpose of automatic
> build machines it to insure that things build and hopefully pass a few
> basic
> tests before the code can be checked in.  i'd assumed that this was not
> working
> since clearly the plan 9 386 port is broken on a regular basis.
>
> if this can't be sorted out, then tracking mainline go is going to be quite
> hard, maybe even impractical.  it's so much harder to get things fixed
> after
> code is merged than before.
>
> - erik
>

[-- Attachment #2: Type: text/html, Size: 1809 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 14:13:51 EST 2013, skip.tavakkolian@gmail.com wrote:
> Please explain this assertion (e.g. TSEMACQUIRE contradicts it).
>
> On Mon, Dec 2, 2013 at 9:25 AM, <lucio@proxima.alt.za> wrote:
>
> > > This current situation is not insurmountable;  it seems that we have
> > enough
> > > people who are interested and a handful who are contributors that we can
> > > make this happen with some coordination.
> >
> > We will almost certainly need more focus from Bell Labs or at least
> > less reluctance.  More than ever, they hold the fate of Plan 9 in
> > their hands and it may be necessary to snatch it from their grasp.

i don't think that's a contradiction.  it's a datapoint.  a generalization
can't be made without a number of datapoints.

the generalization isn't useful though.

let's have a list of things that need doing on the plan 9 side, and let's
see if those demands on the system are really reasonable.  i'm committed
to supporting go in 9atom to the extent that it does not compromise
or corrupt the system substantially.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bakul Shah]

On Mon, 02 Dec 2013 13:33:34 EST erik quanstrom <quanstro@labs.coraid.com> wrote:
> all this is moot unless we can get plan 9 integrated into go's automatic
> build system.  is this doable?  i have resources to make this happen if
> it is.

One simple thing that can be done is to set up a cron job to
fetch updates and if anything changed, to rebuild and mail if
there are any essential differences from previous run.

Requires no additional cooperation from anyone else and can
work for catching breakage early in any third party software.

Another thing that can be done is to maintain a separate
"ports" repository, where you keep patches required to build
but that are not yet integrated in any upstream repo + any
other bits to a s/w package in a platform specific manner.
This is how FreeBSD ports work. Everyone shouldn't have to
track down patches and try merging them individually.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

Some coordination among people who are 9fan-gonut, 9fan-godev or 9dev would
be helpful; at least it will reduce unnecessary head-scratching.

On Mon, Dec 2, 2013 at 11:37 AM, Bakul Shah <bakul@bitblocks.com> wrote:

> On Mon, 02 Dec 2013 13:33:34 EST erik quanstrom <quanstro@labs.coraid.com>
> wrote:
> > all this is moot unless we can get plan 9 integrated into go's automatic
> > build system.  is this doable?  i have resources to make this happen if
> > it is.
>
> One simple thing that can be done is to set up a cron job to
> fetch updates and if anything changed, to rebuild and mail if
> there are any essential differences from previous run.
>
> Requires no additional cooperation from anyone else and can
> work for catching breakage early in any third party software.
>
> Another thing that can be done is to maintain a separate
> "ports" repository, where you keep patches required to build
> but that are not yet integrated in any upstream repo + any
> other bits to a s/w package in a platform specific manner.
> This is how FreeBSD ports work. Everyone shouldn't have to
> track down patches and try merging them individually.
>
>

[-- Attachment #2: Type: text/html, Size: 1615 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

python on plan9 can't even handle the codereview extension.


On Mon, Dec 2, 2013 at 10:39 AM, Kurt H Maier <khm@sciops.net> wrote:

> Specifically:  how will Go enable good things
> that, say, python has not?
>
> khm
>
>
>

[-- Attachment #2: Type: text/html, Size: 695 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 15:10:29 EST 2013, skip.tavakkolian@gmail.com wrote:

> python on plan9 can't even handle the codereview extension.

i believe that's false.  jas' port does a lot of things the
prior port does not.  it's on bitbucket.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[David du Colombier]

The Go builder is now up and running, thanks to the build keys
offered by Chris and Lucio today.

We'll see how it goes over time.

--
David du Colombier

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

i'm getting it now. thanks.


On Mon, Dec 2, 2013 at 12:11 PM, erik quanstrom <quanstro@labs.coraid.com>wrote:

> On Mon Dec  2 15:10:29 EST 2013, skip.tavakkolian@gmail.com wrote:
>
> > python on plan9 can't even handle the codereview extension.
>
> i believe that's false.  jas' port does a lot of things the
> prior port does not.  it's on bitbucket.
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 855 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[David du Colombier]

> > python on plan9 can't even handle the codereview extension.
>
> i believe that's false.  jas' port does a lot of things the
> prior port does not.  it's on bitbucket.

I agree with Erik. Jeff Sickel did a very good job on the
modern Python port.

If you are afraid to compile it, I'm providing up-to-date
binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:

http://www.9legacy.org/download/cpython.mkfs.bz2

--
David du Colombier

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 15:25:33 EST 2013, 0intro@gmail.com wrote:
> > > python on plan9 can't even handle the codereview extension.
> >
> > i believe that's false.  jas' port does a lot of things the
> > prior port does not.  it's on bitbucket.
>
> I agree with Erik. Jeff Sickel did a very good job on the
> modern Python port.
>
> If you are afraid to compile it, I'm providing up-to-date
> binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:

it won't compile unless you're running 9atom. or have integrated
the (extensive) changes to ape, especially in the sockets area.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bakul Shah]

On Mon, 02 Dec 2013 15:38:21 EST erik quanstrom <quanstro@labs.coraid.com> wrote:
> On Mon Dec  2 15:25:33 EST 2013, 0intro@gmail.com wrote:
> > > > python on plan9 can't even handle the codereview extension.
> > >
> > > i believe that's false.  jas' port does a lot of things the
> > > prior port does not.  it's on bitbucket.
> >
> > I agree with Erik. Jeff Sickel did a very good job on the
> > modern Python port.
> >
> > If you are afraid to compile it, I'm providing up-to-date
> > binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:
>
> it won't compile unless you're running 9atom. or have integrated
> the (extensive) changes to ape, especially in the sockets area.

Another reason to break out ports. Ideally they should
work any plan9 fork.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[David du Colombier]

> it won't compile unless you're running 9atom. or have integrated
> the (extensive) changes to ape, especially in the sockets area.

Yes, I had to replace /sys/src/ape and /sys/include/ape with
9atom versions to compile Python. I've put my notes here:

http://www.9legacy.org/9legacy/doc/python/notes

--
David du Colombier

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 15:44:27 EST 2013, bakul@bitblocks.com wrote:
> On Mon, 02 Dec 2013 15:38:21 EST erik quanstrom <quanstro@labs.coraid.com> wrote:
> > On Mon Dec  2 15:25:33 EST 2013, 0intro@gmail.com wrote:
> > > > > python on plan9 can't even handle the codereview extension.
> > > >
> > > > i believe that's false.  jas' port does a lot of things the
> > > > prior port does not.  it's on bitbucket.
> > >
> > > I agree with Erik. Jeff Sickel did a very good job on the
> > > modern Python port.
> > >
> > > If you are afraid to compile it, I'm providing up-to-date
> > > binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:
> >
> > it won't compile unless you're running 9atom. or have integrated
> > the (extensive) changes to ape, especially in the sockets area.
>
> Another reason to break out ports. Ideally they should
> work any plan9 fork.

i don't see how that follows.  changes were needed to ape.
the patches are sitting in /n/sources/patch.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bakul Shah]

On Mon, 02 Dec 2013 15:45:53 EST erik quanstrom <quanstro@labs.coraid.com> wrote:
> On Mon Dec  2 15:44:27 EST 2013, bakul@bitblocks.com wrote:
> > On Mon, 02 Dec 2013 15:38:21 EST erik quanstrom <quanstro@labs.coraid.com>
> wrote:
> > > On Mon Dec  2 15:25:33 EST 2013, 0intro@gmail.com wrote:
> > > > > > python on plan9 can't even handle the codereview extension.
> > > > >
> > > > > i believe that's false.  jas' port does a lot of things the
> > > > > prior port does not.  it's on bitbucket.
> > > >
> > > > I agree with Erik. Jeff Sickel did a very good job on the
> > > > modern Python port.
> > > >
> > > > If you are afraid to compile it, I'm providing up-to-date
> > > > binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:
> > >
> > > it won't compile unless you're running 9atom. or have integrated
> > > the (extensive) changes to ape, especially in the sockets area.
> >
> > Another reason to break out ports. Ideally they should
> > work any plan9 fork.
>
> i don't see how that follows.  changes were needed to ape.
> the patches are sitting in /n/sources/patch.

I am suggesting breaking out just the diffs and new mkfiles in
a separate tree so that one can do

    mk all && mk install

This can fetch the necessary bits, apply patches, build, test,
create downloadable binaries (with crypto signatures if you
care) etc.  As part of this it can mk any dependencies.  One
of which can pull in changes for ape. Right now all this seems
rather manual. As more changes are merged back in the upstream
sources, port diffs can reduce.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> I am suggesting breaking out just the diffs and new mkfiles in
> a separate tree so that one can do
>
>     mk all && mk install
>
> This can fetch the necessary bits, apply patches, build, test,
> create downloadable binaries (with crypto signatures if you
> care) etc.  As part of this it can mk any dependencies.  One
> of which can pull in changes for ape. Right now all this seems
> rather manual. As more changes are merged back in the upstream
> sources, port diffs can reduce.

patches are the plan 9 standard way of doing
things.  they've been submitted, and can be
applied by anyone.

or, you can run 9atom.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

wait! so, you had to make changes to Plan 9 to support Python? :)

(sorry, couldn't resist)


On Mon, Dec 2, 2013 at 12:38 PM, erik quanstrom <quanstro@labs.coraid.com>wrote:

> On Mon Dec  2 15:25:33 EST 2013, 0intro@gmail.com wrote:
> > > > python on plan9 can't even handle the codereview extension.
> > >
> > > i believe that's false.  jas' port does a lot of things the
> > > prior port does not.  it's on bitbucket.
> >
> > I agree with Erik. Jeff Sickel did a very good job on the
> > modern Python port.
> >
> > If you are afraid to compile it, I'm providing up-to-date
> > binaries (Python 2.7.5 and Mercurial 2.8.1) for Plan 9:
>
> it won't compile unless you're running 9atom. or have integrated
> the (extensive) changes to ape, especially in the sockets area.
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 1363 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeff Sickel]

Well, not Plan 9 per se, but APE, yes.  APE was crusty, old, and
not supporting the bulk of the POSIX APIs that all the current
systems are using.

So yes, changes to APE are required.


On Dec 2, 2013, at 3:06 PM, Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:

> wait! so, you had to make changes to Plan 9 to support Python? :)
> 
> (sorry, couldn't resist)
> 

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 16:46:10 EST 2013, jas@corpus-callosum.com wrote:
> Well, not Plan 9 per se, but APE, yes.  APE was crusty, old, and
> not supporting the bulk of the POSIX APIs that all the current
> systems are using.
>
> So yes, changes to APE are required.

but only insofar as ape is wrong!  this is a big difference with go.
more details to come.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 16:08:20 EST 2013, skip.tavakkolian@gmail.com wrote:
> wait! so, you had to make changes to Plan 9 to support Python? :)
>
> (sorry, couldn't resist)

i'll take the bait.

the changes were very different from the changes for go.  python
is an ape program, and ape has been very neglected.  many
of these changes were not strictly necessary for the python port.
none are really python specific.  they fix broken things in ape.
go needs new system calls, and makes other demands, like the
silly libbio macros.

here's a highlevel overview of what was done.  stars by the necessary
changes:

*0.  support for ssl was added to ape by importing mp, sec, bio into
ape.  also compile crypt from /sys/src/libc/port (necessary to avoid using openssl)

*1.  <inttypes.h> was moved to /$objtype/include/ape because it's not
portable.  (necessary for amd64 port)

*2.  inet_ntop, inet_pton, getaddrinfo (and friends) were added.
python doesn't work well with old-style name lookup.

*3.  all the system calls have had their signatures corrected e.g.:
	- extern	int	_RENDEZVOUS(unsigned long, unsigned long);
	+ extern	void*	_RENDEZVOUS(void*, void*);
this is required for amd64.  change brk() and _buf() accordingly.

*4.  fix frexp, modf, (endian issues, subnormal #s) and copysign().

*5.  make a lame attempt to deal with wait4() vs rfork.

*6.  _IO_getc needs to return EOF on eof.

*7.  fix %p  for 64-bit add vfscanf in stdio; use USED() not #pragma ref

*8.  getsrvbyaddr() remember that htons!

*9.  further envp rework.

*10.  socketpair: wrong signature.

and unnecessary but helpful fixes:

0.  all ape compoents are built with -VTw to prevent simple linker
mistakes.  snprintf() now always uses the c99 definition to prevent link errors with -T.

*1.  support for ssl was added to ape by importing mp, sec, bio into
ape.  (necessary to avoid using openssl)

*2.  <inttypes.h> was moved to /$objtype/include/ape because it's not
portable.  (necessary for amd64 port)

3. ip6 support was added.

*4.  inet_ntop, inet_pton, getaddrinfo (and friends) were added.
python doesn't work well with old-style name lookup.

5.  librexexp was fixed in line with fixes to the normal lib.

6.  Rune and wchar_t are an uint/int for 21-bit runes.  decode 21-bit
runes in mbwc.c

7.  getcallerpc/getfcr implemented for all arches.

8. make qsort 64-bit safe.

9.  fix const in strcspn, strpbrk, strrchr, strspn, strstr, rename, pathconf

10.  fix types in setuid, setgid, signal, mkfifo, getgrent, getpwent

11.  fix conflict between bind() [libdraw] and bind [socket].

12.  fmt: fix %p; fix %C (21-bit rune)

13.  utf: 21-bit runes

14.  libv: fix impossible definition of nap().

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

erik quanstrom <quanstro@labs.coraid.com> once said:
> On Mon Dec  2 10:01:48 EST 2013, lucio@proxima.alt.za wrote:
> > > is the threat standing?  that is, if the plan 9 port is broken again
> > > when 1.5 rolls around in just a few more months, does the plan 9
> > > port get booted then, too?
> >
> > The threat is real: Plan 9 is a burden for the developers and lack of
>
> perhaps we're seperated by a common language.  "standing" in the sense
> that for each new release does the same condition hold: if it does not
> pass all the tests, it is evicted from the main line.

Yes. The relevant statement from the new policy is:

    If a builder fails for more than four weeks or is failing at the time
    of a release freeze, and a new maintainer cannot be found, the port
    will be removed from the tree.

    Due to backwards compatibility concerns, removing a formerly working
    port should be a last resort. Finding a new maintainer is always
    preferable.

As of now, both Aram and I are willing to be active maintainers.
(He's just finishing up the Solaris port).

> > The solution is not with "open source" but with rolling up one's
> > sleeves and figuring out how to converge as much as possible the
>
> given that most of the time is spent bailing water out of the boat,
> and fixing things that weren't previously broken, it seems to me that
> like anything it's a two-way street.
>
> anyway, what's the argument for not just forking?

I think forking would actually *increase* the maintenance burden
for us, assuming we want to keep up with point releases.

We would still have to track the changes from upstream and we'd
no longer even be on the Go team's radar. This would naturally
result in more Unix-based assumptions that we'd have to deal with.

For example, I've seen many examples of code that calls certain
functions in the syscall package and just assumes that, e.g.,
Getrusage is defined everywhere. This is even present in the
new Go Performance Dashboard code (written by the Go team).

  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 4245 bytes --]

At a high level it is about excess baggage that needs to be carried to
provide a familiar environment for either language.  Python needs Posix, so
we have to carry its APE baggage, whereas Go brings its own Plan9-ish
baggage (?[cl], lib9, libbio, etc.) and you'll need to give it some hooks
to hang them on.

To the best of my knowledge, until now the following are the changes to
Plan 9 and/or Go to support Go on Plan 9. Do these seem onerous?

1. Addition of TSEMACQUIRE system call.  Does it have other applications?
2. Use of Go's libbio, rather than Plan 9's. Alternatively libbio on Plan 9
can be changed. It's not clear to me why this would be a bad thing.
3. Modifications to 8g to get around an 8c bug (for the curious, see the
test case from Anthony that reproduces it, below). Go team accepted the
patch to get around this, even though it is a bug in Plan 9's 8c.
4. modification to get around having to use runtime·memmove in
runtime·sighandler
(for the curious, this is because runtime·memmove uses SSE MOVOU
instructions, which touch XMM registers).

* Repro for Plan 9 8c bug:
unsigned long long x;

int f(int);

void
test(void)
{
int a;
 a = f(a-x+a);
}



On Mon, Dec 2, 2013 at 1:51 PM, erik quanstrom <quanstro@labs.coraid.com>wrote:

> On Mon Dec  2 16:08:20 EST 2013, skip.tavakkolian@gmail.com wrote:
> > wait! so, you had to make changes to Plan 9 to support Python? :)
> >
> > (sorry, couldn't resist)
>
> i'll take the bait.
>
> the changes were very different from the changes for go.  python
> is an ape program, and ape has been very neglected.  many
> of these changes were not strictly necessary for the python port.
> none are really python specific.  they fix broken things in ape.
> go needs new system calls, and makes other demands, like the
> silly libbio macros.
>
> here's a highlevel overview of what was done.  stars by the necessary
> changes:
>
> *0.  support for ssl was added to ape by importing mp, sec, bio into
> ape.  also compile crypt from /sys/src/libc/port (necessary to avoid using
> openssl)
>
> *1.  <inttypes.h> was moved to /$objtype/include/ape because it's not
> portable.  (necessary for amd64 port)
>
> *2.  inet_ntop, inet_pton, getaddrinfo (and friends) were added.
> python doesn't work well with old-style name lookup.
>
> *3.  all the system calls have had their signatures corrected e.g.:
>         - extern        int     _RENDEZVOUS(unsigned long, unsigned long);
>         + extern        void*   _RENDEZVOUS(void*, void*);
> this is required for amd64.  change brk() and _buf() accordingly.
>
> *4.  fix frexp, modf, (endian issues, subnormal #s) and copysign().
>
> *5.  make a lame attempt to deal with wait4() vs rfork.
>
> *6.  _IO_getc needs to return EOF on eof.
>
> *7.  fix %p  for 64-bit add vfscanf in stdio; use USED() not #pragma ref
>
> *8.  getsrvbyaddr() remember that htons!
>
> *9.  further envp rework.
>
> *10.  socketpair: wrong signature.
>
> and unnecessary but helpful fixes:
>
> 0.  all ape compoents are built with -VTw to prevent simple linker
> mistakes.  snprintf() now always uses the c99 definition to prevent link
> errors with -T.
>
> *1.  support for ssl was added to ape by importing mp, sec, bio into
> ape.  (necessary to avoid using openssl)
>
> *2.  <inttypes.h> was moved to /$objtype/include/ape because it's not
> portable.  (necessary for amd64 port)
>
> 3. ip6 support was added.
>
> *4.  inet_ntop, inet_pton, getaddrinfo (and friends) were added.
> python doesn't work well with old-style name lookup.
>
> 5.  librexexp was fixed in line with fixes to the normal lib.
>
> 6.  Rune and wchar_t are an uint/int for 21-bit runes.  decode 21-bit
> runes in mbwc.c
>
> 7.  getcallerpc/getfcr implemented for all arches.
>
> 8. make qsort 64-bit safe.
>
> 9.  fix const in strcspn, strpbrk, strrchr, strspn, strstr, rename,
> pathconf
>
> 10.  fix types in setuid, setgid, signal, mkfifo, getgrent, getpwent
>
> 11.  fix conflict between bind() [libdraw] and bind [socket].
>
> 12.  fmt: fix %p; fix %C (21-bit rune)
>
> 13.  utf: 21-bit runes
>
> 14.  libv: fix impossible definition of nap().
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 6303 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bakul Shah]

On Mon, 02 Dec 2013 16:03:54 EST erik quanstrom <quanstro@labs.coraid.com> wrote:
> > I am suggesting breaking out just the diffs and new mkfiles in
> > a separate tree so that one can do
> >
> >     mk all && mk install
> >
> > This can fetch the necessary bits, apply patches, build, test,
> > create downloadable binaries (with crypto signatures if you
> > care) etc.  As part of this it can mk any dependencies.  One
> > of which can pull in changes for ape. Right now all this seems
> > rather manual. As more changes are merged back in the upstream
> > sources, port diffs can reduce.
>
> patches are the plan 9 standard way of doing
> things.  they've been submitted, and can be
> applied by anyone.

Right but clearly this is not working well. Anyway, I'll try
to find time to whip up a prototype of what I am talking
about.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Steve Simon]

Whne I looked at Go - maybe 2 years ago, I could not see why
plan9 would not adopt go's 8c/8l and libbio.

obviously others disagree as they have not been back-ported,
but why not? - I'am not trolling, genuine question.

For me, at the time, go's ability to produce (limited) windows 32bit
executables seemed a huge plus - I use windows less these days but it
would still be handy on ocasion.

-Steve

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> > patches are the plan 9 standard way of doing
> > things.  they've been submitted, and can be
> > applied by anyone.
>
> Right but clearly this is not working well. Anyway, I'll try
> to find time to whip up a prototype of what I am talking
> about.

does applying patches oneself not work?  what is lacking in that
approach?

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> At a high level it is about excess baggage that needs to be carried to
> provide a familiar environment for either language.  Python needs Posix, so
> we have to carry its APE baggage, whereas Go brings its own Plan9-ish
> baggage (?[cl], lib9, libbio, etc.) and you'll need to give it some hooks
> to hang them on.

python's requirements are a proper subset of go's, since building go requires
python be installed.

> 2. Use of Go's libbio, rather than Plan 9's. Alternatively libbio on Plan 9
> can be changed. It's not clear to me why this would be a bad thing.

i object to the macros they have been adding.  they haven't been included
in p9p, either.  adding B(put|get)(le|be)(Biobufhdr *b, int width) might
be an interesting convienence, but what they've got for the sake of 1% on
micro benchmarks seems to run counter to the plan 9 culture to me.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

Steve Simon <steve@quintile.net> once said:
> Whne I looked at Go - maybe 2 years ago, I could not see why
> plan9 would not adopt go's 8c/8l and libbio.
>
> obviously others disagree as they have not been back-ported,
> but why not? - I'am not trolling, genuine question.

We couldn't adopt the Go ?[acl] toolchain wholesale
because of incompatible Go-specific changes; e.g.,
disallowing variadic C functions without a #pragma
annotation, a different symtab/pclntab format, etc.

It would take a fair bit of work to make everything
(including libmach) backwards-compatible.

  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

erik quanstrom <quanstro@quanstro.net> once said:
> > 2. Use of Go's libbio, rather than Plan 9's. Alternatively libbio on Plan 9
> > can be changed. It's not clear to me why this would be a bad thing.
>
> i object to the macros they have been adding.  they haven't been included
> in p9p, either.  adding B(put|get)(le|be)(Biobufhdr *b, int width) might
> be an interesting convienence, but what they've got for the sake of 1% on
> micro benchmarks seems to run counter to the plan 9 culture to me.

The claim was a bit stronger than that:

    https://code.google.com/p/go/source/detail?r=1ee3ab13e3b6

but I still agree with you.

  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Aram Hăvărneanu]

On Tue, Dec 3, 2013 at 1:12 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> python's requirements are a proper subset of go's, since building go requires
> python be installed.

No, it doesn't. Using mercurial to clone the tree requires python,
building Go does not require python, nor mercurial.

-- 
Aram Hăvărneanu

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 19:51:11 EST 2013, aram.h@mgk.ro wrote:
> On Tue, Dec 3, 2013 at 1:12 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> > python's requirements are a proper subset of go's, since building go requires
> > python be installed.
>
> No, it doesn't. Using mercurial to clone the tree requires python,
> building Go does not require python, nor mercurial.

have you tried in the last couple of months?  some changes were made
that do some repo checking.  building requires hg now.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Anthony Martin]

erik quanstrom <quanstro@quanstro.net> once said:
> On Mon Dec  2 19:51:11 EST 2013, aram.h@mgk.ro wrote:
> > On Tue, Dec 3, 2013 at 1:12 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> > > python's requirements are a proper subset of go's, since building go requires
> > > python be installed.
> >
> > No, it doesn't. Using mercurial to clone the tree requires python,
> > building Go does not require python, nor mercurial.
>
> have you tried in the last couple of months?  some changes were made
> that do some repo checking.  building requires hg now.

The standard way to get around this requirement is by
placing a VERSION file in the top-level directory.

It can contain any string but it's usually something like:

    devel +7326da92ff4d Mon Dec 02 09:06:41 2013 +1100

Cheers,
  Anthony

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeremy Jackins]

On Mon, Dec 2, 2013 at 5:52 PM, erik quanstrom <quanstro@quanstro.net> wrote:
> have you tried in the last couple of months?  some changes were made
> that do some repo checking.  building requires hg now.

If you put a VERSION file in place manually, it will skip the hg
identify you're talking about.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeff Sickel]

RE: VERSION file

Even if you do stuff a file in to trick the build process into not
searching for hg, you’re still only a partial participant as you
can’t use the codereview.py scripts to synchronize your source and
submit upstream changes.

I’d really appreciate a Go implementation of git/hg somewhere down
the line.

And for the libbio changes, I’m more opposed to the four functions
Bgetle2, Bgetle4, Bputle2, and Bputle4 and the odd use of the
BPUTC && BPUTLE4 macros.  Those implementations are found in other
portions of the code, not specific to libbio, and seem to be
grafted on in a slightly off manner.  That said, I’ll gladly
change my mind if the p9p, Inferno, and a few other forks of the
libbio source give it their blessing.  To date, none have chimed
in to contribute to the conversation in any way so I’m more than
happy to use a patch in Go to still use Plan 9’s libbio but pull
in the four new functions and the macros and go from there.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

On Mon Dec  2 19:16:55 EST 2013, ality@pbrane.org wrote:
> Steve Simon <steve@quintile.net> once said:
> > Whne I looked at Go - maybe 2 years ago, I could not see why
> > plan9 would not adopt go's 8c/8l and libbio.
> >
> > obviously others disagree as they have not been back-ported,
> > but why not? - I'am not trolling, genuine question.
>
> We couldn't adopt the Go ?[acl] toolchain wholesale
> because of incompatible Go-specific changes; e.g.,
> disallowing variadic C functions without a #pragma
> annotation, a different symtab/pclntab format, etc.
>
> It would take a fair bit of work to make everything
> (including libmach) backwards-compatible.

i think one could avoid the backwards-compatibity.  simple
change the formats and recompile the system.  the other
problems in principle are superficial.

the real question that needs adressing is what is plan 9,
and what makes it useful.

for me, i think having a self-contained system that takes over
as early as it can in the boot process with all the source in
/sys and all the source part of the system, and not subject to
forces outside plan 9, is valuable.  the code is amazingly stable,
and it's not too hard for one person to have a pretty good grasp of
the system.

this is super powerful.  instead of putting mental energy in boxing
up hard-to-understand and perhaps not essential bits of the system
so one doesn't have to think about how they work, one can know
how those bits work and put their quirks to good use.  or even
change them.

instead of building another layer on top (to avoid knowing about
lower layers), relatively speaking, plan 9 makes it trivial to get in
and do what needs to be done.

it may be that replacing the compilers with something externally
controlled is not that much of a set back and all told one gains
more than one loses.

even if that is the case, i'm not sure i'd be okay with the ensuing
churn.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> all this is moot unless we can get plan 9 integrated into go's automatic
> build system.  is this doable?  i have resources to make this happen if
> it is.

Let's take note of this, because 9atom certainly deserves attention in
this situation.  I'm trying to wrap my mind around the Go builder and
it would seem to me that even though the platforms are different, we
could categorise builders under the plan 9 label.  If that's the case,
then at least we can get a bird's eye view of progress if enough
contributors take part.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> i thought the language had become more ossified.  as time goes on
> this is less and less of a concern.  i see more performance changes than
> anything these days.

This is intentional when a release is being prepared.  Real changes
will take place once 1.2 (now) is released.  Also, keep in mind, as
Skip points out, that the Go ecosystem is much greater than the
language and is itself good cause to adopt the language.  In my
opinion, very, very good cause.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> Why do you hold this opinion?  While your defensiveness is hilarious, it's a
> simple matter of curiosity.  I'm trying to understand the fervor
> behind another
> language-cum-fashion-accessory.  Specifically:  how will Go enable good things
> that, say, python has not?  Which good things?  Are all of them web based?
>
Stop being sarcastic and you may stop seeing defensiveness and hilariousness, too.

As for the Go-vs-Python issue, (a) Python was
latest-"language-cum-fashion-accessory" itself not too long ago and
(b) enough has been written about it elsewhere by more competent
people than myself; I'm sure you can refer to that literature rather
than expect me to botch a description here.

> I'm not asking this idly.  I have resources available for projects like this,
> but I'm tired of wasting them on projects that go nowhere once the developers'
> attention span shifts to whatever hip new thing is promoted by $COMPANY.

I guess you would not like me to read "for COMPANY=Google shift span"
in the above, but I did and others probably too.  The challenge is out
there, $COMPANY does not want to allocate funding to the Plan 9 port
of Go and Plan 9 and Dragonfly BSD have been found wanting with regard
to the support needed by the developers to get their work done.  That
has caused the entry bar to be raised and I believe that is fair, if
inconvenient.

If you can contribute resources to the project, we may all be
grateful, but if they come with strings attached such as the need to
tolerate your sarcasm, I suspect some of us may well prefer to refuse
them.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> Please explain this assertion (e.g. TSEMACQUIRE contradicts it).

This is not intended to offend Bell Labs or anyone at Bell Labs, it
just exposes the difference in focus that may cost us valuable support
from Google: the MIPS port of Plan 9.  Personally, I found it quite
exciting, but from a Go perspective it was insignificant.

Bell Labs are perfectly entitled to choose their projects and I would
never had raised this matter had I not been challenged to.  What I'm
trying to convey is a need for Bell Labs to make it easier for the Go
contributors to make progress with the few issues that have arisen
(5c's vlong switch labels, libbio, floating point instructions in
syscalls, complex expressions in 8c, etc.), but I don't really know
how they should do that.  Abandoning MIPS development obviously isn't
one such option.  maybe taking Go more seriously or at least
communicating regularly with the Go developers community might well
be.

Again, no offence intended, I'm just pacing out the territory.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[andrey mirtchovski]

> For example, I've seen many examples of code that calls certain
> functions in the syscall package and just assumes that, e.g.,
> Getrusage is defined everywhere. This is even present in the
> new Go Performance Dashboard code (written by the Go team).

not anymore. the getrusage code was split to _unix.go just now.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> i'm committed
> to supporting go in 9atom to the extent that it does not compromise
> or corrupt the system substantially.

I can't assist with the details of kernel requirements, but we all
know that Russ, Rob and Ken will know better and be somewhat committed
so that at least Plan 9 release is not going to be "compromised" by
anything Go is going to need.  You're also overlooking the fact that
OSX, the *BSDs and even some Windows flavours are actively supported
(and Solaris is threatening) so if anything compromises will be
towards generality not specialisation (in other words, no one will try
to turn Plan 9 into Linux).

But (a) your concerns are reasonable and need to be kept sight of and
(b) your idea of listing the necessary changes to Plan 9 dovetails
very much with my idea on how to move forwards from here.  I do wish
we could get Bakul Shah and Anthony Martin to document what they are
working on; and Gorka to document or at least describe to me, so I can
try to document what his focus was when he was still working on the
plan9/arm port.

All of the above would be immensely helpful, in my opinion.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> Another thing that can be done is to maintain a separate
> "ports" repository, where you keep patches required to build
> but that are not yet integrated in any upstream repo + any
> other bits to a s/w package in a platform specific manner.
> This is how FreeBSD ports work. Everyone shouldn't have to
> track down patches and try merging them individually.

If the Go builders system wasn't already in place, that would have
great merit, but as things stand, short of providing a better system
for the Go developers to adopt (a dream, but not a very practical one
- I'm not in love with Mercurial, Python or any other immense GNU-ish
package), we may as well encourage everyone to use what is known to
work adequately.

If you feel we can do better and you are prepared to defend your case,
this (or the go-plan9 google group) will be a reasonable platforms to
present it.  I, for one, will be willing to hear about any such
option.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> Some coordination among people who are 9fan-gonut, 9fan-godev or 9dev would
> be helpful; at least it will reduce unnecessary head-scratching.

Is there one each of these?!  And now go-plan9?!

Do mad people ever scratch their heads?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> python on plan9 can't even handle the codereview extension.

I'll defend Kurt on this one: nor can Go, yet :-)

But I don't know Python, nor care to know it and I happen to like both
Go and Plan 9.  To take your side, Skip, Python is very much a
GNU-ism, like a lot of other products that target Linux almost
exclusively.  Go can break that pattern.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> 1. Addition of TSEMACQUIRE system call.  Does it have other applications?

I missed this; is there any documentation that clarifies this issue?
I think it's important to provide a rationale for structural changes
and Bell Labs are a little too glib with their patch releases, in my
opinion.

> 2. Use of Go's libbio, rather than Plan 9's. Alternatively libbio on Plan 9
> can be changed. It's not clear to me why this would be a bad thing.

The functions are very readily added to libbio without side effects.
The macros, on the other hand, are an embarrassment and trigger dozens
of warnings (to the best of my ability to check).  There really ought
to be no warnings in compiling the Go tool chain for Plan 9; the
bison-related stuff deserves some attention, too.

> 3. Modifications to 8g to get around an 8c bug (for the curious, see the
> test case from Anthony that reproduces it, below). Go team accepted the
> patch to get around this, even though it is a bug in Plan 9's 8c.

The patch was a bit of a scream.  I'm the first to admit that 8c needs
a touch of TLC and that an abort() in the middle of a compiler,
without the slightest attempt to deal with the problem is at least as
embarrassing as the expansion of BGETLE, but the original code that
tripped the compiler was also extremely shoddy.  And the Go
developers' resistance to rewrite a few badly thought out lines of
code did come back to bite them in the ass.

> 4. modification to get around having to use runtime·memmove in
> runtime·sighandler

The right answer here is to have a plan9·memmove until Bell Labs
figure a way to classify some opcodes as other than floating point and
allow them where the architecture permits it.  It's a big problem, but
it needs to be addressed, the world of CPU extensions is still young
and, sadly, not still-born.

The Go developers will eventually need to adjust to this reality too.
Arm64 is a hint of things to come.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 2274 bytes --]

BTW, another big item that I forgot to mention, which ironically has been
the Subject of this thread, is the support for 21bit runes.



On Mon, Dec 2, 2013 at 11:10 PM, <lucio@proxima.alt.za> wrote:

> > 1. Addition of TSEMACQUIRE system call.  Does it have other applications?
>
> I missed this; is there any documentation that clarifies this issue?
> I think it's important to provide a rationale for structural changes
> and Bell Labs are a little too glib with their patch releases, in my
> opinion.
>
> > 2. Use of Go's libbio, rather than Plan 9's. Alternatively libbio on
> Plan 9
> > can be changed. It's not clear to me why this would be a bad thing.
>
> The functions are very readily added to libbio without side effects.
> The macros, on the other hand, are an embarrassment and trigger dozens
> of warnings (to the best of my ability to check).  There really ought
> to be no warnings in compiling the Go tool chain for Plan 9; the
> bison-related stuff deserves some attention, too.
>
> > 3. Modifications to 8g to get around an 8c bug (for the curious, see the
> > test case from Anthony that reproduces it, below). Go team accepted the
> > patch to get around this, even though it is a bug in Plan 9's 8c.
>
> The patch was a bit of a scream.  I'm the first to admit that 8c needs
> a touch of TLC and that an abort() in the middle of a compiler,
> without the slightest attempt to deal with the problem is at least as
> embarrassing as the expansion of BGETLE, but the original code that
> tripped the compiler was also extremely shoddy.  And the Go
> developers' resistance to rewrite a few badly thought out lines of
> code did come back to bite them in the ass.
>
> > 4. modification to get around having to use runtime·memmove in
> > runtime·sighandler
>
> The right answer here is to have a plan9·memmove until Bell Labs
> figure a way to classify some opcodes as other than floating point and
> allow them where the architecture permits it.  It's a big problem, but
> it needs to be addressed, the world of CPU extensions is still young
> and, sadly, not still-born.
>
> The Go developers will eventually need to adjust to this reality too.
> Arm64 is a hint of things to come.
>
> ++L
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2882 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> python's requirements are a proper subset of go's, since building go requires
> python be installed.

Not at all, I have no Python installed on any of my Plan 9 equipment
and can build Go and most of its ecology on each of them (there are
other issues, of course).  What I can't do is "develop" the tool
chain, for that I use Python/Mercurial/Codereview on a NetBSD host
and, where sometimes git is required, I use my Ubuntu workstation.
It's a mess, of course, but I manage to maintain some sort of
discipline despite it all.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> i object to the macros they have been adding.  they haven't been included
> in p9p, either.  adding B(put|get)(le|be)(Biobufhdr *b, int width) might
> be an interesting convienence, but what they've got for the sake of 1% on
> micro benchmarks seems to run counter to the plan 9 culture to me.

I am totally with you on this one.  But the macros are in <bio.h>, so
we may be able to avoid them.  They are obscene and trigger many
annoying warnings.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> On Tue, Dec 3, 2013 at 1:12 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>> python's requirements are a proper subset of go's, since building go requires
>> python be installed.
>
> No, it doesn't. Using mercurial to clone the tree requires python,
> building Go does not require python, nor mercurial.
>
Actually, how do you resolve the VERSION file problem?  Go will not
build from a perfectly clean hg clone without a working "hg".

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> BTW, another big item that I forgot to mention, which ironically has been
> the Subject of this thread, is the support for 21bit runes.

That's water under the mill, is it not?

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> And for the libbio changes, I’m more opposed to the four functions
> Bgetle2, Bgetle4, Bputle2, and Bputle4 and the odd use of the
> BPUTC && BPUTLE4 macros.  Those implementations are found in other
> portions of the code, not specific to libbio, and seem to be
> grafted on in a slightly off manner.  That said, I’ll gladly
> change my mind if the p9p, Inferno, and a few other forks of the
> libbio source give it their blessing.  To date, none have chimed
> in to contribute to the conversation in any way so I’m more than
> happy to use a patch in Go to still use Plan 9’s libbio but pull
> in the four new functions and the macros and go from there.

I'd like to hear more from you about the (mis)use of Bgetle* and
friends (not the macros, we all agree on these).  I think we ought to
prepare a submission to Bell Labs for a reasoned inclusion of these
functions in the library (and the corresponding B*be ones, which for
some reason Go does not need (?!)).  As for the macros, I know Russ
was never in favour of using them, we may have a case to have them
rescinded :-) I suspect that they would all have been dropped, if Russ
had known where to look for them.  If memory serves, he reverted a CL,
but not all pertinent CLs, possibly on request from (members of) the
Plan 9 faction.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

> Stop being sarcastic and you may stop seeing defensiveness and
> hilariousness, too.

Please identify the sarcasm.

> As for the Go-vs-Python issue, (a) Python was
> latest-"language-cum-fashion-accessory" itself not too long ago and
> (b) enough has been written about it elsewhere by more competent
> people than myself; I'm sure you can refer to that literature rather
> than expect me to botch a description here.

(a) that's why I bring it up and (b) no, it hasn't.  I have not been able
to discern why people are excited about Go; that is why I asked the
question.  As far as I can tell, Go is just a de-facto web programming
language, in direct lineage from perl through python.  If that's all the
"there" there, then I'll just ignore it instead.  If there's something
more to be said for it, I would like to know.

Right now I get a bunch of hand-waving from everyone I ask, while some
people mutter about having a lot of libraries available -- as though
anything could compete with CPAN... but we don't see anyone helping get
perl a leg up.  So there has to be something capturing peoples interest.
I'm just trying to ascertain what.

> I guess you would not like me to read "for COMPANY=Google shift span"
> in the above, but I did and others probably too.  The challenge is out
> there, $COMPANY does not want to allocate funding to the Plan 9 port
> of Go and Plan 9 and Dragonfly BSD have been found wanting with regard
> to the support needed by the developers to get their work done.  That
> has caused the entry bar to be raised and I believe that is fair, if
> inconvenient.

Google is not the first company to ride some fashion show into a bunch
of free work from programmers; they won't be the last.  I've seen similar
flurries of wild-eyed excitement about software from Sun and Nokia and
many others.  They all petered out into silence.  What makes this one
different?  This is an actual serious question; someone please answer
it.

The reason I typed '$COMPANY' instead of 'Google' was to indicate that
my statements were more generalized and not directly or uniquely about
Google.  I cannot believe that I have to explain this basic convention.

> If you can contribute resources to the project, we may all be
> grateful, but if they come with strings attached such as the need to
> tolerate your sarcasm, I suspect some of us may well prefer to refuse
> them.

Again, would rectify the sarcasm if there were any.  Instead I asked a
series of simple questions and got shit on for my interest.  You might
want to relax a bit.

khm

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeff Sickel]

On Dec 3, 2013, at 1:31 AM, lucio@proxima.alt.za wrote:
> 
> I am totally with you on this one.  But the macros are in <bio.h>, so
> we may be able to avoid them.  They are obscene and trigger many
> annoying warnings.

See https://codereview.appspot.com/15750047 as it gets around the
issue w/o using the current go supplied <bio.h> but the Plan 9
system supplied bio.h (+ a little mingling in of the additions to
$GOROOT/include/plan9).

I’m still not completely satisfied, but once I figure out cmd/dist
internals a little more then the file I’ve currently dropped in
$GOROOT/src/lib9/bio_plan9.c can be moved to $GOROOT/src/libbio
and just compiled storing the $O in $GOROOT/pkg/obj/plan9_$objtype
and linked in addition to Plan 9’s /$objtype/lib/libbio.a.

I use:
	cmd/8g: work around 64-bit bug in 8c for Plan 9
	https://codereview.appspot.com/14521056

	runtime: do not use memmove in the Plan 9 signal handler
	https://codereview.appspot.com/34640045

	libbio mod (not full replacement, just funcs and macros)
	https://codereview.appspot.com/15750047	libbio

to build go for 386 off a standard sources release with my ape
patch and an install of python 2.7.5 + hg.  There are still issues
in trying to get amd64 working that require changes to cmd/dist
source and some of the pkg/runtime.  I’m close, but need to
eliminate some X11 distractions before really finishing it up.

There is one little bit I’d also like to figure out how to
work around: the install location of go & gofmt: the go_bootstrap
puts those binaries into $GOROOT/bin even though the actual
$objtype targets get installed into $GOROOT/pkg/*$objtype.  It’s
a small mess, but still one that makes multi-architecture
file servers a little awkward (no nearly as awkward as Python’s
attempt at cross-compilation).

-jas

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> I have not been able
> to discern why people are excited about Go; that is why I asked the
> question.  As far as I can tell, Go is just a de-facto web programming
> language, in direct lineage from perl through python.  If that's all the
> "there" there, then I'll just ignore it instead.  If there's something
> more to be said for it, I would like to know.

Almost certainly, Google's involvement is a significant part of the
equation, anyone who denies this is probably delusional (or different
from me, which really makes no difference from my point of view).

The primary objective stated by Rob Pike and in my opinion already
largely achieved in the development of Go was to provide a compiled
language with static typing that could compete in speed of deployment
with the many interpreter where dynamic typing ruled.  It needed to
compile faster than C++ and Java and be comparable in execution speed.

In this respect, Go has been mostly a success, although there are
performance issues that have not yet been sorted out.  The most
important of these is the garbage collector which is undergoing
considerable redesign right now.  The curren model was built out of
necessity without sufficient data points to optimise it; it included a
"stop-the-world" misfeature that has yet to be removed, but is in the
process of being designed out.

Additional objectives have also been achieved: multitasking
primitives, portability across diverse platforms, high readability,
conceptual simplicity, similarity to conventional languages and even
the ability to link to modules written in C and C++, inclusion in the
GNU language package, many more I can't possibly recall.

There is plenty of documentation out there.  If I were to write a
comprehensive dissertation condensing this information into a single
document, I find it difficult to believe that you would trouble
yourself to read it any more than you have read the contents of the
golang.org site.

If you consider such documentation hand-waving, then I doubt you're
interested in hearing more of the same.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> 	runtime: do not use memmove in the Plan 9 signal handler
> 	https://codereview.appspot.com/34640045

I've lost track, but would it make sense to apply go386=387 instead of
that patch (I'm pretty sure that is what I did for at least a while)
in all Plan 9 builds?  I'm thinking that we ought to stay close to the
release wherever possible and in this case Plan 9 needs to come up
with a solution that Go can propagate across present and future
platforms.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> I’m still not completely satisfied, but once I figure out cmd/dist
> internals a little more then the file I’ve currently dropped in
> $GOROOT/src/lib9/bio_plan9.c can be moved to $GOROOT/src/libbio
> and just compiled storing the $O in $GOROOT/pkg/obj/plan9_$objtype
> and linked in addition to Plan 9’s /$objtype/lib/libbio.a.

I saw your CL, but I think your description is a bit too concise, I
can't quite get my mind around it.  Has anyone consulted with Bell
Labs and tried to convince them to add Bgetle* and Bputle* in libbio,
yet?  I have a feeling that a clean patch (I'm sure I have one handy)
will be accepted if we request it with some urgency?

++L

PS: Why the hell is there no Bgetbe* and Bputbe*?

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

On 3 December 2013 08:14, Jeff Sickel <jas@corpus-callosum.com> wrote:

> See https://codereview.appspot.com/15750047 as it gets around the
> issue w/o using the current go supplied <bio.h> but the Plan 9
>

go has its own bio, used by its self-contained source tree.
why can't it just use the one it expects?
i think you explained it once, but i've reverted to being mystified.

[-- Attachment #2: Type: text/html, Size: 874 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Richard Miller]

> * Repro for Plan 9 8c bug:
> unsigned long long x;
>
> int f(int);
>
> void
> test(void)
> {
> int a;
>  a = f(a-x+a);
> }

Is anybody working on fixing this?  If not, I'll volunteer.

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> go has its own bio, used by its self-contained source tree.
> why can't it just use the one it expects?
> i think you explained it once, but i've reverted to being mystified.

Does Go, the tool chain itself, need libbio?  If not, then it is there
only for building Go on platforms other than Plan 9 and the Go libbio
should be a subset of the Plan 9 one.  Breaking this concept seems
unnecessary, not unforgivable: Go does this with libmach, but there
the details are much more complex and a merge of libmachs between Go
and Plan 9 would in any case be an asset.

As an interesting issue, we could have avoided introducing 21-bit
runes in Plan 9 by extracting a libutf from libc for use in Plan 9,
but we would have gained some time and nothing else.  The same, more
or less, applies to libbio, in my opinion.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

>> * Repro for Plan 9 8c bug:
>> unsigned long long x;
>>
>> int f(int);
>>
>> void
>> test(void)
>> {
>> int a;
>>  a = f(a-x+a);
>> }
>
> Is anybody working on fixing this?  If not, I'll volunteer.

I haven't heard any offers, I think you're first.  I seem to remember
Charles suggesting that it is difficult to do.  And as it caught a
rather embarrassing case of sloppy programming, it is tempting to
retain its behaviour, somehow.  But that wouldn't advance the cause of
Plan 9 any, unfortunately.

And that lone

	abort();

against the left margin, is positively ugly.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On 3 December 2013 10:08, <lucio@proxima.alt.za> wrote:

> I seem to remember
> Charles suggesting that it is difficult to do.  And as it caught a
> rather embarrassing case of sloppy programming
>

It's not sloppy programming, but an underlying assumption about RISC
machines in the (then) future
having a respectable number of (truly) general-purpose registers, and the
x86
does not. The S-U numbers guide register allocation, but there is no
spilling code,
except for nested function calls, and a few special cases that don't use
the S-U numbers.
Given that the out of fixed registers diagnostic occurs rarely in the
normal source,
and the S-U numbers might or might not be strictly accurate, the potential
disadvantage
of introducing inefficiency in existing compiled code by spill code
misguided by inaccurate
estimates made it seem not worthwhile at the time.

[-- Attachment #2: Type: text/html, Size: 1434 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> On 3 December 2013 10:08, <lucio@proxima.alt.za> wrote:
>
>> I seem to remember
>> Charles suggesting that it is difficult to do.  And as it caught a
>> rather embarrassing case of sloppy programming
>>
>
> It's not sloppy programming, but an underlying assumption about RISC
> machines in the (then) future
> having a respectable number of (truly) general-purpose registers, and the
> x86
> does not.

Oy! That's not what I was suggesting.  The sloppy programming was in
$GOROOT/src/cmd/8g/ggen.c where it triggered a compiler failure:
multiplication was used in an expression as argument to a function
invoked inside a loop.

In the compiler, the abort() invocation could be called sloppy, but
not in the same sense :-)

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Aram Hăvărneanu]

On Tue, Dec 3, 2013 at 11:04 AM,  <lucio@proxima.alt.za> wrote:
> a merge of libmachs between Go
> and Plan 9 would in any case be an asset.

No, because the Go one changes all the time. E.g., now go puts half
the linker into the compilers; object files will contain real machine
code.

These details change all the time in the Go world.

-- 
Aram Hăvărneanu

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> Does Go, the tool chain itself, need libbio?  If not, then it is there

yes.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> I saw your CL, but I think your description is a bit too concise, I
> can't quite get my mind around it.  Has anyone consulted with Bell
> Labs and tried to convince them to add Bgetle* and Bputle* in libbio,
> yet?  I have a feeling that a clean patch (I'm sure I have one handy)

lucho, as explained before we think this is a *BAD* idea.  macros
are generally avoided.  especially long complicated ones like those.

> PS: Why the hell is there no Bgetbe* and Bputbe*?

because it's just a micro speed hack for the compiler.  i don't think
they're thinking of maintaining libbio as a system library.  there are
reasons not to use this as libbio.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

why can't it just use the one it expects?
why does it have to be the same as the one in /sys/src?


On 3 December 2013 14:42, erik quanstrom <quanstro@quanstro.net> wrote:

> > Does Go, the tool chain itself, need libbio?  If not, then it is there
>
> yes.
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 859 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> BTW, another big item that I forgot to mention, which ironically has been
> the Subject of this thread, is the support for 21bit runes.

that was the trigger for getting them into sources,
but unicode specifies a 21-bit rune.  we were wrong.
and both 9atom and p9p had 21-bit runes several years
before they were added to sources.

by the way, the sources version of sort is broken wrt 21-bit runes.
i believe the 9atom version was correct.  somehow the sources
version got messed up.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> other issues, of course).  What I can't do is "develop" the tool
> chain, for that I use Python/Mercurial/Codereview on a NetBSD host

9atom + jas' python will solve that issue.  and still you have
a netbsd+python dependency.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jeff Sickel]

Because then you’ve got to use additional ifdefs per patch
https://codereview.appspot.com/14604047/.  Sorry Anthony, I
died a little seeing

#if !defined(_WIN32) && !defined(PLAN9)

The cmd/dist/build.c changes are fine as I’m messing around
with similar edge cases.  And of course getting [569]c to use the
ANSI C preprocessor in order to consume the $GOROOT/include/bio.h
is necessary.

In any case, both patches 14604047 and 15750047 fix the Go libbio
addition problem when building on Plan 9.  Neither have made any
progress in getting rolled up into the Go sources.  I’m proposing
modifying my patch, 15750047, to extract the four funcs from
bgetc.c and bputc.c and put them into a blec.c so that cmd/dist
will build just blec.c on Plan 9 and install it into
$GOROOT/pkg/obj/plan9_$objtype/blec.$O for linking.

On Dec 3, 2013, at 8:51 AM, Charles Forsyth <charles.forsyth@gmail.com> wrote:

> why can't it just use the one it expects?
> why does it have to be the same as the one in /sys/src?
> 
> 
> On 3 December 2013 14:42, erik quanstrom <quanstro@quanstro.net> wrote:
> > Does Go, the tool chain itself, need libbio?  If not, then it is there
> 
> yes.
> 
> - erik
> 
> 

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> why can't it just use the one it expects?
> why does it have to be the same as the one in /sys/src?

To keep the Go distribution honest?  Eventually, we'd want as much
convergence as possible, forking the library would make it easier to
diverge without consequences.

Of course, it doesn't really matter because convergence is unlikely,
but I would say that the immediate gratification of taking the
convenience route does not compare with the rewards of sticking to
good principles.  In this case, I think the price to be paid is pretty
minimal.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On 3 December 2013 16:04, <lucio@proxima.alt.za> wrote:

> To keep the Go distribution honest?  Eventually, we'd want as much
> convergence as possible, forking the library would make it easier to
> diverge without consequences.
>

but it's not a question of forking the library. there's a ton of stuff
under go/src,
so what makes libbio special? why not just compile the one there for its
 use, which is the one it expects?
the output goes into a go-specific target directory; what else will care?

[-- Attachment #2: Type: text/html, Size: 919 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

i agree; i was also surprised when i realized this.


On Tue, Dec 3, 2013 at 8:47 AM, Charles Forsyth
<charles.forsyth@gmail.com>wrote:

>
> On 3 December 2013 16:04, <lucio@proxima.alt.za> wrote:
>
>> To keep the Go distribution honest?  Eventually, we'd want as much
>> convergence as possible, forking the library would make it easier to
>> diverge without consequences.
>>
>
> but it's not a question of forking the library. there's a ton of stuff
> under go/src,
> so what makes libbio special? why not just compile the one there for its
>  use, which is the one it expects?
> the output goes into a go-specific target directory; what else will care?
>

[-- Attachment #2: Type: text/html, Size: 1376 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[john francis lee]

[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]

On 12/03/2013 11:47 PM, Charles Forsyth wrote:
>
> On 3 December 2013 16:04, <lucio@proxima.alt.za
> <mailto:lucio@proxima.alt.za>> wrote:
>
>     To keep the Go distribution honest?  Eventually, we'd want as much
>     convergence as possible, forking the library would make it easier to
>     diverge without consequences.
>
>
> but it's not a question of forking the library. there's a ton of stuff
> under go/src,
> so what makes libbio special? why not just compile the one there for
> its  use, which is the one it expects?
> the output goes into a go-specific target directory; what else will care?
google and the nsa will care ?

--
--
Hi there, NSA 'analysts', in-house and/or contracted.

Just reminding you that if you are reading this you are committing a crime, that you are felons mocking the 4th Amendment of our US Constitution ...

   "The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."

... and that someday, really soon I hope, you're going to have to pay for your crimes.

You're breaking international laws as well, so if you're thinking of the 'I was only following orders!' defense ... Please see Nuremberg Principle IV ...

   "The fact that a person acted pursuant to order of his Government or of a superior does not relieve him from responsibility under international law, provided a moral choice was in fact possible to him".

... and start exercising your moral choice. Look upon Thomas Andrews Drake and Edward Snowden as your exemplars and Patron Saints.


[-- Attachment #2: Type: text/html, Size: 2880 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[sl]

> if you are reading this you are committing a crime

This is a public mailing list.

sl

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> but it's not a question of forking the library. there's a ton of stuff
> under go/src,
> so what makes libbio special?

I'm not sure where the wires get crossed, let's see if I can get my
point across or, alternatively, if I can figure out what I'm missing.

In building the Go tool chain, in Plan 9, only libmach, of the
libararies required, differs substantially from the Go release version
to require special functionality.  The differences aren't
irreconcilable, but they are pretty vast.

For a more Posix-y environment, lib9 and libbio are also required to
provide features that Plan 9 has natively.  Lib9 mirrors libc and
libbio is analogous to "the real thing".  My contention is that we
ought to keep these differences to an absolute minimum and,
specifically, we ought to avoid libbio diverging from the Plan 9
native version - at the cost of altering the latter, if necessary - so
that the two systems are kept closer together.

The point here is that once we grant licence for libbio to diverge,
there is no limit to how far it will go and any efforts to bring
either in line with the other will be as difficult as it would be now
for libmach.  Libmach and lib9 both show how far apart this divergence
will go if encouraged.

It is moot, I grant that, but as a responsible citizen, I feel it is
my duty not to create conditions than future generations can't recover
from, at least as long as there is an option.  Yeah, it's a
philosophical line in the sand, but you did ask where I was drawing it
and I'm hoping I have made it a little clearer (even to myself).

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> I'm not sure where the wires get crossed, let's see if I can get my
> point across or, alternatively, if I can figure out what I'm missing.

i don't think any wires are crossed.

> The point here is that once we grant licence for libbio to diverge,
> there is no limit to how far it will go [...]
>
> It is moot, I grant that, but as a responsible citizen, I feel it is
> my duty not to create conditions than future generations can't recover
> from, at least as long as there is an option.  Yeah, it's a
> philosophical line in the sand, but you did ask where I was drawing it
> and I'm hoping I have made it a little clearer (even to myself).

what would we recover from?  divergence?  go never left the building
as it wasn't in the building to begin with.  i think this is likely what
you may be missing.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Jens Staal]

On Wednesday 04 December 2013 06:25:33 lucio@proxima.alt.za wrote:

> For a more Posix-y environment, lib9 and libbio are also required to
> provide features that Plan 9 has natively.  Lib9 mirrors libc and
> libbio is analogous to "the real thing".  My contention is that we
> ought to keep these differences to an absolute minimum and,
> specifically, we ought to avoid libbio diverging from the Plan 9
> native version - at the cost of altering the latter, if necessary - so
> that the two systems are kept closer together.
>

Sorry for hijacking - but out of interest, has anyone approached an
implementation of SIGCHLD either in the "go - plan9 reconciliation effort" or
in any of the "fix APE" efforts (if I remember correctly, it would be needed for
an APE pthreads implementation)?

My personal interest in this is that this is the only thing left to have
MirBSD Ksh fully functional (potentially an up-to-date and actively maintained
sh for APE instead of the current pdksh).

--

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> Sorry for hijacking - but out of interest, has anyone approached an
> implementation of SIGCHLD either in the "go - plan9 reconciliation effort" or
> in any of the "fix APE" efforts (if I remember correctly, it would be needed for
> an APE pthreads implementation)?
>
> My personal interest in this is that this is the only thing left to have
> MirBSD Ksh fully functional (potentially an up-to-date and actively maintained
> sh for APE instead of the current pdksh).

i haven't done anything about SIGCHLD.  in your case, i think a special little
thread to call wait4 and kill(parent, SIGCHILD) would do.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> what would we recover from?  divergence?  go never left the building
> as it wasn't in the building to begin with.  i think this is likely what
> you may be missing.

Are you suggesting that any efforts to keep Go and Plan 9 in sync
should be measured purely against short term gain?  To me, that makes
Plan 9 a superfluous platform for Go: the cost of maintaining the
port(s) would never be recoverable in deployment of Go applications to
any Plan 9 platform.

On the other hand, if Go and Plan 9 continue to influence each other's
development and philosophy, I think both will benefit, as well as
their respective communities.  The discussion here, superficial as it
is, is a case in point.

++L

[9fans] Go port [was Re: Go and 21-bit runes (and a bit of Go status)]

[Jeff Sickel]

The Go ‘builder’ for Plan 9 386 is on the chopping block:

https://groups.google.com/forum/#!topic/golang-dev/QW4zUbMHMBM
https://code.google.com/p/go-wiki/wiki/PortingPolicy


Lucio, there is no symbiotic relationship between Plan 9 and Go.
Go is its own language, and maybe eventually platform, that does
share some history with Plan 9 and various other spin offs.  But
though it started with a few people who happened to have worked
on Plan 9 in the past to hone their ideas, the minimal stubs
used in Go that are similar to core Plan 9 libraries are just that,
mechanisms to bootstrap the language.  Taking libc and libbio and
the initial C toolchain is a great way to start.

I do think they’ve done a very nice job at getting the language
off the ground on several platforms.  And it’s great to see the
Go talks and references back to CSP.  But at the end of the day,
Go is a very different research platform that doesn’t really
map onto what draws many of us to Plan 9.  It’s a great tool to
have in the shop, but like any combo-tool, sometimes you just
have to put it on the bench so you can really use the kW CO2
laser without being distracted.



On Dec 4, 2013, at 12:19 AM, lucio@proxima.alt.za wrote:

>> what would we recover from?  divergence?  go never left the building
>> as it wasn't in the building to begin with.  i think this is likely what
>> you may be missing.
> 
> Are you suggesting that any efforts to keep Go and Plan 9 in sync
> should be measured purely against short term gain?  To me, that makes
> Plan 9 a superfluous platform for Go: the cost of maintaining the
> port(s) would never be recoverable in deployment of Go applications to
> any Plan 9 platform.
> 
> On the other hand, if Go and Plan 9 continue to influence each other's
> development and philosophy, I think both will benefit, as well as
> their respective communities.  The discussion here, superficial as it
> is, is a case in point.
> 
> ++L
> 
> 
> 
> 

Re: [9fans] Go port [was Re: Go and 21-bit runes (and a bit of Go

[lucio]

> The Go ‘builder’ for Plan 9 386 is on the chopping block:
> 
> https://groups.google.com/forum/#!topic/golang-dev/QW4zUbMHMBM
> https://code.google.com/p/go-wiki/wiki/PortingPolicy

Maybe I am misunderstanding, or maybe our perspectives are
irreconcilably different.  As I see it, if we cannot comply with the
need of the Go developers to have sustained maintenance of the Plan 9
port of Go so that they need not provide this maintenance themselves,
then the Plan 9 port will no longer be supported, meaning that the Go
developers are under no obligation to correct any errors that may
affect (exclusively) the port to Plan 9 that may be caused by
development in their other territory.

It also means that the Plan 9 port will be decoupled from the
codereview system and in effect will be left to bitrot wherever it
will be maintained when that happens.

I consider that a disaster, others may think otherwise.

++L

Re: [9fans] Go port [was Re: Go and 21-bit runes (and a bit of Go

[Jeff Sickel]

Russ did release the Porting Policy which gives us a good benchmark.
That said, getting the changes people have submitted to codereview
rolled into the default branch so the builder will work is another
issue all together.

On Dec 4, 2013, at 1:20 AM, lucio@proxima.alt.za wrote:

> It also means that the Plan 9 port will be decoupled from the
> codereview system and in effect will be left to bitrot wherever it
> will be maintained when that happens.
> 
> I consider that a disaster, others may think otherwise.

Codereview has it’s issues.  For me, it’s the inability to actual
construct a query that works to return any CLs related to Plan 9,
or anything for that matter.  Could be that I’m running a non-supported
browser when I’m trying to pull up details.  And I thought this version
of Chrome was the most recent…

Re: [9fans] Go port [was Re: Go and 21-bit runes (and a bit of Go

[lucio]

> Codereview has it’s issues.

It's normal to find fault when things aren't as convenient as one
wishes, but in this case I doubt that we could replace codereview and,
most importantly, the reviewers, with something better.

If there is a better option, I'll be happy to go along with it, I'm
not an intentional reactionary.  But I also don't want to lose sight
of the fact that to me its "Go release" on multiple "Plan 9 platforms"
that I find appealing.  There are far more competent people out there
to deal with all the other options - although netbsd/386 is also of
active interest to me - and I'd rather help where I am able to
contribute what little skills I have.

As for highlighting the Plan 9 issues specifically, I had a brief
conversation, I think it was with Andrew Gerrand, where the conclusion
was that we - the Plan 9 community - were most welcome to use the
issue tracker for Plan 9 issues and it was more or less up to us to
make sure that the issues were labelled correctly.  I'm not familiar
with the issue tracker's innards, but I can investigate this further
if we think it would be the right path to follow.  I do hope it is.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[lucio]

> one does not attempt in a compiler.

I was talking about outputting an explanatory message.  Your point is
valid, but not pertinent here.

++L

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> One does not simply compile.

certainly not.  one simply compiles.

- erik

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[Bence Fábián]

[-- Attachment #1: Type: text/plain, Size: 767 bytes --]

Compile or not compile, there is no try.
One does not simply compile.


2013/12/3 erik quanstrom <quanstro@quanstro.net>

> > The patch was a bit of a scream.  I'm the first to admit that 8c needs
> > a touch of TLC and that an abort() in the middle of a compiler,
> > without the slightest attempt to deal with the problem is at least as
> > embarrassing as the expansion of BGETLE, but the original code that
> > tripped the compiler was also extremely shoddy.  And the Go
>
> one does not attempt in a compiler.  one does correctly or one does
> not do.  "attempting" is the path to madness.  it's been a while since
> i've debugged something down to the asm and found the compiler had
> got it wrong.  we must keep it this way!
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 1195 bytes --]

Re: [9fans] Go and 21-bit runes (and a bit of Go status)

[erik quanstrom]

> The patch was a bit of a scream.  I'm the first to admit that 8c needs
> a touch of TLC and that an abort() in the middle of a compiler,
> without the slightest attempt to deal with the problem is at least as
> embarrassing as the expansion of BGETLE, but the original code that
> tripped the compiler was also extremely shoddy.  And the Go

one does not attempt in a compiler.  one does correctly or one does
not do.  "attempting" is the path to madness.  it's been a while since
i've debugged something down to the asm and found the compiler had
got it wrong.  we must keep it this way!

- erik