[9fans] syscall 53

[9fans] syscall 53

[goo]

p.s. Caps lock is not working. Also copying in 9fat directory shifts file time current time+3 hours, even +6 hours if renaming (mv). My timezone is +3 GMT. Its native plan9 on ibm t42.

Re: [9fans] syscall 53

[Jeff Sickel]

A lot more than Caps lock is not working….
This has been some of the worst 36+ hours I’ve had w/ Plan 9,
and no end in site to get everything running again.

I sure hope the NSEC change is worth it in the long run.
I did want to write some Go code yesterday, but this rebuild
has eaten up most of my hours.

Might be time to see if the old ThinkPad will boot… need
something with a trusty ol’ floppy to get a fresh install going.


On May 17, 2014, at 2:11 PM, goo <puta2001-goo@yahoo.com> wrote:

> 
> p.s. Caps lock is not working. Also copying in 9fat directory shifts file time current time+3 hours, even +6 hours if renaming (mv). My timezone is +3 GMT. Its native plan9 on ibm t42. 

Re: [9fans] syscall 53

[Jeff Sickel]

s/site/sight/

Might not be so bad… one day, but it does mean that handling the
stand-alone file server needs to have a few more controls in
place to prevent a pull.

-jas

On May 17, 2014, at 4:17 PM, Jeff Sickel <jas@corpus-callosum.com> wrote:

> A lot more than Caps lock is not working….
> This has been some of the worst 36+ hours I’ve had w/ Plan 9,
> and no end in site to get everything running again.
> 
> I sure hope the NSEC change is worth it in the long run.
> I did want to write some Go code yesterday, but this rebuild
> has eaten up most of my hours.
> 
> Might be time to see if the old ThinkPad will boot… need
> something with a trusty ol’ floppy to get a fresh install going.
> 
> 
> On May 17, 2014, at 2:11 PM, goo <puta2001-goo@yahoo.com> wrote:
> 
>> 
>> p.s. Caps lock is not working. Also copying in 9fat directory shifts file time current time+3 hours, even +6 hours if renaming (mv). My timezone is +3 GMT. Its native plan9 on ibm t42. 
> 
> 
> 

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

thanks for the heads-up. i'll wait pulling from sources for now.



On Sat, May 17, 2014 at 2:17 PM, Jeff Sickel <jas@corpus-callosum.com>wrote:

> A lot more than Caps lock is not working….
> This has been some of the worst 36+ hours I’ve had w/ Plan 9,
> and no end in site to get everything running again.
>
> I sure hope the NSEC change is worth it in the long run.
> I did want to write some Go code yesterday, but this rebuild
> has eaten up most of my hours.
>
> Might be time to see if the old ThinkPad will boot… need
> something with a trusty ol’ floppy to get a fresh install going.
>
>
> On May 17, 2014, at 2:11 PM, goo <puta2001-goo@yahoo.com> wrote:
>
> >
> > p.s. Caps lock is not working. Also copying in 9fat directory shifts
> file time current time+3 hours, even +6 hours if renaming (mv). My timezone
> is +3 GMT. Its native plan9 on ibm t42.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1317 bytes --]

Re: [9fans] syscall 53

[lucio]

> thanks for the heads-up. i'll wait pulling from sources for now.

I guess we need one of two alternatives at this point: either somebody
documents how to recover from a pull by first reconstructing (or
binding from the dump) the critical commands or we are given some
indication by Bell Labs so that we can wait to do a pull until sources
is consistent and pulling becomes safe again.

Erik kindly offered a recovery procedure, but my impression is that it
compounds effort for only a temporary gain; waiting for Bell Labs, on
the other hand, has the drawback that custom kernels will not be taken
care of automatically.

I seem to remember that there was one header change offered (Erik,
again?) that would help with this last issue, would that allow safe
building of custom kernels without first pulling?

++L

Re: [9fans] syscall 53

[David du Colombier]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

The safe way to upgrade your file server is simply to update the kernel
binaries (for example, with the ones I'm providing) on your 9fat and reboot.

Then, you can pull the updated program binaries from /n/sources.

There is no need to wait, because you have to be running the new kernel
before pulling the new binaires anyway.

In the case you need a custom kernel to boot your file server, you will
have to grab the changes manually (like
http://9fans.eu/pegasus/sources/plan9/plan9-2014-05-15.diff), and
recompile/reinstall/reboot your kernel.

--
David du Colombier

[-- Attachment #2: Type: text/html, Size: 748 bytes --]

Re: [9fans] syscall 53

[Jeff Sickel]

note to self:  add /dev/sdE?/9fat to vac
streamline getting 9LOAD in place w/o corruption

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

i got the impression that sources were in some inconsistent state.

if the only change is the new system call, isn't it sufficient to:

* pull only /sys/src/9
* build the kernels you need and install on boot medium
* reboot
* pull again and rebuild all the binaries

since existing binaries would work on the new kernel?



On Sun, May 18, 2014 at 1:29 AM, David du Colombier <0intro@gmail.com>wrote:

> The safe way to upgrade your file server is simply to update the kernel
> binaries (for example, with the ones I'm providing) on your 9fat and reboot.
>
> Then, you can pull the updated program binaries from /n/sources.
>
> There is no need to wait, because you have to be running the new kernel
> before pulling the new binaires anyway.
>
> In the case you need a custom kernel to boot your file server, you will
> have to grab the changes manually (like
> http://9fans.eu/pegasus/sources/plan9/plan9-2014-05-15.diff), and
> recompile/reinstall/reboot your kernel.
>
> --
> David du Colombier
>

[-- Attachment #2: Type: text/html, Size: 1652 bytes --]

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]

0intro's last paragraph says the same thing.


On Sun, May 18, 2014 at 8:32 AM, Skip Tavakkolian <
skip.tavakkolian@gmail.com> wrote:

> i got the impression that sources were in some inconsistent state.
>
> if the only change is the new system call, isn't it sufficient to:
>
> * pull only /sys/src/9
> * build the kernels you need and install on boot medium
> * reboot
> * pull again and rebuild all the binaries
>
> since existing binaries would work on the new kernel?
>
>
>
> On Sun, May 18, 2014 at 1:29 AM, David du Colombier <0intro@gmail.com>wrote:
>
>> The safe way to upgrade your file server is simply to update the kernel
>> binaries (for example, with the ones I'm providing) on your 9fat and reboot.
>>
>> Then, you can pull the updated program binaries from /n/sources.
>>
>> There is no need to wait, because you have to be running the new kernel
>> before pulling the new binaires anyway.
>>
>> In the case you need a custom kernel to boot your file server, you will
>> have to grab the changes manually (like
>> http://9fans.eu/pegasus/sources/plan9/plan9-2014-05-15.diff), and
>> recompile/reinstall/reboot your kernel.
>>
>> --
>> David du Colombier
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2139 bytes --]

Re: [9fans] syscall 53

[Jeff Sickel]

> * pull only /sys/src/9

 * pull -s sys/src/libc

> * build the kernels you need and install on boot medium
> * reboot


I recovered by using 9fs snap so I could get an earlier state
of /386.  9fs dump triggered the syscall error.

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

rebuilding/installing the binaries from local sources takes very little
time and guarantees that pull will not overwrite them (unless forced).



On Sun, May 18, 2014 at 8:38 AM, Jeff Sickel <jas@corpus-callosum.com>wrote:

>
> > * pull only /sys/src/9
>
>  * pull -s sys/src/libc
>
> > * build the kernels you need and install on boot medium
> > * reboot
>
>
> I recovered by using 9fs snap so I could get an earlier state
> of /386.  9fs dump triggered the syscall error.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 892 bytes --]

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

fyi, pulling/merging (e.g. adding IL back), building the kernels, booting
and building the binaries works as expected for all cpu types in my
environment (pc, bcm, rb and kw).



On Sun, May 18, 2014 at 9:03 AM, Skip Tavakkolian <
skip.tavakkolian@gmail.com> wrote:

> rebuilding/installing the binaries from local sources takes very little
> time and guarantees that pull will not overwrite them (unless forced).
>
>
>
> On Sun, May 18, 2014 at 8:38 AM, Jeff Sickel <jas@corpus-callosum.com>wrote:
>
>>
>> > * pull only /sys/src/9
>>
>>  * pull -s sys/src/libc
>>
>> > * build the kernels you need and install on boot medium
>> > * reboot
>>
>>
>> I recovered by using 9fs snap so I could get an earlier state
>> of /386.  9fs dump triggered the syscall error.
>>
>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 1527 bytes --]

Re: [9fans] syscall 53

[erik quanstrom]

On Sun May 18 18:56:49 EDT 2014, skip.tavakkolian@gmail.com wrote:

> fyi, pulling/merging (e.g. adding IL back), building the kernels, booting
> and building the binaries works as expected for all cpu types in my
> environment (pc, bcm, rb and kw).

i'd put a vote into restoring il to the standard kernels.  there's no
downside.

- erik

Re: [9fans] syscall 53

[Jeff Sickel]

On May 18, 2014, at 8:54 PM, erik quanstrom <quanstro@quanstro.net> wrote:

> On Sun May 18 18:56:49 EDT 2014, skip.tavakkolian@gmail.com wrote:
> 
>> fyi, pulling/merging (e.g. adding IL back), building the kernels, booting
>> and building the binaries works as expected for all cpu types in my
>> environment (pc, bcm, rb and kw).
> 
> i'd put a vote into restoring il to the standard kernels.  there's no
> downside.


I vote getting ether82563.c updated.  I’m merging, but not done yet.

The big fail was my auth server sitting on a Soekris net6501.  Great
machine, on 9atom.  Plan 9 faults if you try to boot it w/o *nomp=1.
And when you do boot it w/ *nomp=1, Plan 9 doesn’t recognize the mSATA
device in it, which kind of ruins the whole idea of using it as an
auth server.

-jas

Re: [9fans] syscall 53

[lucio]

> Great
> machine, on 9atom.

Which raises another question: are 9atom and 9front in synch with the
BL distribution (itself in question) regarding syscall 53?

++L

Re: [9fans] syscall 53

[erik quanstrom]

> Which raises another question: are 9atom and 9front in synch with the
> BL distribution (itself in question) regarding syscall 53?

9atom is not.  i didn't know that it was added, nor do i 
know why nsec was added as a syscall.

i indirectly heard "go needs it", but that is not really a reason
i can understand technically.  why must it be a system call?

getting ahead of myself, if the problem is shared memory vs
shared fds, then the solution is easy: fix nsec in the c library.
don't save a copy of the fd.  that leads to trouble.  (the new
call takes ~6µs on my e3 v2)

if the problem is getting very low-latency timing, or relative
timing, then the solution is still easy: use the timestamp counter.
no version of nsec works for relative timing due to timesync
adjustments!

i'm sure there are other possibilities, i don't think i see them without
an explination.  so if anyone has anything else, that would be
interesting.

- erik

---
; cat /sys/src/libc/9sys/nsec.c
#include <u.h>
#include <libc.h>

vlong
nsec(void)
{
	uchar b[8];
	int fd;

	fd = open("/dev/bintime", OREAD);
	if(fd != -1)
	if(pread(fd, b, sizeof b, 0) == sizeof b){
		close(fd);
		return getbe(b, sizeof b);
	}
	close(fd);
	return 0;
}

Re: [9fans] syscall 53

[lucio]

> i indirectly heard "go needs it", but that is not really a reason
> i can understand technically.  why must it be a system call?

Actually, Go raised an important alert, quite indirectly: when using
high resolution timers, the issue of opening a device, reading it and
converting the input value to a binary value can and in this case is
very expensive.

Curiously, the actual symptom - I cannot remember how it came about -
was that using the timer leaked file descriptors, or, more likely,
gave the impression of leaking file descriptors.  But the reality is
that nanosecond accuracy cannot be achieved from reading a device by
conventional means.

++L

Re: [9fans] syscall 53

[erik quanstrom]

On Mon May 19 10:04:28 EDT 2014, lucio@proxima.alt.za wrote:
> > i indirectly heard "go needs it", but that is not really a reason
> > i can understand technically.  why must it be a system call?
> 
> Actually, Go raised an important alert, quite indirectly: when using
> high resolution timers, the issue of opening a device, reading it and
> converting the input value to a binary value can and in this case is
> very expensive.
>
> Curiously, the actual symptom - I cannot remember how it came about -
> was that using the timer leaked file descriptors, or, more likely,
> gave the impression of leaking file descriptors.  But the reality is
> that nanosecond accuracy cannot be achieved from reading a device by
> conventional means.

i think my original question still stands.  what is the purpose of timing,
what is the desired accuracy and precision, and is a relative or absolute
time wanted?  

a relative time (say a time adjusted with timesync, including leap seconds, etc)
is not what you want if you want relative timing.  something like the
timestamp counter makes a lot more sense.

i took a quick look at the runtime·nanotime, and it looks like it's being
used for gettimeofday, which shouldn't be super performance sensitive.

- erik

Re: [9fans] syscall 53

[erik quanstrom]

On Mon May 19 12:26:00 EDT 2014, quanstro@quanstro.net wrote:
> On Mon May 19 10:04:28 EDT 2014, lucio@proxima.alt.za wrote:
> > > i indirectly heard "go needs it", but that is not really a reason
> > > i can understand technically.  why must it be a system call?
> >
> > Actually, Go raised an important alert, quite indirectly: when using
> > high resolution timers, the issue of opening a device, reading it and
> > converting the input value to a binary value can and in this case is
> > very expensive.
> >
> > Curiously, the actual symptom - I cannot remember how it came about -
> > was that using the timer leaked file descriptors, or, more likely,
> > gave the impression of leaking file descriptors.  But the reality is
> > that nanosecond accuracy cannot be achieved from reading a device by
> > conventional means.
>
> i think my original question still stands.  what is the purpose of timing,
> what is the desired accuracy and precision, and is a relative or absolute
> time wanted?

also, one cannot get close to 1ns precision with a system call.  a system call
takes a bare minimum of 400-500ns on 386/amd64.

- erik

Re: [9fans] syscall 53

[lucio]

> also, one cannot get close to 1ns precision with a system call.  a system call
> takes a bare minimum of 400-500ns on 386/amd64.

Sure, but accessing /dev/time is, if I guess right, orders of
magnitude slower, specially if you have to open the device first.

As far as I know, that was the only choice, to start with.

++L

Re: [9fans] syscall 53

[erik quanstrom]

On Mon May 19 13:17:59 EDT 2014, lucio@proxima.alt.za wrote:
> > also, one cannot get close to 1ns precision with a system call.  a system call
> > takes a bare minimum of 400-500ns on 386/amd64.
> 
> Sure, but accessing /dev/time is, if I guess right, orders of
> magnitude slower, specially if you have to open the device first.

the full operation open/read/close/convert takes 6µs on my machine,
and a system call takes about 750ns.

getting the kitchen time should not need better than 6µs.  if this
were relative timing, i would understand, but it's not.

- erik

Re: [9fans] syscall 53

[lucio]

> i took a quick look at the runtime·nanotime, and it looks like it's being
> used for gettimeofday, which shouldn't be super performance sensitive.

I'm on thin ice here, but I seem to remember that the crucial issue
was the resolution (nanosecond) and the expectation that Plan 9 would
have to match (for portability purposes) the quality available on
other operating system.

Curiously, I'm pretty certain that it was the issue of an fd that
remained open (something to do with caching the /dev/time fd, if I
remember right) that caused some tests to fall apart, probably because
a test for leaking fds actually needed to cache the time of day for
time out purposes.

Two birds with one stone?  On the one hand you gain accuracy and on
the other you actually successfully complete the tests.

++L

Re: [9fans] syscall 53

[erik quanstrom]

> > i took a quick look at the runtime·nanotime, and it looks like it's being
> > used for gettimeofday, which shouldn't be super performance sensitive.
> 
> I'm on thin ice here, but I seem to remember that the crucial issue
> was the resolution (nanosecond) and the expectation that Plan 9 would
> have to match (for portability purposes) the quality available on
> other operating system.

the resolution of the original was already 1ns, the precision and
accuracy were somewhat less, as they would be on any os/hw combination
i am aware of.

> Curiously, I'm pretty certain that it was the issue of an fd that
> remained open (something to do with caching the /dev/time fd, if I
> remember right) that caused some tests to fall apart, probably because
> a test for leaking fds actually needed to cache the time of day for
> time out purposes.

well close the fd then.  it's not exceptionally expensive.

> Two birds with one stone?  On the one hand you gain accuracy and on
> the other you actually successfully complete the tests.

i would be very surprised if there were any gain in accuracy.  the
accuracy is going to be dominated by the most inaccurate term, and
that's likely going to be timesync, and on the order of milliseconds.

- erik

Re: [9fans] syscall 53

[Bakul Shah]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

On Mon, 19 May 2014 13:25:42 EDT erik quanstrom <quanstro@quanstro.net> wrote:
>
> i would be very surprised if there were any gain in accuracy.  the
> accuracy is going to be dominated by the most inaccurate term, and
> that's likely going to be timesync, and on the order of milliseconds.

Speaking of time and accuracy....

I am adding some logic to synchronize with the PPS signal from
the GPS device that I hooked up to a RaspberryPi.  With this
change the TOD clock should be accurate to within 10 to 20 µs.
So I for one welcome the new syscall! [Though its introduction
could've been better managed]

But using a TOD clock for measuring performance seems wrong
since it will also have to account for leapseconds (at the
moment timesync happily ignores leapseconds).

Re: [9fans] syscall 53

[erik quanstrom]

> I am adding some logic to synchronize with the PPS signal from
> the GPS device that I hooked up to a RaspberryPi.  With this
> change the TOD clock should be accurate to within 10 to 20 µs.
> So I for one welcome the new syscall! [Though its introduction
> could've been better managed]

even a syscall on a rpi is going to cost you at least 5-10µs
and clock drift will make this, and your second point very hard.

> But using a TOD clock for measuring performance seems wrong
> since it will also have to account for leapseconds (at the
> moment timesync happily ignores leapseconds).

- erik

Re: [9fans] syscall 53

[ron minnich]

I've been watching the discussion but was hesitant to jump in. But
somebody asked me to say a thing or two.

We put the nsec() system call into NxM because, any way you cut it, it
provides better accuracy than the open/read/close path, particularly
when there's lots of stuff running, and the apps we care about need
that improved precision. Yes, this is measured. Just saying 'use tsc'
ignores the lack of portability of that mechanism, as well as the
other issues that have been found over the years with time registers
(core tsc synchronization, effects from power management, interaction
with virtualization, and so on). You don't have to look very long to
see that the work the kernel does for the three-system-call version is
far higher than the simple nsec system call -- yep, looks simple in
the library, and, nope, lots more going on in the kernel, more than 3x
as much. There are a few uses that break the 'it should look like a
file' model and I think this is one of them. Getting accurate,
jitter-free time is essential to making good decisions in many cases.

Nevertheless, I had not intended to push on the system call idea very
hard. As long as people were happy, there did not seem to be much
reason. So I stopped worrying about it over a year ago.

A recent discussion on golang-dev provided the final impetus: it was
recommended that if the nsec call were available on Plan 9, the Go
ports should use it. That was enough for me to put in a request for
one more review of the patch (which patch I did not put in, BTW), and
the good folks at BL felt the idea was OK, so in it went.

So it's in. It's an architecture neutral kernel interface get time in
nanoseconds in a which-core-are-you-on independent manner. It can be
made faster: on NxM the plan was to put a very fast path in the system
call assembly and return the nsec in %rax, and do the memmove in user
mode, so as to avoid any validaddr or address alignment overhead. A
true system call allows optimizations that the open/read/close
interface can not.

I'm sorry to hear that it caused trouble. That said, the problems were
due (IMHO) to a limitation in the update mechanism, not to the
inclusion of a new system call. The limitation is rarely encountered
because the situation is rarely encountered. The right way to update
the system when the kernel adds a new system call is pretty clear:
kernel first. Updating binaries before kernels is clearly wrong and
shows something could be done better. I think it's a good time to
review how the update path works and fix it.

So, it's good for Go and anything else which wants reasonable accuracy
on time without having to muck with architecture-dependent registers.
It's trivial to add. We found it helped a lot on NxM. The quality of
the results you get are hard if not impossible to equal with the
current interface. The old path still works. I think we're all going
to be ok.

ron

Re: [9fans] syscall 53

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

On 19 May 2014 21:54, ron minnich <rminnich@gmail.com> wrote:

> jitter-free time


Jitter says something about (in)consistency of time periods or intervals.
It will be a function of scheduling decisions, not the overhead of a single
call.
In Nix, on an AP core, sure, because there isn't any scheduling. When there
is scheduling of any sort,
it's the scheduling that results in the jitter, not either of these two
time implementations, surely.
You could clearly say "faster", but I'm not convinced about "jitter-free".
For instance, you can still be pre-empted for variable
time between the invocation of "nsec", its arrival in the kernel, and on
return from the kernel before and after return from "nsec".
Preventing that is a scheduling matter, with both implementations.

[-- Attachment #2: Type: text/html, Size: 1274 bytes --]

Re: [9fans] syscall 53

[ron minnich]

On Mon, May 19, 2014 at 2:30 PM, Charles Forsyth
<charles.forsyth@gmail.com> wrote:

> Jitter says something about (in)consistency of time periods or intervals. It
> will be a function of scheduling decisions, not the overhead of a single
> call.
> In Nix, on an AP core, sure, because there isn't any scheduling. When there
> is scheduling of any sort,
> it's the scheduling that results in the jitter, not either of these two time
> implementations, surely.
> You could clearly say "faster", but I'm not convinced about "jitter-free".
> For instance, you can still be pre-empted for variable
> time between the invocation of "nsec", its arrival in the kernel, and on
> return from the kernel before and after return from "nsec".
> Preventing that is a scheduling matter, with both implementations.

I did not want to make this too long, but, sure, everything you say is
quite correct. It's why we considered taking the implementation even
further on NxM, but we stopped that project before we got to that
point. There are way more opportunities for the introduction of jitter
with the old system as opposed to the new, because there are far more
places that you might get to where scheduling can occur. In other
words, again, you're right as rain in what you're saying.

What can I say? We measured it on NxM, were disappointed with the
results from the old way, and happy(er) with the new way.

And, again, I was not inclined to act on any of this until the
discussion on the golang-dev list, which boiled down to:
"if you can do it with a single system call, you should" with a
response of "we put in a patch, was not accepted yet.". I just renewed
the request to reexamine the patch.

ron

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting ron minnich <rminnich@gmail.com>:

> And, again, I was not inclined to act on any of this until the
> discussion on the golang-dev list, which boiled down to:
> "if you can do it with a single system call, you should" with a
> response of "we put in a patch, was not accepted yet.". I just renewed
> the request to reexamine the patch.

Does anyone know how we can get golang-dev to endorse all of the
ethernet drivers that the labs refuses to merge?  I can send them
some ad money or some web software or whatever motivates them

khm

Re: [9fans] syscall 53

[andrey mirtchovski]

golang-dev has more clout than 9fans nowadays, at least as it pertains to plan9.

On Mon, May 19, 2014 at 5:02 PM, Kurt H Maier <khm@sciops.net> wrote:
> Quoting ron minnich <rminnich@gmail.com>:
>
>> And, again, I was not inclined to act on any of this until the
>> discussion on the golang-dev list, which boiled down to:
>> "if you can do it with a single system call, you should" with a
>> response of "we put in a patch, was not accepted yet.". I just renewed
>> the request to reexamine the patch.
>
>
> Does anyone know how we can get golang-dev to endorse all of the
> ethernet drivers that the labs refuses to merge?  I can send them
> some ad money or some web software or whatever motivates them
>
> khm
>
>

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting andrey mirtchovski <mirtchovski@gmail.com>:

> golang-dev has more clout than 9fans nowadays, at least as it
> pertains to plan9.
>

That's why I'm asking.  We now have three go-related new syscalls, while
lots of actual hardware support gets flushed down the toilet for
unspecified reasons.  I'm not complaining; I'm just wondering how to
adapt to the new system.  Sending code to the labs directly doesn't work;
fine.  How do we lobby golang-dev?

khm

Re: [9fans] syscall 53

[andrey mirtchovski]

I suggest personal notes, flowers, and some hard liquor.

On Mon, May 19, 2014 at 5:12 PM, Kurt H Maier <khm@sciops.net> wrote:
> Quoting andrey mirtchovski <mirtchovski@gmail.com>:
>
>> golang-dev has more clout than 9fans nowadays, at least as it pertains to
>> plan9.
>>
>
> That's why I'm asking.  We now have three go-related new syscalls, while
> lots of actual hardware support gets flushed down the toilet for
> unspecified reasons.  I'm not complaining; I'm just wondering how to
> adapt to the new system.  Sending code to the labs directly doesn't work;
> fine.  How do we lobby golang-dev?
>
> khm
>
>

Re: [9fans] syscall 53

[Jeff Sickel]

On May 19, 2014, at 6:17 PM, andrey mirtchovski <mirtchovski@gmail.com> wrote:

> I suggest personal notes, flowers, and some hard liquor.
> 

/me could use all three after a weekend of sysadmin frustration.

All hope is not lost… this exercise, by not being able to use
my usual cpu servers, gave me time to cut over to a new fossil
raid (the old disk needed to go anyway) while doing an unanticipated
code review.

-jas

Re: [9fans] syscall 53

[Anthony Sorace]

[-- Attachment #1: Type: text/plain, Size: 909 bytes --]

Ron wrote:

> That said, the problems were due (IMHO) to a limitation in the
> update mechanism, not to the inclusion of a new system call.

This is true depending on how you define "update mechanism".
A simple note from whoever made the decision to push the
change out to the effect of "hey, we're going to add a new
syscall, update your kernels before pulling new binaries" a
while before the push would have been sufficient.

> I think it's a good time to review how the update path works
> and fix it.

Again, I agree, provided that definition's broad enough to
include the communication channel. We've gotten notices of
potentially disruptive changes here in the past, and that's been
fine (even if the disruption is inconvenient for some). Without
that sort of communication, doing actual pulls from sources
(regardless of the technical mechanism used) seems dangerous.

Anthony


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 169 bytes --]

Re: [9fans] syscall 53

[erik quanstrom]

> > That said, the problems were due (IMHO) to a limitation in the
> > update mechanism, not to the inclusion of a new system call.
>
> This is true depending on how you define "update mechanism".
> A simple note from whoever made the decision to push the
> change out to the effect of "hey, we're going to add a new
> syscall, update your kernels before pulling new binaries" a
> while before the push would have been sufficient.

technology doesn't solve human communiction problems.

here's the Official Meme.  simply s/spam/update issues/g
and you're good:

https://craphound.com/spamsolutions.txt

- erik

Re: [9fans] syscall 53

[ron minnich]

I have a different perspective. There are millions of chromebooks out
there updating all the time, from the firmware to the kernel to the
root file system to everything. It all works.

If you are telling me that the upgrade technology of Plan 9 can not
handle an automatic upgrade, fine; we have the proof.

If you are telling me Plan 9 should not or never will be able to
handle an automatic upgrade, and is going to require a heads up email
for each kernel change, I have a hard time taking that seriously.

This is not a human communication problem. It's a technology problem,
trivially solved for many years now by many systems.

ron

Re: [9fans] syscall 53

[erik quanstrom]

On Tue May 20 12:42:35 EDT 2014, rminnich@gmail.com wrote:
> I have a different perspective. There are millions of chromebooks out
> there updating all the time, from the firmware to the kernel to the
> root file system to everything. It all works.
>
> If you are telling me that the upgrade technology of Plan 9 can not
> handle an automatic upgrade, fine; we have the proof.
>
> If you are telling me Plan 9 should not or never will be able to
> handle an automatic upgrade, and is going to require a heads up email
> for each kernel change, I have a hard time taking that seriously.
>
> This is not a human communication problem. It's a technology problem,
> trivially solved for many years now by many systems.

leaving aside the fact that plan 9 is software that installs everywhere and
users can be expected to modify any and all system components, and that
android is a hardware appliance running essentially a binary blob....

if we had a perfect update mechanism that did ota updates seamlessly,
it would not address the issue i'm trying to raise.

since people modify the system, and there's a community built around
this, it would be extremely helpful if big system changes where communicated.

- erik

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting ron minnich <rminnich@gmail.com>:

> I have a different perspective. There are millions of chromebooks out
> there updating all the time, from the firmware to the kernel to the
> root file system to everything. It all works.

Millions of carefully-crafted machines updating all the time, from the
firmware
to the kernel, *as long as the user doesn't actually do anything with the
computer*.  Pushing updates to your pet web browser is a very different task
than supporting a general-purpose operating system.

I can only assume you know that, so is this irrelevant tangent deliberate
misdirection or a symptom of an honest misunderstanding of the situation?

khm

Re: [9fans] syscall 53

[ron minnich]

Ah well, back to 'm' for this thread, and I now accept that this
community is unwilling to solve this simple problem, as so many others
have.  Bummer.

ron

Re: [9fans] syscall 53

[erik quanstrom]

On Tue May 20 15:50:56 EDT 2014, rminnich@gmail.com wrote:
> Ah well, back to 'm' for this thread, and I now accept that this
> community is unwilling to solve this simple problem, as so many others
> have.  Bummer.

nobody said that.  there's a difference between noting a strawman
argument, and pointing out that one feels that there is a different
issue that's more important to solve, and being unwilling to address
an issue.

- erik

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting ron minnich <rminnich@gmail.com>:

> Ah well, back to 'm' for this thread, and I now accept that this
> community is unwilling to solve this simple problem, as so many others
> have.  Bummer.
>
> ron

Deliberate misdirection then; got it.

I'm sorry you're sad, but comparing plan freaking 9 to an operating system
backed by a megacorp with a blue-sky r&d budget is silly and in no way
productive.  Your magic update process completely ignores the question of
whether clients even *want* any given patch installed on their system,
which is the actual issue under discussion here.

khm

Re: [9fans] syscall 53

[hiro]

On 5/20/14, ron minnich <rminnich@gmail.com> wrote:
> Ah well, back to 'm' for this thread, and I now accept that this
> community is unwilling to solve this simple problem, as so many others
> have.  Bummer.
>
> ron
>
>

This is wrong.
I've already solved the problem in my local tree by accident.

Basically I've integrated a modified vc (I use an experimental 64bit
MIPS here - provided by the chinese government during my internship)
into systemd, which I ported to plan9 kernel space.
When I boot the system and a syscall is found missing, the kernel
simply recompiles itself (there is some heavy trickery involving
clusters of neural networks in a fiber network to make that work)

So even though this results from my other project as a byproduct, I'm
happy to share it.

Re: [9fans] syscall 53

[Jeff Sickel]

On May 20, 2014, at 11:41 AM, ron minnich <rminnich@gmail.com> wrote:

> This is not a human communication problem. It's a technology problem,
> trivially solved for many years now by many systems.

This event was a communication problem.

The technology, replica, works as advertised.  In general it does
a very good job at maintaining server->client updates and allows
you to work around differences in a way some may prefer over other
options that are out there.

The technology, Plan 9 (and forks), have rather clean mechanisms
already in place to go through the classic development operation
cycle of

	think — edit — make — test ...

The feedback loop, like all, requires communication protocols
to complete the iteration and move the cycle to the next step.

How many of those chromebooks have customized kernels to support
drivers undergoing new development?  And of those that do run custom
kernels, are they getting automatic updates w/o a bill of materials
or other list defining what will be updated?  And are the developers
working on those custom versions working in a vacuum or do they have
communication protocols in place to facilitate their working environment?

For all practical purposes, 9fans is the last remaining forum for people
interested in using Plan 9 related systems.  Please advise if you have
recommendations for other communications outlets that allow people to
develop, test, and maybe improve on ideas brought about by this system
many of us find enough interest in to actually use.

-jas

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 2523 bytes --]

yes, it was a lack of *any* notice; i occasionally check /n/sources/patch
for incoming patches and i don't believe the NSEC patch went through that
channel. in the past, changes that had a system-wide or kernel effect were
announce, and instructions for upgrading were given. certainly, it's not
required; but it is appreciated.

i believe any successful Plan 9 distro will need to provide some
transparency in the change review process. the sources patch process can
work, but with codereview extensions now available, it might make sense to
create/switch to a repository hosted on an hg site so that all the change
requests, discussions and reviews are available to everyone;  perhaps this
is what Charles had in mind for http://code.google.com/p/plan9, but i'm not
sure.

-Skip

On Tue, May 20, 2014 at 1:33 PM, Jeff Sickel <jas@corpus-callosum.com>wrote:

>
> On May 20, 2014, at 11:41 AM, ron minnich <rminnich@gmail.com> wrote:
>
> > This is not a human communication problem. It's a technology problem,
> > trivially solved for many years now by many systems.
>
> This event was a communication problem.
>
> The technology, replica, works as advertised.  In general it does
> a very good job at maintaining server->client updates and allows
> you to work around differences in a way some may prefer over other
> options that are out there.
>
> The technology, Plan 9 (and forks), have rather clean mechanisms
> already in place to go through the classic development operation
> cycle of
>
>         think — edit — make — test ...
>
> The feedback loop, like all, requires communication protocols
> to complete the iteration and move the cycle to the next step.
>
> How many of those chromebooks have customized kernels to support
> drivers undergoing new development?  And of those that do run custom
> kernels, are they getting automatic updates w/o a bill of materials
> or other list defining what will be updated?  And are the developers
> working on those custom versions working in a vacuum or do they have
> communication protocols in place to facilitate their working environment?
>
> For all practical purposes, 9fans is the last remaining forum for people
> interested in using Plan 9 related systems.  Please advise if you have
> recommendations for other communications outlets that allow people to
> develop, test, and maybe improve on ideas brought about by this system
> many of us find enough interest in to actually use.
>
> -jas
>
>
>

[-- Attachment #2: Type: text/html, Size: 3182 bytes --]

Re: [9fans] syscall 53

[lucio]

> but with codereview extensions now available, it might make sense to
> create/switch to a repository hosted on an hg site so that all the change
> requests, discussions and reviews are available to everyone

I think such a beast would provide the foundations for a serious
effort to bring the distributions back together.  I know many resist
such efforts because of Python (a pet hate of mine, even though I
don't know it from Adam), HG and codereview and I resist accusing them
of reactionary behaviour; I do wish we could get past that problem,
though.

++L

Re: [9fans] syscall 53

[erik quanstrom]

> I think such a beast would provide the foundations for a serious
> effort to bring the distributions back together.  I know many resist
> such efforts because of Python (a pet hate of mine, even though I
> don't know it from Adam), HG and codereview and I resist accusing them
> of reactionary behaviour; I do wish we could get past that problem,
> though.

fwiw...

i use a derivative of nemo's patch system.  all changes are applied through
patches.  anyone can comment on a patch, and comments, and patch
dispositions are mailed to everyone on the list.  the list is open to
general discussion.  the patch system allows folks to pull (not executables)
or apply patches themselves, so in spirit it's closer to git than hg.

i'm open to any sort of gateway to hg/codereview/git that folks find useful.
i just don't want hg to be a requirement.  one of the things i value about
plan 9 is the fact that it's a self-contained system.  requiring hg and websites
runs counter this.  i haven't carved out the time to do anything about it,
but patches are welcome.

i think a key bit to collaboration is going to be setting some ground rules.
and the most important one imho is having a clear goal.

off the top of my head, how about having the best plan 9 we can afford,
which runs on as much hardware, and as many vms as possible.  right out
of the box.

what do yall think?

- erik

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

i like git.  as it is a kind of archival file system, one should be able to
build a plan9 file system interface for it.



On Wed, May 21, 2014 at 9:25 AM, erik quanstrom <quanstro@quanstro.net>wrote:

> > I think such a beast would provide the foundations for a serious
> > effort to bring the distributions back together.  I know many resist
> > such efforts because of Python (a pet hate of mine, even though I
> > don't know it from Adam), HG and codereview and I resist accusing them
> > of reactionary behaviour; I do wish we could get past that problem,
> > though.
>
> fwiw...
>
> i use a derivative of nemo's patch system.  all changes are applied through
> patches.  anyone can comment on a patch, and comments, and patch
> dispositions are mailed to everyone on the list.  the list is open to
> general discussion.  the patch system allows folks to pull (not
> executables)
> or apply patches themselves, so in spirit it's closer to git than hg.
>
> i'm open to any sort of gateway to hg/codereview/git that folks find
> useful.
> i just don't want hg to be a requirement.  one of the things i value about
> plan 9 is the fact that it's a self-contained system.  requiring hg and
> websites
> runs counter this.  i haven't carved out the time to do anything about it,
> but patches are welcome.
>
> i think a key bit to collaboration is going to be setting some ground
> rules.
> and the most important one imho is having a clear goal.
>
> off the top of my head, how about having the best plan 9 we can afford,
> which runs on as much hardware, and as many vms as possible.  right out
> of the box.
>
> what do yall think?
>
> - erik
>
>

[-- Attachment #2: Type: text/html, Size: 2193 bytes --]

Re: [9fans] syscall 53

[Steffen Nurpmeso]

Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
 |i like git.  as it is a kind of archival file system, one should be able to
 |build a plan9 file system interface for it.

Funky idea.
After reading through some Plan9 papers i had the impression that
the backing store of git(1) was designed with Venti in mind
(except that garbage collection is possible), so that would be
some kind of coming home.

--steffen

Re: [9fans] syscall 53

[Bakul Shah]

On Wed, 21 May 2014 09:56:26 PDT Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
>
> i like git.  as it is a kind of archival file system, one should be able to
> build a plan9 file system interface for it.

Have you looked at porting git to plan9? 178K lines of *.[ch].
20K lines of shell scripts (+ 100K+ lines of test scripts).
Also python and perl scripts.

All SCM are in some sense archival systems but the actual
storage scheme used is just one tiny part of the system.
There are a few more requirements which a simple filesystem
model won't satisfy by itself.  Consider:

This is the most fundamental operation of a SCM is to manage
source code updates sanely.  Suppose you had to change files
f1..fn to implement a feature/bugfix.  Now you want to check
them in and share these changes with others. But the checkin
should succeed *only if* the latest versions of f1..fn in the
underlying filesystems are the *same* as the ones you started
from. If this is not true, the entire checkin must be aborted.
It is upto the developer to merge his changes with the latest
versions and try again. [Essentially you are doing a
compare-and-swap operation on a set of files]

You can single thread this through a human but the problem
remains the same and a human (without additional tools) will
not be able to check all this reliably.  For a collaborative,
distributed project where different people may work on
different parts, a human in the middle is not a scalable
solution.  The only reason for a manual SCM is to *slow down*
the rate of change.

Then there other operations like maintaining separate
branches, merging/cherry picking changes from another branch,
reverting from local changes, rolling back a bad checkin,
pushing changes to a shared repo, pulling from it, access
control, etc. etc.

You can certainly build something on top of venti to build a
nice SCM but that would be a researchy project.

Given all this I think hg makes a good compromise if you want
to move forward from the status-quo.

[9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Jeff Sickel]

On May 21, 2014, at 7:13 PM, Bakul Shah <bakul@bitblocks.com> wrote:

> On Wed, 21 May 2014 09:56:26 PDT Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
>> 
>> i like git.  as it is a kind of archival file system, one should be able to
>> build a plan9 file system interface for it.
> 
> Have you looked at porting git to plan9? 178K lines of *.[ch].
> 20K lines of shell scripts (+ 100K+ lines of test scripts).
> Also python and perl scripts.

As we’ve managed to migrate towards the topic of version control
systems, I have to add: I don’t like git.  Maybe it’s because
I’ve used darcs and hg so much more, or maybe it’s just that I
don’t like the way git is used in many situations.  But mostly
I think it’s because I’ve found that many of the github projects
lose sight of what I think is the more important portion of
the source history: the history and development process itself.

At the base level I find that sources and sourcesdump are much
more accessible than many of the DSCMs (e.g., darcs, git, hg)
out there.  Yes it’s great to use hg to snapshot state and
allow easy migration across various systems, but it still
clutters the model.

One of the advantages of having a real archival store, like
Venti, is that it changes the conceptual level of how you deal
with metadata about a project.  When the default is everything
is saved and you can manipulate the namespace to represent
various portions of the history then you don’t get caught
up in all the branching, rebasing, queues, merges, and other
general contortions that would make us happy to warp back in
time to an early copy of Dr. Dobb’s Journal of Computer
Calisthenics & Orthodontia when the future looked bright and
we really could do anything with 8 bits.  Sure working with
an automatic snapshot system can be a headache at times, but
it’s one of those that easily passes, not like sitting down for
a [git] root canal because your tooth has been rotting to the
core while you worry about the configuration for the hottest
continuous integration system with a butler name that shows we
really didn’t learn anything about the 18th or 19th century
transitions to the modern age...

Back on topic: be careful of the dependencies required to
get a system bootstrapped.  The FreeBSD community took BIND
out of the default system and re-wrote a new basic resolver
because the BIND 10+ versions would require packaging Python
into the core distribution.  There’s no reason for
bringing in more than is necessary to build, and putting a
dependency on Python would significantly increase the build
time and total lines of code to maintain just to have hg.
Darcs is in the same boat in that you’d have to keep a version
of Haskell in the system.  Git is the closest as it’s just C,
sort of: it’s a whole lot of code.  But why would you want to
bring in “178K lines of *.[ch], 20K lines of shell scripts, 100K+
lines of test scripts” and have to lug in the massive payload
of Python and Perl just to make it functional?

With a payload that large, it would take all the booster
rockets [money] on the planet to get it into orbit.  And it
still might break apart, fall back to Earth, and kill us in the
process.

At the end of the day, it’s the communication with people that’s
the largest benefit.  Let’s continue building systems based on the
ideas that drew us all to Plan 9 in the first place.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> As we’ve managed to migrate towards the topic of version control
> systems, I have to add: I don’t like git.  Maybe it’s because
> I’ve used darcs and hg so much more, or maybe it’s just that I
> don’t like the way git is used in many situations.  But mostly
> I think it’s because I’ve found that many of the github projects
> lose sight of what I think is the more important portion of
> the source history: the history and development process itself.

Thank you for the opened door, Jeff :-)

I've been mulling revision control (actually, strictly source control,
I have little interest in patching binaries, no matter what old tin can
they are fitted with[1]) and one novel idea that keeps nagging me without
quite settling in my head is the use of Plan 9's "proto" to record at
least part of the history.  Is this entirely original of me?  Is it
totally insane, or is there perceived merit in what my subconscious
keeps shouting at me?

++L

[1] See the original cover of Mike Oldfield's "Tubular Bells"

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> putting a
> dependency on Python would significantly increase the build
> time and total lines of code to maintain just to have hg.

Go is in a different league: Heretical as it may seem, we can generate
Go binaries without compelling all Plan 9 installations to include the
Go toolchain, no matter how valuable some of us may perceive it.  HG
without Python is a dead rat.

All for Bind in Go, raise both hands :-)

++L

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> Go is in a different league: Heretical as it may seem, we can generate
> Go binaries without compelling all Plan 9 installations to include the
> Go toolchain, no matter how valuable some of us may perceive it.  HG
> without Python is a dead rat.

that's a partially binary distribution.  a proper, full, distribution
would have the same issue.

and go introduces new issues, it's much more in flux than python.
the risk is that a go update could then break the system.
and only runs on 386.  it does not run on plan 9 mips, arm, or amd64.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> and go introduces new issues, it's much more in flux than python.
> the risk is that a go update could then break the system.
> and only runs on 386.  it does not run on plan 9 mips, arm, or amd64.

These are very valid reservations, I hadn't thought of them.

L.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Bakul Shah]

On Wed, 21 May 2014 22:25:55 CDT Jeff Sickel <jas@corpus-callosum.com> wrote:
> At the base level I find that sources and sourcesdump are much
> more accessible than many of the DSCMs (e.g., darcs, git, hg)
> out there.  Yes it's great to use hg to snapshot state and
> allow easy migration across various systems, but it still
> clutters the model.

Wouldn't something like cinap's hgfs give you the same thing?

> One of the advantages of having a real archival store, like
> Venti, is that it changes the conceptual level of how you deal
> with metadata about a project.  When the default is everything
> is saved and you can manipulate the namespace to represent
> various portions of the history then you don't get caught
> up in all the branching, rebasing, queues, merges, and other
> general contortions that would make us happy to warp back in
> time to an early copy of Dr. Dobb's Journal of Computer
> Calisthenics & Orthodontia when the future looked bright and
> we really could do anything with 8 bits.  Sure working with
> an automatic snapshot system can be a headache at times, but
> it's one of those that easily passes, not like sitting down for
> a [git] root canal because your tooth has been rotting to the
> core while you worry about the configuration for the hottest
> continuous integration system with a butler name that shows we
> really didn't learn anything about the 18th or 19th century
> transitions to the modern age...

Branch/merge features evolved in response to people's needs.
Merging is necessary if you (as an organization) have
substantial local changes for your product (or internal
use) and you also want to use the latest release from your
vendor. No amount of namespace manipulation is going to
help you. Parallel development is inherently more headachy!

> Back on topic: be careful of the dependencies required to
> get a system bootstrapped.  The FreeBSD community took BIND
> out of the default system and re-wrote a new basic resolver
> because the BIND 10+ versions would require packaging Python
> into the core distribution.  There's no reason for
> bringing in more than is necessary to build, and putting a
> dependency on Python would significantly increase the build
> time and total lines of code to maintain just to have hg.
> Darcs is in the same boat in that you'd have to keep a version
> of Haskell in the system.  Git is the closest as it's just C,
> sort of: it's a whole lot of code.  But why would you want to
> bring in "178K lines of *.[ch], 20K lines of shell scripts, 100K+
> lines of test scripts" and have to lug in the massive payload
> of Python and Perl just to make it functional?

I was certainly not suggesting porting git to plan9!  Just
pointing out how painful and big the task would be -- I was in
fact saying use hg!

I looked at a few alternative and felt hg is the only workable
alternative that is usable right now.

A dependency on python doesn't seem much worse than one on
ghostscript. Neither should be built every time you do `mk all'
in /sys/src but it should be possible to build them. And at
least python would be much more useful than gs!

Though the idea of a scmfs (for checkins as well) and using
vac/venti as a backend is starting to appeal to me : )

> At the end of the day, it's the communication with people that's
> the largest benefit.  Let's continue building systems based on the
> ideas that drew us all to Plan 9 in the first place.

Well, nothing prevents us from continuing to use the existing
system.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> I looked at a few alternative and felt hg is the only workable
> alternative that is usable right now.

I'm going to stand right with Bakul on this.  The actual reasons are
not technical, but they are sound.  I don't want to install Python on
my Plan 9 equipment, but I have HG on Ubuntu and NetBSD to provide the
services I need.  I also have no idea if this is viable for a mixture
of Plan 9 and Windows, exclusively, I avoid Windows and I'm trying to
move away from Linux and Android (yes, same thing, indeed).

That said, let me add my encouragement to sample apatch as suggested
by Erik, although any valid objections ought to be raised here.  One,
from me, comes from Erik himself "a modified version of Nemo's
(a)patch" (I don't have the exact quote handy.  Nemo, could we please
start this exercise with one, and only one version of this?

++L

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> That said, let me add my encouragement to sample apatch as suggested
> by Erik, although any valid objections ought to be raised here.  One,
> from me, comes from Erik himself "a modified version of Nemo's
> (a)patch" (I don't have the exact quote handy.  Nemo, could we please
> start this exercise with one, and only one version of this?

the original version is, as far as i know, no longer in use.
i only mentioned the lineage to credit nemo with the work.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> the original version is, as far as i know, no longer in use.
> i only mentioned the lineage to credit nemo with the work.

Out of curiosity, what prompted not using CVS?  I can think of a
number of reasons, but none that echo with your comments up to now.

L.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

On Thu May 22 09:45:08 EDT 2014, lucio@proxima.alt.za wrote:
> > the original version is, as far as i know, no longer in use.
> > i only mentioned the lineage to credit nemo with the work.
>
> Out of curiosity, what prompted not using CVS?  I can think of a
> number of reasons, but none that echo with your comments up to now.

cvs wasn't considered because it's not appropriate.  the 9atom model
is single committer.  very much like linus' model.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> And at
> least python would be much more useful than gs!

I disagree.  I try to use Plan 9 to display PDFs as much as it is able
to.  When it fails, I resort to whatever my recent version of Ubuntu
supports.

++L

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> Though the idea of a scmfs (for checkins as well) and using
> vac/venti as a backend is starting to appeal to me : )

Let's open the discussion, Plan 9 has some baseline tools other OSes
are still thinking about and will probably implement stupidly.  Since
RCS I've been thinking that there has to be a better way and CVS went
a long way to satisfy my personal requirements.  Now may well be the
time to look at fresher options.  I'd like to throw Go into the
picture because I have more or less sold my soul to that devil, but
I'm willing to continue using C (I dislike that Go comes with C
compilers and assemblers that seem to be heading off into the hills -
our little group of Go porters (please forgive me for presuming) ought
to be addressing this issue as well) if the community feels strongly
about it.

L += 1

PS: I've dropped ++ from the list of operators I use in Go. I see no
value in this operator that the more explicit form += 1 does not
satisfy better.  Please do show me how I'm mistaken.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Bakul Shah]

On Thu, 22 May 2014 07:36:54 +0200 lucio@proxima.alt.za wrote:
> > Though the idea of a scmfs (for checkins as well) and using
> > vac/venti as a backend is starting to appeal to me : )
>
> Let's open the discussion, Plan 9 has some baseline tools other OSes
> are still thinking about and will probably implement stupidly.  Since
> RCS I've been thinking that there has to be a better way and CVS went
> a long way to satisfy my personal requirements.  Now may well be the
> time to look at fresher options.

I am attaching an excerpt from an old email (May 26, 2011)
that I never sent.  These ideas are not even half baked.  But
may be they will trigger some creative thoughts.

On Thu, 26 May 2011 20:16:11 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
>
> file servers are great, but hg concepts aren't really file-system oriented.
> it might be nice to be able to
> 	diff -c (default mybranch)^/sys/src/9/pc/mp.c
> but hg diff figures out all the alloying stuff, like the top-level of the
> repo anyway.  also, ideas like push, pull, update, etc., don't map very well.
> so a hg file server seems to me a bit like, "have hammer, see nail".

I thik the filesystem model is more like an electric motor
that powers all sorts of things given the right attachments!

> if i'm missing why this is an interesting idea, i'd love to know what
> i don't see.

I partially agree with you; hence the suggestion about editor
integration.

It is just that I have wondering about just how far the FS
model can be pushed or extended seamlessly in various
directions.

In the context of SCMs, we can map a specific file version to
one specific path -- this is what hgfs above does.  But what
about push/pull/commit etc.? One idea is to to map them to
operations on "magic" files.

For example,
- local file copies appear as normal files.
- cat .hg/status  == hg status
- cat > .hg/commit == hg commit
- cat .hg/log == hg log
- echo -a > .hg/revert == hg revert -a
- echo $url > .hg/push == hg push $url
- echo $url > .hg/pull == hg push -u $url
- ls .hg/help
- cat .hg/help/push

In fact the backend SCM need not be mercurial; it can git,
venti etc. Can we come up with a minimal set of features?

Do we gain anything by mapping $SCM commands to special files?
The same question can be asked about many of existing plan9
filesystems. At least for me visualizing new uses is easier
when more things can be fitted in a simpler model.

Features such as atomic commits, changesets, branches, push,
pull, merge etc. can be useful in multiple contexts so it
would be nice if they can integrated smoothly in an FS.

Example uses:
- A backup is nothing but a previous commit. A nightly
  backup cron job to do "echo `{date} > .commit"
- an offsite backup is a `push'.
- Initial system install is like a clone.
- An OS upgrade is like a pull.
- Installing a package is like a pull (or if you built it
  locally, a commit)
- Uinstall is reverting the change.
- Each machine's config can be in its own branch.
- You can use clone to create sandboxes.
- A commit makes your private temp view permanent and
  potentially visible to others.
- Conversely old commits can be spilled to a backup
  media (current SCMs want the entire history online).
- Without needing a permanent connection you can `pull' data.
  [never have to do `9fs sources; ls /n/sources/contrib'.]

A better integration can inspire new uses:
- a timemachine like a gui can help quickly scroll through
  changes in some file (or even a bird's eye view of changes
  in all the files).
- combining versioning + auto push/pull with other filesystems
  can open up new uses. You can keep your own daily archive of
  http://nyt.com/ or see how a story develops by scrolling through
  changes.

Just some things I was mulling about. As you can see there are
lots of open questions & half-baked ideas to sort through.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> I am attaching an excerpt from an old email (May 26, 2011)
> that I never sent.  These ideas are not even half baked.  But
> may be they will trigger some creative thoughts.

I was hoping for this type of interest, I'm very pleasantly surprised
that it now seems to be materialising.  I guess I'll need to track
this discussion on google groups now, ACME mail is a bit limited in
this regard (and probably also needs to be enhanced both for
discussions like this one and for code review purposes),

L += 1

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> Features such as atomic commits, changesets, branches, push,
> pull, merge etc. can be useful in multiple contexts so it
> would be nice if they can integrated smoothly in an FS.
>
> - Installing a package is like a pull (or if you built it
>   locally, a commit)
> - Uinstall is reverting the change.
> - Each machine's config can be in its own branch.

what is the advantage over seperate files?  imo, this complicates the issue.

> - You can use clone to create sandboxes.
> - A commit makes your private temp view permanent and
>   potentially visible to others.
> - Conversely old commits can be spilled to a backup
>   media (current SCMs want the entire history online).
> - Without needing a permanent connection you can `pull' data.
>   [never have to do `9fs sources; ls /n/sources/contrib'.]

this is a nice list, but i think a key point is not being teased out.
the dump file systems are linear.  there is a full order of system
archives.  in hg, there is a full order of the tip, but not so of
branches in general.  in git, multiple orders (not sure if they're
full or not) can be imposed on a set of hashes.

another key point is that all distributed scms that i've used clone
entire systems.  but what would be more interesting is to clone, say,
/sys/src or some proto-based subset of the system, while using the
main file system for everything else.  imagine you want to work on
the kernel, and imagine that you keep console log files.  clearly
you want to see both the new log entries, and the modified kernel.

i would be concerned that this really challenges the plan 9 model
of namespaces.  one would have to figure out how to keep oneself
out of namespace hell if one were to build this into a file system and
use it heavily.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Bakul Shah]

On Thu, 22 May 2014 08:45:48 EDT erik quanstrom <quanstro@quanstro.net> wrote:
> > Features such as atomic commits, changesets, branches, push,
> > pull, merge etc. can be useful in multiple contexts so it
> > would be nice if they can integrated smoothly in an FS.
> >
> > - Installing a package is like a pull (or if you built it
> >   locally, a commit)
> > - Uinstall is reverting the change.
> > - Each machine's config can be in its own branch.
>
> what is the advantage over seperate files?  imo, this complicates the issue.

I don't quite recall what I was thinking 3 years ago in a 30
minute window but I think the idea was that you have a set of
configuration files which all need to be consistent and
if you wanted to "roll back" changes, you'd have to undo
all those changes.

> > - You can use clone to create sandboxes.
> > - A commit makes your private temp view permanent and
> >   potentially visible to others.
> > - Conversely old commits can be spilled to a backup
> >   media (current SCMs want the entire history online).
> > - Without needing a permanent connection you can `pull' data.
> >   [never have to do `9fs sources; ls /n/sources/contrib'.]
>
> this is a nice list, but i think a key point is not being teased out.
> the dump file systems are linear.  there is a full order of system
> archives.  in hg, there is a full order of the tip, but not so of
> branches in general.  in git, multiple orders (not sure if they're
> full or not) can be imposed on a set of hashes.

So if we do SCM right, backups are just a subset, right?!  No,
don't believe that.  No time to explore this right now but I
think dumps are at a different (lower) level from SCM data.

> another key point is that all distributed scms that i've used clone
> entire systems.  but what would be more interesting is to clone, say,
> /sys/src or some proto-based subset of the system, while using the
> main file system for everything else.  imagine you want to work on
> the kernel, and imagine that you keep console log files.  clearly
> you want to see both the new log entries, and the modified kernel.

Actually with something like venti as the store, `clone' is
trivial! Just find the hash for a particular changeset you want
to clone and you can build the rest on demand. `rebase' or
`pull' will be more painful.

> i would be concerned that this really challenges the plan 9 model
> of namespaces.  one would have to figure out how to keep oneself
> out of namespace hell if one were to build this into a file system and
> use it heavily.

Your concern is a bit premature.  We are just handwaving right
now!  I am interested in finding out just how far we can push
the plan9 model -- and if the current model doesn't naturally
fall out of any extended model, we'd know.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> > another key point is that all distributed scms that i've used clone
> > entire systems.  but what would be more interesting is to clone, say,
> > /sys/src or some proto-based subset of the system, while using the
> > main file system for everything else.  imagine you want to work on
> > the kernel, and imagine that you keep console log files.  clearly
> > you want to see both the new log entries, and the modified kernel.
> 
> Actually with something like venti as the store, `clone' is
> trivial! Just find the hash for a particular changeset you want
> to clone and you can build the rest on demand. `rebase' or
> `pull' will be more painful.

venti is going to be a difficult model.  the objects in scm are typically
arbitrarly sized, and have arbitrary metadata.  θfs has this model for
its object store.  venti does not.

> > i would be concerned that this really challenges the plan 9 model
> > of namespaces.  one would have to figure out how to keep oneself
> > out of namespace hell if one were to build this into a file system and
> > use it heavily.
> 
> Your concern is a bit premature.  We are just handwaving right
> now!  I am interested in finding out just how far we can push
> the plan9 model -- and if the current model doesn't naturally
> fall out of any extended model, we'd know.

i don't know.  this concern was addressed in the very first paper,
and i have delt with some plan 9-based systems that did this, and
didn't get it right.  

but it can be addressed without much trouble.  just have the fs export
em all.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Steve Simon]

Ok, My take on VCS/SCM and fossil+venti

you can copy the repository to a working copy with - dircp.
you can tag a release with - dircp

(don't forget, copying a directory set is very nearly free in terms of disk space).

adding a log message is "cat >> ChangeLog"

the only bit missing is the ability to freeze (seal) a tagged version. You need
the ability for users in a special "dumpers" group to be ablc to take archival snapshots
on demand.

merge can be done using diff3 from ed7 (in my contrib), though this is a bit clunky.

a real graphical merge tool would be great (exercise for the reader ☺)

-Steve

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Aram Hăvărneanu]

On Thu, May 22, 2014 at 7:36 AM,  <lucio@proxima.alt.za> wrote:
> (I dislike that Go comes with C
> compilers and assemblers that seem to be heading off into the hills -
> our little group of Go porters (please forgive me for presuming) ought
> to be addressing this issue as well)

What "issues"?

What are you talking about?

-- 
Aram Hăvărneanu

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> What are you talking about?

Merely that it would be nice if 8c on Plan 9 was the same utility,
whether Go is installed or isn't.  I'm not expecting it to just
happen, but I do think it would be better than what we have now.

L.

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Aram Hăvărneanu]

> Merely that it would be nice if 8c on Plan 9 was the same utility,
> whether Go is installed or isn't.  I'm not expecting it to just
> happen, but I do think it would be better than what we have now.

And what would be the benefit of that?

Plan 9's 8c and Go's 8c are different programs. Different calling
conventions, different output. Right now Go's 8c uses liblink and
generates machine code. .8 files cannot be consumed by Plan 9's 8l.
Even before liblink there were serious incompatibilities in the object
files.

8c is going away soon anyway, after 8g is rewritten in Go.

-- 
Aram Hăvărneanu

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> Branch/merge features evolved in response to people's needs.
> Merging is necessary if you (as an organization) have
> substantial local changes for your product (or internal
> use) and you also want to use the latest release from your
> vendor. No amount of namespace manipulation is going to
> help you. Parallel development is inherently more headachy!

i have not found this to be a problem when maintaining
9atom.  simply evaluate patches as they come in.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 3890 bytes --]

i was thinking more in terms of having a git client (fs) on plan9 and using
any number of public git servers. i'm looking at hgfs now; perhaps it
already does all that's needed.


On Wed, May 21, 2014 at 8:25 PM, Jeff Sickel <jas@corpus-callosum.com>wrote:

>
> On May 21, 2014, at 7:13 PM, Bakul Shah <bakul@bitblocks.com> wrote:
>
> > On Wed, 21 May 2014 09:56:26 PDT Skip Tavakkolian <
> skip.tavakkolian@gmail.com> wrote:
> >>
> >> i like git.  as it is a kind of archival file system, one should be
> able to
> >> build a plan9 file system interface for it.
> >
> > Have you looked at porting git to plan9? 178K lines of *.[ch].
> > 20K lines of shell scripts (+ 100K+ lines of test scripts).
> > Also python and perl scripts.
>
> As we’ve managed to migrate towards the topic of version control
> systems, I have to add: I don’t like git.  Maybe it’s because
> I’ve used darcs and hg so much more, or maybe it’s just that I
> don’t like the way git is used in many situations.  But mostly
> I think it’s because I’ve found that many of the github projects
> lose sight of what I think is the more important portion of
> the source history: the history and development process itself.
>
> At the base level I find that sources and sourcesdump are much
> more accessible than many of the DSCMs (e.g., darcs, git, hg)
> out there.  Yes it’s great to use hg to snapshot state and
> allow easy migration across various systems, but it still
> clutters the model.
>
> One of the advantages of having a real archival store, like
> Venti, is that it changes the conceptual level of how you deal
> with metadata about a project.  When the default is everything
> is saved and you can manipulate the namespace to represent
> various portions of the history then you don’t get caught
> up in all the branching, rebasing, queues, merges, and other
> general contortions that would make us happy to warp back in
> time to an early copy of Dr. Dobb’s Journal of Computer
> Calisthenics & Orthodontia when the future looked bright and
> we really could do anything with 8 bits.  Sure working with
> an automatic snapshot system can be a headache at times, but
> it’s one of those that easily passes, not like sitting down for
> a [git] root canal because your tooth has been rotting to the
> core while you worry about the configuration for the hottest
> continuous integration system with a butler name that shows we
> really didn’t learn anything about the 18th or 19th century
> transitions to the modern age...
>
> Back on topic: be careful of the dependencies required to
> get a system bootstrapped.  The FreeBSD community took BIND
> out of the default system and re-wrote a new basic resolver
> because the BIND 10+ versions would require packaging Python
> into the core distribution.  There’s no reason for
> bringing in more than is necessary to build, and putting a
> dependency on Python would significantly increase the build
> time and total lines of code to maintain just to have hg.
> Darcs is in the same boat in that you’d have to keep a version
> of Haskell in the system.  Git is the closest as it’s just C,
> sort of: it’s a whole lot of code.  But why would you want to
> bring in “178K lines of *.[ch], 20K lines of shell scripts, 100K+
> lines of test scripts” and have to lug in the massive payload
> of Python and Perl just to make it functional?
>
> With a payload that large, it would take all the booster
> rockets [money] on the planet to get it into orbit.  And it
> still might break apart, fall back to Earth, and kill us in the
> process.
>
> At the end of the day, it’s the communication with people that’s
> the largest benefit.  Let’s continue building systems based on the
> ideas that drew us all to Plan 9 in the first place.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 4488 bytes --]

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[lucio]

> i was thinking more in terms of having a git client (fs) on plan9 and using
> any number of public git servers. i'm looking at hgfs now; perhaps it
> already does all that's needed.

A git client would be good.  When I attempted to install git on NetBSD
I ran into trouble because the packaged version (even if compiled from
sources) demanded the X development system to be installed (I need to
run this by the NetBSD folk, but it hardly matters).

I in fact now use Ubuntu to fetch git stuff and the seamlessness of
the Go tool goes out the window.  The Plan 9 side of this is really
for discussion in the Go forum(s), but perhaps others here would care
to comment?

More seriously, though, on the issue of revision control on Plan 9
(and code review, that being the really important aspect) I'd like us
to keep in mind that being able to interface with existing
repositories, difficult as it may be, would be greatly beneficial.  To
what degree one bends over backwards to be compatible is a separate
discussion, to be had about the merits of each system.  But the
underlying system must not preclude the option.

L += 1

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[erik quanstrom]

> More seriously, though, on the issue of revision control on Plan 9
> (and code review, that being the really important aspect) I'd like us
> to keep in mind that being able to interface with existing
> repositories, difficult as it may be, would be greatly beneficial.  To

like i said, a hg gateway hosted on google code or whatever is for me
welcome, but that's not my top priority.  if it is your top priority, then
go to it.  if you need some system changes, submit patches.  if they're
invasive, it might be good to discuss the plan first in public.

- erik

Re: [9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Alex Jordan]

[-- Attachment #1: Type: text/plain, Size: 789 bytes --]

On Wednesday, May 21, 2014, Jeff Sickel <jas@corpus-callosum.com> wrote:

> Git is the closest as it’s just C,
> sort of: it’s a whole lot of code.  But why would you want to
> bring in “178K lines of *.[ch], 20K lines of shell scripts, 100K+
> lines of test scripts” and have to lug in the massive payload
> of Python and Perl just to make it functional?
>
 In case anyone is _really determined_ to do this: libgit2 has only
117K lines of code, better than git's 178K. It also has a mere 200 lines of
shell, total. It doesn't depend on Python or Perl. It's also supposed to be
much more portable than git, though I dunno if that would actually work out
for Plan 9. Probably not that much.
You could try porting libgit2 and then write some small C wrappers using it.

[-- Attachment #2: Type: text/html, Size: 998 bytes --]

[9fans] CMS/MMS (VCS/SCM/DSCM) [was: syscall 53]

[Riddler]

I would throw in a vote in favor of a good git client. It's something
I use daily and I find it works well with distributed people working
on the same project. Which is a situation Linux and plan9 share.

Last time I looked at how it was put together the 'core' was actually
just a small handful of commands most of which where implementing the
content addressed filesystem (that git essentially is).
All the other commands just built on top of those basic ones to
implement handy features or more user friendly-ness. As suggested
above I also think git should fit reasonably well with plan9's
filesystem interface.

To continue throwing out half-baked ideas. I would envision something like:

cp /usr/glenda/myproject /n/git/staging ##Add project files to staging
echo "Add new feature X" > /n/git/commit ###Commit staging directory
with message
cat /n/git/log # view all logs
echo 34nb432 > /n/git/log ###Tell it you want to read log 34nb...
cat /n/git/log ###Reads out log selected with previous command
echo "my-git-repo.example.com:HEAD" > /n/git/pull ###Pull changes from
online repo

Of course, as git just wants another .git directory that could just as easily be
echo "/n/sources" > /n/git/pull ###Pull changes to sources

If it was decided that someone might like to try to port git I'd
imagine you could just port the basic boiler plate commands, and a few
higher level ones, ignoring the fancy util commands like bisect that
are unlikely to be needed often and tackle those as-needed. But that
comment could stand some verification.

http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain is an
interesting read if anyone is interested.

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting Skip Tavakkolian <skip.tavakkolian@gmail.com>:

> i like git.  as it is a kind of archival file system, one should be able to
> build a plan9 file system interface for it.
>

This should be possible for any reasonably sane scm; c.f. cinap's hgfs.

But all the DVCS in the world doesn't let us see code that is never uploaded
in the first place.  I can't even count the number of programs that are only
even known by oral tradition, mentioned only in passing, then never to be
heard of again. "Oh, I'll upload it when it's ready."  Years go past: nobody
has even defined 'ready.'  Nowadays when someone says "it's not ready for
the world to see" I just read "I am a liar and I have nothing."

Plan 9 is fully self-sufficient;  code review tools don't review code.
People review code.  They just have to see the code first.

khm

Re: [9fans] syscall 53

[lucio]

> But all the DVCS in the world doesn't let us see code that is never uploaded
> in the first place.

Obvious, good grounds for a conspiracy theory.  Such code simply does
not exist, no matter how much you harp on it.  Next thing, you'll
insist I need to prove that it does not exist, putting you squarely in
the Creationists camp.

(BTW: do not believe everything South African expatriates have to say.)

++L

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

> Obvious, good grounds for a conspiracy theory.  Such code simply does
> not exist, no matter how much you harp on it.  Next thing, you'll
> insist I need to prove that it does not exist, putting you squarely in
> the Creationists camp.

I don't need anyone to prove anything; I just need them to stop pretending
anyone gives a crap about e.g. 9front except 9front users.  9front is not
some kind of 'mission'; it's there because it serves our needs well.

For the record, hgfs does not depend on python.  It's currently read-only
but can be extended easily, in C.  It works within existing plan 9
paradigms and is one project I'm very excited to watch.

git offers nothing hg does not offer.  hg is already present and working on
my systems.  I do not now have, and do not foresee ever having, any desire
or need to use git on plan 9.  The only real reason people keep braying about
it is their habits on other operating systems.

> (BTW: do not believe everything South African expatriates have to say.)

How about I pretend you didn't just call my wife a liar (a priori and for
absolutely no reason, to boot), and you stop pretending you have any insight
into her motivations and decision-making processes?  Thanks in advance.

khm

Re: [9fans] syscall 53

[Bakul Shah]

On Thu, 22 May 2014 03:43:15 -0000 Kurt H Maier <khm@sciops.net> wrote:
>
> But all the DVCS in the world doesn't let us see code that is never uploaded
> in the first place.  I can't even count the number of programs that are only
> even known by oral tradition, mentioned only in passing, then never to be
> heard of again. "Oh, I'll upload it when it's ready."  Years go past: nobody
> has even defined 'ready.'  Nowadays when someone says "it's not ready for
> the world to see" I just read "I am a liar and I have nothing."

I submit not having a proper DVCS is part of the problem for
this.  The reason github is so successful is because it is so
easy to upload code and then to collaborate, get bug fixes
etc.  While some incomplete code in one's own src tree may not
get looked at for a long time and ultimately may never see the
light of the day. Github should use the slogan "it doesn't
/have/ to be ready for the world to see!".

Re: [9fans] syscall 53

[Aram Hăvărneanu]

> I submit not having a proper DVCS is part of the problem for
> this.  The reason github is so successful is because it is so
> easy to upload code and then to collaborate, get bug fixes
> etc.  While some incomplete code in one's own src tree may not
> get looked at for a long time and ultimately may never see the
> light of the day. Github should use the slogan "it doesn't
> /have/ to be ready for the world to see!".

You are mistaken, nobody prevents people from putting stuff on
sources, and nobody forces people to git push. In every case people
have to decide whether to share something or not.

-- 
Aram Hăvărneanu

Re: [9fans] syscall 53

[lucio]

> i think a key bit to collaboration is going to be setting some ground rules.
> and the most important one imho is having a clear goal.

To keep the ball rolling, let me suggest that we drop the requirement
that Plan 9 be self-contained as a measure to make some progress with
existing expertise.  I wish we could keep Plan 9 as the sole
foundation, but I think that's just not viable, I feel treasonous
suggesting otherwise, but I'm merely stating my sentiments, not
imposing a rule here.

At the same time, the danger that Pyhton-HG-Codeview and web access
all somehow become unviable for the Plan 9 community (who knows,
there's a lot of ugliness out there and, in my opinion, it is
growing), I would like to propose a backup plan to ensure that
developments are tracked outside of the HG.

I used CVS with some success in the past in parallel with HG while Go
for Plan 9 was still in its infancy, but it was a bit of a mission and
I could not sustain it when the transition from "make" to "go-tool"
took place: the changes were coming too thick and fast.  Maybe hiro's
fibre-optic based neural network will make the next efforts succeed (I
restarted the CVS project in an effort to get the ARM developments of
Go for Plan 9 in sync, but three months down the line I'm still
precisely nowhere with that plan).

Something to provide a similar, this time successful, safeguard would
be great, some (including me) would no doubt say essential.  I'd love
to add a new-thought (new-age?) version control system based on venti
and proto to Plan 9 and be the envy of the competition, but it would
be a big project and more pressing issues in Plan 9 have much higher
priority and more likelihood of success, in my opinion.

I think we can all benefit from taking this discussion further, if only
to explore how others are addressing the shortcomings of Plan 9
development and evolution and possibly discover new ways to deal with
our own problems.  No need to get political about it, what we need is
some facts, some good or bad advice, some humour and, because this is
9fans, after all, some conflict.

++L

Re: [9fans] syscall 53

[erik quanstrom]

> To keep the ball rolling, let me suggest that we drop the requirement
> that Plan 9 be self-contained as a measure to make some progress with
> existing expertise.  I wish we could keep Plan 9 as the sole
> foundation, but I think that's just not viable, I feel treasonous
> suggesting otherwise, but I'm merely stating my sentiments, not
> imposing a rule here.

can you explain why is this not viable?  what essential bits would be
missing if hg/git/whatever is not tightly integrated into the process?

- erik

Re: [9fans] syscall 53

[lucio]

> can you explain why is this not viable?  what essential bits would be
> missing if hg/git/whatever is not tightly integrated into the process?

Maybe I didn't explain well: self-contained Plan 9 does not provide
code review tools.  Whereas I can follow (I have learnt to) the
conventions of codereview (as used in Go, by whatever name), I do not
expect less disciplined approaches to be as appealing and successful
as codereview, maybe that's just me, but I am speaking for myself and
measuring the 9fans community by the same metric.

Ergo: Plan 9 does not (yet?) contain sufficient tools to be
self-sustaining.  We're human and we're subjective individuals.  We
have a very weak bond holding this community together, consisting, at
least in my case, more of what other OSes are missing than of any
loyalty to what we have.  Whatever we deploy to provide a platform for
progress needs to be stronger than the criticism that will be levelled
at it; it needs firm buy-in by the community.  I, for one, would need
some hard sell to consider patch and its offspring as sufficient and
much more to convince me that it would be technically superior to
codereview, others may well be even more hard-assed than I am, and
their skills and contributions are too important to sacrifice.

Strong words?  Definitely, 9fans has survived past worse and will
again, so they need not be taken to heart.  But we do risk falling too
far behind to ever be of any significance and that would be our loss
as well as a loss for the entire IT community.

Again, I'm being subjective, by all means keep throwing your stones.
A thin skin here is not going to help.

++L

Re: [9fans] syscall 53

[erik quanstrom]

> Ergo: Plan 9 does not (yet?) contain sufficient tools to be
> self-sustaining.

we've managed for years....

> at it; it needs firm buy-in by the community.  I, for one, would need
> some hard sell to consider patch and its offspring as sufficient and
> much more to convince me that it would be technically superior to
> codereview, others may well be even more hard-assed than I am, and
> their skills and contributions are too important to sacrifice.

i'd encourage you to try participating with apatch, and the mailing
list.

i would find specific issues easier to reason about than
"technically superior".

- erik

Re: [9fans] syscall 53

[lucio]

> i'd encourage you to try participating with apatch, and the mailing
> list.
> 
Conceded.  I never meant to suggest that one shouldn't, merely that it
could be asking for a leap of faith.  I have something waiting
specifically for this opportunity.

So let me ask a few questions.

	Is this the right place to discuss the actual procedure to include
	apatch in one's private Bell Labs' distribution?

	Is it preferable to use apatch within 9atom, or is it reasonably
	portable to the "legacy" (I presume that is what David intends
	with that moniker) distribution?

	Is there a willing participant who is prepared to offer backing
	storage for the target distribution(s) even when he or she
	disagrees with the outcome?  (And be vociferous both about
	disagreeing and accepting the outcome?) David, you've been good
	that way, are there others?  I'd like to think that we can
	leverage Plan 9's distributable properties to have more than one?
	I'd like to offer this myself, but I live at the bottom of an
	Internet gravity well, anyone with sufficient patience is welcome
	to approach me.

	Is it worth it to get going immediately, or should further
	discussion take place?  Whose opinions ought we to be canvassing,
	are there parties that ought to be reached outside of 9fans?  Any
	deities we need to burn offerings to?

	Do we need additional discussion forums (fora, for the pedants)
	for the more thin-skinned contributors (or for any other need to
	keep community members from falling out with each other)?

	Are there any other questions I'm missing from my somewhat limited
	and remote perspective?

> i would find specific issues easier to reason about than
> "technically superior".

Touché.  Although it's kind of obvious I wouldn't be welcome to say
"subjectively superior" instead, right?

++L

Re: [9fans] syscall 53

[erik quanstrom]

> 	Is this the right place to discuss the actual procedure to include
> 	apatch in one's private Bell Labs' distribution?
>
> 	Is it preferable to use apatch within 9atom, or is it reasonably
> 	portable to the "legacy" (I presume that is what David intends
> 	with that moniker) distribution?

apatch is as specialized as patch in that it creates patches against the current
9atom tree.  patch is still there, and i often generate parallel patches for both
9atom and sources.  as long as your source hasn't drifted wrt the atom tree,
you can send an apatch from a 9atom, hybride or standard tree.

> 	Is there a willing participant who is prepared to offer backing
> 	storage for the target distribution(s) even when he or she
> 	disagrees with the outcome?  (And be vociferous both about
> 	disagreeing and accepting the outcome?) David, you've been good
> 	that way, are there others?  I'd like to think that we can
> 	leverage Plan 9's distributable properties to have more than one?
> 	I'd like to offer this myself, but I live at the bottom of an
> 	Internet gravity well, anyone with sufficient patience is welcome
> 	to approach me.

you are aware that you can mount the 9atom sources directly these days?

  nflag=-n srv $nflag -q tcp!atom.9atom.org atom /n/atom atom	# add to 9fs

- erik

Re: [9fans] syscall 53

[lucio]

> you are aware that you can mount the 9atom sources directly these days?

Sure, but then I'd have to commit harakiri as self-immolation is the
only avenue left to appease the Internet gods in the tip of Africa :-)

Still, I'll keep that in mind for occasional experimentation.

More seriously, we do need to start with a benchmark distribution that
everybody can buy into.  Maybe the right idea is to compute the
intersection of common sources for BL, 9atom, 9front, NmX et al.  and
try to shoehorn more into that framework.

Or maybe start with nothing and build from there, one patch at the
time?  What makes sense here?  There has to be some starting point.

Maybe the non-existent unreleased source?

L.

Re: [9fans] syscall 53

[Riddler]

Some thoughts from a new guy.

I would suggest the best way might be to take the 'main' labs sources
distribution and build one patch at a time.
Should in theory leave behind it a nice linear list of patches from a
common base and might get some patches ready to go into sources (if
accepted) for features and/or hardware that 9atom, 9front etc. have
added that sources does not.

It would also force a 'list' of the new features compared to sources
to be created so if someone has quietly added X and no one has noticed
it will be tripped over. Corollary is it would be dropped if it's not
tripped over which may be good or bad depending on your view.

Re: [9fans] syscall 53

[lucio]

>> Ergo: Plan 9 does not (yet?) contain sufficient tools to be
>> self-sustaining.
>
> we've managed for years....

With all respect due to you and Mr Coraid (don't make mne look his
name up, he's so conspicuosly absent on this list, even his hallowed
name has faded - bless him :-) for all you have contributed, Coraid
is as exceptional as Cuba (and more successful, to our relief, no
doubt).

But that was a formidable battle (nay, a full-out war) you waged
against all odds and at some stage it must have felt as if the USSR
stood a greater chance against Reagan and Thatcher combined.

I doff my hat, but I'm not going to start believing in miracles any
time soon, much as this is one miracle I would like to see and I
think, at minimum, you've overcome the five-year survival barrier.

++L

Re: [9fans] syscall 53

[erik quanstrom]

> With all respect due to you and Mr Coraid (don't make mne look his

Coile.

- erik

Re: [9fans] syscall 53

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]

my Plan 9 environment is the only one that i feel i know and have control
over; i don't feel the same about my (Canonical's) ubuntu desktop, my
(Google's) chromebook, my (Apple's) macbook, my (T-Mobile/Google's) android
phone, etc, etc, precisely because of auto-update.  i appreciate that they
want to be my sysadm, but it means that i'm no longer an informed consumer.


On Tue, May 20, 2014 at 9:41 AM, ron minnich <rminnich@gmail.com> wrote:

> I have a different perspective. There are millions of chromebooks out
> there updating all the time, from the firmware to the kernel to the
> root file system to everything. It all works.
>
> If you are telling me that the upgrade technology of Plan 9 can not
> handle an automatic upgrade, fine; we have the proof.
>
> If you are telling me Plan 9 should not or never will be able to
> handle an automatic upgrade, and is going to require a heads up email
> for each kernel change, I have a hard time taking that seriously.
>
> This is not a human communication problem. It's a technology problem,
> trivially solved for many years now by many systems.
>
> ron
>
>

[-- Attachment #2: Type: text/html, Size: 1552 bytes --]

Re: [9fans] syscall 53

[lucio]

> my Plan 9 environment is the only one that i feel i know and have control
> over; i don't feel the same about my (Canonical's) ubuntu desktop, my
> (Google's) chromebook, my (Apple's) macbook, my (T-Mobile/Google's) android
> phone, etc, etc, precisely because of auto-update.  i appreciate that they
> want to be my sysadm, but it means that i'm no longer an informed consumer.

It may not be a coincidence, but I'm going through the same phase.
I'm not yet paranoid, but I have been seriously contemplating moving
my (other) workstation to NetBSD because I am more comofortable that
there are no mysterious daemons lurking in its innards that I ought to
know about.

Like you, I am perfectly comfortable with my Plan 9 network and
continue to wish I did not have to supplement it with any of the
paranoia-inducing offering out there.

+L

PS: I have resurrected an old Nokia (5110, but I'm not sure) phone,
but it's been borrowed and I have my doubts that I will be seeing it
again any time soon.  Maybe this forum can help me decide what GSM
equipment is safe from interference by the networks and their
information masters?  My current hate-object is my Galaxy S4.

Re: [9fans] syscall 53

[erik quanstrom]

> PS: I have resurrected an old Nokia (5110, but I'm not sure) phone,
> but it's been borrowed and I have my doubts that I will be seeing it
> again any time soon.  Maybe this forum can help me decide what GSM
> equipment is safe from interference by the networks and their
> information masters?  My current hate-object is my Galaxy S4.

purchasing a phone directly from google allows one to cut
out the network operator. (to some extent.)  in theory the
cyanogenmod phone is less tied to google.  sadly, it doesn't
bluetooth-le, and i need that.

all options appear to me to boil down to walled gardens.
unless you build it yourself.

- erik

Re: [9fans] syscall 53

[lucio]

> all options appear to me to boil down to walled gardens.
> unless you build it yourself.

I do wish the news was better, but at least I don't have to feel
alone.  Thanks, Erik.

++L

Re: [9fans] syscall 53

[Kurt H Maier]

Quoting lucio@proxima.alt.za:

> PS: I have resurrected an old Nokia (5110, but I'm not sure) phone,
> but it's been borrowed and I have my doubts that I will be seeing it
> again any time soon.  Maybe this forum can help me decide what GSM
> equipment is safe from interference by the networks and their
> information masters?  My current hate-object is my Galaxy S4.

I recommend the nokia 6700 classic, as it's one of the best s40 phones
that still supports real gps (including offline maps and routing).  The
only caveat to s40 is the nokia xpress browser, which does pre-rendering
on nokia (now microsoft) servers, even for https traffic.

khm

Re: [9fans] syscall 53

[hiro]

> I recommend the nokia 6700 classic, as it's one of the best s40 phones
> that still supports real gps (including offline maps and routing).  The
> only caveat to s40 is the nokia xpress browser, which does pre-rendering
> on nokia (now microsoft) servers, even for https traffic.

I don't like s40, it doesn't have a good putty. E72 has good maps,
opera, email, etc.
There are plenty of old bugs that will now never be fixed, but at
least I can be sure it stays stable. Perhaps we should make plan9
binary compatible to s60.

Re: [9fans] syscall 53

[Bakul Shah]

On Mon, 19 May 2014 17:34:24 EDT Anthony Sorace <a@9srv.net> wrote:
>
> Ron wrote:
>
> > That said, the problems were due (IMHO) to a limitation in the
> > update mechanism, not to the inclusion of a new system call.
>
> This is true depending on how you define "update mechanism".
> A simple note from whoever made the decision to push the
> change out to the effect of "hey, we're going to add a new
> syscall, update your kernels before pulling new binaries" a
> while before the push would have been sufficient.

I never understood why binaries are pulled. Even on a lowly
RPi it takes 4 minutes to build everything (half if you cut
out gs). And the 386 binaries are useless on non-386
platforms!

Why not just separate binary and source distributions?  Then
include a file in the source distribution to warn people about
changes such as this one (or the one about 21bit unicode) and
how to avoid painting yourself in a corner. The binary distr.
should have a provision for *only* updating the kernel and
insisting the user boots off of it before further updates can
proceed.

This is a solved problem; not exactly rocket science.  The
harder problem is the social one.

Re: [9fans] syscall 53

[erik quanstrom]

> I never understood why binaries are pulled. Even on a lowly
> RPi it takes 4 minutes to build everything (half if you cut
> out gs). And the 386 binaries are useless on non-386
> platforms!
>
> Why not just separate binary and source distributions?  Then
> include a file in the source distribution to warn people about
> changes such as this one (or the one about 21bit unicode) and
> how to avoid painting yourself in a corner. The binary distr.
> should have a provision for *only* updating the kernel and
> insisting the user boots off of it before further updates can
> proceed.

i think this is a good idea.  the 9atom usb installer builds the
full set of executables for the arches you want as part of the
install.  it reduces the size of the install image by at least 20MB
per installed architecture.  given the sad state of broadband
in many places, i'm hoping this is a good tradeoff.

- erik

Re: [9fans] syscall 53

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

On 19 May 2014 18:13, <lucio@proxima.alt.za> wrote:

> Curiously, I'm pretty certain that it was the issue of an fd that
> remained open (something to do with caching the /dev/time fd, if I
> remember right) that caused some tests to fall apart, probably because
> a test for leaking fds actually needed to cache the time of day for
> time out purposes.
>

That was entirely the result of a botched attempt to optimise something.
Remove that botched optimisation, and that problem goes away.

[-- Attachment #2: Type: text/html, Size: 888 bytes --]

Re: [9fans] syscall 53

[erik quanstrom]

> On 19 May 2014 18:13, <lucio@proxima.alt.za> wrote:
>
> > Curiously, I'm pretty certain that it was the issue of an fd that
> > remained open (something to do with caching the /dev/time fd, if I
> > remember right) that caused some tests to fall apart, probably because
> > a test for leaking fds actually needed to cache the time of day for
> > time out purposes.
> >
>
> That was entirely the result of a botched attempt to optimise something.
> Remove that botched optimisation, and that problem goes away.

+1, or rather, exactly.

- erik

Re: [9fans] syscall 53

[Aram Hăvărneanu]

There are two separate issues here. First, there's the issue of
whether 9front and 9atom should incorporate the change.

For purely egoistic reasons, I'd like that, regardless of the
technical merits of the change. I regulary test labs software on
9front. It would be a pity if I couldn't do that anymore. I have
scripts that set up binds such as to create an environment that
matches both 9atom and the labs distribution. I use these scripts
to collaborate with other people on various projects where my
collaborators and myself use different distributions. This is not
just a thought experiment; this is happening right now, so it affects
me personally.

Simply put, this change is creating (significant) more work for me.
Now, this is not a good enough reason to import this change, but
perhaps if other people are affected in the same way it might be.

Second, there's the issue of the merit of this new system call.
Without more context I believe it's a mistake.

I know that Ron Minnich added a nanotime system call to NxM (or was
it nix?). I don't know why he did that. I heard rumors about
/dev/bintime adding too much jitter, or being too imprecise. The
context of these complains was the Go port to plan9/amd64.

I don't know how much jitter /dev/bintime adds, but I don't think
it matters anyway. If you are latency-sensitive you probably also
want a monotonic clock with relative timing, so /dev/bintime is not
the right tool.

In any case, Go doesn't require an ultra-precise clock and right
now doesn't require so many time calls compared to when the port
was started. The context of the complaints were in some Go *tests*.
The *tests* were written with Linux in mind where gettimeofday is
a fake system call that uses the vdso(7) mappings. The *tests* were
wrong, at least for non-Linux systems.

There was another complaint about /dev/bintime. Some people claimed
that using it leaked file descriptors in multithreaded programs. I
don't understand why this problem can't be solved by opening it
close-on-exec. In fact, this problem doesn't exist in the port of
Go to Plan 9 anymore (although the fix was different)...

In short, I can't justify this new system call. I'm happy to be
educated, however.

-- 
Aram Hăvărneanu

Re: [9fans] syscall 53

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On 19 May 2014 20:15, Aram Hăvărneanu <aram.h@mgk.ro> wrote:

> Some people claimed
> that using it leaked file descriptors in multithreaded programs. I
> don't understand why this problem can't be solved by opening it
> close-on-exec.
>

The optimisation was a static int file descriptor.
That was troublesome in the context of
processes that shared memory but not file descriptor groups,
or somehow rearranged descriptors and then called time/nsec again.
A fork and syslog might be enough.
The improved version cached file descriptors per-process,
but using a table with an entry per-process (but only up to 64,
when it reverted to earlier behaviour) and using a process ID in _tos.
Since a library function has no idea what's happening in rfork(), or
In case file descriptors had been closed meanwhile, it would retry once(!).
In some cases, that might lead it to read from a file descriptor that was
open,
but not on the time, causing much confusion.
Further elaboration of the mechanism had been proposed to deal with that.
I prefer one or other Gordian Knot technique, myself.

[-- Attachment #2: Type: text/html, Size: 1853 bytes --]

Re: [9fans] syscall 53

[erik quanstrom]

> There was another complaint about /dev/bintime. Some people claimed
> that using it leaked file descriptors in multithreaded programs. I
> don't understand why this problem can't be solved by opening it
> close-on-exec. In fact, this problem doesn't exist in the port of
> Go to Plan 9 anymore (although the fix was different)...

close-on-exec doesn't solve any interesting issues.  close on fork might,
but we don't have that.

if two threads share memory, but do not share file descriptor tables, or
if you overflow the file descriptor array, you're really cooked.  things go
seriously wrong in a hurry.

there are other cases that are really terrible, but this is the most glaring one.

- erik

Re: [9fans] syscall 53

[cinap_lenrek]

added the syscall for binary compatibility with sources to 9front kernel.
nsec() remains a library function that reads /dev/bintime. removed the
filedescriptor caching from nsec() like in the 9atom version.

--
cianp

Re: [9fans] syscall 53

[lucio]

> i'd put a vote into restoring il to the standard kernels.  there's no
> downside.

My vote, too.

++L

Re: [9fans] syscall 53

[erik quanstrom]

On Sat May 17 15:16:03 EDT 2014, puta2001-goo@yahoo.com wrote:
>
> p.s.  Caps lock is not working.  Also copying in 9fat directory shifts
> file time current time+3 hours, even +6 hours if renaming (mv).  My
> timezone is +3 GMT.  Its native plan9 on ibm t42.

the standard plan 9 key map maps caps lock -> ctl.  you can change
the mapping with this script

	#!/bin/rc
	kval=0xf862
	if(~ $1 on)
		kval=0xf864
	>/dev/kbmap echo 0 0x3a $kval

if you have a caps lock led, it won't work with sources.  i added support
to 9atom though for both usb and ps/2 keyboards.

the other bit is all fat's fault by keeping times relative to local time not gmt.

- erik