[9fans] NIX this morning

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

The new default branch is 9front. https://github.com/rminnich/9front

I just pushed a commit that:
1. has the execac command use sysr1 for now
2. drops bootrc into a shell before root is mounted so you can poke around
and run execac
3. adds ratrace, execac, and date

When you build nix, look in systab.h, replace
[SYSR1] sysr1,
with
[SYSR1] sysexecac

it's just easier to coopt sysr1 for now

when you boot, make sure you have at least 2 cores; when it drops to a
shell, try this
execac -c 1 /bin/date
That would run /bin/date on core 1.

In a perfect world.

well ...
qunlock called with qlock not held, from 0xffffffff8021e5c2
qunlock called with qlock not held, from 0xffffffff8021e5c2

oops.

If you know how to debug qemu with gdb, well, here's a place to start.
Or just look at what's at that PC in the kernel, and see what it might be

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-Mc1c1fcd703a83579c25cc727
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1977 bytes --]

[9fans] Re: NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 1663 bytes --]

ok that's fixed and:

% ratrace -c execac -c 1 /bin/date
98 execac Running 204326 0x1prepage: base 0x7ffffefff000 top 0x7ffffffff000
prepage: base 0x200000 top 0x400000
prepage: base 0x400000 top 0x406000
prepage: base 0x406000 top 0x406000
/bin/date: timezone: file does not exist: '/env/timezone'
 = process exited
% k

I *think* that means it worked.

First execac I've run in ... well ... a long time.

On Sat, Feb 8, 2025 at 8:17 AM ron minnich <rminnich@gmail.com> wrote:

> The new default branch is 9front. https://github.com/rminnich/9front
>
> I just pushed a commit that:
> 1. has the execac command use sysr1 for now
> 2. drops bootrc into a shell before root is mounted so you can poke around
> and run execac
> 3. adds ratrace, execac, and date
>
> When you build nix, look in systab.h, replace
> [SYSR1] sysr1,
> with
> [SYSR1] sysexecac
>
> it's just easier to coopt sysr1 for now
>
> when you boot, make sure you have at least 2 cores; when it drops to a
> shell, try this
> execac -c 1 /bin/date
> That would run /bin/date on core 1.
>
> In a perfect world.
>
> well ...
> qunlock called with qlock not held, from 0xffffffff8021e5c2
> qunlock called with qlock not held, from 0xffffffff8021e5c2
>
> oops.
>
> If you know how to debug qemu with gdb, well, here's a place to start.
> Or just look at what's at that PC in the kernel, and see what it might be
>
>
>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-M18a4e0c8147ffe90d6d02f40
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2969 bytes --]

Re: [9fans] Re: NIX this morning

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 2195 bytes --]

Nice!  Congratulations!



On Sat, Feb 8, 2025, 9:13 a.m. ron minnich <rminnich@gmail.com> wrote:

> ok that's fixed and:
>
> % ratrace -c execac -c 1 /bin/date
> 98 execac Running 204326 0x1prepage: base 0x7ffffefff000 top 0x7ffffffff000
> prepage: base 0x200000 top 0x400000
> prepage: base 0x400000 top 0x406000
> prepage: base 0x406000 top 0x406000
> /bin/date: timezone: file does not exist: '/env/timezone'
>  = process exited
> % k
>
> I *think* that means it worked.
>
> First execac I've run in ... well ... a long time.
>
> On Sat, Feb 8, 2025 at 8:17 AM ron minnich <rminnich@gmail.com> wrote:
>
>> The new default branch is 9front. https://github.com/rminnich/9front
>>
>> I just pushed a commit that:
>> 1. has the execac command use sysr1 for now
>> 2. drops bootrc into a shell before root is mounted so you can poke
>> around and run execac
>> 3. adds ratrace, execac, and date
>>
>> When you build nix, look in systab.h, replace
>> [SYSR1] sysr1,
>> with
>> [SYSR1] sysexecac
>>
>> it's just easier to coopt sysr1 for now
>>
>> when you boot, make sure you have at least 2 cores; when it drops to a
>> shell, try this
>> execac -c 1 /bin/date
>> That would run /bin/date on core 1.
>>
>> In a perfect world.
>>
>> well ...
>> qunlock called with qlock not held, from 0xffffffff8021e5c2
>> qunlock called with qlock not held, from 0xffffffff8021e5c2
>>
>> oops.
>>
>> If you know how to debug qemu with gdb, well, here's a place to start.
>> Or just look at what's at that PC in the kernel, and see what it might be
>>
>>
>>
>> *9fans <https://9fans.topicbox.com/latest>* / 9fans / see discussions
> <https://9fans.topicbox.com/groups/9fans> + participants
> <https://9fans.topicbox.com/groups/9fans/members> + delivery options
> <https://9fans.topicbox.com/groups/9fans/subscription> Permalink
> <https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-M18a4e0c8147ffe90d6d02f40>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-Mab87d823d9dfd3257de3273b
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 3446 bytes --]

Re: [9fans] Re: NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 4131 bytes --]

NIX this evening. Test on 4 core laptop.

fshalt -r does not end well on my laptop; it's the usual issue with drivers
not dealing that well with a warm reset, and hardware being hard.

So: I modified bootrc to drop me into rc before it starts mounting file
systems and such. I added execac to the image. fshalt -r 9pc64 and I can
run execac as a test.

And it works on my T420 :-)

So, next step: try the fixed time quantum (FTQ) benchmark:
github.com/rminnich/ftq and compare noise results from a TC and an AC.

ftq measures interference that can cause scaling issues in supercomputers,
more here:
https://www.researchgate.net/publication/4123211_Analysis_of_microbenchmarks_for_performance_tuning_of_clusters

We developed FTQ at LANL to measure noise. Some results here:
https://docs.google.com/presentation/d/1_BsQaO_0hdz8RSW1PAzMH6rVDQmdkwupfTmr-x5EXq4/edit?usp=sharing

In 2003 we got far better FTQ results on Plan 9 than on linux, which is why
in 2003 I started the Plan 9 for supercomputing project. We got really good
results on Blue Gene in 2007.

in 2011, I got the best results ever measured, and still the best on just
about everything I've ever seen in 25 years, using NIX on an AC. These
would have been great results on machine in single user mode, but they were
achieved on a fully booted machine running rio -- since the AC was left
alone to do its thing.

Here's hoping that still holds; I'll try it tomorrow. Will be pretty nice
to see if it works out.

But ... NIX is ready enough for you to try. I have no idea if it's any use
for anyone, but thanks to Paul for getting this port off the ground, and
Thierry, who kicked it into gear by asking the right question at the right
time.

github.com/rminnich/9front, 9front branch

On Sat, Feb 8, 2025 at 9:16 AM Paul Lalonde <paul.a.lalonde@gmail.com>
wrote:

> Nice!  Congratulations!
>
>
>
> On Sat, Feb 8, 2025, 9:13 a.m. ron minnich <rminnich@gmail.com> wrote:
>
>> ok that's fixed and:
>>
>> % ratrace -c execac -c 1 /bin/date
>> 98 execac Running 204326 0x1prepage: base 0x7ffffefff000 top
>> 0x7ffffffff000
>> prepage: base 0x200000 top 0x400000
>> prepage: base 0x400000 top 0x406000
>> prepage: base 0x406000 top 0x406000
>> /bin/date: timezone: file does not exist: '/env/timezone'
>>  = process exited
>> % k
>>
>> I *think* that means it worked.
>>
>> First execac I've run in ... well ... a long time.
>>
>> On Sat, Feb 8, 2025 at 8:17 AM ron minnich <rminnich@gmail.com> wrote:
>>
>>> The new default branch is 9front. https://github.com/rminnich/9front
>>>
>>> I just pushed a commit that:
>>> 1. has the execac command use sysr1 for now
>>> 2. drops bootrc into a shell before root is mounted so you can poke
>>> around and run execac
>>> 3. adds ratrace, execac, and date
>>>
>>> When you build nix, look in systab.h, replace
>>> [SYSR1] sysr1,
>>> with
>>> [SYSR1] sysexecac
>>>
>>> it's just easier to coopt sysr1 for now
>>>
>>> when you boot, make sure you have at least 2 cores; when it drops to a
>>> shell, try this
>>> execac -c 1 /bin/date
>>> That would run /bin/date on core 1.
>>>
>>> In a perfect world.
>>>
>>> well ...
>>> qunlock called with qlock not held, from 0xffffffff8021e5c2
>>> qunlock called with qlock not held, from 0xffffffff8021e5c2
>>>
>>> oops.
>>>
>>> If you know how to debug qemu with gdb, well, here's a place to start.
>>> Or just look at what's at that PC in the kernel, and see what it might be
>>>
>>>
>>>
>>> *9fans <https://9fans.topicbox.com/latest>* / 9fans / see discussions
> <https://9fans.topicbox.com/groups/9fans> + participants
> <https://9fans.topicbox.com/groups/9fans/members> + delivery options
> <https://9fans.topicbox.com/groups/9fans/subscription> Permalink
> <https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-Mab87d823d9dfd3257de3273b>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-Mc2f9940ca91aa8c661eb2591
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 6367 bytes --]

Re: [9fans] Re: NIX this morning

[hiro]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

very interesting. is this interference metric normalized taking the
efficiency of the system into account?
e.g. if on one platform 2x as much work is done (for example because the
core frequency is doubled), i wouldn't be surprised if absolute
interference doubles, but in relation to the work being achieved i would
assume the interference metric could be much less then double.
another example: if there's a very low scheduling granularity, and very
short jobs, interference noise floor might be higher, bec. the workload
itself has much less impact in comparison to the idle periods.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T5032253608c13e6b-Maf9b9533784524b9360d8158
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1314 bytes --]

Re: [9fans] Re: NIX this morning

[David Leimbach via 9fans]

[-- Attachment #1: Type: text/html, Size: 2718 bytes --]

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

Paul and I have been trying things out and, as of this morning, we can
successfully run /bin/date on an AC.

Some things still don't work. Just about anything involving a trap won't
end well; the trap in this case is an fpu trap, because something is not
being set up right. It does not happen in all cases, which is what makes it
so much fun.

As always, I pushed all the latest bits to 9front branch.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T3c9993cc6a455c9f-Mad6a4a2d612c578d8a416ce5
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1162 bytes --]

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

OK, I'm on what may be the last problem.

I was having trouble writing a reproducer, until I wrote one by accident,
by fat-fingering %f instead of %d in a print. That caused the problem.

Turns out that I'm still not quite handling the fpu right on the AC, which
will require some more study of what we did in NIX and why.

But programs that don't use float at all work fine.

I think about 1/2 my bug discoveries come from mistakes like this :-)

but ... I have run some commands to completion on an AC at this point, so
it's close.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/Te5fdc53a0157b4a9-M740e1cdb0a3ebfbb9e6eaa97
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1385 bytes --]

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

I've run a command to completion on an AC, with some help. The command is
date. The problem, it turns out, is that I'm somehow corrupting rsp in
the syscall code. Not sure how I pulled that off :-) but that's the issue.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/Tb56a4dce7fab3b4c-M2499942f74c2305ebced71ba
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 976 bytes --]

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 843 bytes --]

I'm now able to run system calls, i.e AC makes system call, and TC runs it.

Things are getting garbled, and at some point, things go wrong enough to
panic
the kernel.

Not quite sure what's wrong, people are invited to take a look,
particularly at pc64/nix.s

but it's closer, by the day.

Part of this was relearning what we did in 2011 ... it was pretty cool. I
can see why it took the 6 or so of us a few days to get it right :-)

I still don't totally believe I'm setting  up the stack right when I go to
reset it:
MOVQ RMACH, RSP
ADDQ $MACHSIZE, SP // hmm. Is that really right? Or too big?

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/Tb67e80a60bd01ca5-M278d594700b917b76c4030b4
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1492 bytes --]

[9fans] nix this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

very, very close, I'm running procs on an AC, system calls are messed up
somehow, but ...
getting there.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T6ec06670104f2d9f-Mff33bf911b26268d9ef481ff
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 801 bytes --]

[9fans] NIX this morning

[ron minnich]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

we're probably starting the process on an AC, the process is calling a
system call and resuming on the TC, it does the system call, and explodes
once it's back running on the AC.

Multiple little bug fixes and that's where we are. We take a page fault in
kernel mode on the AC, still trying to find that last change needed.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T75ffd266fb318010-Me40c0b87f0588b029cf3c583
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1061 bytes --]