[9fans] NIX experience

[9fans] NIX experience

[Andreas.Elding via 9fans]

Hello,

I was wondering if anyone has any experience using the NIX HPC environment? Traditionally, there's a scheduler that keeps track of the resources in the system, what nodes are busy and with which jobs, how much ram is in use and such.

I'm finding very sparse information on the NIX project, so I turn here to ask if anyone has actually used it and can share some details?

The site with the most information on it seems to be https://lsub.org/nix/  but the research papers that I have found there are not too detailed (perhaps I've only found previews?).

Any extra information would be appreciated.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Me7617ff3f90585d71e6e7779
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

Hello, I more or less started that project with a white paper early in
2011 so may be able to help. NIX was inspired by what we learned from
the Blue Gene work and other Plan 9 work sponsored by DOE FAST-OS,
which ran from 2005-2011. During those years, DOE FAST-OS sponsored
the amd64 compiler, k10 kernel, blue gene port, and NIX, to name a few
things. I was at both LANL and SNL over that time period.

A group of us spent May of 2011 at lsub getting the initial NIX system
to work. It was a very productive month :-) The group at lsub were as
good as it gets, and then we had jmk and Charles there too. Quite the
Dream Team.

What would you like to know? I also have an initial broken port to
9front if you'd like to try to bring it to life.

ron

On Thu, Dec 26, 2024 at 9:13 PM Andreas.Elding via 9fans
<9fans@9fans.net> wrote:
> 
> Hello,
> 
> I was wondering if anyone has any experience using the NIX HPC environment? Traditionally, there's a scheduler that keeps track of the resources in the system, what nodes are busy and with which jobs, how much ram is in use and such.
> 
> I'm finding very sparse information on the NIX project, so I turn here to ask if anyone has actually used it and can share some details?
> 
> The site with the most information on it seems to be https://lsub.org/nix/  but the research papers that I have found there are not too detailed (perhaps I've only found previews?).
> 
> Any extra information would be appreciated.
> 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M8ef22bb846390d0a7478c0c9
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

OK, I got curious about when NIX started to happen. Basically, in 2011
or so, we had wrapped up the Blue Gene work, the last Blue Gene
systems having been shipped, and jmk and I were thinking about what to
do; there was still DOE money left.. We decided to revive the k10 work
from 2005 or so. We had stopped the k10 work in 2006, when Fred
Johnson, DOE program manager of FAST-OS, asked the FAST-OS researchers
to start focusing on the upcoming petaflop HPC systems, which were not
going to be x86 clusters, and (so long ago!) were not going to run
Linux. So we went full circle: DOE funded Plan 9 on k10, then we
shifted gears in 2006 to Blue Gene (Power PC), then in 2011, it was
back to ... K10.

I wrote a note summarizing what jmk and I came up with and sent it out
to the lsub folks and jmk on April 21. A lot of it is not what we did
:-) but the core idea, of application cores, we did do.

So, below, the note, showing the core idea in april 2021. The "in May"
reference is lsub's kind offer to fly me out and give me a place to
stay for May 2011. The result was, for me, a chance to work with and
learn from very smart researchers! The 'we' mentioned in the note is
jmk and me. Note that the idea is very unformed. By the time I got
there Nemo had figured out the core operation of switching between AC
and TC, and Charles had convinced me that, since we're running on a
shared memory machine, we might want to take advantage of that fact.

I pushed hard on having only 2M pages, which we later continued on
Harvey. A standard HPC noise benchmark (rminnich/github.com/ftq)
showed this worked very well. Nemo came up with a very nice idea; once
the break got above 1G, just use 1G pages, b/c only 1 or 2 programs
would need it, but we'd save lots of page table pages in doing this.
It worked well.

In retrospect, it's not clear that just having 2M pages is a good
idea. 4K seems clearly too small, but 64K seems a better size all
around.

What was really incredible was just how little of the kernel we had to
change to get it to work, and just how quickly we had the basic system
(about 2 weeks). And it worked so well. We made a minor change to exec
and to rc to make it trivially easy to schedule processes on an AC.

Finally, why did something like this not ever happen? Because GPUs
came along a few years later and that's where all the parallelism in
HPC is nowadays. NIX was a nice idea, but it did not survive in the
GPU era.

"
I think we came to a good conclusion about what to do in May.

The idea is to base our work on the Plan 9 k8 port for the 9k kernel.
I would like to explore the concept of application cores. An
application core is a core that only runs user mode programs. Right
now there are lots of questions about how to do this, but application
cores are seen as a next step in manycore systems. In a system of N^2
cores,vendors are telling me that something like N of then will be
able to run a kernel, and that N^2-N will not be able to. Application
cores save power, heat, money, and die space.

The idea is that to prototype we can run a full-up Plan 9 kernel on
core 0, then have a driver (/dev/apcore) with  clone file
(/dev/apcore/clone) that a process can open to gain access to a core.
The application core process can be assembled by writing to a ctl file
to do such operations as allocating and writing memory, setting
registers, etc., then launched via write to the ctl file. The
application core process talks to the kernel via typed ipc channels
such as we have today -- it will look kind of like an ioproc but the
channels will be highly optimized like the ones in Barrelfish.

All the models we need for this mode exist in Plan 9.

Here's what's neat. You can have application cores, but we can also
have core 0 running a traditional time-shared kernel (Plan 9 in this
case). That way, if you run out of application cores, the traditional
time-shared model is there on core 0. I think this hybrid model is
going to be very powerful.

So I will be booting the 9k/k8 kernel to make sure I know how. That
way, when I get there, we can get a quick start.

thanks
"

On Thu, Dec 26, 2024 at 9:54 PM Ron Minnich <rminnich@p9f.org> wrote:
>
> Hello, I more or less started that project with a white paper early in
> 2011 so may be able to help. NIX was inspired by what we learned from
> the Blue Gene work and other Plan 9 work sponsored by DOE FAST-OS,
> which ran from 2005-2011. During those years, DOE FAST-OS sponsored
> the amd64 compiler, k10 kernel, blue gene port, and NIX, to name a few
> things. I was at both LANL and SNL over that time period.
>
> A group of us spent May of 2011 at lsub getting the initial NIX system
> to work. It was a very productive month :-) The group at lsub were as
> good as it gets, and then we had jmk and Charles there too. Quite the
> Dream Team.
>
> What would you like to know? I also have an initial broken port to
> 9front if you'd like to try to bring it to life.
>
> ron
>
> On Thu, Dec 26, 2024 at 9:13 PM Andreas.Elding via 9fans
> <9fans@9fans.net> wrote:
> > 
> > Hello,
> > 
> > I was wondering if anyone has any experience using the NIX HPC environment? Traditionally, there's a scheduler that keeps track of the resources in the system, what nodes are busy and with which jobs, how much ram is in use and such.
> > 
> > I'm finding very sparse information on the NIX project, so I turn here to ask if anyone has actually used it and can share some details?
> > 
> > The site with the most information on it seems to be https://lsub.org/nix/  but the research papers that I have found there are not too detailed (perhaps I've only found previews?).
> > 
> > Any extra information would be appreciated.
> > 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M00c88605e06db8f37099b970
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

Reading old emails is always interesting. Turns out I was in
discussions with a company building a CPU that was a very good fit to
NIX. I was trying to get that company to ship a research system to
lsub.

They were initially very agreeable but, finally, stopped talking about
Plan 9 on their system.

What stopped them? The Lucent Public License.

That license blocked a number of potential partnerships with companies
back then. I am glad Nokia helped us resolve that problem.

ron
P.S. that CPU never went very far. The GPU tsunami killed it.

On Thu, Dec 26, 2024 at 10:24 PM Ron Minnich <rminnich@p9f.org> wrote:
>
> OK, I got curious about when NIX started to happen. Basically, in 2011
> or so, we had wrapped up the Blue Gene work, the last Blue Gene
> systems having been shipped, and jmk and I were thinking about what to
> do; there was still DOE money left.. We decided to revive the k10 work
> from 2005 or so. We had stopped the k10 work in 2006, when Fred
> Johnson, DOE program manager of FAST-OS, asked the FAST-OS researchers
> to start focusing on the upcoming petaflop HPC systems, which were not
> going to be x86 clusters, and (so long ago!) were not going to run
> Linux. So we went full circle: DOE funded Plan 9 on k10, then we
> shifted gears in 2006 to Blue Gene (Power PC), then in 2011, it was
> back to ... K10.
>
> I wrote a note summarizing what jmk and I came up with and sent it out
> to the lsub folks and jmk on April 21. A lot of it is not what we did
> :-) but the core idea, of application cores, we did do.
>
> So, below, the note, showing the core idea in april 2021. The "in May"
> reference is lsub's kind offer to fly me out and give me a place to
> stay for May 2011. The result was, for me, a chance to work with and
> learn from very smart researchers! The 'we' mentioned in the note is
> jmk and me. Note that the idea is very unformed. By the time I got
> there Nemo had figured out the core operation of switching between AC
> and TC, and Charles had convinced me that, since we're running on a
> shared memory machine, we might want to take advantage of that fact.
>
> I pushed hard on having only 2M pages, which we later continued on
> Harvey. A standard HPC noise benchmark (rminnich/github.com/ftq)
> showed this worked very well. Nemo came up with a very nice idea; once
> the break got above 1G, just use 1G pages, b/c only 1 or 2 programs
> would need it, but we'd save lots of page table pages in doing this.
> It worked well.
>
> In retrospect, it's not clear that just having 2M pages is a good
> idea. 4K seems clearly too small, but 64K seems a better size all
> around.
>
> What was really incredible was just how little of the kernel we had to
> change to get it to work, and just how quickly we had the basic system
> (about 2 weeks). And it worked so well. We made a minor change to exec
> and to rc to make it trivially easy to schedule processes on an AC.
>
> Finally, why did something like this not ever happen? Because GPUs
> came along a few years later and that's where all the parallelism in
> HPC is nowadays. NIX was a nice idea, but it did not survive in the
> GPU era.
>
> "
> I think we came to a good conclusion about what to do in May.
>
> The idea is to base our work on the Plan 9 k8 port for the 9k kernel.
> I would like to explore the concept of application cores. An
> application core is a core that only runs user mode programs. Right
> now there are lots of questions about how to do this, but application
> cores are seen as a next step in manycore systems. In a system of N^2
> cores,vendors are telling me that something like N of then will be
> able to run a kernel, and that N^2-N will not be able to. Application
> cores save power, heat, money, and die space.
>
> The idea is that to prototype we can run a full-up Plan 9 kernel on
> core 0, then have a driver (/dev/apcore) with  clone file
> (/dev/apcore/clone) that a process can open to gain access to a core.
> The application core process can be assembled by writing to a ctl file
> to do such operations as allocating and writing memory, setting
> registers, etc., then launched via write to the ctl file. The
> application core process talks to the kernel via typed ipc channels
> such as we have today -- it will look kind of like an ioproc but the
> channels will be highly optimized like the ones in Barrelfish.
>
> All the models we need for this mode exist in Plan 9.
>
> Here's what's neat. You can have application cores, but we can also
> have core 0 running a traditional time-shared kernel (Plan 9 in this
> case). That way, if you run out of application cores, the traditional
> time-shared model is there on core 0. I think this hybrid model is
> going to be very powerful.
>
> So I will be booting the 9k/k8 kernel to make sure I know how. That
> way, when I get there, we can get a quick start.
>
> thanks
> "
>
> On Thu, Dec 26, 2024 at 9:54 PM Ron Minnich <rminnich@p9f.org> wrote:
> >
> > Hello, I more or less started that project with a white paper early in
> > 2011 so may be able to help. NIX was inspired by what we learned from
> > the Blue Gene work and other Plan 9 work sponsored by DOE FAST-OS,
> > which ran from 2005-2011. During those years, DOE FAST-OS sponsored
> > the amd64 compiler, k10 kernel, blue gene port, and NIX, to name a few
> > things. I was at both LANL and SNL over that time period.
> >
> > A group of us spent May of 2011 at lsub getting the initial NIX system
> > to work. It was a very productive month :-) The group at lsub were as
> > good as it gets, and then we had jmk and Charles there too. Quite the
> > Dream Team.
> >
> > What would you like to know? I also have an initial broken port to
> > 9front if you'd like to try to bring it to life.
> >
> > ron
> >
> > On Thu, Dec 26, 2024 at 9:13 PM Andreas.Elding via 9fans
> > <9fans@9fans.net> wrote:
> > > 
> > > Hello,
> > > 
> > > I was wondering if anyone has any experience using the NIX HPC environment? Traditionally, there's a scheduler that keeps track of the resources in the system, what nodes are busy and with which jobs, how much ram is in use and such.
> > > 
> > > I'm finding very sparse information on the NIX project, so I turn here to ask if anyone has actually used it and can share some details?
> > > 
> > > The site with the most information on it seems to be https://lsub.org/nix/  but the research papers that I have found there are not too detailed (perhaps I've only found previews?).
> > > 
> > > Any extra information would be appreciated.
> > > 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mba95700949bfb1f32c295b1f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
[very interesting stuff]
> 
> Finally, why did something like this not ever happen? Because GPUs
> came along a few years later and that's where all the parallelism in
> HPC is nowadays. NIX was a nice idea, but it did not survive in the
> GPU era.
> 

GPUs are actually wreaking havoc other kernels, with, in the Unix
world, X11 being in a bad shape for several reasons, one being that
GPU are not limited to graphical display---this tends to be
anecdoctical in some sense.

Can you elaborate on the GPUs paradigm break? I tend to think that
there is a main difference between "equals" sharing a same address
space via MMU, and auxiliary processors that are using another address
space. A GPU, as far as I know (this is not much), is an auxiliary
processor when the GPU is discrete, and is a specialized processor
sharing the same address space when integrated (but I guess that a
HPC have discrete GPUs with perhaps a specialized connection).

Do you know good references about:

- organizing processors depending on memory connection---I found
mainly M. J. Flynn's paper(s) about this, but nothing more
recent---and the impact on an OS design;

- IPC vs threads---from your description, it seems that your solution
was multiplying processes so IPC instead of multiplying threads---but
nonetheless the sharing of differing memories remains, and is more
easy to solve with IPC than with threads;

- Present GPU's architecture (supposing it is documented; it seems not
totally from "General-Purpose Graphics Processor Archictectures",
Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
composing hardware by connecting dedicated elements, and vectors vs
SIMT.

Thanks for sharing (what can be shared)!
-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M653db3e6d646319c4c05e9b1
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 3524 bytes --]

GPUs have been my bread and butter for 20+ years.

The best introductory source continues to be Kayvon Fatahalian and Mike
Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197

It says little about the software interface to the GPU, but does a very
good job of motivating and describing the architecture.

The in-depth resource for modern GPU architecture is the Nvidia A100 tensor
architecture paper:
https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.
It's a slog, but clearly shows how compute has changed.  Particularly, much
of the success is in turning branchy workloads with scattered memory
accesses into much more bulk-oriented data streams that match well to the
"natural" structure of the tensor cores.  The performance gains can be
astronomical. I've personally made > 1000x - yes, that's *times* not
percentages - speedups with some workloads.  There is very little compute
that's "cpu-limited" at multi-second scales that can't benefit from these
approaches, hence the death of non-GPU supercomputing.

Paul



On Fri, Dec 27, 2024 at 7:13 AM <tlaronde@kergis.com> wrote:

> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> [very interesting stuff]
> >
> > Finally, why did something like this not ever happen? Because GPUs
> > came along a few years later and that's where all the parallelism in
> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
> > GPU era.
> >
> 
> GPUs are actually wreaking havoc other kernels, with, in the Unix
> world, X11 being in a bad shape for several reasons, one being that
> GPU are not limited to graphical display---this tends to be
> anecdoctical in some sense.
> 
> Can you elaborate on the GPUs paradigm break? I tend to think that
> there is a main difference between "equals" sharing a same address
> space via MMU, and auxiliary processors that are using another address
> space. A GPU, as far as I know (this is not much), is an auxiliary
> processor when the GPU is discrete, and is a specialized processor
> sharing the same address space when integrated (but I guess that a
> HPC have discrete GPUs with perhaps a specialized connection).
> 
> Do you know good references about:
> 
> - organizing processors depending on memory connection---I found
> mainly M. J. Flynn's paper(s) about this, but nothing more
> recent---and the impact on an OS design;
> 
> - IPC vs threads---from your description, it seems that your solution
> was multiplying processes so IPC instead of multiplying threads---but
> nonetheless the sharing of differing memories remains, and is more
> easy to solve with IPC than with threads;
> 
> - Present GPU's architecture (supposing it is documented; it seems not
> totally from "General-Purpose Graphics Processor Archictectures",
> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
> composing hardware by connecting dedicated elements, and vectors vs
> SIMT.
> 
> Thanks for sharing (what can be shared)!
> --
> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>              http://www.kergis.com/
>             http://kertex.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M515bf7357d2a3e968c25260f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 5619 bytes --]

Re: [9fans] NIX experience

[Bakul Shah via 9fans]

[-- Attachment #1: Type: text/plain, Size: 4137 bytes --]

This may be of some use to non-experts:

https://enccs.github.io/gpu-programming/

> On Dec 27, 2024, at 8:32 AM, Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> 
> GPUs have been my bread and butter for 20+ years.
> 
> The best introductory source continues to be Kayvon Fatahalian and Mike Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
> 
> It says little about the software interface to the GPU, but does a very good job of motivating and describing the architecture.
> 
> The in-depth resource for modern GPU architecture is the Nvidia A100 tensor architecture paper: https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.  It's a slog, but clearly shows how compute has changed.  Particularly, much of the success is in turning branchy workloads with scattered memory accesses into much more bulk-oriented data streams that match well to the "natural" structure of the tensor cores.  The performance gains can be astronomical. I've personally made > 1000x - yes, that's *times* not percentages - speedups with some workloads.  There is very little compute that's "cpu-limited" at multi-second scales that can't benefit from these approaches, hence the death of non-GPU supercomputing.
> 
> Paul
> 
> 
> 
> On Fri, Dec 27, 2024 at 7:13 AM <tlaronde@kergis.com <mailto:tlaronde@kergis.com>> wrote:
>> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
>> [very interesting stuff]
>> > 
>> > Finally, why did something like this not ever happen? Because GPUs
>> > came along a few years later and that's where all the parallelism in
>> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
>> > GPU era.
>> > 
>> 
>> GPUs are actually wreaking havoc other kernels, with, in the Unix
>> world, X11 being in a bad shape for several reasons, one being that
>> GPU are not limited to graphical display---this tends to be
>> anecdoctical in some sense.
>> 
>> Can you elaborate on the GPUs paradigm break? I tend to think that
>> there is a main difference between "equals" sharing a same address
>> space via MMU, and auxiliary processors that are using another address
>> space. A GPU, as far as I know (this is not much), is an auxiliary
>> processor when the GPU is discrete, and is a specialized processor
>> sharing the same address space when integrated (but I guess that a
>> HPC have discrete GPUs with perhaps a specialized connection).
>> 
>> Do you know good references about:
>> 
>> - organizing processors depending on memory connection---I found
>> mainly M. J. Flynn's paper(s) about this, but nothing more
>> recent---and the impact on an OS design;
>> 
>> - IPC vs threads---from your description, it seems that your solution
>> was multiplying processes so IPC instead of multiplying threads---but
>> nonetheless the sharing of differing memories remains, and is more
>> easy to solve with IPC than with threads;
>> 
>> - Present GPU's architecture (supposing it is documented; it seems not
>> totally from "General-Purpose Graphics Processor Archictectures",
>> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
>> composing hardware by connecting dedicated elements, and vectors vs
>> SIMT.
>> 
>> Thanks for sharing (what can be shared)!
>> --
>> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>>              http://www.kergis.com/
>>             http://kertex.kergis.com/
>> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> 
> 9fans <https://9fans.topicbox.com/latest> / 9fans / see discussions <https://9fans.topicbox.com/groups/9fans> + participants <https://9fans.topicbox.com/groups/9fans/members> + delivery options <https://9fans.topicbox.com/groups/9fans/subscription>Permalink <https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M515bf7357d2a3e968c25260f>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M65cc13f1936d4f4401738ce6
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 6404 bytes --]

Re: [9fans] NIX experience

[tlaronde]

Thanks for the references! (They are hard, at least for me, to find
when one wants to understand at least a little the why...)

On Fri, Dec 27, 2024 at 08:32:31AM -0800, Paul Lalonde wrote:
> GPUs have been my bread and butter for 20+ years.
> 
> The best introductory source continues to be Kayvon Fatahalian and Mike
> Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
> 
> It says little about the software interface to the GPU, but does a very
> good job of motivating and describing the architecture.
> 
> The in-depth resource for modern GPU architecture is the Nvidia A100 tensor
> architecture paper:
> https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.
> It's a slog, but clearly shows how compute has changed.  Particularly, much
> of the success is in turning branchy workloads with scattered memory
> accesses into much more bulk-oriented data streams that match well to the
> "natural" structure of the tensor cores.  The performance gains can be
> astronomical. I've personally made > 1000x - yes, that's *times* not
> percentages - speedups with some workloads.  There is very little compute
> that's "cpu-limited" at multi-second scales that can't benefit from these
> approaches, hence the death of non-GPU supercomputing.
> 
> Paul
> 
> 
> 
> On Fri, Dec 27, 2024 at 7:13?AM <tlaronde@kergis.com> wrote:
> 
> > On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> > [very interesting stuff]
> > >
> > > Finally, why did something like this not ever happen? Because GPUs
> > > came along a few years later and that's where all the parallelism in
> > > HPC is nowadays. NIX was a nice idea, but it did not survive in the
> > > GPU era.
> > >
> > 
> > GPUs are actually wreaking havoc other kernels, with, in the Unix
> > world, X11 being in a bad shape for several reasons, one being that
> > GPU are not limited to graphical display---this tends to be
> > anecdoctical in some sense.
> > 
> > Can you elaborate on the GPUs paradigm break? I tend to think that
> > there is a main difference between "equals" sharing a same address
> > space via MMU, and auxiliary processors that are using another address
> > space. A GPU, as far as I know (this is not much), is an auxiliary
> > processor when the GPU is discrete, and is a specialized processor
> > sharing the same address space when integrated (but I guess that a
> > HPC have discrete GPUs with perhaps a specialized connection).
> > 
> > Do you know good references about:
> > 
> > - organizing processors depending on memory connection---I found
> > mainly M. J. Flynn's paper(s) about this, but nothing more
> > recent---and the impact on an OS design;
> > 
> > - IPC vs threads---from your description, it seems that your solution
> > was multiplying processes so IPC instead of multiplying threads---but
> > nonetheless the sharing of differing memories remains, and is more
> > easy to solve with IPC than with threads;
> > 
> > - Present GPU's architecture (supposing it is documented; it seems not
> > totally from "General-Purpose Graphics Processor Archictectures",
> > Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
> > composing hardware by connecting dedicated elements, and vectors vs
> > SIMT.
> > 
> > Thanks for sharing (what can be shared)!
> > --
> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >              http://www.kergis.com/
> >             http://kertex.kergis.com/
> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M3254611df71479d1a0c6ec0c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Fri, Dec 27, 2024 at 08:56:32AM -0800, Bakul Shah via 9fans wrote:
> This may be of some use to non-experts:
> 
> https://enccs.github.io/gpu-programming/
> 

I will hence start by this before diving in the references Paul
Lalonde has given. Thanks!

> > On Dec 27, 2024, at 8:32?AM, Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> > 
> > GPUs have been my bread and butter for 20+ years.
> > 
> > The best introductory source continues to be Kayvon Fatahalian and Mike Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
> > 
> > It says little about the software interface to the GPU, but does a very good job of motivating and describing the architecture.
> > 
> > The in-depth resource for modern GPU architecture is the Nvidia A100 tensor architecture paper: https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.  It's a slog, but clearly shows how compute has changed.  Particularly, much of the success is in turning branchy workloads with scattered memory accesses into much more bulk-oriented data streams that match well to the "natural" structure of the tensor cores.  The performance gains can be astronomical. I've personally made > 1000x - yes, that's *times* not percentages - speedups with some workloads.  There is very little compute that's "cpu-limited" at multi-second scales that can't benefit from these approaches, hence the death of non-GPU supercomputing.
> > 
> > Paul
> > 
> > 
> > 
> > On Fri, Dec 27, 2024 at 7:13?AM <tlaronde@kergis.com <mailto:tlaronde@kergis.com>> wrote:
> >> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> >> [very interesting stuff]
> >> > 
> >> > Finally, why did something like this not ever happen? Because GPUs
> >> > came along a few years later and that's where all the parallelism in
> >> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
> >> > GPU era.
> >> > 
> >> 
> >> GPUs are actually wreaking havoc other kernels, with, in the Unix
> >> world, X11 being in a bad shape for several reasons, one being that
> >> GPU are not limited to graphical display---this tends to be
> >> anecdoctical in some sense.
> >> 
> >> Can you elaborate on the GPUs paradigm break? I tend to think that
> >> there is a main difference between "equals" sharing a same address
> >> space via MMU, and auxiliary processors that are using another address
> >> space. A GPU, as far as I know (this is not much), is an auxiliary
> >> processor when the GPU is discrete, and is a specialized processor
> >> sharing the same address space when integrated (but I guess that a
> >> HPC have discrete GPUs with perhaps a specialized connection).
> >> 
> >> Do you know good references about:
> >> 
> >> - organizing processors depending on memory connection---I found
> >> mainly M. J. Flynn's paper(s) about this, but nothing more
> >> recent---and the impact on an OS design;
> >> 
> >> - IPC vs threads---from your description, it seems that your solution
> >> was multiplying processes so IPC instead of multiplying threads---but
> >> nonetheless the sharing of differing memories remains, and is more
> >> easy to solve with IPC than with threads;
> >> 
> >> - Present GPU's architecture (supposing it is documented; it seems not
> >> totally from "General-Purpose Graphics Processor Archictectures",
> >> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
> >> composing hardware by connecting dedicated elements, and vectors vs
> >> SIMT.
> >> 
> >> Thanks for sharing (what can be shared)!
> >> --
> >> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >>              http://www.kergis.com/
> >>             http://kertex.kergis.com/
> >> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> > 
> > 9fans <https://9fans.topicbox.com/latest> / 9fans / see discussions <https://9fans.topicbox.com/groups/9fans> + participants <https://9fans.topicbox.com/groups/9fans/members> + delivery options <https://9fans.topicbox.com/groups/9fans/subscription>Permalink <https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M515bf7357d2a3e968c25260f>

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M719687d43692a16531bb4306
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Kurt H Maier via 9fans]

On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> We had stopped the k10 work in 2006, when Fred
> Johnson, DOE program manager of FAST-OS, asked the FAST-OS researchers
> to start focusing on the upcoming petaflop HPC systems, which were not
> going to be x86 clusters, and (so long ago!) were not going to run
> Linux. 

Which architecture and OS did they wind up with?  I was part of the team
that went on to administer the Coral systems, which were linux on POWER
9+.  Even the early-stage bringup loaders were linux systems.  I
remember they had to ship a couple x86 systems anyway, because Mellanox
wouldn't make a POWER build of UFM.

> Finally, why did something like this not ever happen? Because GPUs
> came along a few years later and that's where all the parallelism in
> HPC is nowadays. NIX was a nice idea, but it did not survive in the
> GPU era.

I always thought NIX would have been a good fit for Xeon Phi MICs, since
part of the bringup involved shipping an entire linux system to the card
for booting anyway.  Sadly, Intel (as usual) gave up on the architecture
about ten minutes after release.  We still have some sitting in a lab
but I have a strict No Plan 9 At Work policy.  The cards are cheap on
ebay if anyone wants to investigate further.

khm

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M20294e9b8dea259d934e1005
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[sirjofri]

27.12.2024 17:33:13 Paul Lalonde <paul.a.lalonde@gmail.com>:
> GPUs have been my bread and butter for 20+ years.

Nice to have another GPU enthusiast in this community. I'm pretty sure you know like 100x more than me though :)

I've been a game developer for >5 years and I'm always surprised by how much GPUs can do if used correctly. It's just incredible.

Can't wait to see what will happen in the future to Plan 9 and GPUs in combination!

sirjofri

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M5e95f70650bf0f067c3b3897
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 1799 bytes --]

Xeon Phi was the last remnant of the first GPU architecture I worked on.
It was evolved from Larrabee, meant to run DX11 graphics workloads.
The first Phi was effectively the Larrabee chip but with the texture
sampling hardware fused off.
The remnants of that work are now living on in the AVX512 instruction set.
The principal problem with Larrabee was that the ring bus connecting some
60+ ring stops was *so wide* (512 bits bidirectional = 1024 bits!) that it
consumed too much power.  It was more flexible than it needed to be
compared to a GPU memory controller/crossbar, and its power consumption
couldn't be reduced sufficiently to make it a useful architecture for the
mobile market.  Instead, graphics continued on the older Intel embedded
graphics path.
Frankly, I'm amazed that Intel followed through with 2 more revisions of
Xeon Phi before canning it.  In all honestly, the core choice for the CPU
(P54C) was nice for micro-optimized predictability of instruction execution
times and reasoning about hazards, but Intel's architectural mastery of
micro-op recompilation and out-of-order pipelines really trumps what the
compilers were doing once it was removed from the graphics space.

On Fri, Dec 27, 2024 at 1:12 PM Kurt H Maier via 9fans <9fans@9fans.net>
wrote:

> I always thought NIX would have been a good fit for Xeon Phi MICs, since
> part of the bringup involved shipping an entire linux system to the card
> for booting anyway.  Sadly, Intel (as usual) gave up on the architecture
> about ten minutes after release.
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Md2d5c0b29b8a6dfa47a3798f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2670 bytes --]

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]

On Fri, Dec 27, 2024 at 1:25 PM sirjofri <sirjofri+ml-9fans@sirjofri.de>
wrote:

> I've been a game developer for >5 years and I'm always surprised by how
> much GPUs can do if used correctly. It's just incredible.
>
Yes, it still doesn't cease to amaze me.  The compute density is just
astounding.


> Can't wait to see what will happen in the future to Plan 9 and GPUs in
> combination!
>

I wish I had better thoughts in this space.  The naive approach says "give
your data regions nice (file)names" but the file abstraction falls apart
completely once you start managing the synchronization primitives.
Add that the drivers are the most complex OS software ever built and it
becomes very difficult to do more than to expose some sort of RPC to a
different computer running a native (non-plan9) workload on the GPU.

That said, now that NVDA has moved a bunch of their "resource manager"
(read, OS) to the GPU itself and simplified the linux DRM module, the
driver layer has simplified significantly.  I'm not sure I have anywhere
near the bandwidth it would take to manage a plan9 port of even the
simplest interfaces, however.

Paul

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M89fcff19fd181cf3b97e35cc
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2448 bytes --]

Re: [9fans] NIX experience

[Kurt H Maier via 9fans]

On Fri, Dec 27, 2024 at 01:24:40PM -0800, Paul Lalonde wrote:
> The remnants of that work are now living on in the AVX512 instruction set.
> The principal problem with Larrabee was that the ring bus connecting some
> 60+ ring stops was *so wide* (512 bits bidirectional = 1024 bits!) that it
> consumed too much power.  It was more flexible than it needed to be
> compared to a GPU memory controller/crossbar, and its power consumption
> couldn't be reduced sufficiently to make it a useful architecture for the
> mobile market.  Instead, graphics continued on the older Intel embedded
> graphics path.

Is the power consumption the reason the cores downclock when you start
sending AVX512 instructions?  By far the most useful results of our
yaers-long Phi experiment was being able to test tons of codes to see
which ones benefit from AVX512 and which ones wind up being penalized so
badly by clock reduction that they weren't worth porting.  It set us up
to be able to deploy efficiently in the followon generations of x86_64.
We never came to a conclusion whether it was power consumption or heat
that required the slowdown.  I guess it could be both.

> Frankly, I'm amazed that Intel followed through with 2 more revisions of
> Xeon Phi before canning it. 

So were we.  The 5110 chips were plagued with reliability problems, and
the software stack was ... tough to get support for.  The host-cpu
Knights Landing systems were much more pleasant to work with, but we
hadn't finished benchmarking them by the time Intil pulled the plug.

khm

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M3198d5ce755d47ebe87068a2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 1290 bytes --]

Not directly.  The AVX512 instructions include some significant
permute/shuffle/mask hardware, available on pretty much all instructions.
These in turn lead to very long capacitance chains (ie, transistors in
series that have to stabilize each clock) and so constrain how fast the
clock can run.  For Larrabee that wasn't a big deal as the clock was
relatively slow, but modern 4-5GHz clocks do not like structures this
long.  Another choice is to split the registers into two banks to run on
alternate clocks and then settle the permutations in a post-process using
register renaming to avoid hazards.  I don't know if any have been
implemented this way as that would be after my time.
It looks a lot like the old CISC vs RISC arguments, only now about
shuffles.  The classical GPU architecture won with what's effectively the
"RISC" version.

Paul

On Fri, Dec 27, 2024 at 1:41 PM Kurt H Maier via 9fans <9fans@9fans.net>
wrote:

> Is the power consumption the reason the cores downclock when you start
> sending AVX512 instructions?
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M6fde8bb81da995615fa76603
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2168 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

On Fri, Dec 27, 2024 at 1:12 PM Kurt H Maier via 9fans <9fans@9fans.net> wrote:

> Which architecture and OS did they wind up with?  I was part of the team
> that went on to administer the Coral systems, which were linux on POWER
> 9+.  Even the early-stage bringup loaders were linux systems.  I
> remember they had to ship a couple x86 systems anyway, because Mellanox
> wouldn't make a POWER build of UFM.

Everything in HPC is a linux now, at least all the top 500 systems ...
as for architecture, it's interesting to see the Top 1 machine
moving from Power to whatever tianhe was to ARM to x86 again ... and
my vague memory is that the early riken were sparc before riken moved
to ARM?
Somebody want to confirm or deny this?

Once HPC was able to fix on Linux, it unlocked the door to using
different architectures.. That was kind of nice. I was sorry to see
that Plan 9 did not work out, but at least Linux did work out,
especially considering the two-year period in the late 90s where DoD
wanted us all to run Windows/NT. Not kidding.

> I always thought NIX would have been a good fit for Xeon Phi MICs, since
> part of the bringup involved shipping an entire linux system to the card
> for booting anyway.

I agree with you. Some part  of the NIX ideas were inspired, if that's
the word, from the Intel ideas of the 2008 or so period.

By the way, thanks also to Gorka, for arranging my time at lsub in
2011; and Enrique, for a lot of nice work getting NIX going.

It was a great month with great people.

I did just notice this,
https://code.google.com/archive/p/nix-os/wikis/GettingStarted.wiki,
which would let you build nix and try it out.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M2fcac096d866e068dddfee56
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 3168 bytes --]

when the plan 9 on blue gene started, and for some time later, there was a
specialised kernel (the "compute kernel"?) for BG,
which was a bit like DOS, so we felt we had a contribution to make, with an
OS designed for an age of
distribution. later it turned out that SC/HPC like everything else just
imports a ton of 3rd party code and languages (eg, Python) and just accepts
burning power and money to do that.
linux might insist on screwing up your latency by running the lpd on a BG
HPC system but at least it supported systemd.

fscfs began as a quick hack in response to Ron's observing that a
presentation at a conference said some python code starting up on a large BG
cluster took many hours, much of it spent on every compute server asking IO
servers whether every combination
of .py, .pyc, existed for every version for every naming convention, on
*every* processor, something like that.
whack a file server.
in our system, we could have fscfs running on appropriate nodes on the way
to the file server to remember bad ideas
and also cache stuff shared by the cpu servers.


On Sat, 28 Dec 2024 at 00:08, Ron Minnich <rminnich@p9f.org> wrote:

> On Fri, Dec 27, 2024 at 1:12 PM Kurt H Maier via 9fans <9fans@9fans.net>
> wrote:
>
> > Which architecture and OS did they wind up with?  I was part of the team
> > that went on to administer the Coral systems, which were linux on POWER
> > 9+.  Even the early-stage bringup loaders were linux systems.  I
> > remember they had to ship a couple x86 systems anyway, because Mellanox
> > wouldn't make a POWER build of UFM.
>
> Everything in HPC is a linux now, at least all the top 500 systems ...
> as for architecture, it's interesting to see the Top 1 machine
> moving from Power to whatever tianhe was to ARM to x86 again ... and
> my vague memory is that the early riken were sparc before riken moved
> to ARM?
> Somebody want to confirm or deny this?
>
> Once HPC was able to fix on Linux, it unlocked the door to using
> different architectures.. That was kind of nice. I was sorry to see
> that Plan 9 did not work out, but at least Linux did work out,
> especially considering the two-year period in the late 90s where DoD
> wanted us all to run Windows/NT. Not kidding.
>
> > I always thought NIX would have been a good fit for Xeon Phi MICs, since
> > part of the bringup involved shipping an entire linux system to the card
> > for booting anyway.
> 
> I agree with you. Some part  of the NIX ideas were inspired, if that's
> the word, from the Intel ideas of the 2008 or so period.
> 
> By the way, thanks also to Gorka, for arranging my time at lsub in
> 2011; and Enrique, for a lot of nice work getting NIX going.
> 
> It was a great month with great people.
> 
> I did just notice this,
> https://code.google.com/archive/p/nix-os/wikis/GettingStarted.wiki,
> which would let you build nix and try it out.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M97bdbbcfae739f7168dc943a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 4885 bytes --]

Re: [9fans] NIX experience

[Anthony Sorace]

I've thought for a while now that NIX might still have interesting things to say in the middle of the space, even if the HPC origins didn't work out. Probably most of us are walking around with systems with asymmetrical cores ("performance" vs. "efficiency") in our pockets right now; it seems like there's lots of space to explore *how* differently to manage these cores (as opposed to just spinning them up or not when needed but treating them as "regular").

I think it's a good idea. But...

> On Dec 27, 2024, at 08:32, Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> 
> There is very little compute that's "cpu-limited" at multi-second scales that can't benefit from these approaches, hence the death of non-GPU supercomputing.

This is really good framing... even if it's bad for my idea. 🤣 "Compute that's "cpu-limited" at multi-second scales" really cuts out most applications at modern scale. Plenty of things like pro workflows, but the higher up you move there, the more likely you're pushing to GPUs anyway. I think the window isn't 0, but it's shrunk quite a bit.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M0d51cc7d067a839af59e6de5
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

all that said, my offer stands: I'd love to help anyone interested to
bring nix back to life. I'd most prefer to do so on 9front, but I'm
not picky.

ron

On Fri, Dec 27, 2024 at 8:18 PM Anthony Sorace <a@9srv.net> wrote:
>
> I've thought for a while now that NIX might still have interesting things to say in the middle of the space, even if the HPC origins didn't work out. Probably most of us are walking around with systems with asymmetrical cores ("performance" vs. "efficiency") in our pockets right now; it seems like there's lots of space to explore *how* differently to manage these cores (as opposed to just spinning them up or not when needed but treating them as "regular").
>
> I think it's a good idea. But...
>
> > On Dec 27, 2024, at 08:32, Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> >
> > There is very little compute that's "cpu-limited" at multi-second scales that can't benefit from these approaches, hence the death of non-GPU supercomputing.
> 
> This is really good framing... even if it's bad for my idea. 🤣 "Compute that's "cpu-limited" at multi-second scales" really cuts out most applications at modern scale. Plenty of things like pro workflows, but the higher up you move there, the more likely you're pushing to GPUs anyway. I think the window isn't 0, but it's shrunk quite a bit.
> 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M7fefe4d849abf2940046f315
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Cyber Fonic]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

Whilst there are many HPC workloads that are well supported by GPGPUs, we
also have multi-core systems such as Ampere One and AMD EPYC with 192 cores
(and soon even more).
I would think that some of the Blue Gene and NIX work might be relevant on
such hardware.

On Sat, 28 Dec 2024 at 15:18, Anthony Sorace <a@9srv.net> wrote:

> I've thought for a while now that NIX might still have interesting things
> to say in the middle of the space, even if the HPC origins didn't work out.
> Probably most of us are walking around with systems with asymmetrical cores
> ("performance" vs. "efficiency") in our pockets right now; it seems like
> there's lots of space to explore *how* differently to manage these cores
> (as opposed to just spinning them up or not when needed but treating them
> as "regular").
>
> I think it's a good idea. But...
>
> > On Dec 27, 2024, at 08:32, Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >
> > There is very little compute that's "cpu-limited" at multi-second scales
> that can't benefit from these approaches, hence the death of non-GPU
> supercomputing.
> 
> This is really good framing... even if it's bad for my idea. 🤣 "Compute
> that's "cpu-limited" at multi-second scales" really cuts out most
> applications at modern scale. Plenty of things like pro workflows, but the
> higher up you move there, the more likely you're pushing to GPUs anyway. I
> think the window isn't 0, but it's shrunk quite a bit.
> 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M2dd6d694eccbdb5ce47209e4
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 3128 bytes --]

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

I'll take 6912 simple cores at 1Ghz over 192 core at 5GHz any day.  So long
as I can spend the 3 months rebuilding the implementation to be cache and
memory friendly.

I love the EPYC line, and have spec'd them into DCs.  But for raw compute
it's just not competitive for HPC-like workloads.

Paul

On Fri, Dec 27, 2024 at 8:44 PM Cyber Fonic <cyberfonic@gmail.com> wrote:

> Whilst there are many HPC workloads that are well supported by GPGPUs, we
> also have multi-core systems such as Ampere One and AMD EPYC with 192 cores
> (and soon even more).
> Permalink
> <https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M2dd6d694eccbdb5ce47209e4>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M7c5247ba28dbf36e9c6a27af
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1624 bytes --]

Re: [9fans] NIX experience

[Jubal Biggs]

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]

Personally I think that there is a significant market for Nix, not in HPC,
but as a better, more distributed hypervisor. With Broadcom mismanaging
VMWare, there is a need for something better than ESXI, Hyper-V or Proxmox
for all the enterprises that aren't 100% on the cloud (which is pretty much
all of them) and for the CSPs themselves too. Something built for a system
that resembles a modern data center, not a single desktop. Managing
workloads across many machines at scale is what makes the cloud work and
will remain a healthy industry for years. I think that a Plan9 could be
used to scrape together resources from multiple, underutilized machines in
the same data center to run applications with tolerance for some latency
and drive up utilization from the current 17% or so norm. If you get above
25% utilization for an existing data center, you would be saving millions
of dollars and avoiding the need to build more at tremendous cost.

On Fri, Dec 27, 2024 at 11:44 PM Ron Minnich <rminnich@p9f.org> wrote:

> all that said, my offer stands: I'd love to help anyone interested to
> bring nix back to life. I'd most prefer to do so on 9front, but I'm
> not picky.
>
> ron
>
> On Fri, Dec 27, 2024 at 8:18 PM Anthony Sorace <a@9srv.net> wrote:
> >
> > I've thought for a while now that NIX might still have interesting
> things to say in the middle of the space, even if the HPC origins didn't
> work out. Probably most of us are walking around with systems with
> asymmetrical cores ("performance" vs. "efficiency") in our pockets right
> now; it seems like there's lots of space to explore *how* differently to
> manage these cores (as opposed to just spinning them up or not when needed
> but treating them as "regular").
> >
> > I think it's a good idea. But...
> >
> > > On Dec 27, 2024, at 08:32, Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> > >
> > > There is very little compute that's "cpu-limited" at multi-second
> scales that can't benefit from these approaches, hence the death of non-GPU
> supercomputing.
> >
> > This is really good framing... even if it's bad for my idea. 🤣 "Compute
> that's "cpu-limited" at multi-second scales" really cuts out most
> applications at modern scale. Plenty of things like pro workflows, but the
> higher up you move there, the more likely you're pushing to GPUs anyway. I
> think the window isn't 0, but it's shrunk quite a bit.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M69cb4cbf5ad0abede5c48239
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 4230 bytes --]

Re: [9fans] NIX experience

[tlaronde]

On Fri, Dec 27, 2024 at 01:31:24PM -0800, Paul Lalonde wrote:
> 
> That said, now that NVDA has moved a bunch of their "resource manager"
> (read, OS) to the GPU itself and simplified the linux DRM module, the
> driver layer has simplified significantly.  I'm not sure I have anywhere
> near the bandwidth it would take to manage a plan9 port of even the
> simplest interfaces, however.

Am I right in thinking that the taxonomy of computers should be
made considering memory and that one can consider that there is a
uniq system when several cores share the same memory (via some MMU)
and that there is a cluster of different systems, connected loosely
or tightly, when they do not share the same memory? i.e. current
computers, with one or more graphic cards, are indeed clusters,
because the GPU is in fact a distinct system (with its own OS so
to speak, and the problem for current OSes being to be able to
"speak" to the GPU system).

The question is then: why an auxiliary system on a card and why not in
fact the GPU system directly on the die, or a general CPU for the
system and the rest of the elements managed by this CPU, this CPU
being the bootable core?---integrated GPU, AFAIK, is not that. (This
is what I call the RISC-V approach, based on what I concluded from
the general presentation of the RISC-V principles.)

And to give an idea of the extent of the problem now for Unices, on
NetBSD, developers imported the whole DRMKMS stuff from Linux and this
code is equal in size to... 50%! of the rest of NetBSD (kernel + userland). And
the problem is that for something like that to work correctly, you
have to have memory management right; but memory management is a
crucial core component, and importing something tied to some alien
memory management is not a way paved with roses...

Just to say that Plan9 has a graphical interface with limitations
and shortcomings.  But the Unix state, with the X11 stack, is not
an advantage over Plan9 in this area. And things have changed so
radically that starting afresh is very probably better than having,
supplementary to the rest, to adapt and keep something that is
starting to look like a huge cemetery.

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mc8e8041e880b9957826e0aec
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

On Sat, 28 Dec 2024 at 04:44, Cyber Fonic <cyberfonic@gmail.com> wrote:
> Whilst there are many HPC workloads that are well supported by GPGPUs, we also have multi-core systems such as Ampere One and AMD EPYC with 192 cores (and soon even more).
> I would think that some of the Blue Gene and NIX work might be relevant on such hardware.

Especially since there's more stuff in it than just the execac thing
everyone knows from the main NIX paper.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mb6855434970697cc24f2229c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Frank D. Engel, Jr.]

Apple Silicon chips may be an interesting counter-example to your view 
of the architecture.  They work directly from system memory; data is not 
copied between different sets of memory or different areas in memory to 
make it available to the GPU.  Consequently the CPU and GPU work 
together much more directly without needing to waste time to shuttle 
data between them.


On 12/28/24 04:03, tlaronde@kergis.com wrote:
> On Fri, Dec 27, 2024 at 01:31:24PM -0800, Paul Lalonde wrote:
>> That said, now that NVDA has moved a bunch of their "resource manager"
>> (read, OS) to the GPU itself and simplified the linux DRM module, the
>> driver layer has simplified significantly.  I'm not sure I have anywhere
>> near the bandwidth it would take to manage a plan9 port of even the
>> simplest interfaces, however.
> Am I right in thinking that the taxonomy of computers should be
> made considering memory and that one can consider that there is a
> uniq system when several cores share the same memory (via some MMU)
> and that there is a cluster of different systems, connected loosely
> or tightly, when they do not share the same memory? i.e. current
> computers, with one or more graphic cards, are indeed clusters,
> because the GPU is in fact a distinct system (with its own OS so
> to speak, and the problem for current OSes being to be able to
> "speak" to the GPU system).
>
> The question is then: why an auxiliary system on a card and why not in
> fact the GPU system directly on the die, or a general CPU for the
> system and the rest of the elements managed by this CPU, this CPU
> being the bootable core?---integrated GPU, AFAIK, is not that. (This
> is what I call the RISC-V approach, based on what I concluded from
> the general presentation of the RISC-V principles.)
>
> And to give an idea of the extent of the problem now for Unices, on
> NetBSD, developers imported the whole DRMKMS stuff from Linux and this
> code is equal in size to... 50%! of the rest of NetBSD (kernel + userland). And
> the problem is that for something like that to work correctly, you
> have to have memory management right; but memory management is a
> crucial core component, and importing something tied to some alien
> memory management is not a way paved with roses...
>
> Just to say that Plan9 has a graphical interface with limitations
> and shortcomings.  But the Unix state, with the X11 stack, is not
> an advantage over Plan9 in this area. And things have changed so
> radically that starting afresh is very probably better than having,
> supplementary to the rest, to adapt and keep something that is
> starting to look like a huge cemetery.
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M450ebf1f351bf9d5605db04c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]

This data-shuttling is one of the things that GPU vendors have been working
on.

Most of the data the GPU needs is never touched by the CPU, except to move
it to GPU memory.  This is wasteful.
But the GPU already sits on the PCIe bus, as does the storage device.  Why
not move the data directly from storage to GPU memory?
Recent iterations of GPUs can support this.  Likewise, Nvidia's NVLink
high-throughput fabric allows loading directly to GPU memory without
touching CPU memory at the same time.

Although CPU and GPU memory are often the same storage architecture, the
memory controllers differ sufficiently to make it a performance hit to
support both CPU-like random access and GPU-like streaming memory
patterns.  UMA certainly works in mobile and in graphics workloads (witness
phones and game consoles), it's more challenging when trying to squeeze the
ultimate performance-per-watt out of HPC workloads.

It's also important not to conflate a uniform memory address space with a
uniformly implemented address space - it's possible to map a chunk of the
GPU memory to the CPU memory space and treat it like RAM from the CPU's
view, but the operations are typically strongly unbalanced with writes
costing significantly more because of the difference in memory consistency
models between the two devices.

Paul

On Sat, Dec 28, 2024 at 8:25 AM Frank D. Engel, Jr. <fde101@fjrhome.net>
wrote:

> Consequently the CPU and GPU work
> together much more directly without needing to waste time to shuttle
> data between them.
>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mf15c6498d2a1be494bc3799c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2705 bytes --]

Re: [9fans] NIX experience

[Skip Tavakkolian]

I think there is a conflict between two different types of usage that
GPU architectures need to support. One is full-speed performance,
where the resource is fully owned and utilized for a single purpose
for a large chunk of time, and the other is where the GPU is a rare
resource that needs to be shared in short intervals (e.g. ML/AI
inference mode)

If I understand correctly, NIX's approach addresses the first case (at
least for cores).  Nvidia's Multiple-Instance GPU (MIG) architecture
seems to help to handle the second type of usage. The first case
requires maximizing data transfer rate, and the second needs clever
scheduling to minimize switching (large/huge) model parameters in/out.


On Fri, Dec 27, 2024 at 8:33 AM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>
> GPUs have been my bread and butter for 20+ years.
>
> The best introductory source continues to be Kayvon Fatahalian and Mike Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
>
> It says little about the software interface to the GPU, but does a very good job of motivating and describing the architecture.
>
> The in-depth resource for modern GPU architecture is the Nvidia A100 tensor architecture paper: https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.  It's a slog, but clearly shows how compute has changed.  Particularly, much of the success is in turning branchy workloads with scattered memory accesses into much more bulk-oriented data streams that match well to the "natural" structure of the tensor cores.  The performance gains can be astronomical. I've personally made > 1000x - yes, that's *times* not percentages - speedups with some workloads.  There is very little compute that's "cpu-limited" at multi-second scales that can't benefit from these approaches, hence the death of non-GPU supercomputing.
>
> Paul
>
>
>
> On Fri, Dec 27, 2024 at 7:13 AM <tlaronde@kergis.com> wrote:
>>
>> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
>> [very interesting stuff]
>> >
>> > Finally, why did something like this not ever happen? Because GPUs
>> > came along a few years later and that's where all the parallelism in
>> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
>> > GPU era.
>> >
>> 
>> GPUs are actually wreaking havoc other kernels, with, in the Unix
>> world, X11 being in a bad shape for several reasons, one being that
>> GPU are not limited to graphical display---this tends to be
>> anecdoctical in some sense.
>> 
>> Can you elaborate on the GPUs paradigm break? I tend to think that
>> there is a main difference between "equals" sharing a same address
>> space via MMU, and auxiliary processors that are using another address
>> space. A GPU, as far as I know (this is not much), is an auxiliary
>> processor when the GPU is discrete, and is a specialized processor
>> sharing the same address space when integrated (but I guess that a
>> HPC have discrete GPUs with perhaps a specialized connection).
>> 
>> Do you know good references about:
>> 
>> - organizing processors depending on memory connection---I found
>> mainly M. J. Flynn's paper(s) about this, but nothing more
>> recent---and the impact on an OS design;
>> 
>> - IPC vs threads---from your description, it seems that your solution
>> was multiplying processes so IPC instead of multiplying threads---but
>> nonetheless the sharing of differing memories remains, and is more
>> easy to solve with IPC than with threads;
>> 
>> - Present GPU's architecture (supposing it is documented; it seems not
>> totally from "General-Purpose Graphics Processor Archictectures",
>> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
>> composing hardware by connecting dedicated elements, and vectors vs
>> SIMT.
>> 
>> Thanks for sharing (what can be shared)!
>> --
>> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>>              http://www.kergis.com/
>>             http://kertex.kergis.com/
>> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M4831fee22f0b3926b832d46f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 5670 bytes --]

MIG is interesting.  It's something we partly devised to deal with sharing
interactive workloads on the same GPU.
Time-slicing is very difficult on a GPU because the state is so large. You
wind up having to drain the workload and that means you're holding on to
resources until the longest job ends.  Utilization becomes very poor, and
even worse if your application uses a long (read, non-terminating) compute
task.
So you want some way to partition the GPU more along spatial lines - giving
each process a subset of the hardware resources.
For some workloads this scheduling is easy - you give each workload enough
front end resources, memory, and threads to donuts job.  But I think we're
still a long way off from the GPU being able to do these allocations
dynamically and efficiently.
There are some really interesting non-uniform co-scheduling constraints
that I've not seen addressed in the literature.

Paul

On Sat, Dec 28, 2024, 4:19 p.m. Skip Tavakkolian <skip.tavakkolian@gmail.com>
wrote:

> I think there is a conflict between two different types of usage that
> GPU architectures need to support. One is full-speed performance,
> where the resource is fully owned and utilized for a single purpose
> for a large chunk of time, and the other is where the GPU is a rare
> resource that needs to be shared in short intervals (e.g. ML/AI
> inference mode)
>
> If I understand correctly, NIX's approach addresses the first case (at
> least for cores).  Nvidia's Multiple-Instance GPU (MIG) architecture
> seems to help to handle the second type of usage. The first case
> requires maximizing data transfer rate, and the second needs clever
> scheduling to minimize switching (large/huge) model parameters in/out.
>
>
> On Fri, Dec 27, 2024 at 8:33 AM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >
> > GPUs have been my bread and butter for 20+ years.
> >
> > The best introductory source continues to be Kayvon Fatahalian and Mike
> Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
> >
> > It says little about the software interface to the GPU, but does a very
> good job of motivating and describing the architecture.
> >
> > The in-depth resource for modern GPU architecture is the Nvidia A100
> tensor architecture paper:
> https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.
> It's a slog, but clearly shows how compute has changed.  Particularly, much
> of the success is in turning branchy workloads with scattered memory
> accesses into much more bulk-oriented data streams that match well to the
> "natural" structure of the tensor cores.  The performance gains can be
> astronomical. I've personally made > 1000x - yes, that's *times* not
> percentages - speedups with some workloads.  There is very little compute
> that's "cpu-limited" at multi-second scales that can't benefit from these
> approaches, hence the death of non-GPU supercomputing.
> >
> > Paul
> >
> >
> >
> > On Fri, Dec 27, 2024 at 7:13 AM <tlaronde@kergis.com> wrote:
> >>
> >> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> >> [very interesting stuff]
> >> >
> >> > Finally, why did something like this not ever happen? Because GPUs
> >> > came along a few years later and that's where all the parallelism in
> >> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
> >> > GPU era.
> >> >
> >>
> >> GPUs are actually wreaking havoc other kernels, with, in the Unix
> >> world, X11 being in a bad shape for several reasons, one being that
> >> GPU are not limited to graphical display---this tends to be
> >> anecdoctical in some sense.
> >>
> >> Can you elaborate on the GPUs paradigm break? I tend to think that
> >> there is a main difference between "equals" sharing a same address
> >> space via MMU, and auxiliary processors that are using another address
> >> space. A GPU, as far as I know (this is not much), is an auxiliary
> >> processor when the GPU is discrete, and is a specialized processor
> >> sharing the same address space when integrated (but I guess that a
> >> HPC have discrete GPUs with perhaps a specialized connection).
> >>
> >> Do you know good references about:
> >>
> >> - organizing processors depending on memory connection---I found
> >> mainly M. J. Flynn's paper(s) about this, but nothing more
> >> recent---and the impact on an OS design;
> >>
> >> - IPC vs threads---from your description, it seems that your solution
> >> was multiplying processes so IPC instead of multiplying threads---but
> >> nonetheless the sharing of differing memories remains, and is more
> >> easy to solve with IPC than with threads;
> >>
> >> - Present GPU's architecture (supposing it is documented; it seems not
> >> totally from "General-Purpose Graphics Processor Archictectures",
> >> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
> >> composing hardware by connecting dedicated elements, and vectors vs
> >> SIMT.
> >>
> >> Thanks for sharing (what can be shared)!
> >> --
> >> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >>              http://www.kergis.com/
> >>             http://kertex.kergis.com/
> >> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> >
> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M5030d3a8098ce71249fd4d89
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 8475 bytes --]

Re: [9fans] NIX experience

[tlaronde]

On Sat, Dec 28, 2024 at 04:26:49PM -0800, Paul Lalonde wrote:
> MIG is interesting.  It's something we partly devised to deal with sharing
> interactive workloads on the same GPU.
> Time-slicing is very difficult on a GPU because the state is so large. You
> wind up having to drain the workload and that means you're holding on to
> resources until the longest job ends.  Utilization becomes very poor, and
> even worse if your application uses a long (read, non-terminating) compute
> task.
> So you want some way to partition the GPU more along spatial lines - giving
> each process a subset of the hardware resources.
> For some workloads this scheduling is easy - you give each workload enough
> front end resources, memory, and threads to donuts job.  But I think we're
> still a long way off from the GPU being able to do these allocations
> dynamically and efficiently.
> There are some really interesting non-uniform co-scheduling constraints
> that I've not seen addressed in the literature.

But an OS is precisely that: implementing a policy of access to
resources. It looks like it would end with an ALU put in the GPU and a
(specialized?) OS i.e. back to square one.

This thread has turned to be a very good one because it contains very
good questions! (that I have read nowhere else...)

What I have also noted in Ron Minnich's mail is that (knowledgeable)
people were able in a few weeks to build a solution starting from
Plan9. And I think here lies a future: a system simple enough (i.e.
simple as a shortest path to the truth) that (knowledgeable) people
can build rapidly a customized solution that can give the desired
result without wasting resources. This probably doesn't mean that
Plan9 can stay as is, but this doesn't mean either that it has to
be discarded as a whole because the current solutions have won
since they are far better---this doesn't look like it at all, the
current solutions seeming more like a patchwork of ad hoc modifications
than a well thought unit.

It is said that generative A.I. is increasing the demand on electrical
power and resources to give a result that was already possible, with
less means, using not generative but specialized A.I.

The description of the wandering concerning the hardware is a
supplementary indication that the dice is still rolling.

[I will now shut up and read the documentation that several have been
nice enough to indicate.]

> 
> On Sat, Dec 28, 2024, 4:19?p.m. Skip Tavakkolian <skip.tavakkolian@gmail.com>
> wrote:
> 
> > I think there is a conflict between two different types of usage that
> > GPU architectures need to support. One is full-speed performance,
> > where the resource is fully owned and utilized for a single purpose
> > for a large chunk of time, and the other is where the GPU is a rare
> > resource that needs to be shared in short intervals (e.g. ML/AI
> > inference mode)
> >
> > If I understand correctly, NIX's approach addresses the first case (at
> > least for cores).  Nvidia's Multiple-Instance GPU (MIG) architecture
> > seems to help to handle the second type of usage. The first case
> > requires maximizing data transfer rate, and the second needs clever
> > scheduling to minimize switching (large/huge) model parameters in/out.
> >
> >
> > On Fri, Dec 27, 2024 at 8:33?AM Paul Lalonde <paul.a.lalonde@gmail.com>
> > wrote:
> > >
> > > GPUs have been my bread and butter for 20+ years.
> > >
> > > The best introductory source continues to be Kayvon Fatahalian and Mike
> > Houston's 2008 CACM paper: https://dl.acm.org/doi/10.1145/1400181.1400197
> > >
> > > It says little about the software interface to the GPU, but does a very
> > good job of motivating and describing the architecture.
> > >
> > > The in-depth resource for modern GPU architecture is the Nvidia A100
> > tensor architecture paper:
> > https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf.
> > It's a slog, but clearly shows how compute has changed.  Particularly, much
> > of the success is in turning branchy workloads with scattered memory
> > accesses into much more bulk-oriented data streams that match well to the
> > "natural" structure of the tensor cores.  The performance gains can be
> > astronomical. I've personally made > 1000x - yes, that's *times* not
> > percentages - speedups with some workloads.  There is very little compute
> > that's "cpu-limited" at multi-second scales that can't benefit from these
> > approaches, hence the death of non-GPU supercomputing.
> > >
> > > Paul
> > >
> > >
> > >
> > > On Fri, Dec 27, 2024 at 7:13?AM <tlaronde@kergis.com> wrote:
> > >>
> > >> On Thu, Dec 26, 2024 at 10:24:23PM -0800, Ron Minnich wrote:
> > >> [very interesting stuff]
> > >> >
> > >> > Finally, why did something like this not ever happen? Because GPUs
> > >> > came along a few years later and that's where all the parallelism in
> > >> > HPC is nowadays. NIX was a nice idea, but it did not survive in the
> > >> > GPU era.
> > >> >
> > >>
> > >> GPUs are actually wreaking havoc other kernels, with, in the Unix
> > >> world, X11 being in a bad shape for several reasons, one being that
> > >> GPU are not limited to graphical display---this tends to be
> > >> anecdoctical in some sense.
> > >>
> > >> Can you elaborate on the GPUs paradigm break? I tend to think that
> > >> there is a main difference between "equals" sharing a same address
> > >> space via MMU, and auxiliary processors that are using another address
> > >> space. A GPU, as far as I know (this is not much), is an auxiliary
> > >> processor when the GPU is discrete, and is a specialized processor
> > >> sharing the same address space when integrated (but I guess that a
> > >> HPC have discrete GPUs with perhaps a specialized connection).
> > >>
> > >> Do you know good references about:
> > >>
> > >> - organizing processors depending on memory connection---I found
> > >> mainly M. J. Flynn's paper(s) about this, but nothing more
> > >> recent---and the impact on an OS design;
> > >>
> > >> - IPC vs threads---from your description, it seems that your solution
> > >> was multiplying processes so IPC instead of multiplying threads---but
> > >> nonetheless the sharing of differing memories remains, and is more
> > >> easy to solve with IPC than with threads;
> > >>
> > >> - Present GPU's architecture (supposing it is documented; it seems not
> > >> totally from "General-Purpose Graphics Processor Archictectures",
> > >> Aamodt, Lun Fung, Rogers, SpringerVerlag) and the RISC-V approach,
> > >> composing hardware by connecting dedicated elements, and vectors vs
> > >> SIMT.
> > >>
> > >> Thanks for sharing (what can be shared)!
> > >> --
> > >> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> > >>              http://www.kergis.com/
> > >>             http://kertex.kergis.com/
> > >> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> > >
> > > 9fans / 9fans / see discussions + participants + delivery options
> > Permalink

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M1b3983cb38a1e4e708c72b7a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[andreas.elding via 9fans]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

Thank you for the response, it was quite an interesting read. Unfortunately, I'm not a great coder, so I can't take you up on that offer. You mentioned that GPUs took over, but not all problems can run on GPUs. There may still be a general interest in this (I know I am interested).

How was it presented to the users? Could they query to see the current utilization of the system?

How did you know that a job completed (or failed)?

You mentioned it was a shared memory system, meaning it was in essence a "very large SMP machine" from the view of the OS?

Could the NIX system only work with shared memory systems like that, or was it possible to take many smaller independent systems and combine their resources?

Anything you can say about the actual usage would be quite interesting - what kind of applications are we talking? Was there commercial interest?
------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mad958dee6fd001441970abf8
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 1763 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

Thanks for the good questions.

On Sun, Dec 29, 2024 at 4:10 PM andreas.elding via 9fans
<9fans@9fans.net> wrote:
> How was it presented to the users? Could they query to see the current utilization of the system?

It looked very normal. To see what was running, you did ps. In the
status, you could see that a process was on an AC. It looked a lot
like a wired proc, save that no kernel code would run and interrupt a
process on an AC. If you are familiar with plan 9 libthread, or
goroutines, it was taking that non-preemptive idea just a bit further:
your proc owned the core, until you were done.

BTW, there are 512- and 1024-core risc-v systems in the works, and NIX
looks pretty good for that kind of CPU.

> How did you know that a job completed (or failed)?

Just as with a process; you read /proc/pid/wait. It was very transparent.

>
> You mentioned it was a shared memory system, meaning it was in essence a "very large SMP machine" from the view of the OS?

Yes, with a slight change in view: the AC looked like a CPU, and there
was shared memory, and it was coherent, but the AC scheduling was a
different bit of code than normal process scheduling.

> Could the NIX system only work with shared memory systems like that, or was it possible to take many smaller independent systems and combine their resources?

My original idea in spring 2011, after talking to Shalf at LBL, was
that we might have CPUs with hardware FIFOs communicating. When I got
to lsub, Charles made the point: "you have a shared memory machine,
might as well use it" -- and that made a lot of sense. So we used
shared memory, and avoided a lot of headaches that Charles, jmk, Eric,
and  I had dealt with on Blue Gene.

>
> Anything you can say about the actual usage would be quite interesting - what kind of applications are we talking? Was there commercial interest?

Nobody used it. In 2006, there was a strong move to Linux, but there
was still room for non-Linux approaches. By 2011, that option was
almost gone. I gave talks on NIX and Plan 9 for about 10 years at
various DOE labs, conferences, and universities; there was always
interest, but "not Linux" became harder and harder to overcome. My
last talk on NIX was 6 months after I left Sandia for Google, and it
was more of a history lesson than anything.

ron

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Meaa99ccddc16c65fcb9a8e51
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Bakul Shah via 9fans]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

I wonder how these many-core systems share memory effectively.

> On Dec 30, 2024, at 8:25 AM, Ron Minnich <rminnich@p9f.org> wrote:
> 
> BTW, there are 512- and 1024-core risc-v systems in the works, and NIX
> looks pretty good for that kind of CPU.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M4c25fea4b6b1e77fa9542acc
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2687 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

On Mon, Dec 30, 2024 at 9:39 AM Bakul Shah via 9fans <9fans@9fans.net> wrote:
>
> I wonder how these many-core systems share memory effectively.

Typically there is an on-chip network, and at least on some systems,
memory blocks scattered among the cores.

See the Esperanto SOC-1 for one example.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M42bdc9a114b187280c9ca3c3
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]

The hard part is memory consistency.

x86 has a strong coherence model, which means that any write is immediately
visible to any other core that loads the same address.  This wreaks havoc
on your cache architecture.  You need to either have a global
synchronization point (effectively a global shared cache) with its
associated performance bottleneck, or you need some method to track which
cores are using which cache lines that might become invalidated by a
write.  Contended cache lines become very expensive in terms of
communication.  Intel's chips deal with this by having "tag directories"
which track globally which caches contain which cache lines.  AMD has a
similar structure on the IO die of their chiplet-based architectures.  In
both cases, you get much better performance if you avoid *ever* using the
same cache line on separate processors.  In the AMD case you can fairly
easily partition your workload across physical memory channels to do this.
On Intel it's much more "magical", and it's not at all clear how well the
magic scales into very large numbers of cores.  You might note that Intel
is not doing as well as AMD these days on the architecture side.

Relaxing the memory model somewhat like ARM and RISC-V can help there by
reducing the number of (false) synchronization points between caches, but
at the cost of more subtle bugs if you miss including the required
synchronization primitives.

Paul


On Mon, Dec 30, 2024 at 10:00 AM Ron Minnich <rminnich@p9f.org> wrote:

> On Mon, Dec 30, 2024 at 9:39 AM Bakul Shah via 9fans <9fans@9fans.net>
> wrote:
> >
> > I wonder how these many-core systems share memory effectively.
> 
> Typically there is an on-chip network, and at least on some systems,
> memory blocks scattered among the cores.
> 
> See the Esperanto SOC-1 for one example.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mf27994b8c88e9e8b8693b871
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 3569 bytes --]

Re: [9fans] NIX experience

[tlaronde]

On Mon, Dec 30, 2024 at 10:15:13AM -0800, Paul Lalonde wrote:
> The hard part is memory consistency.
> 
> x86 has a strong coherence model, which means that any write is immediately
> visible to any other core that loads the same address.  This wreaks havoc
> on your cache architecture.  You need to either have a global
> synchronization point (effectively a global shared cache) with its
> associated performance bottleneck, or you need some method to track which
> cores are using which cache lines that might become invalidated by a
> write.  Contended cache lines become very expensive in terms of
> communication.  Intel's chips deal with this by having "tag directories"
> which track globally which caches contain which cache lines.  AMD has a
> similar structure on the IO die of their chiplet-based architectures.  In
> both cases, you get much better performance if you avoid *ever* using the
> same cache line on separate processors.  In the AMD case you can fairly
> easily partition your workload across physical memory channels to do this.
> On Intel it's much more "magical", and it's not at all clear how well the
> magic scales into very large numbers of cores.  You might note that Intel
> is not doing as well as AMD these days on the architecture side.
> 
> Relaxing the memory model somewhat like ARM and RISC-V can help there by
> reducing the number of (false) synchronization points between caches, but
> at the cost of more subtle bugs if you miss including the required
> synchronization primitives.
> 

Perhaps the target market is not really HPC in the sense of trying to
speed up _one_ task by doing it, parallely, concurrently, on a lot of
cores, but more the "cloud": limited needs for global synchronization,
since, all in all, there are separate VMs running at the same time
but almost orthogonal---even the storage is in fact segregated. So
it is a more tightly coupled cluster of separated systems, allowing
speedy migration of tasks, than a single system (for a single
task---computing weather evolution or simulating complex physics
systems. Just a guess.

> 
> 
> On Mon, Dec 30, 2024 at 10:00?AM Ron Minnich <rminnich@p9f.org> wrote:
> 
> > On Mon, Dec 30, 2024 at 9:39?AM Bakul Shah via 9fans <9fans@9fans.net>
> > wrote:
> > >
> > > I wonder how these many-core systems share memory effectively.
> > 
> > Typically there is an on-chip network, and at least on some systems,
> > memory blocks scattered among the cores.
> > 
> > See the Esperanto SOC-1 for one example.

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Ma32369581e699aa37841140f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 2031 bytes --]

Yes, AMD's EPYC line and derivatives, with their reasonably nice memory
partitioning is *excellent* for running independent VMs.  It does a good
job of letting you scale your core counts appropriately to the size of the
VM.

Nvidia's GeForce NOW game streaming platform runs (ran?  I'm not there
anymore, though this information is publicly available from 2023 at least)
on these AMD CPUs precisely because of the excellent workload isolation and
scalability.

Paul

On Mon, Dec 30, 2024, 12:15 p.m. <tlaronde@kergis.com> wrote:

> Perhaps the target market is not really HPC in the sense of trying to
> speed up _one_ task by doing it, parallely, concurrently, on a lot of
> cores, but more the "cloud": limited needs for global synchronization,
> since, all in all, there are separate VMs running at the same time
> but almost orthogonal---even the storage is in fact segregated. So
> it is a more tightly coupled cluster of separated systems, allowing
> speedy migration of tasks, than a single system (for a single
> task---computing weather evolution or simulating complex physics
> systems. Just a guess.
>
> >
> >
> > On Mon, Dec 30, 2024 at 10:00?AM Ron Minnich <rminnich@p9f.org> wrote:
> >
> > > On Mon, Dec 30, 2024 at 9:39?AM Bakul Shah via 9fans <9fans@9fans.net>
> > > wrote:
> > > >
> > > > I wonder how these many-core systems share memory effectively.
> > >
> > > Typically there is an on-chip network, and at least on some systems,
> > > memory blocks scattered among the cores.
> > >
> > > See the Esperanto SOC-1 for one example.
> 
> --
> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>              http://www.kergis.com/
>             http://kertex.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M72399792ed2cf545ee3b5efe
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 4086 bytes --]

Re: [9fans] NIX experience

[Andreas.Elding via 9fans]

On Monday, December 30th, 2024 at 5:25 PM, Ron Minnich <rminnich@p9f.org> wrote:

> Thanks for the good questions.

This has been a very interesting thread and I'm very glad you've given us your rare insights here, thank you!

There are still some points I'd like to have fleshed out, but I'm not sure how to put it succinctly.

> BTW, there are 512- and 1024-core risc-v systems in the works, and NIX
> looks pretty good for that kind of CPU.
> 
> > How did you know that a job completed (or failed)?
> 
> Just as with a process; you read /proc/pid/wait. It was very transparent.
> 
> > You mentioned it was a shared memory system, meaning it was in essence a "very large SMP machine" from the view of the OS?
> 
> Yes, with a slight change in view: the AC looked like a CPU, and there
> was shared memory, and it was coherent, but the AC scheduling was a
> different bit of code than normal process scheduling.
> 
> > Could the NIX system only work with shared memory systems like that, or was it possible to take many smaller independent systems and combine their resources?
> 
> My original idea in spring 2011, after talking to Shalf at LBL, was
> that we might have CPUs with hardware FIFOs communicating. When I got
> to lsub, Charles made the point: "you have a shared memory machine,
> might as well use it" -- and that made a lot of sense. So we used
> shared memory, and avoided a lot of headaches that Charles, jmk, Eric,
> and I had dealt with on Blue Gene.

When I first heard about NIX, didn't know the architecture, but thought it was many systems working as one. That made a certain amount of sense to me, given the namespaces of Plan9. You could potentially map in the cores of as many independent systems as you needed to your one task.

As you said above, NIX looks quite good for hundreds of cores or more on a single processor.

Do you think the current code could be used in any way to make use of more than a single system?



------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mdb6ffb1cae13ab4498d49f9f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

Timesharing core (TC) processes schedule onto AC (app core) several
ways, one of them being execac.

execac is exec with benefits. The process is started via the normal
exec path, then you can think of it as suspending on the kernel core,
and resuming on an AC.
The way that's done: AC is running a "nanokernel" that is spinning on
a function pointer in a struct in something called the ICC (Inter-Core
Communication).

So TC fills in args, then sets the function pointer, then enters a
loop calling sched() and checking the pointer. Nothing about this
code, and how it works with sched(), is very special.

AC sees that the pointer is non-zero, sets up args and other state,
and exits kernel mode to that address. Your proc is now on the AC, and
will run until it decides not to (system call, fault, whatever).

When the process on the AC needs to do something other than compute,
it exits user mode to the nanokernel. There is nothing special about
this user mode exit, it can
happen as a system call, page fault, or for some other reason. There
are NO changes to libc, or any other source.

The AC saves state, zeros the function pointer, re-enters the
nanokernel spinloop. The AC has no memory of what it was running;
resuming the same process, or some other process, looks the same to
the AC. There is a check in the nanokernel to make sure the stack is
not growing without end.

The TC, as part of normal scheduling, schedules the kernel process
code on the TC, which when it runs, is doing runs the "is the function
pointer 0" test and, if so, resumes the process on the TC. Whatever
needs to be done gets done, and ... resume scheduling on the AC.
Unless there is a note or some other reason to not go back to the AC.

What happens if the ACs all go away while the process is on the TC? It
can keep running on the TC. One of our goals was that NIX processes
always work, even if an AC is not available for some reason.

A couple of things here:
- NIX required a very small, but very careful, set of changes. But the
size of the code? Very small. Just tricky in places. Gorka and I spent
a fair bit of time getting that ICC stuff right.
- Note that the TC offload to AC uses very little change to the sched().
- The nanokernel is less than 60 lines or so IIRC
- I have a standard benchmark that measures kernel noise. It's never
perfect, save on Blue Gene. On NIX, procs on ACs had zero noise. The
noise was perfect. That never happens ...

So, as I'm remembering this, I'm  remembering my surprise at how well
it worked, and how quickly we got it going.

But: you can see that all this kind of needs some sort of shared
memory. I think for non-shared-memory environments, it would have to
be done differently.

Also, the namespaces had no effect, good or bad, on all this working.
They did not come into the picture, and whether a proc was on AC or
TC, namespaces were not impacted.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mbf014d88fa16715e342c3e95
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Tue, Dec 31, 2024 at 11:29:43PM -0800, Ron Minnich wrote:
> Timesharing core (TC) processes schedule onto AC (app core) several
> ways, one of them being execac.
> 

Is the number of TC fixed, or is it at least one TC and the number
can increase if needed (or, put it differently, can a AC, if needed,
switch to a TC and vice-versa)?

> execac is exec with benefits. The process is started via the normal
> exec path, then you can think of it as suspending on the kernel core,
> and resuming on an AC.
> The way that's done: AC is running a "nanokernel" that is spinning on
> a function pointer in a struct in something called the ICC (Inter-Core
> Communication).
> 
> So TC fills in args, then sets the function pointer, then enters a
> loop calling sched() and checking the pointer. Nothing about this
> code, and how it works with sched(), is very special.
> 
> AC sees that the pointer is non-zero, sets up args and other state,
> and exits kernel mode to that address. Your proc is now on the AC, and
> will run until it decides not to (system call, fault, whatever).
> 
> When the process on the AC needs to do something other than compute,
> it exits user mode to the nanokernel. There is nothing special about
> this user mode exit, it can
> happen as a system call, page fault, or for some other reason. There
> are NO changes to libc, or any other source.
> 
> The AC saves state, zeros the function pointer, re-enters the
> nanokernel spinloop. The AC has no memory of what it was running;
> resuming the same process, or some other process, looks the same to
> the AC. There is a check in the nanokernel to make sure the stack is
> not growing without end.
> 
> The TC, as part of normal scheduling, schedules the kernel process
> code on the TC, which when it runs, is doing runs the "is the function
> pointer 0" test and, if so, resumes the process on the TC. Whatever
> needs to be done gets done, and ... resume scheduling on the AC.
> Unless there is a note or some other reason to not go back to the AC.
> 
> What happens if the ACs all go away while the process is on the TC? It
> can keep running on the TC. One of our goals was that NIX processes
> always work, even if an AC is not available for some reason.
> 
> A couple of things here:
> - NIX required a very small, but very careful, set of changes. But the
> size of the code? Very small. Just tricky in places. Gorka and I spent
> a fair bit of time getting that ICC stuff right.
> - Note that the TC offload to AC uses very little change to the sched().
> - The nanokernel is less than 60 lines or so IIRC
> - I have a standard benchmark that measures kernel noise. It's never
> perfect, save on Blue Gene. On NIX, procs on ACs had zero noise. The
> noise was perfect. That never happens ...
> 
> So, as I'm remembering this, I'm  remembering my surprise at how well
> it worked, and how quickly we got it going.
> 

Had you the opportunity to measure how "bad" some application
workloads could be because the number of TC---after the
initialization period---exceeded largely the number of AC with a not zero
pointer?

Because I imagine that the OS can be as smart as possible, if the
applications are badly programmed or if the data is not optimally
organized, no-one can hope solving with the OS a problem that relies on the
application side...

> But: you can see that all this kind of needs some sort of shared
> memory. I think for non-shared-memory environments, it would have to
> be done differently.
> 

Theoretically, could a machine with different kind of cores, perhaps with
differing architectures (specialized cores) but sharing at least
with a common MMU read/write (data) pages (for the kernel shared
data: locks and so on) be possible, with a system such as NIX in fact
scheduling to the matching kind of AC core for the task to be run?

There are cross-compilers, because they deal with bytes stream and
need not execute what they produce. There can be a "cross-OS", that
manage tasks without having to execute directly them (as long as
sharing of the structures for kernel management is possible).

> Also, the namespaces had no effect, good or bad, on all this working.
> They did not come into the picture, and whether a proc was on AC or
> TC, namespaces were not impacted.
> 

I wish the best to the readers for 2025 and thank Ron Minnich, Paul Lalonde and
all the participants to this thread who saved the 2024 ratio
signal/noise of the list :-)

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mf5c9ef6c95a0e34cab5b7860
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

On Wed, Jan 1, 2025 at 4:35 AM <tlaronde@kergis.com> wrote:
> Is the number of TC fixed, or is it at least one TC and the number
> can increase if needed (or, put it differently, can a AC, if needed,
> switch to a TC and vice-versa)?

Fixed, I  believe, at boot time? I no longer recall. Nemo and lsub did
experiment
with dynamic counts, but I came from an HPC/LinuxBios background:
nodes boot in seconds,
so I did not worry about "fixed at boot time" type issues. Want to
change configuration? reboot.
Starting a new job? reboot. And so on.

> Had you the opportunity to measure how "bad" some application
> workloads could be because the number of TC---after the
> initialization period---exceeded largely the number of AC with a not zero
> pointer?
>

That would be good to do, now that we have machines with hundreds of cores.

> Theoretically, could a machine with different kind of cores, perhaps with
> differing architectures (specialized cores) but sharing at least
> with a common MMU read/write (data) pages (for the kernel shared
> data: locks and so on) be possible, with a system such as NIX in fact
> scheduling to the matching kind of AC core for the task to be run?

You have that already, nowadays, starting with big/little, and moving
to the intel
CPUs with widely varying core types. Esperanto has specialized cores too.

I think having incompatible architectures is already there, too, with
GPUs and smart nics like the AWS and Google ones.
Well, hmm, that kind of "sharing with compatible MMU" was started by
quadrics in the 90s, so it's not new.

This has been a very interesting discussion, thanks all. My offer
remains: if anyone wants to revive NIX, I am happy to help.

ron

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M7cfd8da69529213fcf4e109f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 2319 bytes --]

Just saw a review of System76 Thelio Astra (Ampere Altra). An arm64 system
with 128 cores and 512GB of memory for under $7500.  NIX's model seems more
and more applicable to commodity hardware.

On Fri, Jan 3, 2025, 11:11 AM Ron Minnich <rminnich@p9f.org> wrote:

> On Wed, Jan 1, 2025 at 4:35 AM <tlaronde@kergis.com> wrote:
> > Is the number of TC fixed, or is it at least one TC and the number
> > can increase if needed (or, put it differently, can a AC, if needed,
> > switch to a TC and vice-versa)?
>
> Fixed, I  believe, at boot time? I no longer recall. Nemo and lsub did
> experiment
> with dynamic counts, but I came from an HPC/LinuxBios background:
> nodes boot in seconds,
> so I did not worry about "fixed at boot time" type issues. Want to
> change configuration? reboot.
> Starting a new job? reboot. And so on.
>
> > Had you the opportunity to measure how "bad" some application
> > workloads could be because the number of TC---after the
> > initialization period---exceeded largely the number of AC with a not zero
> > pointer?
> >
>
> That would be good to do, now that we have machines with hundreds of cores.
>
> > Theoretically, could a machine with different kind of cores, perhaps with
> > differing architectures (specialized cores) but sharing at least
> > with a common MMU read/write (data) pages (for the kernel shared
> > data: locks and so on) be possible, with a system such as NIX in fact
> > scheduling to the matching kind of AC core for the task to be run?
> 
> You have that already, nowadays, starting with big/little, and moving
> to the intel
> CPUs with widely varying core types. Esperanto has specialized cores too.
> 
> I think having incompatible architectures is already there, too, with
> GPUs and smart nics like the AWS and Google ones.
> Well, hmm, that kind of "sharing with compatible MMU" was started by
> quadrics in the 90s, so it's not new.
> 
> This has been a very interesting discussion, thanks all. My offer
> remains: if anyone wants to revive NIX, I am happy to help.
> 
> ron

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M719d20d891666bd4836e4a53
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 4081 bytes --]

Re: [9fans] NIX experience

[tlaronde]

On Fri, Jan 03, 2025 at 12:32:55PM -0800, Skip Tavakkolian wrote:
> Just saw a review of System76 Thelio Astra (Ampere Altra). An arm64 system
> with 128 cores and 512GB of memory for under $7500.  NIX's model seems more
> and more applicable to commodity hardware.
> 

Since there is a plan9 foundation now, and there will be soon (May
22-24, 2025) the 11th International Workshop on Plan9, there are means
to put everything together, theory and practice, for example by
choosing and buying hardware fit for testing ideas---I would be
personnally OK to contribute for some hundreds of $ to the foundation
for this.

And the goal is, as usual, to find a thundering advertisement. For
example: 

Demonstrate that if the right OS principles were used, and the
right software, the savings in electricity for computation of
weather evolution and generation of academic papers would be so
huge, that the Earth will indeed be facing a glaciation age and not
a global warming, so that Miami would be one of the hugest ski
resort in the world, exploiting ski slopes on icebergs floating near
its costs---put in other words:  that this is the modelisations and the
diarrhea of academic papers (using TeXlive and not kerTeX :-^) on the
present OSes and with the present software that are responsible for the
"global warming".


> On Fri, Jan 3, 2025, 11:11?AM Ron Minnich <rminnich@p9f.org> wrote:
> 
> > On Wed, Jan 1, 2025 at 4:35?AM <tlaronde@kergis.com> wrote:
> > > Is the number of TC fixed, or is it at least one TC and the number
> > > can increase if needed (or, put it differently, can a AC, if needed,
> > > switch to a TC and vice-versa)?
> >
> > Fixed, I  believe, at boot time? I no longer recall. Nemo and lsub did
> > experiment
> > with dynamic counts, but I came from an HPC/LinuxBios background:
> > nodes boot in seconds,
> > so I did not worry about "fixed at boot time" type issues. Want to
> > change configuration? reboot.
> > Starting a new job? reboot. And so on.
> >
> > > Had you the opportunity to measure how "bad" some application
> > > workloads could be because the number of TC---after the
> > > initialization period---exceeded largely the number of AC with a not zero
> > > pointer?
> > >
> >
> > That would be good to do, now that we have machines with hundreds of cores.
> >
> > > Theoretically, could a machine with different kind of cores, perhaps with
> > > differing architectures (specialized cores) but sharing at least
> > > with a common MMU read/write (data) pages (for the kernel shared
> > > data: locks and so on) be possible, with a system such as NIX in fact
> > > scheduling to the matching kind of AC core for the task to be run?
> > 
> > You have that already, nowadays, starting with big/little, and moving
> > to the intel
> > CPUs with widely varying core types. Esperanto has specialized cores too.
> > 
> > I think having incompatible architectures is already there, too, with
> > GPUs and smart nics like the AWS and Google ones.
> > Well, hmm, that kind of "sharing with compatible MMU" was started by
> > quadrics in the 90s, so it's not new.
> > 
> > This has been a very interesting discussion, thanks all. My offer
> > remains: if anyone wants to revive NIX, I am happy to help.
> > 
> > ron

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M66a2043de09d8c30493ea650
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

> This has been a very interesting discussion, thanks all. My offer
> remains: if anyone wants to revive NIX, I am happy to help.

Am I the only one who sees that the Fastcall stuff would be good for
bringing some devices out of the kernel (that are devs only for
performance reasons)?

And then, closer to what Fastcall was actually for (fossil and
venti>disk), you also have ??fs>nusb/disk>disk, which could always do
with a speedup.

(If I'm wrong, what is it I'm missing?)

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M9d89f6803e2d8a5892e66a8a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Bakul Shah via 9fans]

On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
>> This has been a very interesting discussion, thanks all. My offer
>> remains: if anyone wants to revive NIX, I am happy to help.
> 
> Am I the only one who sees that the Fastcall stuff would be good for
> bringing some devices out of the kernel (that are devs only for
> performance reasons)?
> 
> And then, closer to what Fastcall was actually for (fossil and
> venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> with a speedup.

I've been meaning to ask... What is the typical *overhead* of a 9p
call to a user level driver compared to a kernel based driver?
------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mfa21d6c233a6efbd3e250fb7
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Kim Shrier]

> On Jan 3, 2025, at 11:56 AM, Ron Minnich <rminnich@p9f.org> wrote:
> 
> This has been a very interesting discussion, thanks all. My offer
> remains: if anyone wants to revive NIX, I am happy to help.
> 
> ron

I have been interested in NIX since I read about it and would be
willing to assist, if possible, in reviving it.  However, I don’t have
the time to do so right now due to my day job consuming 110%
of my time and other commitments in my personal life consuming
anything I might consider discretionary time for personal projects.

If this effort gets started, I would like to be kept advised of its
status.  And, if possible, I would like to help.

Kim
_
C++ is an off-by-one error





------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M27f94667d3f7bfbdf5cb8fe3
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

let's make a deal.

I will talk to Ampere about what is the right way to simulate their
system. I have friends there. If we get enough people to get NIX
running on a simulator, then I'll try to figure out how to get us some
real hardware, which we could position at a friendly university --
anybody want to volunteer? We used to have a Plan 9 system at LBNL,
back in the day, but all those people left. So we need a new friendly
space.

Deal?

This is hard work, and it's far more important to be persistent over a
long time than to get excited, put out a huge effort, and then drop
it.

It's WAY more important to put in the "think time" to work out how it
gets done than to write code. The amount of code is small. The details
are tricky. Thinking and reading code matters much, much more than
writing code.

 I've been down this road a lot. Just an hour or so a week, applied
consistently over a period of months, can get us there. Our goal
should be to have NIX working on a simulator by May. Believe it or
not, that's just about the right amount of time.

As to which kernel, I lean to 9front. It's the most actively developed
one I know of just now.

On Sat, Jan 4, 2025 at 12:17 PM Kim Shrier <kim@westryn.net> wrote:
> > On Jan 3, 2025, at 11:56 AM, Ron Minnich <rminnich@p9f.org> wrote:
> >
> > This has been a very interesting discussion, thanks all. My offer
> > remains: if anyone wants to revive NIX, I am happy to help.
> >
> > ron
> 
> I have been interested in NIX since I read about it and would be
> willing to assist, if possible, in reviving it.  However, I don’t have
> the time to do so right now due to my day job consuming 110%
> of my time and other commitments in my personal life consuming
> anything I might consider discretionary time for personal projects.
> 
> If this effort gets started, I would like to be kept advised of its
> status.  And, if possible, I would like to help.
> 
> Kim
> _
> C++ is an off-by-one error
> 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M26e47f3c4f8ab1db70cf4f2f
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Thaddeus Woskowiak]

On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
>
> On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >> This has been a very interesting discussion, thanks all. My offer
> >> remains: if anyone wants to revive NIX, I am happy to help.
> >
> > Am I the only one who sees that the Fastcall stuff would be good for
> > bringing some devices out of the kernel (that are devs only for
> > performance reasons)?
> >
> > And then, closer to what Fastcall was actually for (fossil and
> > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > with a speedup.
> 
> I've been meaning to ask... What is the typical *overhead* of a 9p
> call to a user level driver compared to a kernel based driver?

From what I know the only performance issue for 'user-space <->
kernel-space' 9P are context switches. IP is in-kernel to eliminate
context switches for ether(3) <-> ip(3).

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M63b69bcecc79d154ac18e796
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

i think brazil experimented with networking outside the kernel but it was
pushed back in

On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com>
wrote:

> On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net>
> wrote:
> >
> > On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com>
> wrote:
> > >> This has been a very interesting discussion, thanks all. My offer
> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > >
> > > Am I the only one who sees that the Fastcall stuff would be good for
> > > bringing some devices out of the kernel (that are devs only for
> > > performance reasons)?
> > >
> > > And then, closer to what Fastcall was actually for (fossil and
> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > > with a speedup.
> >
> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > call to a user level driver compared to a kernel based driver?
> 
> From what I know the only performance issue for 'user-space <->
> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> context switches for ether(3) <-> ip(3).

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Med712cc7fb24ad3c50592404
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2953 bytes --]

Re: [9fans] NIX experience

[tlaronde]

On Sat, Jan 04, 2025 at 01:29:19PM -0800, Ron Minnich wrote:
> let's make a deal.
> 
> I will talk to Ampere about what is the right way to simulate their
> system. I have friends there. If we get enough people to get NIX
> running on a simulator, then I'll try to figure out how to get us some
> real hardware, which we could position at a friendly university --
> anybody want to volunteer? We used to have a Plan 9 system at LBNL,
> back in the day, but all those people left. So we need a new friendly
> space.

For the hardware, I will put money where my mouth is: I will gladly
donate some hundreds of $ to help to gather the amount
needed to get some hardware. What structure will gather and
administrate the founds is to be decided---the plan9 foundation seemed
to me an obvious choice, but I'm not adamant about this as long as we
don't see again an increase of entropy.

Donate: no strings attached; I'm not claiming "rights" to access
the hardware, or to have a voice to decide who or what will have
it, and neither candidating to be part of a "team": On the kernel
side, I'm largely out of my depth for now with far more questions
than answers and I'm currently mastering a lot of documentation to
reach a reasonable level of understanding, both about the problems
and about the different solutions proposed, now or before---still
hoping that at the end there will still be good questions left
unanswered so that there is perhaps a good path that has still not
be tried ;-)

> 
> Deal?
> 
> This is hard work, and it's far more important to be persistent over a
> long time than to get excited, put out a huge effort, and then drop
> it.
> 
> It's WAY more important to put in the "think time" to work out how it
> gets done than to write code. The amount of code is small. The details
> are tricky. Thinking and reading code matters much, much more than
> writing code.
> 
>  I've been down this road a lot. Just an hour or so a week, applied
> consistently over a period of months, can get us there. Our goal
> should be to have NIX working on a simulator by May. Believe it or
> not, that's just about the right amount of time.
> 
> As to which kernel, I lean to 9front. It's the most actively developed
> one I know of just now.
> 
> On Sat, Jan 4, 2025 at 12:17?PM Kim Shrier <kim@westryn.net> wrote:
> > > On Jan 3, 2025, at 11:56?AM, Ron Minnich <rminnich@p9f.org> wrote:
> > >
> > > This has been a very interesting discussion, thanks all. My offer
> > > remains: if anyone wants to revive NIX, I am happy to help.
> > >
> > > ron
> > 
> > I have been interested in NIX since I read about it and would be
> > willing to assist, if possible, in reviving it.  However, I don?t have
> > the time to do so right now due to my day job consuming 110%
> > of my time and other commitments in my personal life consuming
> > anything I might consider discretionary time for personal projects.
> > 
> > If this effort gets started, I would like to be kept advised of its
> > status.  And, if possible, I would like to help.
> > 
> > Kim
> > _
> > C++ is an off-by-one error
> > 

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M2c09fd275783183bbcb691d2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Daniel Maslowski via 9fans]

[-- Attachment #1: Type: text/plain, Size: 2179 bytes --]

There have been other ideas in similar directions over the years.
E.g.
https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors
about the concepts of ACs and CCs (communication cores).

On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com>
wrote:

> i think brazil experimented with networking outside the kernel but it was
> pushed back in
>
> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com>
> wrote:
>
>> On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net>
>> wrote:
>> >
>> > On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com>
>> wrote:
>> > >> This has been a very interesting discussion, thanks all. My offer
>> > >> remains: if anyone wants to revive NIX, I am happy to help.
>> > >
>> > > Am I the only one who sees that the Fastcall stuff would be good for
>> > > bringing some devices out of the kernel (that are devs only for
>> > > performance reasons)?
>> > >
>> > > And then, closer to what Fastcall was actually for (fossil and
>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
>> > > with a speedup.
>> >
>> > I've been meaning to ask... What is the typical *overhead* of a 9p
>> > call to a user level driver compared to a kernel based driver?
>> 
>> From what I know the only performance issue for 'user-space <->
>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
>> context switches for ether(3) <-> ip(3).
> *9fans <https://9fans.topicbox.com/latest>* / 9fans / see discussions
> <https://9fans.topicbox.com/groups/9fans> + participants
> <https://9fans.topicbox.com/groups/9fans/members> + delivery options
> <https://9fans.topicbox.com/groups/9fans/subscription> Permalink
> <https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Med712cc7fb24ad3c50592404>
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M339ed96afe5ddbf5b510c03b
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 3866 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

No need for money yet!

Let's get this party started. I have queries in to ampere as to how we
can set up a simulator. However, if someone wants to take a first
step, take that 2011 code, bring it to your plan 9 system, and see if
it builds.

Again, the key here is a sustained effort. You don't have to do a lot
each week, but you don't want to start and then drop it. So it needs
to NOT become all consuming. It's all about pacing yourself. Anybody
who's ever spent a few weeks digging ditches can tell you -- set up a
work effort you can sustain. Same thing here.

So, how about we figure out who here is interested, then start off:
get  the code, see if it builds. Who's in? Don't feel out of your
depth: if you can type mk, you're ready to start. Don't assume it's a
slog through code: take time to alternate looking at code, and reading
docs. Do learn how to use something like qemu -- it's a real
timesaver, since you can debug the kernel interactively.

Don't kill yourself if you hit a wall about some code -- bring it
here, and ask questions. That's why we're here.

So, Step 1: anyone? anyone?

Thanks

Ron

On Sun, Jan 5, 2025 at 7:22 AM Daniel Maslowski via 9fans
<9fans@9fans.net> wrote:
>
> There have been other ideas in similar directions over the years.
> E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
>
>
> On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
>>
>> i think brazil experimented with networking outside the kernel but it was pushed back in
>>
>> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
>>>
>>> On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
>>> >
>>> > On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
>>> > >> This has been a very interesting discussion, thanks all. My offer
>>> > >> remains: if anyone wants to revive NIX, I am happy to help.
>>> > >
>>> > > Am I the only one who sees that the Fastcall stuff would be good for
>>> > > bringing some devices out of the kernel (that are devs only for
>>> > > performance reasons)?
>>> > >
>>> > > And then, closer to what Fastcall was actually for (fossil and
>>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
>>> > > with a speedup.
>>> >
>>> > I've been meaning to ask... What is the typical *overhead* of a 9p
>>> > call to a user level driver compared to a kernel based driver?
>>> 
>>> From what I know the only performance issue for 'user-space <->
>>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
>>> context switches for ether(3) <-> ip(3).
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M27f7a378f6816d94910702d6
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Sun, Jan 05, 2025 at 08:36:17AM -0800, Ron Minnich wrote:
> No need for money yet!
> 
> Let's get this party started. I have queries in to ampere as to how we
> can set up a simulator. However, if someone wants to take a first
> step, take that 2011 code, bring it to your plan 9 system, and see if
> it builds.
> 
> Again, the key here is a sustained effort. You don't have to do a lot
> each week, but you don't want to start and then drop it. So it needs
> to NOT become all consuming. It's all about pacing yourself. Anybody
> who's ever spent a few weeks digging ditches can tell you -- set up a
> work effort you can sustain. Same thing here.
> 
> So, how about we figure out who here is interested, then start off:
> get  the code, see if it builds. Who's in? Don't feel out of your
> depth: if you can type mk, you're ready to start. Don't assume it's a
> slog through code: take time to alternate looking at code, and reading
> docs. Do learn how to use something like qemu -- it's a real
> timesaver, since you can debug the kernel interactively.
> 
> Don't kill yourself if you hit a wall about some code -- bring it
> here, and ask questions. That's why we're here.
> 
> So, Step 1: anyone? anyone?
> 

OK me, FW I'm Worth (If I do work, my worthiness will undoubtely improve,
but count it for a low value for now).

T. Laronde

> 
> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> <9fans@9fans.net> wrote:
> >
> > There have been other ideas in similar directions over the years.
> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> >
> >
> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> >>
> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> >>
> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> >>>
> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> >>> >
> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >>> > >> This has been a very interesting discussion, thanks all. My offer
> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> >>> > >
> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> >>> > > bringing some devices out of the kernel (that are devs only for
> >>> > > performance reasons)?
> >>> > >
> >>> > > And then, closer to what Fastcall was actually for (fossil and
> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> >>> > > with a speedup.
> >>> >
> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> >>> > call to a user level driver compared to a kernel based driver?
> >>> 
> >>> From what I know the only performance issue for 'user-space <->
> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> >>> context switches for ether(3) <-> ip(3).
> >
> > 9fans / 9fans / see discussions + participants + delivery options Permalink

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M591dbc5dea1a5be5ebe03e63
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

On Sun, 5 Jan 2025 at 16:39, Ron Minnich <rminnich@p9f.org> wrote:
> take that 2011 code, bring it to your plan 9 system, and see if

But https://github.com/rminnich/nxm has 410 commits after 2011? My
understanding was NIX and "NxM" aren't really different things, that
they can be understood as just a name change since their development
is entirely timewise-disjoint.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Me122206f05c7fe7a2a1395f7
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Jubal Biggs]

[-- Attachment #1: Type: text/plain, Size: 4641 bytes --]

I have a hookup at a company that builds modular compute nodes for DoD.
They make X86 pluggable compute devices the size of a credit card and lots
of types of clustering hardware for them. Each card is equivalent to a
typical desktop PC. I've used Plan 9 on these devices successfully in the
past. If we need to test out a more ambitious architecture with lots of
nodes, it's a very good way to experiment. For experimenting with clusters,
this kind of hardware can be extremely convenient. They are interested in a
more distributed OS/hypervisor for various applications, and I can get some
time with their engineers. Probably a small scale hardware donation and
larger scale deal on something.
Just let me know.
Link: https://addc.com/

On Sun, Jan 5, 2025 at 12:55 PM <tlaronde@kergis.com> wrote:

> On Sun, Jan 05, 2025 at 08:36:17AM -0800, Ron Minnich wrote:
> > No need for money yet!
> >
> > Let's get this party started. I have queries in to ampere as to how we
> > can set up a simulator. However, if someone wants to take a first
> > step, take that 2011 code, bring it to your plan 9 system, and see if
> > it builds.
> >
> > Again, the key here is a sustained effort. You don't have to do a lot
> > each week, but you don't want to start and then drop it. So it needs
> > to NOT become all consuming. It's all about pacing yourself. Anybody
> > who's ever spent a few weeks digging ditches can tell you -- set up a
> > work effort you can sustain. Same thing here.
> >
> > So, how about we figure out who here is interested, then start off:
> > get  the code, see if it builds. Who's in? Don't feel out of your
> > depth: if you can type mk, you're ready to start. Don't assume it's a
> > slog through code: take time to alternate looking at code, and reading
> > docs. Do learn how to use something like qemu -- it's a real
> > timesaver, since you can debug the kernel interactively.
> >
> > Don't kill yourself if you hit a wall about some code -- bring it
> > here, and ask questions. That's why we're here.
> >
> > So, Step 1: anyone? anyone?
> >
>
> OK me, FW I'm Worth (If I do work, my worthiness will undoubtely improve,
> but count it for a low value for now).
>
> T. Laronde
>
> >
> > On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> > <9fans@9fans.net> wrote:
> > >
> > > There have been other ideas in similar directions over the years.
> > > E.g.
> https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors
> about the concepts of ACs and CCs (communication cores).
> > >
> > >
> > > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com>
> wrote:
> > >>
> > >> i think brazil experimented with networking outside the kernel but it
> was pushed back in
> > >>
> > >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <
> tswoskowiak@gmail.com> wrote:
> > >>>
> > >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net>
> wrote:
> > >>> >
> > >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com>
> wrote:
> > >>> > >> This has been a very interesting discussion, thanks all. My
> offer
> > >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > >>> > >
> > >>> > > Am I the only one who sees that the Fastcall stuff would be good
> for
> > >>> > > bringing some devices out of the kernel (that are devs only for
> > >>> > > performance reasons)?
> > >>> > >
> > >>> > > And then, closer to what Fastcall was actually for (fossil and
> > >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could
> always do
> > >>> > > with a speedup.
> > >>> >
> > >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > >>> > call to a user level driver compared to a kernel based driver?
> > >>>
> > >>> From what I know the only performance issue for 'user-space <->
> > >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> > >>> context switches for ether(3) <-> ip(3).
> > >
> > > 9fans / 9fans / see discussions + participants + delivery options
> Permalink
> 
> --
> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>              http://www.kergis.com/
>             http://kertex.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M7dda62524bd4f50df786f87e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 7618 bytes --]

Re: [9fans] NIX experience

[ori]

not sure I have any devices so performance critical that I'd want
to dedicate a core to them. Benchmarks would be interseting.

Quoth Stuart Morrow <morrow.stuart@gmail.com>:
> > This has been a very interesting discussion, thanks all. My offer
> > remains: if anyone wants to revive NIX, I am happy to help.
> 
> Am I the only one who sees that the Fastcall stuff would be good for
> bringing some devices out of the kernel (that are devs only for
> performance reasons)?
> 
> And then, closer to what Fastcall was actually for (fossil and
> venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> with a speedup.
> 
> (If I'm wrong, what is it I'm missing?)

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M2e4563d78a5d99a5ddf3e11e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

On Sun, 5 Jan 2025 at 20:15, <ori@eigenstate.org> wrote:
> not sure I have any devices so performance critical that I'd want
> to dedicate a core to them. Benchmarks would be interseting.

The curried syscall stuff came before the execac stuff.

You were the one wanting userspace devdraw.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M39540462819778e0412e2211
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[ori]

Quoth Stuart Morrow <morrow.stuart@gmail.com>:
> On Sun, 5 Jan 2025 at 20:15, <ori@eigenstate.org> wrote:
> > not sure I have any devices so performance critical that I'd want
> > to dedicate a core to them. Benchmarks would be interseting.
> 
> The curried syscall stuff came before the execac stuff.
> 
> You were the one wanting userspace devdraw.
> 

I don't think moving devdraw to userspace today would be particularly painful;
I suspect a lot of people using devdraw today already do it via a userspace
exportfs over the network.


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Madc4b350b4903c457afbf0bb
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[sirjofri]

05.01.2025 21:48:24 Stuart Morrow <morrow.stuart@gmail.com>:

> On Sun, 5 Jan 2025 at 20:15, <ori@eigenstate.org> wrote:
>> not sure I have any devices so performance critical that I'd want
>> to dedicate a core to them. Benchmarks would be interseting.
>
> The curried syscall stuff came before the execac stuff.
>
> You were the one wanting userspace devdraw.

Working on some variation of it. The end goal would be, having detachable rio sessions.

sirjofri

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M752e99d66ec0fbf932e20631
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

I think it's ok to start with NIX, not NxM.

On Sun, Jan 5, 2025 at 10:45 AM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>
> On Sun, 5 Jan 2025 at 16:39, Ron Minnich <rminnich@p9f.org> wrote:
> > take that 2011 code, bring it to your plan 9 system, and see if
> 
> But https://github.com/rminnich/nxm has 410 commits after 2011? My
> understanding was NIX and "NxM" aren't really different things, that
> they can be understood as just a name change since their development
> is entirely timewise-disjoint.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M9e9ab3b48e8b4b22dce27685
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Christopher Nielsen]

[-- Attachment #1: Type: text/plain, Size: 3516 bytes --]

I'm interested.

Not 100% sure how much work I'll be able to do, but like you said, pace
yourself and be consistent. :-)

Cheers,
Chris

On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:

> No need for money yet!
>
> Let's get this party started. I have queries in to ampere as to how we
> can set up a simulator. However, if someone wants to take a first
> step, take that 2011 code, bring it to your plan 9 system, and see if
> it builds.
>
> Again, the key here is a sustained effort. You don't have to do a lot
> each week, but you don't want to start and then drop it. So it needs
> to NOT become all consuming. It's all about pacing yourself. Anybody
> who's ever spent a few weeks digging ditches can tell you -- set up a
> work effort you can sustain. Same thing here.
>
> So, how about we figure out who here is interested, then start off:
> get  the code, see if it builds. Who's in? Don't feel out of your
> depth: if you can type mk, you're ready to start. Don't assume it's a
> slog through code: take time to alternate looking at code, and reading
> docs. Do learn how to use something like qemu -- it's a real
> timesaver, since you can debug the kernel interactively.
>
> Don't kill yourself if you hit a wall about some code -- bring it
> here, and ask questions. That's why we're here.
>
> So, Step 1: anyone? anyone?
>
> Thanks
>
> Ron
>
> On Sun, Jan 5, 2025 at 7:22 AM Daniel Maslowski via 9fans
> <9fans@9fans.net> wrote:
> >
> > There have been other ideas in similar directions over the years.
> > E.g.
> https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors
> about the concepts of ACs and CCs (communication cores).
> >
> >
> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com>
> wrote:
> >>
> >> i think brazil experimented with networking outside the kernel but it
> was pushed back in
> >>
> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com>
> wrote:
> >>>
> >>> On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net>
> wrote:
> >>> >
> >>> > On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com>
> wrote:
> >>> > >> This has been a very interesting discussion, thanks all. My offer
> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> >>> > >
> >>> > > Am I the only one who sees that the Fastcall stuff would be good
> for
> >>> > > bringing some devices out of the kernel (that are devs only for
> >>> > > performance reasons)?
> >>> > >
> >>> > > And then, closer to what Fastcall was actually for (fossil and
> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always
> do
> >>> > > with a speedup.
> >>> >
> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> >>> > call to a user level driver compared to a kernel based driver?
> >>>
> >>> From what I know the only performance issue for 'user-space <->
> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> >>> context switches for ether(3) <-> ip(3).
> >
> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Me1851dafefdc22d8486190bd
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 6215 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

Do people have a preferred place to start from?

I'm inclined to something like this:
git@github.com:rminnich/nix-os.git

grab that, cd nix/sys/src/nix/k10
mk

and see how it goes. We need a shared place to record our experiences
-- suggestions?

I think our goal this week should be that we figure out who's working
on this; goal for end of next week is that everyone working on it at
least try to get that repo and do the mk and see if fail ;-)

Ori recently fixed 9front git so it can pull from github -- it was a
github bug ...



On Sun, Jan 5, 2025 at 7:57 PM Christopher Nielsen <cnielsen@pobox.com> wrote:
>
> I'm interested.
>
> Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
>
> Cheers,
> Chris
>
> On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
>>
>> No need for money yet!
>>
>> Let's get this party started. I have queries in to ampere as to how we
>> can set up a simulator. However, if someone wants to take a first
>> step, take that 2011 code, bring it to your plan 9 system, and see if
>> it builds.
>>
>> Again, the key here is a sustained effort. You don't have to do a lot
>> each week, but you don't want to start and then drop it. So it needs
>> to NOT become all consuming. It's all about pacing yourself. Anybody
>> who's ever spent a few weeks digging ditches can tell you -- set up a
>> work effort you can sustain. Same thing here.
>>
>> So, how about we figure out who here is interested, then start off:
>> get  the code, see if it builds. Who's in? Don't feel out of your
>> depth: if you can type mk, you're ready to start. Don't assume it's a
>> slog through code: take time to alternate looking at code, and reading
>> docs. Do learn how to use something like qemu -- it's a real
>> timesaver, since you can debug the kernel interactively.
>>
>> Don't kill yourself if you hit a wall about some code -- bring it
>> here, and ask questions. That's why we're here.
>>
>> So, Step 1: anyone? anyone?
>>
>> Thanks
>>
>> Ron
>>
>> On Sun, Jan 5, 2025 at 7:22 AM Daniel Maslowski via 9fans
>> <9fans@9fans.net> wrote:
>> >
>> > There have been other ideas in similar directions over the years.
>> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
>> >
>> >
>> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
>> >>
>> >> i think brazil experimented with networking outside the kernel but it was pushed back in
>> >>
>> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
>> >>>
>> >>> On Sat, Jan 4, 2025 at 1:03 PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
>> >>> >
>> >>> > On Jan 4, 2025, at 9:35 AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
>> >>> > >> This has been a very interesting discussion, thanks all. My offer
>> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
>> >>> > >
>> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
>> >>> > > bringing some devices out of the kernel (that are devs only for
>> >>> > > performance reasons)?
>> >>> > >
>> >>> > > And then, closer to what Fastcall was actually for (fossil and
>> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
>> >>> > > with a speedup.
>> >>> >
>> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
>> >>> > call to a user level driver compared to a kernel based driver?
>> >>>
>> >>> From what I know the only performance issue for 'user-space <->
>> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
>> >>> context switches for ether(3) <-> ip(3).
>> >
>> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M72745473ff2376aa96d457e9
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Sun, Jan 05, 2025 at 10:46:16PM -0800, Ron Minnich wrote:
> Do people have a preferred place to start from?
> 
> I'm inclined to something like this:
> git@github.com:rminnich/nix-os.git
> 
> grab that, cd nix/sys/src/nix/k10
> mk
> 
> and see how it goes. We need a shared place to record our experiences
> -- suggestions?
> 
> I think our goal this week should be that we figure out who's working
> on this; goal for end of next week is that everyone working on it at
> least try to get that repo and do the mk and see if fail ;-)
> 
> Ori recently fixed 9front git so it can pull from github -- it was a
> github bug ...

I'm OK with this scheme. I have decided to reserve some hours on
Saturdays and/or Sundays, to try/work with this.

> 
> On Sun, Jan 5, 2025 at 7:57?PM Christopher Nielsen <cnielsen@pobox.com> wrote:
> >
> > I'm interested.
> >
> > Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
> >
> > Cheers,
> > Chris
> >
> > On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
> >>
> >> No need for money yet!
> >>
> >> Let's get this party started. I have queries in to ampere as to how we
> >> can set up a simulator. However, if someone wants to take a first
> >> step, take that 2011 code, bring it to your plan 9 system, and see if
> >> it builds.
> >>
> >> Again, the key here is a sustained effort. You don't have to do a lot
> >> each week, but you don't want to start and then drop it. So it needs
> >> to NOT become all consuming. It's all about pacing yourself. Anybody
> >> who's ever spent a few weeks digging ditches can tell you -- set up a
> >> work effort you can sustain. Same thing here.
> >>
> >> So, how about we figure out who here is interested, then start off:
> >> get  the code, see if it builds. Who's in? Don't feel out of your
> >> depth: if you can type mk, you're ready to start. Don't assume it's a
> >> slog through code: take time to alternate looking at code, and reading
> >> docs. Do learn how to use something like qemu -- it's a real
> >> timesaver, since you can debug the kernel interactively.
> >>
> >> Don't kill yourself if you hit a wall about some code -- bring it
> >> here, and ask questions. That's why we're here.
> >>
> >> So, Step 1: anyone? anyone?
> >>
> >> Thanks
> >>
> >> Ron
> >>
> >> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> >> <9fans@9fans.net> wrote:
> >> >
> >> > There have been other ideas in similar directions over the years.
> >> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> >> >
> >> >
> >> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> >> >>
> >> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> >> >>
> >> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> >> >>>
> >> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> >> >>> >
> >> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >> >>> > >> This has been a very interesting discussion, thanks all. My offer
> >> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> >> >>> > >
> >> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> >> >>> > > bringing some devices out of the kernel (that are devs only for
> >> >>> > > performance reasons)?
> >> >>> > >
> >> >>> > > And then, closer to what Fastcall was actually for (fossil and
> >> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> >> >>> > > with a speedup.
> >> >>> >
> >> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> >> >>> > call to a user level driver compared to a kernel based driver?
> >> >>>
> >> >>> From what I know the only performance issue for 'user-space <->
> >> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> >> >>> context switches for ether(3) <-> ip(3).
> >> >
> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> >
> > 9fans / 9fans / see discussions + participants + delivery options Permalink

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M6737d80edd9e8e5f16ca3032
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

FYI, as of today, I can not git clone github.com/rminnich/nix-os with
the 9front git. I've asked Ori to take a look. Once I can do that,
I'll start a NOTES file -- I think Charles called his NOTES, or was it
Notes, Charles, either way, I'll follow your model :-), as a way to
record our progress. I'll put it at top level because ...
the big goal is to make the need for the nix-os repo go away, i.e. get
it upstreamd into something, but one thing at a time. First, let's see
if it works any more. The systems we used in 2011 were 4-socket AMD
...


On Mon, Jan 6, 2025 at 4:14 AM <tlaronde@kergis.com> wrote:
>
> On Sun, Jan 05, 2025 at 10:46:16PM -0800, Ron Minnich wrote:
> > Do people have a preferred place to start from?
> >
> > I'm inclined to something like this:
> > git@github.com:rminnich/nix-os.git
> >
> > grab that, cd nix/sys/src/nix/k10
> > mk
> >
> > and see how it goes. We need a shared place to record our experiences
> > -- suggestions?
> >
> > I think our goal this week should be that we figure out who's working
> > on this; goal for end of next week is that everyone working on it at
> > least try to get that repo and do the mk and see if fail ;-)
> >
> > Ori recently fixed 9front git so it can pull from github -- it was a
> > github bug ...
>
> I'm OK with this scheme. I have decided to reserve some hours on
> Saturdays and/or Sundays, to try/work with this.
>
> >
> > On Sun, Jan 5, 2025 at 7:57?PM Christopher Nielsen <cnielsen@pobox.com> wrote:
> > >
> > > I'm interested.
> > >
> > > Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
> > >
> > > Cheers,
> > > Chris
> > >
> > > On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
> > >>
> > >> No need for money yet!
> > >>
> > >> Let's get this party started. I have queries in to ampere as to how we
> > >> can set up a simulator. However, if someone wants to take a first
> > >> step, take that 2011 code, bring it to your plan 9 system, and see if
> > >> it builds.
> > >>
> > >> Again, the key here is a sustained effort. You don't have to do a lot
> > >> each week, but you don't want to start and then drop it. So it needs
> > >> to NOT become all consuming. It's all about pacing yourself. Anybody
> > >> who's ever spent a few weeks digging ditches can tell you -- set up a
> > >> work effort you can sustain. Same thing here.
> > >>
> > >> So, how about we figure out who here is interested, then start off:
> > >> get  the code, see if it builds. Who's in? Don't feel out of your
> > >> depth: if you can type mk, you're ready to start. Don't assume it's a
> > >> slog through code: take time to alternate looking at code, and reading
> > >> docs. Do learn how to use something like qemu -- it's a real
> > >> timesaver, since you can debug the kernel interactively.
> > >>
> > >> Don't kill yourself if you hit a wall about some code -- bring it
> > >> here, and ask questions. That's why we're here.
> > >>
> > >> So, Step 1: anyone? anyone?
> > >>
> > >> Thanks
> > >>
> > >> Ron
> > >>
> > >> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> > >> <9fans@9fans.net> wrote:
> > >> >
> > >> > There have been other ideas in similar directions over the years.
> > >> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> > >> >
> > >> >
> > >> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> > >> >>
> > >> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> > >> >>
> > >> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> > >> >>>
> > >> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> > >> >>> >
> > >> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > >> >>> > >> This has been a very interesting discussion, thanks all. My offer
> > >> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > >> >>> > >
> > >> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> > >> >>> > > bringing some devices out of the kernel (that are devs only for
> > >> >>> > > performance reasons)?
> > >> >>> > >
> > >> >>> > > And then, closer to what Fastcall was actually for (fossil and
> > >> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > >> >>> > > with a speedup.
> > >> >>> >
> > >> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > >> >>> > call to a user level driver compared to a kernel based driver?
> > >> >>>
> > >> >>> From what I know the only performance issue for 'user-space <->
> > >> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> > >> >>> context switches for ether(3) <-> ip(3).
> > >> >
> > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > >
> > > 9fans / 9fans / see discussions + participants + delivery options Permalink
> 
> --
> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>              http://www.kergis.com/
>             http://kertex.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M4d6393eb2b3a57bc03a8174d
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ben Huntsman]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

Hi guys-
   I don't mean to take too much of a tangent, but since the nix-os repo includes a pre-built copy of 9vx for OSX, has anyone looked at updating 9vx to be able to compile on newer Mac OS versions?  One of the big problems is that it's very 32-bit which Apple doesn't support compiling for anymore...

Thank you!

-Ben


------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mf02665786c2af2faa59dfd2e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 2460 bytes --]

Re: [9fans] NIX experience

[ori]

It's not just 32 bit, it depends on the quirks of i386
segmentation registers. It's never going to run on any
processor other than a 32-bit 386.

Quoth Ben Huntsman <ben@huntsmans.net>:
> Hi guys-
>    I don't mean to take too much of a tangent, but since the nix-os repo includes a pre-built copy of 9vx for OSX, has anyone looked at updating 9vx to be able to compile on newer Mac OS versions?  One of the big problems is that it's very 32-bit which Apple doesn't support compiling for anymore...
> 
> Thank you!
> 
> -Ben
> 

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mad33972a2fdb52bd2b3049d8
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[ori]

As far as I can tell, it's a github bug. I've opened
a discussion:

        https://github.com/orgs/community/discussions/148609


Quoth Ron Minnich <rminnich@p9f.org>:
> FYI, as of today, I can not git clone github.com/rminnich/nix-os with
> the 9front git. I've asked Ori to take a look. Once I can do that,
> I'll start a NOTES file -- I think Charles called his NOTES, or was it
> Notes, Charles, either way, I'll follow your model :-), as a way to
> record our progress. I'll put it at top level because ...
> the big goal is to make the need for the nix-os repo go away, i.e. get
> it upstreamd into something, but one thing at a time. First, let's see
> if it works any more. The systems we used in 2011 were 4-socket AMD
> ...
> 
> 
> On Mon, Jan 6, 2025 at 4:14 AM <tlaronde@kergis.com> wrote:
> >
> > On Sun, Jan 05, 2025 at 10:46:16PM -0800, Ron Minnich wrote:
> > > Do people have a preferred place to start from?
> > >
> > > I'm inclined to something like this:
> > > git@github.com:rminnich/nix-os.git
> > >
> > > grab that, cd nix/sys/src/nix/k10
> > > mk
> > >
> > > and see how it goes. We need a shared place to record our experiences
> > > -- suggestions?
> > >
> > > I think our goal this week should be that we figure out who's working
> > > on this; goal for end of next week is that everyone working on it at
> > > least try to get that repo and do the mk and see if fail ;-)
> > >
> > > Ori recently fixed 9front git so it can pull from github -- it was a
> > > github bug ...
> >
> > I'm OK with this scheme. I have decided to reserve some hours on
> > Saturdays and/or Sundays, to try/work with this.
> >
> > >
> > > On Sun, Jan 5, 2025 at 7:57?PM Christopher Nielsen <cnielsen@pobox.com> wrote:
> > > >
> > > > I'm interested.
> > > >
> > > > Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
> > > >
> > > > Cheers,
> > > > Chris
> > > >
> > > > On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
> > > >>
> > > >> No need for money yet!
> > > >>
> > > >> Let's get this party started. I have queries in to ampere as to how we
> > > >> can set up a simulator. However, if someone wants to take a first
> > > >> step, take that 2011 code, bring it to your plan 9 system, and see if
> > > >> it builds.
> > > >>
> > > >> Again, the key here is a sustained effort. You don't have to do a lot
> > > >> each week, but you don't want to start and then drop it. So it needs
> > > >> to NOT become all consuming. It's all about pacing yourself. Anybody
> > > >> who's ever spent a few weeks digging ditches can tell you -- set up a
> > > >> work effort you can sustain. Same thing here.
> > > >>
> > > >> So, how about we figure out who here is interested, then start off:
> > > >> get  the code, see if it builds. Who's in? Don't feel out of your
> > > >> depth: if you can type mk, you're ready to start. Don't assume it's a
> > > >> slog through code: take time to alternate looking at code, and reading
> > > >> docs. Do learn how to use something like qemu -- it's a real
> > > >> timesaver, since you can debug the kernel interactively.
> > > >>
> > > >> Don't kill yourself if you hit a wall about some code -- bring it
> > > >> here, and ask questions. That's why we're here.
> > > >>
> > > >> So, Step 1: anyone? anyone?
> > > >>
> > > >> Thanks
> > > >>
> > > >> Ron
> > > >>
> > > >> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> > > >> <9fans@9fans.net> wrote:
> > > >> >
> > > >> > There have been other ideas in similar directions over the years.
> > > >> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> > > >> >
> > > >> >
> > > >> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> > > >> >>
> > > >> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> > > >> >>
> > > >> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> > > >> >>>
> > > >> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> > > >> >>> >
> > > >> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > > >> >>> > >> This has been a very interesting discussion, thanks all. My offer
> > > >> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > > >> >>> > >
> > > >> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> > > >> >>> > > bringing some devices out of the kernel (that are devs only for
> > > >> >>> > > performance reasons)?
> > > >> >>> > >
> > > >> >>> > > And then, closer to what Fastcall was actually for (fossil and
> > > >> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > > >> >>> > > with a speedup.
> > > >> >>> >
> > > >> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > > >> >>> > call to a user level driver compared to a kernel based driver?
> > > >> >>>
> > > >> >>> From what I know the only performance issue for 'user-space <->
> > > >> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> > > >> >>> context switches for ether(3) <-> ip(3).
> > > >> >
> > > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > > >
> > > > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > 
> > --
> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >              http://www.kergis.com/
> >             http://kertex.kergis.com/
> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Ma1120663efa9b8e1db9f65bb
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ben Huntsman]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

So basically there's no getting 9vx to work on modern Mac OS (on Intel, I don't mean ARM)?

That's truly unfortunate.
________________________________
From: ori@eigenstate.org <ori@eigenstate.org>
Sent: Monday, January 6, 2025 3:02 PM
To: 9fans@9fans.net <9fans@9fans.net>
Subject: Re: [9fans] NIX experience

It's not just 32 bit, it depends on the quirks of i386
segmentation registers. It's never going to run on any
processor other than a 32-bit 386.

Quoth Ben Huntsman <ben@huntsmans.net>:
> Hi guys-
>    I don't mean to take too much of a tangent, but since the nix-os repo includes a pre-built copy of 9vx for OSX, has anyone looked at updating 9vx to be able to compile on newer Mac OS versions?  One of the big problems is that it's very 32-bit which Apple doesn't support compiling for anymore...
>
> Thank you!
>
> -Ben
>

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M529b1d99c51da47a856e8621
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 3197 bytes --]

Re: [9fans] NIX experience

[ori]

I forgot to say -- cloning over ssh is reliable for me, so
if you set up ssh keys on github, that should be sufficient
for now.

Quoth ori@eigenstate.org:
> As far as I can tell, it's a github bug. I've opened
> a discussion:
> 
>         https://github.com/orgs/community/discussions/148609
> 
> 
> Quoth Ron Minnich <rminnich@p9f.org>:
> > FYI, as of today, I can not git clone github.com/rminnich/nix-os with
> > the 9front git. I've asked Ori to take a look. Once I can do that,
> > I'll start a NOTES file -- I think Charles called his NOTES, or was it
> > Notes, Charles, either way, I'll follow your model :-), as a way to
> > record our progress. I'll put it at top level because ...
> > the big goal is to make the need for the nix-os repo go away, i.e. get
> > it upstreamd into something, but one thing at a time. First, let's see
> > if it works any more. The systems we used in 2011 were 4-socket AMD
> > ...
> > 
> > 
> > On Mon, Jan 6, 2025 at 4:14 AM <tlaronde@kergis.com> wrote:
> > >
> > > On Sun, Jan 05, 2025 at 10:46:16PM -0800, Ron Minnich wrote:
> > > > Do people have a preferred place to start from?
> > > >
> > > > I'm inclined to something like this:
> > > > git@github.com:rminnich/nix-os.git
> > > >
> > > > grab that, cd nix/sys/src/nix/k10
> > > > mk
> > > >
> > > > and see how it goes. We need a shared place to record our experiences
> > > > -- suggestions?
> > > >
> > > > I think our goal this week should be that we figure out who's working
> > > > on this; goal for end of next week is that everyone working on it at
> > > > least try to get that repo and do the mk and see if fail ;-)
> > > >
> > > > Ori recently fixed 9front git so it can pull from github -- it was a
> > > > github bug ...
> > >
> > > I'm OK with this scheme. I have decided to reserve some hours on
> > > Saturdays and/or Sundays, to try/work with this.
> > >
> > > >
> > > > On Sun, Jan 5, 2025 at 7:57?PM Christopher Nielsen <cnielsen@pobox.com> wrote:
> > > > >
> > > > > I'm interested.
> > > > >
> > > > > Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
> > > > >
> > > > > Cheers,
> > > > > Chris
> > > > >
> > > > > On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
> > > > >>
> > > > >> No need for money yet!
> > > > >>
> > > > >> Let's get this party started. I have queries in to ampere as to how we
> > > > >> can set up a simulator. However, if someone wants to take a first
> > > > >> step, take that 2011 code, bring it to your plan 9 system, and see if
> > > > >> it builds.
> > > > >>
> > > > >> Again, the key here is a sustained effort. You don't have to do a lot
> > > > >> each week, but you don't want to start and then drop it. So it needs
> > > > >> to NOT become all consuming. It's all about pacing yourself. Anybody
> > > > >> who's ever spent a few weeks digging ditches can tell you -- set up a
> > > > >> work effort you can sustain. Same thing here.
> > > > >>
> > > > >> So, how about we figure out who here is interested, then start off:
> > > > >> get  the code, see if it builds. Who's in? Don't feel out of your
> > > > >> depth: if you can type mk, you're ready to start. Don't assume it's a
> > > > >> slog through code: take time to alternate looking at code, and reading
> > > > >> docs. Do learn how to use something like qemu -- it's a real
> > > > >> timesaver, since you can debug the kernel interactively.
> > > > >>
> > > > >> Don't kill yourself if you hit a wall about some code -- bring it
> > > > >> here, and ask questions. That's why we're here.
> > > > >>
> > > > >> So, Step 1: anyone? anyone?
> > > > >>
> > > > >> Thanks
> > > > >>
> > > > >> Ron
> > > > >>
> > > > >> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> > > > >> <9fans@9fans.net> wrote:
> > > > >> >
> > > > >> > There have been other ideas in similar directions over the years.
> > > > >> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> > > > >> >
> > > > >> >
> > > > >> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> > > > >> >>
> > > > >> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> > > > >> >>
> > > > >> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> > > > >> >>>
> > > > >> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> > > > >> >>> >
> > > > >> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > > > >> >>> > >> This has been a very interesting discussion, thanks all. My offer
> > > > >> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > > > >> >>> > >
> > > > >> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> > > > >> >>> > > bringing some devices out of the kernel (that are devs only for
> > > > >> >>> > > performance reasons)?
> > > > >> >>> > >
> > > > >> >>> > > And then, closer to what Fastcall was actually for (fossil and
> > > > >> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > > > >> >>> > > with a speedup.
> > > > >> >>> >
> > > > >> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > > > >> >>> > call to a user level driver compared to a kernel based driver?
> > > > >> >>>
> > > > >> >>> From what I know the only performance issue for 'user-space <->
> > > > >> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> > > > >> >>> context switches for ether(3) <-> ip(3).
> > > > >> >
> > > > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > > > >
> > > > > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > > 
> > > --
> > > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> > >              http://www.kergis.com/
> > >             http://kertex.kergis.com/
> > > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mcf1fbce5a671a7fb0af29924
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

yep, I set up ssh keys, git pull is fine, next step is try to figure
out how to build it again.

ron

On Mon, Jan 6, 2025 at 3:25 PM <ori@eigenstate.org> wrote:
>
> I forgot to say -- cloning over ssh is reliable for me, so
> if you set up ssh keys on github, that should be sufficient
> for now.
>
> Quoth ori@eigenstate.org:
> > As far as I can tell, it's a github bug. I've opened
> > a discussion:
> >
> >         https://github.com/orgs/community/discussions/148609
> >
> >
> > Quoth Ron Minnich <rminnich@p9f.org>:
> > > FYI, as of today, I can not git clone github.com/rminnich/nix-os with
> > > the 9front git. I've asked Ori to take a look. Once I can do that,
> > > I'll start a NOTES file -- I think Charles called his NOTES, or was it
> > > Notes, Charles, either way, I'll follow your model :-), as a way to
> > > record our progress. I'll put it at top level because ...
> > > the big goal is to make the need for the nix-os repo go away, i.e. get
> > > it upstreamd into something, but one thing at a time. First, let's see
> > > if it works any more. The systems we used in 2011 were 4-socket AMD
> > > ...
> > >
> > >
> > > On Mon, Jan 6, 2025 at 4:14 AM <tlaronde@kergis.com> wrote:
> > > >
> > > > On Sun, Jan 05, 2025 at 10:46:16PM -0800, Ron Minnich wrote:
> > > > > Do people have a preferred place to start from?
> > > > >
> > > > > I'm inclined to something like this:
> > > > > git@github.com:rminnich/nix-os.git
> > > > >
> > > > > grab that, cd nix/sys/src/nix/k10
> > > > > mk
> > > > >
> > > > > and see how it goes. We need a shared place to record our experiences
> > > > > -- suggestions?
> > > > >
> > > > > I think our goal this week should be that we figure out who's working
> > > > > on this; goal for end of next week is that everyone working on it at
> > > > > least try to get that repo and do the mk and see if fail ;-)
> > > > >
> > > > > Ori recently fixed 9front git so it can pull from github -- it was a
> > > > > github bug ...
> > > >
> > > > I'm OK with this scheme. I have decided to reserve some hours on
> > > > Saturdays and/or Sundays, to try/work with this.
> > > >
> > > > >
> > > > > On Sun, Jan 5, 2025 at 7:57?PM Christopher Nielsen <cnielsen@pobox.com> wrote:
> > > > > >
> > > > > > I'm interested.
> > > > > >
> > > > > > Not 100% sure how much work I'll be able to do, but like you said, pace yourself and be consistent. :-)
> > > > > >
> > > > > > Cheers,
> > > > > > Chris
> > > > > >
> > > > > > On Sun, Jan 5, 2025, 08:39 Ron Minnich <rminnich@p9f.org> wrote:
> > > > > >>
> > > > > >> No need for money yet!
> > > > > >>
> > > > > >> Let's get this party started. I have queries in to ampere as to how we
> > > > > >> can set up a simulator. However, if someone wants to take a first
> > > > > >> step, take that 2011 code, bring it to your plan 9 system, and see if
> > > > > >> it builds.
> > > > > >>
> > > > > >> Again, the key here is a sustained effort. You don't have to do a lot
> > > > > >> each week, but you don't want to start and then drop it. So it needs
> > > > > >> to NOT become all consuming. It's all about pacing yourself. Anybody
> > > > > >> who's ever spent a few weeks digging ditches can tell you -- set up a
> > > > > >> work effort you can sustain. Same thing here.
> > > > > >>
> > > > > >> So, how about we figure out who here is interested, then start off:
> > > > > >> get  the code, see if it builds. Who's in? Don't feel out of your
> > > > > >> depth: if you can type mk, you're ready to start. Don't assume it's a
> > > > > >> slog through code: take time to alternate looking at code, and reading
> > > > > >> docs. Do learn how to use something like qemu -- it's a real
> > > > > >> timesaver, since you can debug the kernel interactively.
> > > > > >>
> > > > > >> Don't kill yourself if you hit a wall about some code -- bring it
> > > > > >> here, and ask questions. That's why we're here.
> > > > > >>
> > > > > >> So, Step 1: anyone? anyone?
> > > > > >>
> > > > > >> Thanks
> > > > > >>
> > > > > >> Ron
> > > > > >>
> > > > > >> On Sun, Jan 5, 2025 at 7:22?AM Daniel Maslowski via 9fans
> > > > > >> <9fans@9fans.net> wrote:
> > > > > >> >
> > > > > >> > There have been other ideas in similar directions over the years.
> > > > > >> > E.g. https://www.researchgate.net/publication/342759611_SCE-Comm_A_Real-Time_Inter-Core_Communication_Framework_for_Strictly_Partitioned_Multi-core_Processors about the concepts of ACs and CCs (communication cores).
> > > > > >> >
> > > > > >> >
> > > > > >> > On Sun, 5 Jan 2025, 01:49 Charles Forsyth, <charles.forsyth@gmail.com> wrote:
> > > > > >> >>
> > > > > >> >> i think brazil experimented with networking outside the kernel but it was pushed back in
> > > > > >> >>
> > > > > >> >> On Sun, 5 Jan 2025 at 00:24, Thaddeus Woskowiak <tswoskowiak@gmail.com> wrote:
> > > > > >> >>>
> > > > > >> >>> On Sat, Jan 4, 2025 at 1:03?PM Bakul Shah via 9fans <9fans@9fans.net> wrote:
> > > > > >> >>> >
> > > > > >> >>> > On Jan 4, 2025, at 9:35?AM, Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > > > > >> >>> > >> This has been a very interesting discussion, thanks all. My offer
> > > > > >> >>> > >> remains: if anyone wants to revive NIX, I am happy to help.
> > > > > >> >>> > >
> > > > > >> >>> > > Am I the only one who sees that the Fastcall stuff would be good for
> > > > > >> >>> > > bringing some devices out of the kernel (that are devs only for
> > > > > >> >>> > > performance reasons)?
> > > > > >> >>> > >
> > > > > >> >>> > > And then, closer to what Fastcall was actually for (fossil and
> > > > > >> >>> > > venti>disk), you also have ??fs>nusb/disk>disk, which could always do
> > > > > >> >>> > > with a speedup.
> > > > > >> >>> >
> > > > > >> >>> > I've been meaning to ask... What is the typical *overhead* of a 9p
> > > > > >> >>> > call to a user level driver compared to a kernel based driver?
> > > > > >> >>>
> > > > > >> >>> From what I know the only performance issue for 'user-space <->
> > > > > >> >>> kernel-space' 9P are context switches. IP is in-kernel to eliminate
> > > > > >> >>> context switches for ether(3) <-> ip(3).
> > > > > >> >
> > > > > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > > > > >
> > > > > > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > > >
> > > > --
> > > > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> > > >              http://www.kergis.com/
> > > >             http://kertex.kergis.com/
> > > > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mc174d8e4da9477d0c8efa6d1
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

On Tue, 7 Jan 2025 at 04:24, Ben Huntsman <ben@huntsmans.net> wrote:
>
> So basically there's no getting 9vx to work on modern Mac OS (on Intel, I don't mean ARM)?
>
> That's truly unfortunate.

BoxedWINE exists and works, so I don't see that there can't be a "9pcemu".

And yes, "unfortunate" is right - drawterm exists only as a concession
to the (unfortunate)fact that a user-level operating system is a hard
thing to make and is always unportable.

If you don't believe me then look at how there's never been a drawterm
for Inferno - because Inferno already had a user-level operating
system.

"drawterm has always been a clever hack" - quanstro 2011

It's a rump kernel for Plan 9.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M0ca7e19f405d608863a6b05e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Stuart Morrow]

But what about the name collision with the other Nix OS?

On Mon, 6 Jan 2025 at 03:57, Ron Minnich <rminnich@p9f.org> wrote:
>
> I think it's ok to start with NIX, not NxM.
>
> On Sun, Jan 5, 2025 at 10:45 AM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >
> > On Sun, 5 Jan 2025 at 16:39, Ron Minnich <rminnich@p9f.org> wrote:
> > > take that 2011 code, bring it to your plan 9 system, and see if
> >
> > But https://github.com/rminnich/nxm has 410 commits after 2011? My
> > understanding was NIX and "NxM" aren't really different things, that
> > they can be understood as just a name change since their development
> > is entirely timewise-disjoint.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Ma232c7650da55af5b6665cfc
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

That  name collision question re Nix-OS came up in 2011. I talked to
some folks, they more or less said "don't worry about the name", so I
decided not to.

The name NxM was intended to mean "N kernel cores, M application
cores" -- i.e. to be evocative of the NIX model. We probably could
have called it KxA, but ...

I've looked at the nix tree at github.com/rminnich/nix-os. It had
significant bit rot from 2011-2012, with some half-done experiments.

I added a new branch, first_commit, at the first commit, because the
state of the master branch was ... a mess. The README event referred
to a file that had been removed.

The changes we made from Plan 9 to NIX were not large, and I suspect
porting the first_commit to 9front or other kernel would be easier.

Right now it won't build, but if you look in sys/src/nix/k8 you'll
find pre-built kernels that might work in qemu. The 9front vmx command
won't boot them, they are valid multiboot images but vmx does not
think so. This might be a bug in vmx.

Why NIX?

If you think about it, timesharing is designed for a world where cores
are scarce. But on a machine with hundreds of cores, running Plan 9,
there are < 100 processes. We can assign a core to each process, and
let those processes own the core until they are done. This might be a
useful simplification, it might not, but it's something.

I did run some standard HPC benchmarks on NIX ACs and the results were
good. I was always curious how it would work if we had those
multi-hundred-core machines Intel and IBM and others were telling us
about in 2011. Now that we have them, it would be interesting to try.

Fun fact: when the Power-9 came in to Google, and it had more cores
than bits in a 64-bit word, it caused some angst. It got worked out,
but nobody had really considered that problem. Hardware events
overtook us all. I believe there are similar bitmask issues in Plan 9,
I guess we'll see.

ron

On Tue, Jan 7, 2025 at 4:03 AM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>
> But what about the name collision with the other Nix OS?
>
> On Mon, 6 Jan 2025 at 03:57, Ron Minnich <rminnich@p9f.org> wrote:
> >
> > I think it's ok to start with NIX, not NxM.
> >
> > On Sun, Jan 5, 2025 at 10:45 AM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > >
> > > On Sun, 5 Jan 2025 at 16:39, Ron Minnich <rminnich@p9f.org> wrote:
> > > > take that 2011 code, bring it to your plan 9 system, and see if
> > >
> > > But https://github.com/rminnich/nxm has 410 commits after 2011? My
> > > understanding was NIX and "NxM" aren't really different things, that
> > > they can be understood as just a name change since their development
> > > is entirely timewise-disjoint.

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Md05ec18df1fcd5ff02465148
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[tlaronde]

On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> 
> Why NIX?
> 
> If you think about it, timesharing is designed for a world where cores
> are scarce. But on a machine with hundreds of cores, running Plan 9,
> there are < 100 processes. We can assign a core to each process, and
> let those processes own the core until they are done. This might be a
> useful simplification, it might not, but it's something.
> 
> I did run some standard HPC benchmarks on NIX ACs and the results were
> good. I was always curious how it would work if we had those
> multi-hundred-core machines Intel and IBM and others were telling us
> about in 2011. Now that we have them, it would be interesting to try.

As said previously, I will start wandering and stumbling upon problems
this week-end---I'm a toddler in the area, so it's the way to learn to
walk.

But this brief summary highlight a solution and questions
that are, IMHO, valid questions: remember the "war" between
"micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
separate process (well: there are "administrative" processes,
scheduler and pager but...) but part of the applications. This is also
why it is efficient compared to "message passing" micro-kernels that
are not "near" enough the hardware---so inefficient that, for
ideologic purposes, some have rewritten "micro-kernels" in assembly to
improve the result...

But multiple cores (and even in the smaller machines nowadays, you
find two) present another mean of articulation of the OS code (the
MMU is central for me in the whole picture: not move the data
around, but change the view of the shared data per core). The question
is at least certainly worth asking.

-- 
        Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
                     http://www.kergis.com/
                    http://kertex.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M5aff003c96bbc12fe1554c0d
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

I found the original 2011 paper, which was a sandia report, from may
2011. It's a modification of the original proposal, which I no longer
have; but it is a good summary of where we were at the end of my visit
in May.

This is interesting: "We have changed a surprisingly small amount of
code at this point.
There are about 400 lines of new
assembler source, about 80 lines of platform independent C source, and
about 350 lines of AMD64 C
source code. To this, we have to add a few extra source lines in the
start-up code, system call, and trap han-
dlers. This implementation is being both developed and tested only in
the AMD64 architecture."

I uploaded it to the Plan 9 foundation shared drive:
https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link

On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>
> On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> >
> > Why NIX?
> >
> > If you think about it, timesharing is designed for a world where cores
> > are scarce. But on a machine with hundreds of cores, running Plan 9,
> > there are < 100 processes. We can assign a core to each process, and
> > let those processes own the core until they are done. This might be a
> > useful simplification, it might not, but it's something.
> >
> > I did run some standard HPC benchmarks on NIX ACs and the results were
> > good. I was always curious how it would work if we had those
> > multi-hundred-core machines Intel and IBM and others were telling us
> > about in 2011. Now that we have them, it would be interesting to try.
> 
> As said previously, I will start wandering and stumbling upon problems
> this week-end---I'm a toddler in the area, so it's the way to learn to
> walk.
> 
> But this brief summary highlight a solution and questions
> that are, IMHO, valid questions: remember the "war" between
> "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
> separate process (well: there are "administrative" processes,
> scheduler and pager but...) but part of the applications. This is also
> why it is efficient compared to "message passing" micro-kernels that
> are not "near" enough the hardware---so inefficient that, for
> ideologic purposes, some have rewritten "micro-kernels" in assembly to
> improve the result...
> 
> But multiple cores (and even in the smaller machines nowadays, you
> find two) present another mean of articulation of the OS code (the
> MMU is central for me in the whole picture: not move the data
> around, but change the view of the shared data per core). The question
> is at least certainly worth asking.
> 
> --
> Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>              http://www.kergis.com/
>             http://kertex.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M466cceae3c3bc516f71a077b
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 3950 bytes --]

Ok, I thought, what could do.

So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and
hit "mk".
When that errored out a bunch I realized that I needed /amd64 built, so I
did that.  Just as painless as I remembered.

And now, I get a ways further into the build, but hit an incompatibility
between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.
It seems that at some point since the NIX code was written someone decided
that the program counter should be called pc instead of ip.

Or else, I'm approaching this all wrong, and Ron can shed some light on how
I should be proceeding.

Paul

On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:

> I found the original 2011 paper, which was a sandia report, from may
> 2011. It's a modification of the original proposal, which I no longer
> have; but it is a good summary of where we were at the end of my visit
> in May.
>
> This is interesting: "We have changed a surprisingly small amount of
> code at this point.
> There are about 400 lines of new
> assembler source, about 80 lines of platform independent C source, and
> about 350 lines of AMD64 C
> source code. To this, we have to add a few extra source lines in the
> start-up code, system call, and trap han-
> dlers. This implementation is being both developed and tested only in
> the AMD64 architecture."
>
> I uploaded it to the Plan 9 foundation shared drive:
>
> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>
> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> >
> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> > >
> > > Why NIX?
> > >
> > > If you think about it, timesharing is designed for a world where cores
> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
> > > there are < 100 processes. We can assign a core to each process, and
> > > let those processes own the core until they are done. This might be a
> > > useful simplification, it might not, but it's something.
> > >
> > > I did run some standard HPC benchmarks on NIX ACs and the results were
> > > good. I was always curious how it would work if we had those
> > > multi-hundred-core machines Intel and IBM and others were telling us
> > > about in 2011. Now that we have them, it would be interesting to try.
> >
> > As said previously, I will start wandering and stumbling upon problems
> > this week-end---I'm a toddler in the area, so it's the way to learn to
> > walk.
> >
> > But this brief summary highlight a solution and questions
> > that are, IMHO, valid questions: remember the "war" between
> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
> > separate process (well: there are "administrative" processes,
> > scheduler and pager but...) but part of the applications. This is also
> > why it is efficient compared to "message passing" micro-kernels that
> > are not "near" enough the hardware---so inefficient that, for
> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
> > improve the result...
> >
> > But multiple cores (and even in the smaller machines nowadays, you
> > find two) present another mean of articulation of the OS code (the
> > MMU is central for me in the whole picture: not move the data
> > around, but change the view of the shared data per core). The question
> > is at least certainly worth asking.
> >
> > --
> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >              http://www.kergis.com/
> >             http://kertex.kergis.com/
> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M4d83bfc77a21f3e907bcfc2e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 6221 bytes --]

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 4380 bytes --]

And a bit more digging.  Yes, I'm clearly doing this wrong.  In building
nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure
from nix, not the one in the host system.

How do I re-root this correctly for this build?

Paul

On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com>
wrote:

> Ok, I thought, what could do.
>
> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and
> hit "mk".
> When that errored out a bunch I realized that I needed /amd64 built, so I
> did that.  Just as painless as I remembered.
>
> And now, I get a ways further into the build, but hit an incompatibility
> between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.
> It seems that at some point since the NIX code was written someone decided
> that the program counter should be called pc instead of ip.
>
> Or else, I'm approaching this all wrong, and Ron can shed some light on
> how I should be proceeding.
>
> Paul
>
> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>
>> I found the original 2011 paper, which was a sandia report, from may
>> 2011. It's a modification of the original proposal, which I no longer
>> have; but it is a good summary of where we were at the end of my visit
>> in May.
>>
>> This is interesting: "We have changed a surprisingly small amount of
>> code at this point.
>> There are about 400 lines of new
>> assembler source, about 80 lines of platform independent C source, and
>> about 350 lines of AMD64 C
>> source code. To this, we have to add a few extra source lines in the
>> start-up code, system call, and trap han-
>> dlers. This implementation is being both developed and tested only in
>> the AMD64 architecture."
>>
>> I uploaded it to the Plan 9 foundation shared drive:
>>
>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>>
>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>> >
>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
>> > >
>> > > Why NIX?
>> > >
>> > > If you think about it, timesharing is designed for a world where cores
>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
>> > > there are < 100 processes. We can assign a core to each process, and
>> > > let those processes own the core until they are done. This might be a
>> > > useful simplification, it might not, but it's something.
>> > >
>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
>> > > good. I was always curious how it would work if we had those
>> > > multi-hundred-core machines Intel and IBM and others were telling us
>> > > about in 2011. Now that we have them, it would be interesting to try.
>> >
>> > As said previously, I will start wandering and stumbling upon problems
>> > this week-end---I'm a toddler in the area, so it's the way to learn to
>> > walk.
>> >
>> > But this brief summary highlight a solution and questions
>> > that are, IMHO, valid questions: remember the "war" between
>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
>> > separate process (well: there are "administrative" processes,
>> > scheduler and pager but...) but part of the applications. This is also
>> > why it is efficient compared to "message passing" micro-kernels that
>> > are not "near" enough the hardware---so inefficient that, for
>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
>> > improve the result...
>> >
>> > But multiple cores (and even in the smaller machines nowadays, you
>> > find two) present another mean of articulation of the OS code (the
>> > MMU is central for me in the whole picture: not move the data
>> > around, but change the view of the shared data per core). The question
>> > is at least certainly worth asking.
>> >
>> > --
>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>> >              http://www.kergis.com/
>> >             http://kertex.kergis.com/
>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M90261d7d07b2ef0e41bd249e
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 6920 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

if you look at the first_commit branch, you'll see a sys/src/nix/nix
script, which sets up some binds.

What we did, before building nix, on plan 9, in 2011, was a set of
binds to get the right things such as /sys/include and so on.

This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
I expect it will be strategic changes, and in the end they won't be
all that many lines of code, but there will be some tricky stuff.

Best ot take it slow, when you hit an issue, ruminate it on for a day
or two, then look again. Otherwise you'll just get frustrated (I have
...) But before you make any change, be very sure you know WHY you're
doing it, not just that 'it got me past that mk error.'

Bring issues to the list and, if you want, keep a running doc to which
others can contribute: what you did, what you ran into, what a fix
might be. The old saying; "if you don't write it down it didn't
happen"

But this is the kind of thing you take slowly and carefully, otherwise
it's total misery.

ron

On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>
> And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
>
> How do I re-root this correctly for this build?
>
> Paul
>
> On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>>
>> Ok, I thought, what could do.
>>
>> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
>> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
>>
>> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
>>
>> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
>>
>> Paul
>>
>> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>>>
>>> I found the original 2011 paper, which was a sandia report, from may
>>> 2011. It's a modification of the original proposal, which I no longer
>>> have; but it is a good summary of where we were at the end of my visit
>>> in May.
>>>
>>> This is interesting: "We have changed a surprisingly small amount of
>>> code at this point.
>>> There are about 400 lines of new
>>> assembler source, about 80 lines of platform independent C source, and
>>> about 350 lines of AMD64 C
>>> source code. To this, we have to add a few extra source lines in the
>>> start-up code, system call, and trap han-
>>> dlers. This implementation is being both developed and tested only in
>>> the AMD64 architecture."
>>>
>>> I uploaded it to the Plan 9 foundation shared drive:
>>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>>>
>>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>>> >
>>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
>>> > >
>>> > > Why NIX?
>>> > >
>>> > > If you think about it, timesharing is designed for a world where cores
>>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
>>> > > there are < 100 processes. We can assign a core to each process, and
>>> > > let those processes own the core until they are done. This might be a
>>> > > useful simplification, it might not, but it's something.
>>> > >
>>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
>>> > > good. I was always curious how it would work if we had those
>>> > > multi-hundred-core machines Intel and IBM and others were telling us
>>> > > about in 2011. Now that we have them, it would be interesting to try.
>>> >
>>> > As said previously, I will start wandering and stumbling upon problems
>>> > this week-end---I'm a toddler in the area, so it's the way to learn to
>>> > walk.
>>> >
>>> > But this brief summary highlight a solution and questions
>>> > that are, IMHO, valid questions: remember the "war" between
>>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
>>> > separate process (well: there are "administrative" processes,
>>> > scheduler and pager but...) but part of the applications. This is also
>>> > why it is efficient compared to "message passing" micro-kernels that
>>> > are not "near" enough the hardware---so inefficient that, for
>>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
>>> > improve the result...
>>> >
>>> > But multiple cores (and even in the smaller machines nowadays, you
>>> > find two) present another mean of articulation of the OS code (the
>>> > MMU is central for me in the whole picture: not move the data
>>> > around, but change the view of the shared data per core). The question
>>> > is at least certainly worth asking.
>>> >
>>> > --
>>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>>> >              http://www.kergis.com/
>>> >             http://kertex.kergis.com/
>>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mbea0cdf9396096230312b459
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1: Type: text/plain, Size: 6978 bytes --]

Ok, not a bad first day poking at it.  I have a growing (but not ready) new
nix script to pull the right pieces over top of my build environment.
I have a near-complete build, but with hazards: 9front has evolved in a
number of places with many ulong parameters becoming usize.  I have a list
of those spots, but now they need to be examined for over/underflow.
The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes
the libboot.a6 binary, some source files that match the symbols, and no
mkfile.  Attempting to build also shows some 9front auth changes that need
to be incorporated into doauthenticate.c, calls to convS2M and convM2S that
now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing
at all insurmountable.

Not too daunting.  Next time I have a few moments I'll do a more principled
pass on the nix script so I can share it.  I didn't understand enough when
I first started updating it.

Paul


On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:

> if you look at the first_commit branch, you'll see a sys/src/nix/nix
> script, which sets up some binds.
>
> What we did, before building nix, on plan 9, in 2011, was a set of
> binds to get the right things such as /sys/include and so on.
>
> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
> I expect it will be strategic changes, and in the end they won't be
> all that many lines of code, but there will be some tricky stuff.
>
> Best ot take it slow, when you hit an issue, ruminate it on for a day
> or two, then look again. Otherwise you'll just get frustrated (I have
> ...) But before you make any change, be very sure you know WHY you're
> doing it, not just that 'it got me past that mk error.'
>
> Bring issues to the list and, if you want, keep a running doc to which
> others can contribute: what you did, what you ran into, what a fix
> might be. The old saying; "if you don't write it down it didn't
> happen"
>
> But this is the kind of thing you take slowly and carefully, otherwise
> it's total misery.
>
> ron
>
> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >
> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building
> nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure
> from nix, not the one in the host system.
> >
> > How do I re-root this correctly for this build?
> >
> > Paul
> >
> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >>
> >> Ok, I thought, what could do.
> >>
> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo
> and hit "mk".
> >> When that errored out a bunch I realized that I needed /amd64 built, so
> I did that.  Just as painless as I remembered.
> >>
> >> And now, I get a ways further into the build, but hit an
> incompatibility between the my /amd64/include/ureg.h and
> .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX
> code was written someone decided that the program counter should be called
> pc instead of ip.
> >>
> >> Or else, I'm approaching this all wrong, and Ron can shed some light on
> how I should be proceeding.
> >>
> >> Paul
> >>
> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> >>>
> >>> I found the original 2011 paper, which was a sandia report, from may
> >>> 2011. It's a modification of the original proposal, which I no longer
> >>> have; but it is a good summary of where we were at the end of my visit
> >>> in May.
> >>>
> >>> This is interesting: "We have changed a surprisingly small amount of
> >>> code at this point.
> >>> There are about 400 lines of new
> >>> assembler source, about 80 lines of platform independent C source, and
> >>> about 350 lines of AMD64 C
> >>> source code. To this, we have to add a few extra source lines in the
> >>> start-up code, system call, and trap han-
> >>> dlers. This implementation is being both developed and tested only in
> >>> the AMD64 architecture."
> >>>
> >>> I uploaded it to the Plan 9 foundation shared drive:
> >>>
> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
> >>>
> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> >>> >
> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> >>> > >
> >>> > > Why NIX?
> >>> > >
> >>> > > If you think about it, timesharing is designed for a world where
> cores
> >>> > > are scarce. But on a machine with hundreds of cores, running Plan
> 9,
> >>> > > there are < 100 processes. We can assign a core to each process,
> and
> >>> > > let those processes own the core until they are done. This might
> be a
> >>> > > useful simplification, it might not, but it's something.
> >>> > >
> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results
> were
> >>> > > good. I was always curious how it would work if we had those
> >>> > > multi-hundred-core machines Intel and IBM and others were telling
> us
> >>> > > about in 2011. Now that we have them, it would be interesting to
> try.
> >>> >
> >>> > As said previously, I will start wandering and stumbling upon
> problems
> >>> > this week-end---I'm a toddler in the area, so it's the way to learn
> to
> >>> > walk.
> >>> >
> >>> > But this brief summary highlight a solution and questions
> >>> > that are, IMHO, valid questions: remember the "war" between
> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not
> a
> >>> > separate process (well: there are "administrative" processes,
> >>> > scheduler and pager but...) but part of the applications. This is
> also
> >>> > why it is efficient compared to "message passing" micro-kernels that
> >>> > are not "near" enough the hardware---so inefficient that, for
> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly
> to
> >>> > improve the result...
> >>> >
> >>> > But multiple cores (and even in the smaller machines nowadays, you
> >>> > find two) present another mean of articulation of the OS code (the
> >>> > MMU is central for me in the whole picture: not move the data
> >>> > around, but change the view of the shared data per core). The
> question
> >>> > is at least certainly worth asking.
> >>> >
> >>> > --
> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >>> >              http://www.kergis.com/
> >>> >             http://kertex.kergis.com/
> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> >
> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M17f91ce3175aeaa0be41937d
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 10324 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

if you can document your steps, then others can stand on your
shoulders, possibly, and we can all move forward?

On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>
> Ok, not a bad first day poking at it.  I have a growing (but not ready) new nix script to pull the right pieces over top of my build environment.
> I have a near-complete build, but with hazards: 9front has evolved in a number of places with many ulong parameters becoming usize.  I have a list of those spots, but now they need to be examined for over/underflow.
> The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes the libboot.a6 binary, some source files that match the symbols, and no mkfile.  Attempting to build also shows some 9front auth changes that need to be incorporated into doauthenticate.c, calls to convS2M and convM2S that now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing at all insurmountable.
>
> Not too daunting.  Next time I have a few moments I'll do a more principled pass on the nix script so I can share it.  I didn't understand enough when I first started updating it.
>
> Paul
>
>
> On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
>>
>> if you look at the first_commit branch, you'll see a sys/src/nix/nix
>> script, which sets up some binds.
>>
>> What we did, before building nix, on plan 9, in 2011, was a set of
>> binds to get the right things such as /sys/include and so on.
>>
>> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
>> I expect it will be strategic changes, and in the end they won't be
>> all that many lines of code, but there will be some tricky stuff.
>>
>> Best ot take it slow, when you hit an issue, ruminate it on for a day
>> or two, then look again. Otherwise you'll just get frustrated (I have
>> ...) But before you make any change, be very sure you know WHY you're
>> doing it, not just that 'it got me past that mk error.'
>>
>> Bring issues to the list and, if you want, keep a running doc to which
>> others can contribute: what you did, what you ran into, what a fix
>> might be. The old saying; "if you don't write it down it didn't
>> happen"
>>
>> But this is the kind of thing you take slowly and carefully, otherwise
>> it's total misery.
>>
>> ron
>>
>> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> >
>> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
>> >
>> > How do I re-root this correctly for this build?
>> >
>> > Paul
>> >
>> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> >>
>> >> Ok, I thought, what could do.
>> >>
>> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
>> >> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
>> >>
>> >> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
>> >>
>> >> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
>> >>
>> >> Paul
>> >>
>> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>> >>>
>> >>> I found the original 2011 paper, which was a sandia report, from may
>> >>> 2011. It's a modification of the original proposal, which I no longer
>> >>> have; but it is a good summary of where we were at the end of my visit
>> >>> in May.
>> >>>
>> >>> This is interesting: "We have changed a surprisingly small amount of
>> >>> code at this point.
>> >>> There are about 400 lines of new
>> >>> assembler source, about 80 lines of platform independent C source, and
>> >>> about 350 lines of AMD64 C
>> >>> source code. To this, we have to add a few extra source lines in the
>> >>> start-up code, system call, and trap han-
>> >>> dlers. This implementation is being both developed and tested only in
>> >>> the AMD64 architecture."
>> >>>
>> >>> I uploaded it to the Plan 9 foundation shared drive:
>> >>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>> >>>
>> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>> >>> >
>> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
>> >>> > >
>> >>> > > Why NIX?
>> >>> > >
>> >>> > > If you think about it, timesharing is designed for a world where cores
>> >>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
>> >>> > > there are < 100 processes. We can assign a core to each process, and
>> >>> > > let those processes own the core until they are done. This might be a
>> >>> > > useful simplification, it might not, but it's something.
>> >>> > >
>> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
>> >>> > > good. I was always curious how it would work if we had those
>> >>> > > multi-hundred-core machines Intel and IBM and others were telling us
>> >>> > > about in 2011. Now that we have them, it would be interesting to try.
>> >>> >
>> >>> > As said previously, I will start wandering and stumbling upon problems
>> >>> > this week-end---I'm a toddler in the area, so it's the way to learn to
>> >>> > walk.
>> >>> >
>> >>> > But this brief summary highlight a solution and questions
>> >>> > that are, IMHO, valid questions: remember the "war" between
>> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
>> >>> > separate process (well: there are "administrative" processes,
>> >>> > scheduler and pager but...) but part of the applications. This is also
>> >>> > why it is efficient compared to "message passing" micro-kernels that
>> >>> > are not "near" enough the hardware---so inefficient that, for
>> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
>> >>> > improve the result...
>> >>> >
>> >>> > But multiple cores (and even in the smaller machines nowadays, you
>> >>> > find two) present another mean of articulation of the OS code (the
>> >>> > MMU is central for me in the whole picture: not move the data
>> >>> > around, but change the view of the shared data per core). The question
>> >>> > is at least certainly worth asking.
>> >>> >
>> >>> > --
>> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>> >>> >              http://www.kergis.com/
>> >>> >             http://kertex.kergis.com/
>> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>> >
>> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M7bc8a8dfe29d63e01303d727
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Paul Lalonde]

[-- Attachment #1.1: Type: text/plain, Size: 9613 bytes --]

As you say, Ron.

First, here's my nix script, such as it is, cribbed from the old nix one.
It has holes, guaranteed.  Also, I went and pulled in a "user" directory,
just for old habits dying hard.  Yes, I still use glenda on this old
terminal.  Call me names for it.

#!/bin/rc

unmount /sys/include >[2]/dev/null

unmount /sys/src/libc >[2]/dev/null

bind -b /usr/glenda/nix-os/sys/include /sys/include

bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc

cd /usr/glenda/nix-os/sys

for(d in man/*){

unmount /sys/$d >[2]/dev/null

bind -b $d /sys/$d

}

exit ''


My terminal is a pi 400, so I had to build out the /amd64 tree,
objtype=arm64.  I'll assume folks are clever enough to do this, or to use
an amd64 terminal or cpu to do this work.


Then mk your heart out.  The main pain points are ulong parameters that are
now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.  These changes
appear limited to

M amd64/include/ureg.h

M sys/include/libc.h

M sys/src/boot/pc/lib.h

M sys/src/nix/boot/nopsession.c

M sys/src/nix/k10/acore.c

M sys/src/nix/k10/fpu.c

M sys/src/nix/k10/sipi.h

M sys/src/nix/k10/syscall.c

M sys/src/nix/k10/trap.c

M sys/src/nix/port/lib.h

M sys/src/nix/port/portfns.h

The diffs are attached.  I don't want to commit a branch because as I said,
I don't think my bind mappings are entirely correct, though I'm seeing many
fewer crossed wires now.
Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot which
*almost* makes a full build happen.  parseipmask has gained a v4 parameter
in 9front, which means the fix there needs actual analysis.  qsort is
somehow also complaining, possibly indicating I'm pulling the wrong header
for it, indicating a problem in my bind script.

This feels completely surmountable.


Paul

On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:

> if you can document your steps, then others can stand on your
> shoulders, possibly, and we can all move forward?
>
> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >
> > Ok, not a bad first day poking at it.  I have a growing (but not ready)
> new nix script to pull the right pieces over top of my build environment.
> > I have a near-complete build, but with hazards: 9front has evolved in a
> number of places with many ulong parameters becoming usize.  I have a list
> of those spots, but now they need to be examined for over/underflow.
> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo
> includes the libboot.a6 binary, some source files that match the symbols,
> and no mkfile.  Attempting to build also shows some 9front auth changes
> that need to be incorporated into doauthenticate.c, calls to convS2M and
> convM2S that now need buffer length parameters, and the phasing of Tnop out
> 9p?  Nothing at all insurmountable.
> >
> > Not too daunting.  Next time I have a few moments I'll do a more
> principled pass on the nix script so I can share it.  I didn't understand
> enough when I first started updating it.
> >
> > Paul
> >
> >
> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
> >>
> >> if you look at the first_commit branch, you'll see a sys/src/nix/nix
> >> script, which sets up some binds.
> >>
> >> What we did, before building nix, on plan 9, in 2011, was a set of
> >> binds to get the right things such as /sys/include and so on.
> >>
> >> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
> >> I expect it will be strategic changes, and in the end they won't be
> >> all that many lines of code, but there will be some tricky stuff.
> >>
> >> Best ot take it slow, when you hit an issue, ruminate it on for a day
> >> or two, then look again. Otherwise you'll just get frustrated (I have
> >> ...) But before you make any change, be very sure you know WHY you're
> >> doing it, not just that 'it got me past that mk error.'
> >>
> >> Bring issues to the list and, if you want, keep a running doc to which
> >> others can contribute: what you did, what you ran into, what a fix
> >> might be. The old saying; "if you don't write it down it didn't
> >> happen"
> >>
> >> But this is the kind of thing you take slowly and carefully, otherwise
> >> it's total misery.
> >>
> >> ron
> >>
> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >> >
> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In
> building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos
> structure from nix, not the one in the host system.
> >> >
> >> > How do I re-root this correctly for this build?
> >> >
> >> > Paul
> >> >
> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> >> >>
> >> >> Ok, I thought, what could do.
> >> >>
> >> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os
> repo and hit "mk".
> >> >> When that errored out a bunch I realized that I needed /amd64 built,
> so I did that.  Just as painless as I remembered.
> >> >>
> >> >> And now, I get a ways further into the build, but hit an
> incompatibility between the my /amd64/include/ureg.h and
> .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX
> code was written someone decided that the program counter should be called
> pc instead of ip.
> >> >>
> >> >> Or else, I'm approaching this all wrong, and Ron can shed some light
> on how I should be proceeding.
> >> >>
> >> >> Paul
> >> >>
> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> >> >>>
> >> >>> I found the original 2011 paper, which was a sandia report, from may
> >> >>> 2011. It's a modification of the original proposal, which I no
> longer
> >> >>> have; but it is a good summary of where we were at the end of my
> visit
> >> >>> in May.
> >> >>>
> >> >>> This is interesting: "We have changed a surprisingly small amount of
> >> >>> code at this point.
> >> >>> There are about 400 lines of new
> >> >>> assembler source, about 80 lines of platform independent C source,
> and
> >> >>> about 350 lines of AMD64 C
> >> >>> source code. To this, we have to add a few extra source lines in the
> >> >>> start-up code, system call, and trap han-
> >> >>> dlers. This implementation is being both developed and tested only
> in
> >> >>> the AMD64 architecture."
> >> >>>
> >> >>> I uploaded it to the Plan 9 foundation shared drive:
> >> >>>
> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
> >> >>>
> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> >> >>> >
> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> >> >>> > >
> >> >>> > > Why NIX?
> >> >>> > >
> >> >>> > > If you think about it, timesharing is designed for a world
> where cores
> >> >>> > > are scarce. But on a machine with hundreds of cores, running
> Plan 9,
> >> >>> > > there are < 100 processes. We can assign a core to each
> process, and
> >> >>> > > let those processes own the core until they are done. This
> might be a
> >> >>> > > useful simplification, it might not, but it's something.
> >> >>> > >
> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and the
> results were
> >> >>> > > good. I was always curious how it would work if we had those
> >> >>> > > multi-hundred-core machines Intel and IBM and others were
> telling us
> >> >>> > > about in 2011. Now that we have them, it would be interesting
> to try.
> >> >>> >
> >> >>> > As said previously, I will start wandering and stumbling upon
> problems
> >> >>> > this week-end---I'm a toddler in the area, so it's the way to
> learn to
> >> >>> > walk.
> >> >>> >
> >> >>> > But this brief summary highlight a solution and questions
> >> >>> > that are, IMHO, valid questions: remember the "war" between
> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is
> not a
> >> >>> > separate process (well: there are "administrative" processes,
> >> >>> > scheduler and pager but...) but part of the applications. This is
> also
> >> >>> > why it is efficient compared to "message passing" micro-kernels
> that
> >> >>> > are not "near" enough the hardware---so inefficient that, for
> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in
> assembly to
> >> >>> > improve the result...
> >> >>> >
> >> >>> > But multiple cores (and even in the smaller machines nowadays, you
> >> >>> > find two) present another mean of articulation of the OS code (the
> >> >>> > MMU is central for me in the whole picture: not move the data
> >> >>> > around, but change the view of the shared data per core). The
> question
> >> >>> > is at least certainly worth asking.
> >> >>> >
> >> >>> > --
> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >> >>> >              http://www.kergis.com/
> >> >>> >             http://kertex.kergis.com/
> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006
> F40C
> >> >
> >> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink
> >
> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M168534a1edfe40b2e2ddc134
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #1.2: Type: text/html, Size: 22986 bytes --]

[-- Attachment #2: nixdiff --]
[-- Type: application/octet-stream, Size: 9606 bytes --]

diff a7a7339e5ec509797a3bc3188c6c4bcdb40b3616 uncommitted
--- a/amd64/include/ureg.h
+++ b/amd64/include/ureg.h
@@ -22,7 +22,7 @@
 
 	u64int	type;
 	u64int	error;				/* error code (or zero) */
-	u64int	ip;				/* pc */
+	u64int	pc;				/* pc */
 	u64int	cs;				/* old context */
 	u64int	flags;				/* old flags */
 	u64int	sp;				/* sp */
--- a/sys/include/libc.h
+++ b/sys/include/libc.h
@@ -8,12 +8,12 @@
 /*
  * mem routines
  */
-extern	void*	memccpy(void*, void*, int, ulong);
-extern	void*	memset(void*, int, ulong);
-extern	int	memcmp(void*, void*, ulong);
-extern	void*	memcpy(void*, void*, ulong);
-extern	void*	memmove(void*, void*, ulong);
-extern	void*	memchr(void*, int, ulong);
+extern	void*	memccpy(void*, void*, int, usize);
+extern	void*	memset(void*, int, usize);
+extern	int	memcmp(void*, void*, usize);
+extern	void*	memcpy(void*, void*, usize);
+extern	void*	memmove(void*, void*, usize);
+extern	void*	memchr(void*, int, usize);
 
 /*
  * string routines
--- a/sys/src/boot/pc/lib.h
+++ b/sys/src/boot/pc/lib.h
@@ -8,9 +8,9 @@
  * mem routines
  */
 extern	void*	memccpy(void*, void*, int, ulong);
-extern	void*	memset(void*, int, ulong);
-extern	int	memcmp(void*, void*, ulong);
-extern	void*	memmove(void*, void*, ulong);
+extern	void*	memset(void*, int, usize);
+extern	int	memcmp(void*, void*, usize);
+extern	void*	memmove(void*, void*, usize);
 extern	void*	memchr(void*, int, ulong);
 
 /*
--- a/sys/src/nix/boot/nopsession.c
+++ b/sys/src/nix/boot/nopsession.c
@@ -10,11 +10,11 @@
 rpc(int fd, int type)
 {
 	int n, l;
-	char buf[128], *p;
+	uchar buf[128], *p;
 
 	hdr.type = type;
 	hdr.tag = NOTAG;
-	n = convS2M(&hdr, buf);
+	n = convS2M(&hdr, buf, 128);
 	if(write(fd, buf, n) != n)
 		fatal("write rpc");
 
@@ -30,7 +30,7 @@
 		p += n;
 		l += n;
 	}
-	if(convM2S(buf, &hdr, n) == 0){
+	if(convM2S(buf, n, &hdr) == 0){
 		print("%ux %ux %ux\n", buf[0], buf[1], buf[2]);
 		fatal("rpc format");
 	}
@@ -48,5 +48,5 @@
 nop(int fd)
 {
 	print("nop");
-	rpc(fd, Tnop);
+	/*rpc(fd, Tnop);*/ /* PAL: DANGER - stubbing? */
 }
--- a/sys/src/nix/k10/acore.c
+++ b/sys/src/nix/k10/acore.c
@@ -135,7 +135,7 @@
 	acfpusysprocsetup(m->proc);
 
 	u = m->proc->dbgreg;
-	DBG("cpu%d: touser usp = %#p entry %#p\n", m->machno, u->sp, u->ip);
+	DBG("cpu%d: touser usp = %#p entry %#p\n", m->machno, u->sp, u->pc);
 	xactouser(u->sp);
 	panic("actouser");
 }
--- a/sys/src/nix/k10/fpu.c
+++ b/sys/src/nix/k10/fpu.c
@@ -312,8 +312,8 @@
 	_stts();
 	up->fpustate = Idle;
 
-	if(ureg->ip & KZERO)
-		panic("#MF: ip=%#p", ureg->ip);
+	if(ureg->pc & KZERO)
+		panic("#MF: pc=%#p", ureg->pc);
 
 	/*
 	 * Notify the user process.
@@ -375,8 +375,8 @@
 	_stts();
 	up->fpustate = Idle;
 
-	if(ureg->ip & KZERO)
-		panic("#MF: ip=%#p rip=%#p", ureg->ip, fpusave->rip);
+	if(ureg->pc & KZERO)
+		panic("#MF: pc=%#p rip=%#p", ureg->pc, fpusave->rip);
 
 	/*
 	 * Notify the user process.
@@ -419,7 +419,7 @@
 	 * #NM - Device Not Available (Vector 7).
 	 */
 	if(up == nil)
-		panic("#NM: fpu in kernel: ip %#p\n", ureg->ip);
+		panic("#NM: fpu in kernel: pc %#p\n", ureg->pc);
 
 	/*
 	 * Someone tried to use the FPU in a note handler.
@@ -428,14 +428,14 @@
 	if(up->fpustate & Hold)
 		return "sys: floating point in note handler";
 
-	if(ureg->ip & KZERO)
-		panic("#NM: proc %d %s state %d ip %#p\n",
-			up->pid, up->text, up->fpustate, ureg->ip);
+	if(ureg->pc & KZERO)
+		panic("#NM: proc %d %s state %d pc %#p\n",
+			up->pid, up->text, up->fpustate, ureg->pc);
 
 	switch(up->fpustate){
 	case Busy:
 	default:
-		panic("#NM: state %d ip %#p\n", up->fpustate, ureg->ip);
+		panic("#NM: state %d pc %#p\n", up->fpustate, ureg->pc);
 		break;
 	case Init:
 		/*
--- a/sys/src/nix/k10/sipi.h
+++ b/sys/src/nix/k10/sipi.h
@@ -16,10 +16,10 @@
 0x0f,0x20,0xc2,0x81,0xe2,0xf5,0xff,0xff,0x9f,0x81,0xca,0x00,0x00,0x01,0x80,0x0f,
 0x22,0xc2,0xea,0xf1,0x30,0x00,0x00,0x18,0x00,0x48,0xc7,0xc0,0xfa,0x30,0x00,0xf0,
 0xff,0xe0,0x48,0xc7,0xc0,0x4e,0x30,0x00,0xf0,0x0f,0x01,0x10,0x48,0x31,0xd2,0x8e,
-0xda,0x8e,0xc2,0x8e,0xe2,0x8e,0xea,0x8e,0xd2,0x63,0xf6,0x48,0x89,0xf0,0x48,0x05,
+0xda,0x8e,0xc2,0x8e,0xe2,0x8e,0xea,0x8e,0xd2,0x8b,0xf6,0x48,0x89,0xf0,0x48,0x05,
 0x00,0x00,0x00,0xf0,0x48,0x89,0xc4,0x48,0x89,0x10,0x0f,0x22,0xde,0x48,0xc7,0xc0,
-0x00,0x30,0x00,0xf0,0x8b,0x70,0xfc,0x63,0xf6,0x48,0x89,0xf0,0x48,0x05,0x00,0x00,
+0x00,0x30,0x00,0xf0,0x8b,0x70,0xfc,0x8b,0xf6,0x48,0x89,0xf0,0x48,0x05,0x00,0x00,
 0x00,0xf0,0x48,0x89,0xc4,0x48,0x05,0x00,0x50,0x00,0x00,0x49,0x89,0xc7,0x49,0x89,
-0xd6,0x52,0x9d,0x63,0xed,0x55,0x49,0x8b,0x47,0x08,0xff,0xd0,0xeb,0xfe,
+0xd6,0x52,0x9d,0x8b,0xed,0x55,0x49,0x8b,0x47,0x08,0xff,0xd0,0xeb,0xfe,
 
 };
--- a/sys/src/nix/k10/syscall.c
+++ b/sys/src/nix/k10/syscall.c
@@ -71,10 +71,10 @@
 	switch((int)arg0){
 	case NCONT:
 	case NRSTR:
-		if(!okaddr(nur->ip, BY2SE, 0) || !okaddr(nur->sp, BY2SE, 0)){
+		if(!okaddr(nur->pc, BY2SE, 0) || !okaddr(nur->sp, BY2SE, 0)){
 			qunlock(&up->debug);
 			pprint("suicide: trap in noted pc=%#p sp=%#p\n",
-				nur->ip, nur->sp);
+				nur->pc, nur->sp);
 			pexit("Suicide", 0);
 		}
 		up->ureg = nf->old;
@@ -81,10 +81,10 @@
 		qunlock(&up->debug);
 		break;
 	case NSAVE:
-		if(!okaddr(nur->ip, BY2SE, 0) || !okaddr(nur->sp, BY2SE, 0)){
+		if(!okaddr(nur->pc, BY2SE, 0) || !okaddr(nur->sp, BY2SE, 0)){
 			qunlock(&up->debug);
 			pprint("suicide: trap in noted pc=%#p sp=%#p\n",
-				nur->ip, nur->sp);
+				nur->pc, nur->sp);
 			pexit("Suicide", 0);
 		}
 		qunlock(&up->debug);
@@ -144,7 +144,7 @@
 		l = strlen(note.msg);
 		if(l > ERRMAX-sizeof(" pc=0x0123456789abcdef"))
 			l = ERRMAX-sizeof(" pc=0x0123456789abcdef");
-		sprint(note.msg+l, " pc=%#p", ureg->ip);
+		sprint(note.msg+l, " pc=%#p", ureg->pc);
 	}
 
 	if(note.flag != NUser && (up->notified || up->notify == nil)){
@@ -164,7 +164,7 @@
 		qunlock(&up->debug);
 		pexit(note.msg, note.flag != NDebug);
 	}
-	if(!okaddr(PTR2UINT(up->notify), sizeof(ureg->ip), 0)){
+	if(!okaddr(PTR2UINT(up->notify), sizeof(ureg->pc), 0)){
 		qunlock(&up->debug);
 		pprint("suicide: bad function address %#p in notify\n",
 			up->notify);
@@ -189,7 +189,7 @@
 	nf->ip = 0;
 
 	ureg->sp = sp;
-	ureg->ip = PTR2UINT(up->notify);
+	ureg->pc = PTR2UINT(up->notify);
 	up->notified = 1;
 	up->nnote--;
 	memmove(&up->lastnote, &note, sizeof(Note));
@@ -240,7 +240,7 @@
 	up->nsyscall++;
 	up->nqsyscall++;
 	up->insyscall = 1;
-	up->pc = ureg->ip;
+	up->pc = ureg->pc;
 	up->dbgreg = ureg;
 	sp = ureg->sp;
 	startns = 0;
@@ -276,7 +276,7 @@
 	if(!waserror()){
 		if(scallnr >= nsyscall || systab[scallnr].f == nil){
 			pprint("bad sys call number %d pc %#llux\n",
-				scallnr, ureg->ip);
+				scallnr, ureg->pc);
 			postnote(up, 1, "sys: bad sys call", NDebug);
 			error(Ebadarg);
 		}
@@ -390,7 +390,7 @@
 
 	ureg = up->dbgreg;
 	ureg->sp = PTR2UINT(sp);
-	ureg->ip = entry;
+	ureg->pc = entry;
 	ureg->type = 64;			/* fiction for acid */
 
 	/*
--- a/sys/src/nix/k10/trap.c
+++ b/sys/src/nix/k10/trap.c
@@ -401,7 +401,7 @@
 			nmienable();
 			if(m->machno != 0){
 				iprint("cpu%d: PC %#llux\n",
-					m->machno, ureg->ip);
+					m->machno, ureg->pc);
 				for(;;);
 			}
 		}
@@ -471,7 +471,7 @@
 	iprint("ureg fs\t%#ux\n", *(unsigned int *)&ureg->ds);
 	iprint("type\t%#llux\n", ureg->type);
 	iprint("error\t%#llux\n", ureg->error);
-	iprint("pc\t%#llux\n", ureg->ip);
+	iprint("pc\t%#llux\n", ureg->pc);
 	iprint("cs\t%#llux\n", ureg->cs);
 	iprint("flags\t%#llux\n", ureg->flags);
 	iprint("sp\t%#llux\n", ureg->sp);
@@ -510,7 +510,7 @@
 callwithureg(void (*fn)(Ureg*))
 {
 	Ureg ureg;
-	ureg.ip = getcallerpc(&fn);
+	ureg.pc = getcallerpc(&fn);
 	ureg.sp = PTR2UINT(&fn);
 	fn(&ureg);
 }
@@ -530,7 +530,7 @@
 	iprint("dumpstack\n");
 
 	x = 0;
-	x += iprint("ktrace 9%s %#p %#p\n", strrchr(conffile, '/')+1, ureg->ip, ureg->sp);
+	x += iprint("ktrace 9%s %#p %#p\n", strrchr(conffile, '/')+1, ureg->pc, ureg->sp);
 	i = 0;
 	if(up != nil
 //	&& (uintptr)&l >= (uintptr)up->kstack
@@ -577,7 +577,7 @@
 	if(up == 0)
 		panic("kernel bpt");
 	/* restore pc to instruction that caused the trap */
-	ureg->ip--;
+	ureg->pc--;
 	sprint(buf, "sys: breakpoint");
 	postnote(up, 1, buf, NDebug);
 }
@@ -618,7 +618,7 @@
 	 */
 	if(up == nil){
 		panic("fault with up == nil; pc %#llux addr %#llux\n",
-			ureg->ip, addr);
+			ureg->pc, addr);
 	}
 	read = !(ureg->error & 2);
 
@@ -656,7 +656,7 @@
 {
 	if(ureg == nil)
 		ureg = up->dbgreg;
-	return ureg->ip;
+	return ureg->pc;
 }
 
 /* This routine must save the values of registers the user is not permitted
@@ -692,7 +692,7 @@
 void
 setkernur(Ureg* ureg, Proc* p)
 {
-	ureg->ip = p->sched.pc;
+	ureg->pc = p->sched.pc;
 	ureg->sp = p->sched.sp+BY2SE;
 }
 
@@ -705,5 +705,5 @@
 	if(ureg == 0)
 		return 0;
 
-	return ureg->ip;
+	return ureg->pc;
 }
--- a/sys/src/nix/port/lib.h
+++ b/sys/src/nix/port/lib.h
@@ -8,11 +8,11 @@
 /*
  * mem routines
  */
-extern	void*	memccpy(void*, void*, int, ulong);
-extern	void*	memset(void*, int, ulong);
-extern	int	memcmp(void*, void*, ulong);
-extern	void*	memmove(void*, void*, ulong);
-extern	void*	memchr(void*, int, ulong);
+extern	void*	memccpy(void*, void*, int, usize);
+extern	void*	memset(void*, int, usize);
+extern	int	memcmp(void*, void*, usize);
+extern	void*	memmove(void*, void*, usize);
+extern	void*	memchr(void*, int, usize);
 
 /*
  * string routines
--- a/sys/src/nix/port/portfns.h
+++ b/sys/src/nix/port/portfns.h
@@ -124,7 +124,7 @@
 void		freepte(Segment*, Pte*);
 void		getcolor(ulong, ulong*, ulong*, ulong*);
 char*		getconfenv(void);
-int		getpgszi(ulong);
+int		getpgszi(usize);
 Segment*	getzkseg(void);
 void		gotolabel(Label*);
 int		haswaitq(void*);

[-- Attachment #3: mkfile --]
[-- Type: application/octet-stream, Size: 288 bytes --]

</$objtype/mkfile

LIB=/$objtype/lib/libventi.a

OFILES=\
	aux.$O\
	boot.$O\
	bootauth.$O\
	bootcache.$O\
	bootip.$O\
	embed.$O\
	getpasswd.$O\
	local.$O\
	nopsession.$O\
	paq.$O\
	printstub.$O\
	sac.$O\
	settime.$O

NOBUILD=doauthenticate.$O


</sys/src/cmd/mksyslib

CFLAGS=$CFLAGS -I.

Re: [9fans] NIX experience

[Ron Minnich]

so for work like this, my motto is commit early, commit often, to a
branch we can always drop later. no harm.  It's easier (for me anyway)
than shuffling patches around in email.

I'm happy to accept a pull request against rminnich/nix-os, , let's
call the branch regen.

thanks

On Tue, Jan 7, 2025 at 9:52 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>
> As you say, Ron.
>
> First, here's my nix script, such as it is, cribbed from the old nix one.  It has holes, guaranteed.  Also, I went and pulled in a "user" directory, just for old habits dying hard.  Yes, I still use glenda on this old terminal.  Call me names for it.
>
> #!/bin/rc
>
> unmount /sys/include >[2]/dev/null
>
> unmount /sys/src/libc >[2]/dev/null
>
> bind -b /usr/glenda/nix-os/sys/include /sys/include
>
> bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc
>
> cd /usr/glenda/nix-os/sys
>
> for(d in man/*){
>
> unmount /sys/$d >[2]/dev/null
>
> bind -b $d /sys/$d
>
> }
>
> exit ''
>
>
> My terminal is a pi 400, so I had to build out the /amd64 tree, objtype=arm64.  I'll assume folks are clever enough to do this, or to use an amd64 terminal or cpu to do this work.
>
>
> Then mk your heart out.  The main pain points are ulong parameters that are now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.  These changes appear limited to
>
> M amd64/include/ureg.h
>
> M sys/include/libc.h
>
> M sys/src/boot/pc/lib.h
>
> M sys/src/nix/boot/nopsession.c
>
> M sys/src/nix/k10/acore.c
>
> M sys/src/nix/k10/fpu.c
>
> M sys/src/nix/k10/sipi.h
>
> M sys/src/nix/k10/syscall.c
>
> M sys/src/nix/k10/trap.c
>
> M sys/src/nix/port/lib.h
>
> M sys/src/nix/port/portfns.h
>
> The diffs are attached.  I don't want to commit a branch because as I said, I don't think my bind mappings are entirely correct, though I'm seeing many fewer crossed wires now.
> Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot which *almost* makes a full build happen.  parseipmask has gained a v4 parameter in 9front, which means the fix there needs actual analysis.  qsort is somehow also complaining, possibly indicating I'm pulling the wrong header for it, indicating a problem in my bind script.
>
> This feels completely surmountable.
>
>
> Paul
>
>
> On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:
>>
>> if you can document your steps, then others can stand on your
>> shoulders, possibly, and we can all move forward?
>>
>> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> >
>> > Ok, not a bad first day poking at it.  I have a growing (but not ready) new nix script to pull the right pieces over top of my build environment.
>> > I have a near-complete build, but with hazards: 9front has evolved in a number of places with many ulong parameters becoming usize.  I have a list of those spots, but now they need to be examined for over/underflow.
>> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes the libboot.a6 binary, some source files that match the symbols, and no mkfile.  Attempting to build also shows some 9front auth changes that need to be incorporated into doauthenticate.c, calls to convS2M and convM2S that now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing at all insurmountable.
>> >
>> > Not too daunting.  Next time I have a few moments I'll do a more principled pass on the nix script so I can share it.  I didn't understand enough when I first started updating it.
>> >
>> > Paul
>> >
>> >
>> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
>> >>
>> >> if you look at the first_commit branch, you'll see a sys/src/nix/nix
>> >> script, which sets up some binds.
>> >>
>> >> What we did, before building nix, on plan 9, in 2011, was a set of
>> >> binds to get the right things such as /sys/include and so on.
>> >>
>> >> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
>> >> I expect it will be strategic changes, and in the end they won't be
>> >> all that many lines of code, but there will be some tricky stuff.
>> >>
>> >> Best ot take it slow, when you hit an issue, ruminate it on for a day
>> >> or two, then look again. Otherwise you'll just get frustrated (I have
>> >> ...) But before you make any change, be very sure you know WHY you're
>> >> doing it, not just that 'it got me past that mk error.'
>> >>
>> >> Bring issues to the list and, if you want, keep a running doc to which
>> >> others can contribute: what you did, what you ran into, what a fix
>> >> might be. The old saying; "if you don't write it down it didn't
>> >> happen"
>> >>
>> >> But this is the kind of thing you take slowly and carefully, otherwise
>> >> it's total misery.
>> >>
>> >> ron
>> >>
>> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> >> >
>> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
>> >> >
>> >> > How do I re-root this correctly for this build?
>> >> >
>> >> > Paul
>> >> >
>> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> >> >>
>> >> >> Ok, I thought, what could do.
>> >> >>
>> >> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
>> >> >> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
>> >> >>
>> >> >> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
>> >> >>
>> >> >> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
>> >> >>
>> >> >> Paul
>> >> >>
>> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>> >> >>>
>> >> >>> I found the original 2011 paper, which was a sandia report, from may
>> >> >>> 2011. It's a modification of the original proposal, which I no longer
>> >> >>> have; but it is a good summary of where we were at the end of my visit
>> >> >>> in May.
>> >> >>>
>> >> >>> This is interesting: "We have changed a surprisingly small amount of
>> >> >>> code at this point.
>> >> >>> There are about 400 lines of new
>> >> >>> assembler source, about 80 lines of platform independent C source, and
>> >> >>> about 350 lines of AMD64 C
>> >> >>> source code. To this, we have to add a few extra source lines in the
>> >> >>> start-up code, system call, and trap han-
>> >> >>> dlers. This implementation is being both developed and tested only in
>> >> >>> the AMD64 architecture."
>> >> >>>
>> >> >>> I uploaded it to the Plan 9 foundation shared drive:
>> >> >>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>> >> >>>
>> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>> >> >>> >
>> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
>> >> >>> > >
>> >> >>> > > Why NIX?
>> >> >>> > >
>> >> >>> > > If you think about it, timesharing is designed for a world where cores
>> >> >>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
>> >> >>> > > there are < 100 processes. We can assign a core to each process, and
>> >> >>> > > let those processes own the core until they are done. This might be a
>> >> >>> > > useful simplification, it might not, but it's something.
>> >> >>> > >
>> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
>> >> >>> > > good. I was always curious how it would work if we had those
>> >> >>> > > multi-hundred-core machines Intel and IBM and others were telling us
>> >> >>> > > about in 2011. Now that we have them, it would be interesting to try.
>> >> >>> >
>> >> >>> > As said previously, I will start wandering and stumbling upon problems
>> >> >>> > this week-end---I'm a toddler in the area, so it's the way to learn to
>> >> >>> > walk.
>> >> >>> >
>> >> >>> > But this brief summary highlight a solution and questions
>> >> >>> > that are, IMHO, valid questions: remember the "war" between
>> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
>> >> >>> > separate process (well: there are "administrative" processes,
>> >> >>> > scheduler and pager but...) but part of the applications. This is also
>> >> >>> > why it is efficient compared to "message passing" micro-kernels that
>> >> >>> > are not "near" enough the hardware---so inefficient that, for
>> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
>> >> >>> > improve the result...
>> >> >>> >
>> >> >>> > But multiple cores (and even in the smaller machines nowadays, you
>> >> >>> > find two) present another mean of articulation of the OS code (the
>> >> >>> > MMU is central for me in the whole picture: not move the data
>> >> >>> > around, but change the view of the shared data per core). The question
>> >> >>> > is at least certainly worth asking.
>> >> >>> >
>> >> >>> > --
>> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>> >> >>> >              http://www.kergis.com/
>> >> >>> >             http://kertex.kergis.com/
>> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>> >> >
>> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>> >
>> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M3769343bc21b9d2e2f27b00c
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

NIX is moving forward, thank you paul!

The branch is called regen, we have our first commit in many years.
Please take a look. If you submit a PR, please add a signed-off-by:
line.

thanks

On Tue, Jan 7, 2025 at 10:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>
> so for work like this, my motto is commit early, commit often, to a
> branch we can always drop later. no harm.  It's easier (for me anyway)
> than shuffling patches around in email.
>
> I'm happy to accept a pull request against rminnich/nix-os, , let's
> call the branch regen.
>
> thanks
>
> On Tue, Jan 7, 2025 at 9:52 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> >
> > As you say, Ron.
> >
> > First, here's my nix script, such as it is, cribbed from the old nix one.  It has holes, guaranteed.  Also, I went and pulled in a "user" directory, just for old habits dying hard.  Yes, I still use glenda on this old terminal.  Call me names for it.
> >
> > #!/bin/rc
> >
> > unmount /sys/include >[2]/dev/null
> >
> > unmount /sys/src/libc >[2]/dev/null
> >
> > bind -b /usr/glenda/nix-os/sys/include /sys/include
> >
> > bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc
> >
> > cd /usr/glenda/nix-os/sys
> >
> > for(d in man/*){
> >
> > unmount /sys/$d >[2]/dev/null
> >
> > bind -b $d /sys/$d
> >
> > }
> >
> > exit ''
> >
> >
> > My terminal is a pi 400, so I had to build out the /amd64 tree, objtype=arm64.  I'll assume folks are clever enough to do this, or to use an amd64 terminal or cpu to do this work.
> >
> >
> > Then mk your heart out.  The main pain points are ulong parameters that are now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.  These changes appear limited to
> >
> > M amd64/include/ureg.h
> >
> > M sys/include/libc.h
> >
> > M sys/src/boot/pc/lib.h
> >
> > M sys/src/nix/boot/nopsession.c
> >
> > M sys/src/nix/k10/acore.c
> >
> > M sys/src/nix/k10/fpu.c
> >
> > M sys/src/nix/k10/sipi.h
> >
> > M sys/src/nix/k10/syscall.c
> >
> > M sys/src/nix/k10/trap.c
> >
> > M sys/src/nix/port/lib.h
> >
> > M sys/src/nix/port/portfns.h
> >
> > The diffs are attached.  I don't want to commit a branch because as I said, I don't think my bind mappings are entirely correct, though I'm seeing many fewer crossed wires now.
> > Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot which *almost* makes a full build happen.  parseipmask has gained a v4 parameter in 9front, which means the fix there needs actual analysis.  qsort is somehow also complaining, possibly indicating I'm pulling the wrong header for it, indicating a problem in my bind script.
> >
> > This feels completely surmountable.
> >
> >
> > Paul
> >
> >
> > On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:
> >>
> >> if you can document your steps, then others can stand on your
> >> shoulders, possibly, and we can all move forward?
> >>
> >> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> >> >
> >> > Ok, not a bad first day poking at it.  I have a growing (but not ready) new nix script to pull the right pieces over top of my build environment.
> >> > I have a near-complete build, but with hazards: 9front has evolved in a number of places with many ulong parameters becoming usize.  I have a list of those spots, but now they need to be examined for over/underflow.
> >> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes the libboot.a6 binary, some source files that match the symbols, and no mkfile.  Attempting to build also shows some 9front auth changes that need to be incorporated into doauthenticate.c, calls to convS2M and convM2S that now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing at all insurmountable.
> >> >
> >> > Not too daunting.  Next time I have a few moments I'll do a more principled pass on the nix script so I can share it.  I didn't understand enough when I first started updating it.
> >> >
> >> > Paul
> >> >
> >> >
> >> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
> >> >>
> >> >> if you look at the first_commit branch, you'll see a sys/src/nix/nix
> >> >> script, which sets up some binds.
> >> >>
> >> >> What we did, before building nix, on plan 9, in 2011, was a set of
> >> >> binds to get the right things such as /sys/include and so on.
> >> >>
> >> >> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
> >> >> I expect it will be strategic changes, and in the end they won't be
> >> >> all that many lines of code, but there will be some tricky stuff.
> >> >>
> >> >> Best ot take it slow, when you hit an issue, ruminate it on for a day
> >> >> or two, then look again. Otherwise you'll just get frustrated (I have
> >> >> ...) But before you make any change, be very sure you know WHY you're
> >> >> doing it, not just that 'it got me past that mk error.'
> >> >>
> >> >> Bring issues to the list and, if you want, keep a running doc to which
> >> >> others can contribute: what you did, what you ran into, what a fix
> >> >> might be. The old saying; "if you don't write it down it didn't
> >> >> happen"
> >> >>
> >> >> But this is the kind of thing you take slowly and carefully, otherwise
> >> >> it's total misery.
> >> >>
> >> >> ron
> >> >>
> >> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> >> >> >
> >> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
> >> >> >
> >> >> > How do I re-root this correctly for this build?
> >> >> >
> >> >> > Paul
> >> >> >
> >> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> >> >> >>
> >> >> >> Ok, I thought, what could do.
> >> >> >>
> >> >> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
> >> >> >> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
> >> >> >>
> >> >> >> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
> >> >> >>
> >> >> >> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
> >> >> >>
> >> >> >> Paul
> >> >> >>
> >> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> >> >> >>>
> >> >> >>> I found the original 2011 paper, which was a sandia report, from may
> >> >> >>> 2011. It's a modification of the original proposal, which I no longer
> >> >> >>> have; but it is a good summary of where we were at the end of my visit
> >> >> >>> in May.
> >> >> >>>
> >> >> >>> This is interesting: "We have changed a surprisingly small amount of
> >> >> >>> code at this point.
> >> >> >>> There are about 400 lines of new
> >> >> >>> assembler source, about 80 lines of platform independent C source, and
> >> >> >>> about 350 lines of AMD64 C
> >> >> >>> source code. To this, we have to add a few extra source lines in the
> >> >> >>> start-up code, system call, and trap han-
> >> >> >>> dlers. This implementation is being both developed and tested only in
> >> >> >>> the AMD64 architecture."
> >> >> >>>
> >> >> >>> I uploaded it to the Plan 9 foundation shared drive:
> >> >> >>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
> >> >> >>>
> >> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> >> >> >>> >
> >> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> >> >> >>> > >
> >> >> >>> > > Why NIX?
> >> >> >>> > >
> >> >> >>> > > If you think about it, timesharing is designed for a world where cores
> >> >> >>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
> >> >> >>> > > there are < 100 processes. We can assign a core to each process, and
> >> >> >>> > > let those processes own the core until they are done. This might be a
> >> >> >>> > > useful simplification, it might not, but it's something.
> >> >> >>> > >
> >> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
> >> >> >>> > > good. I was always curious how it would work if we had those
> >> >> >>> > > multi-hundred-core machines Intel and IBM and others were telling us
> >> >> >>> > > about in 2011. Now that we have them, it would be interesting to try.
> >> >> >>> >
> >> >> >>> > As said previously, I will start wandering and stumbling upon problems
> >> >> >>> > this week-end---I'm a toddler in the area, so it's the way to learn to
> >> >> >>> > walk.
> >> >> >>> >
> >> >> >>> > But this brief summary highlight a solution and questions
> >> >> >>> > that are, IMHO, valid questions: remember the "war" between
> >> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
> >> >> >>> > separate process (well: there are "administrative" processes,
> >> >> >>> > scheduler and pager but...) but part of the applications. This is also
> >> >> >>> > why it is efficient compared to "message passing" micro-kernels that
> >> >> >>> > are not "near" enough the hardware---so inefficient that, for
> >> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
> >> >> >>> > improve the result...
> >> >> >>> >
> >> >> >>> > But multiple cores (and even in the smaller machines nowadays, you
> >> >> >>> > find two) present another mean of articulation of the OS code (the
> >> >> >>> > MMU is central for me in the whole picture: not move the data
> >> >> >>> > around, but change the view of the shared data per core). The question
> >> >> >>> > is at least certainly worth asking.
> >> >> >>> >
> >> >> >>> > --
> >> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> >> >> >>> >              http://www.kergis.com/
> >> >> >>> >             http://kertex.kergis.com/
> >> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> >> >> >
> >> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> >> >
> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> >
> > 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M61a12703b44240d058a66a56
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Ron Minnich]

WOW! Paul got it to build.

git/clone git@github.com:rminnich/nix-os
git/branch -b origin/regen -n regen
cd sys/src/nix
# HEY ANYONE! WANT TO FIX THIS!
rc -x nix # set the x bits?
# make it so it does not have to be in $home/nix-os?

cd boot
mk
cd ../k10
mk
# it may seem like it hangs, it's actually waiting for your nvram key.
# HEY ANYONE! the prompt for nvram gets buried in output. Want to fix this?

vmx 9k8cpu # HEY ANYONE! vmx thinks the multiboot header in 9k8cpu is
wrong, but it's not. This is an easy one, Look at the multiboot header
in l32p.s
# and see why vmx does not like it.

Or just netboot a cpu server with 9k8cpu

Note we decided to leave a few things for people to take a try at
fixing. These are great little exercises. Learn to use git, learn a
workflow, building a kernel, etc. etc.

contributing:
The github workflow is to fork github.com/rminnich/nix-os, checkout a
branch based on regen, hack hack, commit -s, push to your branch, that
will make a pull request.
Very standard stuff, we don't know how to make it all work with 9front git yet.

Questions? Put them here, and thanks in advance.

ron

On Wed, Jan 8, 2025 at 4:19 PM Ron Minnich <rminnich@p9f.org> wrote:
>
> NIX is moving forward, thank you paul!
>
> The branch is called regen, we have our first commit in many years.
> Please take a look. If you submit a PR, please add a signed-off-by:
> line.
>
> thanks
>
> On Tue, Jan 7, 2025 at 10:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> >
> > so for work like this, my motto is commit early, commit often, to a
> > branch we can always drop later. no harm.  It's easier (for me anyway)
> > than shuffling patches around in email.
> >
> > I'm happy to accept a pull request against rminnich/nix-os, , let's
> > call the branch regen.
> >
> > thanks
> >
> > On Tue, Jan 7, 2025 at 9:52 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> > >
> > > As you say, Ron.
> > >
> > > First, here's my nix script, such as it is, cribbed from the old nix one.  It has holes, guaranteed.  Also, I went and pulled in a "user" directory, just for old habits dying hard.  Yes, I still use glenda on this old terminal.  Call me names for it.
> > >
> > > #!/bin/rc
> > >
> > > unmount /sys/include >[2]/dev/null
> > >
> > > unmount /sys/src/libc >[2]/dev/null
> > >
> > > bind -b /usr/glenda/nix-os/sys/include /sys/include
> > >
> > > bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc
> > >
> > > cd /usr/glenda/nix-os/sys
> > >
> > > for(d in man/*){
> > >
> > > unmount /sys/$d >[2]/dev/null
> > >
> > > bind -b $d /sys/$d
> > >
> > > }
> > >
> > > exit ''
> > >
> > >
> > > My terminal is a pi 400, so I had to build out the /amd64 tree, objtype=arm64.  I'll assume folks are clever enough to do this, or to use an amd64 terminal or cpu to do this work.
> > >
> > >
> > > Then mk your heart out.  The main pain points are ulong parameters that are now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.  These changes appear limited to
> > >
> > > M amd64/include/ureg.h
> > >
> > > M sys/include/libc.h
> > >
> > > M sys/src/boot/pc/lib.h
> > >
> > > M sys/src/nix/boot/nopsession.c
> > >
> > > M sys/src/nix/k10/acore.c
> > >
> > > M sys/src/nix/k10/fpu.c
> > >
> > > M sys/src/nix/k10/sipi.h
> > >
> > > M sys/src/nix/k10/syscall.c
> > >
> > > M sys/src/nix/k10/trap.c
> > >
> > > M sys/src/nix/port/lib.h
> > >
> > > M sys/src/nix/port/portfns.h
> > >
> > > The diffs are attached.  I don't want to commit a branch because as I said, I don't think my bind mappings are entirely correct, though I'm seeing many fewer crossed wires now.
> > > Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot which *almost* makes a full build happen.  parseipmask has gained a v4 parameter in 9front, which means the fix there needs actual analysis.  qsort is somehow also complaining, possibly indicating I'm pulling the wrong header for it, indicating a problem in my bind script.
> > >
> > > This feels completely surmountable.
> > >
> > >
> > > Paul
> > >
> > >
> > > On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:
> > >>
> > >> if you can document your steps, then others can stand on your
> > >> shoulders, possibly, and we can all move forward?
> > >>
> > >> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> > >> >
> > >> > Ok, not a bad first day poking at it.  I have a growing (but not ready) new nix script to pull the right pieces over top of my build environment.
> > >> > I have a near-complete build, but with hazards: 9front has evolved in a number of places with many ulong parameters becoming usize.  I have a list of those spots, but now they need to be examined for over/underflow.
> > >> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes the libboot.a6 binary, some source files that match the symbols, and no mkfile.  Attempting to build also shows some 9front auth changes that need to be incorporated into doauthenticate.c, calls to convS2M and convM2S that now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing at all insurmountable.
> > >> >
> > >> > Not too daunting.  Next time I have a few moments I'll do a more principled pass on the nix script so I can share it.  I didn't understand enough when I first started updating it.
> > >> >
> > >> > Paul
> > >> >
> > >> >
> > >> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
> > >> >>
> > >> >> if you look at the first_commit branch, you'll see a sys/src/nix/nix
> > >> >> script, which sets up some binds.
> > >> >>
> > >> >> What we did, before building nix, on plan 9, in 2011, was a set of
> > >> >> binds to get the right things such as /sys/include and so on.
> > >> >>
> > >> >> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
> > >> >> I expect it will be strategic changes, and in the end they won't be
> > >> >> all that many lines of code, but there will be some tricky stuff.
> > >> >>
> > >> >> Best ot take it slow, when you hit an issue, ruminate it on for a day
> > >> >> or two, then look again. Otherwise you'll just get frustrated (I have
> > >> >> ...) But before you make any change, be very sure you know WHY you're
> > >> >> doing it, not just that 'it got me past that mk error.'
> > >> >>
> > >> >> Bring issues to the list and, if you want, keep a running doc to which
> > >> >> others can contribute: what you did, what you ran into, what a fix
> > >> >> might be. The old saying; "if you don't write it down it didn't
> > >> >> happen"
> > >> >>
> > >> >> But this is the kind of thing you take slowly and carefully, otherwise
> > >> >> it's total misery.
> > >> >>
> > >> >> ron
> > >> >>
> > >> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> > >> >> >
> > >> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
> > >> >> >
> > >> >> > How do I re-root this correctly for this build?
> > >> >> >
> > >> >> > Paul
> > >> >> >
> > >> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
> > >> >> >>
> > >> >> >> Ok, I thought, what could do.
> > >> >> >>
> > >> >> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
> > >> >> >> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
> > >> >> >>
> > >> >> >> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
> > >> >> >>
> > >> >> >> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
> > >> >> >>
> > >> >> >> Paul
> > >> >> >>
> > >> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> > >> >> >>>
> > >> >> >>> I found the original 2011 paper, which was a sandia report, from may
> > >> >> >>> 2011. It's a modification of the original proposal, which I no longer
> > >> >> >>> have; but it is a good summary of where we were at the end of my visit
> > >> >> >>> in May.
> > >> >> >>>
> > >> >> >>> This is interesting: "We have changed a surprisingly small amount of
> > >> >> >>> code at this point.
> > >> >> >>> There are about 400 lines of new
> > >> >> >>> assembler source, about 80 lines of platform independent C source, and
> > >> >> >>> about 350 lines of AMD64 C
> > >> >> >>> source code. To this, we have to add a few extra source lines in the
> > >> >> >>> start-up code, system call, and trap han-
> > >> >> >>> dlers. This implementation is being both developed and tested only in
> > >> >> >>> the AMD64 architecture."
> > >> >> >>>
> > >> >> >>> I uploaded it to the Plan 9 foundation shared drive:
> > >> >> >>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
> > >> >> >>>
> > >> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> > >> >> >>> >
> > >> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
> > >> >> >>> > >
> > >> >> >>> > > Why NIX?
> > >> >> >>> > >
> > >> >> >>> > > If you think about it, timesharing is designed for a world where cores
> > >> >> >>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
> > >> >> >>> > > there are < 100 processes. We can assign a core to each process, and
> > >> >> >>> > > let those processes own the core until they are done. This might be a
> > >> >> >>> > > useful simplification, it might not, but it's something.
> > >> >> >>> > >
> > >> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
> > >> >> >>> > > good. I was always curious how it would work if we had those
> > >> >> >>> > > multi-hundred-core machines Intel and IBM and others were telling us
> > >> >> >>> > > about in 2011. Now that we have them, it would be interesting to try.
> > >> >> >>> >
> > >> >> >>> > As said previously, I will start wandering and stumbling upon problems
> > >> >> >>> > this week-end---I'm a toddler in the area, so it's the way to learn to
> > >> >> >>> > walk.
> > >> >> >>> >
> > >> >> >>> > But this brief summary highlight a solution and questions
> > >> >> >>> > that are, IMHO, valid questions: remember the "war" between
> > >> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
> > >> >> >>> > separate process (well: there are "administrative" processes,
> > >> >> >>> > scheduler and pager but...) but part of the applications. This is also
> > >> >> >>> > why it is efficient compared to "message passing" micro-kernels that
> > >> >> >>> > are not "near" enough the hardware---so inefficient that, for
> > >> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
> > >> >> >>> > improve the result...
> > >> >> >>> >
> > >> >> >>> > But multiple cores (and even in the smaller machines nowadays, you
> > >> >> >>> > find two) present another mean of articulation of the OS code (the
> > >> >> >>> > MMU is central for me in the whole picture: not move the data
> > >> >> >>> > around, but change the view of the shared data per core). The question
> > >> >> >>> > is at least certainly worth asking.
> > >> >> >>> >
> > >> >> >>> > --
> > >> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> > >> >> >>> >              http://www.kergis.com/
> > >> >> >>> >             http://kertex.kergis.com/
> > >> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
> > >> >> >
> > >> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > >> >
> > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
> > >
> > > 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mc84faae8239e0743d68adfc2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Daniel Maslowski via 9fans]

[-- Attachment #1: Type: text/plain, Size: 13495 bytes --]

Fantastic!

Ron, to make it easier, you can set the regen branch as the new default
branch in the repo settings on GitHub, so people don't accidentally file
against master.

On Thu, 9 Jan 2025, 23:22 Ron Minnich, <rminnich@p9f.org> wrote:

> WOW! Paul got it to build.
>
> git/clone git@github.com:rminnich/nix-os
> git/branch -b origin/regen -n regen
> cd sys/src/nix
> # HEY ANYONE! WANT TO FIX THIS!
> rc -x nix # set the x bits?
> # make it so it does not have to be in $home/nix-os?
>
> cd boot
> mk
> cd ../k10
> mk
> # it may seem like it hangs, it's actually waiting for your nvram key.
> # HEY ANYONE! the prompt for nvram gets buried in output. Want to fix this?
>
> vmx 9k8cpu # HEY ANYONE! vmx thinks the multiboot header in 9k8cpu is
> wrong, but it's not. This is an easy one, Look at the multiboot header
> in l32p.s
> # and see why vmx does not like it.
>
> Or just netboot a cpu server with 9k8cpu
>
> Note we decided to leave a few things for people to take a try at
> fixing. These are great little exercises. Learn to use git, learn a
> workflow, building a kernel, etc. etc.
>
> contributing:
> The github workflow is to fork github.com/rminnich/nix-os, checkout a
> branch based on regen, hack hack, commit -s, push to your branch, that
> will make a pull request.
> Very standard stuff, we don't know how to make it all work with 9front git
> yet.
>
> Questions? Put them here, and thanks in advance.
>
> ron
>
> On Wed, Jan 8, 2025 at 4:19 PM Ron Minnich <rminnich@p9f.org> wrote:
> >
> > NIX is moving forward, thank you paul!
> >
> > The branch is called regen, we have our first commit in many years.
> > Please take a look. If you submit a PR, please add a signed-off-by:
> > line.
> >
> > thanks
> >
> > On Tue, Jan 7, 2025 at 10:01 PM Ron Minnich <rminnich@p9f.org> wrote:
> > >
> > > so for work like this, my motto is commit early, commit often, to a
> > > branch we can always drop later. no harm.  It's easier (for me anyway)
> > > than shuffling patches around in email.
> > >
> > > I'm happy to accept a pull request against rminnich/nix-os, , let's
> > > call the branch regen.
> > >
> > > thanks
> > >
> > > On Tue, Jan 7, 2025 at 9:52 PM Paul Lalonde <paul.a.lalonde@gmail.com>
> wrote:
> > > >
> > > > As you say, Ron.
> > > >
> > > > First, here's my nix script, such as it is, cribbed from the old nix
> one.  It has holes, guaranteed.  Also, I went and pulled in a "user"
> directory, just for old habits dying hard.  Yes, I still use glenda on this
> old terminal.  Call me names for it.
> > > > #!/bin/rc
> > > >
> > > > unmount /sys/include >[2]/dev/null
> > > >
> > > > unmount /sys/src/libc >[2]/dev/null
> > > >
> > > > bind -b /usr/glenda/nix-os/sys/include /sys/include
> > > >
> > > > bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc
> > > >
> > > > cd /usr/glenda/nix-os/sys
> > > >
> > > > for(d in man/*){
> > > >
> > > > unmount /sys/$d >[2]/dev/null
> > > >
> > > > bind -b $d /sys/$d
> > > >
> > > > }
> > > >
> > > > exit ''
> > > >
> > > >
> > > > My terminal is a pi 400, so I had to build out the /amd64 tree,
> objtype=arm64.  I'll assume folks are clever enough to do this, or to use
> an amd64 terminal or cpu to do this work.
> > > >
> > > >
> > > > Then mk your heart out.  The main pain points are ulong parameters
> that are now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.
> These changes appear limited to
> > > >
> > > > M amd64/include/ureg.h
> > > >
> > > > M sys/include/libc.h
> > > >
> > > > M sys/src/boot/pc/lib.h
> > > >
> > > > M sys/src/nix/boot/nopsession.c
> > > >
> > > > M sys/src/nix/k10/acore.c
> > > >
> > > > M sys/src/nix/k10/fpu.c
> > > >
> > > > M sys/src/nix/k10/sipi.h
> > > >
> > > > M sys/src/nix/k10/syscall.c
> > > >
> > > > M sys/src/nix/k10/trap.c
> > > >
> > > > M sys/src/nix/port/lib.h
> > > >
> > > > M sys/src/nix/port/portfns.h
> > > >
> > > > The diffs are attached.  I don't want to commit a branch because as
> I said, I don't think my bind mappings are entirely correct, though I'm
> seeing many fewer crossed wires now.
> > > > Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot
> which *almost* makes a full build happen.  parseipmask has gained a v4
> parameter in 9front, which means the fix there needs actual analysis.
> qsort is somehow also complaining, possibly indicating I'm pulling the
> wrong header for it, indicating a problem in my bind script.
> > > >
> > > > This feels completely surmountable.
> > > >
> > > >
> > > > Paul
> > > >
> > > >
> > > > On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:
> > > >>
> > > >> if you can document your steps, then others can stand on your
> > > >> shoulders, possibly, and we can all move forward?
> > > >>
> > > >> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <
> paul.a.lalonde@gmail.com> wrote:
> > > >> >
> > > >> > Ok, not a bad first day poking at it.  I have a growing (but not
> ready) new nix script to pull the right pieces over top of my build
> environment.
> > > >> > I have a near-complete build, but with hazards: 9front has
> evolved in a number of places with many ulong parameters becoming usize.  I
> have a list of those spots, but now they need to be examined for
> over/underflow.
> > > >> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo
> includes the libboot.a6 binary, some source files that match the symbols,
> and no mkfile.  Attempting to build also shows some 9front auth changes
> that need to be incorporated into doauthenticate.c, calls to convS2M and
> convM2S that now need buffer length parameters, and the phasing of Tnop out
> 9p?  Nothing at all insurmountable.
> > > >> >
> > > >> > Not too daunting.  Next time I have a few moments I'll do a more
> principled pass on the nix script so I can share it.  I didn't understand
> enough when I first started updating it.
> > > >> >
> > > >> > Paul
> > > >> >
> > > >> >
> > > >> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org>
> wrote:
> > > >> >>
> > > >> >> if you look at the first_commit branch, you'll see a
> sys/src/nix/nix
> > > >> >> script, which sets up some binds.
> > > >> >>
> > > >> >> What we did, before building nix, on plan 9, in 2011, was a set
> of
> > > >> >> binds to get the right things such as /sys/include and so on.
> > > >> >>
> > > >> >> This  won't be just a 'mk and it builds'. There's 13 years of
> bitrot.
> > > >> >> I expect it will be strategic changes, and in the end they won't
> be
> > > >> >> all that many lines of code, but there will be some tricky stuff.
> > > >> >>
> > > >> >> Best ot take it slow, when you hit an issue, ruminate it on for
> a day
> > > >> >> or two, then look again. Otherwise you'll just get frustrated (I
> have
> > > >> >> ...) But before you make any change, be very sure you know WHY
> you're
> > > >> >> doing it, not just that 'it got me past that mk error.'
> > > >> >>
> > > >> >> Bring issues to the list and, if you want, keep a running doc to
> which
> > > >> >> others can contribute: what you did, what you ran into, what a
> fix
> > > >> >> might be. The old saying; "if you don't write it down it didn't
> > > >> >> happen"
> > > >> >>
> > > >> >> But this is the kind of thing you take slowly and carefully,
> otherwise
> > > >> >> it's total misery.
> > > >> >>
> > > >> >> ron
> > > >> >>
> > > >> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <
> paul.a.lalonde@gmail.com> wrote:
> > > >> >> >
> > > >> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.
> In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos
> structure from nix, not the one in the host system.
> > > >> >> >
> > > >> >> > How do I re-root this correctly for this build?
> > > >> >> >
> > > >> >> > Paul
> > > >> >> >
> > > >> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <
> paul.a.lalonde@gmail.com> wrote:
> > > >> >> >>
> > > >> >> >> Ok, I thought, what could do.
> > > >> >> >>
> > > >> >> >> So I went to my rPi 400, set up SSH for github, got Ron's
> nix-os repo and hit "mk".
> > > >> >> >> When that errored out a bunch I realized that I needed /amd64
> built, so I did that.  Just as painless as I remembered.
> > > >> >> >>
> > > >> >> >> And now, I get a ways further into the build, but hit an
> incompatibility between the my /amd64/include/ureg.h and
> .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX
> code was written someone decided that the program counter should be called
> pc instead of ip.
> > > >> >> >>
> > > >> >> >> Or else, I'm approaching this all wrong, and Ron can shed
> some light on how I should be proceeding.
> > > >> >> >>
> > > >> >> >> Paul
> > > >> >> >>
> > > >> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org>
> wrote:
> > > >> >> >>>
> > > >> >> >>> I found the original 2011 paper, which was a sandia report,
> from may
> > > >> >> >>> 2011. It's a modification of the original proposal, which I
> no longer
> > > >> >> >>> have; but it is a good summary of where we were at the end
> of my visit
> > > >> >> >>> in May.
> > > >> >> >>>
> > > >> >> >>> This is interesting: "We have changed a surprisingly small
> amount of
> > > >> >> >>> code at this point.
> > > >> >> >>> There are about 400 lines of new
> > > >> >> >>> assembler source, about 80 lines of platform independent C
> source, and
> > > >> >> >>> about 350 lines of AMD64 C
> > > >> >> >>> source code. To this, we have to add a few extra source
> lines in the
> > > >> >> >>> start-up code, system call, and trap han-
> > > >> >> >>> dlers. This implementation is being both developed and
> tested only in
> > > >> >> >>> the AMD64 architecture."
> > > >> >> >>>
> > > >> >> >>> I uploaded it to the Plan 9 foundation shared drive:
> > > >> >> >>>
> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
> > > >> >> >>>
> > > >> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
> > > >> >> >>> >
> > > >> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich
> wrote:
> > > >> >> >>> > >
> > > >> >> >>> > > Why NIX?
> > > >> >> >>> > >
> > > >> >> >>> > > If you think about it, timesharing is designed for a
> world where cores
> > > >> >> >>> > > are scarce. But on a machine with hundreds of cores,
> running Plan 9,
> > > >> >> >>> > > there are < 100 processes. We can assign a core to each
> process, and
> > > >> >> >>> > > let those processes own the core until they are done.
> This might be a
> > > >> >> >>> > > useful simplification, it might not, but it's something.
> > > >> >> >>> > >
> > > >> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and
> the results were
> > > >> >> >>> > > good. I was always curious how it would work if we had
> those
> > > >> >> >>> > > multi-hundred-core machines Intel and IBM and others
> were telling us
> > > >> >> >>> > > about in 2011. Now that we have them, it would be
> interesting to try.
> > > >> >> >>> >
> > > >> >> >>> > As said previously, I will start wandering and stumbling
> upon problems
> > > >> >> >>> > this week-end---I'm a toddler in the area, so it's the way
> to learn to
> > > >> >> >>> > walk.
> > > >> >> >>> >
> > > >> >> >>> > But this brief summary highlight a solution and questions
> > > >> >> >>> > that are, IMHO, valid questions: remember the "war" between
> > > >> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the
> kernel is not a
> > > >> >> >>> > separate process (well: there are "administrative"
> processes,
> > > >> >> >>> > scheduler and pager but...) but part of the applications.
> This is also
> > > >> >> >>> > why it is efficient compared to "message passing"
> micro-kernels that
> > > >> >> >>> > are not "near" enough the hardware---so inefficient that,
> for
> > > >> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in
> assembly to
> > > >> >> >>> > improve the result...
> > > >> >> >>> >
> > > >> >> >>> > But multiple cores (and even in the smaller machines
> nowadays, you
> > > >> >> >>> > find two) present another mean of articulation of the OS
> code (the
> > > >> >> >>> > MMU is central for me in the whole picture: not move the
> data
> > > >> >> >>> > around, but change the view of the shared data per core).
> The question
> > > >> >> >>> > is at least certainly worth asking.
> > > >> >> >>> >
> > > >> >> >>> > --
> > > >> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
> > > >> >> >>> >              http://www.kergis.com/
> > > >> >> >>> >             http://kertex.kergis.com/
> > > >> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95
> 6006 F40C
> > > >> >> >
> > > >> >> > 9fans / 9fans / see discussions + participants + delivery
> options Permalink
> > > >> >
> > > >> > 9fans / 9fans / see discussions + participants + delivery options
> Permalink
> > > >
> > > > 9fans / 9fans / see discussions + participants + delivery options
> Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mb71d11397ae3ece9bfa9087b
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 20973 bytes --]

Re: [9fans] NIX experience

[Ron Minnich]

Thanks, Daniel, done.


On Thu, Jan 9, 2025 at 5:59 PM Daniel Maslowski via 9fans
<9fans@9fans.net> wrote:
>
> Fantastic!
>
> Ron, to make it easier, you can set the regen branch as the new default branch in the repo settings on GitHub, so people don't accidentally file against master.
>
> On Thu, 9 Jan 2025, 23:22 Ron Minnich, <rminnich@p9f.org> wrote:
>>
>> WOW! Paul got it to build.
>>
>> git/clone git@github.com:rminnich/nix-os
>> git/branch -b origin/regen -n regen
>> cd sys/src/nix
>> # HEY ANYONE! WANT TO FIX THIS!
>> rc -x nix # set the x bits?
>> # make it so it does not have to be in $home/nix-os?
>>
>> cd boot
>> mk
>> cd ../k10
>> mk
>> # it may seem like it hangs, it's actually waiting for your nvram key.
>> # HEY ANYONE! the prompt for nvram gets buried in output. Want to fix this?
>>
>> vmx 9k8cpu # HEY ANYONE! vmx thinks the multiboot header in 9k8cpu is
>> wrong, but it's not. This is an easy one, Look at the multiboot header
>> in l32p.s
>> # and see why vmx does not like it.
>>
>> Or just netboot a cpu server with 9k8cpu
>>
>> Note we decided to leave a few things for people to take a try at
>> fixing. These are great little exercises. Learn to use git, learn a
>> workflow, building a kernel, etc. etc.
>>
>> contributing:
>> The github workflow is to fork github.com/rminnich/nix-os, checkout a
>> branch based on regen, hack hack, commit -s, push to your branch, that
>> will make a pull request.
>> Very standard stuff, we don't know how to make it all work with 9front git yet.
>>
>> Questions? Put them here, and thanks in advance.
>>
>> ron
>>
>> On Wed, Jan 8, 2025 at 4:19 PM Ron Minnich <rminnich@p9f.org> wrote:
>> >
>> > NIX is moving forward, thank you paul!
>> >
>> > The branch is called regen, we have our first commit in many years.
>> > Please take a look. If you submit a PR, please add a signed-off-by:
>> > line.
>> >
>> > thanks
>> >
>> > On Tue, Jan 7, 2025 at 10:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>> > >
>> > > so for work like this, my motto is commit early, commit often, to a
>> > > branch we can always drop later. no harm.  It's easier (for me anyway)
>> > > than shuffling patches around in email.
>> > >
>> > > I'm happy to accept a pull request against rminnich/nix-os, , let's
>> > > call the branch regen.
>> > >
>> > > thanks
>> > >
>> > > On Tue, Jan 7, 2025 at 9:52 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> > > >
>> > > > As you say, Ron.
>> > > >
>> > > > First, here's my nix script, such as it is, cribbed from the old nix one.  It has holes, guaranteed.  Also, I went and pulled in a "user" directory, just for old habits dying hard.  Yes, I still use glenda on this old terminal.  Call me names for it.
>> > > >
>> > > > #!/bin/rc
>> > > >
>> > > > unmount /sys/include >[2]/dev/null
>> > > >
>> > > > unmount /sys/src/libc >[2]/dev/null
>> > > >
>> > > > bind -b /usr/glenda/nix-os/sys/include /sys/include
>> > > >
>> > > > bind -c /usr/glenda/nix-os/sys/src/libc /sys/src/libc
>> > > >
>> > > > cd /usr/glenda/nix-os/sys
>> > > >
>> > > > for(d in man/*){
>> > > >
>> > > > unmount /sys/$d >[2]/dev/null
>> > > >
>> > > > bind -b $d /sys/$d
>> > > >
>> > > > }
>> > > >
>> > > > exit ''
>> > > >
>> > > >
>> > > > My terminal is a pi 400, so I had to build out the /amd64 tree, objtype=arm64.  I'll assume folks are clever enough to do this, or to use an amd64 terminal or cpu to do this work.
>> > > >
>> > > >
>> > > > Then mk your heart out.  The main pain points are ulong parameters that are now usize in 9front, and the renaming of Ureg.ip to Ureg.pc.  These changes appear limited to
>> > > >
>> > > > M amd64/include/ureg.h
>> > > >
>> > > > M sys/include/libc.h
>> > > >
>> > > > M sys/src/boot/pc/lib.h
>> > > >
>> > > > M sys/src/nix/boot/nopsession.c
>> > > >
>> > > > M sys/src/nix/k10/acore.c
>> > > >
>> > > > M sys/src/nix/k10/fpu.c
>> > > >
>> > > > M sys/src/nix/k10/sipi.h
>> > > >
>> > > > M sys/src/nix/k10/syscall.c
>> > > >
>> > > > M sys/src/nix/k10/trap.c
>> > > >
>> > > > M sys/src/nix/port/lib.h
>> > > >
>> > > > M sys/src/nix/port/portfns.h
>> > > >
>> > > > The diffs are attached.  I don't want to commit a branch because as I said, I don't think my bind mappings are entirely correct, though I'm seeing many fewer crossed wires now.
>> > > > Attached is the (trivial) mkfile I built for nix-os/sys/nix/boot which *almost* makes a full build happen.  parseipmask has gained a v4 parameter in 9front, which means the fix there needs actual analysis.  qsort is somehow also complaining, possibly indicating I'm pulling the wrong header for it, indicating a problem in my bind script.
>> > > >
>> > > > This feels completely surmountable.
>> > > >
>> > > >
>> > > > Paul
>> > > >
>> > > >
>> > > > On Tue, Jan 7, 2025 at 9:29 PM Ron Minnich <rminnich@p9f.org> wrote:
>> > > >>
>> > > >> if you can document your steps, then others can stand on your
>> > > >> shoulders, possibly, and we can all move forward?
>> > > >>
>> > > >> On Tue, Jan 7, 2025 at 9:08 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> > > >> >
>> > > >> > Ok, not a bad first day poking at it.  I have a growing (but not ready) new nix script to pull the right pieces over top of my build environment.
>> > > >> > I have a near-complete build, but with hazards: 9front has evolved in a number of places with many ulong parameters becoming usize.  I have a list of those spots, but now they need to be examined for over/underflow.
>> > > >> > The last puzzle of the day is nix-os/sys/src/nix/boot.  The repo includes the libboot.a6 binary, some source files that match the symbols, and no mkfile.  Attempting to build also shows some 9front auth changes that need to be incorporated into doauthenticate.c, calls to convS2M and convM2S that now need buffer length parameters, and the phasing of Tnop out 9p?  Nothing at all insurmountable.
>> > > >> >
>> > > >> > Not too daunting.  Next time I have a few moments I'll do a more principled pass on the nix script so I can share it.  I didn't understand enough when I first started updating it.
>> > > >> >
>> > > >> > Paul
>> > > >> >
>> > > >> >
>> > > >> > On Tue, Jan 7, 2025 at 6:58 PM Ron Minnich <rminnich@p9f.org> wrote:
>> > > >> >>
>> > > >> >> if you look at the first_commit branch, you'll see a sys/src/nix/nix
>> > > >> >> script, which sets up some binds.
>> > > >> >>
>> > > >> >> What we did, before building nix, on plan 9, in 2011, was a set of
>> > > >> >> binds to get the right things such as /sys/include and so on.
>> > > >> >>
>> > > >> >> This  won't be just a 'mk and it builds'. There's 13 years of bitrot.
>> > > >> >> I expect it will be strategic changes, and in the end they won't be
>> > > >> >> all that many lines of code, but there will be some tricky stuff.
>> > > >> >>
>> > > >> >> Best ot take it slow, when you hit an issue, ruminate it on for a day
>> > > >> >> or two, then look again. Otherwise you'll just get frustrated (I have
>> > > >> >> ...) But before you make any change, be very sure you know WHY you're
>> > > >> >> doing it, not just that 'it got me past that mk error.'
>> > > >> >>
>> > > >> >> Bring issues to the list and, if you want, keep a running doc to which
>> > > >> >> others can contribute: what you did, what you ran into, what a fix
>> > > >> >> might be. The old saying; "if you don't write it down it didn't
>> > > >> >> happen"
>> > > >> >>
>> > > >> >> But this is the kind of thing you take slowly and carefully, otherwise
>> > > >> >> it's total misery.
>> > > >> >>
>> > > >> >> ron
>> > > >> >>
>> > > >> >> On Tue, Jan 7, 2025 at 5:34 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> > > >> >> >
>> > > >> >> > And a bit more digging.  Yes, I'm clearly doing this wrong.  In building nix-os/sys/src/k10/trap.c it should absolutely be using the Tos structure from nix, not the one in the host system.
>> > > >> >> >
>> > > >> >> > How do I re-root this correctly for this build?
>> > > >> >> >
>> > > >> >> > Paul
>> > > >> >> >
>> > > >> >> > On Tue, Jan 7, 2025 at 4:47 PM Paul Lalonde <paul.a.lalonde@gmail.com> wrote:
>> > > >> >> >>
>> > > >> >> >> Ok, I thought, what could do.
>> > > >> >> >>
>> > > >> >> >> So I went to my rPi 400, set up SSH for github, got Ron's nix-os repo and hit "mk".
>> > > >> >> >> When that errored out a bunch I realized that I needed /amd64 built, so I did that.  Just as painless as I remembered.
>> > > >> >> >>
>> > > >> >> >> And now, I get a ways further into the build, but hit an incompatibility between the my /amd64/include/ureg.h and .../nix-os/amd64/include/ureg.h.  It seems that at some point since the NIX code was written someone decided that the program counter should be called pc instead of ip.
>> > > >> >> >>
>> > > >> >> >> Or else, I'm approaching this all wrong, and Ron can shed some light on how I should be proceeding.
>> > > >> >> >>
>> > > >> >> >> Paul
>> > > >> >> >>
>> > > >> >> >> On Tue, Jan 7, 2025 at 4:01 PM Ron Minnich <rminnich@p9f.org> wrote:
>> > > >> >> >>>
>> > > >> >> >>> I found the original 2011 paper, which was a sandia report, from may
>> > > >> >> >>> 2011. It's a modification of the original proposal, which I no longer
>> > > >> >> >>> have; but it is a good summary of where we were at the end of my visit
>> > > >> >> >>> in May.
>> > > >> >> >>>
>> > > >> >> >>> This is interesting: "We have changed a surprisingly small amount of
>> > > >> >> >>> code at this point.
>> > > >> >> >>> There are about 400 lines of new
>> > > >> >> >>> assembler source, about 80 lines of platform independent C source, and
>> > > >> >> >>> about 350 lines of AMD64 C
>> > > >> >> >>> source code. To this, we have to add a few extra source lines in the
>> > > >> >> >>> start-up code, system call, and trap han-
>> > > >> >> >>> dlers. This implementation is being both developed and tested only in
>> > > >> >> >>> the AMD64 architecture."
>> > > >> >> >>>
>> > > >> >> >>> I uploaded it to the Plan 9 foundation shared drive:
>> > > >> >> >>> https://drive.google.com/file/d/1F41_4MFpio3UsnxOpTJBiypUrHjkinL-/view?usp=share_link
>> > > >> >> >>>
>> > > >> >> >>> On Tue, Jan 7, 2025 at 10:18 AM <tlaronde@kergis.com> wrote:
>> > > >> >> >>> >
>> > > >> >> >>> > On Tue, Jan 07, 2025 at 09:20:06AM -0800, Ron Minnich wrote:
>> > > >> >> >>> > >
>> > > >> >> >>> > > Why NIX?
>> > > >> >> >>> > >
>> > > >> >> >>> > > If you think about it, timesharing is designed for a world where cores
>> > > >> >> >>> > > are scarce. But on a machine with hundreds of cores, running Plan 9,
>> > > >> >> >>> > > there are < 100 processes. We can assign a core to each process, and
>> > > >> >> >>> > > let those processes own the core until they are done. This might be a
>> > > >> >> >>> > > useful simplification, it might not, but it's something.
>> > > >> >> >>> > >
>> > > >> >> >>> > > I did run some standard HPC benchmarks on NIX ACs and the results were
>> > > >> >> >>> > > good. I was always curious how it would work if we had those
>> > > >> >> >>> > > multi-hundred-core machines Intel and IBM and others were telling us
>> > > >> >> >>> > > about in 2011. Now that we have them, it would be interesting to try.
>> > > >> >> >>> >
>> > > >> >> >>> > As said previously, I will start wandering and stumbling upon problems
>> > > >> >> >>> > this week-end---I'm a toddler in the area, so it's the way to learn to
>> > > >> >> >>> > walk.
>> > > >> >> >>> >
>> > > >> >> >>> > But this brief summary highlight a solution and questions
>> > > >> >> >>> > that are, IMHO, valid questions: remember the "war" between
>> > > >> >> >>> > "micro-kernels" and "monolithic kernels"? In Unix, the kernel is not a
>> > > >> >> >>> > separate process (well: there are "administrative" processes,
>> > > >> >> >>> > scheduler and pager but...) but part of the applications. This is also
>> > > >> >> >>> > why it is efficient compared to "message passing" micro-kernels that
>> > > >> >> >>> > are not "near" enough the hardware---so inefficient that, for
>> > > >> >> >>> > ideologic purposes, some have rewritten "micro-kernels" in assembly to
>> > > >> >> >>> > improve the result...
>> > > >> >> >>> >
>> > > >> >> >>> > But multiple cores (and even in the smaller machines nowadays, you
>> > > >> >> >>> > find two) present another mean of articulation of the OS code (the
>> > > >> >> >>> > MMU is central for me in the whole picture: not move the data
>> > > >> >> >>> > around, but change the view of the shared data per core). The question
>> > > >> >> >>> > is at least certainly worth asking.
>> > > >> >> >>> >
>> > > >> >> >>> > --
>> > > >> >> >>> > Thierry Laronde <tlaronde +AT+ kergis +dot+ com>
>> > > >> >> >>> >              http://www.kergis.com/
>> > > >> >> >>> >             http://kertex.kergis.com/
>> > > >> >> >>> > Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>> > > >> >> >
>> > > >> >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>> > > >> >
>> > > >> > 9fans / 9fans / see discussions + participants + delivery options Permalink
>> > > >
>> > > > 9fans / 9fans / see discussions + participants + delivery options Permalink
>
> 9fans / 9fans / see discussions + participants + delivery options Permalink

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-M1548153856197d17c27a6fc3
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] NIX experience

[Andreas.Elding via 9fans]

> What would you like to know? I also have an initial broken port to > 9front if you'd like to try to bring it to life.

Thank you for the response, it was quite an interesting read. Unfortunately, I'm not a great coder, so I can't take you up on that offer. You mentioned that GPUs took over, but not all problems can run on GPUs. There may still be a general interest in this (I know I am interested).

How was it presented to the users? Could they query to see the current utilization of the system?

How did you know that a job completed (or failed)? 

You mentioned it was a shared memory system, meaning it was in essence a "very large SMP machine" from the view of the OS?

Could the NIX system only work with shared memory systems like that, or was it possible to take many smaller independent systems and combine their resources?

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/Tacd1c3c541277b7f-M3d221a244a4e706d0a7f15d7
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription