[9front] Thoughts on Wayland?

[9front] Thoughts on Wayland?

[Willow Liquorice]

Hello again 9front,

(Warning: long email is long)

I've been leafing through the Wayland protocol, comparing it to rio and 
/dev/draw. To me, a system that could speak Wayland sounds much more 
compatible with the wider software ecosystem.

Has there been much thought put into Wayland in 9front? I'd poke around 
in the list archives, but I haven't figured out how to 9P on my Linux 
machine yet.

Here are my thoughts:

1. The Wayland compositor and clients talk to each other over a Unix 
domain socket. I'm guessing Plan 9 doesn't have those (and doesn't want 
them) as there's nothing in sys/man on the subject. Could pipe(3) do the 
trick?

2. Shared memory (where wl_buffers are stored) is achieved by passing a 
file descriptor from the client to the compositor, which the compositor 
mmaps as needed.

If I understand correctly, that already sounds pretty plan9-y, though 
I'm not sure whether you could then treat the file *like* shared memory 
under 9front. Again, couldn't find any references to "mmap" in the 
manuals, so I guess there's no direct analogue. Could you achieve the 
same (or similar) effect with a ramfs?

3. protocol/wayland.xml is apparently used to generate all of the 
function prototypes you need to speak Wayland. If the underlying 
components (shared memory and a domain socket lookalike) work, making a 
Wayland library for Plan 9 (Way 9?) *might* boil down to a 
paint-by-numbers exercise.

Wayland's been around a while, so maybe none of these observations are 
new. I don't know! I'm a novice.

Regards,
	Willow

Re: [9front] Thoughts on Wayland?

[Stuart Morrow]

How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

And here is where the trouble begins lol.

On 04/08/2024 20:44, Stuart Morrow wrote:
> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

always better to be a willow that bends than the strongest oak in a storm :)

there's a common misconception that plan 9 is an OS for a computer. plan 9
is an OS for a network. drawing fast locally is not a plan 9 concept. plan
9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to
draw somewhere else on the network would be good and appreciated and a
relevant plan 9 concept and people are working on it... but for a fast
local wayland implementation, that's what linux is, and linux can be and is
a part of plan 9 which is an OS for a network

On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:

> How do you share memory over rcpu?
>

[-- Attachment #2: Type: text/html, Size: 1035 bytes --]

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

actually, drawing fast locally (and GPU stuff especially for
computation) is definitely a plan 9 concept too and is also being
worked on separately from wayland, and possibly on linux... so... plan
9 is an OS for networks that may include plan 9 machines, linux
machines, iPhones, etc... it's a "unix made out of a network of
computers" not a "network made out of unix computers"

On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
>
> always better to be a willow that bends than the strongest oak in a storm :)
>
> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
>
> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>>
>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

Though I guess there's nothing stopping you putting the draw on the same 
machine as the client program…

On 04/08/2024 20:54, Eli Cohen wrote:
> always better to be a willow that bends than the strongest oak in a storm :)
> 
> there's a common misconception that plan 9 is an OS for a computer. plan 
> 9 is an OS for a network. drawing fast locally is not a plan 9 concept. 
> plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that 
> renders to draw somewhere else on the network would be good and 
> appreciated and a relevant plan 9 concept and people are working on 
> it... but for a fast local wayland implementation, that's what linux is, 
> and linux can be and is a part of plan 9 which is an OS for a network
> 
> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com 
> <mailto:morrow.stuart@gmail.com>> wrote:
> 
>     How do you share memory over rcpu?
> 

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

also as far as mmap goes and what it even means, I have no idea, but a
stupid younger version of me, who was myself at the time, managed to
change this audio driver
http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
to support some mmap'ed I/O ac97 chips. the part where it says ismmio
= 1, that's a flag that those chips are mmap'ed ac97 chips. but now
I'm older and still honestly have no idea what you're talking about

On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
>
> actually, drawing fast locally (and GPU stuff especially for
> computation) is definitely a plan 9 concept too and is also being
> worked on separately from wayland, and possibly on linux... so... plan
> 9 is an OS for networks that may include plan 9 machines, linux
> machines, iPhones, etc... it's a "unix made out of a network of
> computers" not a "network made out of unix computers"
>
> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> >
> > always better to be a willow that bends than the strongest oak in a storm :)
> >
> > there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
> >
> > On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >>
> >> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

no, draw or anything else can be anywhere on the network, at least as
long as the interconnection is fast enough for it to work properly

On Sun, Aug 4, 2024 at 1:04 PM Eli Cohen <echoline@gmail.com> wrote:
>
> also as far as mmap goes and what it even means, I have no idea, but a
> stupid younger version of me, who was myself at the time, managed to
> change this audio driver
> http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
> to support some mmap'ed I/O ac97 chips. the part where it says ismmio
> = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
> I'm older and still honestly have no idea what you're talking about
>
> On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
> >
> > actually, drawing fast locally (and GPU stuff especially for
> > computation) is definitely a plan 9 concept too and is also being
> > worked on separately from wayland, and possibly on linux... so... plan
> > 9 is an OS for networks that may include plan 9 machines, linux
> > machines, iPhones, etc... it's a "unix made out of a network of
> > computers" not a "network made out of unix computers"
> >
> > On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> > >
> > > always better to be a willow that bends than the strongest oak in a storm :)
> > >
> > > there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
> > >
> > > On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > >>
> > >> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[mkf9]

mcf had one called wl9,

I think.

Aug 5, 2024 12:25:13 AM Eli Cohen <echoline@gmail.com>:

> always better to be a willow that bends than the strongest oak in a 
> storm :)
>
> there's a common misconception that plan 9 is an OS for a computer. 
> plan 9
> is an OS for a network. drawing fast locally is not a plan 9 concept. 
> plan
> 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that 
> renders to
> draw somewhere else on the network would be good and appreciated and a
> relevant plan 9 concept and people are working on it... but for a fast
> local wayland implementation, that's what linux is, and linux can be 
> and is
> a part of plan 9 which is an OS for a network
>
> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> 
> wrote:
>
>> How do you share memory over rcpu?
>>

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

I presume the Wayland spec is talking about mmap(2) under Linux, which 
allows you to treat a section of a file as you would program memory.

On 04/08/2024 21:04, Eli Cohen wrote:
> also as far as mmap goes and what it even means, I have no idea, but a
> stupid younger version of me, who was myself at the time, managed to
> change this audio driver
> http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
> to support some mmap'ed I/O ac97 chips. the part where it says ismmio
> = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
> I'm older and still honestly have no idea what you're talking about
> 
> On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
>>
>> actually, drawing fast locally (and GPU stuff especially for
>> computation) is definitely a plan 9 concept too and is also being
>> worked on separately from wayland, and possibly on linux... so... plan
>> 9 is an OS for networks that may include plan 9 machines, linux
>> machines, iPhones, etc... it's a "unix made out of a network of
>> computers" not a "network made out of unix computers"
>>
>> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
>>>
>>> always better to be a willow that bends than the strongest oak in a storm :)
>>>
>>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
>>>
>>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>>>>
>>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

I think that's vmap in the Plan 9 kernel? I honestly have no idea what
you're talking about.

also, I am on the wrong mailing list. 9front is a fork of Plan 9 that
is helping Plan 9 be an OS for computers but is still Plan 9

On Sun, Aug 4, 2024 at 1:13 PM Willow Liquorice <willow@howhill.com> wrote:
>
> I presume the Wayland spec is talking about mmap(2) under Linux, which
> allows you to treat a section of a file as you would program memory.
>
> On 04/08/2024 21:04, Eli Cohen wrote:
> > also as far as mmap goes and what it even means, I have no idea, but a
> > stupid younger version of me, who was myself at the time, managed to
> > change this audio driver
> > http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
> > to support some mmap'ed I/O ac97 chips. the part where it says ismmio
> > = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
> > I'm older and still honestly have no idea what you're talking about
> >
> > On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
> >>
> >> actually, drawing fast locally (and GPU stuff especially for
> >> computation) is definitely a plan 9 concept too and is also being
> >> worked on separately from wayland, and possibly on linux... so... plan
> >> 9 is an OS for networks that may include plan 9 machines, linux
> >> machines, iPhones, etc... it's a "unix made out of a network of
> >> computers" not a "network made out of unix computers"
> >>
> >> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> >>>
> >>> always better to be a willow that bends than the strongest oak in a storm :)
> >>>
> >>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
> >>>
> >>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >>>>
> >>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[Pavel Renev]

>2. Shared memory (where wl_buffers are stored) is achieved by passing a file descriptor from the client to the compositor, which the compositor mmaps as needed.
>
>Could you achieve the same (or similar) effect with a ramfs?

IIRC, Haiku's Wayland shim indeed uses a file on ramdisk as a substitute for mmap. I am not familiar with the details however.

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

Here? https://git.sr.ht/~mcf/wl9

Seems pretty quiet, but it might be worth a poke around in. It's far 
from done by their own admission.

On 04/08/2024 21:08, mkf9 wrote:
> mcf had one called wl9,
> 
> I think.
> 
> Aug 5, 2024 12:25:13 AM Eli Cohen <echoline@gmail.com>:
> 
>> always better to be a willow that bends than the strongest oak in a 
>> storm :)
>>
>> there's a common misconception that plan 9 is an OS for a computer. 
>> plan 9
>> is an OS for a network. drawing fast locally is not a plan 9 concept. 
>> plan
>> 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that 
>> renders to
>> draw somewhere else on the network would be good and appreciated and a
>> relevant plan 9 concept and people are working on it... but for a fast
>> local wayland implementation, that's what linux is, and linux can be 
>> and is
>> a part of plan 9 which is an OS for a network
>>
>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> 
>> wrote:
>>
>>> How do you share memory over rcpu?
>>>

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

It sounds like the "shared memory" concept Wayland uses is the tricky 
bit, then, because you can't put the same RAM stick in two different 
machines.

Though, if I'm understanding pipe(3) correctly, it reads like the right 
tool to talk to the compositor.

Thinking about it, I guess the compositor and the in-use graphics 
resources (hardware, software, data) ought to be resident on the same 
physical system: a "GPU server", by analogy with a CPU server, though 
"composition server" is probably a more accurate term.

Thinking about it even more, if *all* graphics resources, in use or not, 
are resident on the GPU server, you don't have to shove huge fonts or 
textures over the network, just the rendered images (or diffs!) and some 
RPCs to a process resident on the GPU server to get it drawing what you 
want.

How on earth that makes a program that uses Pango+Cairo, or something 
like that, play nicely on 9 I have no idea… more shims for client programs?

I suppose every terminal is, in and of itself, a GPU server. Maybe I'm 
overthinking it.

On 04/08/2024 20:59, Eli Cohen wrote:
> actually, drawing fast locally (and GPU stuff especially for
> computation) is definitely a plan 9 concept too and is also being
> worked on separately from wayland, and possibly on linux... so... plan
> 9 is an OS for networks that may include plan 9 machines, linux
> machines, iPhones, etc... it's a "unix made out of a network of
> computers" not a "network made out of unix computers"
> 
> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
>>
>> always better to be a willow that bends than the strongest oak in a storm :)
>>
>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
>>
>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>>>
>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[ori]

Yes, plan 9 doesn't support that -- resources can be located anywhere
on the network, which comes with odd implications about failure domains.

With mmap, this is is very difficult to paper over on a single machine;
it gets worse when multiple pieces of hardware are involved.

While it's theoretically possible to do mmap over 9p, it's not going to
be a good experience.

Quoth Willow Liquorice <willow@howhill.com>:
> I presume the Wayland spec is talking about mmap(2) under Linux, which 
> allows you to treat a section of a file as you would program memory.
> 
> On 04/08/2024 21:04, Eli Cohen wrote:
> > also as far as mmap goes and what it even means, I have no idea, but a
> > stupid younger version of me, who was myself at the time, managed to
> > change this audio driver
> > http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
> > to support some mmap'ed I/O ac97 chips. the part where it says ismmio
> > = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
> > I'm older and still honestly have no idea what you're talking about
> > 
> > On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
> >>
> >> actually, drawing fast locally (and GPU stuff especially for
> >> computation) is definitely a plan 9 concept too and is also being
> >> worked on separately from wayland, and possibly on linux... so... plan
> >> 9 is an OS for networks that may include plan 9 machines, linux
> >> machines, iPhones, etc... it's a "unix made out of a network of
> >> computers" not a "network made out of unix computers"
> >>
> >> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> >>>
> >>> always better to be a willow that bends than the strongest oak in a storm :)
> >>>
> >>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
> >>>
> >>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> >>>>
> >>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[sirjofri]

04.08.2024 22:02:42 Willow Liquorice <willow@howhill.com>:
> Though I guess there's nothing stopping you putting the draw on the same machine as the client program…

What's the client, what's the server?

Devdraw is a server and rio would be a client. On plan 9, where those programs run doesn't matter at all.

Regarding GPUs, the gpu will be the server and part of the client program can be loaded into that server (a "shader") which would speed up calculation a lot by using the power of the gpu and decreasing the latency in that part of the program. It's clear how the GPU generally works and can be integrated, the server side is in an early stage of development (beginning with drawterm but that project is currently stalled due to a lack of a computer). When it is done, you can think of the GPU as another part of the plan 9 network/installation.

sirjofri

Re: [9front] Thoughts on Wayland?

[ori]

Quoth Willow Liquorice <willow@howhill.com>:
> Thinking about it even more, if *all* graphics resources, in use or not, 
> are resident on the GPU server, you don't have to shove huge fonts or 
> textures over the network, just the rendered images (or diffs!) and some 
> RPCs to a process resident on the GPU server to get it drawing what you 
> want.

That's what devdraw does.

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

[-- Attachment #1: Type: text/plain, Size: 3374 bytes --]

well, ok, the topic here is kind of that a wayland implementation opens us
to more things that use wayland. and this is actually already true on
linux... and linux already exists and already does all of this. and Plan 9
networks can (and in my personal opinion, (only speaking for myself though)
should) include linux machines. so if a wayland compositor is a draw
program at the bottom framebuffer or whatever we're calling it, and draw is
at the bottom, then it can interoperate well with Plan 9 draw displays
anywhere on the network. or pango, cairo, x11, etc... if they draw to a
draw display it can be anywhere on the Plan 9 network. the display and the
program don't have to be on the same computer but they can be. for a
program that already runs on linux, in my personal opinion, just use
linux... but if it draws to draw, that would be awesome for a Plan 9 network

On Sun, Aug 4, 2024, 2:15 PM Willow Liquorice <willow@howhill.com> wrote:

> It sounds like the "shared memory" concept Wayland uses is the tricky
> bit, then, because you can't put the same RAM stick in two different
> machines.
>
> Though, if I'm understanding pipe(3) correctly, it reads like the right
> tool to talk to the compositor.
>
> Thinking about it, I guess the compositor and the in-use graphics
> resources (hardware, software, data) ought to be resident on the same
> physical system: a "GPU server", by analogy with a CPU server, though
> "composition server" is probably a more accurate term.
>
> Thinking about it even more, if *all* graphics resources, in use or not,
> are resident on the GPU server, you don't have to shove huge fonts or
> textures over the network, just the rendered images (or diffs!) and some
> RPCs to a process resident on the GPU server to get it drawing what you
> want.
>
> How on earth that makes a program that uses Pango+Cairo, or something
> like that, play nicely on 9 I have no idea… more shims for client programs?
>
> I suppose every terminal is, in and of itself, a GPU server. Maybe I'm
> overthinking it.
>
> On 04/08/2024 20:59, Eli Cohen wrote:
> > actually, drawing fast locally (and GPU stuff especially for
> > computation) is definitely a plan 9 concept too and is also being
> > worked on separately from wayland, and possibly on linux... so... plan
> > 9 is an OS for networks that may include plan 9 machines, linux
> > machines, iPhones, etc... it's a "unix made out of a network of
> > computers" not a "network made out of unix computers"
> >
> > On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> >>
> >> always better to be a willow that bends than the strongest oak in a
> storm :)
> >>
> >> there's a common misconception that plan 9 is an OS for a computer.
> plan 9 is an OS for a network. drawing fast locally is not a plan 9
> concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc
> that renders to draw somewhere else on the network would be good and
> appreciated and a relevant plan 9 concept and people are working on it...
> but for a fast local wayland implementation, that's what linux is, and
> linux can be and is a part of plan 9 which is an OS for a network
> >>
> >> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com>
> wrote:
> >>>
> >>> How do you share memory over rcpu?
>

[-- Attachment #2: Type: text/html, Size: 4058 bytes --]

Re: [9front] Thoughts on Wayland?

[ori]

Quoth Willow Liquorice <willow@howhill.com>:
> Hello again 9front,
> 
> (Warning: long email is long)
> 
> I've been leafing through the Wayland protocol, comparing it to rio and 
> /dev/draw. To me, a system that could speak Wayland sounds much more 
> compatible with the wider software ecosystem.
> 

A system that fully implements the posix APIs would also be more
compatible with the wider software ecosystem, however it would not
be more pleasant to program for or interact with; The appeal of
Plan 9 is that it's simplified things to the essentials, and
doesn't pull in a bunch of overengineered junk.

So, as for Wayland specifically: The biggest problem with it is
that it's not a protocol. It's got a core that's insufficient for
actual use, and a set of mutually incompatible extensisons that
forces each user to write programs that explicitly support wlroots,
gnome, and KDE with separate hacks. To pick on an example, cursor
warping in drawterm only works in some environments, because others
fail to support those specific extensisons.

On top of that, the protocol's model of resources is rather
overengineered and painful to use.

More or less, Wayland was misdesigned on different axes than X11;
it's not clear which one is a worse approach.

However, Wayland is clearly a worse fit for Plan 9 than devdraw;
it's designed for the case where not only does the GPU live on the
same machine as the compositor, it's designed for the case where
every single piece of software doing drawing lives on the same
machine as well.

Re: [9front] Thoughts on Wayland?

[ori]

('that', of course, being mmap)

Quoth ori@eigenstate.org:
> Yes, plan 9 doesn't support that -- resources can be located anywhere
> on the network, which comes with odd implications about failure domains.
> 
> With mmap, this is is very difficult to paper over on a single machine;
> it gets worse when multiple pieces of hardware are involved.
> 
> While it's theoretically possible to do mmap over 9p, it's not going to
> be a good experience.
> 
> Quoth Willow Liquorice <willow@howhill.com>:
> > I presume the Wayland spec is talking about mmap(2) under Linux, which 
> > allows you to treat a section of a file as you would program memory.
> > 
> > On 04/08/2024 21:04, Eli Cohen wrote:
> > > also as far as mmap goes and what it even means, I have no idea, but a
> > > stupid younger version of me, who was myself at the time, managed to
> > > change this audio driver
> > > http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
> > > to support some mmap'ed I/O ac97 chips. the part where it says ismmio
> > > = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
> > > I'm older and still honestly have no idea what you're talking about
> > > 
> > > On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
> > >>
> > >> actually, drawing fast locally (and GPU stuff especially for
> > >> computation) is definitely a plan 9 concept too and is also being
> > >> worked on separately from wayland, and possibly on linux... so... plan
> > >> 9 is an OS for networks that may include plan 9 machines, linux
> > >> machines, iPhones, etc... it's a "unix made out of a network of
> > >> computers" not a "network made out of unix computers"
> > >>
> > >> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> > >>>
> > >>> always better to be a willow that bends than the strongest oak in a storm :)
> > >>>
> > >>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
> > >>>
> > >>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
> > >>>>
> > >>>> How do you share memory over rcpu?
> 

Re: [9front] Thoughts on Wayland?

[David Leimbach]

> 
>> 
>> While it's theoretically possible to do mmap over 9p, it's not going to
>> be a good experience.

Is it substantially worse than mmap’ing over NFS? I can imagine a lot of panic or deadlock cases but…. Yeah that’s true for NFS also. At least a core dump.

>> 
>> Quoth Willow Liquorice <willow@howhill.com>:
>>> I presume the Wayland spec is talking about mmap(2) under Linux, which
>>> allows you to treat a section of a file as you would program memory.
>>> 
>>> On 04/08/2024 21:04, Eli Cohen wrote:
>>>> also as far as mmap goes and what it even means, I have no idea, but a
>>>> stupid younger version of me, who was myself at the time, managed to
>>>> change this audio driver
>>>> http://git.9front.org/plan9front/plan9front/HEAD/sys/src/9/pc/audioac97.c/raw
>>>> to support some mmap'ed I/O ac97 chips. the part where it says ismmio
>>>> = 1, that's a flag that those chips are mmap'ed ac97 chips. but now
>>>> I'm older and still honestly have no idea what you're talking about
>>>> 
>>>> On Sun, Aug 4, 2024 at 12:59 PM Eli Cohen <echoline@gmail.com> wrote:
>>>>> 
>>>>> actually, drawing fast locally (and GPU stuff especially for
>>>>> computation) is definitely a plan 9 concept too and is also being
>>>>> worked on separately from wayland, and possibly on linux... so... plan
>>>>> 9 is an OS for networks that may include plan 9 machines, linux
>>>>> machines, iPhones, etc... it's a "unix made out of a network of
>>>>> computers" not a "network made out of unix computers"
>>>>> 
>>>>> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
>>>>>> 
>>>>>> always better to be a willow that bends than the strongest oak in a storm :)
>>>>>> 
>>>>>> there's a common misconception that plan 9 is an OS for a computer. plan 9 is an OS for a network. drawing fast locally is not a plan 9 concept. plan 9 is a 9P multiplexer. a wayland shim for linux/mac/bsd/etc that renders to draw somewhere else on the network would be good and appreciated and a relevant plan 9 concept and people are working on it... but for a fast local wayland implementation, that's what linux is, and linux can be and is a part of plan 9 which is an OS for a network
>>>>>> 
>>>>>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> wrote:
>>>>>>> 
>>>>>>> How do you share memory over rcpu?
>> 
> 

Re: [9front] Thoughts on Wayland?

[ori]

Quoth David Leimbach <leimy2k@icloud.com>:
> 
> > 
> >> 
> >> While it's theoretically possible to do mmap over 9p, it's not going to
> >> be a good experience.
> 
> Is it substantially worse than mmap’ing over NFS? I can imagine a lot of panic or deadlock cases but…. Yeah that’s true for NFS also. At least a core dump.
> 

I'm not familiar enough with the internals of the NFS
protocol to say for sure.

However, dont' just consider deadlock, consider coherence.
What happens if you put a mutex into shm, and two machines
try to lock it at the same time?

How do you avoid turning every single memory access into
a network round trip without some sort of sideband protocol?

What happens if a laptop participating in that coherence
mechanism gets its lid closed and hibernates for an hour or
three, and then comes back?

Re: [9front] Thoughts on Wayland?

[Dave MacFarlane]

On August 5, 2024 4:26:10 a.m. GMT+09:00, Willow Liquorice <willow@howhill.com> wrote:
>Hello again 9front,
>
>(Warning: long email is long)
>
>I've been leafing through the Wayland protocol, comparing it to rio and /dev/draw. To me, a system that could speak Wayland sounds much more compatible with the wider software ecosystem.
>
>Has there been much thought put into Wayland in 9front? I'd poke around in the list archives, but I haven't figured out how to 9P on my Linux machine yet.
>
>Here are my thoughts:
>
>1. The Wayland compositor and clients talk to each other over a Unix domain socket. I'm guessing Plan 9 doesn't have those (and doesn't want them) as there's nothing in sys/man on the subject. Could pipe(3) do the trick?
>
>2. Shared memory (where wl_buffers are stored) is achieved by passing a file descriptor from the client to the compositor, which the compositor mmaps as needed.
>
>If I understand correctly, that already sounds pretty plan9-y, though I'm not sure whether you could then treat the file *like* shared memory under 9front. Again, couldn't find any references to "mmap" in the manuals, so I guess there's no direct analogue. Could you achieve the same (or similar) effect with a ramfs?
>
>3. protocol/wayland.xml is apparently used to generate all of the function prototypes you need to speak Wayland. If the underlying components (shared memory and a domain socket lookalike) work, making a Wayland library for Plan 9 (Way 9?) *might* boil down to a paint-by-numbers exercise.
>
>Wayland's been around a while, so maybe none of these observations are new. I don't know! I'm a novice.
>
>Regards,
>	Willow
>

What is your goal?

If you want to learn about /dev/draw and Wayland trying to port Wayland for yourself is an ambitious goal but you would learn a lot.

If you want to improve the architecture replacing a Plan 9 solution for graphics with a Linux one is of dubious value.

If you want to make it easier to port software you're trying to run before you walk. Anything that uses Wayland almost certainly needs all kinds of other Linux-specific libraries or modern C extensions or LLVM or etc and you'd get more value working on APE* and starting with something more concrete (a specific, preferrably small, program you'd like to port.)

* I am not sure if 9front is really interested in a more complete APE, having just removed ape/diff.. I'd suggest reaching out to the Harvey people since this idea is more up their lane but I think Harvey is dead. 

- Dave

Re: [9front] Thoughts on Wayland?

[David Leimbach]

Sent from my iPhone

> On Aug 4, 2024, at 3:28 PM, ori@eigenstate.org wrote:
> 
> Quoth David Leimbach <leimy2k@icloud.com>:
>> 
>>> 
>>>> 
>>>> While it's theoretically possible to do mmap over 9p, it's not going to
>>>> be a good experience.
>> 
>> Is it substantially worse than mmap’ing over NFS? I can imagine a lot of panic or deadlock cases but…. Yeah that’s true for NFS also. At least a core dump.
>> 
> 
> I'm not familiar enough with the internals of the NFS
> protocol to say for sure.
> 
> However, dont' just consider deadlock, consider coherence.
> What happens if you put a mutex into shm, and two machines
> try to lock it at the same time?
> 
> How do you avoid turning every single memory access into
> a network round trip without some sort of sideband protocol?
> 
> What happens if a laptop participating in that coherence
> mechanism gets its lid closed and hibernates for an hour or
> three, and then comes back?


It’s a nasty game of timeouts and hangups… like everything in distributed systems. 

Re: [9front] Thoughts on Wayland?

[ori]

Quoth David Leimbach <leimy2k@icloud.com>:
> 
> Sent from my iPhone
> 
> > On Aug 4, 2024, at 3:28 PM, ori@eigenstate.org wrote:
> > 
> > Quoth David Leimbach <leimy2k@icloud.com>:
> >> 
> >>> 
> >>>> 
> >>>> While it's theoretically possible to do mmap over 9p, it's not going to
> >>>> be a good experience.
> >> 
> >> Is it substantially worse than mmap’ing over NFS? I can imagine a lot of panic or deadlock cases but…. Yeah that’s true for NFS also. At least a core dump.
> >> 
> > 
> > I'm not familiar enough with the internals of the NFS
> > protocol to say for sure.
> > 
> > However, dont' just consider deadlock, consider coherence.
> > What happens if you put a mutex into shm, and two machines
> > try to lock it at the same time?
> > 
> > How do you avoid turning every single memory access into
> > a network round trip without some sort of sideband protocol?
> > 
> > What happens if a laptop participating in that coherence
> > mechanism gets its lid closed and hibernates for an hour or
> > three, and then comes back?
> 
> 
> It’s a nasty game of timeouts and hangups… like everything in distributed systems. 
> 

ah, except in a distributed system, locks are provably broken;
they can't even work in theory.

Imagine two cases:
1) a machine takes a lock, pauses for an arbitrary amount
   of time, and then tries to proceed with its critical
   section, assuming that nobody has mutated its state

2) a macine takes a lock, and then dies permanently, never
   to return.

you can do one of two things here: you can either time out
the lock, or hold it forever.

in case 1, if you hold it forever, the system is correct.
If you time it out, you've just done critical work outside
a critical section and corrupted data.

in case 2, if you hold it forever, you have a deadlock that
can't be recovered from; the machine is never coming back.

you need a correct distributed system to obey wait-free like
properties.

sofware needs to be designed from the ground up to be aware
of the complexity of distributed systems; if you try to take
a system that was designed to run on a single failure domain
and make it distributed by layering over something like mmap,
either things get very complex, or they get very broken; that
is not an exclusive or.

Re: [9front] Thoughts on Wayland?

[Noam Preil]

> However, Wayland is clearly a worse fit for Plan 9 than devdraw;
> it's designed for the case where not only does the GPU live on the
> same machine as the compositor, it's designed for the case where
> every single piece of software doing drawing lives on the same
> machine as well.

In that respect, it's strictly a worse fit for Plan 9 than X, which at
least is designed with networking in mind, I suspect?

Though, there is a tool for networking transparency on Linux with
Wayland, so it's not impossible.

Neither sounds like a fun time to work on, though both could be used for
nicer vmx integration at the least...

Re: [9front] Thoughts on Wayland?

[Noam Preil]

Not speaking for anyone but myself, but as a 9fronter, broader
compatibility support within Sigrid's NPE (a replacement for APE, to
oversimplify) would definitely be appreciated :)

- Noam Preil

Re: [9front] Thoughts on Wayland?

[hiro]

> In that respect, it's strictly a worse fit for Plan 9 than X, which at
> least is designed with networking in mind, I suspect?

Both Plan 9 and X are designed with networking in mind.

> Though, there is a tool for networking transparency on Linux with
> Wayland, so it's not impossible.

I bet they are multiplexing all graphical output as a lossy video stream.
Careful design from the ground up on the remote protocol level is
needed to solve the problem in a lossless, bandwidth-efficient, and
roundtrip-reducing way. And then all applications using the protocol
have to use it in the right way and not try to sabotage all the
efforts that got put into this mechanism: modern x11 programs suffer
from the same problem that wayland does, kde/gnome/google/mozilla et
all started to use x11 inefficiently, introducing bad
round-trip-sensitive implementations for unimportant cosmetic features
like animations.

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

I gave this some more thought, and "just shim Cairo" has a lot of 
potential footguns, and it comes down to memory (again). What if you 
tried using one of the functions in memory(2) – common to both OSs – on 
a piece of memory that is touched by Cairo (and is, therefore, remote).

On 04/08/2024 22:10, Willow Liquorice wrote:
> It sounds like the "shared memory" concept Wayland uses is the tricky 
> bit, then, because you can't put the same RAM stick in two different 
> machines.
> 
> Though, if I'm understanding pipe(3) correctly, it reads like the right 
> tool to talk to the compositor.
> 
> Thinking about it, I guess the compositor and the in-use graphics 
> resources (hardware, software, data) ought to be resident on the same 
> physical system: a "GPU server", by analogy with a CPU server, though 
> "composition server" is probably a more accurate term.
> 
> Thinking about it even more, if *all* graphics resources, in use or not, 
> are resident on the GPU server, you don't have to shove huge fonts or 
> textures over the network, just the rendered images (or diffs!) and some 
> RPCs to a process resident on the GPU server to get it drawing what you 
> want.
> 
> How on earth that makes a program that uses Pango+Cairo, or something 
> like that, play nicely on 9 I have no idea… more shims for client programs?
> 
> I suppose every terminal is, in and of itself, a GPU server. Maybe I'm 
> overthinking it.
> 
> On 04/08/2024 20:59, Eli Cohen wrote:
>> actually, drawing fast locally (and GPU stuff especially for
>> computation) is definitely a plan 9 concept too and is also being
>> worked on separately from wayland, and possibly on linux... so... plan
>> 9 is an OS for networks that may include plan 9 machines, linux
>> machines, iPhones, etc... it's a "unix made out of a network of
>> computers" not a "network made out of unix computers"
>>
>> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
>>>
>>> always better to be a willow that bends than the strongest oak in a 
>>> storm :)
>>>
>>> there's a common misconception that plan 9 is an OS for a computer. 
>>> plan 9 is an OS for a network. drawing fast locally is not a plan 9 
>>> concept. plan 9 is a 9P multiplexer. a wayland shim for 
>>> linux/mac/bsd/etc that renders to draw somewhere else on the network 
>>> would be good and appreciated and a relevant plan 9 concept and 
>>> people are working on it... but for a fast local wayland 
>>> implementation, that's what linux is, and linux can be and is a part 
>>> of plan 9 which is an OS for a network
>>>
>>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> 
>>> wrote:
>>>>
>>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[ori]

shared memory just isn't a very good model for distributed systems.

devdraw is more or less what you get if you make a slimmed down cairo
the native drawing model, and talk to it over a pipe for rpcs, but your
musings are gradually reinventing devdraw :)

Quoth Willow Liquorice <willow@howhill.com>:
> I gave this some more thought, and "just shim Cairo" has a lot of 
> potential footguns, and it comes down to memory (again). What if you 
> tried using one of the functions in memory(2) – common to both OSs – on 
> a piece of memory that is touched by Cairo (and is, therefore, remote).

cairo supports many backends; a devdraw backend would not be that
difficult; shared memory is not necessary.

A good one would be harder, but even so it would not be that bad;
devdraw's model is still a reasonable fit for cairo -- there are
several missing features, but the model matches. Specifically,
off the top of my head, cairo would need:

	* scaling matrices
	* antialiasing
	* gradients

as devdraw rpcs, and then cairo could do all of the heavy lifting
in devdraw.
 
> On 04/08/2024 22:10, Willow Liquorice wrote:
> > It sounds like the "shared memory" concept Wayland uses is the tricky 
> > bit, then, because you can't put the same RAM stick in two different 
> > machines.
> > 
> > Though, if I'm understanding pipe(3) correctly, it reads like the right 
> > tool to talk to the compositor.
> > 
> > Thinking about it, I guess the compositor and the in-use graphics 
> > resources (hardware, software, data) ought to be resident on the same 
> > physical system: a "GPU server", by analogy with a CPU server, though 
> > "composition server" is probably a more accurate term.
> > 
> > Thinking about it even more, if *all* graphics resources, in use or not, 
> > are resident on the GPU server, you don't have to shove huge fonts or 
> > textures over the network, just the rendered images (or diffs!) and some 
> > RPCs to a process resident on the GPU server to get it drawing what you 
> > want.
> > 
> > How on earth that makes a program that uses Pango+Cairo, or something 
> > like that, play nicely on 9 I have no idea… more shims for client programs?
> > 
> > I suppose every terminal is, in and of itself, a GPU server. Maybe I'm 
> > overthinking it.
> > 
> > On 04/08/2024 20:59, Eli Cohen wrote:
> >> actually, drawing fast locally (and GPU stuff especially for
> >> computation) is definitely a plan 9 concept too and is also being
> >> worked on separately from wayland, and possibly on linux... so... plan
> >> 9 is an OS for networks that may include plan 9 machines, linux
> >> machines, iPhones, etc... it's a "unix made out of a network of
> >> computers" not a "network made out of unix computers"
> >>
> >> On Sun, Aug 4, 2024 at 12:54 PM Eli Cohen <echoline@gmail.com> wrote:
> >>>
> >>> always better to be a willow that bends than the strongest oak in a 
> >>> storm :)
> >>>
> >>> there's a common misconception that plan 9 is an OS for a computer. 
> >>> plan 9 is an OS for a network. drawing fast locally is not a plan 9 
> >>> concept. plan 9 is a 9P multiplexer. a wayland shim for 
> >>> linux/mac/bsd/etc that renders to draw somewhere else on the network 
> >>> would be good and appreciated and a relevant plan 9 concept and 
> >>> people are working on it... but for a fast local wayland 
> >>> implementation, that's what linux is, and linux can be and is a part 
> >>> of plan 9 which is an OS for a network
> >>>
> >>> On Sun, Aug 4, 2024, 12:46 PM Stuart Morrow <morrow.stuart@gmail.com> 
> >>> wrote:
> >>>>
> >>>> How do you share memory over rcpu?

Re: [9front] Thoughts on Wayland?

[sirjofri]

05.08.2024 10:34:45 ori@eigenstate.org:
> shared memory just isn't a very good model for distributed systems.
> devdraw is more or less what you get if you make a slimmed down cairo
> the native drawing model, and talk to it over a pipe for rpcs, but your
> musings are gradually reinventing devdraw :)

That's what I like about GPUs: they have dedicated memory and you have to upload manually. No synchronization issues, just what you upload is there. It's basically a different machine.

Also, the design is extremely similar to devdraw, just that you can upload programs and arbitrary data. Imagine all your graphics routines are executed on the devdraw server side, and you just kick off those programs from the client. That would mean fluent animations with many frames, and the cpu that runs the client application can just relax.

With complex applications with hungry code and hungry graphics (many primitive draws) it would be a win-win: the hungry code gets more time on the client, because the client doesn't have to draw primitives, and you still get fancy graphics, because the hungry primitive drawing routines happens on the devdraw server side.

My current plan for gpufs needs another round trip yet, because it doesn't integrate with devdraw at all. You'd have to kick off the drawing in gpufs, you have to download the resulting image, and load that into devdraw. Over network, it means a loss of performance, but I'm thinking about some kind of coupling, so that some devdraw-images are automatically linked with gpufs image buffers. At least on integrated systems like drawterm that should be possible. Still not perfect, but better than network round-trips.

Anyways, early research.

sirjofri

Re: [9front] Thoughts on Wayland?

[Noam Preil]

Of course, for e.g. treason, it's trickier, since frame buffers are
decoded on the CPU side, and the entire frame buffer gets uploaded to
devdraw.

Best case, with gpufs, would be tossing the encoded data instead -
which _would_ be a major bandwidth reduction. That might actually be
sufficient for treason over 9p?

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

I read about gpufs in the latest IWP9 proceedings (once I found it, the 
Foundation didn't put it in their news section). Exciting stuff!

On 05/08/2024 09:52, sirjofri wrote:
> 05.08.2024 10:34:45 ori@eigenstate.org:
>> shared memory just isn't a very good model for distributed systems.
>> devdraw is more or less what you get if you make a slimmed down cairo
>> the native drawing model, and talk to it over a pipe for rpcs, but your
>> musings are gradually reinventing devdraw :)
> 
> That's what I like about GPUs: they have dedicated memory and you have to upload manually. No synchronization issues, just what you upload is there. It's basically a different machine.
> 
> Also, the design is extremely similar to devdraw, just that you can upload programs and arbitrary data. Imagine all your graphics routines are executed on the devdraw server side, and you just kick off those programs from the client. That would mean fluent animations with many frames, and the cpu that runs the client application can just relax.
> 
> With complex applications with hungry code and hungry graphics (many primitive draws) it would be a win-win: the hungry code gets more time on the client, because the client doesn't have to draw primitives, and you still get fancy graphics, because the hungry primitive drawing routines happens on the devdraw server side.
> 
> My current plan for gpufs needs another round trip yet, because it doesn't integrate with devdraw at all. You'd have to kick off the drawing in gpufs, you have to download the resulting image, and load that into devdraw. Over network, it means a loss of performance, but I'm thinking about some kind of coupling, so that some devdraw-images are automatically linked with gpufs image buffers. At least on integrated systems like drawterm that should be possible. Still not perfect, but better than network round-trips.
> 
> Anyways, early research.
> 
> sirjofri

Re: [9front] Thoughts on Wayland?

[sirjofri]

05.08.2024 10:58:57 Noam Preil <noam@pixelhero.dev>:
> Of course, for e.g. treason, it's trickier, since frame buffers are
> decoded on the CPU side, and the entire frame buffer gets uploaded to
> devdraw.
>
> Best case, with gpufs, would be tossing the encoded data instead -
> which _would_ be a major bandwidth reduction. That might actually be
> sufficient for treason over 9p?

It could even be better: you can probably stream the data of a few frames. It could decode asynchronously what's there, which is likely more than just that one frame. You'd end up with a few frames before you need them, and you only have to clock the frames to get the exact frame you need (e.g. from a swapchain). Older frames can just safely be discarded.

Even better would be, streaming the data and decode it using the builtin GPU "hardware" decoders. That way you could also decode DRM-protected stuff, but I think those are also bound to direct display from the GPU on a monitor that supports DRM/HDCP/whatever. DRM stuff is horrible...

That feature is also currently not planned, I want the GPU as just a giant array of compute shaders with its own fast memory (GPGPU). Theoretically, you can call it gpgpufs, but since it's likely ending up in /dev as a driver and not as a client program, it's probably more like devgpu. Gpufs could be a client program for cpu emulation to make programs work without GPUs.

It might even make more sense to allow access to the GPU at an even lower level (basically the individual API calls) and have filesystems for the higher level stuff (e.g vulkanfs), but since that would mean more 9p messages that would probably decrease performance even more.

sirjofri

Re: [9front] Thoughts on Wayland?

[Shawn Rutledge]

On Mon, Aug 05, 2024 at 10:52:58AM +0200, sirjofri wrote:
> 05.08.2024 10:34:45 ori@eigenstate.org:
> > shared memory just isn't a very good model for distributed systems.
> > devdraw is more or less what you get if you make a slimmed down cairo
> > the native drawing model, and talk to it over a pipe for rpcs, but your
> > musings are gradually reinventing devdraw :)
> 
> That's what I like about GPUs: they have dedicated memory and you have
> to upload manually. No synchronization issues, just what you upload is
> there. It's basically a different machine.
>
> Also, the design is extremely similar to devdraw, just that you can
> upload programs and arbitrary data. Imagine all your graphics routines
> are executed on the devdraw server side, and you just kick off those
> programs from the client. That would mean fluent animations with many
> frames, and the cpu that runs the client application can just relax.

Dedicated GPUs are like that, so portable APIs need to support working
that way. But often on integrated graphics, GPU memory is just a chunk
of system memory, which makes the "upload" trivial in practice. Perhaps
it can even be zero-copy sometimes, if there's a way to just map a chunk
into GPU-managed space after it's populated?

It's very useful for AI applications when there's a lot of system memory
and the boundary for "dedicated" GPU memory is fully flexible, as on
Apple's unified memory. (For example I found that I can run a fully
offline LLM with ollama on a modern mac; and after that, I stay away
from all the online LLMs... I don't trust them, they are ultimately
greedy, and if something is "free" it must be that we're helping them
train it better by using their throttled interfaces.) So I'm not sure,
but maybe we can expect that kind of architecture to be more common in
the future. 

And it would be nice to have a way to avoid spending system memory at
all for GPU resources, to be able to stream directly from the original
source (e.g. a file) to GPU memory. This is an issue in Qt (my day
job)... we have caching features in multiple places, we try to avoid
reading files and remote resources more than once; but if you're playing
an animation, and the desired end result is to have the frames as GPU
textures, and you have enough GPU memory, then it should be ok to lose
the CPU-side cache. Especially if the cache was not already in
GPU-texture form. Decoding simple formats like png and gif is fast
enough that it doesn't even matter if you need to do it multiple times:
not worth caching frames from them, IMO, unless the cache is on the GPU
side. But I think the tradeoffs are different for different sizes. In
some cases, an animated gif can be very compact and yet a full set of
decoded frames can be enormous, so it doesn't make sense to cache it
anywhere. Decoding one frame at a time is the cheapest. Even if you
had to keep re-reading the file, doesn't the OS cache the file contents
in RAM anyway? (A controversial position, I'm sure.) So how and whether
to decide at runtime what to cache how and where, or leave it up to the
application developer by exposing all the suitable APIs, is Qt's
problem... sorry for the digression, my point is just that
upload-and-forget is not the only way that a GPU needs to be used.
Likewise large games are often streaming assets and geometry to the GPU
more or less continuously, from what I've heard: depends which assets
they can reasonably expect to be reused and to have enough GPU memory to
retain them, I suppose?

It's also my daydream to get the GPU to take care of UI rendering more
completely, even for simple 2D stuff, and free up the CPU. It's one
thing I'm hoping to achieve with my 9p scenegraph project (which is
a slow-moving side project, not a Qt project). But in general, there
might also be a backlash against excessive GPU usage coming, if people
expect to use the GPU mainly for "hard problems" or embarrassingly-
parallel algorithms like AI and 3D graphics, and not load it down with
simple stuff that the CPU can just as well do for itself. And battery
consumption might be a concern sometimes too. My attitude towards old
CPU-based paint engines like draw and QPainter has been kindof negative
since I started at Qt, because we've been trying to sell the idea that
you have a GPU, so you might as well use it to get nice AA on all your
graphics, animations "for free", alpha blending, and stuff like that. I
still think AA is really a killer feature though. Just about makes
2D-on-the-gpu worthwhile all on its own. But Plan 9's draw could not
have AA on everything, could it?

So while _I'm_ still interested in 2D on the GPU, I admit that you might
be onto something with your gpufs proposal, to focus on treating it more
like a computing resource than a fundamental piece of the graphics
pipeline. But I think we should have both options.  ;-)

> With complex applications with hungry code and hungry graphics (many primitive draws)

Many draw calls are the enemy of performance when it comes to GPU
graphics. This is the main reason for needing a scene graph: first get
the whole collection of everything that needs to go to the screen
together into one pile, in an efficient form that is quick to traverse,
and then traverse it and figure out how to combine the draw calls.
(There is an impedance mismatch between any turtle-graphics or
paint-engine API, and the GPU. You can solve it if you can intercept the
draw calls and have them only populate the scene graph instead. Never do
any drawing immediately on-demand. Or, give up on the imperative API
altogether: be declarative.) This is the thing that Qt Quick is good at.
The interesting question for me now is how best to map that idea to a
filesystem.

Are you trying to have gpufs as an endpoint for uploading assets that
could be invoked via draw calls too? Or just as a way to invoke the GPU
to do embarrassingly-parallel computation at whatever stage one wants to
use it for? (Why not make it possible to use it both ways, eventually?)
But I would expect that this fs will have a close-to-the-hardware
design, right?

Re: [9front] Thoughts on Wayland?

[David Arnold]

> On 5 Aug 2024, at 18:54, sirjofri <sirjofri+ml-9front@sirjofri.de> wrote:


> Also, the design is extremely similar to devdraw, just that you can upload programs and arbitrary data. Imagine all your graphics routines are executed on the devdraw server side, and you just kick off those programs from the client. That would mean fluent animations with many frames, and the cpu that runs the client application can just relax.

So, like DisplayPostscript or NeWS?




d

Re: [9front] Thoughts on Wayland?

[hiro]

> That's what I like about GPUs: they have dedicated memory and you have to upload manually. No synchronization issues, just what you upload is there. It's basically a different machine.

whether you call it upload or synchronization makes no difference in practice.
devdraw also has dedicated memory and you upload images into it
manually. it doesn't only behave as if it was a different machine, but
it is inspired by the difficulties that arose from having to run it on
a dedicated terminal machine.
from the point of view of devdraw and the whole development toolchain
a gpu is a step back, bec. gpus aren't real computers, so the
compilers stop working (i.e. the hardware target differs, just like
the original blit). unlike with other cpu archs it's probably not
worth getting our compilers ported into shaderland.

> With complex applications with hungry code and hungry graphics (many primitive draws) it would be a win-win: the hungry code gets more time on the client, because the client doesn't have to draw primitives, and you still get fancy graphics, because the hungry primitive drawing routines happens on the devdraw server side.

i'm not sure why you care so much. mostly devdraw is a bandwidth
optimization (preloaded textures).
the primitives we have are more than enough to display 2D content
efficiently. what usecase are you unable to implement with the given
technique?

> My current plan for gpufs needs another round trip yet, because it doesn't integrate with devdraw at all. You'd have to kick off the drawing in gpufs, you have to download the resulting image, and load that into devdraw. Over network, it means a loss of performance, but I'm thinking about some kind of coupling, so that some devdraw-images are automatically linked with gpufs image buffers. At least on integrated systems like drawterm that should be possible. Still not perfect, but better than network round-trips.

what are the ratios here? current perceived number of roundtrips for
example graphical interaction, vs. theoretical minimum number of
roundtrips  vs. your optimized -1 roundtrip?

Re: [9front] Thoughts on Wayland?

[Shawn Rutledge]

On Mon, Aug 05, 2024 at 10:02:10AM +0200, hiro wrote:
> > In that respect, it's strictly a worse fit for Plan 9 than X, which at
> > least is designed with networking in mind, I suspect?
> 
> Both Plan 9 and X are designed with networking in mind.
> 
> > Though, there is a tool for networking transparency on Linux with
> > Wayland, so it's not impossible.
> 
> I bet they are multiplexing all graphical output as a lossy video stream.
> Careful design from the ground up on the remote protocol level is
> needed to solve the problem in a lossless, bandwidth-efficient, and
> roundtrip-reducing way. And then all applications using the protocol
> have to use it in the right way and not try to sabotage all the
> efforts that got put into this mechanism: modern x11 programs suffer
> from the same problem that wayland does, kde/gnome/google/mozilla et
> all started to use x11 inefficiently, introducing bad
> round-trip-sensitive implementations for unimportant cosmetic features
> like animations.

Yes that's an old problem. One place where I worked over 20 years ago,
the boss had set up dual networks: one for internal stuff and one for the
Internet. He had a pile of surplus NCD X-terminals, which he made us use
for Internet stuff, running the actual programs on a shared Sun machine: so
fully remote over 100mbit ethernet. (Of course he was also routinely spying
on us to make sure we didn't waste too much time web-surfing or
socializing.) It quickly became clear which programs were efficient that
way and which weren't. Qt and GTK were both already trying to render the
whole UI into one window, doing widget compositing themselves; I think this
explanation here is from that time period (and offers workarounds):
https://doc.qt.io/qt-6/qwidget.html#native-widgets-vs-alien-widgets
That just made them feel much slower under such conditions, because often
the buffer for the whole window had to be resent. (VNC works better than
that, because of tiling.) But the tradeoff is if each widget generates
resources on the X11 server side (which in this case had a rather small
amount of memory, a few megabytes), there is a limit to how complex the UI
can be (so you are stuck either way; even using too many colors was
expensive on those terminals). And the window-per-widget architecture is
slower in the more common case where the X server is on the same machine
where the application runs, shared memory is possible, and you might even
have a GPU.  Don't trust the X server to be the quickest way to render your
2D stuff: do it on your own if you want to be sure that it's as quick as it
needs to be. Plus, AA really benefits everyone, and
https://en.wikipedia.org/wiki/X_Rendering_Extension is only for fonts, not
for arbitrary graphics. (Same problem as on Plan 9. Should we blame Keith
Packard for not going further than just font rendering?) There's no way to
have AA in arbitrary 2D graphics rendered on the X server, AFAIK. So
basically the widget toolkits gave up on supporting efficient networked X11
ages ago, and not for completely frivolous reasons. There were only a few
users, maybe not vocal enough. Not sure what's the state-of-the-art toolkit
that is actually good at it, though. TK was ok, as I remember.

(The work itself at that company was interesting: on the upside, where else
could I have been paid to write Scheme full-time for a year? I still use
Scheme now because of what I learned there. And I was working on remote-UI
stuff there too. But got fed up with the low pay and control-freak
attitude, and left as soon as I got a better offer.)

Anyway the toolkits got away with that on X11, which maybe contributed to
the assumption that shared memory is fine for Wayland as well, and
remote-desktop use cases are a niche case that can be served with things
like VNC, RDP, and video streaming. The KDE/Plasma project is pursuing all
of these, by the way. There's pipewire.org which I got the impression was
for audio first, and then got video shoehorned in as well. Accordingly
there's https://invent.kde.org/plasma/kpipewire ...not that I've tried it.
And https://conf.kde.org/event/5/contributions/133/ ... that one surprised
me, because I thought RDP is a complex Microsoft beast. Seems like a lot of
work to support that in open source.

But if video streaming is the solution, it seems to me an assumption is
being made that the server (where the applications run) has a GPU then: it
would help for rendering the applications (with AA, blending, animations
and all), and it would help for video compression too. Somewhat of a break
from tradition then. I agree that a solution designed specifically with low
bandwidth and low latency in mind ought to be better. But is 9p really for
that? I'm trying to work that way; time will tell...

Re: [9front] Thoughts on Wayland?

[hiro]

it makes zero sense to decode video remotely and then send it in raw
video over the network link over devdraw rpcs.
you'd have to integrate image and video compression into devdraw. and
enable such a new video-streaming feature. maybe call it /dev/video
we already have compression in theory, but it's lz77 and not very
efficient in comparison with lossy image and video codecs

On Mon, Aug 5, 2024 at 10:58 AM Noam Preil <noam@pixelhero.dev> wrote:
>
> Of course, for e.g. treason, it's trickier, since frame buffers are
> decoded on the CPU side, and the entire frame buffer gets uploaded to
> devdraw.
>
> Best case, with gpufs, would be tossing the encoded data instead -
> which _would_ be a major bandwidth reduction. That might actually be
> sufficient for treason over 9p?
>

Re: [9front] Thoughts on Wayland?

[hiro]

> Many draw calls are the enemy of performance when it comes to GPU
> graphics. This is the main reason for needing a scene graph: first get
> the whole collection of everything that needs to go to the screen
> together into one pile, in an efficient form that is quick to traverse,
> and then traverse it and figure out how to combine the draw calls.

yes, that's already how devdraw works, you can flush whenever you
want, draw RPCs get batched, if you want.

Re: [9front] Thoughts on Wayland?

[sirjofri]

05.08.2024 13:09:41 Shawn Rutledge <lists@ecloud.org>:
> Dedicated GPUs are like that, so portable APIs need to support working
> that way. But often on integrated graphics, GPU memory is just a chunk
> of system memory, which makes the "upload" trivial in practice. Perhaps
> it can even be zero-copy sometimes, if there's a way to just map a chunk
> into GPU-managed space after it's populated?

I don't know so much about the integrated graphics, but in the end they plaster the whole beast with an API middleware like OpenGL or DirectX, which takes care of anything that's happening underneath. I assume that these also handle copying or mapping with integrated graphics. With dedicated GPUs, they surely upload the data, either directly (blocking) or nonblocking (save until the GPU is ready, then upload).

> It's very useful for AI applications when there's a lot of system memory
> and the boundary for "dedicated" GPU memory is fully flexible, as on
> Apple's unified memory. (For example I found that I can run a fully
> offline LLM with ollama on a modern mac; and after that, I stay away
> from all the online LLMs... I don't trust them, they are ultimately
> greedy, and if something is "free" it must be that we're helping them
> train it better by using their throttled interfaces.) So I'm not sure,
> but maybe we can expect that kind of architecture to be more common in
> the future.

Probably, who knows. The GPU is becoming a core part of any system, and AMD is working on having dedicated GPU power in the same package as the CPU. Modern APUs (as in handheld gaming consoles) are incredibly powerful enough (still not comparable to true dedicated graphics, but you can run modern games on them), and ARM also follows the market of integrating GPUs (Mali and stuff). I assume that with the rise of AI in standard computer systems, the market is forced to having dedicated GPU power, and unified memory (apple) sounds like an interesting way to go forward.

> And it would be nice to have a way to avoid spending system memory at
> all for GPU resources, to be able to stream directly from the original
> source (e.g. a file) to GPU memory. This is an issue in Qt (my day
> job)... we have caching features in multiple places, we try to avoid
> reading files and remote resources more than once; but if you're playing
> an animation, and the desired end result is to have the frames as GPU
> textures, and you have enough GPU memory, then it should be ok to lose
> the CPU-side cache. Especially if the cache was not already in
> GPU-texture form. Decoding simple formats like png and gif is fast
> enough that it doesn't even matter if you need to do it multiple times:
> not worth caching frames from them, IMO, unless the cache is on the GPU
> side. But I think the tradeoffs are different for different sizes. In
> some cases, an animated gif can be very compact and yet a full set of
> decoded frames can be enormous, so it doesn't make sense to cache it
> anywhere. Decoding one frame at a time is the cheapest. Even if you
> had to keep re-reading the file, doesn't the OS cache the file contents
> in RAM anyway? (A controversial position, I'm sure.) So how and whether
> to decide at runtime what to cache how and where, or leave it up to the
> application developer by exposing all the suitable APIs, is Qt's
> problem... sorry for the digression, my point is just that
> upload-and-forget is not the only way that a GPU needs to be used.
> Likewise large games are often streaming assets and geometry to the GPU
> more or less continuously, from what I've heard: depends which assets
> they can reasonably expect to be reused and to have enough GPU memory to
> retain them, I suppose?

Streaming textures (and other data) to the GPU is indeed a complex topic. Large games usually calculate what's needed, then load the data that's not already in the GPU and upload it. They make a big difference between streamed data (e.g world textures) and non-streamed data (e.g UI textures).

Microsoft is also working on DirectStorage, which makes decoding/unpacking on the CPU obsolete by just uploading the data to the GPU and unpacking it there. I think they are also working on using other bus systems to transfer the data, but as far as I know, as the hardware is currently built, it's not that easily possible and you always need the CPU (and system memory) for streaming. I don't know the exact details though...

Modern game engines (I'm biased towards Unreal) really perfected streaming, also considering the incredible speeds of SSDs. Nanite, for example, describes a hierarchy of clusters of polygons, each cluster with its own bounds. The cluster data is uploaded to the GPU, and the GPU (shader) does some fancy culling (frustum, occlusion, and size). This data is then used to "tell" the CPU which polygons (clusters) to load, and then only this data is loaded. This data is also stored on the GPU for as long as it's needed, the data is reused each frame. So per frame, only new clusters are streamed in, for example when moving around. In practice, each frame is slightly different, so you always have a few clusters here and there, but compared to full draw calls like in a classic pipeline, that's a huge difference!

> It's also my daydream to get the GPU to take care of UI rendering more
> completely, even for simple 2D stuff, and free up the CPU. It's one
> thing I'm hoping to achieve with my 9p scenegraph project (which is
> a slow-moving side project, not a Qt project). But in general, there
> might also be a backlash against excessive GPU usage coming, if people
> expect to use the GPU mainly for "hard problems" or embarrassingly-
> parallel algorithms like AI and 3D graphics, and not load it down with
> simple stuff that the CPU can just as well do for itself. And battery
> consumption might be a concern sometimes too. My attitude towards old
> CPU-based paint engines like draw and QPainter has been kindof negative
> since I started at Qt, because we've been trying to sell the idea that
> you have a GPU, so you might as well use it to get nice AA on all your
> graphics, animations "for free", alpha blending, and stuff like that. I
> still think AA is really a killer feature though. Just about makes
> 2D-on-the-gpu worthwhile all on its own. But Plan 9's draw could not
> have AA on everything, could it?

In fact, GPUs can still be used as 2d accelerators. In the end, it comes down to how you program it.

In the beginning, the GPU was more like devdraw, it could only draw 2d stuff based on simple draw calls. With shaders, you are free to do what you want. You can, in fact, upload some scene data structure, and your shader does interpolation, rasterization, etc. Sometimes it's even cheaper to do that in a shader than using dedicated hardware components.

> So while _I'm_ still interested in 2D on the GPU, I admit that you might
> be onto something with your gpufs proposal, to focus on treating it more
> like a computing resource than a fundamental piece of the graphics
> pipeline. But I think we should have both options.  ;-)

The trend goes towards indirect rendering for reasons. For example, the standard graphics pipeline is very strict about what it does and what it expects, and also how it works. But for your specific application, you maybe don't need all the components, and you maybe need some components to work differently.

This is also the example of Nanite: large triangles are faster to rasterize on the hardware, using the dedicated hardware rasterizer, but smaller triangles are faster to render with a custom rasterizer engine. It becomes even more complex when thinking about other use cases, like particle rendering and displacement.

Additionally, when using the standard pipeline, you have to run a lot of boilerplate code that you don't really need. For Nanite, they even plan to implement compute-based shading (running the pixel shader as a compute shader), and they expect a performance gain.

I expect that at some point we won't have the hardware for this specialized rendering pipeline anymore. It will still exist as a concept, but the API (OpenGL, DirectX, ...) will simulate that.

>> With complex applications with hungry code and hungry graphics (many primitive draws)
>
> Many draw calls are the enemy of performance when it comes to GPU
> graphics. This is the main reason for needing a scene graph: first get
> the whole collection of everything that needs to go to the screen
> together into one pile, in an efficient form that is quick to traverse,
> and then traverse it and figure out how to combine the draw calls.
> (There is an impedance mismatch between any turtle-graphics or
> paint-engine API, and the GPU. You can solve it if you can intercept the
> draw calls and have them only populate the scene graph instead. Never do
> any drawing immediately on-demand. Or, give up on the imperative API
> altogether: be declarative.) This is the thing that Qt Quick is good at.
> The interesting question for me now is how best to map that idea to a
> filesystem.

It's not uncommon nowadays to have a command list builder that accepts commands from many different cpu threads. Those command lists will then be uploaded in bundles, which reduces the number of draw calls.

> Are you trying to have gpufs as an endpoint for uploading assets that
> could be invoked via draw calls too? Or just as a way to invoke the GPU
> to do embarrassingly-parallel computation at whatever stage one wants to
> use it for? (Why not make it possible to use it both ways, eventually?)
> But I would expect that this fs will have a close-to-the-hardware
> design, right?

Currently, gpufs is designed after vulkan, which is much closer to the hardware than OpenGL. It focuses on interacting with the hardware, instead of drawing graphics.

I'd like to treat assets and graphics as "just data". Your application (including shaders) defines what the data is. It could be a 3d model, a texture, a frame buffer, an animation, it fully depends on the application.

That way, you can upload your assets and your programs, and in the end you get the final frame as data, which can be interpreted as an image, for example. You could however als upload assets and your programs and get the final data, which is an animation. As a game developer, I have to build graphics though...

sirjofri

Re: [9front] Thoughts on Wayland?

[sirjofri]

05.08.2024 13:48:41 hiro <23hiro@gmail.com>:
>> That's what I like about GPUs: they have dedicated memory and you have to upload manually. No synchronization issues, just what you upload is there. It's basically a different machine.
>
> whether you call it upload or synchronization makes no difference in practice.
> devdraw also has dedicated memory and you upload images into it
> manually. it doesn't only behave as if it was a different machine, but
> it is inspired by the difficulties that arose from having to run it on
> a dedicated terminal machine.
> from the point of view of devdraw and the whole development toolchain
> a gpu is a step back, bec. gpus aren't real computers, so the
> compilers stop working (i.e. the hardware target differs, just like
> the original blit). unlike with other cpu archs it's probably not
> worth getting our compilers ported into shaderland.

It surely isn't worth it, and I'd also not recommend it. It's a special case.

I'd more think about compiling the shaders "from source, on demand", which is common since the application cannot know which hardware the shader is run on.

Gpufs is based around vulkan, so you would compile your high level shader program to the SPIR-V intermediate language and just upload that to the filesystem. This cannot be run, but has to be compiled to the actual architecture of the GPU.

With that, in an ideal world, a host computer (cpu) runs and exports the gpufs and does the compilation of the shaders at this last stage, all bundled in the gpufs. My plan is to extend drawterm to do that, but it could also be just a single linux process, or a native plan 9 machine (if we ever get drivers).

>> With complex applications with hungry code and hungry graphics (many primitive draws) it would be a win-win: the hungry code gets more time on the client, because the client doesn't have to draw primitives, and you still get fancy graphics, because the hungry primitive drawing routines happens on the devdraw server side.
>
> i'm not sure why you care so much. mostly devdraw is a bandwidth
> optimization (preloaded textures).
> the primitives we have are more than enough to display 2D content
> efficiently. what usecase are you unable to implement with the given
> technique?

I'm a professional game developer working with Unreal. I could tell more, but I think that's enough.

>> My current plan for gpufs needs another round trip yet, because it doesn't integrate with devdraw at all. You'd have to kick off the drawing in gpufs, you have to download the resulting image, and load that into devdraw. Over network, it means a loss of performance, but I'm thinking about some kind of coupling, so that some devdraw-images are automatically linked with gpufs image buffers. At least on integrated systems like drawterm that should be possible. Still not perfect, but better than network round-trips.
>
> what are the ratios here? current perceived number of roundtrips for
> example graphical interaction, vs. theoretical minimum number of
> roundtrips  vs. your optimized -1 roundtrip?

Since this is early research, I don't have numbers. In the end, it can be all or nothing. As long as I'm unable to profile it, I can only speculate and say that there could be issues.

In a perfect world, the CPU sees an address and the GPU already knows the data, but that's magic, not engineering.

sirjofri

Re: [9front] Thoughts on Wayland?

[Willow Liquorice]

I see your point. Re: the extensions. I was poking around on 
wayland.app/protocols, but I didn't scroll down far enough to see the 
mountain of extensions labelled "wlr", "KDE", "Weston", and "External". 
Jesus Christ…

I must agree: *Plan 9 is no place for a Wayland compositor.*

Still, it might be a place for a remote Wayland client, Arch Wiki 
summarises the state of the art:

https://wiki.archlinux.org/title/Wayland#Remote_display

They all use different solutions of course. I suppose the Plan 9 
solution would be a file server on the compositor machine.

I find it funny that most of my interactions with this list have boiled 
down to "_ can't be that hard, right?", followed by "oh fuck, _ is that 
hard!" within the space of 24 hours.

I really should try and get 9front running on something instead of being 
a theoretical hacker. Have the Raspberry Pi images been tested on the Pi 
Zero?

	- Willow

P.S. I wasn't expecting this to blow up as much as it did. This has been 
a really informative thread, and the talk about GPUs on Plan 9 has been 
fascinating.

On 04/08/2024 22:31, ori@eigenstate.org wrote:
> Quoth Willow Liquorice <willow@howhill.com>:
>> Hello again 9front,
>>
>> (Warning: long email is long)
>>
>> I've been leafing through the Wayland protocol, comparing it to rio and
>> /dev/draw. To me, a system that could speak Wayland sounds much more
>> compatible with the wider software ecosystem.
>>
> 
> A system that fully implements the posix APIs would also be more
> compatible with the wider software ecosystem, however it would not
> be more pleasant to program for or interact with; The appeal of
> Plan 9 is that it's simplified things to the essentials, and
> doesn't pull in a bunch of overengineered junk.
> 
> So, as for Wayland specifically: The biggest problem with it is
> that it's not a protocol. It's got a core that's insufficient for
> actual use, and a set of mutually incompatible extensisons that
> forces each user to write programs that explicitly support wlroots,
> gnome, and KDE with separate hacks. To pick on an example, cursor
> warping in drawterm only works in some environments, because others
> fail to support those specific extensisons.
> 
> On top of that, the protocol's model of resources is rather
> overengineered and painful to use.
> 
> More or less, Wayland was misdesigned on different axes than X11;
> it's not clear which one is a worse approach.
> 
> However, Wayland is clearly a worse fit for Plan 9 than devdraw;
> it's designed for the case where not only does the GPU live on the
> same machine as the compositor, it's designed for the case where
> every single piece of software doing drawing lives on the same
> machine as well.
> 

[9front] Fortune worthy

[Steve simon]

 "_ can't be that hard, right?", followed by "oh fuck, _ is that hard!"
 Willow Liquorice, on Weyland plan9 integration

Re: [9front] Thoughts on Wayland?

[Jon Sharp]

On Mon, 5 Aug 2024, Willow Liquorice wrote:

> I really should try and get 9front running on something instead of being a 
> theoretical hacker. Have the Raspberry Pi images been tested on the Pi Zero?

I can confirm the latest release images work great on the zero. 
(performance of games/nes notwithstanding... ;) )

--
Jon

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

I still think it's sort of a place for a Wayland compositor... this
thread got long. has anyone mentioned what halfwit was working on? a
way log in to *nix machines and just run firefox or something the same
as you would do anything else on Plan 9

On Mon, Aug 5, 2024 at 1:11 PM Jon Sharp <jrsharp@sdf.org> wrote:
>
> On Mon, 5 Aug 2024, Willow Liquorice wrote:
>
> > I really should try and get 9front running on something instead of being a
> > theoretical hacker. Have the Raspberry Pi images been tested on the Pi Zero?
>
> I can confirm the latest release images work great on the zero.
> (performance of games/nes notwithstanding... ;) )
>
> --
> Jon

Re: [9front] Thoughts on Wayland?

[Eli Cohen]

well I mean... then I guess the Wayland compositor is still on the
*nix machine end. it's draw by the time it's going over the network

On Mon, Aug 5, 2024 at 5:07 PM Eli Cohen <echoline@gmail.com> wrote:
>
> I still think it's sort of a place for a Wayland compositor... this
> thread got long. has anyone mentioned what halfwit was working on? a
> way log in to *nix machines and just run firefox or something the same
> as you would do anything else on Plan 9
>
> On Mon, Aug 5, 2024 at 1:11 PM Jon Sharp <jrsharp@sdf.org> wrote:
> >
> > On Mon, 5 Aug 2024, Willow Liquorice wrote:
> >
> > > I really should try and get 9front running on something instead of being a
> > > theoretical hacker. Have the Raspberry Pi images been tested on the Pi Zero?
> >
> > I can confirm the latest release images work great on the zero.
> > (performance of games/nes notwithstanding... ;) )
> >
> > --
> > Jon

Re: [9front] Thoughts on Wayland?

[Michael Misch]

I (halfwit) have no real idea whether my stuff is even viable until it’s done, idk

> On Aug 5, 2024, at 17:09, Eli Cohen <echoline@gmail.com> wrote:
> 
> well I mean... then I guess the Wayland compositor is still on the
> *nix machine end. it's draw by the time it's going over the network
> 
> On Mon, Aug 5, 2024 at 5:07 PM Eli Cohen <echoline@gmail.com> wrote:
>> 
>> I still think it's sort of a place for a Wayland compositor... this
>> thread got long. has anyone mentioned what halfwit was working on? a
>> way log in to *nix machines and just run firefox or something the same
>> as you would do anything else on Plan 9
>> 
>> On Mon, Aug 5, 2024 at 1:11 PM Jon Sharp <jrsharp@sdf.org> wrote:
>>> 
>>> On Mon, 5 Aug 2024, Willow Liquorice wrote:
>>> 
>>>> I really should try and get 9front running on something instead of being a
>>>> theoretical hacker. Have the Raspberry Pi images been tested on the Pi Zero?
>>> 
>>> I can confirm the latest release images work great on the zero.
>>> (performance of games/nes notwithstanding... ;) )
>>> 
>>> --
>>> Jon

Re: [9front] Thoughts on Wayland?

[Emil Tomczyk]

6 sie 2024 02:13:57 Eli Cohen <echoline@gmail.com>:

> I still think it's sort of a place for a Wayland compositor... this
> thread got long. has anyone mentioned what halfwit was working on? a
> way log in to *nix machines and just run firefox or something the same
> as you would do anything else on Plan 9

You mean something like waypipe? Or rather VNC?

Re: [9front] Thoughts on Wayland?

[hiro]

> needs to be. Plus, AA really benefits everyone, and
> https://en.wikipedia.org/wiki/X_Rendering_Extension is only for fonts, not
> for arbitrary graphics. (Same problem as on Plan 9. Should we blame Keith
> Packard for not going further than just font rendering?) There's no way to
> have AA in arbitrary 2D graphics rendered on the X server, AFAIK.

What do you need AA for at all? It just makes everything look blurry
to me, and even for fonts I often disable this behavior (Though at my
low resolution that works only for high quality fonts that have good
hinting parameters embedded).
A lot of software depends on hand-crafted pixel art in their icons, I
see only downsides, let's avoid resampling and AA completely unless we
render our own oversampled images from something of much higher or
infinite (vector) resolution (for example like with fonts).

> So
> basically the widget toolkits gave up on supporting efficient networked X11
> ages ago, and not for completely frivolous reasons.

If they tried at all!

> But if video streaming is the solution, it seems to me an assumption is
> being made that the server (where the applications run) has a GPU then: it
> would help for rendering the applications (with AA, blending, animations
> and all), and it would help for video compression too. Somewhat of a break
> from tradition then. I agree that a solution designed specifically with low
> bandwidth and low latency in mind ought to be better. But is 9p really for
> that? I'm trying to work that way; time will tell...

My guess would be it's the other way around, the blit specific
multiplexing idea survives in a generalized way as 9p.
The precursor to the draw RPCs were meant to be multiplexed together
with other concurrent streams from and to the blit terminal, only
later 9p generalized this approach to cover not just terminals and
their mouse, keyboard and display. Instead, every ressource can be
accessed as a file and all file access multiplexed over a single 9p
mount. This is nicer than a new custom multiplexing protocol for every
service.
Back then multiplexing multiple concurrent accesses over a single
synchronous bidirectional pipe must have seemed like magic. I fear it
still does on the other Unix-descendants.
It still seems like magic when I see them reinventing it over and over
again in the HTTP2 and Quic fields.

Re: [9front] Thoughts on Wayland?

[sirjofri]

06.08.2024 18:40:31 hiro <23hiro@gmail.com>:
>> needs to be. Plus, AA really benefits everyone, and
>> https://en.wikipedia.org/wiki/X_Rendering_Extension is only for fonts, not
>> for arbitrary graphics. (Same problem as on Plan 9. Should we blame Keith
>> Packard for not going further than just font rendering?) There's no way to
>> have AA in arbitrary 2D graphics rendered on the X server, AFAIK.
>
> What do you need AA for at all? It just makes everything look blurry
> to me, and even for fonts I often disable this behavior (Though at my
> low resolution that works only for high quality fonts that have good
> hinting parameters embedded).
> A lot of software depends on hand-crafted pixel art in their icons, I
> see only downsides, let's avoid resampling and AA completely unless we
> render our own oversampled images from something of much higher or
> infinite (vector) resolution (for example like with fonts).

As for fonts, you are right. But for line rendering and graphics, AA has some benefits in many cases.

Take a line that's thinner than a pixel. Can you see it without any subpixel calculation? When they're diagonally you often see just a few portions of that line.

Or a grid, textures in the distance. Moire patterns?

sirjofri

Re: [9front] Thoughts on Wayland?

[Steve simon]

Not using anti aliasing on fonts does make them look sharper on a monitor.

However if you resize the resulting image, e.g. using a scaling monitor,
or resizing a screen-grab the result will most probably look vile; covered
in extra fringes or echos around letters (spacial alias)

Though interlaced video is becoming a thing of the past, people do still use interlace.
If you use interlaced,  and display text with non-anti-aliased fonts then you will get
temporal aliasing which is even nastier - the aliases will now flicker at interlace rate.

[ aside:
If you are suspicious that there may be temporal aliasing in moving video you can strobe out the alias frequency quite easily.
Look straight at the image and chomp your teeth together - The impact causes your eyeballs to oscillate,
The natural resonance of the muscles that control your your eyeballs is of the order of 20-30Hz.
Any temporal aliasing show now appear as static (or near static) spacial aliasing and be easy to spot.
I learnt this from Martin Weston, Very probably the cleverest person I have ever met
 ]

Re: [9front] Thoughts on Wayland?

[hiro]

> However if you resize the resulting image, e.g. using a scaling monitor,
> or resizing a screen-grab the result will most probably look vile; covered
> in extra fringes or echos around letters (spacial alias)

It will still look like vile whether you antialiased once at the
beginning or not. Because every time you resample you have to do it
again.
And it only gets worse each time.

Personally I found I have to turn off scaling in my monitor and use
it's native resolution, anything else is unusable.
If displayport cables had infinite bandwidth, i'd be like you and
render everything at a *higher* than native resolution and make the
monitor downsample. that would also be the day i start using
antialiasing even on fonts and start scaling each viewport
(separately) up and down as I please.

Sadly that is not possible, so I'm saving bandwidth which happens to
also increase link reliability: i continue to let my gpu output my
monitor's native resolution. it's a good sweet spot.

The goal should always be to avoid resampling, or to only downsample
once, with Lanczos, as near to the monitor as possible.
Sadly this means we have to render at the correct DPI already on the
application layer, and demand that nothing else afterwards resamples
our outputs. This makes the application more complicated. But I see no
valid way around it.
And don't think I don't regret the many downsides: For example, on the
UI side what also would help is to make it easy to snap back to the
original resolution and make it easy to pan the image with a very low
end-user delay.
Hard with the current designs, you'd need to first make your own
devdraw scaling monitor for this, and then offload zooming from page
(our image/pdf viewer) into devdraw itself. that's a slippery slope
though, cause next you'll ask for html and pdf rendering inside
devdraw, else you'd have to render at a very high and wasteful
resolution bec. for quality results you want to only downsample, never
upsample. Harder to find the sweet spot here, but I'm sure there's
one, if you care enough ;)

with modern systems the assumption seems to be that everything needs
to be resampled 5 times. the result hurts my brain. And they still
require GPUs for the other parts of the programs regardless, they are
just not used for anything useful but animations, video codecs,
website ads with crypto-miners, and 3D games.

If we can make a devdraw scaling monitor, it should support Lanczos
resampling which includes anti-aliasing, and support for common lossy
image codecs (instead of lz77). Then we can avoid the worst-case
scenario where we decode a lossy image and then upsample it to various
wasteful resolutions as we keep zooming in, which only gets slower and
slower at each iteration, infuriating the user. Tried zooming in on a
screenshare in a modern mainstream app? It also feels like that.

about interlacing: thankfully that's not a common VGA mode.

bec. all modern software sucks, my favourite resolution remains
1600x1200 and 60Hz is enough for my light mousing around. i never need
to upsample cause of my superior low resolution. and i'm quite sure
that my eyesight will degrade faster than software development efforts
are bearing fruits.