How is smp handled in file services?

How is smp handled in file services?

[Trevor Higgins]

Just trying ti understand the data flow through the kernel (as compared 
to *nix).

Given the following easy to understand example:

Assume that three processes are writing to /dev/audio through the mixfs 
file server. Given that all processes are on the same terminal server, 
and that there are ample idle cpu cores (ignore locking and other side 
effects), what is the data flow path in mixfs:

    1. All three procs sleep while some proc processes the write 
requests sequentially, returning the results sequentially.

    2. All three procs enter the file service and process one request 
concurrently and return result directly to own process.

I am assuming that unless designed specifically in the file server, that 
the answer is 1.


-- 
We need another plan

Re: [9front] How is smp handled in file services?

[ori]

> Just trying ti understand the data flow through the kernel (as compared 
> to *nix).
> 
> Given the following easy to understand example:
> 
> Assume that three processes are writing to /dev/audio through the mixfs 
> file server. Given that all processes are on the same terminal server, 
> and that there are ample idle cpu cores (ignore locking and other side 
> effects), what is the data flow path in mixfs:
> 
>     1. All three procs sleep while some proc processes the write 
> requests sequentially, returning the results sequentially.
>
>     2. All three procs enter the file service and process one request 
> concurrently and return result directly to own process.
> 
> I am assuming that unless designed specifically in the file server, that 
> the answer is 1.

If you're in userspace, the file server is just reading from
an FD. it's up to you if you want to add concurrency around
those reads.

If you're in the kernel, it's concurrent. a process enters
the kernel from sysfile.c:/^sysread, which pretty much just
calls sysfile.c:/^read.

There aren't any locks held up to:

		nnn = nn = devtab[c->type]->read(c, p, n, off);

Smokey the bear says: Only you can prevent corrupted data
structures.

Re: [9front] How is smp handled in file services?

[Trevor Higgins]

On 03/11/2020 02:03 PM, ori@eigenstate.org wrote:

I dont mean to offend , but that seems more than a bit sucky.

Listening to Rob Pike wax lyrically about the benefits of CSP over 
parallel , seems to ring a bit hollow in the face of
all the evidence to the contrary.
(I must admit his metaphor of 'book burning' to explain CSP now looks 
quite prescient given the current climate of intolerance to free speech).

So 15 of my 16 mk tasks come to a shit shattering halt when ever I/O 
needs to be done. This explains to me why ramfs does not provide much of 
a speed increase when building.

This probably will translate very badly for performance when file 
systems are stacked on top of each other. Rather than achieving
high concurrency ,  it will degenerate into a linear flow through the 
stack and back.

Probably also explains why 9p over the interweb is so slow.

What I don't see in the 'Plan' is how to introduce a pipeline into the 
Fileserver processing to take advantage of SMP. Even if you
break each I/O request into 10 steps, there is no way to separate this 
out into 10 programs all working concurrently. So as you say
each file service has to make it's own determination on how many tasks 
to run concurrently, which is also sub-optimal.

The design of P9 is to have FS in userspace and not kernel so 
optimizations there are not going to help.

>> Just trying ti understand the data flow through the kernel (as compared
>> to *nix).
>>
>> Given the following easy to understand example:
>>
>> Assume that three processes are writing to /dev/audio through the mixfs
>> file server. Given that all processes are on the same terminal server,
>> and that there are ample idle cpu cores (ignore locking and other side
>> effects), what is the data flow path in mixfs:
>>
>>      1. All three procs sleep while some proc processes the write
>> requests sequentially, returning the results sequentially.
>>
>>      2. All three procs enter the file service and process one request
>> concurrently and return result directly to own process.
>>
>> I am assuming that unless designed specifically in the file server, that
>> the answer is 1.
> If you're in userspace, the file server is just reading from
> an FD. it's up to you if you want to add concurrency around
> those reads.
>
> If you're in the kernel, it's concurrent. a process enters
> the kernel from sysfile.c:/^sysread, which pretty much just
> calls sysfile.c:/^read.
>
> There aren't any locks held up to:
>
> 		nnn = nn = devtab[c->type]->read(c, p, n, off);
>
> Smokey the bear says: Only you can prevent corrupted data
> structures.
>

-- 
We need another plan

Re: [9front] How is smp handled in file services?

[ori]

> On 03/11/2020 02:03 PM, ori@eigenstate.org wrote:
> 
> I dont mean to offend , but that seems more than a bit sucky.
> 
> Listening to Rob Pike wax lyrically about the benefits of CSP over 
> parallel , seems to ring a bit hollow in the face of
> all the evidence to the contrary.
> (I must admit his metaphor of 'book burning' to explain CSP now looks 
> quite prescient given the current climate of intolerance to free speech).
> 
> So 15 of my 16 mk tasks come to a shit shattering halt when ever I/O 
> needs to be done. This explains to me why ramfs does not provide much of 
> a speed increase when building.
> 
> This probably will translate very badly for performance when file 
> systems are stacked on top of each other. Rather than achieving
> high concurrency ,  it will degenerate into a linear flow through the 
> stack and back.
> 
> Probably also explains why 9p over the interweb is so slow.
> 
> What I don't see in the 'Plan' is how to introduce a pipeline into the 
> Fileserver processing to take advantage of SMP. Even if you
> break each I/O request into 10 steps, there is no way to separate this 
> out into 10 programs all working concurrently. So as you say
> each file service has to make it's own determination on how many tasks 
> to run concurrently, which is also sub-optimal.
> 
> The design of P9 is to have FS in userspace and not kernel so 
> optimizations there are not going to help.

Disk file systems like cwfs tend to be fairly parallel. Code can't
be automatically parallelized, you need to write it that way. Using
CSP if you like.

As for latency -- this is due to design choices in 9p that will
need to be fixed at some point, and adding parallelism in the file
server won't help.

Re: [9front] How is smp handled in file services?

[hiro]

latency is not *due* to 9p.
9p is slow *due* to latency.
if at all.
try random disk access latency on linux. it's magnitudes worse.
bandwidth is low, yes. and that is because right now 9p doesn't help
you keep the buffers full.