Re: [9fans] zero copy & 9p (was Re: PDP11 (Was: Re: what heavy negativity!)

[Steven Stallion <sstallion@gmail.com>]

> On Oct 10, 2018, at 2:54 PM, Steven Stallion <sstallion@gmail.com> wrote:
>
> You seem to be saying zero-copy wouldn't buy anything until these
> other problems are solved, right?

Fundamentally zero-copy requires that the kernel and user process
share the same virtual address space mapped for the given operation.
This can't always be done and the kernel will be forced to perform a
copy anyway. To wit, one of the things I added to the exynos kernel
early on was a 1:1 mapping of the virtual kernel address space such
that something like zero-copy could be possible in the future (it was
also very convenient to limit MMU swaps on the Cortex-A15). That said,
the problem gets harder when you're working on something more general
that can handle the entire address space. In the end, you trade the
complexity/performance hit of MMU management versus making a copy.
Believe it or not, sometimes copies can be faster, especially on
larger NUMA systems.

> Suppose you could replace 9p based FS with something of your choice.
> Would it have made your jobs easier? Code less grotty? In other
> words, is the complexity of the driver to achieve high throughput
> due to the complexity of hardware or is it due to 9p's RPC model?
> For streaming data you pretty much have to have some sort of
> windowing protocol (data prefetch or write behind with mmap is a
> similar thing).

This is one of those problems that afflicts storage more than any
other subsystem, but like most things it's a tradeoff. Having a
filesystem that doesn't support 9P doesn't seem to make much sense on
Plan 9 given the ubiquity of the protocol. Dealing with the multiple
outstanding issue does make filesystem support much more complex and
would have a far-reaching effect on existing code (not to mention the
kernel).

It's completely possible to support prefetch and/or streaming I/O
using existing kernel interfaces. cinap's comment about read not
returning until the entire buffer is read is an implementation detail
of the underlying device. A read call is free to return fewer bytes
than requested; it's not uncommon for a driver to return partial data
to favor latency over throughput. In other words, there's no magic
behind mmap - it's a convenience interface. If you look at how other
kernels tend to implement I/O, there are generally fundamental calls
to the a read/write interface - there are no special provisions for
mmap beyond the syscall layer.

The beauty of 9P is you can wrap driver filesystems for added
functionality. Want a block caching interface? Great! Slap a kernel
device on top of a storage driver that handles caching and prefetch.
I'm sure you can see where this is going...

> Looks like people who have worked on the plan9 kernel have learned
> a lot of lessons and have a lot of good advice to offer. I'd love
> to learn from that. Except usually I rarely see anyone criticizing
> plan9.

Something, something, in polite company :-)