[TUHS] Re: If forking is bad, how about buffering?

[Bakul Shah via TUHS <tuhs@tuhs.org>]

On May 15, 2024, at 9:42 AM, G. Branden Robinson <g.branden.robinson@gmail.com> wrote:
> 
> I contemplated following up Bakul Shah's
> post with a mention of Jim Gettys's work on bufferbloat.[1]  So let me
> do that here, and venture the opinion that a "buffer" as popularly
> conceived and implemented (more or less just a hunk of memory to house
> data) is too damn dumb a data structure for many of the uses to which it
> is put.

Note that even if you remove every RAM buffer between the two
endpoints of a TCP connection, you still have a "buffer". Example:
If you have a 1Gbps pipe between SF & NYC, the pipe itself can store
something like 3.5MB to 4MB in each direction! As the pipe can be
lossy, you have to buffer up N (=bandwidth*latency) bytes at the
sending end (until you see an ack for the previous Nth byte), if you
want to utilize the full bandwidth.

Now what happens if the sender program exits right after sending
the last byte? Something on behalf of the sender has to buffer up
and stick around to complete the TCP dance. Even if the sender is
cat -u, the kernel or a network daemon process atop a microkernel
has to buffer this data[1].

Unfortunately you can't abolish latency! But where to put buffers
is certainly an engineering choice that can impact compositionality
or other problems such as bufferbloat.

[1] This brings up a separate point: in a microkernel even a simple
thing like "foo | bar" would require a third process - a "pipe
service", to buffer up the output of foo! You may have reduced
the overhead of individual syscalls but you will have more of
cross-domain calls!