Re: [9fans] "Blocks" in C

[Charles Forsyth <forsyth@terzarima.net>]

> i mentioned that the pool library can act as a big
> kernel lock a few weeks ago.  i don't know if anyone
> has thoughts on how to deal with this.

it isn't really a `big kernel lock' in the linux sense.
the big kernel lock was the device by which operating systems
written with only a uniprocessor in mind were made to `work' on a multiprocessor.
typically, the equivalent of qlock and qunlock of a
single global lock were placed at strategic points to
bracket (say) every system call that affected non-trivial state.
then the developers started splitting the lock region.
it's easier than thinking. processes would queue for use of
the kernel (but they'd go to sleep, not spin).

for the pool library it's a spin lock, which is intended to
protect small blocks of code. really the effects of collision
should not be as bad as they are.
although it's probably good especially with many cores to have some processor-local allocation
of cpu and memory resources without interlock, or make the main storage pool locks more refined,
or using wait-free cleverness,
and indeed there are schemes to do all those things, the big initial problem with
the pool library is that it's remarkably slow. in fact,
that's been my experience with every tree-based allocator
i've had to use, so it's not just that pool is replete
with error checking. an alternative allocator that uses quickfit on top of
a variant of the simple allocator in kernighan and ritchie is
roughly 30 times faster on some malloc-thrashing tests (i'm just remembering that
from tests i did last year, so i might have the number wrong, but it's substantial).
for a reasonable choice of block quantum it also has better fragmentation properties,
which is amusing since that's supposed to be one of the advantages of the tree-based allocators.