[9fans] thread STACK size

[9fans] thread STACK size

[Mathieu Lonjaret]

Hi all,

A while ago, while working on btfs, I stumbled upon some sort of
overflow (http://9fans.net/archive/2009/07/77) which was in fact due
to the thread STACK being too small (and hence if I understood
correctly things would get written out of it, in the heap).
To be on the safe side, I have it set to 16384 now, but as I think I'm
getting near something usable with btfs, I'd like to go back to a more
fitting value.  I think it's pretty important to have it as low as
possible since the number of threads/coroutines will grow linearly
with the number of peers connected (to be honest, I don't even know if
that can even scale in terms of memory use).

So the question is, how can I evualuate what's the minimal value I can
set that to without getting into trouble again?  Is there anything
smarter than just trial and error?

Thanks,
Mathieu

[9fans] thread STACK size

[Sape Mullender]

> A while ago, while working on btfs, I stumbled upon some sort of
> overflow (http://9fans.net/archive/2009/07/77) which was in fact due
> to the thread STACK being too small (and hence if I understood
> correctly things would get written out of it, in the heap).
> To be on the safe side, I have it set to 16384 now, but as I think I'm
> getting near something usable with btfs, I'd like to go back to a more
> fitting value. I think it's pretty important to have it as low as
> possible since the number of threads/coroutines will grow linearly
> with the number of peers connected (to be honest, I don't even know if
> that can even scale in terms of memory use).
>
> So the question is, how can I evualuate what's the minimal value I can
> set that to without getting into trouble again? Is there anything
> smarter than just trial and error?

There's no good way, really.  One thing you might do is change the thread
library to initialize the stack to some pattern (zeroing it will probably
do, but you can let your phantasy go wild here).  You can then, when your
code has been running for a while, use acid -lthread and a bit of scripting
to scan your stacks for the higest point where the pattern is disturbed.


As a general rule in threaded programs, avoid declaring local arrays
or large structs.  Instead, malloc them and free them when you're done.
A file server, as an example, should never allocate an 8K message
buffer on the stack.  If you can manage to obey the rule of not having
arrays on the stack (as local variables), you can usually comfortably
make use of 4K or 8K stacks.

	Sape

Re: [9fans] thread STACK size

[Federico G. Benavento]

also if you're using bio(8) notice that a biobuf is >8kb

On Wed, May 19, 2010 at 6:20 AM, Sape Mullender
<sape@plan9.bell-labs.com> wrote:
>> A while ago, while working on btfs, I stumbled upon some sort of
>> overflow (http://9fans.net/archive/2009/07/77) which was in fact due
>> to the thread STACK being too small (and hence if I understood
>> correctly things would get written out of it, in the heap).
>> To be on the safe side, I have it set to 16384 now, but as I think I'm
>> getting near something usable with btfs, I'd like to go back to a more
>> fitting value. I think it's pretty important to have it as low as
>> possible since the number of threads/coroutines will grow linearly
>> with the number of peers connected (to be honest, I don't even know if
>> that can even scale in terms of memory use).
>>
>> So the question is, how can I evualuate what's the minimal value I can
>> set that to without getting into trouble again? Is there anything
>> smarter than just trial and error?
>
> There's no good way, really.  One thing you might do is change the thread
> library to initialize the stack to some pattern (zeroing it will probably
> do, but you can let your phantasy go wild here).  You can then, when your
> code has been running for a while, use acid -lthread and a bit of scripting
> to scan your stacks for the higest point where the pattern is disturbed.
>
>
> As a general rule in threaded programs, avoid declaring local arrays
> or large structs.  Instead, malloc them and free them when you're done.
> A file server, as an example, should never allocate an 8K message
> buffer on the stack.  If you can manage to obey the rule of not having
> arrays on the stack (as local variables), you can usually comfortably
> make use of 4K or 8K stacks.
>
>        Sape
>
>



-- 
Federico G. Benavento

Re: [9fans] thread STACK size

[Francisco J Ballesteros]

I've found it useful to use the testing of the program to also
force it to get into what I think is a worst case and then printing the
stack size (doing this is simple by printing argument addresses).

hth

On Wed, May 19, 2010 at 11:55 AM, Federico G. Benavento
<benavento@gmail.com> wrote:
> also if you're using bio(8) notice that a biobuf is >8kb
>
> On Wed, May 19, 2010 at 6:20 AM, Sape Mullender
> <sape@plan9.bell-labs.com> wrote:
>>> A while ago, while working on btfs, I stumbled upon some sort of
>>> overflow (http://9fans.net/archive/2009/07/77) which was in fact due
>>> to the thread STACK being too small (and hence if I understood
>>> correctly things would get written out of it, in the heap).
>>> To be on the safe side, I have it set to 16384 now, but as I think I'm
>>> getting near something usable with btfs, I'd like to go back to a more
>>> fitting value. I think it's pretty important to have it as low as
>>> possible since the number of threads/coroutines will grow linearly
>>> with the number of peers connected (to be honest, I don't even know if
>>> that can even scale in terms of memory use).
>>>
>>> So the question is, how can I evualuate what's the minimal value I can
>>> set that to without getting into trouble again? Is there anything
>>> smarter than just trial and error?
>>
>> There's no good way, really.  One thing you might do is change the thread
>> library to initialize the stack to some pattern (zeroing it will probably
>> do, but you can let your phantasy go wild here).  You can then, when your
>> code has been running for a while, use acid -lthread and a bit of scripting
>> to scan your stacks for the higest point where the pattern is disturbed.
>>
>>
>> As a general rule in threaded programs, avoid declaring local arrays
>> or large structs.  Instead, malloc them and free them when you're done.
>> A file server, as an example, should never allocate an 8K message
>> buffer on the stack.  If you can manage to obey the rule of not having
>> arrays on the stack (as local variables), you can usually comfortably
>> make use of 4K or 8K stacks.
>>
>>        Sape
>>
>>
>
>
>
> --
> Federico G. Benavento
>
>

Re: [9fans] thread STACK size

[erik quanstrom]

> As a general rule in threaded programs, avoid declaring local arrays
> or large structs.  Instead, malloc them and free them when you're done.
> A file server, as an example, should never allocate an 8K message
> buffer on the stack.  If you can manage to obey the rule of not having
> arrays on the stack (as local variables), you can usually comfortably
> make use of 4K or 8K stacks.

i've always followed this rule myself.  but it just occurs to me that
there might be another way of looking at this.

if our mythical program needs a fixed-size message buffer for every
request, there are advantages to the stack.  dynamic allocation
with the pool library is slow and single-threaded.  and of course,
any dynamic allocation is more work and can be fatal.

- erik