Re: [9fans] Do the Pool Voodoo like I do

Re: [9fans] Do the Pool Voodoo like I do

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

You're running past the allocated memory.  'mem user overflow' means that
malloc detected that your program has written past the end of the allocated buffer.
It can do this because it actually may 'overallocate', i.e., allocating more memory
than you asked for.  It puts a predetermined pattern in the extra bytes.  If it
finds  that pattern has been altered, it complains its little ass off.

[-- Attachment #2: Type: message/rfc822, Size: 2861 bytes --]

From: dbailey27@ameritech.net
To: 9fans@cse.psu.edu
Subject: [9fans] Do the Pool Voodoo like I do
Date: Thu, 19 Feb 2004 19:52:58 -0500
Message-ID: <8a1ec9efc59551c37ee2638fb0d006c1@yourdomain.dom>

Hey all,
	I'm having a bit of trouble with the malloc
library. I've got an algorithm that dynamically
allocates memory char by char using realloc in
a loop. However, the lib seems to be having some
internal problem. I've got seriously unoptimized
code handling the token stream this algorithm
parses, so there are two cases for the crash below:
	1) my unoptimized code is allocating too much
	    heap and that's causing the panic
	2) there is an internal alloc bug that I'm triggering

Here's the panic info:

mem user overflow
pool sbrkmem block d9a0
hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe
tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020
user data 00 3e 1f 00  00 24 ff 23 | 00 fe f1 f0  ca be 00 15
panic: pool panic
8.out 703: suicide: sys: trap: fault read addr=0x0 pc=0x000065fb
apple%

Any ideas? More info provided on request.

Don (north_)

Re: [9fans] Do the Pool Voodoo like I do

[Russ Cox]

> mem user overflow
> pool sbrkmem block d9a0
> hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe
> tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020
> user data 00 3e 1f 00  00 24 ff 23 | 00 fe f1 f0  ca be 00 15
> panic: pool panic
> 8.out 703: suicide: sys: trap: fault read addr=0x0 pc=0x000065fb
> apple%

> You're running past the allocated memory.  'mem user overflow' means that
> malloc detected that your program has written past the end of the allocated buffer.
> It can do this because it actually may 'overallocate', i.e., allocating more memory
> than you asked for.  It puts a predetermined pattern in the extra bytes.  If it
> finds  that pattern has been altered, it complains its little ass off.

In particular you have allocated three bytes and
stored four bytes -- 24 FF 23 00 into it.  Maybe you
are using string routines and not counting the
trailing nul?

The displayed info is a gold mine.  If you do any
debugging of malloc problems, it's worth learning
what it means.  It's also useful for debugging stray
pointer problems.  We spend a fair amount of space
on extra info useful for debugging:

> hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe

This is the first six words of the block.  The user data
only starts four words in, meaning if you're in acid
and want to dump the block for pointer p, you need
mem(p-16, "10X") or so.

> tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020

This is the last words of the data space, then a |, then
the two tail words.

Thus the whole block is

0a110c09 - magic number for an allocated block
00000020 - size of entire block, header and tail included
000023b5 - pc of malloc for block
00001f3e - pc of realloc for block
0023ff24 - begin actual data space
caf0f1fe
ef1500be - this is four bytes: be 00 15 ef.  the be and ef
	are magic numbers, the 00 15 is the amount of header
	and tail in the block, not counting the two pc words.
	since the block is 0x20, the data size is 0x20 - 0x15 - 2*4 = 3.
00000020 - size of entire block, header and tail included

Thus the actual data is 24 FF 23.  When there are extra bytes
in the data space, the first up to four extra bytes are initialized
to FE, F1, F0, FA, rotated so that FE ends up on a four-aligned
address.

Russ

Re: [9fans] Do the Pool Voodoo like I do

[dbailey27]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

Nice! Thanks for the detailed explanation, Russ.
Fortunately, I didn't have to worry about going very
deep. As soon as Presotto noticed that the problem
was a *user* overflow, I went back through my code.

Sure enough, the problem was that I was moving a
block of memory, but based on the strlen(), not the
strlen() plus the terminating null. My logic fumbled
somewhere through the recursion.

On the bright side, my preprocessor is finished *yay*.

Thanks for the help, as always, guys.

Don (north_)

[-- Attachment #2: Type: message/rfc822, Size: 4695 bytes --]

From: "Russ Cox" <rsc@swtch.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Do the Pool Voodoo like I do
Date: Sat, 21 Feb 2004 20:09:41 -0500
Message-ID: <E1Aui7p-000ORF-M7@t40.swtch.com>

> mem user overflow
> pool sbrkmem block d9a0
> hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe
> tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020
> user data 00 3e 1f 00  00 24 ff 23 | 00 fe f1 f0  ca be 00 15
> panic: pool panic
> 8.out 703: suicide: sys: trap: fault read addr=0x0 pc=0x000065fb
> apple%

> You're running past the allocated memory.  'mem user overflow' means that
> malloc detected that your program has written past the end of the allocated buffer.
> It can do this because it actually may 'overallocate', i.e., allocating more memory
> than you asked for.  It puts a predetermined pattern in the extra bytes.  If it
> finds  that pattern has been altered, it complains its little ass off.

In particular you have allocated three bytes and
stored four bytes -- 24 FF 23 00 into it.  Maybe you
are using string routines and not counting the
trailing nul?

The displayed info is a gold mine.  If you do any
debugging of malloc problems, it's worth learning
what it means.  It's also useful for debugging stray
pointer problems.  We spend a fair amount of space
on extra info useful for debugging:

> hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe

This is the first six words of the block.  The user data
only starts four words in, meaning if you're in acid
and want to dump the block for pointer p, you need
mem(p-16, "10X") or so.

> tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020

This is the last words of the data space, then a |, then
the two tail words.

Thus the whole block is

0a110c09 - magic number for an allocated block
00000020 - size of entire block, header and tail included
000023b5 - pc of malloc for block
00001f3e - pc of realloc for block
0023ff24 - begin actual data space
caf0f1fe
ef1500be - this is four bytes: be 00 15 ef.  the be and ef
	are magic numbers, the 00 15 is the amount of header
	and tail in the block, not counting the two pc words.
	since the block is 0x20, the data size is 0x20 - 0x15 - 2*4 = 3.
00000020 - size of entire block, header and tail included

Thus the actual data is 24 FF 23.  When there are extra bytes
in the data space, the first up to four extra bytes are initialized
to FE, F1, F0, FA, rotated so that FE ends up on a four-aligned
address.

Russ

Re: [9fans] Do the Pool Voodoo like I do

[Geoff Collyer]

Try the suggestions in DIAGNOSTICS of malloc(2), and follow the
pointers in the SEE ALSO section, pool(2) in particular.

[9fans] Do the Pool Voodoo like I do

[dbailey27]

Hey all,
	I'm having a bit of trouble with the malloc
library. I've got an algorithm that dynamically
allocates memory char by char using realloc in
a loop. However, the lib seems to be having some
internal problem. I've got seriously unoptimized
code handling the token stream this algorithm
parses, so there are two cases for the crash below:
	1) my unoptimized code is allocating too much
	    heap and that's causing the panic
	2) there is an internal alloc bug that I'm triggering

Here's the panic info:

mem user overflow
pool sbrkmem block d9a0
hdr 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe
tail 0a110c09 00000020 000023b5 00001f3e 0023ff24 caf0f1fe | ef1500be 00000020
user data 00 3e 1f 00  00 24 ff 23 | 00 fe f1 f0  ca be 00 15
panic: pool panic
8.out 703: suicide: sys: trap: fault read addr=0x0 pc=0x000065fb
apple%

Any ideas? More info provided on request.

Don (north_)