When is mem.c / malloc called?

When is mem.c / malloc called?

[Sebastian Gniazdowski]

Hello,
I've did:

@@ -1201,6 +1201,8 @@ static struct m_hdr *m_l;
 MALLOC_RET_T
 malloc(MALLOC_ARG_T size)
 {
+    fprintf( stderr, "Hello\n" );
+    fflush( stderr );
     struct m_hdr *m, *mp, *mt;

And compiled once without --enable-zsh-mem, once with it. Then in a
module I've called zshcalloc() and malloc(). Didn't get the message. Zsh
startup also didn't produce anything.

Is there a configuration when non-standard malloc() will be used?

I'm writing a module that needs thread safe memory allocation. So
dupstring() is out. System malloc() is as I researched good. I have to
stick to it when allocating hash, array and variable. As dupstring()
etc. is a temporary memory, I should be fine assuming that I won't have
to use zhalloc() when allocating permanent variables. But custom
internal malloc() would kill the thing.

-- 
  Sebastian Gniazdowski
  psprint2@fastmail.com

Re: When is mem.c / malloc called?

[Sebastian Gniazdowski]

Don't know why, but now when I have the fprintf(), and do:

make clean ; ./configure --enable-zsh-debug --enable-zsh-mem && make
CFLAGS="-g -O0" && make install

then I see "Hello", and shell startup always ends in:

 mem.c:1341: MEM: allocation error at sbrk, size 20480.

i.e. aborts, so I guess I'm safe – the pathway to use non-standard,
thread unsafe malloc() is blocked because it's not working.

-- 
  Sebastian Gniazdowski
  psprint2@fastmail.com

Re: When is mem.c / malloc called?

[Bart Schaefer]

On Feb 10,  7:17am, Sebastian Gniazdowski wrote:
} Subject: Re: When is mem.c / malloc called?
}
} Don't know why, but now when I have the fprintf() ...
} then I see "Hello", and shell startup always ends in:
} 
}  mem.c:1341: MEM: allocation error at sbrk, size 20480.
} 
} i.e. aborts, so I guess I'm safe - the pathway to use non-standard,
} thread unsafe malloc() is blocked because it's not working.

What's not working is your debugging technique.  fprintf/fflush invoke
memory allocation, so if you use them *inside* the allocator, things
break.  You would need to use low-level system calls e.g. write()
directly to avoid this, or use an actual debugger like GDB and put a
breakpoint in malloc.

Re: When is mem.c / malloc called?

[Bart Schaefer]

On Feb 10,  4:24am, Sebastian Gniazdowski wrote:
}
} ... compiled once without --enable-zsh-mem, once with it. Then in a
} module I've called zshcalloc() and malloc(). Didn't get the message.

--enable-zsh-mem should turn on the internal malloc, see line 1026 of
mem.c.  However, if you didn't relink separately-compiled modules,
they might still link at runtime to the system malloc.

} I'm writing a module that needs thread safe memory allocation.

This sounds pretty dangerous, as the rest of zsh (signal handling in
particular, update of the environment on e.g. "export", etc.) is not
going to be thread-safe.

Re: When is mem.c / malloc called?

[Sebastian Gniazdowski]

On Fri, Feb 10, 2017, at 10:25 AM, Bart Schaefer wrote:
> } I'm writing a module that needs thread safe memory allocation.
> 
> This sounds pretty dangerous, as the rest of zsh (signal handling in
> particular, update of the environment on e.g. "export", etc.) is not
> going to be thread-safe.
> 

I've ended up copying multiple Zsh functions, calling them
my_{original_name}, creating separate call tree. The goal is to do this
in multi-threaded manner:

# print -rl -- "a,b" "c,d" | zpopulator -D , -vA myhash
# typeset -p myhash
typeset -A myhash=( a b c d )

I basically create a thread after I do:

fdopen( dup( fileno( stdin ) ), "r" );

in module's builtin – zpopulator. I was counting on: Zsh will think
module command ended because main thread will do "return 0", it will not
break the pipe because it gets broken by itself when reader or writer
exits. This way the forked "print -rl" and the thread-splitted
"zpopulator" could work in background. However I just discovered that:

# ls -R / | zpopulator -D , -vA myhash

DOES block shell.. I successfully obtained no-block behavior but it was
zpopulator that was persisting, not the writing process (forked one). 

Have you any idea how to cause above to not block shell? zpopulator's
thread exits from module's builtin, this should be close to unblock
shell..

Code:
https://github.com/psprint/zpopulator/blob/41f40f7b35c67fed907d660b620f4ec5bda6022e/module/Src/psprint/zpopulator.c#L438-L442

-- 
Sebastian Gniazdowski