Multithreading support in pluggable module, but with problems

[Sebastian Gniazdowski <psprint2@fastmail.com>]

Hello,
being able to build modules outside Zsh I've created one that allows
(this works):

# zpin 'print -rl "a:b" "c:d"' | zpopulator -A my_hash
# print -r -- ${(kv)my_hash[@]}
a b c d

This is done entirely in background. zpin does fork() and this way makes
Zsh think it's finished – main process exits. zpopulator starts a thread
and makes main thread exit – this is also sufficient to make Zsh think
that there is no background job.

The aim is e.g. to do:

zpin "ls -R1 /home/user" | zpopulator -a my_files
# echo $zpworkers_count
1
<later..>
# echo $zpworkers_count
0

And work in shell as normal. Also, I plan to modify my syntax
highlighting so that paths are checked in background. This is to avoid
stalls when paths are on auto-mounted directory.

However I cannot resolve all problems:

1. I duplicate stdin of "zpopulator" (reading thread):

    /* Duplicate standard input */
    oconf->stream = fdopen( dup( fileno( stdin ) ), "r" );
    /* Prepare standard input replacement */
    oconf->r_devnull = fopen( "/dev/null", "r");
    /* Replace standard input with /dev/null */
    dup2( fileno( oconf->r_devnull ), STDIN_FILENO );
    fclose( oconf->r_devnull );

I tried with only first line of code, without last (fclose) line, and
result is always: 50% of runs has error:

"zpopulator: Warning: could not close input stream: Bad file descriptor"

This is from fclose(). There is a fcntl() check before:
        flags = fcntl( oconf->stream->_file, F_GETFD );

and it results in:

"zpopulator: Indeed bad descriptor 11: Bad file descriptor"

After fclose(), the ->_file, the descriptor, is even turned into -1.

Any idea why duplicated descriptor, used only in worker thread, is
invalid? What could Zsh do with stdin/out file descriptors of commands
it runs to cause such problems?

https://github.com/psprint/zpopulator

-- 
  Sebastian Gniazdowski
  psprint2@fastmail.com