check if coproc has output

check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

Hello, I want to populate an array from a long running process, something
like:

long_running_process | while read line
do
  array+=$line
done

problem is, since long_running_process does not continually produce output
but only some times, when i try to interrupt this pipeline with ctrl-c I
have to wait until it produces some output for the process to terminate
(because, as I understand it, when it tries to write to the pipe it
receives a sigpipe due to it being already closed)

I have tried various constructs with trap and even coproc to be able to
intercept the ctrl-c and send a sigpipe to the long running process, but to
no result.

I wonder if someone has some solution to share for this problem, thanks

Pier Paolo Grassi

[-- Attachment #2: Type: text/html, Size: 1105 bytes --]

Re: check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]

I realize now the subject doesn't reflect the final version of the mail,
sorry for that

Pier Paolo Grassi


Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
pierpaolog@gmail.com> ha scritto:

> Hello, I want to populate an array from a long running process, something
> like:
>
> long_running_process | while read line
> do
>   array+=$line
> done
>
> problem is, since long_running_process does not continually produce output
> but only some times, when i try to interrupt this pipeline with ctrl-c I
> have to wait until it produces some output for the process to terminate
> (because, as I understand it, when it tries to write to the pipe it
> receives a sigpipe due to it being already closed)
>
> I have tried various constructs with trap and even coproc to be able to
> intercept the ctrl-c and send a sigpipe to the long running process, but to
> no result.
>
> I wonder if someone has some solution to share for this problem, thanks
>
> Pier Paolo Grassi
>

[-- Attachment #2: Type: text/html, Size: 1741 bytes --]

Re: check if coproc has output

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]

You could get pid from $jobtexts?

On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
wrote:

> I realize now the subject doesn't reflect the final version of the mail,
> sorry for that
>
> Pier Paolo Grassi
>
>
> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
> pierpaolog@gmail.com> ha scritto:
>
>> Hello, I want to populate an array from a long running process, something
>> like:
>>
>> long_running_process | while read line
>> do
>>   array+=$line
>> done
>>
>> problem is, since long_running_process does not continually produce
>> output but only some times, when i try to interrupt this pipeline with
>> ctrl-c I have to wait until it produces some output for the process to
>> terminate (because, as I understand it, when it tries to write to the pipe
>> it receives a sigpipe due to it being already closed)
>>
>> I have tried various constructs with trap and even coproc to be able to
>> intercept the ctrl-c and send a sigpipe to the long running process, but to
>> no result.
>>
>> I wonder if someone has some solution to share for this problem, thanks
>>
>> Pier Paolo Grassi
>>
>

-- 
Best regards,
Sebastian Gniazdowski

[-- Attachment #2: Type: text/html, Size: 2337 bytes --]

Re: check if coproc has output

[Roman Perepelitsa]

On Thu, Jan 19, 2023 at 4:34 PM Pier Paolo Grassi <pierpaolog@gmail.com> wrote:
>
> Hello, I want to populate an array from a long running process, something like:
>
> long_running_process | while read line
> do
>   array+=$line
> done
>
> problem is, since long_running_process does not continually produce output but only some times, when i try to interrupt this pipeline with ctrl-c I have to wait until it produces some output for the process to terminate (because, as I understand it, when it tries to write to the pipe it receives a sigpipe due to it being already closed)

When you press Ctrl-C, zsh sends SIGINT to long_running_process.
Ideally, it should honor the signal and terminate. Do you know why it
doesn't do that?

Roman.

Re: check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

I'm sorry, what is $jobtexts? it seems an empty variable to me and doesn't
appear in zshall manual

Pier Paolo Grassi


Il giorno gio 19 gen 2023 alle ore 17:02 Sebastian Gniazdowski <
sgniazdowski@gmail.com> ha scritto:

> You could get pid from $jobtexts?
>
> On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
> wrote:
>
>> I realize now the subject doesn't reflect the final version of the mail,
>> sorry for that
>>
>> Pier Paolo Grassi
>>
>>
>> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
>> pierpaolog@gmail.com> ha scritto:
>>
>>> Hello, I want to populate an array from a long running process,
>>> something like:
>>>
>>> long_running_process | while read line
>>> do
>>>   array+=$line
>>> done
>>>
>>> problem is, since long_running_process does not continually produce
>>> output but only some times, when i try to interrupt this pipeline with
>>> ctrl-c I have to wait until it produces some output for the process to
>>> terminate (because, as I understand it, when it tries to write to the pipe
>>> it receives a sigpipe due to it being already closed)
>>>
>>> I have tried various constructs with trap and even coproc to be able to
>>> intercept the ctrl-c and send a sigpipe to the long running process, but to
>>> no result.
>>>
>>> I wonder if someone has some solution to share for this problem, thanks
>>>
>>> Pier Paolo Grassi
>>>
>>
>
> --
> Best regards,
> Sebastian Gniazdowski
>
>

[-- Attachment #2: Type: text/html, Size: 3057 bytes --]

Re: check if coproc has output

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1966 bytes --]

Are you running zsh 3.0? That would only explain lack of following in
zshall:

       jobtexts
              This associative array maps job numbers to the texts of the
command
              lines that were used to start the jobs.

              Handling  of  the keys of the associative array is as
described for
              jobdirs above.


On Thu, 19 Jan 2023 at 16:08, Pier Paolo Grassi <pierpaolog@gmail.com>
wrote:

> I'm sorry, what is $jobtexts? it seems an empty variable to me and doesn't
> appear in zshall manual
>
> Pier Paolo Grassi
>
>
> Il giorno gio 19 gen 2023 alle ore 17:02 Sebastian Gniazdowski <
> sgniazdowski@gmail.com> ha scritto:
>
>> You could get pid from $jobtexts?
>>
>> On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
>> wrote:
>>
>>> I realize now the subject doesn't reflect the final version of the mail,
>>> sorry for that
>>>
>>> Pier Paolo Grassi
>>>
>>>
>>> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
>>> pierpaolog@gmail.com> ha scritto:
>>>
>>>> Hello, I want to populate an array from a long running process,
>>>> something like:
>>>>
>>>> long_running_process | while read line
>>>> do
>>>>   array+=$line
>>>> done
>>>>
>>>> problem is, since long_running_process does not continually produce
>>>> output but only some times, when i try to interrupt this pipeline with
>>>> ctrl-c I have to wait until it produces some output for the process to
>>>> terminate (because, as I understand it, when it tries to write to the pipe
>>>> it receives a sigpipe due to it being already closed)
>>>>
>>>> I have tried various constructs with trap and even coproc to be able to
>>>> intercept the ctrl-c and send a sigpipe to the long running process, but to
>>>> no result.
>>>>
>>>> I wonder if someone has some solution to share for this problem, thanks
>>>>
>>>> Pier Paolo Grassi
>>>>
>>>
>>
>> --
>> Best regards,
>> Sebastian Gniazdowski
>>
>>

-- 
Best regards,
Sebastian Gniazdowski

[-- Attachment #2: Type: text/html, Size: 4246 bytes --]

Re: check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 2315 bytes --]

I did again the search in the manual and now I found it, maybe I mistyped
the first time
where do you suggest I try to print this array?

Pier Paolo Grassi


Il giorno gio 19 gen 2023 alle ore 17:10 Sebastian Gniazdowski <
sgniazdowski@gmail.com> ha scritto:

> Are you running zsh 3.0? That would only explain lack of following in
> zshall:
>
>        jobtexts
>               This associative array maps job numbers to the texts of the
> command
>               lines that were used to start the jobs.
>
>               Handling  of  the keys of the associative array is as
> described for
>               jobdirs above.
>
>
> On Thu, 19 Jan 2023 at 16:08, Pier Paolo Grassi <pierpaolog@gmail.com>
> wrote:
>
>> I'm sorry, what is $jobtexts? it seems an empty variable to me and
>> doesn't appear in zshall manual
>>
>> Pier Paolo Grassi
>>
>>
>> Il giorno gio 19 gen 2023 alle ore 17:02 Sebastian Gniazdowski <
>> sgniazdowski@gmail.com> ha scritto:
>>
>>> You could get pid from $jobtexts?
>>>
>>> On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
>>> wrote:
>>>
>>>> I realize now the subject doesn't reflect the final version of the
>>>> mail, sorry for that
>>>>
>>>> Pier Paolo Grassi
>>>>
>>>>
>>>> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
>>>> pierpaolog@gmail.com> ha scritto:
>>>>
>>>>> Hello, I want to populate an array from a long running process,
>>>>> something like:
>>>>>
>>>>> long_running_process | while read line
>>>>> do
>>>>>   array+=$line
>>>>> done
>>>>>
>>>>> problem is, since long_running_process does not continually produce
>>>>> output but only some times, when i try to interrupt this pipeline with
>>>>> ctrl-c I have to wait until it produces some output for the process to
>>>>> terminate (because, as I understand it, when it tries to write to the pipe
>>>>> it receives a sigpipe due to it being already closed)
>>>>>
>>>>> I have tried various constructs with trap and even coproc to be able
>>>>> to intercept the ctrl-c and send a sigpipe to the long running process, but
>>>>> to no result.
>>>>>
>>>>> I wonder if someone has some solution to share for this problem, thanks
>>>>>
>>>>> Pier Paolo Grassi
>>>>>
>>>>
>>>
>>> --
>>> Best regards,
>>> Sebastian Gniazdowski
>>>
>>>
>
> --
> Best regards,
> Sebastian Gniazdowski
>
>

[-- Attachment #2: Type: text/html, Size: 5042 bytes --]

Re: check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 1618 bytes --]

Hi Roman, I do not. I cannot even strace the process because it is an .exe
invoked from a windows wsl environment, but it shouldn't
 be a problem regarding the handling of the int signal, other scripts I
made in the same manner handle it properly..
I haven't mentioned this in the first place because I had similar problems
in the past with pure linux pipelines.
I have now discovered this:

if I invoke the .exe directly, eg:

long_running_process.exe | while etc

the ctrl+c is handled correctly, whether if I enclose it in a function like
this:

winclip(){
  local cmd=($DOTNET_PRJ/pasteclip/pasteclip.exe $args)
  $cmd "$@"
}

doing

winclip | while etc

shows the ctrl-c problem

zsh (under wsl) is 5.4.2

Pier Paolo Grassi


Il giorno gio 19 gen 2023 alle ore 17:02 Roman Perepelitsa <
roman.perepelitsa@gmail.com> ha scritto:

> On Thu, Jan 19, 2023 at 4:34 PM Pier Paolo Grassi <pierpaolog@gmail.com>
> wrote:
> >
> > Hello, I want to populate an array from a long running process,
> something like:
> >
> > long_running_process | while read line
> > do
> >   array+=$line
> > done
> >
> > problem is, since long_running_process does not continually produce
> output but only some times, when i try to interrupt this pipeline with
> ctrl-c I have to wait until it produces some output for the process to
> terminate (because, as I understand it, when it tries to write to the pipe
> it receives a sigpipe due to it being already closed)
>
> When you press Ctrl-C, zsh sends SIGINT to long_running_process.
> Ideally, it should honor the signal and terminate. Do you know why it
> doesn't do that?
>
> Roman.
>

[-- Attachment #2: Type: text/html, Size: 2677 bytes --]

Re: check if coproc has output

[Roman Perepelitsa]

On Thu, Jan 19, 2023 at 5:24 PM Pier Paolo Grassi <pierpaolog@gmail.com> wrote:
>
> if I invoke the .exe directly, eg:
>
> long_running_process.exe | while etc
>
> the ctrl+c is handled correctly, whether if I enclose it in a function like this:
>
> winclip(){
>   local cmd=($DOTNET_PRJ/pasteclip/pasteclip.exe $args)
>   $cmd "$@"
> }
>
> doing
>
> winclip | while etc
>
> shows the ctrl-c problem

I recall there were recently emails related to unkillable pipelines
and maybe even commits with fixes. You might be able to find them on
zsh-workers.

In any case, you could try to create a minimal test case. For example,
is it necessary to have a `while` loop or can you replace that with a
single read? Is it necessary to invoke the specific executable or can
you invoke something like `sleep 10` instead? Maybe the following
hangs for you?

    f() sleep 10
    f | read

If not, what must be changed so that it starts hanging?

Roman.

Re: check if coproc has output

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 2852 bytes --]

I've read your post briefly and saw that you need to terminate the
background process. I recall that I was doing it in configure-wrapper that
I once wrote and that I used jobtexts. I do recall that it does contain
entry for coproc, so you could examine it in a INT trap and send a signal
to the coproc writing process.

On Thu, 19 Jan 2023 at 16:14, Pier Paolo Grassi <pierpaolog@gmail.com>
wrote:

> I did again the search in the manual and now I found it, maybe I mistyped
> the first time
> where do you suggest I try to print this array?
>
> Pier Paolo Grassi
>
>
> Il giorno gio 19 gen 2023 alle ore 17:10 Sebastian Gniazdowski <
> sgniazdowski@gmail.com> ha scritto:
>
>> Are you running zsh 3.0? That would only explain lack of following in
>> zshall:
>>
>>        jobtexts
>>               This associative array maps job numbers to the texts of the
>> command
>>               lines that were used to start the jobs.
>>
>>               Handling  of  the keys of the associative array is as
>> described for
>>               jobdirs above.
>>
>>
>> On Thu, 19 Jan 2023 at 16:08, Pier Paolo Grassi <pierpaolog@gmail.com>
>> wrote:
>>
>>> I'm sorry, what is $jobtexts? it seems an empty variable to me and
>>> doesn't appear in zshall manual
>>>
>>> Pier Paolo Grassi
>>>
>>>
>>> Il giorno gio 19 gen 2023 alle ore 17:02 Sebastian Gniazdowski <
>>> sgniazdowski@gmail.com> ha scritto:
>>>
>>>> You could get pid from $jobtexts?
>>>>
>>>> On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
>>>> wrote:
>>>>
>>>>> I realize now the subject doesn't reflect the final version of the
>>>>> mail, sorry for that
>>>>>
>>>>> Pier Paolo Grassi
>>>>>
>>>>>
>>>>> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
>>>>> pierpaolog@gmail.com> ha scritto:
>>>>>
>>>>>> Hello, I want to populate an array from a long running process,
>>>>>> something like:
>>>>>>
>>>>>> long_running_process | while read line
>>>>>> do
>>>>>>   array+=$line
>>>>>> done
>>>>>>
>>>>>> problem is, since long_running_process does not continually produce
>>>>>> output but only some times, when i try to interrupt this pipeline with
>>>>>> ctrl-c I have to wait until it produces some output for the process to
>>>>>> terminate (because, as I understand it, when it tries to write to the pipe
>>>>>> it receives a sigpipe due to it being already closed)
>>>>>>
>>>>>> I have tried various constructs with trap and even coproc to be able
>>>>>> to intercept the ctrl-c and send a sigpipe to the long running process, but
>>>>>> to no result.
>>>>>>
>>>>>> I wonder if someone has some solution to share for this problem,
>>>>>> thanks
>>>>>>
>>>>>> Pier Paolo Grassi
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Sebastian Gniazdowski
>>>>
>>>>
>>
>> --
>> Best regards,
>> Sebastian Gniazdowski
>>
>>

-- 
Best regards,
Sebastian Gniazdowski

[-- Attachment #2: Type: text/html, Size: 5927 bytes --]

Re: check if coproc has output

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 3371 bytes --]

The pid for the coproc is easily obtained with $!, but when I tried to use
it in a trap I couldn't kill the process, maybe because of the bug Roman is
pointing out to. I will now try to recreate the problem with a simpler
example like Roman asked so it can be better understand
thanks

Pier Paolo Grassi


Il giorno gio 19 gen 2023 alle ore 17:36 Sebastian Gniazdowski <
sgniazdowski@gmail.com> ha scritto:

> I've read your post briefly and saw that you need to terminate the
> background process. I recall that I was doing it in configure-wrapper that
> I once wrote and that I used jobtexts. I do recall that it does contain
> entry for coproc, so you could examine it in a INT trap and send a signal
> to the coproc writing process.
>
> On Thu, 19 Jan 2023 at 16:14, Pier Paolo Grassi <pierpaolog@gmail.com>
> wrote:
>
>> I did again the search in the manual and now I found it, maybe I mistyped
>> the first time
>> where do you suggest I try to print this array?
>>
>> Pier Paolo Grassi
>>
>>
>> Il giorno gio 19 gen 2023 alle ore 17:10 Sebastian Gniazdowski <
>> sgniazdowski@gmail.com> ha scritto:
>>
>>> Are you running zsh 3.0? That would only explain lack of following in
>>> zshall:
>>>
>>>        jobtexts
>>>               This associative array maps job numbers to the texts of
>>> the command
>>>               lines that were used to start the jobs.
>>>
>>>               Handling  of  the keys of the associative array is as
>>> described for
>>>               jobdirs above.
>>>
>>>
>>> On Thu, 19 Jan 2023 at 16:08, Pier Paolo Grassi <pierpaolog@gmail.com>
>>> wrote:
>>>
>>>> I'm sorry, what is $jobtexts? it seems an empty variable to me and
>>>> doesn't appear in zshall manual
>>>>
>>>> Pier Paolo Grassi
>>>>
>>>>
>>>> Il giorno gio 19 gen 2023 alle ore 17:02 Sebastian Gniazdowski <
>>>> sgniazdowski@gmail.com> ha scritto:
>>>>
>>>>> You could get pid from $jobtexts?
>>>>>
>>>>> On Thu, 19 Jan 2023 at 15:43, Pier Paolo Grassi <pierpaolog@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I realize now the subject doesn't reflect the final version of the
>>>>>> mail, sorry for that
>>>>>>
>>>>>> Pier Paolo Grassi
>>>>>>
>>>>>>
>>>>>> Il giorno gio 19 gen 2023 alle ore 16:32 Pier Paolo Grassi <
>>>>>> pierpaolog@gmail.com> ha scritto:
>>>>>>
>>>>>>> Hello, I want to populate an array from a long running process,
>>>>>>> something like:
>>>>>>>
>>>>>>> long_running_process | while read line
>>>>>>> do
>>>>>>>   array+=$line
>>>>>>> done
>>>>>>>
>>>>>>> problem is, since long_running_process does not continually produce
>>>>>>> output but only some times, when i try to interrupt this pipeline with
>>>>>>> ctrl-c I have to wait until it produces some output for the process to
>>>>>>> terminate (because, as I understand it, when it tries to write to the pipe
>>>>>>> it receives a sigpipe due to it being already closed)
>>>>>>>
>>>>>>> I have tried various constructs with trap and even coproc to be able
>>>>>>> to intercept the ctrl-c and send a sigpipe to the long running process, but
>>>>>>> to no result.
>>>>>>>
>>>>>>> I wonder if someone has some solution to share for this problem,
>>>>>>> thanks
>>>>>>>
>>>>>>> Pier Paolo Grassi
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Sebastian Gniazdowski
>>>>>
>>>>>
>>>
>>> --
>>> Best regards,
>>> Sebastian Gniazdowski
>>>
>>>
>
> --
> Best regards,
> Sebastian Gniazdowski
>
>

[-- Attachment #2: Type: text/html, Size: 6850 bytes --]