Github messages for mblaze
 help / color / mirror / Atom feed
* [ISSUE] mpick: multi redir test hangs on Alpine
@ 2021-03-13 19:27 nmeum
  2021-03-13 21:07 ` Duncaen
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: nmeum @ 2021-03-13 19:27 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1733 bytes --]

New issue by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203

Description:
While trying to get the mblaze test suite running on Alpine Linux, I noticed that the `multi redir` test from `2000-mpick.t` does not terminate, even with current git HEAD. To reproduce:

```
$ cat /tmp/input
inbox/cur/1:2,S
inbox/cur/2:2,ST
inbox/cur/3:2,SRT
inbox/cur/4:2,SRFT
inbox/cur/5:2,T
inbox/cur/6:2,SRF
inbox/cur/7:2,SR
inbox/cur/8:2,SF
inbox/cur/9:2,
$ cat expr
let foo = from.addr == "peter@example.org"
let bar = from.disp == "Peter Example"
in
  foo |"sed ""s/^/1:&/""" && bar |"sed ""s/^/2:&/""" && skip
$ mpick -F ./expr < /tmp/input
1:inbox/cur/1:2,S
1:inbox/cur/9:2,
2:inbox/cur/1:2,S
2:inbox/cur/9:2,
<process does not terminate>
```

I debugged this a bit with `strace(1)`. The output of the 3 processes (mpick, sed, and sed) is available here:

* [sed.25493.txt](https://github.com/leahneukirchen/mblaze/files/6135400/sed.25493.txt)
* [sed.25494.txt](https://github.com/leahneukirchen/mblaze/files/6135401/sed.25494.txt)
* [sed.25495.txt](https://github.com/leahneukirchen/mblaze/files/6135402/sed.25495.txt)

I am a bit confused as to what is happening here. The mpick process (PID 25493) blocks because it waits for the first sed process (PID 25494) to terminate (since mpick uses `pclose(3)`). The sed process on the other hand doesn't terminate because it is still reading from the reading end of the pipe. However, I would expect that read to return with an EOF because the mpick process already closed the writing end of the pipe. Yet for some reason, it is stuck in a read system call. Maybe I am missing something obvious here, any idea what might be causing this?

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
@ 2021-03-13 21:07 ` Duncaen
  2021-03-13 21:08 ` Duncaen
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Duncaen @ 2021-03-13 21:07 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

New comment by Duncaen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-798785916

Comment:
This looks like a bug in musl, the second popen'ed process ends up inheriting the writing end of the first popen.

> The popen() function shall ensure that any streams from previous popen() calls that remain open in the parent process are closed in the new child process.
https://pubs.opengroup.org/onlinepubs/9699919799/

```
echo "Hello\nWorld" | extrace busybox awk '{print $0 |"cat -n"; print $0 |"cat  -n"}'
```

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
  2021-03-13 21:07 ` Duncaen
@ 2021-03-13 21:08 ` Duncaen
  2021-03-13 21:40 ` nmeum
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Duncaen @ 2021-03-13 21:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 546 bytes --]

New comment by Duncaen on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-798785916

Comment:
This looks like a bug in musl, the second popen'ed process ends up inheriting the writing end of the first popen.

> The popen() function shall ensure that any streams from previous popen() calls that remain open in the parent process are closed in the new child process.
https://pubs.opengroup.org/onlinepubs/9699919799/

```
echo "Hello\nWorld" | busybox awk '{print $0 |"cat -n"; print $0 |"cat  -n"}'
```

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
  2021-03-13 21:07 ` Duncaen
  2021-03-13 21:08 ` Duncaen
@ 2021-03-13 21:40 ` nmeum
  2021-03-14 16:08 ` nmeum
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: nmeum @ 2021-03-13 21:40 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

New comment by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-798790009

Comment:
Ah, indeed. That would also explain why closing the pipes in reverse order fixes it. And fd 4 does also show up in `/proc/259495/fd`. I will report this upstream.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
                   ` (2 preceding siblings ...)
  2021-03-13 21:40 ` nmeum
@ 2021-03-14 16:08 ` nmeum
  2021-06-25 23:06 ` nmeum
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: nmeum @ 2021-03-14 16:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 211 bytes --]

New comment by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-798933222

Comment:
> I will report this upstream.

https://www.openwall.com/lists/musl/2021/03/14/1

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ISSUE] [CLOSED] mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
                   ` (4 preceding siblings ...)
  2021-06-25 23:06 ` nmeum
@ 2021-06-25 23:06 ` nmeum
  2021-06-25 23:06 ` nmeum
  6 siblings, 0 replies; 8+ messages in thread
From: nmeum @ 2021-06-25 23:06 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

Closed issue by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203

Description:
While trying to get the mblaze test suite running on Alpine Linux, I noticed that the `multi redir` test from `2000-mpick.t` does not terminate, even with current git HEAD. To reproduce:

```
$ cat /tmp/input
inbox/cur/1:2,S
inbox/cur/2:2,ST
inbox/cur/3:2,SRT
inbox/cur/4:2,SRFT
inbox/cur/5:2,T
inbox/cur/6:2,SRF
inbox/cur/7:2,SR
inbox/cur/8:2,SF
inbox/cur/9:2,
$ cat expr
let foo = from.addr == "peter@example.org"
let bar = from.disp == "Peter Example"
in
  foo |"sed ""s/^/1:&/""" && bar |"sed ""s/^/2:&/""" && skip
$ mpick -F ./expr < /tmp/input
1:inbox/cur/1:2,S
1:inbox/cur/9:2,
2:inbox/cur/1:2,S
2:inbox/cur/9:2,
<process does not terminate>
```

I debugged this a bit with `strace(1)`. The output of the 3 processes (mpick, sed, and sed) is available here:

* [sed.25493.txt](https://github.com/leahneukirchen/mblaze/files/6135400/sed.25493.txt)
* [sed.25494.txt](https://github.com/leahneukirchen/mblaze/files/6135401/sed.25494.txt)
* [sed.25495.txt](https://github.com/leahneukirchen/mblaze/files/6135402/sed.25495.txt)

I am a bit confused as to what is happening here. The mpick process (PID 25493) blocks because it waits for the first sed process (PID 25494) to terminate (since mpick uses `pclose(3)`). The sed process on the other hand doesn't terminate because it is still reading from the reading end of the pipe. However, I would expect that read to return with an EOF because the mpick process already closed the writing end of the pipe (fd 4). Yet for some reason, it is stuck in a read system call. Maybe I am missing something obvious here, any idea what might be causing this?

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
                   ` (3 preceding siblings ...)
  2021-03-14 16:08 ` nmeum
@ 2021-06-25 23:06 ` nmeum
  2021-06-25 23:06 ` [ISSUE] [CLOSED] " nmeum
  2021-06-25 23:06 ` nmeum
  6 siblings, 0 replies; 8+ messages in thread
From: nmeum @ 2021-06-25 23:06 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

New comment by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-868875311

Comment:
This seems to have been fixed recently on the master branch https://git.musl-libc.org/cgit/musl/commit/?id=e1a51185ceb4386481491e11f6dd39569b9e54f7 As such, the mblaze tests should pass with the upcoming musl release. I think we can close this now.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: mpick: multi redir test hangs on Alpine
  2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
                   ` (5 preceding siblings ...)
  2021-06-25 23:06 ` [ISSUE] [CLOSED] " nmeum
@ 2021-06-25 23:06 ` nmeum
  6 siblings, 0 replies; 8+ messages in thread
From: nmeum @ 2021-06-25 23:06 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

New comment by nmeum on mblaze repository

https://github.com/leahneukirchen/mblaze/issues/203#issuecomment-868875311

Comment:
This seems to have been fixed recently on the musl master branch https://git.musl-libc.org/cgit/musl/commit/?id=e1a51185ceb4386481491e11f6dd39569b9e54f7 As such, the mblaze tests should pass with the upcoming musl release. I think we can close this now.

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-06-25 23:06 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-13 19:27 [ISSUE] mpick: multi redir test hangs on Alpine nmeum
2021-03-13 21:07 ` Duncaen
2021-03-13 21:08 ` Duncaen
2021-03-13 21:40 ` nmeum
2021-03-14 16:08 ` nmeum
2021-06-25 23:06 ` nmeum
2021-06-25 23:06 ` [ISSUE] [CLOSED] " nmeum
2021-06-25 23:06 ` nmeum

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).