Combination of pipes and &! leaks file descriptors

Combination of pipes and &! leaks file descriptors

[Jure Varlec]

[-- Attachment #1.1: Type: text/plain, Size: 542 bytes --]

Hello,

I noticed a file descriptor leak today in a script that was forking
jobs in a loop and ceased to work after 1000 jobs. A (quick, admittedly)
search of the mailing list found nothing relevant. A minimal example is
attached, using `lsof` to demonstrate the problem. I also attached its
output in case there are difficulties reproducing the problem. It is
interesting to see that only the `&!` operator causes the problem, the
`disown` command does not. A pipe needs to be present in the
backgrounded command.

Regards
Jure

[-- Attachment #1.2: forkleak.sh --]
[-- Type: application/x-shellscript, Size: 422 bytes --]

[-- Attachment #1.3: Type: text/plain, Size: 1898 bytes --]

Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=forkleak.out

V2l0aCB0aGVgZGlzb3duYCBrZXl3b3JkOgoxYzEKPCBmb3JrbGVhay4gMjcwNjYgZXh6b21iaWUg
ICAxN3IgIEZJRk8gICAgICAgICAgICAgICAwLDEwICAgICAgMHQwICAxOTg4ODE5IHBpcGUgMjcw
NjcsZm9ya2xlYWsuLDF3IDI3MDY5LGdyZXAsMXcKLS0tCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9t
YmllICAgMTdyICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4OTA5MSBwaXBl
IDI3MDkyLGZvcmtsZWFrLiwxdyAyNzA5NCxncmVwLDF3CgpXaXRoIHRoZSBgJiFgIG9wZXJhdG9y
OgoxYzEsMTEKPCBmb3JrbGVhay4gMjcwNjYgZXh6b21iaWUgICAxN3IgIEZJRk8gICAgICAgICAg
ICAgICAwLDEwICAgICAgMHQwICAxOTg4ODE5IHBpcGUgMjcwNjcsZm9ya2xlYWsuLDF3IDI3MDY5
LGdyZXAsMXcKLS0tCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMTdyICBGSUZPICAgICAg
ICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4MzE4NiBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4
em9tYmllICAgMThyICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4ODAxMCBw
aXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMTlyICBGSUZPICAgICAgICAgICAgICAg
MCwxMCAgICAgIDB0MCAgMTk4ODAxMyBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAg
MjByICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4ODAxNiBwaXBlCj4gZm9y
a2xlYWsuIDI3MDY2IGV4em9tYmllICAgMjFyICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAg
IDB0MCAgMTk4ODAxOSBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMjJyICBGSUZP
ICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4ODAyMiBwaXBlCj4gZm9ya2xlYWsuIDI3
MDY2IGV4em9tYmllICAgMjNyICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4
ODAyNSBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMjRyICBGSUZPICAgICAgICAg
ICAgICAgMCwxMCAgICAgIDB0MCAgMTk4MzE4OSBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9t
YmllICAgMjVyICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4MzE5MSBwaXBl
Cj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMjZyICBGSUZPICAgICAgICAgICAgICAgMCwx
MCAgICAgIDB0MCAgMTk4ODAzMCBwaXBlCj4gZm9ya2xlYWsuIDI3MDY2IGV4em9tYmllICAgMjdy
ICBGSUZPICAgICAgICAgICAgICAgMCwxMCAgICAgIDB0MCAgMTk4MzI0MiBwaXBlIDI3MTE3LGZv
cmtsZWFrLiwxdyAyNzExOSxncmVwLDF3Cg==

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Combination of pipes and &! leaks file descriptors

[Peter Stephenson]

On Tue, 2 May 2017 16:19:25 +0200
Jure Varlec <jure@varlec.si> wrote:
> I noticed a file descriptor leak today in a script that was forking
> jobs in a loop and ceased to work after 1000 jobs. A (quick, admittedly)
> search of the mailing list found nothing relevant. A minimal example is
> attached, using `lsof` to demonstrate the problem. I also attached its
> output in case there are difficulties reproducing the problem. It is
> interesting to see that only the `&!` operator causes the problem, the
> `disown` command does not. A pipe needs to be present in the
> backgrounded command.

Thanks, this made it easy to track down.

pws

diff --git a/Src/exec.c b/Src/exec.c
index 9a75dd2..debb0ae 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -1612,6 +1612,7 @@ execpline(Estate state, wordcode slcode, int how, int last1)
 	    zclose(opipe[0]);
 	}
 	if (how & Z_DISOWN) {
+	    pipecleanfilelist(jobtab[thisjob].filelist, 0);
 	    deletejob(jobtab + thisjob, 1);
 	    thisjob = -1;
 	}