File descriptor madness

File descriptor madness

[Peter Stephenson]

Playing with ztcp and zselect has raised my awareness of file descriptor
usage to a higher plane.

In particular, getpwuid() on Solaris (2.8) is opening an fd to its new
(so new it's not properly documented) `doors' system as an interface to
NIS.  If I remember right, we've always had some sort of problem with
NIS and its use of file descriptors.  In this case it's tying up fd 3,
right in the paltry set of 9 fd's we have available.  It never releases
it.  (endpwent() doesn't help, by the way, in case you thought it
might.)

One solution to this problem is to open (or dup something such as fd 0
onto) all fd's 3 to 9 during initialisation and then close them again
before any user code is executed (keeping careful track of any that were
already open).  It's not particularly efficient, but not *that*
inefficient either.

Any comments?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: File descriptor madness

[Danek Duvall]

Just checked the source for this, and it doesn't look like this is
scheduled for a change any time soon; this behavior is on purpose.

That said, you may be able to get away with closing the newly opened
file descriptor afterwards, but I think it would be better to
artificially force getpwuid() to use a higher fd, or perhaps dup2() to
move it after the call.

I'm sure it's been explained before, but why does the limit of 9 fds
exist?  Is it just because of the limitations in redirection constructs?

Danek

Re: File descriptor madness

[Bart Schaefer]

On Thu, 9 May 2002, Peter Stephenson wrote:

> In this case it's tying up fd 3, right in the paltry set of 9 fd's we
> have available. [...]
> One solution to this problem is to open (or dup something such as fd 0
> onto) all fd's 3 to 9 during initialisation and then close them again
> before any user code is executed

It'd have to be an actual open() or the equivalent, because there's no way
to guarantee that any particular descriptor (even 0) is valid when the
shell starts up.

What's the cheapest/fastest way to create descriptors -- pipe(), perhaps?

On Thu, 9 May 2002, Danek Duvall wrote:

> That said, you may be able to get away with closing the newly opened
> file descriptor afterwards, but I think it would be better to
> artificially force getpwuid() to use a higher fd, or perhaps dup2() to
> move it after the call.

Moving it after the call won't help, as something in the library almost
certainly will still be referring to the old number.

> I'm sure it's been explained before, but why does the limit of 9 fds
> exist?  Is it just because of the limitations in redirection constructs?

It's entirely because of limitations in the redirection syntax.  Really,
there isn't any limit of 10 (0 through 9) -- this works fine if fd 12 is
already open for writing:

	echo twelve >&12

However, only a single digit to the *left* of the redirect operator can be
parsed as an fd number without breaking sh syntax.

Re: File descriptor madness

[Peter Stephenson]

Bart Schaefer wrote:
> On Thu, 9 May 2002, Peter Stephenson wrote:
> 
> > In this case it's tying up fd 3, right in the paltry set of 9 fd's we
> > have available. [...]
> > One solution to this problem is to open (or dup something such as fd 0
> > onto) all fd's 3 to 9 during initialisation and then close them again
> > before any user code is executed
> 
> It'd have to be an actual open() or the equivalent, because there's no way
> to guarantee that any particular descriptor (even 0) is valid when the
> shell starts up.

Apart from 0, which I suppose we need to test with open(), I had in mind
just using dup() to dup 0 and see where the fd ended up.  This would in
one go show open fd's (avoided) and open new fd's to block them.  Or
doesn't that work?

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: File descriptor madness

[Bart Schaefer]

On Thu, 9 May 2002, Peter Stephenson wrote:

> Apart from 0, which I suppose we need to test with open(), I had in mind
> just using dup() to dup 0 and see where the fd ended up.  This would in
> one go show open fd's (avoided) and open new fd's to block them.  Or
> doesn't that work?

Just one dup() will only show you where the first hole in the sequence of
fd numbers is, not whether there are any more descriptors open above that.
So you'd have to do (psuedo-code)

	if (!is_a_valid_fd(0)) { was_closed[0] = open(...); }
	while ((i = dup(0)) < 10) { was_closed[i] = 1; }
	/* ... initialize the shell ... */
	for (i = 0; i < 10; ++i) { if (was_closed[i]) close(i); }

I was merely wondering aloud whether it would be faster to do

	for (i = 0; !pipe(&just_opened[i]); ++i) {
	  if (just_opened[++i]) >= 9) break;
	}
	/* ... initialize the shell ... */
	while (i >= 0) { close(just_opened[i--]); }

Replace pipe() with something else if there's a faster way to create new
descriptors.  It's possible/probable that dup() is a lot faster, but it'd
be nice not to need to hit the filesystem for that initial open(...) in
the first example.

Re: File descriptor madness

[Peter Stephenson]

Bart Schaefer wrote:
> On Thu, 9 May 2002, Peter Stephenson wrote:
> 
> > Apart from 0, which I suppose we need to test with open(), I had in mind
> > just using dup() to dup 0 and see where the fd ended up.  This would in
> > one go show open fd's (avoided) and open new fd's to block them.  Or
> > doesn't that work?
> 
> Just one dup() will only show you where the first hole in the sequence of
> fd numbers is, not whether there are any more descriptors open above that.

Yes, I thought of that.  Here's an attempt.  It seems to work, according
to lsof.

I can't think of a good test of whether it's necessary, though.  Perhaps
there are enough systems potentially using NIS or something similar that
it's always reasonable.

Index: Src/init.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/init.c,v
retrieving revision 1.22
diff -u -r1.22 init.c
--- Src/init.c	6 May 2002 14:50:13 -0000	1.22
+++ Src/init.c	10 May 2002 17:16:11 -0000
@@ -613,14 +613,11 @@
 #endif
     struct timezone dummy_tz;
     char *ptr;
-#ifdef HAVE_GETRLIMIT
-    int i;
-#endif
+    int i, j;
 #if defined(SITEFPATH_DIR) || defined(FPATH_DIR)
     char **fpathptr;
 # if defined(FPATH_DIR) && defined(FPATH_SUBDIRS)
     char *fpath_subdirs[] = FPATH_SUBDIRS;
-    int j;
 # endif
 # ifdef SITEFPATH_DIR
     int fpathlen = 1;
@@ -628,6 +625,43 @@
     int fpathlen = 0;
 # endif
 #endif
+    int close_fds[10], tmppipe[2];
+
+    /*
+     * Workaround a problem with NIS (in one guise or another) which
+     * grabs file descriptors and keeps them for future reference.
+     * We don't want these to be in the range where the user can
+     * open fd's, i.e. 0 to 9 inclusive.  So we make sure all
+     * fd's in that range are in use.
+     */
+    memset(close_fds, 0, 10*sizeof(int));
+    if (pipe(tmppipe) == 0) {
+	/*
+	 * Strategy:  Make sure we have at least fd 0 open (hence
+	 * the pipe).  From then on, keep dup'ing until we are
+	 * up to 9.  If we go over the top, close immediately, else
+	 * mark for later closure.
+	 */
+	i = -1;			/* max fd we have checked */
+	while (i < 9) {
+	    /* j is current fd */
+	    if (i < tmppipe[0])
+		j = tmppipe[0];
+	    else if (i < tmppipe[1])
+		j = tmppipe[1];
+	    else {
+		j = dup(0);
+		if (j == -1)
+		    break;
+	    }
+	    if (j < 10)
+		close_fds[j] = 1;
+	    else
+		close(j);
+	    if (i < j)
+		i = j;
+	}
+    }
 
     addhookdefs(argzero, zshhooks, sizeof(zshhooks)/sizeof(*zshhooks));
 
@@ -813,6 +847,11 @@
     }
 
     times(&shtms);
+
+    /* Close the file descriptors we opened to block off 0 to 9 */
+    for (i = 0; i < 10; i++)
+	if (close_fds[i])
+	    close(i);
 }
 
 /* Initialize signal handling */

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: File descriptor madness

[Peter Stephenson]

Peter Stephenson wrote:
> Yes, I thought of that.  Here's an attempt.  It seems to work, according
> to lsof.

I missed the case where all 10 fd's are already used, in which case the
pipe's descriptors leak.  Add

	if (i < tmppipe[0])
	    close(tmppipe[0]);
	if (i < tmppipe[1])
	    close(tmppipe[1]);

and stir.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: File descriptor madness

[Peter Stephenson]

Peter Stephenson wrote:
> Peter Stephenson wrote:
> > Yes, I thought of that.  Here's an attempt.  It seems to work, according
> > to lsof.
> 
> I missed the case where all 10 fd's are already used, in which case the
> pipe's descriptors leak.

I've committed these two changes.  If anyone notices odd behaviour on
start-up or with file descriptors this can be reviewed.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Slightly OT Re: File descriptor madness

[Borsenkow Andrej]

В Птн, 10.05.2002, в 21:24, Peter Stephenson написал:
> +    memset(close_fds, 0, 10*sizeof(int));

Do we really assume memset is unconditionally available? Well, judging
by the fact there were no bug reports it must be on all modern systems.

-andrej