From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11782 Path: news.gmane.org!.POSTED!not-for-mail From: Denys Vlasenko Newsgroups: gmane.linux.lib.musl.general Subject: Re: shell needs to change fd in a FILE Date: Wed, 2 Aug 2017 14:08:09 +0200 Message-ID: References: <20170731201827.GA1627@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1501675728 11749 195.159.176.226 (2 Aug 2017 12:08:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 2 Aug 2017 12:08:48 +0000 (UTC) To: musl Original-X-From: musl-return-11795-gllmg-musl=m.gmane.org@lists.openwall.com Wed Aug 02 14:08:43 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1dcsS7-0002jn-OH for gllmg-musl@m.gmane.org; Wed, 02 Aug 2017 14:08:39 +0200 Original-Received: (qmail 10140 invoked by uid 550); 2 Aug 2017 12:08:42 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 10119 invoked from network); 2 Aug 2017 12:08:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=y+Gm7BIsGZ9e9TmU+PzNs5BP68ti7jFGO30qQsH2U78=; b=UhCRNI2TEcx9e3BBzTd4vUa68MzbnFGBvmflk74v7YHW5HXlJlr6s1OasYfgHJsdPI dLES7J+E9S6fEkVi1g0yRfNRNFhBJXCMAaLftAaUEhALF6hpjNqz0VOEnGBL/9iygZTP E5Dxb0YwzJPG4m6DpsUt170aCL407lqYnp0DcjjlOZXmZyKhCJOahpFqaa67IVzgnuUF yQyHRYdvvH+k6vg5+1eLTQGlXqJ5YdHcqw89FFvacHFJg5HqOxyD2YHdwGvLpaNDpv1X edf9i+hoPb8HIbQPUX5YSyDpqxewFP7qylzZY4mz58kSWPnUtr4V0TA9D8ooijIqVUxM dfRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=y+Gm7BIsGZ9e9TmU+PzNs5BP68ti7jFGO30qQsH2U78=; b=KCCBH+k2u9ZLvyuN13QLdGT+bElJPq+7WU8Sre9V94wsN9xZ83k3/km9kcgggzee1X j79pTZwy0Qe/1idorVKADhco/O5bg5BP2XjTqVE2PSLQM3PvZXWctkoX/gCddC0y6cOF KPD5O0LiR/HShYXQUc1MSMA73KJvBvfvB6GXCahpPY8cAO1t+fY1GP4vz+hVVcQSBV1v QRu0F5WkJiDq4j1NrEJpmWN7qkclqddI9fY6Uh9Scp8gNB7uVLROspWxYTYzRxcQ6A5K 3dy8YW0XeRPI7xU61niTZ0U8c+UBspb541wuvimspTie1SkOJ+mZQKdZAK3P0aXEtE+R fZKQ== X-Gm-Message-State: AIVw111jOmTZaW7ruwCW7FTSufNyAb1FoNON2FHb5kFWUaqTIwz3qpLp /272UwMdxz6RvS+jPFzojfvpuGGLRFJc X-Received: by 10.80.177.228 with SMTP id n33mr20718327edd.145.1501675710296; Wed, 02 Aug 2017 05:08:30 -0700 (PDT) In-Reply-To: <20170731201827.GA1627@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:11782 Archived-At: On Mon, Jul 31, 2017 at 10:18 PM, Rich Felker wrote: > On Mon, Jul 31, 2017 at 05:05:24PM +0200, Denys Vlasenko wrote: >> Hi, >> >> I'm using ordinary FILE i/o for reading scripts in hush shell, >> instead of rolling my own implementation, so that I can reuse >> buffering code in libc, through the use of fgetc(). >> >> This works for almost all cases, except this one: if in script >> I have a redirect which says shell wants to open a fd which happens >> to be equal to the fd (say, 10) shell already used for script FILE: >> >> exec 10>FILE >> >> What other shells do in this situation is they simply >> dup and close script fd [in real code, they use fcntl(F_DUPFD) >> instead of dup() since they want to avoid getting low fds], >> so that fd is "moved" and no longer collides with the redirect. > > The right solution to this problem is not to actually reassign fds in > the shell process at all, but instead reassign them in the child > process to match their nominal (per the redirection operator) value. > This is easy with fork+exec, hard or impossible to do safely with > vfork+exec, and easy with posix_spawn. > > For commands which are internal (no child process), then, the nominal > fd number is not the actual fd number but it doesn't matter; the > internal logic can just remap it. "Just remap it" is actually quite a PITA in this case. This means "can't use standard functions emitting messages to stderr, because my stderr in builtins may be not on fd 2 but somewhere else". Would you enjoy this turn of events? It's not like shell logic is such an easy piece of code before you add this fd remapping thing. In busybox, echo, printf, kill, test shell builtins reuse code with standalone (non-shell) tools. Now all of those also can't use stdin and stderr as usual, they need to account for the possibility of remapped fds. Apart from builtins, busubox has these optional so-called NOFORK applets: they won't fork/exec/waitpid when run by shell, they simply call _main(). There are 25 NOFORKs. For example, printenv, pwd, whoami - these all output stuff to stdout. Many of them also can emit error messages. Now all of them also can't use stdin and stderr with usual functions. This is too WAY too much uglification. > Not saying you have to do it this way, but it's the clean (and the > only strictly-conforming, since POSIX allows implementation-internal > fd use that you *can't* safely move). For the functions used in shell, there is no reason for libc to use hidden fds. Shells don't use openlog() or DNS resolution functions. So this is only a theoretical thing to worry in this case. >> I can do this trick, but since I use FILE interface, then >> I need to inform libc that it needs to use new fd for this FILE. >> >> "fileno(fp) = new_fd;" is non-portable and does not work in either >> musl or glibc: it's a function, not a macro referencing >> (fp)->field_holding_fd. >> >> "fclose(fp); fp = fdopen(new_fd);" is not good since fp may have >> some buffered input, which will be lost by such code. > > I don't see how this is a problem unless you can read scripts from a > non-seekable stream, which sounds really dubious. I just tried: "bash /dev/tty" and "echo uname | bash /dev/fd/0" works. I prefer to not needlessly prevent my shell from being able to do that too. >> How about adding a "set_fileno(fp, fd)" extension to musl, >> with some easy define to probe for to conditionally use it? > > I really don't want to get into the business of doing > application-specific stdio extensions on request. We did a couple for > gnulib, but that's because it's already crept into everything and the > alternative was them doing awful hacks trying to poke at stdio > internals. I think we can come up with a better solution here. One easy solution is to ditch FILE and use my own buffering. ash/dash already did exactly that many years ago, for this very reason. This would mean that libc has some 25 kb of really good, well-debugged I/O buffering code which I am not using because it misses just one (!) trivial operation in the API, fp->fd = new_fd. This was the thought which prompted me to write the email: by seeing how much this would help, you might agree.