From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/2920 Path: news.gmane.org!not-for-mail From: Zvi Gilboa Newsgroups: gmane.linux.lib.musl.general Subject: Re: question: hard-coded file descriptors in stdin/stdout/stderr Date: Fri, 15 Mar 2013 10:46:54 -0400 Message-ID: <5143345E.4080209@eservices.virginia.edu> References: <5141F86D.8010000@eservices.virginia.edu> <20130314171752.GB19010@port70.net> <51420E17.9030305@eservices.virginia.edu> <20130315083357.GH20323@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1363358833 14971 80.91.229.3 (15 Mar 2013 14:47:13 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 15 Mar 2013 14:47:13 +0000 (UTC) To: Original-X-From: musl-return-2921-gllmg-musl=m.gmane.org@lists.openwall.com Fri Mar 15 15:47:35 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1UGVuw-0005DI-5u for gllmg-musl@plane.gmane.org; Fri, 15 Mar 2013 15:47:34 +0100 Original-Received: (qmail 22134 invoked by uid 550); 15 Mar 2013 14:47:08 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 22126 invoked from network); 15 Mar 2013 14:47:07 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130308 Thunderbird/17.0.4 In-Reply-To: <20130315083357.GH20323@brightrain.aerifal.cx> X-Originating-IP: [68.229.98.213] Xref: news.gmane.org gmane.linux.lib.musl.general:2920 Archived-At: >> [Rich wrote:] Not only are the numbers 0/1/2 specified and widely hard-coded in applications; >> [LM wrote:] I've been trying to port msh to Windows and it uses hard-coded 0, 1 and 2 with pipes and expects them to mean stdin, stdout and stderr. Thank you for this valuable input! Clearly, remapping 0/1/2 from within psxcalls would be the only responsible approach:) >> [LM wrote, regarding the absence of fork() on Windows:] That's pretty much what I do when I port applications to Windows; I substitute spawn. Seems easier to replace vfork than fork that way though. This seems to be the common in-lack-of-a-better-solution approach... but I'm sure you'd agree it is both inaccurate and inconvenient, not to mention all those file handles and .data and .bss variables that have to be manually synchronized... >> [LM wrote:] One other issue I hit when porting to Windows is the lack of certain signal handling capabilities. For instance, when attempting to port flrec, in place of kill(pid_sox,SIGSTOP);, I'm trying to use NtSuspendProcess and in place of SIGCONT, I'm trying to use NtResumeProcess. That kind of "translation" is an essential part of the psxcalls project. Surprisingly (or not), the vast majority of differences between the Native API and POSIX are about syntax, not functionality. In other words, there exist only very few POSIX features that cannot be implemented using the Native API in a straight-forward way. There are for sure some cases that require programming acrobatics (fork() and exec()) being the most obvious examples), but that's the exception, not the rule. Whenever I implement one of the functions, I try to phrase the task in the form of a question, for instance: "using the Native API, how would you map a memory page to the address space of a user applications?" That kind of approach to writing the library surely entails a lot of work, but is also very exciting -- like solving a puzzle every single day... On 03/15/2013 04:33 AM, Rich Felker wrote: > On Thu, Mar 14, 2013 at 01:51:19PM -0400, Zvi Gilboa wrote: >>>> which ones? >> .... since you are asking... inspired by musl-libc, I am currently >> writing a win32/win64 open-source library that implements/provides >> POSIX system calls (see note below). I believe that having a >> powerful libc with an MIT license available on Windows would >> actually be of great value to the open source community for all >> possible reasons, but that is of course irrelevant to my question:) > This is actually something I've wanted to see done for a long time, so > I wish you the greatest success on your project. Despite my general > aversion to using the STDIN_FILENO etc. macros, if your project takes > off and this issue is a major obstacle to pulling up-to-date code from > musl, I would probably consider just using them. But... > >> The main issue here is that the standard file descriptors on Windows >> are -10 (STD_INPUT_HANDLE), -11 (STD_OUTPUT_HANDLE), and -12 >> (STD_ERROR_HANDLE). I could of course compensate for that in my >> code and "translate" the POSIX special file descriptor numbers to >> the Windows ones, however it would be more elegant if musl-libc used >> the descriptors defined in instead of hard-coded >> numbers. > When I originally proposed an idea similar to what you're doing, one > of the open-ended questions was "how much" of POSIX such a libc (for > Windows) should aim for. Since we weren't actually doing the project, > a lot of questions were left unanswered, but I think this sort of file > descriptor remapping is something you really should do. Not only are > the numbers 0/1/2 specified and widely hard-coded in applications; the > Windows "file handle" values for stdin/out/err are not usable as file > descriptors because they are negative. They definitely aren't > compatible with select(), and probably have a lot of other issues > where they clash with the interpretation of a negative return value as > an error (e.g. dup2(fd,-10) would return -10, which applications would > interpret as an error). If you want to make sockets and regular file > file descriptors a common space (which is needed for close() to work, > as well as select()/poll()) I think that also requires a remapping of > windows file/socket handles to POSIX-style file descriptor numbers. > >> * as for psxcalls: this is a C library, that exclusively uses the >> Native API. It attaches to ntdll.dll during run-time, and can thus >> be compiled as a "native" static library with no external >> dependencies. While it is currently in its very initial stage (and >> not yet online), the major "obstacle" features -- including fork() >> -- have already been implemented. To remove all doubts, I am aware >> of Cygwin's existence, yet am looking for a high-performance >> solution that would be both "clean" >> (psxcalls-->libc-->user-library-or-application), and flexibly >> licensed. > Nice to hear. Incidentally, fork() was one of the interfaces I thought > we could just sacrifice in doing such a libc (posix_spawn or the > similar Windows function can replace most correct uses of fork()), so > it's nice to hear you've solved the problem. > > Rich