From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11113 invoked from network); 6 Jun 2006 22:50:39 -0000 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,FORGED_RCVD_HELO autolearn=ham version=3.1.3 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 6 Jun 2006 22:50:39 -0000 Received: (qmail 83438 invoked from network); 6 Jun 2006 22:50:32 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 6 Jun 2006 22:50:32 -0000 Received: (qmail 17951 invoked by alias); 6 Jun 2006 22:50:24 -0000 Mailing-List: contact zsh-users-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 10354 Received: (qmail 17940 invoked from network); 6 Jun 2006 22:50:22 -0000 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by sunsite.dk with SMTP; 6 Jun 2006 22:50:22 -0000 Received: (qmail 82353 invoked from network); 6 Jun 2006 22:50:21 -0000 Received: from mta08-winn.ispmail.ntl.com (HELO mtaout02-winn.ispmail.ntl.com) (81.103.221.48) by a.mx.sunsite.dk with SMTP; 6 Jun 2006 22:50:20 -0000 Received: from aamtaout03-winn.ispmail.ntl.com ([81.103.221.35]) by mtaout02-winn.ispmail.ntl.com with ESMTP id <20060606225019.MRRJ29040.mtaout02-winn.ispmail.ntl.com@aamtaout03-winn.ispmail.ntl.com> for ; Tue, 6 Jun 2006 23:50:19 +0100 Received: from pwslaptop.csr.com ([81.107.41.155]) by aamtaout03-winn.ispmail.ntl.com with ESMTP id <20060606225018.SDYN16286.aamtaout03-winn.ispmail.ntl.com@pwslaptop.csr.com> for ; Tue, 6 Jun 2006 23:50:18 +0100 Received: from pwslaptop.csr.com (pwslaptop.csr.com [127.0.0.1]) by pwslaptop.csr.com (8.13.6/8.13.4) with ESMTP id k56Mo5cS001920; Tue, 6 Jun 2006 23:50:05 +0100 Message-Id: <200606062250.k56Mo5cS001920@pwslaptop.csr.com> From: Peter Stephenson To: ZSH Users , pws@pwslaptop.csr.com Subject: Re: Writing Interpreter Testing Framework In-Reply-To: Message from "Johann 'Myrkraverk' Oskarsson" of "Tue, 06 Jun 2006 21:51:10 -0000." Date: Tue, 06 Jun 2006 23:50:05 +0100 "Johann 'Myrkraverk' Oskarsson" wrote: > For the sake of argument, let's say I'm testing an interpreter* which > basically means I want to run an application, feed it some commands > from a file to STDIN, and depending on the results on its STDOUT, I > want to feed some other file. > > That is, something like the following interactive session: > > % interpreter < file_of_commands_A > Some output from interpreter: B > % interpreter < file_of_commands_A < file_of_commands_B > > Though I'm not excacly sure the above will work, and, if possible, I'd > like to skip running the interpreter again, to preserve all possible > random-set states. I'm assuming you're interested in an interactive zsh way of doing this, since you've asked the question here. There are lots of script based ways from expect to Perl libraries designed to do this which I will ignore; but if you're really interested in robust scripts and don't care about interactive use, that's probably a better way to go. I'd recommend Perl or another "real" language of your choice rather than expect, since TCL is otherwise rather limited. (Our test engineer has just started writing scripts in Perl rather than expect and apparently it all works much better. I'm sure you can do it in Python, too.) The standard zsh/ksh way of doing this is with coprocesses. Here's a trivial example with an "interpreter" that is just a simple subshell loop that modifies the input and spits it out. I've stripped intermediate prompts for clarity: % coproc (while read line; do print -r "You said: $line" done) % print this is a message >&p % read input <&p % print $input You said: this is a message If you need the "interpreter" to run in a (pseudo)terminal, investigate the zpty command provided by the zsh/zpty module and documented in the zshmodules manual page. This works similarly to coprocesses, although the syntax is different. If you need the output to go straight to the (real) terminal, you need to be a bit cleverer. You can get some of the way by using some intermediate stage for the input, for example a named pipe: % mkfifo ~/tmp/fifo % (while true; do; while read line; do print -r "You said: $line" done <~/tmp/fifo done) & % print Hello >~/tmp/fifo You said: Hello % print Goodbye >~/tmp/fifo You said: Goodbye There are two subtleties here. First, the double "while" loop. That's because the one-off prints to the FIFO cause an end-of-file, which terminates the inner loop. You'd need to take account of that in any interpreter. Second, the output from the background process is asynchronous and the shell isn't expecting it, so although I showed the output as neat lines between the shell intput you won't actually get that. If you still want better interaction with the shell, you might want to consider using the TCP framework supplied with the shell and documented in the zshtcpsys manual entry. It works most naturally if your interpreter is running on a TCP port. See below for a simple-minded function that does this: it's not that hard to knock up a daemon to wrap an interpreter that does that, though you'd better make sure you have a good firewall if you go this route. However, the system allows you to come in at a lower level. For example: # Make two FIFOs: we need that because FIFOs get confused if two # processes read from the same one. That's one advantage of a real # TCP solution: you can use one file descriptor with no problem. % mkfifo ~/tmp/infifo % mkfifo ~/tmp/outfifo # Start the "interpreter". Note we don't need any hacky extra loop this # time. % (while read line; do print -r "You said: $line" done <~/tmp/infifo >~/tmp/outfifo) & # Make fd 5 our output to this interpreter's input. % exec 5>~/tmp/infifo # Open a session called "output" on this fd % tcp_open -f 5 output Session output (fd 5) opened OK. Setting default TCP session output # Make fd 4 our input from the interpreter's output. % exec 4<~/tmp/outfifo # Open a session called "input" on this fd % tcp_open -f 4 input Session input (fd 4) opened OK. Now comes the good bit: % tcp_send Hello <-[input] You said: Hello (You might see an empty line before the reply comes back if the line editor starts up again before the reply is ready, but the display will still be cleared for the output.) tcp_send sends output to the default session which, as you can see from the messages above, is the session "output" which writes to the interpreter's input. When we opened the session "input", the shell automatically supplied the fd 4 to the line editor (using the builtin "zle -F"). Now the shell listens for input not just on the editing terminal, but on that fd. When input comes, it alerts the TCP system, which clears the terminal and displays the input with the prompt "<-[input] " (this is configurable). When you want to interact with the interpreter programmatically, you can write functions like this: fn() { emulate -L zsh setopt localoptions local -a match mbegin mend tcp_send "This is a message" if tcp_expect -s input '*You said: (#b)(*)'; then print -r "It said I said: $match[1]" fi } (Note that because the input and output are different sessions in our FIFO approach you need to tell tcp_expect the session to use explicitly.) You'll see that you still get the raw line echoed; add "local TCP_SILENT=1" to the top of the function to stop that. If you are happy about the security aspect, here's how to use the full majesty of TCP to do this: # We need the interpreter to be a single command... % interpreter() { while read line; do print -r "You said: $line" done } # Start the interpreter running on a random port % tcp_proxy 5114 interpreter & # We are now done with the interpreter... in fact, all the above # could be in a different terminal or even on a different host. # Now start the session "session" to talk to the interpreter % tcp_open localhost 5114 session Session session (host localhost, port 5114 fd 3) opened OK. Setting default TCP session session % tcp_send Hello <-[session] You said: Hello The tcp_expect stuff works as before, though this time you've only got one session to worry about so don't need any "-s" arguments. I use this system all the time at work for interacting with my company's range of digital radio chips and I find it highly flexible. -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/