Fwd: input redirect from a variable

Fwd: input redirect from a variable

[Peter Stephenson]

------- Forwarded Message

Mime-Version: 1.0
Message-Id: <pdy98830623095bbf58c3401e47@[192.168.1.2]>
Date: Thu, 22 Sep 2005 13:30:00 -0700
To: Peter Stephenson <pws@csr.com>
From: Dave Yost <Dave@Yost.com>
Subject: input redirect from a variable
Content-Type: text/plain; charset="us-ascii" ; format="flowed"

Hi.

It seems to me that there should be a way to do something like this:

foo1="$(...)"
foo2="$(...)"

comm -3 <<<<$foo1 <<<<$foo2

where the <<<$foo1 syntax says to output $foo1 to a tmp file, then 
use that filenamne as the argument, then delete that file.

Thanks

Dave


 To report this email as spam click https://www.mailcontrol.com/sr/dV5V1+NNvfz0l6E4pUhSMFqZ4C4toP8iu3W+wAq8XC!nhDpvbkN9LMBQa+TOk!M04e16bR4uI6G!jl5aNChCqde!Gqru15OTP1ncoqn8kbiVYaW8HVlGUtBYkbssd1PnpA4zSIG5OEHHXnBgUQNc1DDc7H2KqzK8bLzMluGArFap!K!fvkhzBLQDcu6!oYIymNRxq!w9HD4fHNgmFGL4XWvOCwpikgbX .

------- End of Forwarded Message



**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: input redirect from a variable

[Mikael Magnusson]

On 9/23/05, Peter Stephenson <pws@csr.com> wrote:
>
> ------- Forwarded Message
>
> Mime-Version: 1.0
> Message-Id: <pdy98830623095bbf58c3401e47@[192.168.1.2]>
> Date: Thu, 22 Sep 2005 13:30:00 -0700
> To: Peter Stephenson <pws@csr.com>
> From: Dave Yost <Dave@Yost.com>
> Subject: input redirect from a variable
> Content-Type: text/plain; charset="us-ascii" ; format="flowed"
>
> Hi.
>
> It seems to me that there should be a way to do something like this:
>
> foo1="$(...)"
> foo2="$(...)"
>
> comm -3 <<<<$foo1 <<<<$foo2
>
> where the <<<$foo1 syntax says to output $foo1 to a tmp file, then
> use that filenamne as the argument, then delete that file.
>
> Thanks
>
> Dave

I use this function for comparing the hexdump of two files,
hexdiff () {
	diff -u <(hexdump "$1") <(hexdump "$2")
}

so in your case, what you want to do is comm -3 <(...) <(...)
or if you really want
foo1=$(...)
foo2=$(...)
comm -3 <(echo "$foo1") <(echo "$foo2")
but that seems stupid :)
Also note this syntax will provide pipes, if you use =(...) instead
you will get temp files like you asked for.

--
Mikael Magnusson

Re: input redirect from a variable

[Dave Yost]

Thanks.

<(foo) and =(foo) should be mentioned in the REDIRECTION section, where I would have found them.

Dave

At 12:15 PM +0200 2005-09-23, Mikael Magnusson wrote:
>On 9/23/05, Peter Stephenson <pws@csr.com> wrote:
>>
>> ------- Forwarded Message
>>
>> Mime-Version: 1.0
>> Message-Id: <pdy98830623095bbf58c3401e47@[192.168.1.2]>
>> Date: Thu, 22 Sep 2005 13:30:00 -0700
>> To: Peter Stephenson <pws@csr.com>
>> From: Dave Yost <Dave@Yost.com>
>> Subject: input redirect from a variable
>> Content-Type: text/plain; charset="us-ascii" ; format="flowed"
>>
>> Hi.
>>
>> It seems to me that there should be a way to do something like this:
>>
>> foo1="$(...)"
>> foo2="$(...)"
>>
>> comm -3 <<<<$foo1 <<<<$foo2
>>
>> where the <<<$foo1 syntax says to output $foo1 to a tmp file, then
>> use that filenamne as the argument, then delete that file.
>>
>> Thanks
>>
>> Dave
>
>I use this function for comparing the hexdump of two files,
>hexdiff () {
>	diff -u <(hexdump "$1") <(hexdump "$2")
>}
>
>so in your case, what you want to do is comm -3 <(...) <(...)
>or if you really want
>foo1=$(...)
>foo2=$(...)
>comm -3 <(echo "$foo1") <(echo "$foo2")
>but that seems stupid :)
>Also note this syntax will provide pipes, if you use =(...) instead
>you will get temp files like you asked for.
>
>--
>Mikael Magnusson

Re: input redirect from a variable

[Bart Schaefer]

On Sep 23,  6:14am, Dave Yost wrote:
} Subject: Re: input redirect from a variable
}
} <(foo) and =(foo) should be mentioned in the REDIRECTION section,

But they *aren't* redirections.

Redirection implies changing a file descriptor.  <(foo) and =(foo)
create file name strings and substitute them into the command line.

Re: input redirect from a variable

[Bart Schaefer]

On Sep 23, 12:15pm, Mikael Magnusson wrote:
}
} > foo1="$(...)"
} > foo2="$(...)"
} >
} > comm -3 <<<<$foo1 <<<<$foo2
} 
} comm -3 <(echo "$foo1") <(echo "$foo2")
} but that seems stupid :)

Incidentally:

    comm -3 =(<<<$foo1) =(<<<$foo2)

There already is a <<< syntax, but it turns a string into standard input
as a "here document".  Wrapping that in =(...) causes zsh to read that
input and put it in the temp file.

A nice optimization (which has not yet been done) would be to have this
special case avoid forking a new process, much as $(<file) reads a file.
Right now =(<<<...) is equivalent to =($NULLCMD <<<...).  Which actually
seems like a bug to me; shouldn't it use $READNULLCMD instead?  (Not that
one really wants "more" rather than "cat" inside a substitution, but even
outside of substitutions, here-documents are fed to $NULLCMD rather than
to $READNULLCMD.)

Re: input redirect from a variable

[Dave Yost]

At 2:21 PM +0000 2005-09-23, Bart Schaefer wrote:
>On Sep 23,  6:14am, Dave Yost wrote:
>} Subject: Re: input redirect from a variable
>}
>} <(foo) and =(foo) should be mentioned in the REDIRECTION section,
>
>But they *aren't* redirections.
>
>Redirection implies changing a file descriptor.  <(foo) and =(foo)
>create file name strings and substitute them into the command line.

Yes, you're right, now that I think of it.  But they should still be mentioned under redirections with a pointer to where to find them discussed.  (The use of "<" is a little confusing, too.)  Any shell user knows about redirection, but then when you imagine this new facility you want (as I did), you don't have a name for it, so redirection is the first place you look.

Dave

Re: input redirect from a variable

[Peter Stephenson]

Bart Schaefer wrote:
> A nice optimization (which has not yet been done) would be to have this
> special case avoid forking a new process, much as $(<file) reads a file.

This isn't too difficult since most of the hairy test for a simple
enough redirection already exists (and presumably works, though I
certainly can't tell by looking at it).

> Right now =(<<<...) is equivalent to =($NULLCMD <<<...).  Which actually
> seems like a bug to me; shouldn't it use $READNULLCMD instead?  (Not that
> one really wants "more" rather than "cat" inside a substitution, but even
> outside of substitutions, here-documents are fed to $NULLCMD rather than
> to $READNULLCMD.)

It's a bit curious, but maybe the thinking was that you aren't likely to
want to look at a here-doc or here-string via a pager, which is the
typical use of $READNULLCMD.

Index: Doc/Zsh/expn.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v
retrieving revision 1.55
diff -u -r1.55 expn.yo
--- Doc/Zsh/expn.yo	2 Sep 2005 11:38:42 -0000	1.55
+++ Doc/Zsh/expn.yo	23 Sep 2005 16:40:08 -0000
@@ -340,6 +340,15 @@
 form for a program that expects to lseek (see manref(lseek)(2))
 on the input file.
 
+There is an optimisation for substitutions of the form
+tt(=LPAR()<<<)var(arg)tt(RPAR()), where var(arg) is a single-word argument
+to the here-string redirection tt(<<<).  This form produces a file name
+containing the value of var(arg) after any substitutions have been
+performed.  This is handled entirely within the current shell.  This is
+effectively the reverse of the special form tt($LPAR()<)var(arg)tt(RPAR())
+which treats var(arg) as a file name and replaces it with the file's
+contents.
+
 The tt(=) form is useful as both the tt(/dev/fd) and the named pipe
 implementation of tt(<LPAR())var(...)tt(RPAR()) have drawbacks.  In 
 the former case, some programmes may automatically close the file
Index: Src/exec.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
retrieving revision 1.92
diff -u -r1.92 exec.c
--- Src/exec.c	8 Aug 2005 17:01:37 -0000	1.92
+++ Src/exec.c	23 Sep 2005 16:40:23 -0000
@@ -2941,33 +2941,50 @@
     return fd;
 }
 
-/* $(...) */
+/*
+ * Test if some wordcode starts with a simple redirection of type
+ * redir_type.  If it does, return the name of the file, copied onto
+ * the heap.  If it doesn't, return NULL.
+ */
 
-/**/
-LinkList
-getoutput(char *cmd, int qt)
+static char *
+simple_redir_name(Eprog prog, int redir_type)
 {
-    Eprog prog;
-    int pipes[2];
-    pid_t pid;
     Wordcode pc;
 
-    if (!(prog = parse_string(cmd)))
-	return NULL;
-
     pc = prog->prog;
     if (prog != &dummy_eprog &&
 	wc_code(pc[0]) == WC_LIST && (WC_LIST_TYPE(pc[0]) & Z_END) &&
 	wc_code(pc[1]) == WC_SUBLIST && !WC_SUBLIST_FLAGS(pc[1]) &&
 	WC_SUBLIST_TYPE(pc[1]) == WC_SUBLIST_END &&
 	wc_code(pc[2]) == WC_PIPE && WC_PIPE_TYPE(pc[2]) == WC_PIPE_END &&
-	wc_code(pc[3]) == WC_REDIR && WC_REDIR_TYPE(pc[3]) == REDIR_READ && 
+	wc_code(pc[3]) == WC_REDIR && WC_REDIR_TYPE(pc[3]) == redir_type &&
 	!WC_REDIR_VARID(pc[3]) &&
 	!pc[4] &&
 	wc_code(pc[6]) == WC_SIMPLE && !WC_SIMPLE_ARGC(pc[6])) {
+	return dupstring(ecrawstr(prog, pc + 5, NULL));
+    }
+
+    return NULL;
+}
+
+/* $(...) */
+
+/**/
+LinkList
+getoutput(char *cmd, int qt)
+{
+    Eprog prog;
+    int pipes[2];
+    pid_t pid;
+    char *s;
+
+    if (!(prog = parse_string(cmd)))
+	return NULL;
+
+    if ((s = simple_redir_name(prog, REDIR_READ))) {
 	/* $(< word) */
 	int stream;
-	char *s = dupstring(ecrawstr(prog, pc + 5, NULL));
 
 	singsub(&s);
 	if (errflag)
@@ -3101,6 +3118,7 @@
     char *nam;
     Eprog prog;
     int fd;
+    char *s;
 
     if (thisjob == -1)
 	return NULL;
@@ -3109,13 +3127,36 @@
     if (!(nam = gettempname(NULL, 0)))
 	return NULL;
 
+    if ((s = simple_redir_name(prog, REDIR_HERESTR))) {
+	/*
+	 * =(<<<stuff).  Optimise a la $(<file).  It's
+	 * effectively the reverse, converting a string into a file name
+	 * rather than vice versa.
+	 */
+	singsub(&s);
+	if (errflag)
+	    s = NULL;
+	else
+	    untokenize(s);
+    }
+
     if (!jobtab[thisjob].filelist)
 	jobtab[thisjob].filelist = znewlinklist();
     zaddlinknode(jobtab[thisjob].filelist, nam);
 
-    child_block();
+    if (!s)
+	child_block();
     fd = open(nam, O_WRONLY | O_CREAT | O_EXCL | O_NOCTTY, 0600);
 
+    if (s) {
+	/* optimised here-string */
+	int len;
+	unmetafy(s, &len);
+	write(fd, s, len);
+	close(fd);
+	return nam;
+    }
+
     if (fd < 0 || (cmdoutpid = pid = zfork(NULL)) == -1) {
 	/* fork or open error */
 	child_unblock();
Index: Test/A04redirect.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/A04redirect.ztst,v
retrieving revision 1.9
diff -u -r1.9 A04redirect.ztst
--- Test/A04redirect.ztst	16 Apr 2005 23:24:57 -0000	1.9
+++ Test/A04redirect.ztst	23 Sep 2005 16:40:24 -0000
@@ -269,3 +269,11 @@
   exec {myfd}>&-
 1:Error closing file descriptor using readonly variable
 ?(eval):4: can't close file descriptor from readonly parameter myfd
+
+# This tests the here-string to filename optimisation; we can't
+# test that it's actually being optimised, but we can test that it
+# still works.
+  cat =(<<<$'This string has been replaced\nby a file containing it.\n')
+0:Optimised here-string to filename
+>This string has been replaced
+>by a file containing it.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************