questions - calling matheval() for printf

questions - calling matheval() for printf

[Oliver Kiddle]

I have a few questions related to the use of matheval() for numeric
arguments to printf. According to the POSIX definition (which you can
read at http://www.opengroup.org/onlinepubs/007908799/xcu/printf.html),
with the %d format specification, and assuming 32-bit ints, the
following are the expect results:

argument	stdout		stderr
5a		5		printf: "5a" not completely converted
9999999999	2147483647	printf: "9999999999" arithmetic overflow
-9999999999	-2147483648	printf: "-9999999999" arithmetic overflow
ABC		0		printf: "ABC" expected numeric value

In the first case, the different error message doesn't worry me but
math evaluation doesn't return `5'. It is conceivable that someone
relies on this in a script, passing a string starting with a number and
redirecting stderr. I could successively chop characters off the right
of the string until matheval returns without an error. This would need
a hack to avoid repeated error messages from matheval(). Any other,
perhaps better suggestions?

Is there any way I can find out when arithmetic overflows have occurred
for the second and third cases? matheval() just allows the numbers to
wrap?

In the last case, we'll just have to be non-compliant (it returns $ABC)
but that doesn't worry me much.

What is going on with this errval and zerr/zwarn stuff? I don't
entirely understand which I should be calling on errors and whether I
am right to zero errval after calling matheval() and reading it?

Can a 64-bit integer be something other than long long? It seems I only
have ZSH_64_BIT_TYPE to check and it might need a different format
specifier for printf(3) if this is something other than long long.

Aside from this, using matheval() works very smoothly and it is
certainly better that we won't lose precision due to converting floats
via ASCII before formatting..

Oliver

Re: questions - calling matheval() for printf

[Bart Schaefer]

On Oct 12,  3:03pm, Oliver Kiddle wrote:
}
} What is going on with this errval and zerr/zwarn stuff? I don't
} entirely understand which I should be calling on errors and whether I
} am right to zero errval after calling matheval() and reading it?

(What's "errval"?  I grepped the source but didn't find it.)

In the long ago and far away, there was only zerr().  Calling zerr() would
cause zsh to print a message to stderr and abort whatever it was doing.

Aborting wasn't always what was wanted, though, so sometimes the code
called zerr() and then zeroed the global error flag.

Later zwarn() was introduced, so one could call zerr() when zsh should
abort and zwarn() when all it needed to do was print the error message.
A sweep was done to change zerr() calls to zwarn() as appropriate, but
some of them either weren't caught or it was unclear what should happen.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: questions - calling matheval() for printf

[Peter Stephenson]

Sorry I didn't reply to this (the bits Bart didn'g) earlier...

Oliver Kiddle wrote:
> argument	stdout		stderr
> 5a		5		printf: "5a" not completely converted
> 
> In the first case, the different error message doesn't worry me but
> math evaluation doesn't return `5'. It is conceivable that someone
> relies on this in a script, passing a string starting with a number and
> redirecting stderr. I could successively chop characters off the right
> of the string until matheval returns without an error. This would need
> a hack to avoid repeated error messages from matheval(). Any other,
> perhaps better suggestions?

If we need this, it would be better to hack matheval to give it an option
to stop and return a pointer to the end, just like strtol().  But I don't
think we need to adhere to POSIX printf semantics that closely.  I'd be
more interested in what other shells do.

> Is there any way I can find out when arithmetic overflows have occurred
> for the second and third cases? matheval() just allows the numbers to
> wrap?

Again, this would be more work and I'm not sure it's worth it in a shell.

> Can a 64-bit integer be something other than long long? It seems I only
> have ZSH_64_BIT_TYPE to check and it might need a different format
> specifier for printf(3) if this is something other than long long.

On Alpha, it will normally be long rather than long long.  However, I
compiled sizeof(long long) with gcc on the Alpha at Sourceforge, and that
seems to be 64-bit too.  I don't think it's a good idea to rely on that,
though --- there's no standard for such things.

Can we make %q do just the same as what ${(q)foo} would do?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: questions - calling matheval() for printf

[Oliver Kiddle]

Peter Stephenson wrote:
> 
> Sorry I didn't reply to this (the bits Bart didn'g) earlier...

Thanks. I've got another question here I'm afraid.

> If we need this, it would be better to hack matheval to give it an option
> to stop and return a pointer to the end, just like strtol().  But I don't
> think we need to adhere to POSIX printf semantics that closely.  I'd be
> more interested in what other shells do.

Ksh handles things in the way that zsh perhaps ideally should:

$ printf '%d\n' 4a
ksh: printf: 4a: arithmetic syntax error
ksh: printf: warning: invalid argument of type d
4
$ printf '%d\n' 123456789012
ksh: printf: warning: 123456789012: overflow exception
2147483647

(though I see no need for two error messages in the first case)

Bash produces an error in each case though it fails to output 4 and max
int respectively as POSIX stipulates. Note that bash doesn't attempt
arithmetic evaluation here.

> > Can a 64-bit integer be something other than long long? It seems I only
> > have ZSH_64_BIT_TYPE to check and it might need a different format
> > specifier for printf(3) if this is something other than long long.
> 
> On Alpha, it will normally be long rather than long long.  However, I
> compiled sizeof(long long) with gcc on the Alpha at Sourceforge, and that

I'll do a bit of testing on Sourceforge then. Thanks.

> Can we make %q do just the same as what ${(q)foo} would do?

Yes. That is what it does at the moment so I'll just ditch the patch.

I have below another experimental feature. This allows the argument to
be specified in the format in much the same way as is possible with
printf(3). e.g:

printf '%2$s %s %1$s\n' World Good Morning

will print `Good Morning World'

Note that glibc's printf(3) would print `Good Good World' for this but
it seems to be the exception.

The question is how should this interact with the printf(1) feature of
reusing the format if more arguments remain. The easy answer would be
to not reuse the format if this feature had been used. As an
experiment, I've made it remove all arguments up to the last one used.
This allows interesting things like:

% printf '%2$s %1$s ' 1 2 3 4 5 6 ;echo
2 1 4 3 6 5

I can see this having some uses but I can also see it being a problem
as this is likely to be used for picking out fields where the arguments
are some command in $(...). A possibly quite good option would be an
option to print to disable format reuse (in which case any suggestion
on the choice of option letter?). Any other ideas on this?

Note that there is no ksh precedent here as ksh seg faults if you give
it these argument specifiers (I've got a lot of bug reporting to do
when I finish this).

If you try the patch, be aware that things may go awry if you try
specifying widths and precisions.

Oliver

Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.58
diff -u -r1.58 builtin.c
--- Src/builtin.c	2001/10/15 11:34:27	1.58
+++ Src/builtin.c	2001/10/15 16:54:35
@@ -2892,7 +2892,8 @@
 int
 bin_print(char *name, char **args, char *ops, int func)
 {
-    int flen, width, prec, type, argc, n, nnl = 0, ret = 0;
+    int flen, width, prec, type, argc, n, gotwidth;
+    int nnl = 0, ret = 0, maxarg = 0;
     int flags[5], *len;
     char *start, *endptr, *c, *d, *flag, spec[11], *fmt = NULL;
     char **first, *flagch = "0+- #", save, nullstr = '\0';
@@ -3095,6 +3096,11 @@
     /* printf style output */
     *spec='%';
     do {
+    	if (maxarg) {
+	    first += maxarg;
+	    argc -= maxarg;
+    	    maxarg = 0;
+	}
 	for (c = fmt;c-fmt < flen;c++) {
 	    if (*c != '%') {
 		putc(*c, fout);
@@ -3110,7 +3116,7 @@
 	    }
 
 	    type = prec = -1;
-	    width = 0;
+	    width = gotwidth = 0;
 	    d = spec + 1;
 
 	    /* copy only one of each flag as spec has finite size */
@@ -3123,13 +3129,32 @@
 	    	c++;
 	    }
 
-	    if (*c == '*') {
-		if (*args) width = (int)mathevali(*args++);
-		c++;
-	    } else if (idigit(*c)) {
+	    if (idigit(*c)) {
 		width = strtoul(c, &endptr, 0);
 		c = endptr;
+		if (*c == '$') {
+		    DPUTS(width <= 0, "specified zero or negative arg");
+		    if (width > argc) {
+		    	zwarnnam(name, "%d: argument specifier out of range",
+			    0, width);
+			ret = 1;
+		    } else {
+		    	if (width > maxarg) maxarg = width;
+		    	args = first + width - 1;
+		    }
+		    if (idigit(*++c)) {
+		    	width = strtoul(c, &endptr, 0);
+			c = endptr;
+			gotwidth = 1;
+		    } else
+		    	width = 0;
+		} else
+		    gotwidth = 1;
 	    }
+	    if (!gotwidth && *c == '*') {
+		if (*args) width = (int)mathevali(*args++);
+		c++;
+	    }
 	    *d++ = '*';
 
 	    if (*c == '.') {
@@ -3224,7 +3249,7 @@
 		    	*d++ = 'l', *d++ = *c, *d = '\0';
 			zlongval = (*args) ? mathevali(*args++) : 0;
 			if (errflag) {
-			    zlongval = 0;
+			    doubleval = 0;
 			    errflag = 0;
 			}
 			print_val(zlongval)
@@ -3255,8 +3280,11 @@
 		    }
 		}
 	    }
+	    if (maxarg && (args - first > maxarg))
+	    	maxarg = args - first;
 	}
 
+    	if (maxarg) args = first + maxarg;
     /* if there are remaining args, reuse format string */
     } while (*args && args != first);

_____________________________________________________________________
This message has been checked for all known viruses by the 
MessageLabs Virus Scanning Service. For further information visit
http://www.messagelabs.com/stats.asp

Re: questions - calling matheval() for printf

[Peter Stephenson]

Oliver Kiddle wrote:
> The question is how should this interact with the printf(1) feature of
> reusing the format if more arguments remain. The easy answer would be
> to not reuse the format if this feature had been used. As an
> experiment, I've made it remove all arguments up to the last one used.
> This allows interesting things like:
> 
> % printf '%2$s %1$s ' 1 2 3 4 5 6 ;echo
> 2 1 4 3 6 5
> 
> I can see this having some uses but I can also see it being a problem
> as this is likely to be used for picking out fields where the arguments
> are some command in $(...).

Even in that case, the problem is really with the reuse of the format,
rather than the special argument-picking syntax.  Maybe it would be best to
have a command-line option to turn it (the reuse of the format specifier,
that is) off --- or even on, since it might be regarded as a little florid
for default behaviour.  But I suppose we're going to have to stick with ksh
if we're trying to match it.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************