hzoli change: $foo:s//r/

hzoli change: $foo:s//r/

[Heading, Anthony]

Sorry I missed this first time round.

hzoli changelog of Jul 10 says:
- $foo:s//r/ gives ... error message ... (194)

Now the following used to work

% array=(a b c d e)
% echo $array:s//\&-\&/
a-a b-b c-c d-d e-e

I remember I had a use for this once...

Anthony

Re: hzoli change: $foo:s//r/

[Zoltan Hidvegi]

Anthony Heading wrote:
> 
> Sorry I missed this first time round.
> 
> hzoli changelog of Jul 10 says:
> - $foo:s//r/ gives ... error message ... (194)
> 
> Now the following used to work
> 
> % array=(a b c d e)
> % echo $array:s//\&-\&/
> a-a b-b c-c d-d e-e
> 
> I remember I had a use for this once...

The problem is that the manual says that zsh should use the left-hand-side of
the last substitution if the substitution string is empty.  And the history
code also does this.  Also the code in hist.c which produced that feature you
describe was buggy and it sometimes got into an infinite loop (or did some
even worse thing).  So what you used so far was probably an undocumented bug.
Maybe we can find an other syntax for that.

Zoltan

Re: hzoli change: $foo:s//r/

[Thorsten Meinecke]

In article "archive/latest/313", Zoltan Hidvegi wrote:
> Anthony Heading wrote:

> > hzoli changelog of Jul 10 says:
> > - $foo:s//r/ gives ... error message ... (194)
> > 
> > Now the following used to work
> > 
> > % array=(a b c d e)
> > % echo $array:s//\&-\&/
> > a-a b-b c-c d-d e-e

> Maybe we can find an other syntax for that.


In plain unmodified hzoli10.3 the unadorned backslash does exactly that.

$ ./zsh -fc 'echo $ZSH_VERSION
             array=(a b c d e)
             echo $array:s/\/\&-\&/
            '
2.6-beta10-hzoli10.3
a-a b-b c-c d-d e-e
$

Is this just another undocumented feature? Since there's no real need
to escape the delimiter--just choose a different one--the backslash 
seems one of the best choices to denote "substitute whole string/ele-
ments of array" to me. 


Sincere apologies for my GNUisms breaking hzoli10.2.
Please forgive me.
  Thorsten


BTW it is "substitution", isn't it?

--- zsh-2.6-beta10-hzoli10.3/Src/subst.c.orig	Wed Aug 16 00:46:45 1995
+++ zsh-2.6-beta10-hzoli10.3/Src/subst.c	Wed Aug 16 01:23:57 1995
@@ -905,7 +905,7 @@
 		*str = aptr + 1;
 		return n;
 	    } else {
-		zerr("bad subtitution", NULL, 0);
+		zerr("bad substitution", NULL, 0);
 		return NULL;
 	    }
 	else
@@ -1465,7 +1467,7 @@
 		del = *ptr1++;
 		for (ptr2 = ptr1; *ptr2 != del && *ptr2; ptr2++);
 		if (!*ptr2) {
-		    zerr("bad subtitution", NULL, 0);
+		    zerr("bad substitution", NULL, 0);
 		    return;
 		}
 		*ptr2++ = '\0';

-- 
Thorsten Meinecke
<kaefer@aglaia.snafu.de>

Re: hzoli change: $foo:s//r/

[Zoltan Hidvegi]

Thorsten Meinecke wrote:
> In plain unmodified hzoli10.3 the unadorned backslash does exactly that.
> 
> $ ./zsh -fc 'echo $ZSH_VERSION
>              array=(a b c d e)
>              echo $array:s/\/\&-\&/
>             '
> 2.6-beta10-hzoli10.3
> a-a b-b c-c d-d e-e
> $
> 
> Is this just another undocumented feature? Since there's no real need
> to escape the delimiter--just choose a different one--the backslash 
> seems one of the best choices to denote "substitute whole string/ele-
> ments of array" to me. 

Yes, it really works.  This is because the backslash is removed as a nularg.
However it does not work if the substitution is quoted with double quotes.

There is a bug here I discovered testing this: nulargs are not properly
removed from the substitution strings.  This bug is also present in stock
beta10 zsh.  The patch below fixes that.  After the patch :s/''/\&-\&/ will
also do the same as :s/\/\&-\&/ but the '' syntax is more readable I think
(and also in the future \ may quote the separator string so I wouldn't
recommend its usage).

> Sincere apologies for my GNUisms breaking hzoli10.2.
> Please forgive me.
>   Thorsten

It is partially my mistake.  I knew that -q is not a universal grep option but
somehow I forgot it.  If I had tried configure on all systems I have acess to
befor releasing 10.2 I would have noticed it.

> BTW it is "substitution", isn't it?

Yes.

Zoltan

*** 1.14	1995/07/10 18:02:35
--- Src/subst.c	1995/08/16 11:48:46
***************
*** 1482,1491 ****
  		}
  		for (tt = hsubl; *tt; tt++)
  		    if (INULL(*tt))
! 			chuck(tt);
  		for (tt = hsubr = ztrdup(ptr2); *tt; tt++)
  		    if (INULL(*tt))
! 			chuck(tt);
  		ptr2[-1] = del;
  		if (sav)
  		    ptr3[-1] = sav;
--- 1482,1491 ----
  		}
  		for (tt = hsubl; *tt; tt++)
  		    if (INULL(*tt))
! 			chuck(tt--);
  		for (tt = hsubr = ztrdup(ptr2); *tt; tt++)
  		    if (INULL(*tt))
! 			chuck(tt--);
  		ptr2[-1] = del;
  		if (sav)
  		    ptr3[-1] = sav;

Re: hzoli change: $foo:s//r/

[Thorsten Meinecke]

Another very minor problem with Zoltan's substitution code.

If hsubr is non-NULL, and a "bad/no previous substitution" error 
occurs, hsubr is free()d. This may corrupt memory. Example:

  $ PS1='[%!] ' ./zsh -f
  [1] echo $foo:s/x//
  
  [2] echo $foo:s/ //
  zsh: bad substitution
  [3] echo $foo:s/ //
  zsh: attempt to free already free storage

This fix only free()s hsubr when there was *no* error.

--- zsh-2.6-beta10-hzoli10.3/Src/subst.c.orig	Wed Aug 16 16:15:45 1995
+++ zsh-2.6-beta10-hzoli10.3/Src/subst.c	Wed Aug 16 16:37:07 1995
@@ -1461,5 +1461,4 @@
 		c = **ptr;
 		(*ptr)++;
-		zsfree(hsubr);
 		ptr1 = *ptr;
 		del = *ptr1++;
@@ -1481,4 +1480,5 @@
 		    return;
 		}
+		zsfree(hsubr);
 		for (tt = hsubl; *tt; tt++)
 		    if (INULL(*tt))

-- 
Thorsten Meinecke
<kaefer@aglaia.snafu.de>

Re: hzoli change: $foo:s//r/

[Zoltan Hidvegi]

> 
> Another very minor problem with Zoltan's substitution code.
> 
> If hsubr is non-NULL, and a "bad/no previous substitution" error 
> occurs, hsubr is free()d. This may corrupt memory. Example:
> 
>   $ PS1='[%!] ' ./zsh -f
>   [1] echo $foo:s/x//
>   
>   [2] echo $foo:s/ //
>   zsh: bad substitution
>   [3] echo $foo:s/ //
>   zsh: attempt to free already free storage
> 
> This fix only free()s hsubr when there was *no* error.

You are right here.  But there is an other problem: the code assumes in many
places that extern variables are initialized to zero.  If hsubl/hsubr were not
zero on startup it would cause the same problem even after this patch.  The
patch below initializes hsubl/hsubr in globals.h and probably there are some
other variables which require similar treatment.

Zoltan

*** 1.10	1995/07/24 17:12:46
--- Src/globals.h	1995/08/14 17:09:11
***************
*** 36,43 ****
--- 36,45 ----
  
  #ifdef GLOBALS
  # define EXTERN
+ # define CHARPTR(X) char *X = NULL;
  #else
  # define EXTERN extern
+ # define CHARPTR(X) extern char *X;
  #endif
   
  #ifdef GLOBALS
***************
*** 284,294 ****
   
  /* the last l for s/l/r/ history substitution */
   
! EXTERN char *hsubl;
  
  /* the last r for s/l/r/ history substitution */
   
! EXTERN char *hsubr;
   
  /* We cache `USERNAME' and use check cached_uid *
   * so we know when to recompute it.             */
--- 286,296 ----
   
  /* the last l for s/l/r/ history substitution */
   
! CHARPTR(hsubl)
  
  /* the last r for s/l/r/ history substitution */
   
! CHARPTR(hsubr)
   
  /* We cache `USERNAME' and use check cached_uid *
   * so we know when to recompute it.             */

Re: hzoli change: $foo:s//r/

[Barton E. Schaefer]

On Aug 16,  5:20pm, Zoltan Hidvegi wrote:
} Subject: Re: hzoli change:  $foo:s//r/
}
} > Another very minor problem with Zoltan's substitution code.
} > 
} > If hsubr is non-NULL, and a "bad/no previous substitution" error 
} > occurs, hsubr is free()d. This may corrupt memory.
} 
} You are right here.  But there is an other problem: the code assumes in many
} places that extern variables are initialized to zero.

This is a safe assumption.  It's required that the compiler initialize
global variables to zero, at least as far back as K&R and probably farther.

-- 
Bart Schaefer                     Vice President, Technology, Z-Code Software
schaefer@z-code.com                  Division of NCD Software Corporation
http://www.well.com/www/barts

Re: hzoli change: $foo:s//r/

[Wayne Davison]

If you wish to initialize globals in a file that is used to either
declare or define, I find the following syntax more readable:

--------------------------------------------------------
#ifdef GLOBALS
# define EXTERN
# define INIT(x) = x
#else
# define EXTERN extern
# define INIT(x)
#endif

[...]

/* the last l for s/l/r/ history substitution */

EXTERN char *hsubl INIT(NULL);

/* the last r for s/l/r/ history substitution */

EXTERN char *hsubr INIT(NULL);
--------------------------------------------------------

This can be used for any type variable and could simplify a few
of the other #ifdef GLOBALS initializations in the file.

..wayne..

Re: hzoli change: $foo:s//r/

[Zoltan Hidvegi]

Wayne Davison wrote:
> 
> If you wish to initialize globals in a file that is used to either
> declare or define, I find the following syntax more readable:
> 
> --------------------------------------------------------
> #ifdef GLOBALS
> # define EXTERN
> # define INIT(x) = x
> #else
> # define EXTERN extern
> # define INIT(x)
> #endif
> 
> [...]
> 
> /* the last l for s/l/r/ history substitution */
> 
> EXTERN char *hsubl INIT(NULL);
> 
> /* the last r for s/l/r/ history substitution */
> 
> EXTERN char *hsubr INIT(NULL);
> --------------------------------------------------------
> 
> This can be used for any type variable and could simplify a few
> of the other #ifdef GLOBALS initializations in the file.

Yes, it is really a good idea.  But if it is really guaranteed that externs
are initialized to zere therese initializations are unnecessary.  The fact
that zsh worked so far shows that it is quite safe to assume that global
variables are initialized to zero, but what is in the ANSI-C standard?

Zoltan

Re: hzoli change: $foo:s//r/

[Zefram]

>Yes, it is really a good idea.  But if it is really guaranteed that externs
>are initialized to zere therese initializations are unnecessary.  The fact
>that zsh worked so far shows that it is quite safe to assume that global
>variables are initialized to zero, but what is in the ANSI-C standard?

ANSI requires that all objects of static storage duration (i.e.
non-auto variables) are initialised to zero(*), except when explicitly
initialised to something else.  (*) Integers are initialised to
all-bits zero, which is numerically zero, but floating-point values are
initialised to floating-point zero, and pointers are initialised to a
NULL value which is not required to be all-bits zero.

This requirement has existed throughout C's lifetime; it initially
appeared, I think, because such variables under Unix are put in the BSS
area, which the kernel initialises to all-bits zero.  The ANSI
committee had no choice but to uphold the assumption made by most C
programmers that this initialisation would take place, but they made
the required initialisation effectively "numerically zero" rather than
"all-bits zero" because of the unspecified nature of pointer and
floating-point representations.

Under Unix it's quite safe to assume that this initialisation will
happen (to all-bits zero, which under Unix is good for a NULL pointer)
even in languages that make no such guarantee.

-zefram

RE: hzoli change: $foo:s//r/

[Heading, Anthony]

Umm, I remember patching this a couple of years ago to give its recent
behaviour. It used to loop infinitely I think. The change never made it into
the manual, so I agree it's undocumented and it's quite reasonable to
change it.   But it did work, I don't consider it a bug, quite a number
of my scripts turned out to use it [they all fell over today :-( ] and I'm
a bit upset at seeing it go just to introduce an unnecessary error message.

Can't we just document it?

A
_______________________________________________________________________________
From: Zoltan Hidvegi on Tue, Aug 15, 1995 1:05 pm
Subject: Re: hzoli change:  $foo:s//r/
To: Heading Anthony
Cc: zsh-workers

Anthony Heading wrote:
> 
> Sorry I missed this first time round.
> 
> hzoli changelog of Jul 10 says:
> - $foo:s//r/ gives ... error message ... (194)
> 
> Now the following used to work
> 
> % array=(a b c d e)
> % echo $array:s//\&-\&/
> a-a b-b c-c d-d e-e
> 
> I remember I had a use for this once...

The problem is that the manual says that zsh should use the left-hand-side of
the last substitution if the substitution string is empty.  And the history
code also does this.  Also the code in hist.c which produced that feature you
describe was buggy and it sometimes got into an infinite loop (or did some
even worse thing).  So what you used so far was probably an undocumented bug.
Maybe we can find an other syntax for that.

Zoltan

Re: hzoli change: $foo:s//r/

[Zoltan Hidvegi]

But :s//foo/ should either use the last substitution as documented already in
the manual or use the whole string to be modified as you want.  Other people
requested the later behaviour (that's why I changed this and that's what the
present maual says about it, I did not change the manual I only changed the
code to conform the maual).

So what to do now?  I'd prefer the present solution as it is consistent with
the history substitution behaviour.

Zoltan