zsh-workers
 help / color / mirror / code / Atom feed
* Deprecation of egrep
@ 2022-09-10 16:04 Vin Shelton
  2022-09-10 17:26 ` Ellenor Bjornsdottir
  2022-09-11  0:10 ` Bart Schaefer
  0 siblings, 2 replies; 11+ messages in thread
From: Vin Shelton @ 2022-09-10 16:04 UTC (permalink / raw)
  To: Zsh Hackers' List

[-- Attachment #1: Type: text/plain, Size: 398 bytes --]

Greetings!

The latest release of grep officially deprecates egrep in favor of "grep
-E".  In the build process, we have handled this in our autoconf script,
but egrep persists in our test scripts, causing failures.  Changing all
occurrences of "egrep" to "grep -E" in our test scripts will fix this.  How
do we want to handle this?

Regards,
  Vin Shelton

-- 

*Never for money, always for love*

[-- Attachment #2: Type: text/html, Size: 1266 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-10 16:04 Deprecation of egrep Vin Shelton
@ 2022-09-10 17:26 ` Ellenor Bjornsdottir
  2022-09-12  5:32   ` Jun T
  2022-09-11  0:10 ` Bart Schaefer
  1 sibling, 1 reply; 11+ messages in thread
From: Ellenor Bjornsdottir @ 2022-09-10 17:26 UTC (permalink / raw)
  To: Zsh Hackers' List

[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]

grep from which system? GNU? `egrep` would appear to be a GNU extension, mimicked by some other UNIXes like FreeBSD. "Releases of grep" as a thing separate from the OS is a concept foreign to me, other than GNU.

I am not from zsh-land (though I do use it as my login shell), so I doubt I've a say here, but I'd suggest this looks right. I looked at the SUSv2 grep manpage and this will perform as expected if one's grep complies with SUSv2. (For the count, that's a very old version of POSIX.)

On 10 September 2022 16:04:49 UTC, Vin Shelton <acs@alumni.princeton.edu> wrote:
>Greetings!
>
>The latest release of grep officially deprecates egrep in favor of "grep
>-E".  In the build process, we have handled this in our autoconf script,
>but egrep persists in our test scripts, causing failures.  Changing all
>occurrences of "egrep" to "grep -E" in our test scripts will fix this.  How
>do we want to handle this?
>
>Regards,
>  Vin Shelton
>
>-- 
>
>*Never for money, always for love*

-- 
Ellenor Bjornsdottir (she)
sysadmin umbrellix.net

[-- Attachment #2: Type: text/html, Size: 1960 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-10 16:04 Deprecation of egrep Vin Shelton
  2022-09-10 17:26 ` Ellenor Bjornsdottir
@ 2022-09-11  0:10 ` Bart Schaefer
  1 sibling, 0 replies; 11+ messages in thread
From: Bart Schaefer @ 2022-09-11  0:10 UTC (permalink / raw)
  To: Vin Shelton; +Cc: Zsh Hackers' List

On Sat, Sep 10, 2022 at 9:06 AM Vin Shelton <acs@alumni.princeton.edu> wrote:
>
> The latest release of grep officially deprecates egrep in favor of "grep -E".

A quick check reveals six uses, all of which could be replaced with
zsh pattern matching ... five are scanning the output of "locale -a"
and the last the output of "setopt".


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-10 17:26 ` Ellenor Bjornsdottir
@ 2022-09-12  5:32   ` Jun T
  2022-09-12  7:30     ` Jun T
  0 siblings, 1 reply; 11+ messages in thread
From: Jun T @ 2022-09-12  5:32 UTC (permalink / raw)
  To: zsh-workers


> 2022/09/11 2:26, Ellenor Bjornsdottir <ellenor@umbrellix.net> wrote:
> 
> grep from which system? GNU?

GNU grep-3.8. At least Debian-sid and the latest Arch Linux already use it.

> 2022/09/11 1:04, Vin Shelton <acs@alumni.princeton.edu> wrote:
> 
> Changing all occurrences of "egrep" to "grep -E" in our test scripts will fix this.

It works on Linux, {Free,Open,Net,Dragonfly}BSD, macOS and Cygwin.
But I'm not sure if it works on Solaris (and other commercial UNIXes) or not.
Online Solaris manpage indicates that /usr/bin/grep does not accept -E
(although /usr/xpg4/bin/grep accepts it).

Could someone test on Solaris etc.?

> 2022/09/11 9:10, Bart Schaefer <schaefer@brasslantern.com> wrote:
> 
> A quick check reveals six uses, all of which could be replaced with
> zsh pattern matching 

If we can not simply replace egrep by 'grep -E', this would be the simplest
fix; see the patch below.

configure sets EGREP to either 'grep -E' or 'egrep', so propagating this
to ztst.zsh (in some way) would be another (more complex) possibility.


diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index aca275c1c..248180b34 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -94,6 +94,14 @@ tail() {
   command tail "$argv[@]"
 }
 
+# Define our egrep (works only as a pipe). GNU grep-3.8 forces us to use
+# 'grep -E' instead of egrep, but on some systems grep may not accept -E.
+egrep () {
+  while read line; do
+    [[ $line = *(${~1})* ]] && print $line
+  done
+}
+
 # The source directory is not necessarily the current directory,
 # but if $0 doesn't contain a `/' assume it is.
 if [[ $0 = */* ]]; then





^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12  5:32   ` Jun T
@ 2022-09-12  7:30     ` Jun T
  2022-09-12  9:08       ` Daniel Shahaf
  0 siblings, 1 reply; 11+ messages in thread
From: Jun T @ 2022-09-12  7:30 UTC (permalink / raw)
  To: zsh-workers


> 2022/09/12 14:32, I wrote:

> If we can not simply replace egrep by 'grep -E', this would be the simplest
> fix; see the patch below.

Sorry, I forgot 'local line':

diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index aca275c1c..d0e779f94 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -94,6 +94,15 @@ tail() {
  command tail "$argv[@]"
}

+# Define our egrep (works only as a pipe). GNU grep-3.8 forces us to use
+# 'grep -E' instead of egrep, but on some systems grep may not accept -E.
+egrep () {
+  local line
+  while read line; do
+    [[ $line = *(${~1})* ]] && print $line
+  done
+}
+
# The source directory is not necessarily the current directory,
# but if $0 doesn't contain a `/' assume it is.
if [[ $0 = */* ]]; then




^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12  7:30     ` Jun T
@ 2022-09-12  9:08       ` Daniel Shahaf
  2022-09-12 10:15         ` Jun T
  0 siblings, 1 reply; 11+ messages in thread
From: Daniel Shahaf @ 2022-09-12  9:08 UTC (permalink / raw)
  To: zsh-workers

Jun T wrote on Mon, 12 Sep 2022 07:30 +00:00:
> +++ b/Test/ztst.zsh
> @@ -94,6 +94,15 @@ tail() {
> +egrep () {
> +    [[ $line = *(${~1})* ]] && print $line

May I suggest naming the function something else?  «egrep foo»
should interpret foo as an extended regular expression; this function
doesn't.

As to propagating configure's EGREP to ztst.zsh: propagating a single
variable should be straightforward enough, but what about cross builds?
configure runs on the build host and the test suite necessarily runs on
the target host.

Cheers,

Daniel


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12  9:08       ` Daniel Shahaf
@ 2022-09-12 10:15         ` Jun T
  2022-09-12 10:49           ` Peter Stephenson
  0 siblings, 1 reply; 11+ messages in thread
From: Jun T @ 2022-09-12 10:15 UTC (permalink / raw)
  To: zsh-workers


> 2022/09/12 18:08, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> 
> May I suggest naming the function something else?  «egrep foo»
> should interpret foo as an extended regular expression; this function
> doesn't.

Yes; I just wanted to patch only single file.
If we are going to patch each file that uses egrep, we can use
zsh pattern matching in each of them; for example:

diff --git a/Test/D07multibyte.ztst b/Test/D07multibyte.ztst
index e2e9a25ef..cde3f2b81 100644
--- a/Test/D07multibyte.ztst
+++ b/Test/D07multibyte.ztst
@@ -6,7 +6,7 @@
   unset -m LC_\*
   mb_ok=
   langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
+	 ${(M)$(locale -a 2>/dev/null):#*.(utf8|UTF-8)} )
   for LANG in $langs; do
     if [[ é = ? ]]; then
       mb_ok=1

I don't know which is better.

> As to propagating configure's EGREP to ztst.zsh: propagating a single
> variable should be straightforward enough, but what about cross builds?

Hmm, then, if we _know_ the system(s) on which 'grep -E' does not work
we can define EGREP in ztst.zsh depending on $OSTYPE. But I feel this
is not a good idea.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12 10:15         ` Jun T
@ 2022-09-12 10:49           ` Peter Stephenson
  2022-09-12 15:27             ` Vin Shelton
  0 siblings, 1 reply; 11+ messages in thread
From: Peter Stephenson @ 2022-09-12 10:49 UTC (permalink / raw)
  To: Jun T, zsh-workers

> On 12/09/2022 11:15 Jun T <takimoto-j@kba.biglobe.ne.jp> wrote:
> If we are going to patch each file that uses egrep, we can use
> zsh pattern matching in each of them; for example:
> 
> diff --git a/Test/D07multibyte.ztst b/Test/D07multibyte.ztst
> index e2e9a25ef..cde3f2b81 100644
> --- a/Test/D07multibyte.ztst
> +++ b/Test/D07multibyte.ztst
> @@ -6,7 +6,7 @@
>    unset -m LC_\*
>    mb_ok=
>    langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
> -	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
> +	 ${(M)$(locale -a 2>/dev/null):#*.(utf8|UTF-8)} )
>    for LANG in $langs; do
>      if [[ é = ? ]]; then
>        mb_ok=1
> 
> I don't know which is better.

I tend to agree with Bart that a change like the above is probably best
in the long run.  Directly using zsh's own features inline would be
clearer (given the core audience here is zsh experts anyway).

pws


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12 10:49           ` Peter Stephenson
@ 2022-09-12 15:27             ` Vin Shelton
  2022-09-13  4:33               ` Jun T
  0 siblings, 1 reply; 11+ messages in thread
From: Vin Shelton @ 2022-09-12 15:27 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: Jun T, Zsh Hackers' List

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

On Mon, Sep 12, 2022 at 6:54 AM Peter Stephenson <
p.w.stephenson@ntlworld.com> wrote:

>
> I tend to agree with Bart that a change like the above is probably best
> in the long run.  Directly using zsh's own features inline would be
> clearer (given the core audience here is zsh experts anyway).


I agree that a zsh-specific solution is best.  FWIW, for clarity I would
remove the 'langs' assignment entirely and put the entire construct on the
'for' statement:

  for LANG in en_{US,GB}.{UTF-,utf}8 en.UTF-8 ${(M)$(locale -a
2>/dev/null):#*.(utf8|UTF-8)} ; do

[-- Attachment #2: Type: text/html, Size: 1454 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-12 15:27             ` Vin Shelton
@ 2022-09-13  4:33               ` Jun T
  2022-09-23 17:14                 ` Jun. T
  0 siblings, 1 reply; 11+ messages in thread
From: Jun T @ 2022-09-13  4:33 UTC (permalink / raw)
  To: zsh-workers


> 2022/09/13 0:27, Vin Shelton <acs@alumni.princeton.edu> wrote:
> 
> I agree that a zsh-specific solution is best.  FWIW, for clarity I would remove the 'langs' assignment entirely and put the entire construct on the 'for' statement:
> 
>   for LANG in en_{US,GB}.{UTF-,utf}8 en.UTF-8 ${(M)$(locale -a 2>/dev/null):#*.(utf8|UTF-8)} ; do

I think using the array 'langs' allows us to continue long line
without using back slash (but I have no objection for someone to
later include the above simplification).

In the path below, I moved the repeated code for finding UTF-8 locale
to a single function in ztst.zsh. The only other use of egrep is
in E01options.zsh.


diff --git a/Test/D07multibyte.ztst b/Test/D07multibyte.ztst
index e2e9a25ef..6909346cb 100644
--- a/Test/D07multibyte.ztst
+++ b/Test/D07multibyte.ztst
@@ -1,19 +1,7 @@
 %prep
 
-# Find a UTF-8 locale.
-  setopt multibyte
-# Don't let LC_* override our choice of locale.
-  unset -m LC_\*
-  mb_ok=
-  langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
-  for LANG in $langs; do
-    if [[ é = ? ]]; then
-      mb_ok=1
-      break;
-    fi
-  done
-  if [[ -z $mb_ok ]]; then
+  LANG=$(ZTST_find_UTF8)
+  if [[ -z $LANG ]]; then
     ZTST_unimplemented="no UTF-8 locale or multibyte mode is not implemented"
   else
     print -u $ZTST_fd Testing multibyte with locale $LANG
diff --git a/Test/E01options.ztst b/Test/E01options.ztst
index 2acbfd357..d38fbed74 100644
--- a/Test/E01options.ztst
+++ b/Test/E01options.ztst
@@ -651,7 +651,7 @@
 >noktarg1
 >0 1
 
-  showopt() { setopt | egrep 'localoptions|ksharrays'; }
+  showopt() { echo ${(FM)${(@f)"$(setopt)"}:#(localoptions|ksharrays)*} }
   f1() { setopt localoptions ksharrays; showopt }
   f2() { setopt ksharrays; showopt }
   setopt kshoptionprint
diff --git a/Test/V07pcre.ztst b/Test/V07pcre.ztst
index c9c844d2a..ca13419e5 100644
--- a/Test/V07pcre.ztst
+++ b/Test/V07pcre.ztst
@@ -6,20 +6,8 @@
     return 0
   fi
   setopt rematch_pcre
-# Find a UTF-8 locale.
-  setopt multibyte
-# Don't let LC_* override our choice of locale.
-  unset -m LC_\*
-  mb_ok=
-  langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
-  for LANG in $langs; do
-    if [[ é = ? ]]; then
-      mb_ok=1
-      break;
-    fi
-  done
-  if [[ -z $mb_ok ]]; then
+  LANG=$(ZTST_find_UTF8)
+  if [[ -z $LANG ]]; then
     ZTST_unimplemented="no UTF-8 locale or multibyte mode is not implemented"
   else
     print -u $ZTST_fd Testing PCRE multibyte with locale $LANG
diff --git a/Test/X02zlevi.ztst b/Test/X02zlevi.ztst
index 8146d6752..203c13c32 100644
--- a/Test/X02zlevi.ztst
+++ b/Test/X02zlevi.ztst
@@ -1,16 +1,7 @@
 # Tests of the vi mode of ZLE
 
 %prep
-  unset -m LC_\*
-  ZSH_TEST_LANG=
-  langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
-  for LANG in $langs; do
-    if [[ é = ? ]]; then
-      ZSH_TEST_LANG=$LANG 
-      break;
-    fi
-  done
+  ZSH_TEST_LANG=$(ZTST_find_UTF8)
   if ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     comptestinit -v -z $ZTST_testdir/../Src/zsh
diff --git a/Test/X03zlebindkey.ztst b/Test/X03zlebindkey.ztst
index 43692a85b..5277332a7 100644
--- a/Test/X03zlebindkey.ztst
+++ b/Test/X03zlebindkey.ztst
@@ -3,16 +3,7 @@
 # into bindings.  The latter is particularly tricky with multibyte sequences.
 
 %prep
-  unset -m LC_\*
-  ZSH_TEST_LANG=
-  langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-	 $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
-  for LANG in $langs; do
-    if [[ é = ? ]]; then
-      ZSH_TEST_LANG=$LANG
-      break;
-    fi
-  done
+  ZSH_TEST_LANG=$(ZTST_find_UTF8)
   if ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     comptestinit -z $ZTST_testdir/../Src/zsh
diff --git a/Test/Y01completion.ztst b/Test/Y01completion.ztst
index 6af0efc6d..f976f9f91 100644
--- a/Test/Y01completion.ztst
+++ b/Test/Y01completion.ztst
@@ -1,16 +1,7 @@
 # Tests for completion system.
 
 %prep
-  unset -m LC_\*
-  ZSH_TEST_LANG=
-  langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
-         $(locale -a 2>/dev/null | egrep 'utf8|UTF-8'))
-  for LANG in $langs; do
-    if [[ é = ? ]]; then
-      ZSH_TEST_LANG=$LANG
-      break;
-    fi
-  done
+  ZSH_TEST_LANG=$(ZTST_find_UTF8)
   if ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     mkdir comp.tmp
diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index aca275c1c..d95b726e7 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -37,6 +37,21 @@ emulate -R zsh
 # LANG must be passed to child zsh.
 export LANG
 
+# find UTF-8 locale
+ZTST_find_UTF8 () {
+  setopt multibyte
+  # Don't let LC_* override our choice of locale.
+  unset -m LC_\*
+  local langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
+               ${(M)$(locale -a 2>/dev/null):#*.(utf8|UTF-8)})
+  for LANG in $langs; do
+    if [[ é = ? ]]; then
+      echo $LANG
+      return
+    fi
+  done
+}
+
 # Don't propagate variables that are set by default in the shell.
 typeset +x WORDCHARS
 






^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Deprecation of egrep
  2022-09-13  4:33               ` Jun T
@ 2022-09-23 17:14                 ` Jun. T
  0 siblings, 0 replies; 11+ messages in thread
From: Jun. T @ 2022-09-23 17:14 UTC (permalink / raw)
  To: zsh-workers


> 2022/09/13 13:33, Jun T <takimoto-j@kba.biglobe.ne.jp> wrote:
> 
> In the path below, I moved the repeated code for finding UTF-8 locale
> to a single function in ztst.zsh. The only other use of egrep is
> in E01options.zsh.

Sorry, with this patch D07 fails if users explicitly export LC_CTYPE=C
when running the test. Unsetting LC_\* in ZTST_find_UTF8() is not
sufficient.

I think we can simply unset LC_\* in ztst.zsh for ALL the tests
and rely only on LANG.


diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index d95b726e7..ea1b016d5 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -25,23 +25,13 @@
 # still not be good enough.  Maybe we should trick it somehow.
 emulate -R zsh
 
-# Ensure the locale does not screw up sorting.  Don't supply a locale
-# unless there's one set, to minimise problems.
-[[ -n $LC_ALL ]] && LC_ALL=C
-[[ -n $LC_CTYPE ]] && LC_CTYPE=C
-[[ -n $LC_COLLATE ]] && LC_COLLATE=C
-[[ -n $LC_NUMERIC ]] && LC_NUMERIC=C
-[[ -n $LC_MESSAGES ]] && LC_MESSAGES=C
-[[ -n $LANG ]] && LANG=C
-# Test file may (or may not) set LANG to other locales. In either case,
-# LANG must be passed to child zsh.
-export LANG
+# By default tests are run in C locale. LANG must be passed to child zsh.
+unset -m LC_\*
+export LANG=C
 
 # find UTF-8 locale
 ZTST_find_UTF8 () {
   setopt multibyte
-  # Don't let LC_* override our choice of locale.
-  unset -m LC_\*
   local langs=(en_{US,GB}.{UTF-,utf}8 en.UTF-8
                ${(M)$(locale -a 2>/dev/null):#*.(utf8|UTF-8)})
   for LANG in $langs; do





^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-09-23 17:15 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-10 16:04 Deprecation of egrep Vin Shelton
2022-09-10 17:26 ` Ellenor Bjornsdottir
2022-09-12  5:32   ` Jun T
2022-09-12  7:30     ` Jun T
2022-09-12  9:08       ` Daniel Shahaf
2022-09-12 10:15         ` Jun T
2022-09-12 10:49           ` Peter Stephenson
2022-09-12 15:27             ` Vin Shelton
2022-09-13  4:33               ` Jun T
2022-09-23 17:14                 ` Jun. T
2022-09-11  0:10 ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).