[COFF] Requesting thoughts on extended regular expressions in grep.

[Ralph Corderoy <ralph@inputplus.co.uk>]

Hi Ed,

> I have made an attempt to make my RE stuff readable and supportable.

Readable to you, which is fine because you're the prime future reader.
But it's less readable than the regexp to those that know and read them
because of the indirection introduced by the variables.  You've created
your own little language of CAPITALS rather than the lingua franca of
regexps.  :-)

> Machine language was unreadable and then along came assembly language.
> Assembly language was unreadable, then came higher level languages.

Each time the original language was readable because practitioners had
to read and write it.  When its replacement came along, the old skill
was no longer learnt and the language became ‘unreadable’.

> So far, I can do that for this RE program that works for small files,
> large files, binary files and text files for exactly one pattern:
>     YYYY[-MM-DD]
> I constructed this RE with code like this:
>     # ymdt is YYYY-MM-DD RE in text.
>     # looking only for 1900s and 2000s years and no later than today.
>     _YYYY = "(19\d\d|20[01]\d|202" + "[0-" + lastYearRE) + "]" + "){1}"

‘{1}’ is redundant.

>     # months
>     _MM   = "(0[1-9]|1[012])"
>     # days
>     _DD   = "(0[1-9]|[12]\d|3[01])"
>     ymdt = _YYYY + '[' + _INTERNALSEP +
>                          _MM          +
>                          _INTERNALSEP +
>                    ']'{0,1)

I think we're missing something as the ‘'['’ is starting a character
class which is odd for wrapping the month and the ‘{0,1)’ doesn't have
matching brackets and is outside the string.

BTW, ‘{0,1}’ is more readable to those who know regexps as ‘?’.

> For the whole file, RE I used
>     ymdthf = _FRSEP + ymdt + _BASEP
> where FRSEP is front separator which includes
> a bunch of possible separators, excluding numbers and letters, or-ed
> with the up arrow "beginning of line" RE mark.

It sounds like you're wanting a word boundary; something provided by
regexps.  In Python, it's ‘\b’.

    >>> re.search(r'\bfoo\b', 'endfoo foostart foo ends'),
    (<re.Match object; span=(16, 19), match='foo'>,)

Are you aware of the /x modifier to a regexp which ignores internal
whitespace, including linefeeds?  This allows a large regexp to be split
over lines.  There's a comment syntax too.  See
https://docs.python.org/3/library/re.html#re.X

GNU grep isn't too shabby at looking through binary files.  I can't use
/x with grep so in a bash script, I'd do it manually.  \< and \> match
the start and end of a word, a bit like Python's \b.

    re='
        .?\<
            (19[0-9][0-9]|20[01][0-9]|202[0-3])
            (
                ([-:._])
                (0[1-9]|1[0-2])
                \3
                (0[1-9]|[12][0-9]|3[01])
            )?
        \>.?
    '
    re=${re//$'\n'/}
    re=${re// /}

    printf '%s\n' 2001-04-01,1999_12_31 1944.03.01,1914! 2000-01.01 >big-binary-file
    LC_ALL=C grep -Eboa "$re" big-binary-file | sed -n l

which gives

    0:2001-04-01,$
    11:1999_12_31$
    22:1944.03.01,$
    33:1914!$
    39:2000-$

showing:

- the byte offset within the file of each match,
- along with the any before and after byte if it's not a \n and not
  already matched, just to show the word-boundary at work,
- with any non-printables escaped into octal by sed.

> I thought I was on the COFF mailing list.

I'm sending this to just the list.

> I received this email by direct mail to from Larry.

Perhaps your account on the list is configured to not send you an email
if it sees your address in the header's fields.

-- 
Cheers, Ralph.