libzglob?

libzglob?

[Jarkko Hietaniemi]

Hi,

has there been any talk / thought / action as to separate the zsh
globbing code into its own library?  (as Zle nowadays seems to be,
sort of)

Background: I am one of the Perl Porters and there is one sore wound:
Perl uses csh for filename globbing (you may boo now).  Using external
program is a) ugly b) slow c) security risk d) in certain environments
csh globbing is plain broken (`glob *.c` may not return NUL-separated
string of filenames, glob cannot return >1kB stuff, things like that).
All that goes against the grain of Perl. *)

We briefly looked into tcsh -- but why stop there when there is a
much better glob out there.

What would be needed would be as small subdirectory as possible
in addition to zsh's glob.c that could be built stand alone into
a (at least) libzglob.a.

I realize that this is not trivial because the glob.c seems to
use a lot of general zsh structs and call other zsh functions,
ansi2knr, makepro.sh, for some things it may be useful to use
the "parent" software's (Perl in this case) function instead of
the zsh ones (for example garbage collection comes to mind).
But some of you zsh-workers is interested I am prepared to
co-operate with you in modularizing the Zsh Glob.

If this happens I can see one zsh API change need right now: glob()
should be 'thread-safe' in that the various glob options should be
passed in as arguments (a bitmask? a struct?) instead of being part of
the 'global' zsh state -- as there is no zsh opts[] if only the glob
lib is being used.

*) somebody might argue the a) point here :-)

-- 
$jhi++; # http://www.iki.fi/~jhi/
        # There is this special biologist word we use for 'stable'.
        # It is 'dead'. -- Jack Cohen

Re: libzglob?

[Zoltan Hidvegi]

> has there been any talk / thought / action as to separate the zsh
> globbing code into its own library?  (as Zle nowadays seems to be,
> sort of)

glob.c is quite independent of the rest of the code, I think it would not
be hard to make it a standalone library.

> I realize that this is not trivial because the glob.c seems to
> use a lot of general zsh structs and call other zsh functions,
> ansi2knr, makepro.sh, for some things it may be useful to use

ansi2knr is only needed for very old traditional compilers which cannot
compile programs with ANSI C prototypes.  Makepro is just a simple
prototype generator.

> If this happens I can see one zsh API change need right now: glob()
> should be 'thread-safe' in that the various glob options should be
> passed in as arguments (a bitmask? a struct?) instead of being part of
> the 'global' zsh state -- as there is no zsh opts[] if only the glob
> lib is being used.

There are several static variables used by glob.c, so if you want real
re-entrant glob, more changes are necessary.

The scanner in zsh glob can be used for perl, however the pattern
matching part has some problems.  One for example is that (ab#)# does not
match ab.  For perl, a zsh glob -> regexp translator would be much more
roboust and relyable.

Zoltan

Re: libzglob?

[Andrew Main]

Zoltan Hidvegi wrote:
>The scanner in zsh glob can be used for perl, however the pattern
>matching part has some problems.  One for example is that (ab#)# does not
>match ab.

Why?

>           For perl, a zsh glob -> regexp translator would be much more
>roboust and relyable.

But then in order to do actual file globbing, a second regexp execution
engine would be required.  This is not necessarily a bad approach --
it has the potential to allow globbing based on full Perl regexps --
but it might be better to stick with zsh's special purpose code.

-zefram

Re: libzglob?

[Zoltan T. Hidvegi]

Andrew Main wrote:
> Zoltan Hidvegi wrote:
> >The scanner in zsh glob can be used for perl, however the pattern
> >matching part has some problems.  One for example is that (ab#)# does not
> >match ab.
>
> Why?

Because there is no proper backtracking, the glob code sees that `a' alone
matches `ab#' and then it tries to match the rest of the string, `b' with
the rest of the glob pattern (which is nothing).

> >           For perl, a zsh glob -> regexp translator would be much more
> >roboust and relyable.
>
> But then in order to do actual file globbing, a second regexp execution
> engine would be required.  This is not necessarily a bad approach --
> it has the potential to allow globbing based on full Perl regexps --
> but it might be better to stick with zsh's special purpose code.

Yes, for zsh.  But the question was to use the zsh glob within perl, which
already have the regexp engine.  I'd thing that even for zsh the regexp
engine would be superior, but it is quite big, and zsh is big enough even
without regexps.  Of course a loadable regexp module can be added to zsh.

Zoli

Re: libzglob?

[Andrew Main]

Zoltan T. Hidvegi wrote:
>Yes, for zsh.  But the question was to use the zsh glob within perl, which
>already have the regexp engine.  I'd thing that even for zsh the regexp
>engine would be superior, but it is quite big, and zsh is big enough even
>without regexps.  Of course a loadable regexp module can be added to zsh.

That's not what I meant.  We're talking about doing globbing.  A normal
regexp engine answers questions of the form "does this string match this
pattern?", but for globbing we need an engine that can answer "what are
all the filenames that match this pattern?".  libzglob would really have
to contain its own globbing engine -- Perl currently doesn't have one,
though it has a good regexp engine.

-zefram

Re: [ZSH] libzglob?

[Chip Salzenberg]

According to Andrew Main:
> libzglob would really have to contain its own globbing engine ...

Quite.  I'm involved with Perl development, and I'd _really_ like to
see the hypothetical libzglob used in Perl, if only as an optional
extension.
-- 
Chip Salzenberg          - a.k.a. -           <chip@pobox.com>
 (Roller coaster on camera:)  "Wow, this square quarter mile
    has unbelievably good light rail transit!"  // MST3K

Re: libzglob?

[Zoltan T. Hidvegi]

Andrew Main wrote:
> Zoltan T. Hidvegi wrote:
> >Yes, for zsh.  But the question was to use the zsh glob within perl, which
> >already have the regexp engine.  I'd thing that even for zsh the regexp
> >engine would be superior, but it is quite big, and zsh is big enough even
> >without regexps.  Of course a loadable regexp module can be added to zsh.
>
> That's not what I meant.  We're talking about doing globbing.  A normal
> regexp engine answers questions of the form "does this string match this
> pattern?", but for globbing we need an engine that can answer "what are
> all the filenames that match this pattern?".  libzglob would really have
> to contain its own globbing engine -- Perl currently doesn't have one,
> though it has a good regexp engine.

But glob.c really has three main functions: pattern compiler to struct
comp, pattern matcher (domatch()) and scanner.  The scanner scans the
filesystem, and calls domatch() to see if a filename matches a pattern.
This is a bit simplified description, but not completely far from reality.
domatch() can be replaced by a regexp engine.

Zoltan