Re: [TUHS] v7 K&R C [really lexers]

Re: [TUHS] v7 K&R C [really lexers]

[Nelson H. F. Beebe]

Brantley Coile <brantley@coraid.com> wrote on Sun, 17 May 2020 01:36:16 +0000:

>> It looks like only grap and pic have mkfiles that invoke lex.

Both of those are Brian Kernighan's work, and from the FIXES file
in his nawk, I can offer this quote:

>> ...
>> Aug 9, 1997:
>> 	somewhat regretfully, replaced the ancient lex-based lexical
>> 	analyzer with one written in C.  it's longer, generates less code,
>> 	and more portable; the old one depended too much on mysterious
>> 	properties of lex that were not preserved in other environments.
>> 	in theory these recognize the same language.
>> ...

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Re: [TUHS] v7 K&R C [really lexers]

[Doug McIlroy]

Interesting. My "speak" program had a trivial lexer that
recognized literal tokens, many of which were prefixes
of others, by maximum-munch binary search in a list of
1600 entries. Entries gave token+translation+rewrite.
The whole thing fit in 15K.

Many years later I wrote a regex recognizer that special-cased
alternations of lots of literals. I believe Gnu's regex.c does
that, too. (My regex also supported conjunction and negation--
legitimate regular-language operations--implemented by
continuation-passing to avoid huge finite-state machines.)

We have here a case of imperfect communication in 1127. Had I
been conscious of the lex-explosion problem, I might have
thought of speak and put support for speak-like tables
into lex. As it happened, I only used yacc/lex once, quite
successfully, for a small domain-specific language. 

Doug

Steve Johnson wrote:

I also gave up on lex for parsing fairly early.   The problem was
reserved words.  These looked like identifiers, but the state machine to
pick out a couple of dozen reserved words out of all identifiers was too
big for the PDP-11.   When I wrote spell, I ran into the same problem.
I had some rules that wanted to convert plurals to singular forms that
would be found in the dictionary.   Writing a rule to recognize .*ies
and convert the "ies" to "y" blew out the memory after only a handful of
patterns.   My solution was to pick up words and reverse them before
passing them through lex, so I looked for the pattern "sei.*", converted
it to "y" and then reversed the word again.  As it turned out, I only
owned spell for a few weeks because Doug and others grabbed it and ran
with it.

Re: [TUHS] v7 K&R C

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 3180 bytes --]

Interesting that Go had only what you call "typed typdefs" until we needed
to add "untyped typedefs" so we could provide aliasing for forwarding
declarations. And that necessity made me unhappy. But the short version: Go
went the other way with what "typedef" means.

-rob


On Mon, May 11, 2020 at 10:28 AM <scj@yaccman.com> wrote:

> Following up on Rob's comment, I always took the point of view that Dennis
> owned the C description, and what he said goes.  Not that I didn't make
> suggestions that he accepted.  One of the better ones (actually in B) was ^
> for exclusive OR.  One of the worse ones was the syntax for casts.  We
> looked at about 5 different ideas and hated all of them.   And most of them
> couldn't be easily compiled with Yacc.  So I took the grammar for
> declarations, removed the variable name, and voila, it expressed everything
> we wanted in the way of semantics, had a simple rule of construction, and
> we badly needed the functionality for the Interdata port.   I quickly came
> to hate it, though -- the casts we were using looked like a teletype threw
> up in the middle of the code.
>
> With respect to enums, there is a feature I've wanted for years: a typed
> typedef.  Saying typetdef int foo would make foo an integer, but if you
> passed an ordinary int to something declared as foo it would be an error.
> Even if it was an integer constant unless cast.
>
> The amount of mechanism required to get that behavior from both C and C++
> is horrible, so far as I know, although C++ has accreted so much stuff
> maybe it's there now...
>
> Steve
> ---
>
>
>
> On 2020-04-24 19:54, Rob Pike wrote:
>
> Another debate at the time was caused by a disagreement between pcc and cc
> regarding enums: are they a type or just a way to declare constant? I
> remember getting annoyed by pcc not letting me declare a constant with an
> enum and use it as an int. I protested to scj and dmr and after some to-ing
> and fro-ing Steve changed pcc to treat them as constants.
>
> Not sure it was the right decision, but C desperately wanted a non-macro
> way to define a constant. I'd probably argue the same way today. The real
> lesson is how propinquity affects progress.
>
> -rbo
>
>
> On Sat, Apr 25, 2020 at 12:51 PM Rob Pike <robpike@gmail.com> wrote:
>
> The ability to call a function pointer fp with the syntax fp() rather than
> (*fp)() came rather late, I think at Bjarne's suggestion or example. Pretty
> sure it was not in v7 C, as you observe.
>
> Convenient though the shorthand may be, it always bothered me as
> inconsistent and misleading. (I am pretty sure I used it sometimes
> regardless.)
>
> -rob
>
>
> On Sat, Apr 25, 2020 at 12:48 PM Adam Thornton <athornton@gmail.com>
> wrote:
>
>
>
> On Apr 24, 2020, at 7:37 PM, Charles Anthony <charles.unix.pro@gmail.com>
> wrote:
>
>
>
> On Fri, Apr 24, 2020 at 7:00 PM Adam Thornton <athornton@gmail.com> wrote:
>
> This doesn't like the function pointer.
>
>
> $ cc -c choparg.c
> choparg.c:11: Call of non-function
>
>
> Perhaps:
>
>     (*fcn)(arg);
>
>
> We have a winner!
>
> Also, Kartik, dunno where it is on the net, but if you install a v7
> system, /usr/src/cmd/c
>
> Adam
>
>

[-- Attachment #2: Type: text/html, Size: 5575 bytes --]

Re: [TUHS] v7 K&R C

[Larry McVoy]

My mail is screwed up, I see Rob's reply to Steve but didn't see Steve's
original.

> On Mon, May 11, 2020 at 10:28 AM <scj@yaccman.com> wrote:
> > With respect to enums, there is a feature I've wanted for years: a typed
> > typedef.  Saying typetdef int foo would make foo an integer, but if you
> > passed an ordinary int to something declared as foo it would be an error.
> > Even if it was an integer constant unless cast.

Steve, I couldn't agree more, you are 100% right, this is how it should
work.  I wanted to like enums because I naively thought they'd have these
semantics but then learned they really aren't any different than a well
managed list of #defines.

IMHO, without your semantics, enums are pretty useless, #define is good
enough and more clear.

--lm

Re: [TUHS] v7 K&R C

[Greg A. Woods]

[-- Attachment #1: Type: text/plain, Size: 1971 bytes --]

At Sun, 10 May 2020 17:57:46 -0700, Larry McVoy <lm@mcvoy.com> wrote:
Subject: Re: [TUHS] v7 K&R C
>
> > On Mon, May 11, 2020 at 10:28 AM <scj@yaccman.com> wrote:
> > > With respect to enums, there is a feature I've wanted for years: a typed
> > > typedef.  Saying typetdef int foo would make foo an integer, but if you
> > > passed an ordinary int to something declared as foo it would be an error.
> > > Even if it was an integer constant unless cast.
>
> Steve, I couldn't agree more, you are 100% right, this is how it should
> work.  I wanted to like enums because I naively thought they'd have these
> semantics but then learned they really aren't any different than a well
> managed list of #defines.

Absolutely agreed!

The lameness of typedef (and in how enum is related to typedef) is one
of the saddest parts of C.  (The other is the default promotion to int.)

It would be trivial to fix too -- for a "new" C, that is.  Making it
backward compatible for legacy code would be tough, even with tooling to
help fix the worst issues.  I've seen far too much code that would be
hard to fix by hand, e.g. some that even goes so far as to assume things
like arithmetic on enum values will produce other valid enum values.

Ideally enums could be a value in any native type, including float/double.

> IMHO, without your semantics, enums are pretty useless, #define is good
> enough and more clear.

Actually that's no longer true with a good modern toolchain, especially
with respect to the debugger.  A good debugger can now show the enum
symbol for a (matching) value of a properly typedefed variable.

(In fact I never thouth that a #define macro was more clear, even before
debugger support -- the debugger support just gave me a better excuse to
use to explain my preference!)

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [TUHS] v7 K&R C

[Paul Winalski]

On 5/11/20, Greg A. Woods <woods@robohack.ca> wrote:
>
> The lameness of typedef (and in how enum is related to typedef) is one
> of the saddest parts of C.  (The other is the default promotion to int.)

I would add a third:  file-scope declarations being global by default.
One must use the keyword "static" to restrict a file-scope declaration
to the file it's declared in.  And why "static"?  All file-scope
declarations have static allocation.  Why isn't the keyword "local" or
"own"?  Anyway, the way it ought to be is that file-scope declarations
are restricted to the file they're declared in.  To make the symbol
visible outside its file, you should have to explicitly say "global".

> It would be trivial to fix too -- for a "new" C, that is.  Making it
> backward compatible for legacy code would be tough, even with tooling to
> help fix the worst issues.  I've seen far too much code that would be
> hard to fix by hand, e.g. some that even goes so far as to assume things
> like arithmetic on enum values will produce other valid enum values.

This ought to be easy to fix using a compiler command line option for
the legacy behavior.  Many C compilers do this already to support K&R
semantics vs. standard C semantics.

> Ideally enums could be a value in any native type, including float/double.

Except pointers, of course.

>> IMHO, without your semantics, enums are pretty useless, #define is good
>> enough and more clear.
>
> Actually that's no longer true with a good modern toolchain, especially
> with respect to the debugger.  A good debugger can now show the enum
> symbol for a (matching) value of a properly typedefed variable.

Indeed.

-Paul W.

Re: [TUHS] v7 K&R C

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]

On Mon, May 11, 2020 at 2:25 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> This ought to be easy to fix using a compiler command line option for
> the legacy behavior.  Many C compilers do this already to support K&R
> semantics vs. standard C semantics.
>
Hrrrumph

Point taken but ...

C++ is an example in my mind of not listening to Dennis' words:

   - “C is quirky, flawed, and an enormous success.”
   <https://www.inspiringquotes.us/quotes/2Rki_bGM7zqTA>
   - “When I read commentary about suggestions for where C should go, I
   often think back and give thanks that it wasn't developed under the advice
   of a worldwide crowd.”
   <https://www.inspiringquotes.us/quotes/eDQR_hqwtHAC9>
   - “A language that doesn't have everything is actually easier to program
   in than some that do”
   <https://www.inspiringquotes.us/quotes/R86z_ybYw9JTS>

[-- Attachment #2: Type: text/html, Size: 2197 bytes --]

Re: [TUHS] v7 K&R C

[Paul Winalski]

On 5/11/20, Clem Cole <clemc@ccc.com> wrote:
>
> C++ is an example in my mind of not listening to Dennis' words:
>
>    - “C is quirky, flawed, and an enormous success.”

Ditto Fortran.

>    - “When I read commentary about suggestions for where C should go, I
>    often think back and give thanks that it wasn't developed under the
> advice
>    of a worldwide crowd.”

The old saying of an elephant being a mouse designed by committee comes to mind.

Language standards committees tend to be like a pack of dogs
contemplating a tree.  Each dog isn't satisfied with the tree until
he's peed on it.

>    - “A language that doesn't have everything is actually easier to program
>    in than some that do”

Big, comprehensive languages such as PL/I, Ada, and C++ tend to have
more of their share of toxic language features--things that shouldn't
be used if you want reliable, easily maintained and understood code.
Ada failed for two reasons:  [1] it had cooties because of its
military origins, and [2] it collapsed under the weight of all of its
features.

-Paul W.

Re: [TUHS] v7 K&R C

[joe mcguckin]

Maybe it’s time for C++ subset ‘G'


Joe McGuckin
ViaNet Communications

joe@via.net
650-207-0372 cell
650-213-1302 office
650-969-2124 fax



> On May 11, 2020, at 12:12 PM, Paul Winalski <paul.winalski@gmail.com> wrote:
> 
> On 5/11/20, Clem Cole <clemc@ccc.com> wrote:
>> 
>> C++ is an example in my mind of not listening to Dennis' words:
>> 
>>   - “C is quirky, flawed, and an enormous success.”
> 
> Ditto Fortran.
> 
>>   - “When I read commentary about suggestions for where C should go, I
>>   often think back and give thanks that it wasn't developed under the
>> advice
>>   of a worldwide crowd.”
> 
> The old saying of an elephant being a mouse designed by committee comes to mind.
> 
> Language standards committees tend to be like a pack of dogs
> contemplating a tree.  Each dog isn't satisfied with the tree until
> he's peed on it.
> 
>>   - “A language that doesn't have everything is actually easier to program
>>   in than some that do”
> 
> Big, comprehensive languages such as PL/I, Ada, and C++ tend to have
> more of their share of toxic language features--things that shouldn't
> be used if you want reliable, easily maintained and understood code.
> Ada failed for two reasons:  [1] it had cooties because of its
> military origins, and [2] it collapsed under the weight of all of its
> features.
> 
> -Paul W.

Re: [TUHS] v7 K&R C

[Larry McVoy]

Isn't that effectively what companies do now?  Don't they all have a
"Here is what you can use, this and nothing else" doc?

On Mon, May 11, 2020 at 12:57:01PM -0700, joe mcguckin wrote:
> Maybe it???s time for C++ subset ???G'
> 
> 
> Joe McGuckin
> ViaNet Communications
> 
> joe@via.net
> 650-207-0372 cell
> 650-213-1302 office
> 650-969-2124 fax
> 
> 
> 
> > On May 11, 2020, at 12:12 PM, Paul Winalski <paul.winalski@gmail.com> wrote:
> > 
> > On 5/11/20, Clem Cole <clemc@ccc.com> wrote:
> >> 
> >> C++ is an example in my mind of not listening to Dennis' words:
> >> 
> >>   - ???C is quirky, flawed, and an enormous success.???
> > 
> > Ditto Fortran.
> > 
> >>   - ???When I read commentary about suggestions for where C should go, I
> >>   often think back and give thanks that it wasn't developed under the
> >> advice
> >>   of a worldwide crowd.???
> > 
> > The old saying of an elephant being a mouse designed by committee comes to mind.
> > 
> > Language standards committees tend to be like a pack of dogs
> > contemplating a tree.  Each dog isn't satisfied with the tree until
> > he's peed on it.
> > 
> >>   - ???A language that doesn't have everything is actually easier to program
> >>   in than some that do???
> > 
> > Big, comprehensive languages such as PL/I, Ada, and C++ tend to have
> > more of their share of toxic language features--things that shouldn't
> > be used if you want reliable, easily maintained and understood code.
> > Ada failed for two reasons:  [1] it had cooties because of its
> > military origins, and [2] it collapsed under the weight of all of its
> > features.
> > 
> > -Paul W.

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

Re: [TUHS] v7 K&R C

[Paul Winalski]

On 5/11/20, Larry McVoy <lm@mcvoy.com> wrote:
> Isn't that effectively what companies do now?  Don't they all have a
> "Here is what you can use, this and nothing else" doc?
>
> On Mon, May 11, 2020 at 12:57:01PM -0700, joe mcguckin wrote:
>> Maybe it???s time for C++ subset ???G'

Absolutely.  The projects that I ran effectively used C++ as a
stronger-typed version of C.  A small subset of C++ features were
allowed, but among the prohibited features were:

o multiple inheritance
o operator overloading
o friend classes
o C++ exception handling
o all std:: and STL functions

The last two of these are mainly for performance reasons.  throw and
catch play merry hell with compiler optimizations, especially of
global variables.

-Paul W.

Re: [TUHS] v7 K&R C

[Dave Horsfall]

On Tue, 12 May 2020, Paul Winalski wrote:

> Absolutely.  The projects that I ran effectively used C++ as a 
> stronger-typed version of C.  A small subset of C++ features were 
> allowed, but among the prohibited features were:

[...]

> o operator overloading

[...]

I never could figure out why Stroustrup implemented that "feature"; let's 
see, this operator usually means this, except when you use it in that 
situation in which case it means something else.  Now, try debugging that.

I had to learn C++ for a project at $WORK years ago (the client demanded 
it), and boy was I glad when I left...

-- Dave

Re: [TUHS] v7 K&R C

[Larry McVoy]

On Thu, May 14, 2020 at 09:36:57AM +1000, Dave Horsfall wrote:
> I had to learn C++ for a project at $WORK years ago (the client demanded
> it), and boy was I glad when I left...

Amen.  I'm being a whiney grumpy old man, but I'm sort of glad I'm at the
tail end of my career.  Going into it now, there are some bright spots,
and some dim ones, Go seems nice, Rust could have been nice but they just
had to come up with a different syntax, I can't see why anyone would do
anything other than an improved C like syntax, Java and C++ seem awful,
D tried but threw too much into the language like C++ did, if D had had
some restraint like Go does, D would probably be my language of choice.

Personally, I just want a modernized C.  If you want to see what I want
take a look at https://www.little-lang.org/

It's got some perl goodness, regexps are part of the syntax, switches
work on strings or regexps as well as constants, it's pleasant.  And 
completely doable as an extension to C.

Oh, and it has reference counting on auto allocated stuff so when it
goes out of scope, free() is automatic.  

--lm

Re: [TUHS] v7 K&R C

[Tony Finch]

Larry McVoy <lm@mcvoy.com> wrote:
>
> It's got some perl goodness, regexps are part of the syntax, ....

I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
riff on lex being its Swiss Army Knife. I came to appreciate lex
regrettably late: lex makes it remarkably easy to chew through a huge pile
of text and feed the pieces to some library code written in C. I've been
using re2c recently (http://re2c.org/), which is differently weird than
lex, though it still uses YY in all its variable names. It's remarkable
how much newer lexer/parser generators can't escape from the user
interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
Trafalgar: Cyclonic 6 to gale 8. Rough occasionally very rough in west and
south. Thundery showers. Good, occasionally poor.

Re: [TUHS] v7 K&R C

[Steffen Nurpmeso]

Tony Finch wrote in
<alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>:
 |Larry McVoy <lm@mcvoy.com> wrote:
 |>
 |> It's got some perl goodness, regexps are part of the syntax, ....
 |
 |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
 |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
 |riff on lex being its Swiss Army Knife. I came to appreciate lex
 |regrettably late: lex makes it remarkably easy to chew through a huge pile
 |of text and feed the pieces to some library code written in C. I've been
 |using re2c recently (http://re2c.org/), which is differently weird than
 |lex, though it still uses YY in all its variable names. It's remarkable
 |how much newer lexer/parser generators can't escape from the user
 |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/

P.S.: i really hate automated lexers.  I never ever got used to
use them.  For learning i once tried to use flex/bison, but
i failed really hard.  I like that blood, sweat and tears thing,
and using a lexer seems so shattered, all the pieces.  And i find
them really hard to read.

If you can deal with them they are surely a relief, especially in
rapidly moving syntax situations.  But if i look at settled source
code which uses it, for example usr.sbin/ospfd/parse.y, or
usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am
happy that i do not need to maintain that code.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] v7 K&R C [really lexers]

[Jon Steinhart]

Steffen Nurpmeso writes:
> Tony Finch wrote in
> <alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>:
>  |Larry McVoy <lm@mcvoy.com> wrote:
>  |>
>  |> It's got some perl goodness, regexps are part of the syntax, ....
>  |
>  |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
>  |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
>  |riff on lex being its Swiss Army Knife. I came to appreciate lex
>  |regrettably late: lex makes it remarkably easy to chew through a huge pile
>  |of text and feed the pieces to some library code written in C. I've been
>  |using re2c recently (http://re2c.org/), which is differently weird than
>  |lex, though it still uses YY in all its variable names. It's remarkable
>  |how much newer lexer/parser generators can't escape from the user
>  |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/
>
> P.S.: i really hate automated lexers.  I never ever got used to
> use them.  For learning i once tried to use flex/bison, but
> i failed really hard.  I like that blood, sweat and tears thing,
> and using a lexer seems so shattered, all the pieces.  And i find
> them really hard to read.
>
> If you can deal with them they are surely a relief, especially in
> rapidly moving syntax situations.  But if i look at settled source
> code which uses it, for example usr.sbin/ospfd/parse.y, or
> usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am
> happy that i do not need to maintain that code.
>
> --steffen

Wow, I've had the opposite experience.  I find lex/yacc/flex/bison really
easy to use.  The issue, which I believe was covered in the early docs,
is that some languages are not designed with regularity in mind which makes
for ugly code.  But to be fair, that code is at least as ugly with hand-crafted
code.

I believe that the original wisecrack was directed towards FORTRAN.  My ancient
experience was that it was using lex/yacc for HSPICE was not going to work so I
had to hand-craft code for that.

Jon

Re: [TUHS] v7 K&R C [really lexers]

[Brantley Coile]

[-- Attachment #1: Type: text/plain, Size: 2297 bytes --]

“The asteroid to kill this dinosaur is still in orbit.“

    —- Plan 9 lex man page

I always hand craft my lexers and use yacc to parse. Most  code on plan 9 does that as well.

  Brantley


On May 16, 2020, at 8:00 PM, Jon Steinhart <jon@fourwinds.com> wrote:

Steffen Nurpmeso writes:
Tony Finch wrote in
<alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>:
|Larry McVoy <lm@mcvoy.com> wrote:
|>
|> It's got some perl goodness, regexps are part of the syntax, ....
|
|I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
|Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
|riff on lex being its Swiss Army Knife. I came to appreciate lex
|regrettably late: lex makes it remarkably easy to chew through a huge pile
|of text and feed the pieces to some library code written in C. I've been
|using re2c recently (http://re2c.org/), which is differently weird than
|lex, though it still uses YY in all its variable names. It's remarkable
|how much newer lexer/parser generators can't escape from the user
|interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/

P.S.: i really hate automated lexers.  I never ever got used to
use them.  For learning i once tried to use flex/bison, but
i failed really hard.  I like that blood, sweat and tears thing,
and using a lexer seems so shattered, all the pieces.  And i find
them really hard to read.

If you can deal with them they are surely a relief, especially in
rapidly moving syntax situations.  But if i look at settled source
code which uses it, for example usr.sbin/ospfd/parse.y, or
usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am
happy that i do not need to maintain that code.

--steffen

Wow, I've had the opposite experience.  I find lex/yacc/flex/bison really
easy to use.  The issue, which I believe was covered in the early docs,
is that some languages are not designed with regularity in mind which makes
for ugly code.  But to be fair, that code is at least as ugly with hand-crafted
code.

I believe that the original wisecrack was directed towards FORTRAN.  My ancient
experience was that it was using lex/yacc for HSPICE was not going to work so I
had to hand-craft code for that.

Jon

[-- Attachment #2: Type: text/html, Size: 4835 bytes --]

Re: [TUHS] v7 K&R C [really lexers]

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 2646 bytes --]

On Sat, May 16, 2020, 6:05 PM Brantley Coile <brantley@coraid.com> wrote:

> “The asteroid to kill this dinosaur is still in orbit.“
>
>     —- Plan 9 lex man page
>
>
> I always hand craft my lexers and use yacc to parse. Most  code on plan 9
> does that as well.
>

Wow! That is the most awesome thing I've seen in a while....

Warner


  Brantley
>
>
> On May 16, 2020, at 8:00 PM, Jon Steinhart <jon@fourwinds.com> wrote:
>
> Steffen Nurpmeso writes:
>
> Tony Finch wrote in
>
> <alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>:
>
> |Larry McVoy <lm@mcvoy.com> wrote:
>
> |>
>
> |> It's got some perl goodness, regexps are part of the syntax, ....
>
> |
>
> |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
>
> |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
>
> |riff on lex being its Swiss Army Knife. I came to appreciate lex
>
> |regrettably late: lex makes it remarkably easy to chew through a huge pile
>
> |of text and feed the pieces to some library code written in C. I've been
>
> |using re2c recently (http://re2c.org/), which is differently weird than
>
> |lex, though it still uses YY in all its variable names. It's remarkable
>
> |how much newer lexer/parser generators can't escape from the user
>
> |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/
>
>
> P.S.: i really hate automated lexers.  I never ever got used to
>
> use them.  For learning i once tried to use flex/bison, but
>
> i failed really hard.  I like that blood, sweat and tears thing,
>
> and using a lexer seems so shattered, all the pieces.  And i find
>
> them really hard to read.
>
>
> If you can deal with them they are surely a relief, especially in
>
> rapidly moving syntax situations.  But if i look at settled source
>
> code which uses it, for example usr.sbin/ospfd/parse.y, or
>
> usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am
>
> happy that i do not need to maintain that code.
>
>
> --steffen
>
>
> Wow, I've had the opposite experience.  I find lex/yacc/flex/bison really
> easy to use.  The issue, which I believe was covered in the early docs,
> is that some languages are not designed with regularity in mind which makes
> for ugly code.  But to be fair, that code is at least as ugly with
> hand-crafted
> code.
>
> I believe that the original wisecrack was directed towards FORTRAN.  My
> ancient
> experience was that it was using lex/yacc for HSPICE was not going to work
> so I
> had to hand-craft code for that.
>
> Jon
>
>

[-- Attachment #2: Type: text/html, Size: 5765 bytes --]

Re: [TUHS] v7 K&R C [really lexers]

[Brantley Coile]

It looks like only grap and pic have mkfiles that invoke lex.


> On May 16, 2020, at 9:23 PM, Warner Losh <imp@bsdimp.com> wrote:
> 
> 
> 
> On Sat, May 16, 2020, 6:05 PM Brantley Coile <brantley@coraid.com> wrote:
> “The asteroid to kill this dinosaur is still in orbit.“
>     —- Plan 9 lex man page
> 
> I always hand craft my lexers and use yacc to parse. Most  code on plan 9 does that as well. 
> 
> Wow! That is the most awesome thing I've seen in a while....
> 
> Warner
> 
> 
>   Brantley
> 
> 
>> On May 16, 2020, at 8:00 PM, Jon Steinhart <jon@fourwinds.com> wrote:
>> 
>> Steffen Nurpmeso writes:
>>> Tony Finch wrote in
>>> <alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>:
>>> |Larry McVoy <lm@mcvoy.com> wrote:
>>> |>
>>> |> It's got some perl goodness, regexps are part of the syntax, ....
>>> |
>>> |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw
>>> |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a
>>> |riff on lex being its Swiss Army Knife. I came to appreciate lex
>>> |regrettably late: lex makes it remarkably easy to chew through a huge pile
>>> |of text and feed the pieces to some library code written in C. I've been
>>> |using re2c recently (http://re2c.org/), which is differently weird than
>>> |lex, though it still uses YY in all its variable names. It's remarkable
>>> |how much newer lexer/parser generators can't escape from the user
>>> |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/
>>> 
>>> P.S.: i really hate automated lexers.  I never ever got used to
>>> use them.  For learning i once tried to use flex/bison, but
>>> i failed really hard.  I like that blood, sweat and tears thing,
>>> and using a lexer seems so shattered, all the pieces.  And i find
>>> them really hard to read.
>>> 
>>> If you can deal with them they are surely a relief, especially in
>>> rapidly moving syntax situations.  But if i look at settled source
>>> code which uses it, for example usr.sbin/ospfd/parse.y, or
>>> usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am
>>> happy that i do not need to maintain that code.
>>> 
>>> --steffen
>> 
>> Wow, I've had the opposite experience.  I find lex/yacc/flex/bison really
>> easy to use.  The issue, which I believe was covered in the early docs,
>> is that some languages are not designed with regularity in mind which makes
>> for ugly code.  But to be fair, that code is at least as ugly with hand-crafted
>> code.
>> 
>> I believe that the original wisecrack was directed towards FORTRAN.  My ancient
>> experience was that it was using lex/yacc for HSPICE was not going to work so I
>> had to hand-craft code for that.
>> 
>> Jon

Re: [TUHS] v7 K&R C [really lexers]

[scj]

[-- Attachment #1: Type: text/plain, Size: 3284 bytes --]

I also gave up on lex for parsing fairly early.   The problem was
reserved words.  These looked like identifiers, but the state machine to
pick out a couple of dozen reserved words out of all identifiers was too
big for the PDP-11.   When I wrote spell, I ran into the same problem. 
I had some rules that wanted to convert plurals to singular forms that
would be found in the dictionary.   Writing a rule to recognize .*ies
and convert the "ies" to "y" blew out the memory after only a handful of
patterns.   My solution was to pick up words and reverse them before
passing them through lex, so I looked for the pattern "sei.*", converted
it to "y" and then reversed the word again.  As it turned out, I only
owned spell for a few weeks because Doug and others grabbed it and ran
with it... 

Steve

---

On 2020-05-16 18:23, Warner Losh wrote:

> On Sat, May 16, 2020, 6:05 PM Brantley Coile <brantley@coraid.com> wrote: 
> 
>> "The asteroid to kill this dinosaur is still in orbit."
>> 
>> --- Plan 9 lex man page
>> 
>> I always hand craft my lexers and use yacc to parse. Most  code on plan 9 does that as well.
> 
> Wow! That is the most awesome thing I've seen in a while.... 
> 
> Warner 
> 
> Brantley 
> 
> On May 16, 2020, at 8:00 PM, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> Steffen Nurpmeso writes:
> Tony Finch wrote in <alpine.DEB.2.20.2005142316170.3374@grey.csi.cam.ac.uk>: |Larry McVoy <lm@mcvoy.com> wrote: |> |> It's got some perl goodness, regexps are part of the syntax, .... | |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a |riff on lex being its Swiss Army Knife. I came to appreciate lex |regrettably late: lex makes it remarkably easy to chew through a huge pile |of text and feed the pieces to some library code written in C. I've been |using re2c recently (http://re2c.org/), which is differently weird than |lex, though it still uses YY in all its variable names. It's remarkable |how much newer lexer/parser generators can't escape from the user |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/ P.S.: i really hate automated lexers.  I never ever got used to use them.  For learning i once tried to use flex/bison, but i failed really hard.  I like
that blood, sweat and tears thing, and using a lexer seems so shattered, all the pieces.  And i find them really hard to read. If you can deal with them they are surely a relief, especially in rapidly moving syntax situations.  But if i look at settled source code which uses it, for example usr.sbin/ospfd/parse.y, or usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am happy that i do not need to maintain that code. --steffen 
> Wow, I've had the opposite experience.  I find lex/yacc/flex/bison really
> easy to use.  The issue, which I believe was covered in the early docs,
> is that some languages are not designed with regularity in mind which makes
> for ugly code.  But to be fair, that code is at least as ugly with hand-crafted
> code.
> 
> I believe that the original wisecrack was directed towards FORTRAN.  My ancient
> experience was that it was using lex/yacc for HSPICE was not going to work so I
> had to hand-craft code for that.
> 
> Jon

[-- Attachment #2: Type: text/html, Size: 8802 bytes --]

Re: [TUHS] v7 K&R C [really lexers]

[arnold]

scj@yaccman.com wrote:

> As it turned out, I only owned spell for a few weeks because Doug and
> others grabbed it and ran with it...
>
> Steve

Who else besides Doug worked on spell?

Do I understand correctly that you invented the original pipeline based
spell:

	tr ... |				# split words to one line
		tr ... | 			# lower case
			sort -u |		# sort and uniq
				comm -12 - dict	# find words not in dictionary

I didn't know that you'd worked on a rewrite in C.

Thanks,

Arnold

Re: [TUHS] v7 K&R C [really lexers]

[Richard Salz]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

Does anyone have the Usenix paper (80's timeframe I think) about making lex
go fast?  It was by Vern Paxson or Van Jacobson IIRC. Maybe only an
abstract was published.  The talk ended with a chart showing some CPU
times, and the modified lex was only slightly slower than cat. Maybe Vern
since he contributed to what became flex.

[-- Attachment #2: Type: text/html, Size: 362 bytes --]

Re: [TUHS] v7 K&R C [really lexers]

[Ralph Corderoy]

Hi Rich,

> Does anyone have the Usenix paper (80's timeframe I think) about making lex
> go fast?  It was by Vern Paxson or Van Jacobson IIRC. Maybe only an
> abstract was published.  The talk ended with a chart showing some CPU
> times, and the modified lex was only slightly slower than cat. Maybe Vern
> since he contributed to what became flex.

Google Scholar's ‘Vancouver’ style citation:

    Jacobson V. Tuning UNIX Lex or it's NOT true what they say
    about Lex.  InUSENIX Conference Proceedings (Washington,
    DC, Winter 1987) 1987 (pp. 163-164).

Lack of space after ‘In’ is Google, not me.

I didn't find the paper itself, just citations of it.

-- 
Cheers, Ralph.

Re: [TUHS] v7 K&R C [really lexers]

[arnold]

Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> Hi Rich,
>
> > Does anyone have the Usenix paper (80's timeframe I think) about making lex
> > go fast?  It was by Vern Paxson or Van Jacobson IIRC. Maybe only an
> > abstract was published.  The talk ended with a chart showing some CPU
> > times, and the modified lex was only slightly slower than cat. Maybe Vern
> > since he contributed to what became flex.
>
> Google Scholar's ‘Vancouver’ style citation:
>
>     Jacobson V. Tuning UNIX Lex or it's NOT true what they say
>     about Lex.  InUSENIX Conference Proceedings (Washington,
>     DC, Winter 1987) 1987 (pp. 163-164).
>
> Lack of space after ‘In’ is Google, not me.
>
> I didn't find the paper itself, just citations of it.

I'm pretty sure I have those proceedings. Given that it's 2 pages,
it's probably just an abstract. If there's interest, I can try to scan
the pages.

Arnold

Re: [TUHS] v7 K&R C [really lexers]

[Ralph Corderoy]

Hi Arnold,

> I'm pretty sure I have those proceedings. Given that it's 2 pages,
> it's probably just an abstract.

Yes, it's just the abstract, says
https://compilers.iecc.com/comparch/article/87-05-012

-- 
Cheers, Ralph.

Re: [TUHS] v7 K&R C [really lexers]

[Warren Toomey]

On Sun, Jun 14, 2020 at 03:48:19PM +0100, Ralph Corderoy wrote:
> Hi Arnold,
> 
> > I'm pretty sure I have those proceedings. Given that it's 2 pages,
> > it's probably just an abstract.
> 
> Yes, it's just the abstract, says
> https://compilers.iecc.com/comparch/article/87-05-012

Clem was kind enough to send the scan to me, and it's now here:
https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf

Cheers, Warren

Re: [TUHS] v7 K&R C [really lexers]

[Warren Toomey]

On Mon, Jun 15, 2020 at 11:12:30AM +1000, Warren Toomey wrote:
> Clem was kind enough to send the scan to me, and it's now here:
> https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf

John R. Levine might be a person to ask to see if he has a copy of
the paper.
	Warren

Re: [TUHS] v7 K&R C [really lexers]

[arnold]

Warren Toomey <wkt@tuhs.org> wrote:

> On Mon, Jun 15, 2020 at 11:12:30AM +1000, Warren Toomey wrote:
> > Clem was kind enough to send the scan to me, and it's now here:
> > https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf
>
> John R. Levine might be a person to ask to see if he has a copy of
> the paper.
> 	Warren

I've just done so, by posting to comp.compilers.

Arnold

Re: [TUHS] v7 K&R C [really lexers]

[Paul Winalski]

Regarding lex/yacc/flex/bison, I remember (ca. 1980) when DEC's
compiler group first got their hands on lex and yacc.  For yucks they
put the BLISS grammar through yacc.  It came back with an error
message that the grammar was ambiguous.  And it turned out that, yes,
Wulf's grammar for BLISS had an obscure corner case that *was*
ambiguous.  That caused quite a stir.

-Paul W.