[9fans] all you yacc experts

[9fans] all you yacc experts

[Scato Logic]

Let me get this straight. The goal is to get a successful full native build
of go under plan9. Currently, the build requires bison which no one wants
to port to plan9 but can be run via linuxemu. Brucee has offered to modify
his limbo version of yacc to make things work but that yacc will then need
to be run via the Inferno emu. Is this a comedy skit?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.9fans.net/private/9fans/attachments/20111114/a4b4e958/attachment.html>

[9fans] all you yacc experts

[Bruce Ellis]

Reread what I posted. I said that I will modify yacc for different
reasons and gave my reasons for doing so. I NEVER said RUN INFERNO.

Yes ron, what a lot of noise. And I'll SHOUT whenever I think it might
help, Mr aptly named Scatomancer.

brucee

On 14 November 2011 09:30, Scato Logic <go.scatology at gmail.com> wrote:
> Let me get this straight. The goal is to get a successful full native build
> of go under plan9. Currently, the build requires bison which no one wants to
> port to plan9 but can be run via linuxemu. Brucee has offered to modify his
> limbo version of yacc to make things work but that yacc will then need to be
> run via the Inferno emu. Is this a comedy skit?



-- 
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[9fans] all you yacc experts

[erik quanstrom]

On Fri Nov 11 11:08:13 EST 2011, rminnich at gmail.com wrote:
> Go is pretty solid on 386 and I'm slowly puzzling my way through NIX support.
> 
> One thing that stands in the way of full native build is the bison issue.
> 
> If somebody wants to look at enhancing yacc so that the extra bison
> bits can be supported, that would probably do the trick. I have no
> idea of the level of effort, I have not looked.
> 
> We could run bison under linuxemu; I don't think this approach is as
> good because it still leaves us a bit dependent on some outside force
> for bison binaries. But that might be a good stopgap.
> 
> I'm looking forward to Go 1 because I'm pretty sure that most of what
> we've had to do for this version of Go will go away :-)
> 
> I'm still amused that the best way to write portable C is to just ship
> a reasonable C compiler with Go and use gcc to build that C compiler,
> and then compile your portable C with it :-)

at one time the go .y wasn't bison dependent.  is the new dependency
a choice or requirement?

- erik

[9fans] all you yacc experts

[Lucio De Re]

> at one time the go .y wasn't bison dependent.  is the new dependency
> a choice or requirement?

It makes it possible to produce clever error messages, but it depends
on a bison extension.  I agree with Russ that it is worth it, but it
does throw a massive spanner in the works.

++L

[9fans] all you yacc experts

[ron minnich]

i'm well aware we can tweak go and remove better error messages. that
doesn't count IMHO. We're working to get to the point where we don't
tweak the go source at all and it just builds. I don't see the point
in making a go for Plan 9 that is in some way compromised. So, getting
back to my original comment, if somebody wants to dig into this it
would be helpful.

ron

[9fans] all you yacc experts

[Lucio De Re]

> i'm well aware we can tweak go and remove better error messages. that
> doesn't count IMHO. We're working to get to the point where we don't
> tweak the go source at all and it just builds. I don't see the point
> in making a go for Plan 9 that is in some way compromised. So, getting
> back to my original comment, if somebody wants to dig into this it
> would be helpful.
> 
Ron, I'm with you on that.

I'll be looking at bison before long.  Right now, I'm still trying to
catch up with your efforts, unfortunately on a parallel track rather
than in the same lane as you.

There are a few changes I want to submit to the Go Authors in the next
couple of days, then I'm going to compare my efforts with what you and
Andrey have been doing, try to find the middle ground that will save
everybody unnecessary pain.  Bison/yacc is just not urgent enough and
I have a suspicion that there are better qualified programmers out
there.  But if no one beats me to it, I'm prepared to get the work
done.

++L

PS: I'm a libmach build away from posting a mkfs archive to sources.
Probably sometime in the morning.

[9fans] all you yacc experts

[ron minnich]

Go is pretty solid on 386 and I'm slowly puzzling my way through NIX support.

One thing that stands in the way of full native build is the bison issue.

If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

We could run bison under linuxemu; I don't think this approach is as
good because it still leaves us a bit dependent on some outside force
for bison binaries. But that might be a good stopgap.

I'm looking forward to Go 1 because I'm pretty sure that most of what
we've had to do for this version of Go will go away :-)

I'm still amused that the best way to write portable C is to just ship
a reasonable C compiler with Go and use gcc to build that C compiler,
and then compile your portable C with it :-)

Thanks

ron

[9fans] all you yacc experts

[Steve Simon]

I believe go can be tweeked a little build correctly without
bison, but some of the error messages gc generates will be less
helpful.

for me the best solution would be for yacc to be modified
to produce the error message tables directly, rather than
mimicing bison - producing a debug file withch can be post
processed with awk.

-Steve

[9fans] all you yacc experts

[Bakul Shah]

On Fri, 11 Nov 2011 08:07:21 PST ron minnich <rminnich at gmail.com>  wrote:
> If somebody wants to look at enhancing yacc so that the extra bison
> bits can be supported, that would probably do the trick. I have no
> idea of the level of effort, I have not looked.

After some googling I see that src/cmd/gc/bisonerrors was
added by Russ in an evening to improve go error messages.

Clinton Jeffery in his "Generating LR syntax error messages
from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
-- not sure if this is the same paper as the one in TOPLAS)
says his Merr program works with Berkeley yacc, AT&T yacc &
bison.  He also refers to a modified byacc called iyacc.

Might it be worth looking Merr or iyacc?  Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

[9fans] all you yacc experts

[ron minnich]

On Fri, Nov 11, 2011 at 12:54 PM, Bakul Shah <bakul at bitblocks.com> wrote:

> Might it be worth looking Merr or iyacc? ?Porting bison to
> plan9 seems like a hugh punishment for a quick hack:-)
> Implementing Jeffery's directly in yacc might benefit other
> parsers as well.

If it's worth a look then I hope someone looks. I'm more trying to push that
than having this discussion :-)

I think porting bison is the wrong way to go, personally.

ron

[9fans] all you yacc experts

[Bakul Shah]

On Fri, 11 Nov 2011 12:58:30 PST ron minnich <rminnich at gmail.com>  wrote:
> On Fri, Nov 11, 2011 at 12:54 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> 
> > Might it be worth looking Merr or iyacc? Porting bison to
> > plan9 seems like a hugh punishment for a quick hack:-)
> > Implementing Jeffery's directly in yacc might benefit other
> > parsers as well.
>
> If it's worth a look then I hope someone looks. I'm more trying to push that
> than having this discussion :-)

I will take a look but don't hold your breath; my plate is
already full.

One further issue is whether go has any other bison
dependencies.

> I think porting bison is the wrong way to go, personally.

They lost me at configure.  Why should a parser generator need
on anything more than string.h, stdio,h and may be stdlib.h is
beyond me.  All it does is manipulate lists, sets and vectors!

[9fans] all you yacc experts

[Steve Simon]

> One further issue is whether go has any other bison
> dependencies.

as off march (when I last played with a port) it didn't,
there where a few bits of bison syntax which are different
in yacc but these could be papered over whith a few lines of
sed - ur yacc could even be taught this alternative syntax I suspose.

-Steve

[9fans] all you yacc experts

[Lucio De Re]

> One further issue is whether go has any other bison
> dependencies.

None as serious as the error message generation.  But your research is
invaluable, let's hope it leads in the right direction.

++L

[9fans] all you yacc experts

[Bruce Ellis]

I'm modifying yacc for other purposes, and yes I know how it works.

If bison is not too vomituos I will try and help.

brucee

On 12 November 2011 15:51, Lucio De Re <lucio at proxima.alt.za> wrote:
>> One further issue is whether go has any other bison
>> dependencies.
>
> None as serious as the error message generation. ?But your research is
> invaluable, let's hope it leads in the right direction.
>
> ++L
>
>
>



-- 
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[9fans] all you yacc experts

[Lucio De Re]

> I'm modifying yacc for other purposes, and yes I know how it works.
> 
Nobody could doubt that :-)

> If bison is not too vomituos I will try and help.

Don't overlook Bakul's suggestions, they may have much merit.  A quick
glance at my previous efforts suggests having to fight with the wide
character functions in Unix.  I seem to recall that fgb has done some
work in that direction?

++L

[9fans] all you yacc experts

[Lucio De Re]

> I think porting bison is the wrong way to go, personally.

Having tried it, I can only agree.  But I'm going to give it another
go, if only to remind myself why I didn't complete the task the first
time 'round.

++L

[9fans] all you yacc experts

[Iruatã Souza]

On Fri, Nov 11, 2011 at 6:54 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> On Fri, 11 Nov 2011 08:07:21 PST ron minnich <rminnich at gmail.com> ?wrote:
>> If somebody wants to look at enhancing yacc so that the extra bison
>> bits can be supported, that would probably do the trick. I have no
>> idea of the level of effort, I have not looked.
>
> After some googling I see that src/cmd/gc/bisonerrors was
> added by Russ in an evening to improve go error messages.
>
> Clinton Jeffery in his "Generating LR syntax error messages
> from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
> -- not sure if this is the same paper as the one in TOPLAS)
> says his Merr program works with Berkeley yacc, AT&T yacc &
> bison. ?He also refers to a modified byacc called iyacc.
>
> Might it be worth looking Merr or iyacc? ?Porting bison to
> plan9 seems like a hugh punishment for a quick hack:-)
> Implementing Jeffery's directly in yacc might benefit other
> parsers as well.
>
>

Sorry if it obvious, but Russ posted about the improvement on Go
errors: http://research.swtch.com/2010/01/generating-good-syntax-errors.html.

iru

[9fans] all you yacc experts

[Bakul Shah]

On Fri, 11 Nov 2011 12:54:28 PST Bakul Shah <bakul at bitblocks.com>  wrote:
> On Fri, 11 Nov 2011 08:07:21 PST ron minnich <rminnich at gmail.com>  wrote:
> > If somebody wants to look at enhancing yacc so that the extra bison
> > bits can be supported, that would probably do the trick. I have no
> > idea of the level of effort, I have not looked.
> 
> After some googling I see that src/cmd/gc/bisonerrors was
> added by Russ in an evening to improve go error messages.
> 
> Clinton Jeffery in his "Generating LR syntax error messages
> from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
> -- not sure if this is the same paper as the one in TOPLAS)
> says his Merr program works with Berkeley yacc, AT&T yacc &
> bison.  He also refers to a modified byacc called iyacc.
> 
> Might it be worth looking Merr or iyacc?  Porting bison to
> plan9 seems like a hugh punishment for a quick hack:-)
> Implementing Jeffery's directly in yacc might benefit other
> parsers as well.

Reading Merr code was useful but it is written in Icon so....

After staring at go.y, bisonerrors, go.errors, bison's
y.output, and generated yerr.h for a while I think I
understand what is going on.  I believe everything needed is
already in plan9's y.output -- no need to change yacc.  It may
be easier to just make bisonerrors handle this than convert
plan9's y.output to bison compatible y.output.  Of course,
things may look much bleaker in the morning but this is what I
think now!

Given that an array of <state, terminal, message> is generated
from y.output & go.errors, it is may be worth integrating this
logic in yacc. But that is for another day.

Ideally something like go.errors is interactively generated
from actual errors (sort of like how a spellchecker works --
you are presented with an error + its context and you indicate
the error message &/or fix). This is then fed back to yacc
along with the grammar.

[9fans] all you yacc experts

[Bruce Ellis]

If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

On my list.

brucee

On 12 November 2011 20:23, Bakul Shah <bakul at bitblocks.com> wrote:
> On Fri, 11 Nov 2011 12:54:28 PST Bakul Shah <bakul at bitblocks.com> ?wrote:
>> On Fri, 11 Nov 2011 08:07:21 PST ron minnich <rminnich at gmail.com> ?wrote:
>> > If somebody wants to look at enhancing yacc so that the extra bison
>> > bits can be supported, that would probably do the trick. I have no
>> > idea of the level of effort, I have not looked.
>>
>> After some googling I see that src/cmd/gc/bisonerrors was
>> added by Russ in an evening to improve go error messages.
>>
>> Clinton Jeffery in his "Generating LR syntax error messages
>> from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
>> -- not sure if this is the same paper as the one in TOPLAS)
>> says his Merr program works with Berkeley yacc, AT&T yacc &
>> bison. ?He also refers to a modified byacc called iyacc.
>>
>> Might it be worth looking Merr or iyacc? ?Porting bison to
>> plan9 seems like a hugh punishment for a quick hack:-)
>> Implementing Jeffery's directly in yacc might benefit other
>> parsers as well.
>
> Reading Merr code was useful but it is written in Icon so....
>
> After staring at go.y, bisonerrors, go.errors, bison's
> y.output, and generated yerr.h for a while I think I
> understand what is going on. ?I believe everything needed is
> already in plan9's y.output -- no need to change yacc. ?It may
> be easier to just make bisonerrors handle this than convert
> plan9's y.output to bison compatible y.output. ?Of course,
> things may look much bleaker in the morning but this is what I
> think now!
>
> Given that an array of <state, terminal, message> is generated
> from y.output & go.errors, it is may be worth integrating this
> logic in yacc. But that is for another day.
>
> Ideally something like go.errors is interactively generated
> from actual errors (sort of like how a spellchecker works --
> you are presented with an error + its context and you indicate
> the error message &/or fix). This is then fed back to yacc
> along with the grammar.
>
>



-- 
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[9fans] all you yacc experts

[Lucio De Re]

> If all the information is available in y.output then surely modifying
> yacc will cut out the middle man.
> 
That's not quite how I remember it, I'm sure that there was no trivial
way to alter the output from yacc to conform to the needs of Russ'
error messages.  That said, even if y.output does include the
essentials, it would have been extremly insightful of Russ to design a
message format that would be worth including in yacc itself.  It
certainly isn't the impression I got from looking at the available
documentation and sources,

I don't want to rain on anybody's parade, but I think a middle man
provides more flexibility and allows the developers on the yacc side
to focus on generating the type of output that makes the most sense,
rather than what the Go toolchain requires.

> On my list.

Ron isn't the only developer that will benefit from your efforts, I'd
like to show my appreciation as well.

++L

[9fans] all you yacc experts

[Bruce Ellis]

my motivation is that the limbo version of yacc is out of date
severely (doesn't cope with cc.y) and it seems like a good time to
have a 5c in limbo that copes with all and sundry arm variants - two
more were released this week - with loadable arch modules. and to
write shorter sentences.

brucee

On 13 November 2011 15:48, Lucio De Re <lucio at proxima.alt.za> wrote:
>> If all the information is available in y.output then surely modifying
>> yacc will cut out the middle man.
>>
> That's not quite how I remember it, I'm sure that there was no trivial
> way to alter the output from yacc to conform to the needs of Russ'
> error messages. ?That said, even if y.output does include the
> essentials, it would have been extremly insightful of Russ to design a
> message format that would be worth including in yacc itself. ?It
> certainly isn't the impression I got from looking at the available
> documentation and sources,
>
> I don't want to rain on anybody's parade, but I think a middle man
> provides more flexibility and allows the developers on the yacc side
> to focus on generating the type of output that makes the most sense,
> rather than what the Go toolchain requires.
>
>> On my list.
>
> Ron isn't the only developer that will benefit from your efforts, I'd
> like to show my appreciation as well.
>
> ++L
>
>
>



-- 
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[9fans] all you yacc experts

[Steve Simon]

Perhaps I'am way off base but surely the neatest solution would be to
modify yacc to produce a nice clean message format, modify the gc
to use that format.

The reason for not using just the same format blindly is it was designed 
with the structure of gc in mind rather than a clean output format.

Who knows, by delving into yacc perhaps more or even better error
status can be extracted.

Then we just need to convince russ/the go authors to include plan9's yacc
as part of the go distribution, it wouldn't be unprecendented as go already
includes 8c, 8l and friends.

-Steve

[9fans] all you yacc experts

[Bruce Ellis]

that's what i said ...

If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

brucee

On 13 November 2011 21:08, Steve Simon <steve at quintile.net> wrote:
> Perhaps I'am way off base but surely the neatest solution would be to
> modify yacc to produce a nice clean message format, modify the gc
> to use that format.
>
> The reason for not using just the same format blindly is it was designed
> with the structure of gc in mind rather than a clean output format.
>
> Who knows, by delving into yacc perhaps more or even better error
> status can be extracted.
>
> Then we just need to convince russ/the go authors to include plan9's yacc
> as part of the go distribution, it wouldn't be unprecendented as go already
> includes 8c, 8l and friends.
>
> -Steve
>
>



-- 
Don't meddle in the mouth -- MVS (0416935147, +1-513-3BRUCEE)

[9fans] all you yacc experts

[ron minnich]

The bisonerrors script is 124 lines. This email thread is now 724
lines. I figure when the number of lines of this thread is 10x the
size of the bisonerrors script it would be nice to replace the endless
discussion with some code :-)

ron

Re: [9fans] all you yacc experts

[Anthony Martin]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

Attached is a modified version of p9p yacc that
supports the Go grammar.  I'll be sending a
version of Plan 9 yacc later today.

The following is a description of the changes.

  1. The %error-verbose directive is ignored.

  2. A description of the final grammar is
     printed before the state descriptions
	 in y.output.

  3. The 'x' format for character literals is
     now used instead of prefixing with a space.

  4. The YYEMPTY define is now used to clear
     the lookahead token (instead of an explicit
	 negative one).

  5. Make yychar and yystate globals so they
     can be inspected by external code.

  5. Support C++ style // comments in actions.

  6. Add a usage message.

  7. Fix a few uses of sprint and strcpy.


I've also sent out a changeset to the Go
development list which adds support for
using Plan 9 yacc to generate the special
errors.

One tiny nit is that Plan 9 uses the name
yytoknames for debugging where Bison uses
yytname.  I've just used sed for this.

Any questions?
  Anthony

[-- Attachment #2: yacc.diff --]
[-- Type: text/plain, Size: 11953 bytes --]

diff -r 44a7194d00cf lib/yaccpar
--- a/lib/yaccpar	Sat Nov 12 11:52:10 2011 -0800
+++ b/lib/yaccpar	Tue Nov 15 13:10:13 2011 -0800
@@ -2,7 +2,7 @@
 #define YYERROR		goto yyerrlab
 #define YYACCEPT	return(0)
 #define YYABORT		return(1)
-#define	yyclearin	yychar = -1
+#define	yyclearin	yychar = YYEMPTY
 #define	yyerrok		yyerrflag = 0

 #ifdef	yydebug
@@ -51,6 +51,8 @@
 	return x;
 }

+long yychar;
+
 static long
 #ifdef YYARG
 yylex1(struct Yyarg *yyarg)
@@ -58,7 +60,6 @@
 yylex1(void)
 #endif
 {
-	long yychar;
 	const long *t3p;
 	int c;

@@ -68,6 +69,7 @@
 	yychar = yylex();
 #endif
 	if(yychar <= 0) {
+		yychar = 0;
 		c = yytok1[0];
 		goto out;
 	}
@@ -99,6 +101,8 @@
 	return c;
 }

+int yystate;
+
 int
 #ifdef YYARG
 yyparse(struct Yyarg *yyarg)
@@ -112,8 +116,8 @@
 		int	yys;
 	} yys[YYMAXDEPTH], *yyp, *yypt;
 	const short *yyxi;
-	int yyj, yym, yystate, yyn, yyg;
-	long yychar;
+	int yyj, yym, yyn, yyg;
+	long yyc;
 #ifndef YYARG
 	YYSTYPE save1, save2;
 	int save3, save4;
@@ -125,7 +129,8 @@
 #endif

 	yystate = 0;
-	yychar = -1;
+	yychar = YYEMPTY;
+	yyc = YYEMPTY;
 	yynerrs = 0;
 	yyerrflag = 0;
 	yyp = &yys[-1];
@@ -151,7 +156,7 @@
 yystack:
 	/* put a state and value onto the stack */
 	if(yydebug >= 4)
-		fprint(2, "char %s in %s", yytokname(yychar), yystatname(yystate));
+		fprint(2, "char %s in %s", yytokname(yyc), yystatname(yystate));

 	yyp++;
 	if(yyp >= &yys[YYMAXDEPTH]) {
@@ -165,18 +170,19 @@
 	yyn = yypact[yystate];
 	if(yyn <= YYFLAG)
 		goto yydefault; /* simple state */
-	if(yychar < 0)
+	if(yyc < 0)
 #ifdef YYARG
-		yychar = yylex1(yyarg);
+		yyc = yylex1(yyarg);
 #else
-		yychar = yylex1();
+		yyc = yylex1();
 #endif
-	yyn += yychar;
+	yyn += yyc;
 	if(yyn < 0 || yyn >= YYLAST)
 		goto yydefault;
 	yyn = yyact[yyn];
-	if(yychk[yyn] == yychar) { /* valid shift */
-		yychar = -1;
+	if(yychk[yyn] == yyc) { /* valid shift */
+		yyc = YYEMPTY;
+		yychar = YYEMPTY;
 		yyval = yylval;
 		yystate = yyn;
 		if(yyerrflag > 0)
@@ -188,11 +194,11 @@
 	/* default state action */
 	yyn = yydef[yystate];
 	if(yyn == -2) {
-		if(yychar < 0)
+		if(yyc < 0)
 #ifdef YYARG
-		yychar = yylex1(yyarg);
+			yyc = yylex1(yyarg);
 #else
-		yychar = yylex1();
+			yyc = yylex1();
 #endif

 		/* look through exception table */
@@ -201,21 +207,24 @@
 				break;
 		for(yyxi += 2;; yyxi += 2) {
 			yyn = yyxi[0];
-			if(yyn < 0 || yyn == yychar)
+			if(yyn < 0 || yyn == yyc)
 				break;
 		}
 		yyn = yyxi[1];
-		if(yyn < 0)
+		if(yyn < 0) {
+			yyc = YYEMPTY;
+			yychar = YYEMPTY;
 			goto ret0;
+		}
 	}
 	if(yyn == 0) {
 		/* error ... attempt to resume parsing */
 		switch(yyerrflag) {
 		case 0:   /* brand new error */
 			yyerror("syntax error");
-			if(yydebug >= 1) {
+			if(yydebug >= 2) {
 				fprint(2, "%s", yystatname(yystate));
-				fprint(2, "saw %s\n", yytokname(yychar));
+				fprint(2, "saw %s\n", yytokname(yyc));
 			}
 			goto yyerrlab;
 		yyerrlab:
@@ -245,10 +254,11 @@

 		case 3:  /* no shift yet; clobber input char */
 			if(yydebug >= 2)
-				fprint(2, "error recovery discards %s\n", yytokname(yychar));
-			if(yychar == YYEOFCODE)
+				fprint(2, "error recovery discards %s\n", yytokname(yyc));
+			if(yyc == YYEOFCODE)
 				goto ret1;
-			yychar = -1;
+			yyc = YYEMPTY;
+			yychar = YYEMPTY;
 			goto yynewstate;   /* try again in the same state */
 		}
 	}
diff -r 44a7194d00cf src/cmd/yacc.c
--- a/src/cmd/yacc.c	Sat Nov 12 11:52:10 2011 -0800
+++ b/src/cmd/yacc.c	Tue Nov 15 13:10:13 2011 -0800
@@ -92,6 +92,7 @@
 	TYPEDEF,
 	TYPENAME,
 	UNION,
+	IGNORE,

 	ENDFILE		= 0,

@@ -319,6 +320,9 @@
 	"token",	TERM,
 	"type",		TYPEDEF,
 	"union",	UNION,
+
+	/* ignored bison directives */
+	"error-verbose",	IGNORE,
 	0,
 };

@@ -330,6 +334,7 @@
 char*	writem(int*);
 char*	symnam(int);
 void	summary(void);
+void	grammar(void);
 void	error(char*, ...);
 void	aryfil(int*, int, int);
 int	setunion(int*, int*);
@@ -388,6 +393,7 @@
 	cempty();		/* make a table of which nonterminals can match the empty string */
 	cpfir();		/* make a table of firsts of nonterminals */
 	stagen();		/* generate the states */
+	grammar();
 	output();		/* write the states and the tables */
 	go2out();
 	hideprod();
@@ -531,14 +537,14 @@
 		;
 	p = prdptr[-*p];
 	q = chcopy(sarr, nontrst[*p-NTBASE].name);
-	q = chcopy(q, ": ");
+	q = chcopy(q, ":");
 	for(;;) {
 		*q = ' ';
 		p++;
-		if(p == pp)
-			*q = '.';
 		q++;
 		*q = '\0';
+		if(p == pp)
+			q = chcopy(q, ". ");
 		i = *p;
 		if(i <= 0)
 			break;
@@ -550,7 +556,7 @@
 	/* an item calling for a reduction */
 	i = *pp;
 	if(i < 0 ) {
-		q = chcopy(q, "    (");
+		q = chcopy(q, "   (");
 		sprint(q, "%d)", -i);
 	}
 	return sarr;
@@ -562,12 +568,41 @@
 char*
 symnam(int i)
 {
-	char* cp;
+	return (i >= NTBASE)? nontrst[i-NTBASE].name: tokset[i].name;
+}

-	cp = (i >= NTBASE)? nontrst[i-NTBASE].name: tokset[i].name;
-	if(*cp == ' ')
-		cp++;
-	return cp;
+/*
+ * output the grammar rules on y.output
+ */
+void
+grammar(void)
+{
+	int i, j, n, prev;
+	int *p;
+
+	if(foutput == 0)
+		return;
+
+	Bprint(foutput, "\nGrammar\n");
+	prev = 0;
+	n = 0;
+	PLOOP(0, i) {
+		p = prdptr[i];
+		j = 0;
+		if(p[0] == prev)
+			Bprint(foutput, "\t%d \t|", n++);
+		else {
+			Bprint(foutput, "\n\t%d %s:", n++, symnam(p[0]));
+			if(p[1] <= 0)
+				Bprint(foutput, " /* empty */");
+		}
+		for(j = 1; p[j] > 0; j++) {
+			Bprint(foutput, " %s", symnam(p[j]));
+		}
+		Bputc(foutput, '\n');
+		prev = p[0];
+	}
+	Bprint(foutput, "\n");
 }

 /*
@@ -1173,10 +1208,17 @@
 }

 void
+usage(void)
+{
+	fprint(2, "usage: yacc [-Dn] [-vdS] [-o outputfile] [-s stem] grammar\n");
+	exits("usage");
+}
+
+void
 setup(int argc, char *argv[])
 {
 	long c, t;
-	int i, j, fd, lev, ty, ytab, *p;
+	int i, j, lev, ty, ytab, *p;
 	int vflag, dflag, stem;
 	char actnm[8], *stemc, *s, dirbuf[128];
 	Biobuf *fout;
@@ -1195,7 +1237,7 @@
 		vflag++;
 		break;
 	case 'D':
-		yydebug = ARGF();
+		yydebug = EARGF(usage());
 		break;
 	case 'a':
 		yyarg = 1;
@@ -1208,7 +1250,7 @@
 		break;
 	case 'o':
 		ytab++;
-		ytabc = ARGF();
+		ytabc = EARGF(usage());
 		break;
 	case 's':
 		stem++;
@@ -1221,18 +1263,11 @@
 		error("illegal option: %c", ARGC());
 	}ARGEND
 	openup(stemc, dflag, vflag, ytab, ytabc);
-	fout = dflag?fdefine:ftable;
-	if(yyarg){
+	if(yyarg)
 		Bprint(ftable, "#define\tYYARG\t1\n\n");
-	}
-	if((fd = mkstemp(ttempname)) >= 0){
-		tempname = ttempname;
-		ftemp = Bfdopen(fd, OWRITE);
-	}
-	if((fd = mkstemp(tactname)) >= 0){
-		actname = tactname;
-		faction = Bfdopen(fd, OWRITE);
-	}
+
+	ftemp = Bopen(tempname = mktemp(ttempname), OWRITE);
+	faction = Bopen(actname = mktemp(tactname), OWRITE);
 	if(ftemp == 0 || faction == 0)
 		error("cannot open temp file");
 	if(argc < 1)
@@ -1375,6 +1410,10 @@
 		t = gettok();
 		continue;

+	case IGNORE:
+		t = gettok();
+		continue;
+
 	default:
 		error("syntax error");
 	}
@@ -1396,8 +1435,11 @@
 		Bprint(ftable, "YYSTYPE	yylval;\n");
 		Bprint(ftable, "YYSTYPE	yyval;\n");
 	}else{
-		if(dflag)
+		fout = ftable;
+		if(dflag){
+			fout = fdefine;
 			Bprint(ftable, "#include \"%s.%s\"\n\n", stemc, FILED);
+		}
 		Bprint(fout, "struct Yyarg {\n");
 		Bprint(fout, "\tint\tyynerrs;\n");
 		Bprint(fout, "\tint\tyyerrflag;\n");
@@ -1406,6 +1448,7 @@
 		Bprint(fout, "\tYYSTYPE\tyylval;\n");
 		Bprint(fout, "};\n\n");
 	}
+
 	prdptr[0] = mem;

 	/* added production */
@@ -1552,6 +1595,7 @@
 	Bterm(faction);
 	Bprint(ftable, "#define YYEOFCODE %d\n", 1);
 	Bprint(ftable, "#define YYERRCODE %d\n", 2);
+	Bprint(ftable, "#define YYEMPTY (%d)\n", -2);
 }

 /*
@@ -1581,17 +1625,17 @@

 	/* establish value for token */
 	/* single character literal */
-	if(s[0] == ' ') {
+	if(s[0] == '\'') {
 		val = chartorune(&rune, &s[1]);
-		if(s[val+1] == 0) {
+		if(s[val+1] == '\'') {
 			val = rune;
 			goto out;
 		}
 	}

 	/* escape sequence */
-	if(s[0] == ' ' && s[1] == '\\') {
-		if(s[3] == 0) {
+	if(s[0] == '\'' && s[1] == '\\') {
+		if(s[3] == '\'') {
 			/* single character escape sequence */
 			switch(s[2]) {
 			case 'n':	val = '\n'; break;
@@ -1625,6 +1669,7 @@
 	val = extval++;

 out:
+	//print("%s = %d\n", s, val);
 	tokset[ntokens].value = val;
 	toklev[ntokens] = 0;
 	return ntokens;
@@ -1642,7 +1687,7 @@
 	for(i=ndefout; i<=ntokens; i++) {
 		/* non-literals */
 		c = tokset[i].name[0];
-		if(c != ' ' && c != '$') {
+		if(c != '\'' && c != '$') {
 			Bprint(ftable, "#define	%s	%d\n",
 				tokset[i].name, tokset[i].value);
 			if(fdefine)
@@ -1737,7 +1782,7 @@
 	case '"':
 	case '\'':
 		match = c;
-		tokname[0] = ' ';
+		tokname[0] = '\'';
 		i = 1;
 		for(;;) {
 			c = Bgetrune(finput);
@@ -1756,6 +1801,8 @@
 			if(i < NAMESIZE)
 				i += c;
 		}
+		tokname[i] = '\'';
+		i++;
 		break;

 	case '%':
@@ -1847,7 +1894,7 @@
 {
 	int i;

-	if(s[0] == ' ')
+	if(s[0] == '\'')
 		t = 0;
 	TLOOP(i)
 		if(!strcmp(s, tokset[i].name))
@@ -1915,6 +1962,7 @@
 void
 cpycode(void)
 {
+
 	long c;

 	c = Bgetrune(finput);
@@ -1956,17 +2004,22 @@

 	/* i is the number of lines skipped */
 	i = 0;
-	if(Bgetrune(finput) != '*')
+	c = Bgetrune(finput);
+	if(c == '/'){			/* C++ //: skip to end of line */
+		while((c = Bgetrune(finput)) != Beof)
+			if(c == '\n')
+				return 1;
+	}else if(c == '*'){		/* normal C comment */
+		while((c = Bgetrune(finput)) != Beof) {
+			while(c == '*')
+				if((c = Bgetrune(finput)) == '/')
+					return i;
+			if(c == '\n')
+				i++;
+		}
+	}else
 		error("illegal comment");
-	c = Bgetrune(finput);
-	while(c != Beof) {
-		while(c == '*')
-			if((c=Bgetrune(finput)) == '/')
-				return i;
-		if(c == '\n')
-			i++;
-		c = Bgetrune(finput);
-	}
+
 	error("EOF inside comment");
 	return 0;
 }
@@ -2094,22 +2147,30 @@
 		/* look for comments */
 		Bputrune(faction, c);
 		c = Bgetrune(finput);
-		if(c != '*')
+		switch(c) {
+		case '/':
+			while(c != Beof) {
+				if(c == '\n')
+					goto swt;
+				Bputrune(faction, c);
+				c = Bgetrune(finput);
+			}
+			break;
+		case '*':
+			while(c != Beof) {
+				while(c == '*') {
+					Bputrune(faction, c);
+					if((c = Bgetrune(finput)) == '/')
+						goto lcopy;
+				}
+				Bputrune(faction, c);
+				if(c == '\n')
+					lineno++;
+				c = Bgetrune(faction);
+			}
+			break;
+		default:
 			goto swt;
-
-		/* it really is a comment */
-		Bputrune(faction, c);
-		c = Bgetrune(finput);
-		while(c >= 0) {
-			while(c == '*') {
-				Bputrune(faction, c);
-				if((c=Bgetrune(finput)) == '/')
-					goto lcopy;
-			}
-			Bputrune(faction, c);
-			if(c == '\n')
-				lineno++;
-			c = Bgetrune(finput);
 		}
 		error("EOF inside comment");

@@ -2158,26 +2219,26 @@
 	char buf[256];

 	if(vflag) {
-		sprint(buf, "%s.%s", stem, FILEU);
+		snprint(buf, sizeof buf, "%s.%s", stem, FILEU);
 		foutput = Bopen(buf, OWRITE);
 		if(foutput == 0)
 			error("cannot open %s", buf);
 	}
 	if(yydebug) {
-		sprint(buf, "%s.%s", stem, FILEDEBUG);
+		snprint(buf, sizeof buf, "%s.%s", stem, FILEDEBUG);
 		if((fdebug = Bopen(buf, OWRITE)) == 0)
 			error("can't open %s", buf);
 	}
 	if(dflag) {
-		sprint(buf, "%s.%s", stem, FILED);
+		snprint(buf, sizeof buf, "%s.%s", stem, FILED);
 		fdefine = Bopen(buf, OWRITE);
 		if(fdefine == 0)
 			error("can't create %s", buf);
 	}
 	if(ytab == 0)
-		sprint(buf, "%s.%s", stem, OFILE);
+		snprint(buf, sizeof buf, "%s.%s", stem, OFILE);
 	else
-		strcpy(buf, ytabc);
+		strecpy(buf, buf+sizeof buf, ytabc);
 	ftable = Bopen(buf, OWRITE);
 	if(ftable == 0)
 		error("cannot open table file %s", buf);

Re: [9fans] all you yacc experts

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

Pretty cool!

On Tue, Nov 15, 2011 at 1:10 PM, Anthony Martin <ality@pbrane.org> wrote:

> Attached is a modified version of p9p yacc that
> supports the Go grammar.  I'll be sending a
> version of Plan 9 yacc later today.
>
> The following is a description of the changes.
>
>  1. The %error-verbose directive is ignored.
>
>  2. A description of the final grammar is
>     printed before the state descriptions
>         in y.output.
>
>  3. The 'x' format for character literals is
>     now used instead of prefixing with a space.
>
>  4. The YYEMPTY define is now used to clear
>     the lookahead token (instead of an explicit
>         negative one).
>
>  5. Make yychar and yystate globals so they
>     can be inspected by external code.
>
>  5. Support C++ style // comments in actions.
>
>  6. Add a usage message.
>
>  7. Fix a few uses of sprint and strcpy.
>
>
> I've also sent out a changeset to the Go
> development list which adds support for
> using Plan 9 yacc to generate the special
> errors.
>
> One tiny nit is that Plan 9 uses the name
> yytoknames for debugging where Bison uses
> yytname.  I've just used sed for this.
>
> Any questions?
>   Anthony
>

[-- Attachment #2: Type: text/html, Size: 1630 bytes --]