-column issue

-column issue

[Jason McIntyre]

hi. there's a formatting issue for -column lists. the best example i can
find is in openbsd's sysctl.3 (it's riddled with these):

	.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
	.It Sy Second level name	Type	Changeable
	.It Dv FS_POSIX_SETUID No "	integer	yes"
	.El

(the items are tab-separated)

groff formats it thusly:

	Second level name          Type          Changeable
	FS_POSIX_SETUID            integer       yes

all 3 column headers are displayed in Sy font.

but mandoc gets the spacing wrong:

	Second level name          Type          Changeable
	FS_POSIX_SETUID           integer       yes

additionally, only "Second level name" is rendered in Sy font. "Type"
and "Changeable" are unaffected.

these -column lists are icky!
jmc
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Kristaps Dzonsons]

> hi. there's a formatting issue for -column lists. the best example i can
> find is in openbsd's sysctl.3 (it's riddled with these):
> 
> 	.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
> 	.It Sy Second level name	Type	Changeable
> 	.It Dv FS_POSIX_SETUID No "	integer	yes"
> 	.El

UGH.  This is gross.  Why why why didn't they do:

     46  .Bl -column "Second level nameXXXXXX" integerXXX -offset indent
     47  .It Sy Second level name        Type    Changeable
     48  .It Dv FS_POSIX_SETUID No       integer yes

(With tabs between columns.)  It renders just fine in both mandoc and 
new/old groff.

I think this "bug" should be set aside.  It goes like this.  The quoted 
part for FS_POSIX_SETUID is in its own column, as is what follows.  So 
the `No' nixes the space before the entire column, which in turn makes 
it be off by one.  This is intuitive (to me, at least) because what 
follows `No' shouldn't have a leading space.  Throwing in a corner-case 
for columns seems wrong.  However, see below for my musings.

> additionally, only "Second level name" is rendered in Sy font. "Type"
> and "Changeable" are unaffected.
> 
> these -column lists are icky!

Right, this has been fairly extensively discussed.

In groff, tabbed lists are a little magical in that they allow macros to 
extend across the whole line (ONLY if they're specified in the first 
column).  mandoc, on the other hand, treats each column like a fresh new 
context.

I'm still on the fence whether this should be "fixed".

An option is to depend on Ingo's tab-code and simply throw out special 
tab-handling for lists altogether.  I'll look into this in the coming 
weeks.  If it simplifies the code, I'll probably do it.
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Jason McIntyre]

On Fri, Aug 06, 2010 at 07:08:11PM +0200, Kristaps Dzonsons wrote:
> >hi. there's a formatting issue for -column lists. the best example i can
> >find is in openbsd's sysctl.3 (it's riddled with these):
> >
> >	.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
> >	.It Sy Second level name	Type	Changeable
> >	.It Dv FS_POSIX_SETUID No "	integer	yes"
> >	.El
> 
> UGH.  This is gross.  Why why why didn't they do:
> 
>     46  .Bl -column "Second level nameXXXXXX" integerXXX -offset indent
>     47  .It Sy Second level name        Type    Changeable
>     48  .It Dv FS_POSIX_SETUID No       integer yes
> 
> (With tabs between columns.)  It renders just fine in both mandoc and 
> new/old groff.
> 

"they" didn;t do it, presumably, because whichever poor bastard wrote it
probably just jiggled the list until it formatted correctly. that is
roughly how one goes about constructing a -column list.

anyway, are you sure that your solution is correct? look carefully at how
groff formats it:

Second level name          Type          Changeable
FS_POSIX_SETUID No         integer       yes

i.e. it does not parse "No". mandoc does not display the "No". same if i
move the "No" into the next column (which seems more logical).

jmc
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Kristaps Dzonsons]

[-- Attachment #1: Type: text/plain, Size: 1479 bytes --]

Jason McIntyre wrote:
> On Fri, Aug 06, 2010 at 07:08:11PM +0200, Kristaps Dzonsons wrote:
>>> hi. there's a formatting issue for -column lists. the best example i can
>>> find is in openbsd's sysctl.3 (it's riddled with these):
>>>
>>> 	.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
>>> 	.It Sy Second level name	Type	Changeable
>>> 	.It Dv FS_POSIX_SETUID No "	integer	yes"
>>> 	.El
>> UGH.  This is gross.  Why why why didn't they do:
>>
>>     46  .Bl -column "Second level nameXXXXXX" integerXXX -offset indent
>>     47  .It Sy Second level name        Type    Changeable
>>     48  .It Dv FS_POSIX_SETUID No       integer yes
>>
>> (With tabs between columns.)  It renders just fine in both mandoc and 
>> new/old groff.
>>
> 
> "they" didn;t do it, presumably, because whichever poor bastard wrote it
> probably just jiggled the list until it formatted correctly. that is
> roughly how one goes about constructing a -column list.
> 
> anyway, are you sure that your solution is correct? look carefully at how
> groff formats it:
> 
> Second level name          Type          Changeable
> FS_POSIX_SETUID No         integer       yes
> 
> i.e. it does not parse "No". mandoc does not display the "No". same if i
> move the "No" into the next column (which seems more logical).

Jason, I can't replicate this.  Enclosed is the input (so as to preserve 
the tabbing...).  I tested with OpenBSD and a new groff.  Do you still 
see this behaviour?

Kristaps

[-- Attachment #2: foo.1 --]
[-- Type: text/plain, Size: 219 bytes --]

.Dd $Mdocdate$
.Dt FOO 1
.Os
.Sh NAME
.Nm foo
.Nd bar
.Sh DESCRIPTION
.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
.It Sy Second level name	Type	Changeable
.It Dv FS_POSIX_SETUID No 	integer	yes 
.El

Re: -column issue

[Jason McIntyre]

On Fri, Aug 06, 2010 at 08:13:22PM +0200, Kristaps Dzonsons wrote:
> >
> >Second level name          Type          Changeable
> >FS_POSIX_SETUID No         integer       yes
> >
> >i.e. it does not parse "No". mandoc does not display the "No". same if i
> >move the "No" into the next column (which seems more logical).
> 
> Jason, I can't replicate this.  Enclosed is the input (so as to preserve 
> the tabbing...).  I tested with OpenBSD and a new groff.  Do you still 
> see this behaviour?
> 
> Kristaps


hmm. so the reason was that you used "No<TAB><space>", where i used
"No<TAB>". and the extra space is as you explained previously about
lining up the columns correctly, and the way mandoc handles -column
lists, yes?

so, it seems non-intuitive. i would expect to do sth like this:

	FS_POSIX_SETUID<TAB>No<SPACE>integer<TAB>yes

mandoc handles it, but groff does not. sigh.

so, do you recommend i "fix" sysctl.3?
jmc
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Kristaps Dzonsons]

Jason McIntyre wrote:
> On Fri, Aug 06, 2010 at 08:13:22PM +0200, Kristaps Dzonsons wrote:
>>> Second level name          Type          Changeable
>>> FS_POSIX_SETUID No         integer       yes
>>>
>>> i.e. it does not parse "No". mandoc does not display the "No". same if i
>>> move the "No" into the next column (which seems more logical).
>> Jason, I can't replicate this.  Enclosed is the input (so as to preserve 
>> the tabbing...).  I tested with OpenBSD and a new groff.  Do you still 
>> see this behaviour?
>>
>> Kristaps
> 
> 
> hmm. so the reason was that you used "No<TAB><space>", where i used
> "No<TAB>". and the extra space is as you explained previously about
> lining up the columns correctly, and the way mandoc handles -column
> lists, yes?
> 
> so, it seems non-intuitive. i would expect to do sth like this:
> 
> 	FS_POSIX_SETUID<TAB>No<SPACE>integer<TAB>yes
> 
> mandoc handles it, but groff does not. sigh.
> 
> so, do you recommend i "fix" sysctl.3?

Oomf, I hadn't noticed that.  Both old and new groff do indeed print out 
the `No'.  I fail to understand this one.  Let me see if I can think of 
a more elegant solution... but somehow I doubt it.

--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Ingo Schwarze]

Hi Jason,

> so, do you recommend i "fix" sysctl.3?

I don't think we will get mandoc fully fixed in this respect
before release, we are not even sure which is the best way to go,
and changing this could easily cause considerable breakage.

You could just use

  .It Dv FS_POSIX_SETUID Ta integer Ta yes

That is quite easy to understand and works with old groff,
new groff and mandoc.  When there are many words per column,
old groff sometimes needs quoting to reduce the number of
arguments:

  .It Dv FS_POSIX_SETUID Ta "1 2 3 4 5" Ta yes

When there are many columns, Ta may not be an option with old
groff at all.  But here, none of these issues seem to occur.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Ingo Schwarze]

Hi Jason and Kristaps,

Jason McIntyre wrote on Fri, Aug 06, 2010 at 06:51:36PM +0059:
> On Fri, Aug 06, 2010 at 07:08:11PM +0200, Kristaps Dzonsons wrote:
>> Jason wrote:

>>> hi. there's a formatting issue for -column lists. the best example
>>> i can find is in openbsd's sysctl.3 (it's riddled with these):
>>>
>>>	.Bl -column "Second level nameXXXXXX" integerXXX -offset indent
>>>	.It Sy Second level name	Type	Changeable
>>>	.It Dv FS_POSIX_SETUID No "	integer	yes"
>>>	.El

>> UGH.  This is gross.  Why why why didn't they do:
>> 
>>     46  .Bl -column "Second level nameXXXXXX" integerXXX -offset indent
>>     47  .It Sy Second level name        Type    Changeable
>>     48  .It Dv FS_POSIX_SETUID No       integer yes
>> 
>> (With tabs between columns.)  It renders just fine in both mandoc
>> and new/old groff.

[...]
> anyway, are you sure that your solution is correct?
> look carefully at how groff formats it:
> 
> Second level name          Type          Changeable
> FS_POSIX_SETUID No         integer       yes
> 
> i.e. it does not parse "No". mandoc does not display the "No".
> same if i move the "No" into the next column
> (which seems more logical).

I guess now i understand.
This has nothing to do with .Bl -column.
Rather, we are talking about tab characters.

The rules are:

1. Either a space or a tab delimit the initial macro on a line.
   If the delimiter is a tab, that tab is lost.

   Examples:
   .Fl<space>Fl       ->  Fl() Fl()  ->  "--"
   .Fl<tab>Fl         ->  Fl() Fl()  ->  "--"
   .Fl<tab><space>Fl  ->  Fl() Fl()  ->  "--"

   In this respect, there is a bug in mandoc.
   A tab character does not delimit the initial macro on a line,
   but .Fl<tab> gives you:
   ERROR: bad character  <-- that's the tab
   ERROR: unknown macro will be lost: unknown macro: Fl<tab>

2. Inside a macro line, from the point of view of the *parser*,
   the tab is a normal character and doesn't delimit anything.
   In the parser, it is preserved verbatim.
   Later on, the formatter will expand it.

   Examples:
   .Fl<space><tab>Fl         ->  Fl("<tab>Fl")     ->  "-    Fl"
   .Fl<space><tab><space>Fl  ->  Fl("<tab>") Fl()  ->  "-     -"
   .Op<space>Fl<tab>Fl       ->  Op("Fl<tab>Fl")   ->  "[Fl   Fl]"

   In this respect, mandoc actually seems to be correct.

Thus,

  .It<sp>Dv<sp>FS_POSIX_SETUID<sp>No<tab>integer<tab>yes  ->
  It(Dv(FS_POSIX_SETUID) "No<tab>integer<tab>yes")        ->
  "FS_POSIX_SETUID No         integer       yes"

In the line

  .It Dv FS_POSIX_SETUID No "     integer yes"

the space after the No is vital, whereas the quotes are not.
I guess the quotes were only put to make it plain that there is
a syntactically significant space preceding the tab.

Oh, and the reason why

  .It<sp>Dv<sp>FS_POSIX_SETUID<tab>No<sp>integer<tab>yes

cannot work is now plain too:  It gives

  It(Dv("FS_POSIX_SETUID<tab>No") "integer<tab>yes")  ->
  "FS_POSIX_SETUID            No integer    yes"

On the other hand, you cannot put a space after the tab
because it will mess up the alignment of the second column.
Hence the need to put the formatting macro for the second column
before the tab, and hence the idiom: macro, space, quote, tab.

Enjoy roff,
  Ingo


What remains is the quoted spacing issue in mandoc:

  .Bl -column "Second level nameXXXXXX" integerXXX -offset indent
  .It Sy Second level name	Type	Changeable
  .It Dv FS_POSIX_SETUID No 	integer	yes
  .It Dv FS_POSIX_SETUID No "	integer	yes"
  .El

gives

           Second level name          Type          Changeable
           FS_POSIX_SETUID            integer       yes
           FS_POSIX_SETUID            integer       yes

with groff (both old and new) but

           Second level name          Type          Changeable
           FS_POSIX_SETUID            integer       yes
           FS_POSIX_SETUID           integer       yes

with mandoc.
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: -column issue

[Kristaps Dzonsons]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

> The rules are:
> 
> 1. Either a space or a tab delimit the initial macro on a line.
>    If the delimiter is a tab, that tab is lost.
> 
>    Examples:
>    .Fl<space>Fl       ->  Fl() Fl()  ->  "--"
>    .Fl<tab>Fl         ->  Fl() Fl()  ->  "--"
>    .Fl<tab><space>Fl  ->  Fl() Fl()  ->  "--"
> 
>    In this respect, there is a bug in mandoc.
>    A tab character does not delimit the initial macro on a line,
>    but .Fl<tab> gives you:
>    ERROR: bad character  <-- that's the tab
>    ERROR: unknown macro will be lost: unknown macro: Fl<tab>

Ingo,

Does the enclosed patch solve this behaviour?  I've enclosed a handy 
test as well.

Kristaps

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 2258 bytes --]

? .foo.1.swp
? config.h
? config.log
? foo.1
? foo.1.html
? foo.1.pdf
? foo.1.ps
? foo.3
? foo.pdf
? mandoc
? mdoc.7.ps.gz
? patch.txt
? regress/mandoc.core
? regress/output
Index: TODO
===================================================================
RCS file: /usr/vhosts/mdocml.bsd.lv/cvs/mdocml/TODO,v
retrieving revision 1.35
diff -u -r1.35 TODO
--- TODO	27 Jul 2010 08:46:30 -0000	1.35
+++ TODO	7 Aug 2010 17:30:07 -0000
@@ -63,6 +63,9 @@
 - prohibit `Nm' from having non-text HEAD children
   (e.g., NetBSD mDNSShared/dns-sd.1)
   (mdoc_html.c and mdoc_term.c `Nm' handlers can be slightly simplified)
+
+- allow `Qq', `Dq', `Sq', `Aq', `Bq' to have 0 arguments
+  noted by Alex Kozlov 08/06/10 23:05
   
 ************************************************************************
 * formatting issues: ugly output
Index: mdoc.c
===================================================================
RCS file: /usr/vhosts/mdocml.bsd.lv/cvs/mdocml/mdoc.c,v
retrieving revision 1.159
diff -u -r1.159 mdoc.c
--- mdoc.c	18 Jul 2010 17:00:26 -0000	1.159
+++ mdoc.c	7 Aug 2010 17:30:07 -0000
@@ -764,11 +764,11 @@
 
 	i = offs;
 
-	/* Accept whitespace after the initial control char. */
+	/* Accept tabs/whitespace after the initial control char. */
 
-	if (' ' == buf[i]) {
+	if (' ' == buf[i] || '\t' == buf[i]) {
 		i++;
-		while (buf[i] && ' ' == buf[i])
+		while (buf[i] && (' ' == buf[i] || '\t' == buf[i]))
 			i++;
 		if ('\0' == buf[i])
 			return(1);
@@ -776,15 +776,19 @@
 
 	sv = i;
 
-	/* Copy the first word into a nil-terminated buffer. */
+	/* 
+	 * Copy the first word into a nil-terminated buffer.  Stop
+	 * copying when a tab, space, or eoln is encountered.
+	 */
 
 	for (j = 0; j < 4; j++, i++) {
 		if ('\0' == (mac[j] = buf[i]))
 			break;
-		else if (' ' == buf[i])
+		else if (' ' == buf[i] || '\t' == buf[i])
 			break;
 
 		/* Check for invalid characters. */
+		/* TODO: remove me, already done in main.c. */
 
 		if (isgraph((u_char)buf[i]))
 			continue;
@@ -807,7 +811,12 @@
 		return(1);
 	}
 
-	/* The macro is sane.  Jump to the next word. */
+	/* Disregard the first trailing tab, if applicable. */
+
+	if ('\t' == buf[i])
+		i++;
+
+	/* Jump to the next non-whitespace word. */
 
 	while (buf[i] && ' ' == buf[i])
 		i++;

[-- Attachment #3: foo.1 --]
[-- Type: text/plain, Size: 326 bytes --]

.Dd $Mdocdate$
.Dt FOO 1
.Os
.Sh NAME
.Nm foo
.Nd bar
.Sh SYNOPSIS
.In header.h
.Vt extern const char *global;
.Ft "char *"
.Fn foo "const char *src"
.Ft "char *"
.Fn bar "const char *src"
.Sh DESCRIPTION
.	\r	Fl Fl
.Pp
.	 	Fl Fl
.Pp
.	Fl Fl
.Pp
.Fl		Fl
.Pp
.Fl	Fl
.Pp
.Fl Fl
.Pp
.Fl	 Fl
.Pp
.Fl 	Fl
.Pp
.Fl	 	Fl
.Pp
.Op Fl	Fl

Re: -column issue

[Ingo Schwarze]

Hi Kristaps,

Kristaps Dzonsons wrote on Sat, Aug 07, 2010 at 07:32:06PM +0200:
> Ingo Schwarze wrote:

>> The rules are:
>>
>> 1. Either a space or a tab delimit the initial macro on a line.
>>    If the delimiter is a tab, that tab is lost.
>>
>>    Examples:
>>    .Fl<space>Fl       ->  Fl() Fl()  ->  "--"
>>    .Fl<tab>Fl         ->  Fl() Fl()  ->  "--"
>>    .Fl<tab><space>Fl  ->  Fl() Fl()  ->  "--"
>>
>>    In this respect, there is a bug in mandoc.
>>    A tab character does not delimit the initial macro on a line,
>>    but .Fl<tab> gives you:
>>    ERROR: bad character  <-- that's the tab
>>    ERROR: unknown macro will be lost: unknown macro: Fl<tab>

> Does the enclosed patch solve this behaviour?

Yes, it does.  Thanks.
Please commit to bsd.lv, i'm committing to OpenBSD, too.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv