From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from smtp1.rz.uni-karlsruhe.de (Debian-exim@smtp1.rz.uni-karlsruhe.de [129.13.185.217])
	by krisdoz.my.domain (8.14.3/8.14.3) with ESMTP id p0TB6DBY022393
	for <tech@mdocml.bsd.lv>; Sat, 29 Jan 2011 06:06:13 -0500 (EST)
Received: from hekate.usta.de (asta-nat.asta.uni-karlsruhe.de [172.22.63.82])
	by smtp1.rz.uni-karlsruhe.de with esmtp (Exim 4.63 #1)
	id 1Pj8d8-00054B-BP; Sat, 29 Jan 2011 12:06:12 +0100
Received: from donnerwolke.usta.de ([172.24.96.3])
	by hekate.usta.de with esmtp (Exim 4.72)
	(envelope-from <schwarze@usta.de>)
	id 1Pj8d8-0003cZ-91; Sat, 29 Jan 2011 12:06:10 +0100
Received: from iris.usta.de ([172.24.96.5] helo=usta.de)
	by donnerwolke.usta.de with esmtp (Exim 4.69)
	(envelope-from <schwarze@usta.de>)
	id 1Pj8d8-0000T2-82; Sat, 29 Jan 2011 12:06:10 +0100
Received: from schwarze by usta.de with local (Exim 4.72)
	(envelope-from <schwarze@usta.de>)
	id 1Pj8d8-000295-7H; Sat, 29 Jan 2011 12:06:10 +0100
Date: Sat, 29 Jan 2011 12:06:10 +0100
From: Ingo Schwarze <schwarze@usta.de>
To: Matthieu Herrb <matthieu.herrb@laas.fr>
Cc: tech@mdocml.bsd.lv, jmc@openbsd.org
Subject: Re: [PATCH] implement roff N escape (used in Xenocara)
Message-ID: <20110129110610.GA15220@iris.usta.de>
References: <20110129101818.GB12351@iris.usta.de>
 <20110129105342.GB20999@bluenote.herrb.net>
X-Mailinglist: mdocml-tech
Reply-To: tech@mdocml.bsd.lv
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <20110129105342.GB20999@bluenote.herrb.net>
User-Agent: Mutt/1.5.21 (2010-09-15)

Hi Matthieu,

Matthieu Herrb wrote on Sat, Jan 29, 2011 at 11:53:42AM +0100:
> On Sat, Jan 29, 2011 at 11:18:18AM +0100, Ingo Schwarze wrote:

>> even though the roff \N character number escape is marked deprecated
>> and non-portable even in the Heirloom roff manual, Xenocara docs
>> use it extensively, so we better implement it, if only for backward
>> compatibility.
>>=20
>> OK?

> Tried it on several X drivers manual pages. Works fine and makes them
> a lot more readable. So ok from me as far as I'm concerned.

Good, thanks for testing!

I'm planning to commit on Sunday unless problems are found with the patch.

> Thanks for doing that.

>> Hm, now that i'm sending the patch, i see that the mandoc_char(7)
>> manual part is still missing, so i guess i need to add that when
>> committing.

And here is the manual patch as well.

Yours,
  Ingo


Index: mandoc_char.7
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/share/man/man7/mandoc_char.7,v
retrieving revision 1.6
diff -u -r1.6 mandoc_char.7
--- mandoc_char.7	28 Oct 2010 18:13:52 -0000	1.6
+++ mandoc_char.7	29 Jan 2011 11:00:33 -0000
@@ -513,6 +513,21 @@
 .It \e*(aa   Ta \*(aa       Ta acute
 .It \e*(ga   Ta \*(ga       Ta grave
 .El
+.Sh NUMBERED CHARACTERS
+For backward compatibility with existing manuals,
+.Xr mandoc 1
+also supports the
+.Pp
+.Dl \eN\(aq Ns Ar number Ns \(aq
+.Pp
+escape sequence, inserting the character
+.Ar number
+from the current character set into the output.
+Of course, this is inherently non-portable, and it is already marked
+as deprecated in the Heirloom roff manual.
+For example, do not use \eN'34', use \e(dq, or even the plain
+.Sq \(dq
+character where possible.
 .Sh COMPATIBILITY
 This section documents compatibility of
 .Nm

 ----- 8< ----- schnipp ----- >8 ----- 8< ----- schnapp ----- >8 -----

Index: usr.bin/mandoc/chars.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/chars.c,v
retrieving revision 1.15
diff -a -u -p -r1.15 chars.c
--- usr.bin/mandoc/chars.c	4 Jan 2011 22:28:17 -0000	1.15
+++ usr.bin/mandoc/chars.c	29 Jan 2011 10:06:07 -0000
@@ -1,6 +1,7 @@
 /*	$Id: chars.c,v 1.15 2011/01/04 22:28:17 schwarze Exp $ */
 /*
  * Copyright (c) 2009, 2010 Kristaps Dzonsons <kristaps@bsd.lv>
+ * Copyright (c) 2011 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -145,6 +146,27 @@ chars_res2cp(void *arg, const char *p, s
 	if (NULL =3D=3D ln)
 		return(-1);
 	return(ln->unicode);
+}
+
+
+/*
+ * Numbered character to literal character,
+ * represented as a null-terminated string for additional safety.
+ */
+const char *
+chars_num2char(const char *p, size_t sz)
+{
+	int		  i;
+	static char	  c[2];
+
+	if (sz > 3)
+		return(NULL);
+	i =3D atoi(p);
+	if (i < 0 || i > 255)
+		return(NULL);
+	c[0] =3D (char)i;
+	c[1] =3D '\0';
+	return(c);
 }
=20
=20
Index: usr.bin/mandoc/chars.h
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/chars.h,v
retrieving revision 1.5
diff -a -u -p -r1.5 chars.h
--- usr.bin/mandoc/chars.h	25 Jul 2010 18:05:54 -0000	1.5
+++ usr.bin/mandoc/chars.h	29 Jan 2011 10:06:07 -0000
@@ -1,6 +1,7 @@
 /*	$Id: chars.h,v 1.5 2010/07/25 18:05:54 schwarze Exp $ */
 /*
  * Copyright (c) 2008, 2009, 2010 Kristaps Dzonsons <kristaps@bsd.lv>
+ * Copyright (c) 2011 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -25,6 +26,7 @@ enum	chars {
 };
=20
 void		 *chars_init(enum chars);
+const char	 *chars_num2char(const char *, size_t);
 const char	 *chars_spec2str(void *, const char *, size_t, size_t *);
 int		  chars_spec2cp(void *, const char *, size_t);
 const char	 *chars_res2str(void *, const char *, size_t, size_t *);
Index: usr.bin/mandoc/html.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/html.c,v
retrieving revision 1.22
diff -a -u -p -r1.22 html.c
--- usr.bin/mandoc/html.c	16 Jan 2011 19:41:16 -0000	1.22
+++ usr.bin/mandoc/html.c	29 Jan 2011 10:06:08 -0000
@@ -1,6 +1,7 @@
 /*	$Id: html.c,v 1.22 2011/01/16 19:41:16 schwarze Exp $ */
 /*
  * Copyright (c) 2008, 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv>
+ * Copyright (c) 2011 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -89,6 +90,7 @@ static	const char	*const htmlattrs[ATTR_
 	"colspan", /* ATTR_COLSPAN */
 };
=20
+static	void		  print_num(struct html *, const char *, size_t);
 static	void		  print_spec(struct html *, enum roffdeco,
 				const char *, size_t);
 static	void		  print_res(struct html *, const char *, size_t);
@@ -210,6 +212,17 @@ print_gen_head(struct html *h)
=20
=20
 static void
+print_num(struct html *h, const char *p, size_t len)
+{
+	const char	*rhs;
+
+	rhs =3D chars_num2char(p, len);
+	if (rhs)
+		putchar((int)*rhs);
+}
+
+
+static void
 print_spec(struct html *h, enum roffdeco d, const char *p, size_t len)
 {
 	int		 cp;
@@ -329,6 +342,9 @@ print_encode(struct html *h, const char=20
 		len =3D a2roffdeco(&deco, &seq, &sz);
=20
 		switch (deco) {
+		case (DECO_NUMBERED):
+			print_num(h, seq, sz);
+			break;
 		case (DECO_RESERVED):
 			print_res(h, seq, sz);
 			break;
Index: usr.bin/mandoc/out.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/out.c,v
retrieving revision 1.11
diff -a -u -p -r1.11 out.c
--- usr.bin/mandoc/out.c	25 Jan 2011 12:07:26 -0000	1.11
+++ usr.bin/mandoc/out.c	29 Jan 2011 10:06:08 -0000
@@ -1,6 +1,7 @@
 /*	$Id: out.c,v 1.11 2011/01/25 12:07:26 schwarze Exp $ */
 /*
  * Copyright (c) 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv>
+ * Copyright (c) 2011 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -247,6 +248,49 @@ a2roffdeco(enum roffdeco *d, const char=20
 			break;
 		}
 		break;
+
+	case ('N'):
+
+		/*
+		 * Sequence of characters:  backslash,  'N' (i =3D 0),
+		 * starting delimiter (i =3D 1), character number (i =3D 2).
+		 */
+
+		*word =3D wp + 2;
+		*sz =3D 0;
+
+		/*
+		 * Cannot use a digit as a starting delimiter;
+		 * but skip the digit anyway.
+		 */
+
+		if (isdigit((int)wp[1]))
+			return(2);
+
+		/*
+		 * Any non-digit terminates the character number.
+		 * That is, the terminating delimiter need not
+		 * match the starting delimiter.
+		 */
+
+		for (i =3D 2; isdigit((int)wp[i]); i++)
+			(*sz)++;
+
+		/*
+		 * This is only a numbered character
+		 * if the character number has at least one digit.
+		 */
+
+		if (*sz)
+			*d =3D DECO_NUMBERED;
+
+		/*
+		 * Skip the terminating delimiter, even if it does not
+		 * match, and even if there is no character number.
+		 */
+
+		return(++i);
+
 	case ('h'):
 		/* FALLTHROUGH */
 	case ('v'):
Index: usr.bin/mandoc/out.h
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/out.h,v
retrieving revision 1.7
diff -a -u -p -r1.7 out.h
--- usr.bin/mandoc/out.h	9 Jan 2011 14:30:48 -0000	1.7
+++ usr.bin/mandoc/out.h	29 Jan 2011 10:06:08 -0000
@@ -52,6 +52,7 @@ enum	roffscale {
=20
 enum	roffdeco {
 	DECO_NONE,
+	DECO_NUMBERED, /* numbered character */
 	DECO_SPECIAL, /* special character */
 	DECO_SSPECIAL, /* single-char special */
 	DECO_RESERVED, /* reserved word */
Index: usr.bin/mandoc/term.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/usr.bin/mandoc/term.c,v
retrieving revision 1.55
diff -a -u -p -r1.55 term.c
--- usr.bin/mandoc/term.c	4 Jan 2011 22:28:17 -0000	1.55
+++ usr.bin/mandoc/term.c	29 Jan 2011 10:06:08 -0000
@@ -1,7 +1,7 @@
 /*	$Id: term.c,v 1.55 2011/01/04 22:28:17 schwarze Exp $ */
 /*
  * Copyright (c) 2008, 2009, 2010 Kristaps Dzonsons <kristaps@bsd.lv>
- * Copyright (c) 2010 Ingo Schwarze <schwarze@openbsd.org>
+ * Copyright (c) 2010, 2011 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -349,6 +349,17 @@ term_vspace(struct termp *p)
=20
=20
 static void
+numbered(struct termp *p, const char *word, size_t len)
+{
+	const char	*rhs;
+
+	rhs =3D chars_num2char(word, len);
+	if (rhs)=20
+		encode(p, rhs, 1);
+}
+
+
+static void
 spec(struct termp *p, enum roffdeco d, const char *word, size_t len)
 {
 	const char	*rhs;
@@ -507,6 +518,9 @@ term_word(struct termp *p, const char *w
 		word +=3D a2roffdeco(&deco, &seq, &ssz);
=20
 		switch (deco) {
+		case (DECO_NUMBERED):
+			numbered(p, seq, ssz);
+			break;
 		case (DECO_RESERVED):
 			res(p, seq, ssz);
 			break;
Index: regress/usr.bin/mandoc/char/Makefile
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvs/src/regress/usr.bin/mandoc/char/Makefile,v
retrieving revision 1.1
diff -a -u -p -r1.1 Makefile
--- regress/usr.bin/mandoc/char/Makefile	23 Dec 2009 23:40:58 -0000	1.1
+++ regress/usr.bin/mandoc/char/Makefile	29 Jan 2011 10:06:08 -0000
@@ -1,6 +1,6 @@
 # $OpenBSD: Makefile,v 1.1 2009/12/23 23:40:58 schwarze Exp $
=20
-SUBDIR+=3D space
+SUBDIR+=3D space N
=20
 groff groff-clean: _SUBDIRUSE
=20
Index: regress/usr.bin/mandoc/char/N/Makefile
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: regress/usr.bin/mandoc/char/N/Makefile
diff -N regress/usr.bin/mandoc/char/N/Makefile
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ regress/usr.bin/mandoc/char/N/Makefile	29 Jan 2011 10:06:08 -0000
@@ -0,0 +1,6 @@
+# $OpenBSD: Makefile,v 1.2 2010/01/01 20:08:17 schwarze Exp $
+
+REGRESS_TARGETS=3Dbasic
+GROFF_TARGETS=3Dbasic
+
+.include <bsd.regress.mk>
Index: regress/usr.bin/mandoc/char/N/basic.in
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: regress/usr.bin/mandoc/char/N/basic.in
diff -N regress/usr.bin/mandoc/char/N/basic.in
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ regress/usr.bin/mandoc/char/N/basic.in	29 Jan 2011 10:06:08 -0000
@@ -0,0 +1,23 @@
+.TH N-BASIC 1 "January 29, 2011"
+.SH NAME
+N-basic \- basic handling of character number escapes
+.SH DESCRIPTION
+basic usage: x\N'65'x
+.br
+too large: x\N'259'x
+.br
+much too large: x\N'2259'x
+.br
+null: x\N'0'x
+.br
+non-numerical content: x\N'XX'x
+.br
+mixed content: x\N'65XX'x
+.br
+empty: x\N''x
+.br
+no quoting: x\N65x
+.br
+non-matching quoting characters: x\NX65Yx
+.br
+end of test document
Index: regress/usr.bin/mandoc/char/N/basic.out_ascii
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: regress/usr.bin/mandoc/char/N/basic.out_ascii
diff -N regress/usr.bin/mandoc/char/N/basic.out_ascii
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ regress/usr.bin/mandoc/char/N/basic.out_ascii	29 Jan 2011 10:06:08 -0000
@@ -0,0 +1,21 @@
+N-BASIC(1)                                                          N-BASI=
C(1)
+
+
+
+N=08NA=08AM=08ME=08E
+       N-basic - basic handling of character number escapes
+
+D=08DE=08ES=08SC=08CR=08RI=08IP=08PT=08TI=08IO=08ON=08N
+       basic usage: xAx
+       too large: xx
+       much too large: xx
+       null: x
+       mixed content: xAX'x
+       empty: xx
+       no quoting: x5x
+       non-matching quoting characters: xAx
+       end of test document
+
+
+
--
 To unsubscribe send an email to tech+unsubscribe@mdocml.bsd.lv