Hi, i just released portable mandoc-1.14.2. It is available now from http://mandoc.bsd.lv/ . All downstream maintainers are encouraged to update their ports and packages from 1.14.1 to 1.14.2. Mandoc 1.14.2 is a feature release introducing * a new -Tmarkdown output mode * anchors for deep linking into -Thtml manual pages * a superset of the functionality of the former mdoclint(1) utility * a new -Wstyle message level with several new messages * automatic line breaking inside individual tbl(7) cells * a rewrite of the eqn(7) lexer, and some eqn(7) rendering improvements * support for many additional low-level roff(7) features * and various smaller features and bug fixes. For more details, see: http://mandoc.bsd.lv/NEWS With the improved mandoc features, only twenty-five out of the ten thousand software packages in the OpenBSD ports tree still need groff to format their manual pages. Since the project has been called "mandoc" rather than "mdocml" for several years now, the website, the distribution tarball, and the source extraction directory are now also called "mandoc" rather than "mdocml". The release was tested on the following systems: * OpenBSD-current and OpenBSD-stable * NetBSD-current * illumos * Debian Linux * Void Linux x86_64 glibc and musl * Crux Linux * SunOS 5.11.2, 5.10, and 5.9 As before, catman(8) and the regression suite cannot be used on SunOS 5.10 and SunOS 5.9. A big thanks to everybody who provided patches, bug reports, feature suggestions, advice, and help with testing! Yours, Ingo -- To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
On Fri, 28 Jul 2017 20:12:44 +0200, Ingo Schwarze wrote:
> Hi,
>
> i just released portable mandoc-1.14.2.
> It is available now from http://mandoc.bsd.lv/ .
>
> All downstream maintainers are encouraged to update their ports
> and packages from 1.14.1 to 1.14.2.
>
> Mandoc 1.14.2 is a feature release introducing
> * a new -Tmarkdown output mode
> * anchors for deep linking into -Thtml manual pages
> * a superset of the functionality of the former mdoclint(1) utility
> * a new -Wstyle message level with several new messages
> * automatic line breaking inside individual tbl(7) cells
> * a rewrite of the eqn(7) lexer, and some eqn(7) rendering improvements
> * support for many additional low-level roff(7) features
> * and various smaller features and bug fixes.
> For more details, see: http://mandoc.bsd.lv/NEWS
>
> With the improved mandoc features, only twenty-five out of the
> ten thousand software packages in the OpenBSD ports tree still
> need groff to format their manual pages.
>
> Since the project has been called "mandoc" rather than "mdocml"
> for several years now, the website, the distribution tarball,
> and the source extraction directory are now also called "mandoc"
> rather than "mdocml".
>
> The release was tested on the following systems:
> * OpenBSD-current and OpenBSD-stable
> * NetBSD-current
> * illumos
> * Debian Linux
> * Void Linux x86_64 glibc and musl
> * Crux Linux
> * SunOS 5.11.2, 5.10, and 5.9
>
> As before, catman(8) and the regression suite cannot be used on
> SunOS 5.10 and SunOS 5.9.
>
> A big thanks to everybody who provided patches, bug reports,
> feature suggestions, advice, and help with testing!
Hi Ingo,
I'm in the process of updating the mandoc version in our tree, and have
noticed that man pages bundled with the release are not entirely '-Tlint
-Wstyle' clean as reported by 1.14.2 version that I've just built, a lot
of noise about man.options.1 (which we don't use though), and the ones
below:
mandoc: /home/yuri/mandoc-1.14.2/mandoc.1:99:20: STYLE: no blank before
trailing delimiter: Oo ...;
mandoc: /home/yuri/mandoc-1.14.2/mandoc_char.7:204:11: STYLE: no blank
before trailing delimiter: Sq \e&.
mandoc: /home/yuri/mandoc-1.14.2/mandoc_html.3:272:6: STYLE: no blank
before trailing delimiter: Cm s?
mandoc: /home/yuri/mandoc-1.14.2/mdoc.7:3076:8: STYLE: no blank before
trailing delimiter: Sq \e&.
mandoc: /home/yuri/mandoc-1.14.2/roff.7:1868:7: STYLE: trailing
delimiter: Ss \e,
--
To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
Hi Yuri, Yuri Pankov Illumos wrote on Sat, Jul 29, 2017 at 04:35:37PM +0300: > I'm in the process of updating the mandoc version in our tree, Great. You are working fast. :-) > and have noticed that man pages bundled with the release are not > entirely '-Tlint -Wstyle' clean as reported by 1.14.2 version that > I've just built, a lot of noise about man.options.1 (which we don't > use though), Indeed, that one isn't really intended to be installed. The target audience are mandoc developers (and developers of other man(1) systems), just like for the *.3 pages, which are not installed by default either, in particular the internal ones like man.cgi.3, mandoc_headers.3, mandoc_html.3. The man.options.1 file is not a clean mdoc(7) page but uses some low-level roff(7) features that are not needed in normal manual pages. > and the ones below: > > mandoc: /home/yuri/mandoc-1.14.2/mandoc.1:99:20: STYLE: no blank before > trailing delimiter: Oo ...; > mandoc: /home/yuri/mandoc-1.14.2/mandoc_char.7:204:11: STYLE: no blank > before trailing delimiter: Sq \e&. > mandoc: /home/yuri/mandoc-1.14.2/mandoc_html.3:272:6: STYLE: no blank > before trailing delimiter: Cm s? > mandoc: /home/yuri/mandoc-1.14.2/mdoc.7:3076:8: STYLE: no blank before > trailing delimiter: Sq \e&. > mandoc: /home/yuri/mandoc-1.14.2/roff.7:1868:7: STYLE: trailing > delimiter: Ss \e, That is known, too. It is not easy to get style messages 100% free of false positives. That is even documented, see mandoc(1): style An input file uses dubious or discouraged style. This is not a complaint about the syntax, and probably neither formatting nor portability are in danger. While great care is taken to avoid false positives on the higher message levels, the style level tries to reduce the probability that issues go unnoticed, so it may occasionally issue bogus suggestions. Please use your good judgement to decide whether any particular style suggestion really justifies a change to the input file. If you cannot tolerate a small number of false positives in the build, consider using -Wwarning instead. It wouldn't really help to change the mandoc manuals themselves to not issue these five false positives. Probably, there are also a few false positives in the rest of your tree. Warning and error messages should almost always be fixed - or else reported to me as bogus when they cause false positives, such that i can fix them. But making the criteria for style messages even stricter such that they never cause false positives would degrade their usefulness by making them miss many real issues. Yours, Ingo -- To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
On Sat, 29 Jul 2017 16:28:27 +0200, Ingo Schwarze wrote:
> Hi Yuri,
>
> Yuri Pankov Illumos wrote on Sat, Jul 29, 2017 at 04:35:37PM +0300:
>
>> I'm in the process of updating the mandoc version in our tree,
>
> Great. You are working fast. :-)
>
>> and have noticed that man pages bundled with the release are not
>> entirely '-Tlint -Wstyle' clean as reported by 1.14.2 version that
>> I've just built, a lot of noise about man.options.1 (which we don't
>> use though),
>
> Indeed, that one isn't really intended to be installed. The target
> audience are mandoc developers (and developers of other man(1) systems),
> just like for the *.3 pages, which are not installed by default either,
> in particular the internal ones like man.cgi.3, mandoc_headers.3,
> mandoc_html.3.
>
> The man.options.1 file is not a clean mdoc(7) page but uses some
> low-level roff(7) features that are not needed in normal manual
> pages.
>
>> and the ones below:
>>
>> mandoc: /home/yuri/mandoc-1.14.2/mandoc.1:99:20: STYLE: no blank before
>> trailing delimiter: Oo ...;
>> mandoc: /home/yuri/mandoc-1.14.2/mandoc_char.7:204:11: STYLE: no blank
>> before trailing delimiter: Sq \e&.
>> mandoc: /home/yuri/mandoc-1.14.2/mandoc_html.3:272:6: STYLE: no blank
>> before trailing delimiter: Cm s?
>> mandoc: /home/yuri/mandoc-1.14.2/mdoc.7:3076:8: STYLE: no blank before
>> trailing delimiter: Sq \e&.
>> mandoc: /home/yuri/mandoc-1.14.2/roff.7:1868:7: STYLE: trailing
>> delimiter: Ss \e,
>
> That is known, too. It is not easy to get style messages 100% free
> of false positives. That is even documented, see mandoc(1):
>
> style An input file uses dubious or discouraged style. This is not a
> complaint about the syntax, and probably neither formatting nor
> portability are in danger. While great care is taken to avoid
> false positives on the higher message levels, the style level
> tries to reduce the probability that issues go unnoticed, so it
> may occasionally issue bogus suggestions. Please use your good
> judgement to decide whether any particular style suggestion
> really justifies a change to the input file.
>
> If you cannot tolerate a small number of false positives in the build,
> consider using -Wwarning instead. It wouldn't really help to change
> the mandoc manuals themselves to not issue these five false positives.
> Probably, there are also a few false positives in the rest of your
> tree.
>
> Warning and error messages should almost always be fixed - or else
> reported to me as bogus when they cause false positives, such that
> i can fix them. But making the criteria for style messages even
> stricter such that they never cause false positives would degrade
> their usefulness by making them miss many real issues.
Got it, thanks for explanation! I'll switch the checks to the -Wwarning
level.
There's another issue though which doesn't look entirely correct, sorry,
I didn't check the lint, only formatted output previously.
We have a man page that looks like this (and should stay like that):
$ cat foo_bar.1
.Dd July 29, 2017
.Dt FOO_BAR 1
.Os
.Sh NAME
.Nm foo_bar
.Nd baz
.Sh SYNOPSIS
.Nm foo
.Fl F Sy bar
.Sh DESCRIPTION
baz
.Sh SEE ALSO
.Xr foo 1
and here's the warning:
$ mandoc -Tlint -Wwarning foo_bar.1
mandoc: foo_bar.1:13:5: WARNING: cross reference to self: Xr foo 1
Shouldn't we look only at the NAME section for this check?
--
To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
Hi Yuri, Yuri Pankov Illumos wrote on Sat, Jul 29, 2017 at 06:12:54PM +0300: > There's another issue though which doesn't look entirely correct, > sorry, I didn't check the lint, only formatted output previously. No problem, nobody can be expected to find everything at once. > We have a man page that looks like this (and should stay like that): > > $ cat foo_bar.1 > .Dd July 29, 2017 > .Dt FOO_BAR 1 > .Os > .Sh NAME > .Nm foo_bar > .Nd baz > .Sh SYNOPSIS > .Nm foo > .Fl F Sy bar That should probably be ".Fl F Cm bar", but that's unrelated. > .Sh DESCRIPTION > baz > .Sh SEE ALSO > .Xr foo 1 Oh. You mean like git-commit(1)? > and here's the warning: > > $ mandoc -Tlint -Wwarning foo_bar.1 > mandoc: foo_bar.1:13:5: WARNING: cross reference to self: Xr foo 1 > > Shouldn't we look only at the NAME section for this check? Very good question. My first impulse was to say "your manual page is broken". But that is maybe not the right answer. We don't have that problem in OpenBSD because we do not build tools so large that they can't be described in a single page. Not so much out of lack of time to write that much code, but more out of pity on the poor users who would have to read all that documentation and spend so much time trying to learn using such large tools. But opinions on good software design and sane limits on complexity and bloat aside, documentation tools probably shouldn't make it impossible to document seriously bloated software like git(1). It's OK if documentation tools make documentating such bloat seriously harder, because maintaining and using such bloated tools is extremely difficult anyway. But it shouldn't be outright impossible. That said, the technical problem here is that mandoc(1) uses - slightly simplifying the algorithm and not describing priorities - 1. the filename(s) of of manual page files 2. the .Nm arguments in the NAME section 3. the .Nm HEAD arguments in the SYNOPSIS section as names of the manual page. Item 3 is useful because some authors neglect putting all the names in the NAME section - even though they should. Using "foo" as a name of the manual page has many implications: Try "man -w foo". You will probably see both foo(1) and foo_bar(1). Try "man -a foo". You will probably get shown foo(1), foo_bar(1), and possibly quite some other pages, too. Which may actually be useful for searching through all these pages at once, inside less(1). Such searching is even more important for large than for small tools. Try "man -k Nm~'^foo$'". You will probably see something like foo(1) - top-level description of the foo utility foo(1), foo_bar(1) - baz Delete foo.1 from the system and run "makewhatis", then try "man foo". You will probably be shown foo_bar(1) or some similar page. So this is not just about the warning that may seem annoying at first. That warning actually indicates that there are real, deeper issues. What can we do? a) Stop harvesting names in the SYNOPSIS section. Instead, more forcefully tell people to fix their NAME sections. b) Somehow provide a way to say, in the SYNOPSIS section, "This name is not topical." In the case of git, you could already do that by saying .Nm "git commit" rather than .Nm git Cm commit in the SYNOPSIS. In your case, with the nasty flag in between, that's less obvious. .Nm "foo \-F bar" seems to be what the page documents, but it sure looks ugly. I'm very hesitant to introduce new syntax just to mark a name as "not topical". c) Not warn about .Xr to names only found in the SYNOPSIS. I'm not enthusiastic because that hides potential problems with the aspects described above even for legitimate use, and it can also hide actual bugs, namely, pages that *do* reference themselves but have an incomplete NAME section. d) Something else? Maybe i'm leaning towards a). It's the simplest to understand for users and authors. But i'm not really sure yet... Yours, Ingo -- To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
On Sat, 29 Jul 2017 19:38:58 +0200, Ingo Schwarze wrote: > Hi Yuri, > > Yuri Pankov Illumos wrote on Sat, Jul 29, 2017 at 06:12:54PM +0300: > >> There's another issue though which doesn't look entirely correct, >> sorry, I didn't check the lint, only formatted output previously. > > No problem, nobody can be expected to find everything at once. > >> We have a man page that looks like this (and should stay like that): >> >> $ cat foo_bar.1 >> .Dd July 29, 2017 >> .Dt FOO_BAR 1 >> .Os >> .Sh NAME >> .Nm foo_bar >> .Nd baz >> .Sh SYNOPSIS >> .Nm foo >> .Fl F Sy bar > > That should probably be ".Fl F Cm bar", but that's unrelated. > >> .Sh DESCRIPTION >> baz >> .Sh SEE ALSO >> .Xr foo 1 > > Oh. You mean like git-commit(1)? > >> and here's the warning: >> >> $ mandoc -Tlint -Wwarning foo_bar.1 >> mandoc: foo_bar.1:13:5: WARNING: cross reference to self: Xr foo 1 >> >> Shouldn't we look only at the NAME section for this check? > > Very good question. > > My first impulse was to say "your manual page is broken". > But that is maybe not the right answer. > > We don't have that problem in OpenBSD because we do not build > tools so large that they can't be described in a single page. > Not so much out of lack of time to write that much code, but > more out of pity on the poor users who would have to read all > that documentation and spend so much time trying to learn using > such large tools. > > But opinions on good software design and sane limits on complexity > and bloat aside, documentation tools probably shouldn't make it > impossible to document seriously bloated software like git(1). It's > OK if documentation tools make documentating such bloat seriously > harder, because maintaining and using such bloated tools is extremely > difficult anyway. But it shouldn't be outright impossible. It's not about bloat. We have mount(1M) and share(1M), documenting generic options, and e.g. mount_nfs(1M) and share_nfs(1M), documenting NFS protocol specific ones, along with lots of protocol specific notes. Dumping all that in mount(1M) and share(1M) would actually be bloating them IMO :-) For example, share_nfs.1m: .Dd March 23, 2017 .Dt SHARE_NFS 1M .Os .Sh NAME .Nm share_nfs .Nd make local NFS file systems available for mounting by remote systems .Sh SYNOPSIS .Nm share .Fl F Cm nfs .Op Fl d Ar description .Op Fl o Ar specific_options .Ar pathname > That said, the technical problem here is that mandoc(1) uses - > slightly simplifying the algorithm and not describing priorities - > 1. the filename(s) of of manual page files > 2. the .Nm arguments in the NAME section > 3. the .Nm HEAD arguments in the SYNOPSIS section > as names of the manual page. Item 3 is useful because some authors > neglect putting all the names in the NAME section - even though they > should. > > Using "foo" as a name of the manual page has many implications: > > Try "man -w foo". You will probably see both foo(1) and foo_bar(1). > > Try "man -a foo". You will probably get shown foo(1), foo_bar(1), > and possibly quite some other pages, too. Which may actually be > useful for searching through all these pages at once, inside less(1). > Such searching is even more important for large than for small tools. > > Try "man -k Nm~'^foo$'". You will probably see something like > > foo(1) - top-level description of the foo utility > foo(1), foo_bar(1) - baz > > Delete foo.1 from the system and run "makewhatis", then try "man foo". > You will probably be shown foo_bar(1) or some similar page. > > So this is not just about the warning that may seem annoying at first. > That warning actually indicates that there are real, deeper issues. > > > What can we do? > > a) Stop harvesting names in the SYNOPSIS section. > Instead, more forcefully tell people to fix their NAME sections. > > b) Somehow provide a way to say, in the SYNOPSIS section, > "This name is not topical." > In the case of git, you could already do that by saying > .Nm "git commit" > rather than > .Nm git Cm commit > in the SYNOPSIS. > > In your case, with the nasty flag in between, that's less obvious. > .Nm "foo \-F bar" > seems to be what the page documents, but it sure looks ugly. > > I'm very hesitant to introduce new syntax just to mark a name > as "not topical". > > c) Not warn about .Xr to names only found in the SYNOPSIS. > I'm not enthusiastic because that hides potential problems > with the aspects described above even for legitimate use, > and it can also hide actual bugs, namely, pages that *do* > reference themselves but have an incomplete NAME section. > > d) Something else? > > Maybe i'm leaning towards a). It's the simplest to understand for > users and authors. > > But i'm not really sure yet... I'd say a) as well, but looking for a solution in the mean time -- this is the only one left for the moment :-) TIA -- To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv
Hi Yuri, Yuri Pankov Illumos wrote on Sat, Jul 29, 2017 at 11:09:50PM +0300: > It's not about bloat. We have mount(1M) and share(1M), documenting > generic options, and e.g. mount_nfs(1M) and share_nfs(1M), documenting > NFS protocol specific ones, along with lots of protocol specific notes. > Dumping all that in mount(1M) and share(1M) would actually be bloating > them IMO :-) > > For example, share_nfs.1m: > > .Dd March 23, 2017 > .Dt SHARE_NFS 1M > .Os > .Sh NAME > .Nm share_nfs > .Nd make local NFS file systems available for mounting by remote systems > .Sh SYNOPSIS > .Nm share > .Fl F Cm nfs > .Op Fl d Ar description > .Op Fl o Ar specific_options > .Ar pathname Interesting. On OpenBSD, mount_nfs(8) is an actual command /sbin/mount_nfs, and documented as such, with .Sh SYNOPSIS .Nm mount_nfs ... Of course, you can also invoke it as "mount -t nfs", which is documented in mount(8). But that implies the clash doesn't occur here. > Ingo Schwarze wrote: >> What can we do? >> >> a) Stop harvesting names in the SYNOPSIS section. >> c) Not warn about .Xr to names only found in the SYNOPSIS. I just committed a fix doing that, see below. > I'd say a) as well, but looking for a solution in the mean time -- > this is the only one left for the moment :-) Since Markus Waldner and Leah Neukirchen found a very serious regression in 1.14.2 right after release, i'm going to follow up with a pure bug fix release 1.14.3 shortly. It will also contain the fix below. Yours, Ingo Log Message: ----------- No longer use names that only occur in the SYNOPSIS section as names for man(1) lookup. For OpenBSD base and Xenocara, that functionality was never intended to be required, and i just fixed the last handful of offenders using it - not counting the horribly ill-designed interfaces engine(3) and lh_new(3) which are impossible to properly document in the first place. Of course, apropos(1) and whatis(1) continue to use SYNOPSIS .Nm, .Fn, and .Fo macros, so "man -k ENGINE_get_load_privkey_function" still works. This change also gets rid of a few bogus warnings "cross reference to self" which actually are *not* to self, like in yp(8). This former functionality was intended to help third-party software in the ports tree and on non-OpenBSD systems containing manual pages with incomplete or corrupt NAME sections. But it turned out it did more harm than good, and caused more confusion than relief, specifically for third party manuals and for maintainers of mandoc-portable on other operating systems. So kill it. Problems reported, among others, by Yuri Pankov (illumos). OK jmc@ Modified Files: -------------- mandoc: mansearch.c mdoc_validate.c Revision Data ------------- Index: mdoc_validate.c =================================================================== RCS file: /home/cvs/mandoc/mandoc/mdoc_validate.c,v retrieving revision 1.351 retrieving revision 1.352 diff -Lmdoc_validate.c -Lmdoc_validate.c -u -p -r1.351 -r1.352 --- mdoc_validate.c +++ mdoc_validate.c @@ -1137,8 +1137,6 @@ post_fname(POST_ARGS) if ( ! (cp[0] == '\0' || (cp[0] == '(' && cp[1] == '*'))) mandoc_msg(MANDOCERR_FN_PAREN, mdoc->parse, n->line, n->pos + pos, n->string); - if (n->sec == SEC_SYNOPSIS && mdoc->meta.msec != NULL) - mandoc_xr_add(mdoc->meta.msec, n->string, -1, -1); } static void @@ -1205,9 +1203,8 @@ post_nm(POST_ARGS) n = mdoc->last; - if ((n->sec == SEC_NAME || n->sec == SEC_SYNOPSIS) && - n->child != NULL && n->child->type == ROFFT_TEXT && - mdoc->meta.msec != NULL) + if (n->sec == SEC_NAME && n->child != NULL && + n->child->type == ROFFT_TEXT && mdoc->meta.msec != NULL) mandoc_xr_add(mdoc->meta.msec, n->child->string, -1, -1); if (n->last != NULL && Index: mansearch.c =================================================================== RCS file: /home/cvs/mandoc/mandoc/mansearch.c,v retrieving revision 1.75 retrieving revision 1.76 diff -Lmansearch.c -Lmansearch.c -u -p -r1.75 -r1.76 --- mansearch.c +++ mansearch.c @@ -171,7 +171,9 @@ mansearch(const struct mansearch *search page = dbm_page_get(rp->page); if (lstmatch(search->sec, page->sect) == 0 || - lstmatch(search->arch, page->arch) == 0) + lstmatch(search->arch, page->arch) == 0 || + (search->argmode == ARG_NAME && + rp->bits <= (int32_t)(NAME_SYN & NAME_MASK))) continue; if (res == NULL) { @@ -452,14 +454,28 @@ lstlen(const char *cp, size_t sep) { size_t sz; - for (sz = 0;; sz++) { - if (cp[0] == '\0') { - if (cp[1] == '\0') - break; - sz += sep - 1; - } else if (cp[0] < ' ') - sz--; - cp++; + for (sz = 0; *cp != '\0'; cp++) { + + /* Skip names appearing only in the SYNOPSIS. */ + if (*cp <= (char)(NAME_SYN & NAME_MASK)) { + while (*cp != '\0') + cp++; + continue; + } + + /* Skip name class markers. */ + if (*cp < ' ') + cp++; + + /* Print a separator before each but the first string. */ + if (sz) + sz += sep; + + /* Copy one string. */ + while (*cp != '\0') { + sz++; + cp++; + } } return sz; } @@ -471,19 +487,34 @@ lstlen(const char *cp, size_t sep) static void lstcat(char *buf, size_t *i, const char *cp, const char *sep) { - const char *s; + const char *s; + size_t i_start; - for (;;) { - if (cp[0] == '\0') { - if (cp[1] == '\0') - break; + for (i_start = *i; *cp != '\0'; cp++) { + + /* Skip names appearing only in the SYNOPSIS. */ + if (*cp <= (char)(NAME_SYN & NAME_MASK)) { + while (*cp != '\0') + cp++; + continue; + } + + /* Skip name class markers. */ + if (*cp < ' ') + cp++; + + /* Print a separator before each but the first string. */ + if (*i > i_start) { s = sep; while (*s != '\0') buf[(*i)++] = *s++; - } else if (cp[0] >= ' ') - buf[(*i)++] = cp[0]; - cp++; + } + + /* Copy one string. */ + while (*cp != '\0') + buf[(*i)++] = *cp++; } + } /* -- To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv