discuss@mandoc.bsd.lv
 help / color / mirror / Atom feed
* pod2mdoc, docbook2mdoc
@ 2014-03-30 18:23 ` Kristaps Dzonsons
  2014-03-31  9:09   ` Thomas Klausner
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-03-30 18:23 UTC (permalink / raw)
  To: discuss

Hiya folks,

Long story short: any Perl POD or DocBook people out there?

In recent chatter on groff's mailing list, there's been talk about 
manpages.  For reasons (maybe) clear to anybody reading these, I thought 
I'd write some tools that might help people start putting their weight 
behind mdoc.

To wit,

  http://mdocml.bsd.lv/pod2mdoc
  http://mdocml.bsd.lv/docbook2mdoc

The second requires libexpat (for the XML parsing), but beyond that, 
they're both standalone utilities.  No Perl.  No weird... XSLT... 
jade... whatever things.

I've little time these days, so I thought maybe somebody would be 
interested in delving into these utilities and seeing how they stack up 
against the real world.  There's a lot of meat to both.

pod2mdoc does a decent job given the many POD manuals out there, but 
could use more "semantic suck".  (E.g., pod2man is able to "recognise" 
functions--that'd be swell for us, too.  The SYNOPSIS is a horror, too.)

docbook2mdoc is a larger project just because docbook has so many 
elements.  So mechanically larger, not larger in terms of brains.  It 
also needs a bit of "doclifter desucking", as many docbook manuals I've 
seen are from doclifter, which mangles some things (e.g., Xr).

Anybody interested?  Bueller?  Bueller?

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-30 18:23 ` pod2mdoc, docbook2mdoc Kristaps Dzonsons
@ 2014-03-31  9:09   ` Thomas Klausner
  2014-03-31 10:30     ` Kristaps Dzonsons
  0 siblings, 1 reply; 17+ messages in thread
From: Thomas Klausner @ 2014-03-31  9:09 UTC (permalink / raw)
  To: discuss

On Sun, Mar 30, 2014 at 08:23:05PM +0200, Kristaps Dzonsons wrote:
> In recent chatter on groff's mailing list, there's been talk about manpages.
> For reasons (maybe) clear to anybody reading these, I thought I'd write some
> tools that might help people start putting their weight behind mdoc.
> 
> To wit,
> 
>  http://mdocml.bsd.lv/pod2mdoc
>  http://mdocml.bsd.lv/docbook2mdoc
> 
> The second requires libexpat (for the XML parsing), but beyond that, they're
> both standalone utilities.  No Perl.  No weird... XSLT... jade... whatever
> things.
> 
> I've little time these days, so I thought maybe somebody would be interested
> in delving into these utilities and seeing how they stack up against the
> real world.  There's a lot of meat to both.

This sounds interesting. To make it easier for pkgsrc people to do
that, I've just added packages for both of them: textproc/pod2mdoc and
textproc/docbook2mdoc.

Thanks for writing these!
 Thomas
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31  9:09   ` Thomas Klausner
@ 2014-03-31 10:30     ` Kristaps Dzonsons
  2014-03-31 16:13       ` Ingo Schwarze
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-03-31 10:30 UTC (permalink / raw)
  To: discuss

> This sounds interesting. To make it easier for pkgsrc people to do
> that, I've just added packages for both of them: textproc/pod2mdoc and
> textproc/docbook2mdoc.

Thomas and Anthony,

Thank you for doing this!  Though I worry... is this not moving too 
fast? These tools are fresh--real fresh--and moving fast.  Is it not too 
soon for committing to a port?  pod2mdoc works over all the POD I found 
(not being a Perl junkie, however, I just used what's in the Perl 
distribution), but docbook2mdoc still needs a lot of work.  Fact is, I 
just don't know the "real world" landscape of ports using docbook (or 
POD, for that matter).  Would it not be better to have somebody using 
(or porting) POD or DocBook systems to break them in before putting them 
out for general consumption?  Thoughts?

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 10:30     ` Kristaps Dzonsons
@ 2014-03-31 16:13       ` Ingo Schwarze
  2014-03-31 19:40         ` Kristaps Dzonsons
  0 siblings, 1 reply; 17+ messages in thread
From: Ingo Schwarze @ 2014-03-31 16:13 UTC (permalink / raw)
  To: Kristaps Dzonsons; +Cc: discuss

Hi Kristaps,

Kristaps Dzonsons wrote on Mon, Mar 31, 2014 at 12:30:17PM +0200:

> Thank you for doing this!  Though I worry... is this not moving too
> fast? These tools are fresh--real fresh--and moving fast.  Is it not
> too soon for committing to a port?

I wouldn't worry too much about that.  The version number 0.0.5
is telling enough, and the package description can say a few words
too.  So i see no big risk there, but possible benefit allowing
cleaner packaging, installation and deinstallation of experimental
tools.

> pod2mdoc works over all the POD
> I found (not being a Perl junkie, however, I just used what's in the
> Perl distribution), but docbook2mdoc still needs a lot of work.

Indeed.  As far as i can see, the biggest problem right now is that
it usually exits in the middle of each processed file, rendering
merely the beginning.  To be usable, it has to become more resilient.

At first sight, it seems that the "ps->stop = 1;" in the
function xml_elem_start() cause the problem, but i didn't
try to come up with a fix yet.

  cvs.1.xml:33:6: unknown node "indexterm"
  lynx.1.xml:83:0: unknown node "literallayout"
  sqlite3.1.xml:108:0: unknown node "markup"

> Fact is, I just don't know the "real world" landscape of ports using
> docbook (or POD, for that matter).  Would it not be better to have
> somebody using (or porting) POD or DocBook systems to break them in
> before putting them out for general consumption?  Thoughts?

A good corpus of test input would be the man(7) manuals in OpenBSD
base, imho.  As far as these are POD input, they are good testing
candidates for pod2mdoc; i consider the test chain POD -> pod2man ->
man(7) -> doclifter -> DocBook-XML -> docbook2mdoc mostly useless.
As far as they are real man(7), not from POD, they are good testing
candidates for docbook2mdoc, i.e. using the testing chain
man(7) -> doclifter -> DocBook-XML -> docbook2mdoc -> mdoc(7).

At least at first, i think the main use for docbook2mdoc will
be as a postprocessor for doclifter, so getting it to handle
the subset of DocBook-XML produced by doclifter is the second
most useful step to take after preventing it from exiting
prematurely (see above).

In any case, docbook2mdoc seems much more useful than pod2mdoc,
at least in a short-term perspective.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 16:13       ` Ingo Schwarze
@ 2014-03-31 19:40         ` Kristaps Dzonsons
  2014-03-31 20:57           ` Ingo Schwarze
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-03-31 19:40 UTC (permalink / raw)
  To: Ingo Schwarze; +Cc: discuss

> I wouldn't worry too much about that.  The version number 0.0.5
> is telling enough, and the package description can say a few words
> too.  So i see no big risk there, but possible benefit allowing
> cleaner packaging, installation and deinstallation of experimental
> tools.

Ingo,

Thank you for your input!  This looks logical to me.  I'll drop a note 
here when I bump versions in a meaningful way--does that work?  Note for 
those with commit access that the modules are in 
/usr/vhosts/mdocml.bsd.lv/pod2mdoc and so.  (The commit hook in CVSROOT 
should probably be updated to mail commits on the new modules...)

>> pod2mdoc works over all the POD
>> I found (not being a Perl junkie, however, I just used what's in the
>> Perl distribution), but docbook2mdoc still needs a lot of work.
>
> Indeed.  As far as i can see, the biggest problem right now is that
> it usually exits in the middle of each processed file, rendering
> merely the beginning.  To be usable, it has to become more resilient.

Yes.  This is because I don't support a huge set of docbook elements 
yet--and the parser exits when it encounters an unknown one to spur me 
to add as many new ones as possible.  My plan (as you've noted below) is 
to start by having it accept all of doclifter's DocBook, then focus on 
the rest, possibly relaxing the parser to accept unknown nodes if we get 
lazy.

Fortunately, it's easy to add new nodes to docbook2mdoc!  They need an 
entry in extern.h, then the rules of parenting (this is the most 
annoying part--both children and parents need to be added in) in 
rules.c, then a handler in pnode_print().  Most handler types have 
already been implemented, so it's just putting together the blocks and 
going by example.  Attributes work more or less the same way.

> At first sight, it seems that the "ps->stop = 1;" in the
> function xml_elem_start() cause the problem, but i didn't
> try to come up with a fix yet.
>
>    cvs.1.xml:33:6: unknown node "indexterm"
>    lynx.1.xml:83:0: unknown node "literallayout"
>    sqlite3.1.xml:108:0: unknown node "markup"

This is by design--it just means that I need to add a NODE_INDEXTERM, 
etc.  As mentioned, I'm going to start with doclifter's nodes.

>> Fact is, I just don't know the "real world" landscape of ports using
>> docbook (or POD, for that matter).  Would it not be better to have
>> somebody using (or porting) POD or DocBook systems to break them in
>> before putting them out for general consumption?  Thoughts?
>
> A good corpus of test input would be the man(7) manuals in OpenBSD
> base, imho.  As far as these are POD input, they are good testing
> candidates for pod2mdoc; i consider the test chain POD -> pod2man ->
> man(7) -> doclifter -> DocBook-XML -> docbook2mdoc mostly useless.
> As far as they are real man(7), not from POD, they are good testing
> candidates for docbook2mdoc, i.e. using the testing chain
> man(7) -> doclifter -> DocBook-XML -> docbook2mdoc -> mdoc(7).
>
> At least at first, i think the main use for docbook2mdoc will
> be as a postprocessor for doclifter, so getting it to handle
> the subset of DocBook-XML produced by doclifter is the second
> most useful step to take after preventing it from exiting
> prematurely (see above).

Agreed!

> In any case, docbook2mdoc seems much more useful than pod2mdoc,
> at least in a short-term perspective.

My general thought on pod2mdoc was to avoid invoking perl(1) just to run 
pod2man(1) in the normal chain of looking at POD documents.  I also 
thought that, with a few more pod2mdoc smarts, we'll be able to deeply 
mine those documents with mandocdb.  From what I understand, they're 
kinda opaque right now.

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 19:40         ` Kristaps Dzonsons
@ 2014-03-31 20:57           ` Ingo Schwarze
  2014-03-31 21:30             ` Thomas Klausner
  0 siblings, 1 reply; 17+ messages in thread
From: Ingo Schwarze @ 2014-03-31 20:57 UTC (permalink / raw)
  To: discuss

Hi Kristaps,

Kristaps Dzonsons wrote on Mon, Mar 31, 2014 at 09:40:10PM +0200:

> Note for those with commit access that the modules are in
> /usr/vhosts/mdocml.bsd.lv/pod2mdoc and so.

Err, not quite, there is a "cvs/" missing in the middle.
The following works (if you have commit access):

  export CVSROOT=${LOGIN}@bsd.lv:/usr/vhosts/mdocml.bsd.lv/cvs
  cvs co pod2mdoc
  cvs co docbook2mdoc

> (The commit hook in CVSROOT should probably be updated to mail
> commits on the new modules...)

Done.

The commitinfo/loginfo syntax requires BREs, so there is no support
for '|', so i had to copy the respective lines, but nothing much
is wrong with that, i think.

> Ingo Schwarze wrote:

>> In any case, docbook2mdoc seems much more useful than pod2mdoc,
>> at least in a short-term perspective.

> My general thought on pod2mdoc was to avoid invoking perl(1) just to
> run pod2man(1) in the normal chain of looking at POD documents.  I
> also thought that, with a few more pod2mdoc smarts, we'll be able to
> deeply mine those documents with mandocdb.  From what I understand,
> they're kinda opaque right now.

All true.  I didn't mean to call pod2mdoc useless, no doubt it can
bring us benefits when it will be polished.

What i meant to say is that docbook2mdoc is even better because
it might help people convert man(7) manuals to mdoc(7), thus
thwarting esr's evil plan to migrate the world to DocBook.  :-)

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 20:57           ` Ingo Schwarze
@ 2014-03-31 21:30             ` Thomas Klausner
  2014-03-31 21:54               ` Kristaps Dzonsons
  0 siblings, 1 reply; 17+ messages in thread
From: Thomas Klausner @ 2014-03-31 21:30 UTC (permalink / raw)
  To: discuss

On Mon, Mar 31, 2014 at 10:57:36PM +0200, Ingo Schwarze wrote:
> > My general thought on pod2mdoc was to avoid invoking perl(1) just to
> > run pod2man(1) in the normal chain of looking at POD documents.  I
> > also thought that, with a few more pod2mdoc smarts, we'll be able to
> > deeply mine those documents with mandocdb.  From what I understand,
> > they're kinda opaque right now.
> 
> All true.  I didn't mean to call pod2mdoc useless, no doubt it can
> bring us benefits when it will be polished.

Actually, I think it can be very useful. NetBSD regularly imports
openssl, and the documentation for that is written in .pod files.
So for me the most interesting input files are these :) *hinthint*
Just in case you don't have enough test cases yet :)

Thanks,
 Thomas
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 21:30             ` Thomas Klausner
@ 2014-03-31 21:54               ` Kristaps Dzonsons
  2014-03-31 22:21                 ` Ingo Schwarze
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-03-31 21:54 UTC (permalink / raw)
  To: discuss

>>> My general thought on pod2mdoc was to avoid invoking perl(1) just to
>>> run pod2man(1) in the normal chain of looking at POD documents.  I
>>> also thought that, with a few more pod2mdoc smarts, we'll be able to
>>> deeply mine those documents with mandocdb.  From what I understand,
>>> they're kinda opaque right now.
>>
>> All true.  I didn't mean to call pod2mdoc useless, no doubt it can
>> bring us benefits when it will be polished.
>
> Actually, I think it can be very useful. NetBSD regularly imports
> openssl, and the documentation for that is written in .pod files.
> So for me the most interesting input files are these :) *hinthint*
> Just in case you don't have enough test cases yet :)

Thomas,

Can you send a quick tarball of these interesting files?

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 21:54               ` Kristaps Dzonsons
@ 2014-03-31 22:21                 ` Ingo Schwarze
  2014-03-31 22:31                   ` Thomas Klausner
  0 siblings, 1 reply; 17+ messages in thread
From: Ingo Schwarze @ 2014-03-31 22:21 UTC (permalink / raw)
  To: discuss

Hi,

Kristaps Dzonsons wrote on Mon, Mar 31, 2014 at 11:54:43PM +0200:
> Thomas Klausner wrote:

>> Actually, I think it can be very useful. NetBSD regularly imports
>> openssl, and the documentation for that is written in .pod files.
>> So for me the most interesting input files are these :) *hinthint*
>> Just in case you don't have enough test cases yet :)

Oh, indeed, the NetBSD base system does not contain Perl, right?
So you can't easily run pod2man(1) during the build?
I didn't think about that because in OpenBSD, we do just that...

> Can you send a quick tarball of these interesting files?

On your OpenBSD system, they are:

  /usr/src/lib/libssl/src/*/*/*.pod
  /usr/src/lib/libssl/src/*/*.pod

So no need to install an additional tarball...  :-)

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 22:21                 ` Ingo Schwarze
@ 2014-03-31 22:31                   ` Thomas Klausner
  2014-04-03 12:02                     ` Kristaps Dzonsons
  0 siblings, 1 reply; 17+ messages in thread
From: Thomas Klausner @ 2014-03-31 22:31 UTC (permalink / raw)
  To: discuss

On Tue, Apr 01, 2014 at 12:21:18AM +0200, Ingo Schwarze wrote:
> Oh, indeed, the NetBSD base system does not contain Perl, right?
> So you can't easily run pod2man(1) during the build?
> I didn't think about that because in OpenBSD, we do just that...

Yes, that's right.
After import, pod2man is run manually once.
I'm not very happy about the output and expect better from Kristaps :)

> So no need to install an additional tarball...  :-)

Oh, ok. I had sent it in the meantime.
 Thomas
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-03-31 22:31                   ` Thomas Klausner
@ 2014-04-03 12:02                     ` Kristaps Dzonsons
  2014-04-03 12:17                       ` Thomas Klausner
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-04-03 12:02 UTC (permalink / raw)
  To: discuss

>> Oh, indeed, the NetBSD base system does not contain Perl, right?
>> So you can't easily run pod2man(1) during the build?
>> I didn't think about that because in OpenBSD, we do just that...
>
> Yes, that's right.
> After import, pod2man is run manually once.
> I'm not very happy about the output and expect better from Kristaps :)

Thomas,

I suggest you check out the latest version... run as:

  pod2mdoc foo.pm | mandoc

The output is much nicer than pod2man in the same way.  Lots more 
semantic content as mentioned in the "Smarts" subsection of the manual. 
  Most visibly, since the SYNOPSIS is now rendered with Nm, Oo/Oc, Ar, 
and Fl, mandoc can properly format sections.

I specifically used the SSL manpages and Perl distribution as a basis.

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 12:02                     ` Kristaps Dzonsons
@ 2014-04-03 12:17                       ` Thomas Klausner
  2014-04-03 12:33                         ` Kristaps Dzonsons
  0 siblings, 1 reply; 17+ messages in thread
From: Thomas Klausner @ 2014-04-03 12:17 UTC (permalink / raw)
  To: discuss

Hi Kristaps!

On Thu, Apr 03, 2014 at 02:02:41PM +0200, Kristaps Dzonsons wrote:
> I suggest you check out the latest version... run as:
> 
>  pod2mdoc foo.pm | mandoc
> 
> The output is much nicer than pod2man in the same way.  Lots more semantic
> content as mentioned in the "Smarts" subsection of the manual.  Most
> visibly, since the SYNOPSIS is now rendered with Nm, Oo/Oc, Ar, and Fl,
> mandoc can properly format sections.

Thanks.

I've tried DSA_SIG_new.pod, basically at random. Here's the output:

.Dd July 20, 2009
.Dt DSA_SIG_NEW 1
.Os
.Sh NAME
.Nm DSA_SIG_new ,
.Nm DSA_SIG_free 
.Nd allocate and free DSA signature objects
.Sh SYNOPSIS
.Bd -literal
 #include <openssl/dsa.h>
.Ed
.Bd -literal
 DSA_SIG *DSA_SIG_new(void);
.Ed
.Bd -literal
 void   DSA_SIG_free(DSA_SIG *a);
.Ed
.Sh DESCRIPTION
DSA_SIG_new() allocates and initializes a 
.Sy DSA_SIG
structure.
.Pp
DSA_SIG_free() frees the 
.Sy DSA_SIG
structure and its components. The
values are erased before the memory is returned to the system.
.Sh RETURN VALUES
If the allocation fails, DSA_SIG_new() returns 
.Sy NULL
and sets an
error code that can be obtained by
.Xr ERR_get_error 3 Ns  .
Otherwise it returns a pointer
to the newly allocated structure.
.Pp
DSA_SIG_free() returns no value.
.Sh SEE ALSO
.Xr dsa 3 Ns  ,
.Xr ERR_get_error 3 Ns  ,
.Xr DSA_do_sign 3
.Sh HISTORY
DSA_SIG_new() and DSA_SIG_free() were added in OpenSSL 0.9.3.

It's already quite nice!

Some suggestions:
* foo() -> .Fn foo
* too many Ns before punctuation
* section number is 1 instead of 3 (not sure how to detect that)
* .Bd -literal + #include <foo> + Ed -> .In foo
* splitting up the functions in the SYNOPSIS the same way is probably too much effort?

Related question:
How do people use Sy vs. Dv?

I personally always mark up NULL as a Dv and similarly for most other
defined C symbols.

Thanks,
 Thomas
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 12:17                       ` Thomas Klausner
@ 2014-04-03 12:33                         ` Kristaps Dzonsons
  2014-04-03 13:53                           ` Ingo Schwarze
  0 siblings, 1 reply; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-04-03 12:33 UTC (permalink / raw)
  To: discuss

Thomas,

Thanks for the suggestions!  In general, most of these fall into the 
"Smarts" category.  My aim is to start with simple markup and 
properness, then get into the smarty stuff.  The biggest improvement 
since the last version is in non-library applications (e.g. 
libssl/src/doc/apps).

> Some suggestions:
> * foo() -> .Fn foo

I know that pod2man does this--that'd be an easy smart.

> * too many Ns before punctuation

This is a big one, but complicated to handle properly.  Getting the 
spacing right was really a nightmare.  Consider:

B<foo>B< bar>
B<foo >B<bar>
B<foo B<bar Z<>><B foo> >

and so on.  pod2mdoc does fine with all but one combination noted in the 
manual.  As mandoc(1) will ignore the superfluous "Nm", it's safer to 
leave it as-is than try to be smarter.

> * section number is 1 instead of 3 (not sure how to detect that)

perlpod says that, absent a suffix of ".pm", the manual should be 
considered a section 1.  I guess that ".pod" should be treated 
similarly--can you verify that?  I also have a #define for which section 
should be the Perl module section.  OpenBSD has 3p.  (And a note that 
pod2mdoc needs to be changed if it's redefined.)

> * .Bd -literal + #include <foo> + Ed -> .In foo

Yes, this is TODO for more Smarts.

> * splitting up the functions in the SYNOPSIS the same way is probably too much effort?

Same thing: more Smarts...

> Related question:
> How do people use Sy vs. Dv?

I use Dv for preprocessor symbols and NULL.  I've never used Sy on my 
own.  A fairly low-hanging smart would be to decorate NULL.

> I personally always mark up NULL as a Dv and similarly for most other
> defined C symbols.

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 12:33                         ` Kristaps Dzonsons
@ 2014-04-03 13:53                           ` Ingo Schwarze
  2014-04-03 16:51                             ` Kristaps Dzonsons
  2014-04-03 22:06                             ` Thomas Klausner
  0 siblings, 2 replies; 17+ messages in thread
From: Ingo Schwarze @ 2014-04-03 13:53 UTC (permalink / raw)
  To: discuss

Hi Kristaps and Thomas,

Kristaps Dzonsons wrote on Thu, Apr 03, 2014 at 02:33:50PM +0200:
> Thomas Klausner wrote:

>> Some suggestions:

Thomas, you are already aiming high.
All your suggestions definitely make sense, but
they are beyond what the current state-of-the-art tools can do.

For comparison, i have run:

  pod2man --official --release="OpenBSD 5.5" --center=OpenSSL \
    --section=3 --name=DSA_SIG_NEW DSA_SIG_new.pod > DSA_SIG_new.man
  doclifter < DSA_SIG_new.man > DSA_SIG_new.xml

The idea is that

 * pod2man contains some Smarts, so the .man is already going to be
   semantically enriched compared to the *.pod.
 * doclifter contains many more Smarts, so the .xml is going to be
   much more semantically enriched compared to the *.man.
   This grade of enrichment is what i call the state of the art
   regarding Smarts.  Eric S. Raymond basically says that it took
   him a decade to get there, and he calls it very difficult.
   Which doesn't mean that Kristaps can't do better in a few days,
   but...  well...  ESR isn't exactly known for stupidity in the
   free software world, either, so bear with Kristaps.  :)

>> * foo() -> .Fn foo

> I know that pod2man does this--that'd be an easy smart.

Wait, pod2man(1) code is not exactly easy.  It does contain
quite some Smarts, and it isn't pretty.  That's exactly why
i chose not to embark on this project some time ago.

But let's see what current tools do:

 - *.pod has no markup at all.
 - *.man has:
      - in the SYNOPSIS: .Vb, which is basically .nf (almost nothing)
      - in the text: \fI...\fR (presentational)
 - *.xml has:
      - in the SYNOPSIS:
        <funcdef><function>DSA_SIG</function> *DSA_SIG_new</funcdef>
        which is outright WRONG (DSA_SIG is not a function,
        and DSA_SIG_new lacks markup)
        <funcdef>void <function>DSA_SIG_free</function></funcdef>
        which is a bit better, but the function type still lacks markup
      - in the text:
        <emphasis remap='I'>DSA_SIG_new()</emphasis>
        <emphasis remap='I'>DSA_SIG_free()</emphasis>
        So even though DSA_SIG_free() is recognized as a function
        in the SYNOPSIS, doclifter is not smart enough to remember
        that for text formatting.

So what you ask for is well beyond the state of the art.

I'm not impressed by doclifter here.

>> * too many Ns before punctuation

> This is a big one, but complicated to handle properly.  Getting the
> spacing right was really a nightmare.  Consider:
> 
> B<foo>B< bar>
> B<foo >B<bar>
> B<foo B<bar Z<>><B foo> >
> 
> and so on.  pod2mdoc does fine with all but one combination noted in
> the manual.  As mandoc(1) will ignore the superfluous "Nm", it's
> safer to leave it as-is than try to be smarter.

Sure, the .Ns isn't incorrect, just superfluous.

By the way, mandoc(1) already contains the necessary logic,
in file mdoc.c, function isdelim().  When you are about to print .Ns,
the next word is a single character, and isdelim() returns DELIM_CLOSE,
you can skip the .Ns.  With sugar on top.

>> * section number is 1 instead of 3 (not sure how to detect that)

There is no safe way.  The build system is supposed to supply that
information externally, see the pod2man command above.

That said, you could try to be Smart.  If the SYNOPSIS contains .Fn,
you are more likely to be in section 3 than section 1.

> perlpod says that, absent a suffix of ".pm", the manual should be
> considered a section 1.  I guess that ".pod" should be treated
> similarly--can you verify that?

No, it should not.  The *.pod suffix just means "plain old documentation",
that can be any section, even 3p - though 1, 3, 5, 7, 8 are maybe more
likely than 3p because most Perlmodule docs are embedded in the *.pm.
Anyway, you shouldn't conclude anything from a *.pod filename.

> I also have a #define for which
> section should be the Perl module section.  OpenBSD has 3p.  (And a
> note that pod2mdoc needs to be changed if it's redefined.)

>> * .Bd -literal + #include <foo> + Ed -> .In foo

> Yes, this is TODO for more Smarts.

 - *.pod only has a leading blank character.
 - *.man has .Vb, see above, almost nothing.
 - *.xml has:

   <funcsynopsisinfo>
    #include &lt;openssl/dsa.h&gt;

   </funcsynopsisinfo>

That's pathetic.  It doesn't even remove the leading blank
and provides almost no semantics.  So again, Thomas, you aim
well beyond the state of the art.  That's already the second
case of an easy smart that doclifter fails to deliver.

>> * splitting up the functions in the SYNOPSIS the same way
>>   is probably too much effort?

> Same thing: more Smarts...

See above:  The state of the art is to try and fail.  :-o

>> Related question:
>> How do people use Sy vs. Dv?
>> I personally always mark up NULL as a Dv and similarly for most
>> other defined C symbols.

> I use Dv for preprocessor symbols and NULL.  I've never used Sy on
> my own.  A fairly low-hanging smart would be to decorate NULL.

Confirmed, i'd call .Dv NULL best practice.

 - *.pod has B<NULL>
 - *.man has \fB\s-1NULL\s0\fR
 - *.xml has <emphasis remap='B'><?troff ps -1?>NULL<?troff ps 0?></emphasis>

Again, pathetic.  No attempt at being smart.

So, to summarize the state of the art:

 * #include: Do not even try.
 * functions in the SYNOPSIS: Try in a half-assed way
   and partially fail even regarding the part that was tried.
 * functions in text: Do not even try. 
 * NULL: Do not even try.

No, i'm not impressed by the state of the art.

The bright side is that this means Kristaps will almost certainly
be able to do better, without prohibitive effort.  Given the
amount of boasting by ESR, that's a bit surprising...

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 13:53                           ` Ingo Schwarze
@ 2014-04-03 16:51                             ` Kristaps Dzonsons
  2014-04-03 22:06                             ` Thomas Klausner
  1 sibling, 0 replies; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-04-03 16:51 UTC (permalink / raw)
  To: discuss

>>> * .Bd -literal + #include <foo> + Ed -> .In foo
>
>> Yes, this is TODO for more Smarts.
>
>   - *.pod only has a leading blank character.
>   - *.man has .Vb, see above, almost nothing.
>   - *.xml has:
>
>     <funcsynopsisinfo>
>      #include &lt;openssl/dsa.h&gt;
>
>     </funcsynopsisinfo>
>
> That's pathetic.  It doesn't even remove the leading blank
> and provides almost no semantics.  So again, Thomas, you aim
> well beyond the state of the art.  That's already the second
> case of an easy smart that doclifter fails to deliver.

With this last commit, leading #include statements in SYNOPSIS verbatim 
paragraphs are rendered with In.

Example:

=head1 SYNOPSIS

  #include <openssl/ssl.h>
  #include <asdfopenssl/ssl.h>
  SSL *SSL_new(SSL_CTX *ctx);

Now rendered as:

.Sh SYNOPSIS
.In openssl/ssl.h
.In osdfpenssl/ssl.h
.Bd -literal
  SSL *SSL_new(SSL_CTX *ctx);
.Ed

So state of the art!
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 13:53                           ` Ingo Schwarze
  2014-04-03 16:51                             ` Kristaps Dzonsons
@ 2014-04-03 22:06                             ` Thomas Klausner
  2014-04-04 13:43                               ` Kristaps Dzonsons
  1 sibling, 1 reply; 17+ messages in thread
From: Thomas Klausner @ 2014-04-03 22:06 UTC (permalink / raw)
  To: discuss

On Thu, Apr 03, 2014 at 03:53:02PM +0200, Ingo Schwarze wrote:
> Thomas, you are already aiming high.

Yes!

> All your suggestions definitely make sense, but
> they are beyond what the current state-of-the-art tools can do.
...
>    Eric S. Raymond basically says that it took
>    him a decade to get there, and he calls it very difficult.

A few years ago when I last tried doclifter I wasn't very impressed,
and your comparison today doesn't impress me much either...
Thanks for that comparison, btw!

>    Which doesn't mean that Kristaps can't do better in a few days,
>    but...  well...  ESR isn't exactly known for stupidity in the
>    free software world, either, so bear with Kristaps.  :)

I hope I wasn't misunderstood:
Kristaps, you're doing great!
I didn't mean to put you down in any way.

> No, it should not.  The *.pod suffix just means "plain old documentation",
> that can be any section, even 3p - though 1, 3, 5, 7, 8 are maybe more
> likely than 3p because most Perlmodule docs are embedded in the *.pm.
> Anyway, you shouldn't conclude anything from a *.pod filename.

So worst-case we could tell pod2man with a flag which section the man
page belongs to.
 Thomas
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: pod2mdoc, docbook2mdoc
  2014-04-03 22:06                             ` Thomas Klausner
@ 2014-04-04 13:43                               ` Kristaps Dzonsons
  0 siblings, 0 replies; 17+ messages in thread
From: Kristaps Dzonsons @ 2014-04-04 13:43 UTC (permalink / raw)
  To: discuss

>> No, it should not.  The *.pod suffix just means "plain old documentation",
>> that can be any section, even 3p - though 1, 3, 5, 7, 8 are maybe more
>> likely than 3p because most Perlmodule docs are embedded in the *.pm.
>> Anyway, you shouldn't conclude anything from a *.pod filename.
>
> So worst-case we could tell pod2man with a flag which section the man
> page belongs to.

Thomas,

This feature is of course already in pod2mdoc.  I just copied all the 
relevant flags from pod2man (-s is the one you're looking for).  See the 
manpage for details...

Best,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2014-04-04 13:43 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <sfid-H20140330-202333-+022.64-1@spamfilter.osbf.lua>
2014-03-30 18:23 ` pod2mdoc, docbook2mdoc Kristaps Dzonsons
2014-03-31  9:09   ` Thomas Klausner
2014-03-31 10:30     ` Kristaps Dzonsons
2014-03-31 16:13       ` Ingo Schwarze
2014-03-31 19:40         ` Kristaps Dzonsons
2014-03-31 20:57           ` Ingo Schwarze
2014-03-31 21:30             ` Thomas Klausner
2014-03-31 21:54               ` Kristaps Dzonsons
2014-03-31 22:21                 ` Ingo Schwarze
2014-03-31 22:31                   ` Thomas Klausner
2014-04-03 12:02                     ` Kristaps Dzonsons
2014-04-03 12:17                       ` Thomas Klausner
2014-04-03 12:33                         ` Kristaps Dzonsons
2014-04-03 13:53                           ` Ingo Schwarze
2014-04-03 16:51                             ` Kristaps Dzonsons
2014-04-03 22:06                             ` Thomas Klausner
2014-04-04 13:43                               ` Kristaps Dzonsons

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).