Mdocdate

Mdocdate

[Dag-Erling Smørgrav]

[-- Attachment #1: Type: text/plain, Size: 2327 bytes --]

The attached patch improves the Mdocdate code and updates the
documentation to match.

 - in a2time():

   Skip past "$Mdocdate: " if it exists instead of relying on the caller
   to specify it in the format string.

   Accept trailing text after a successful match (e.g. "$").

   If the first character after a successful match is 'Z', use UTC
   instead of the local time zone.

 - in time2a():

   Use a sufficiently large stack buffer and mandoc_strdup() instead of
   trying to guess at the correct length.

 - in mandoc_normdate():

   Fix several logic errors (inverted tests), add a test for the
   Subversion %d format, and simplify the code.

 - in the man page:

   Document how to use Mdocdate with Subversion.

Given the following test cases:

% cat tests
February 7 2036
February  7 2036
February 07 2036
February 7, 2036
February  7, 2036
February 07, 2036
Feb 7 2036
Feb  7 2036
Feb 07 2036
Feb 7, 2036
Feb  7, 2036
Feb 07, 2036
2036-02-07
2036-02-7
2036-2-07
2036-2-7
$Mdocdate: February 7 2036 $
$Mdocdate: 2036-02-07 06:28:16Z $

The current code (1.13.3) produces the following output:

% while read d ; do echo ".Dd $d" | mandoc | tail -1 ; done <tests
                                February 7 2036
                                February 7 2036
                               February 07 2036
                               February 7, 2036
                               February 7, 2036
                               February 7, 2036
                                  Feb 7 2036
                                  Feb 7 2036
                                  Feb 07 2036
                               February 7, 2036
                               February 7, 2036
                               February 7, 2036
                                  2036-02-07
                                   2036-02-7
                                   2036-2-07
                                   2036-2-7
                               February 7, 2036
                       $Mdocdate: 2036-02-07 06:28:16Z $

My version produces the same output for all cases:

% while read d ; do echo ".Dd $d" | ./mandoc | tail -1 ; done <tests | uniq
                               February 7, 2036

DES
--
Dag-Erling Smørgrav - des@des.no


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: mdocdate.diff --]
[-- Type: text/x-patch, Size: 2611 bytes --]

--- mandoc.c.orig
+++ mandoc.c
@@ -468,13 +468,15 @@
 	char		*pp;
 
 	memset(&tm, 0, sizeof(struct tm));
+	if (strncmp(p, "$" "Mdocdate: ", 11) == 0)
+		p += 11;
 
 	pp = NULL;
 #if HAVE_STRPTIME
 	pp = strptime(p, fmt, &tm);
 #endif
-	if (NULL != pp && '\0' == *pp) {
-		*t = mktime(&tm);
+	if (NULL != pp) {
+		*t = ('Z' == *p) ? timegm(&tm) : mktime(&tm);
 		return(1);
 	}
 
@@ -485,59 +487,32 @@
 time2a(time_t t)
 {
 	struct tm	*tm;
-	char		*buf, *p;
-	size_t		 ssz;
-	int		 isz;
+	char		 buf[32];
 
 	tm = localtime(&t);
-	if (tm == NULL)
-		return(NULL);
-
-	/*
-	 * Reserve space:
-	 * up to 9 characters for the month (September) + blank
-	 * up to 2 characters for the day + comma + blank
-	 * 4 characters for the year and a terminating '\0'
-	 */
-	p = buf = mandoc_malloc(10 + 4 + 4 + 1);
-
-	if (0 == (ssz = strftime(p, 10 + 1, "%B ", tm)))
-		goto fail;
-	p += (int)ssz;
-
-	if (-1 == (isz = snprintf(p, 4 + 1, "%d, ", tm->tm_mday)))
-		goto fail;
-	p += isz;
-
-	if (0 == strftime(p, 4 + 1, "%Y", tm))
-		goto fail;
-	return(buf);
-
-fail:
-	free(buf);
-	return(NULL);
+	if (tm == NULL || 0 == strftime(buf, sizeof buf, "%B %d, %Y", tm))
+		return (NULL);
+	return(mandoc_strdup(buf));
 }
 
 char *
 mandoc_normdate(struct mparse *parse, char *in, int ln, int pos)
 {
-	char		*out;
 	time_t		 t;
 
-	if (NULL == in || '\0' == *in ||
-	    0 == strcmp(in, "$" "Mdocdate$")) {
+	if (NULL == in || '\0' == *in || 0 == strcmp(in, "$" "Mdocdate$")) {
 		mandoc_msg(MANDOCERR_DATE_MISSING, parse, ln, pos, NULL);
-		time(&t);
+		return(time2a(time(NULL)));
 	}
-	else if (a2time(&t, "%Y-%m-%d", in))
-		t = 0;
-	else if (!a2time(&t, "$" "Mdocdate: %b %d %Y $", in) &&
-	    !a2time(&t, "%b %d, %Y", in)) {
+	else if (a2time(&t, "%Y-%m-%d %H:%M:%S", in) ||
+	    a2time(&t, "%Y-%m-%d", in) || a2time(&t, "%b %d, %Y", in) ||
+	    a2time(&t, "%b %d %Y", in)) {
+		return(time2a(t));
+	}
+	else {
 		mandoc_msg(MANDOCERR_DATE_BAD, parse, ln, pos, in);
-		t = 0;
+		return(mandoc_strdup(in));
 	}
-	out = t ? time2a(t) : NULL;
-	return(out ? out : mandoc_strdup(in));
 }
 
 int
--- mdoc.7.orig
+++ mdoc.7
@@ -1225,6 +1225,12 @@
 the special string
 .Dq $\&Mdocdate$
 can be given as an argument.
+The same effect can be achieved with
+.Xr svn 1
+by setting the
+.Va svn:keywords
+property to
+.Li Mdocdate=%d .
 .It
 The traditional, purely numeric
 .Xr man 7
@@ -1240,7 +1246,10 @@
 Examples:
 .Dl \&.Dd $\&Mdocdate$
 .Dl \&.Dd $\&Mdocdate: July 21 2007$
+.Dl \&.Dd $\&Mdocdate: 2007-07-21 12:34:56Z foo $
 .Dl \&.Dd July 21, 2007
+.Dl \&.Dd July 21  2007
+.Dl \&.Dd 2007-07-21
 .Pp
 See also
 .Sx \&Dt

Re: Mdocdate

[Ingo Schwarze]

Hi Dag-Erling,

thanks for caring about mandoc!


Dag-Erling Smoergrav wrote on Thu, Nov 12, 2015 at 10:25:42AM +0100:

> The attached patch improves the Mdocdate code and updates the
> documentation to match.
> 
>  - in a2time():
> 
>    Skip past "$Mdocdate: " if it exists instead of relying on the
>    caller to specify it in the format string.

No, that would introduce a bug.  The format is different depending
on whether or not Mdocdate is used.  Maybe deplorable, but about a
decade too late to change.  Well, it could be changed with a lot of
churn in the tree, but for what benefit?  People don't need to type
in the Mdocdate format manually, after all, and even if they do that
and get it wrong, cvs(1) will helpfully fix it for them.

>    Accept trailing text after a successful match (e.g. "$").

Why?  That seems like a step backwards, weakening syntax validation.

>    If the first character after a successful match is 'Z', use UTC
>    instead of the local time zone.

That makes no sense.  Neither .TH nor .Dd syntax allows time formats
more fine-grained than days, so it makes no sense to worry about
time zones.

>  - in time2a():
> 
>    Use a sufficiently large stack buffer and mandoc_strdup() instead of
>    trying to guess at the correct length.

This does look useful, it simplifies the code.

>  - in mandoc_normdate():
> 
>    Fix several logic errors (inverted tests),

Err, what?  Can you be more specific?  I don't see any logic errors
in there.  But your changes break several features.

> add a test for the Subversion %d format,

That is not valid mdoc(7) .Dd syntax, so it can't be added.

> and simplify the code.

Yes, some of that seems useful.

>  - in the man page:
> 
>    Document how to use Mdocdate with Subversion.

No.  Please don't introduce yet another format, use something like

  Mdocdate="%b %d %Y "

or whatever the subversion syntax is for svn:keywords.

> Given the following test cases:
> 
> % cat tests
> February 7 2036
> February  7 2036
> February 07 2036

These are all invalid and must generate a warning.

> February 7, 2036
> February  7, 2036
> February 07, 2036

These are valid, but not with Mdocdate.

> Feb 7 2036
> Feb  7 2036
> Feb 07 2036

Invalid.

> Feb 7, 2036
> Feb  7, 2036
> Feb 07, 2036
> 2036-02-07
> 2036-02-7
> 2036-2-07
> 2036-2-7

Valid, but not with Mdocdate.

> $Mdocdate: February 7 2036 $

Valid.

> $Mdocdate: 2036-02-07 06:28:16Z $

Invalid.

> The current code (1.13.3) produces the following output:
> 
> % while read d ; do echo ".Dd $d" | mandoc | tail -1 ; done <tests
>                                 February 7 2036
>                                 February 7 2036
>                                February 07 2036
>                                February 7, 2036
>                                February 7, 2036
>                                February 7, 2036
>                                   Feb 7 2036
>                                   Feb 7 2036
>                                   Feb 07 2036
>                                February 7, 2036
>                                February 7, 2036
>                                February 7, 2036
>                                   2036-02-07
>                                    2036-02-7
>                                    2036-2-07
>                                    2036-2-7
>                                February 7, 2036
>                        $Mdocdate: 2036-02-07 06:28:16Z $

Yes, that is correct behaviour.

> My version produces the same output for all cases:
> 
> % while read d ; do echo ".Dd $d" | ./mandoc | tail -1 ; done <tests | uniq
>                                February 7, 2036

That is wrong.

I admit that validity rules for .Dd and .TH dates are a mess, for
historical reasons.  They allow much more than they should.
So i would gladly make them stricter, but that needs care to
not break too many existing documents.  I should probably look
into that anyway, time seems about ripe.

But adding to the mess by allowing yet more different formats
looks like the opposite of the direction we should go.


> --- mandoc.c.orig
> +++ mandoc.c
> @@ -468,13 +468,15 @@
>  	char		*pp;
>  
>  	memset(&tm, 0, sizeof(struct tm));
> +	if (strncmp(p, "$" "Mdocdate: ", 11) == 0)
> +		p += 11;

Wrong, see above, syntax differs.

>  	pp = NULL;
>  #if HAVE_STRPTIME
>  	pp = strptime(p, fmt, &tm);
>  #endif
> -	if (NULL != pp && '\0' == *pp) {
> -		*t = mktime(&tm);
> +	if (NULL != pp) {
> +		*t = ('Z' == *p) ? timegm(&tm) : mktime(&tm);
>  		return(1);
>  	}

Wrong, "Z" is not allowed here.

>  char *
>  mandoc_normdate(struct mparse *parse, char *in, int ln, int pos)
>  {
> -	char		*out;
>  	time_t		 t;
>  
> -	if (NULL == in || '\0' == *in ||
> -	    0 == strcmp(in, "$" "Mdocdate$")) {
> +	if (NULL == in || '\0' == *in || 0 == strcmp(in, "$" "Mdocdate$")) {

I see no change here?

>  		mandoc_msg(MANDOCERR_DATE_MISSING, parse, ln, pos, NULL);
> -		time(&t);
> +		return(time2a(time(NULL)));
>  	}

Yes, that seems simpler.

> -	else if (a2time(&t, "%Y-%m-%d", in))
> -		t = 0;
> -	else if (!a2time(&t, "$" "Mdocdate: %b %d %Y $", in) &&
> -	    !a2time(&t, "%b %d, %Y", in)) {
> +	else if (a2time(&t, "%Y-%m-%d %H:%M:%S", in) ||

Wrong, see above.

> +	    a2time(&t, "%Y-%m-%d", in) || a2time(&t, "%b %d, %Y", in) ||
> +	    a2time(&t, "%b %d %Y", in)) {
> +		return(time2a(t));

Wrong in several ways.  Including two invalid formats and
converting the legacy man(7) format to the mdoc(7) format.

> +	}
> +	else {
>  		mandoc_msg(MANDOCERR_DATE_BAD, parse, ln, pos, in);
> -		t = 0;
> +		return(mandoc_strdup(in));

Yes, that seems simpler.

>  	}
> -	out = t ? time2a(t) : NULL;
> -	return(out ? out : mandoc_strdup(in));
>  }
>  
>  int
> --- mdoc.7.orig
> +++ mdoc.7
> @@ -1225,6 +1225,12 @@
>  the special string
>  .Dq $\&Mdocdate$
>  can be given as an argument.
> +The same effect can be achieved with
> +.Xr svn 1
> +by setting the
> +.Va svn:keywords
> +property to
> +.Li Mdocdate=%d .
>  .It
>  The traditional, purely numeric
>  .Xr man 7
> @@ -1240,7 +1246,10 @@
>  Examples:
>  .Dl \&.Dd $\&Mdocdate$
>  .Dl \&.Dd $\&Mdocdate: July 21 2007$
> +.Dl \&.Dd $\&Mdocdate: 2007-07-21 12:34:56Z foo $
>  .Dl \&.Dd July 21, 2007
> +.Dl \&.Dd July 21  2007

No, the latter is plain wrong.

> +.Dl \&.Dd 2007-07-21

That kind of works with mandoc, which is probably a bug.
It doesn't work with groff, so it certainly mustn't be advertised
in the manual.

I'll test the useful simplifications and probably commit them.

Thanks!
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Ingo Schwarze]

Hi Dag-Erling,

Ingo Schwarze wrote on Thu, Nov 12, 2015 at 10:31:35PM +0100:
> Dag-Erling Smoergrav wrote on Thu, Nov 12, 2015 at 10:25:42AM +0100:

>>  - in time2a():
>> 
>>    Use a sufficiently large stack buffer and mandoc_strdup() instead of
>>    trying to guess at the correct length.

> This does look useful, it simplifies the code.

Unfortunately, testing revealed that it breaks the output.

I have added a comment explaining why the seemingly weird
formatting code is required, such that people don't write
the same patch again.

However, i did simplify the code in mandoc_normdate()
the way you suggested and added some comments there, too.
It is much more readable now, thanks for the idea.

Yours,
  Ingo


Log Message:
-----------
Simplify the logic in mandoc_normdate() and add some comments.
Also add a comment in time2a() explaining why it isn't possible
to use just one single call to strftime().
Do some style cleanup while here.
No functional change.
Triggered by a very different patch from des@FreeBSD.

Modified Files:
--------------
    mdocml:
        mandoc.c

Revision Data
-------------
Index: mandoc.c
===================================================================
RCS file: /home/cvs/mdocml/mdocml/mandoc.c,v
retrieving revision 1.97
retrieving revision 1.98
diff -Lmandoc.c -Lmandoc.c -u -p -r1.97 -r1.98
--- mandoc.c
+++ mandoc.c
@@ -478,17 +478,27 @@ time2a(time_t t)
 	 * up to 2 characters for the day + comma + blank
 	 * 4 characters for the year and a terminating '\0'
 	 */
+
 	p = buf = mandoc_malloc(10 + 4 + 4 + 1);
 
-	if (0 == (ssz = strftime(p, 10 + 1, "%B ", tm)))
+	if ((ssz = strftime(p, 10 + 1, "%B ", tm)) == 0)
 		goto fail;
 	p += (int)ssz;
 
-	if (-1 == (isz = snprintf(p, 4 + 1, "%d, ", tm->tm_mday)))
+	/*
+	 * The output format is just "%d" here, not "%2d" or "%02d".
+	 * That's also the reason why we can't just format the
+	 * date as a whole with "%B %e, %Y" or "%B %d, %Y".
+	 * Besides, the present approach is less prone to buffer
+	 * overflows, in case anybody should ever introduce the bug
+	 * of looking at LC_TIME.
+	 */
+
+	if ((isz = snprintf(p, 4 + 1, "%d, ", tm->tm_mday)) == -1)
 		goto fail;
 	p += isz;
 
-	if (0 == strftime(p, 4 + 1, "%Y", tm))
+	if (strftime(p, 4 + 1, "%Y", tm) == 0)
 		goto fail;
 	return buf;
 
@@ -500,23 +510,29 @@ fail:
 char *
 mandoc_normdate(struct mparse *parse, char *in, int ln, int pos)
 {
-	char		*out;
 	time_t		 t;
 
-	if (NULL == in || '\0' == *in ||
-	    0 == strcmp(in, "$" "Mdocdate$")) {
+	/* No date specified: use today's date. */
+
+	if (in == NULL || *in == '\0' || strcmp(in, "$" "Mdocdate$") == 0) {
 		mandoc_msg(MANDOCERR_DATE_MISSING, parse, ln, pos, NULL);
-		time(&t);
+		return time2a(time(NULL));
 	}
-	else if (a2time(&t, "%Y-%m-%d", in))
-		t = 0;
-	else if (!a2time(&t, "$" "Mdocdate: %b %d %Y $", in) &&
-	    !a2time(&t, "%b %d, %Y", in)) {
+
+	/* Valid mdoc(7) date format. */
+
+	if (a2time(&t, "$" "Mdocdate: %b %d %Y $", in) ||
+	    a2time(&t, "%b %d, %Y", in))
+		return time2a(t);
+
+	/* Do not warn about the legacy man(7) format. */
+
+	if ( ! a2time(&t, "%Y-%m-%d", in))
 		mandoc_msg(MANDOCERR_DATE_BAD, parse, ln, pos, in);
-		t = 0;
-	}
-	out = t ? time2a(t) : NULL;
-	return out ? out : mandoc_strdup(in);
+
+	/* Use any non-mdoc(7) date verbatim. */
+
+	return mandoc_strdup(in);
 }
 
 int
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Dag-Erling Smørgrav]

Ingo Schwarze <schwarze@usta.de> writes:
> Dag-Erling Smørgrav <des@des.no> writes:
> >  - in a2time():
> > 
> >    Skip past "$Mdocdate: " if it exists instead of relying on the
> >    caller to specify it in the format string.
> No, that would introduce a bug.  The format is different depending
> on whether or not Mdocdate is used.  Maybe deplorable, but about a
> decade too late to change.  Well, it could be changed with a lot of
> churn in the tree, but for what benefit?  People don't need to type
> in the Mdocdate format manually, after all, and even if they do that
> and get it wrong, cvs(1) will helpfully fix it for them.

I would argue that's a feature.  My version will accept several
different formats for $Mdocdate$, including both the existing OpenBSD
format and the format which is possible to achieve with Subversion.

> >    Accept trailing text after a successful match (e.g. "$").
> Why?  That seems like a step backwards, weakening syntax validation.

Postel's law.  In this situation, you're not going for strict
validation, but for maximum flexibility.  You want to give the author
the benefit of the doubt, as long as you're reasonably sure that what
you saw is a date.

> >    If the first character after a successful match is 'Z', use UTC
> >    instead of the local time zone.
> That makes no sense.  Neither .TH nor .Dd syntax allows time formats
> more fine-grained than days, so it makes no sense to worry about
> time zones.

It can make a difference of ± 1 day if a) the commit date was close to
midnight or b) you are relatively close to the date line.  Subversion
timestamps are always in UTC and include the time followed by a Z.

> >  - in mandoc_normdate():
> > 
> >    Fix several logic errors (inverted tests),
> Err, what?  Can you be more specific?  I don't see any logic errors
> in there.  But your changes break several features.

Please explain how

 if (valid_date(str))
    /* we didn't find a valid date */ ;

is a feature.

> > add a test for the Subversion %d format,
> That is not valid mdoc(7) .Dd syntax, so it can't be added.

Says who?  Once again: Postel's law.

> >  - in the man page:
> >    Document how to use Mdocdate with Subversion.
> No.  Please don't introduce yet another format, use something like
>
>   Mdocdate="%b %d %Y "
>
> or whatever the subversion syntax is for svn:keywords.

That's not possible.  Mdocdate=%d is the closest you will get, otherwise
I wouldn't have needed this patch.

> > Given the following test cases:
> > [...]
> These are all invalid and must generate a warning.
> [...]
> These are valid, but not with Mdocdate.
> [...]
> Invalid.
> [...]
> Valid, but not with Mdocdate.
> [...]
> Invalid.

Once again: who says these dates shouldn't be accepted?  Is there some
sort of EU directive I'm not aware of that dictates the precise syntax
of a .Dd statement in an mdoc file?

>> The current code (1.13.3) produces the following output:
>> 
>> % while read d ; do echo ".Dd $d" | mandoc | tail -1 ; done <tests
>>                                 February 7 2036
>>                                 February 7 2036
>>                                February 07 2036
>>                                February 7, 2036
>>                                February 7, 2036
>>                                February 7, 2036
>>                                   Feb 7 2036
>>                                   Feb 7 2036
>>                                   Feb 07 2036
>>                                February 7, 2036
>>                                February 7, 2036
>>                                February 7, 2036
>>                                   2036-02-07
>>                                    2036-02-7
>>                                    2036-2-07
>>                                    2036-2-7
>>                                February 7, 2036
>>                        $Mdocdate: 2036-02-07 06:28:16Z $
>
> Yes, that is correct behaviour.

Not according to what you wrote earlier (the %Y-%m-%d forms should have
been accepted but were not due to the logic bugs I mentioned which you
claim are not bugs)

I must say I'm really surprised by your attitude.  You seem to value
rigid adherence to forty-year-old specifications over flexibility, ease
of use and compatibility with third-party systems, even when the latter
can be achieved without sacrificing the former.  That's laudable when
you're implementing TLS or a message parser in a SOLAS system, but not
when you're writing a text preparation system.  You'll just end up
driving even more people away from mdoc and towards DocBook or Doxygen
or whatever the flavor of the month is, for absolutely no benefit.

DES
-- 
Dag-Erling Smørgrav - des@des.no
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Ingo Schwarze]

Hi Dag-Erling,

it looks like you are missing one of the chief design goals of
mandoc(1):  Compatibility.  That means, among other aspects,
byte-by-byte identical output with groff where that is possible
with reasonable effort and not manifestly absurd (in maybe a dozen
cases, mandoc(1) deliberatly deviates from groff output, but all
these are cases of blatantly invalid input syntax, where groff
produces completely garbled output and mandoc can easily produce
something less broken, of course providing prominent error messages
when run with -Werror, -Wall, or -Tlint).  For input that is not
blatantly invalid, we do not want to become incompatible with groff.

We also do not want to encourage sloppy input that may work with
mandoc(1) - if you have a version that is new enough - but won't
work with other formatters.

It is quite frustrating for document authors when they find out
much later that their pages, even though they looked good with
mandoc(1) and mandoc -Tlint didn't report any warnings, are just
plain broken elsewhere.

I think what you really want to do is generate .Dd lines with SVN.
In that case, it doesn't help you to change the rules of the game
in just mandoc only.  You need to fix *all* formatters, or you
generate non-portable output.  Actually, it might be possible with
a bit of effort to get similar patches into groff and Heirloom,
Werner and Carsten are usually very helpful - but i'm not convinced
the format you propose makes much sense.  We really don't need
seconds and timezones in .Dd, and i wouldn't know how to explain
the need to Werner and Carsten.  Rather than trying to change the
world, i think it makes more sense to teach SVN to generate .Dd
lines that everyone will understand.


Dag-Erling Smoergrav wrote on Fri, Nov 13, 2015 at 06:52:57AM +0100:
> Ingo Schwarze <schwarze@usta.de> writes:
>> Dag-Erling Smoergrav <des@des.no> writes:

>>> Accept trailing text after a successful match (e.g. "$").

>> Why?  That seems like a step backwards, weakening syntax validation.

> Postel's law.

Exactly.  Authoring manuals is *sending* information to other people.
So the tools for writing documentation should help authors to be
conservative with that.

Note that mandoc(1) already is liberal with respect to the *accepting*
part.  If the date found in a document is invalid, no matter in which
way, it is shown to the end-user verbatim.  Mandoc already is more
liberal than groff in that respect.

What you are proposing is weakening the -Wall / -Tlint warnings
that authors see.  That will make end-users suffer on systems
mismatching the author's system.


>>> If the first character after a successful match is 'Z', use UTC
>>> instead of the local time zone.

>> That makes no sense.  Neither .TH nor .Dd syntax allows time formats
>> more fine-grained than days, so it makes no sense to worry about
>> time zones.

> It can make a difference of 1 day if a) the commit date was close to
> midnight or b) you are relatively close to the date line.  Subversion
> timestamps are always in UTC and include the time followed by a Z.

Yes, but so what?  Including that information will make groff display
*the current date* (the date the manual is formatted or displayed)
because the language definition doesn't allow it.  Giving a sizeable
fraction of users something completely wrong all the time because
you want to prevent occasional subtle off-by-one errors in the
last-change date is not a reasonable tradeoff.

>>>  - in mandoc_normdate():
>>>    Fix several logic errors (inverted tests),

>> Err, what?  Can you be more specific?  I don't see any logic errors
>> in there.  But your changes break several features.

> Please explain how
> 
>  if (valid_date(str))
>     /* we didn't find a valid date */ ;
> 
> is a feature.

You misread the code, maybe the new version (which i revised
using your suggestions, but without functional change) is clearer:

	/* Do not warn about the legacy man(7) format. */
	
	if ( ! a2time(&t, "%Y-%m-%d", in))
		mandoc_msg(MANDOCERR_DATE_BAD, parse, ln, pos, in);
	
	/* Use any non-mdoc(7) date verbatim. */
	
	return mandoc_strdup(in);

So the point is, a legacy man(7) format date is returned verbatim
and *NOT* converted to mdoc(7) format.  Groff behaves the same way
for man(7) documents, and that's where these dates occur in practice.
Note that in mdoc(7) documents, groff does *NOT* accept the man(7)
format at all, causing the current date to be displayed.  That's why
i should probably warn about man(7) style dates in mdoc(7) documents.
I didn't come round to doing that yet, it's a bit tricky to implement.

>>> add a test for the Subversion %d format,

>> That is not valid mdoc(7) .Dd syntax, so it can't be added.

> Says who?  Once again: Postel's law.

Admittedly, there is no mdoc(7) RFC.  But groff-mdoc(7) has been
the de-facto standard for two decades and still behaves that way.
mandoc(1) is becoming the de-facto standard as we speak and takes
great care to remain compatible with groff.  Cynthia Livingston's
original mdoc(7) v3 also behaves that way, see

  https://svnweb.freebsd.org/csrg/share/tmac/doc-common

and so does Heirloom roff.

>>> - in the man page:
>>>   Document how to use Mdocdate with Subversion.

>> No.  Please don't introduce yet another format, use something like
>>
>>   Mdocdate="%b %d %Y "
>>
>> or whatever the subversion syntax is for svn:keywords.

> That's not possible.  Mdocdate=%d is the closest you will get,
> otherwise I wouldn't have needed this patch.

Oh.  That's bad.  I assumed SVN would take a format string...

It doesn't buy you much to let SVN put in stuff that almost
no mdoc(7) parser understands.  If you want to autogenerate
mdoc(7) code from SVN, you will probably have to teach SVN
how to do it properly.  We did the same for CVS long ago.
I hope it won't be that hard...

>>> The current code (1.13.3) produces the following output:
>>> 
>>> % while read d ; do echo ".Dd $d" | mandoc | tail -1 ; done <tests
>>>                                 February 7 2036
>>>                                 February 7 2036
>>>                                February 07 2036
>>>                                February 7, 2036
>>>                                February 7, 2036
>>>                                February 7, 2036
>>>                                   Feb 7 2036
>>>                                   Feb 7 2036
>>>                                   Feb 07 2036
>>>                                February 7, 2036
>>>                                February 7, 2036
>>>                                February 7, 2036
>>>                                   2036-02-07
>>>                                    2036-02-7
>>>                                    2036-2-07
>>>                                    2036-2-7
>>>                                February 7, 2036
>>>                        $Mdocdate: 2036-02-07 06:28:16Z $

>> Yes, that is correct behaviour.

> Not according to what you wrote earlier (the %Y-%m-%d forms should have
> been accepted but were not due to the logic bugs I mentioned which you
> claim are not bugs)

You are confusing *validity* and *output*.

If something is valid, that doesn't imply that it will be converted
to some other format.  It means that -Wall / -Tlint will produce
no warning.

The input is used verbatim both for valid legacy man(7) format
and for invalid input.

> I must say I'm really surprised by your attitude.  You seem to value
> rigid adherence to forty-year-old specifications over flexibility, ease
> of use and compatibility with third-party systems, even when the latter
> can be achieved without sacrificing the former.  That's laudable when
> you're implementing TLS or a message parser in a SOLAS system, but not
> when you're writing a text preparation system.  You'll just end up
> driving even more people away from mdoc and towards DocBook or Doxygen
> or whatever the flavor of the month is, for absolutely no benefit.

No, that's not what i'm trying to do.  I'm trying to make sure that
authors can find out whether what they write will be portable.
And i'm also trying to make sure that even when authors screwed up,
end-users see something reasonable.

You propose to make it impossible for authors to figure out whether
what they wrote might work elsewhere, and to show output to end-users
that is gratuitiously different from what other formatters would
show them.  We have to solve your task in some better way than with
your original patch.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Steffen Nurpmeso]

Hello,

Ingo Schwarze <schwarze@usta.de> wrote:
 |Hi Dag-Erling,

 |generate non-portable output.  Actually, it might be possible with
 |a bit of effort to get similar patches into groff and Heirloom,
 |Werner and Carsten are usually very helpful - but i'm not convinced
 |the format you propose makes much sense.  We really don't need
 |seconds and timezones in .Dd, and i wouldn't know how to explain
 |the need to Werner and Carsten.  Rather than trying to change the
 |world, i think it makes more sense to teach SVN to generate .Dd

but i think ISO 8601 / RFC 3339 "full-date" would be a useful
extension since they are (a) standardized and well-known and (b)
locale / charset agnostic and worldwide relatively (it is a
Gregorian date in the end) easy perceivable.
I lack the overview, but localized manpages have to ship with
those english / american dates, too?  So no, now that groff's mdoc
has been extended to support mdocml's cvs(1) tag natively i think
i'll add ISO 8601 YYYY-MM-DD to my mdoc copy once i come to
development of my roff fork, and add a Savannah bug report with
the patch, then.

--steffen
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Ingo Schwarze]

Hi Steffen,

Steffen Nurpmeso wrote on Sat, Nov 14, 2015 at 04:02:01PM +0100:
> Ingo Schwarze <schwarze@usta.de> wrote:

>> generate non-portable output.  Actually, it might be possible with
>> a bit of effort to get similar patches into groff and Heirloom,
>> Werner and Carsten are usually very helpful - but i'm not convinced
>> the format you propose makes much sense.  We really don't need
>> seconds and timezones in .Dd, and i wouldn't know how to explain
>> the need to Werner and Carsten.  Rather than trying to change the
>> world, i think it makes more sense to teach SVN to generate .Dd
 
> but i think ISO 8601 / RFC 3339 "full-date" would be a useful
> extension since they are (a) standardized and well-known

As a whole, ISO 8601 provides *much* too many options.
Allowing many different formats only confuses authors,
endangers portability, and provides a fertile ground for bugs.

A manual page language should encourage *one* simple, uniform
format.  That is fully sufficient for the purpose, easiest to
implement, easiest to document, and easiest to remember for
authors.

Actually, we are almost there.  For the legacy man(7) language, the
traditional format used is YYYY-MM-DD.  For the mdoc(7) language,
the only format fully supported has always been "Month [D]D, YYYY".
It's not ideal that both are different, but man(7) is slowly dying
anyway, so things settle on "Month [D]D, YYYY".

> and (b) locale / charset agnostic and worldwide relatively (it
> is a Gregorian date in the end) easy perceivable.
>
> I lack the overview, but localized manpages have to ship with
> those english / american dates, too?

Yes.

Of course, if you translate a manual page, you can choose to provide
a .Dd argument in chinese letters using a chinese calendar.  But
you should understand the consequences:  All formatters except
mandoc(1) will ignore your chinese date, show the date of *formatting*
of the manual instead, using english conventions.  Mandoc will warn
that the date is invalid and non-portable and will show it verbatim.

Besides, portability of translated manuals is poor anyway.
For example, you cannot translate the "SYNOPSIS" section header
without breaking important functionality.  A very ugly workaround
exists (.nr nS) that works with groff and mandoc, but isn't
portable to any other formatters.  There are lots and lots of
similar gotchas all over the place that are more important
than the date format.

You should also be aware that there is almost no interest in
translated documentation.  I have talked to Japanese people multiple
times, and they invariably said:  "Yes, now and then somebody
translates a few pages, but then they get outdated soon and are
abondoned.  Nobody really uses translated documentation.  People
really value that using the authoritative english versions, you can
be sure that what you read is true."  What russian people told me
hardly sounded different.  And those - Japan and Russia - are two
countries where speaking english is somewhat less common than in
western and central Europe.  I have yet to meet french or german
people who feel that french or german manuals would be useful.

Don't get me wrong:  If somebody wants to translate a manual, i
don't want the tools to hinder them.  That's why we support UTF-8
output for a long time and have improved UTF-8 input recently.
That's why i regularly ask people at conferences whether they are
aware of any issues with non-english documentation.  Such conversation
rarely reveal any issues, and in the few cases where they do, they
can often be fixed without collecting technical debt in the code.

But it is very important to not weaken the language definition for
the sake of straw-man internationalization arguments.  It is not
even possible to properly *design* comprehensive internationalization
interfaces simply because demand is so elusive.  Unless some new
code is put to intensive use right away, the design will inevitably
become a bad design-by-committee one.  That will only hurt everyone.

> So no, now that groff's mdoc
> has been extended to support mdocml's cvs(1) tag natively i think
> i'll add ISO 8601 YYYY-MM-DD to my mdoc copy once i come to
> development of my roff fork, and add a Savannah bug report with
> the patch, then.

Sounds like a bad idea.  Mdocdate support was added to groff
a decade after thousands of manuals using that convention existed
in the wild.  It was the first system ever to support autogeneration
of .Dd content.  Those are sufficiently strong reasons for adding
a new feature.

None of these reasons apply to allowing YYYY-MM-DD in .Dd.
No such manuals exist in the wild yet using that format, and it's
still time to prevent bloating the definition of this user interface.
Adding another format is not required; an established format for
autogeneration already exists.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Steffen Nurpmeso]

Hallo Ingo!

Ingo Schwarze <schwarze@usta.de> wrote:
 |Steffen Nurpmeso wrote on Sat, Nov 14, 2015 at 04:02:01PM +0100:
 |> Ingo Schwarze <schwarze@usta.de> wrote:

 |>> Werner and Carsten are usually very helpful - but i'm not convinced
 |>> the format you propose makes much sense.  We really don't need
 |>> seconds and timezones in .Dd, and i wouldn't know how to explain
 |>> the need to Werner and Carsten.  Rather than trying to change the
 |>> world, i think it makes more sense to teach SVN to generate .Dd
 | 
 |> but i think ISO 8601 / RFC 3339 "full-date" would be a useful
 |> extension since they are (a) standardized and well-known
 |
 |As a whole, ISO 8601 provides *much* too many options.

Yes, i think so.  I don't know for sure since the PDF costs about
120 Euro and i never was and am not interested enough to spend
that much money for the standard.  (And that is Switzerland where
refugees live in atomic bunkers and may only walk on the land some
hours a day since more than a decade at least, yet everybody knows
that a lot of weapon dealing happens to happen there.  So that is
not schizophrenic but quite the opposite.)

 |Allowing many different formats only confuses authors,
 |endangers portability, and provides a fertile ground for bugs.
 |
 |A manual page language should encourage *one* simple, uniform
 |format.  That is fully sufficient for the purpose, easiest to
 |implement, easiest to document, and easiest to remember for
 |authors.
 |
 |Actually, we are almost there.  For the legacy man(7) language, the
 |traditional format used is YYYY-MM-DD.  For the mdoc(7) language,
 |the only format fully supported has always been "Month [D]D, YYYY".
 |It's not ideal that both are different, but man(7) is slowly dying
 |anyway, so things settle on "Month [D]D, YYYY".

I agree, and if someone will write a good markdown to mdoc
converter then world dominition is only a small step away. ^_^
For what is shown in the middle of the bottom line of manuals
quite a furious mixture can be seen, including nothing at all.
So at least on the inhomogenous GNU/Linux environment i'm writing
this on the man(1)ual pages of the GNU tools wc(1), grep(1) and
find(1) differ in that the first shows "July 2015", the second
"GNU grep 2.22" and the last nothing (only "FIND(1)" in the lower
right).  (Starting in 2001 i wished regulary reoccurring to have
a BSD userland on a Linux kernel.  _That_ would be nice!)
But i for one disagree with the date format, since

 |> and (b) locale / charset agnostic and worldwide relatively (it
 |> is a Gregorian date in the end) easy perceivable.

it is at least neutral.
Granted there already is a world domination in the world of
programming and normal end-users will rarely see manual pages, and
even if it is a single entry of the last line of a manual page.

But YYYY-MM-DD could be easily converted to the calendar of
a locale user, which is not the case for the current format.
There is not even strptime_l(), so you either had to unroll
a parser or (less likely) setlocale() to C/POSIX in order to
strptime() the english names.

 |> I lack the overview, but localized manpages have to ship with
 |> those english / american dates, too?
 |
 |Yes.
 |
 |Of course, if you translate a manual page, you can choose to provide
 |a .Dd argument in chinese letters using a chinese calendar.  But
 |you should understand the consequences:  All formatters except
 |mandoc(1) will ignore your chinese date, show the date of *formatting*
 |of the manual instead, using english conventions.  Mandoc will warn
 |that the date is invalid and non-portable and will show it verbatim.
 |
 |Besides, portability of translated manuals is poor anyway.
 |For example, you cannot translate the "SYNOPSIS" section header
 |without breaking important functionality.  A very ugly workaround
 |exists (.nr nS) that works with groff and mandoc, but isn't
 |portable to any other formatters.  There are lots and lots of
 |similar gotchas all over the place that are more important
 |than the date format.

But this is today.  What will be in 20, 30 or even fourty years?
I think decisions have to be made with some context, and then the
above is plain unacceptable!  Helmut Schmidt for example pointed
out that whereas in the 1950s more than 25% of the worlds goods
(so to say) where "created" by the Europeans (or their slaves, of
course), it can be expected that in the middle of the century no
more than about 7% will remain.  Going the fortress way and
massively pushing own interest on the outside (just remember that
before Fischer/Schröder corruption in foreign countries was
tax-deductible in Germany!) does only seem to be the right way ...
unless the three fingers that point back at you are visible.  So
to say (ah, it's a monday).

 |You should also be aware that there is almost no interest in
 |translated documentation.  I have talked to Japanese people multiple
 |times, and they invariably said:  "Yes, now and then somebody
 |translates a few pages, but then they get outdated soon and are
 |abondoned.  Nobody really uses translated documentation.  People
 |really value that using the authoritative english versions, you can
 |be sure that what you read is true."  What russian people told me
 |hardly sounded different.  And those - Japan and Russia - are two
 |countries where speaking english is somewhat less common than in
 |western and central Europe.  I have yet to meet french or german
 |people who feel that french or german manuals would be useful.

You won't find one in me either.  I'm always stunned that suddenly
everything becomes so german on Linux, and was like that on Apple.
Talking about messages.  (My shared .profile only sets LC_ALL.
But there LC_MESSAGES really is necessary.  I mean it's fine and
great, *if* i could selectively install what i want.  E.g., about
15 years ago with earliest KDE it was terrible to see that all
those menu entry definition files contained all the available
translations.  And AlpineLinux doesn't install any manuals, but
that is also weird; wondering why their image is so much larger
than that of Void.)

 |Don't get me wrong:  If somebody wants to translate a manual, i
 |don't want the tools to hinder them.  That's why we support UTF-8
 |output for a long time and have improved UTF-8 input recently.
 |That's why i regularly ask people at conferences whether they are
 |aware of any issues with non-english documentation.  Such conversation
 |rarely reveal any issues, and in the few cases where they do, they
 |can often be fixed without collecting technical debt in the code.
 |
 |But it is very important to not weaken the language definition for
 |the sake of straw-man internationalization arguments.  It is not

But i don't agree nonetheless.  I think localized documentation is
very important and will be happily consumed if it's available.
No, and i don't think that people you meet at some conference will
be of any value in determining wether it is important or not.
Professionals tend to say things like "of course *everybody* knows
that and that" and whereas it's true it is so only for the people
they usually have contact with.  It's the outcome of an ivory
tower's point of view, and i guess that is true most of the time.

But even if.  That is today.  Outdated localized translations are
a management problem only -- simply don't bundle them!  (The
translation status of not few programs in the GNU/Linux world are
really good, i cannot comment on the translation quality, of
course.)  The future will bring many, many new users in places of
the world which are considered third world by ourselves.  And in
my opinion it is a matter of livable human co-existence to try to
offer support if available.

In this respect i would say that using english style dates was an
error right from the start, but which anyway should have been
corrected many years ago.  And whereas it may be easier to convert
the dates to some locale format as they are today, and talking
about roff macros, i think having a standardized variant like
YYYY-MM-DD will be better over time.  I.e., i think it may make
sense to offer a new date-localization troff command that can be
used from within man/mdoc/xy before the date is actually
displayed.  And why not giving the user an option to define her or
his preferred format in her or his ~/.roffrc or so?  And then
troff has the same problem with strptime() that i pointed out
above.  (And not to talk about true calendar support that is
humiliatingly missing from ISO C and POSIX.)

 |even possible to properly *design* comprehensive internationalization
 |interfaces simply because demand is so elusive.  Unless some new
 |code is put to intensive use right away, the design will inevitably
 |become a bad design-by-committee one.  That will only hurt everyone.

 |> So no, now that groff's mdoc
 |> has been extended to support mdocml's cvs(1) tag natively i think
 |> i'll add ISO 8601 YYYY-MM-DD to my mdoc copy once i come to
 |> development of my roff fork, and add a Savannah bug report with
 |> the patch, then.
 |
 |Sounds like a bad idea.  Mdocdate support was added to groff
 |a decade after thousands of manuals using that convention existed
 |in the wild.  It was the first system ever to support autogeneration
 |of .Dd content.  Those are sufficiently strong reasons for adding
 |a new feature.
 |
 |None of these reasons apply to allowing YYYY-MM-DD in .Dd.
 |No such manuals exist in the wild yet using that format, and it's
 |still time to prevent bloating the definition of this user interface.
 |Adding another format is not required; an established format for
 |autogeneration already exists.

So i continue to disagree due to the reasons as above.  I think
i will not only retain the idea of the RFC 3339 "full-date"
format, but also extend my (future) troff to support date and time
localization, such functionality is missing iirc.
Ciao,

--steffen
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Svyatoslav Mishyn]

Hello.

(Mon, 16 Nov 14:35) Steffen Nurpmeso:
> ..if someone will write a good markdown to mdoc
> converter then world dominition is only a small step away.

mkd2man ?
https://github.com/faelys/libsoldout/blob/trunk/mkd2man.c

But yes, it needs some enhancements.


-- 
http://www.juef.tk/
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Steffen Nurpmeso]

Hi!

Svyatoslav Mishyn <juef@openmailbox.org> wrote:
 |(Mon, 16 Nov 14:35) Steffen Nurpmeso:
 |> ..if someone will write a good markdown to mdoc
 |> converter then world dominition is only a small step away.
 |
 |mkd2man ?
 |https://github.com/faelys/libsoldout/blob/trunk/mkd2man.c

Oh, yes that looks pretty cool...  It seems it has been turned
from MAN to MDOC on January 12th 2015 thanks to Baptiste Daroussin
of FreeBSD!

Hmm...

 |But yes, it needs some enhancements.

Ok.  That..seems..to..be....overly true. :)
Well i guess someone had to write a pandoc(1) extension or
something rather likewise.  (Not that i know pandoc.)
It supports man(7), which i find unbelievable given that this
format doesn't even support active references.  And now they even
use blue font for .. nothing!  mdoc(7) can better than that!
Ah i don't know why people don't like this incredibly simple, easy
pars- and understandable documentation format!  I always enjoy
glancing over the 9483 self-containing lines of my MUA's manual.

--steffen
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv

Re: Mdocdate

[Michael Dexter]

On 11/12/15 1:31 PM, Ingo Schwarze wrote:
> thanks for caring about mandoc!

+1

Michael
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv