discuss@mandoc.bsd.lv
 help / color / mirror / Atom feed
* Crash in makewhatis on FreeBSD's man9 manuals
@ 2014-08-16 17:56 Ulrich Spörlein
  2014-08-16 20:02 ` Ingo Schwarze
  0 siblings, 1 reply; 2+ messages in thread
From: Ulrich Spörlein @ 2014-08-16 17:56 UTC (permalink / raw)
  To: discuss

[-- Attachment #1: Type: text/plain, Size: 3240 bytes --]

Hey,

release 1.13.1, running makewhatis /usr/share/man I get:

% gdb ./makewhatis
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
(gdb) r /usr/share/man
Starting program: /home/uqs/ports/textproc/mdocml/work/mdocml-1.13.1/makewhatis /usr/share/man
[New LWP 101640]
[New Thread 801806400 (LWP 101640/makewhatis)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 801806400 (LWP 101640/makewhatis)]
0x0000000000410ba6 in post_it (mdoc=0x801819200) at mdoc_validate.c:1209
1209            lt = nbl->norm->Bl.type;
Current language:  auto; currently minimal
(gdb) bt
#0  0x0000000000410ba6 in post_it (mdoc=0x801819200) at mdoc_validate.c:1209
#1  0x000000000040e488 in rew_last (mdoc=0x801819200, to=0x802961240) at mdoc_macro.c:289
#2  0x000000000040e635 in rew_sub (t=<value optimized out>, mdoc=0x801819200, tok=<value optimized out>, line=<value optimized out>, ppos=<value optimized out>)
    at mdoc_macro.c:589
#3  0x000000000040c604 in blk_full (mdoc=0x801819200, tok=MDOC_Sh, line=610, ppos=1, pos=0x7fffffffd9a0, buf=0x80251e200 ".Sh RETURN VALUES") at mdoc_macro.c:1059
#4  0x000000000040a7a2 in mdoc_parseln (mdoc=0x801819200, ln=610, buf=0x80251e200 ".Sh RETURN VALUES", offs=1) at mdoc.c:886
#5  0x00000000004188fc in mparse_buf_r (curp=0x801817080, start=1) at read.c:565
#6  0x0000000000417da5 in mparse_parse_buffer (curp=0x801817080, file=<value optimized out>) at read.c:719
#7  0x00000000004180e0 in mparse_readfd (curp=0x801817080, fd=<value optimized out>, file=0x801b23100 "man9/callout_reset_on.9.gz") at read.c:765
#8  0x00000000004040ea in mpages_merge (mc=0x80181b058, mp=0x801817080) at mandocdb.c:1158
#9  0x0000000000403128 in main (argc=<value optimized out>, argv=<value optimized out>) at mandocdb.c:515
(gdb) f 0
#0  0x0000000000410ba6 in post_it (mdoc=0x801819200) at mdoc_validate.c:1209
1209            lt = nbl->norm->Bl.type;
(gdb) l
1204            nit = mdoc->last;
1205            if (MDOC_BLOCK != nit->type)
1206                    return(1);
1207    
1208            nbl = nit->parent->parent;
1209            lt = nbl->norm->Bl.type;
1210    
1211            switch (lt) {
1212            case LIST_tag:
1213                    /* FALLTHROUGH */
(gdb) p *nbl
$1 = {parent = 0x80183e840, child = 0x802961100, last = 0x0, next = 0x0, prev = 0x8024177e0, nchild = 2, line = 107, pos = 1, lastline = 378, tok = MDOC_Sh, flags = 24, 
  type = MDOC_BLOCK, sec = SEC_DESCRIPTION, norm = 0x0, prev_font = 0x0, args = 0x0, pending = 0x0, head = 0x802961100, body = 0x802961240, tail = 0x0, string = 0x0, 
  span = 0x0, eqn = 0x0, end = ENDBODY_NOT}
(gdb) 


Moving callout_reset_on.9 out of the way just makes it die at some other
section 9 manual. Good news however is that if I remove man9 altogether
it seemingly runs to completion. I'll attach the man-page in question,
hoping you'll be able to reproduce. Thanks!

Cheers,
Uli

[-- Attachment #2: callout_reset_sbt_on.9 --]
[-- Type: text/plain, Size: 18223 bytes --]

.\"	$NetBSD: timeout.9,v 1.2 1996/06/23 22:32:34 pk Exp $
.\"
.\" Copyright (c) 1996 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" This code is derived from software contributed to The NetBSD Foundation
.\" by Paul Kranenburg.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE
.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
.\" POSSIBILITY OF SUCH DAMAGE.
.\"
.\" $FreeBSD: head/share/man/man9/timeout.9 268369 2014-07-07 16:55:07Z imp $
.\"
.Dd August 23, 2013
.Dt TIMEOUT 9
.Os
.Sh NAME
.Nm timeout ,
.Nm untimeout ,
.Nm callout_handle_init ,
.Nm callout_init ,
.Nm callout_init_mtx ,
.Nm callout_init_rm ,
.Nm callout_init_rw ,
.Nm callout_stop ,
.Nm callout_drain ,
.Nm callout_reset ,
.Nm callout_reset_on ,
.Nm callout_reset_curcpu ,
.Nm callout_reset_sbt ,
.Nm callout_reset_sbt_on ,
.Nm callout_reset_sbt_curcpu ,
.Nm callout_schedule ,
.Nm callout_schedule_on ,
.Nm callout_schedule_curcpu ,
.Nm callout_pending ,
.Nm callout_active ,
.Nm callout_deactivate
.Nd execute a function after a specified length of time
.Sh SYNOPSIS
.In sys/types.h
.In sys/systm.h
.Bd -literal
typedef void timeout_t (void *);
.Ed
.Ft struct callout_handle
.Fn timeout "timeout_t *func" "void *arg" "int ticks"
.Ft void
.Fn callout_handle_init "struct callout_handle *handle"
.Bd -literal
struct callout_handle handle = CALLOUT_HANDLE_INITIALIZER(&handle);
.Ed
.Ft void
.Fn untimeout "timeout_t *func" "void *arg" "struct callout_handle handle"
.Ft void
.Fn callout_init "struct callout *c" "int mpsafe"
.Ft void
.Fn callout_init_mtx "struct callout *c" "struct mtx *mtx" "int flags"
.Fn void
.Fn callout_init_rm "struct callout *c" "struct rmlock *rm" "int flags"
.Ft void
.Fn callout_init_rw "struct callout *c" "struct rwlock *rw" "int flags"
.Ft int
.Fn callout_stop "struct callout *c"
.Ft int
.Fn callout_drain "struct callout *c"
.Ft int
.Fn callout_reset "struct callout *c" "int ticks" "timeout_t *func" "void *arg"
.Ft int
.Fn callout_reset_on "struct callout *c" "int ticks" "timeout_t *func" \
"void *arg" "int cpu"
.Ft int
.Fn callout_reset_sbt_on "struct callout *c" "sbintime_t sbt" \
"sbintime_t pr" "timeout_t *func" "void *arg" "int cpu" "int flags"
.Ft int
.Fn callout_reset_curcpu "struct callout *c" "int ticks" "timeout_t *func" \
"void *arg"
.Ft int
.Fn callout_schedule "struct callout *c" "int ticks"
.Ft int
.Fn callout_schedule_on "struct callout *c" "int ticks" "int cpu"
.Ft int
.Fn callout_schedule_curcpu "struct callout *c" "int ticks"
.Ft int
.Fn callout_pending "struct callout *c"
.Ft int
.Fn callout_active "struct callout *c"
.Fn callout_deactivate "struct callout *c"
.Sh DESCRIPTION
The function
.Fn timeout
schedules a call to the function given by the argument
.Fa func
to take place after
.Fa ticks Ns No /hz
seconds.
Non-positive values of
.Fa ticks
are silently converted to the value
.Sq 1 .
.Fa func
should be a pointer to a function that takes a
.Fa void *
argument.
Upon invocation,
.Fa func
will receive
.Fa arg
as its only argument.
The return value from
.Fn timeout
is a
.Ft struct callout_handle
which can be used in conjunction with the
.Fn untimeout
function to request that a scheduled timeout be canceled.
The
.Fn timeout
call is the old style and new code should use the
.Fn callout_*
functions.
.Pp
The function
.Fn callout_handle_init
can be used to initialize a handle to a state which will cause
any calls to
.Fn untimeout
with that handle to return with no side
effects.
.Pp
Assigning a callout handle the value of
.Fn CALLOUT_HANDLE_INITIALIZER
performs the same function as
.Fn callout_handle_init
and is provided for use on statically declared or global callout handles.
.Pp
The function
.Fn untimeout
cancels the timeout associated with
.Fa handle
using the
.Fa func
and
.Fa arg
arguments to validate the handle.
If the handle does not correspond to a timeout with
the function
.Fa func
taking the argument
.Fa arg
no action is taken.
.Fa handle
must be initialized by a previous call to
.Fn timeout ,
.Fn callout_handle_init ,
or assigned the value of
.Fn CALLOUT_HANDLE_INITIALIZER "&handle"
before being passed to
.Fn untimeout .
The behavior of calling
.Fn untimeout
with an uninitialized handle
is undefined.
The
.Fn untimeout
call is the old style and new code should use the
.Fn callout_*
functions.
.Pp
As handles are recycled by the system, it is possible (although unlikely)
that a handle from one invocation of
.Fn timeout
may match the handle of another invocation of
.Fn timeout
if both calls used the same function pointer and argument, and the first
timeout is expired or canceled before the second call.
The timeout facility offers O(1) running time for
.Fn timeout
and
.Fn untimeout .
Timeouts are executed from
.Fn softclock
with the
.Va Giant
lock held.
Thus they are protected from re-entrancy.
.Pp
The functions
.Fn callout_init ,
.Fn callout_init_mtx ,
.Fn callout_init_rm ,
.Fn callout_init_rw ,
.Fn callout_stop ,
.Fn callout_drain ,
.Fn callout_reset
and
.Fn callout_schedule
are low-level routines for clients who wish to allocate their own
callout structures.
.Pp
The function
.Fn callout_init
initializes a callout so it can be passed to
.Fn callout_stop ,
.Fn callout_drain ,
.Fn callout_reset
or
.Fn callout_schedule
without any side effects.
If the
.Fa mpsafe
argument is zero,
the callout structure is not considered to be
.Dq multi-processor safe ;
that is,
the Giant lock will be acquired before calling the callout function,
and released when the callout function returns.
.Pp
The
.Fn callout_init_mtx
function may be used as an alternative to
.Fn callout_init .
The parameter
.Fa mtx
specifies a mutex that is to be acquired by the callout subsystem
before calling the callout function, and released when the callout
function returns.
The following
.Fa flags
may be specified:
.Bl -tag -width ".Dv CALLOUT_RETURNUNLOCKED"
.It Dv CALLOUT_RETURNUNLOCKED
The callout function will release
.Fa mtx
itself, so the callout subsystem should not attempt to unlock it
after the callout function returns.
.El
.Pp
The
.Fn callout_init_rw
and the
.Fn callout_init_rm
fuctions serve the need of using rwlocks and rmlocks in conjunction
with callouts.
The functions do the same as
.Fn callout_init
with the possibility of specifying an extra
.Fa rw
or
.Fa rm
argument.
If an
.Fa rm
argument is specified, the lock should be created without passing the
.Dv RM_SLEEPABLE
flag.
The usable lock classes are currently limited to mutexes, rwlocks and
non-sleepable rmlocks, because callout handlers run in softclock swi,
so they cannot sleep nor acquire sleepable locks like sx or lockmgr.
The following
.Fa flags
may be specified:
.Bl -tag -width ".Dv CALLOUT_SHAREDLOCK"
.It Dv CALLOUT_SHAREDLOCK
The lock is only acquired in read mode when running the callout handler.
It has no effects when used in conjunction with
.Fa mtx .
.El
.Pp
The function
.Fn callout_stop
cancels a callout if it is currently pending.
If the callout is pending, then
.Fn callout_stop
will return a non-zero value.
If the callout is not set, has already been serviced or is currently
being serviced, then zero will be returned.
If the callout has an associated mutex, then that mutex must be
held when this function is called.
.Pp
The function
.Fn callout_drain
is identical to
.Fn callout_stop
except that it will wait for the callout to be completed if it is
already in progress.
This function MUST NOT be called while holding any
locks on which the callout might block, or deadlock will result.
Note that if the callout subsystem has already begun processing this
callout, then the callout function may be invoked during the execution of
.Fn callout_drain .
However, the callout subsystem does guarantee that the callout will be
fully stopped before
.Fn callout_drain
returns.
.Pp
The function
.Fn callout_reset
first performs the equivalent of
.Fn callout_stop
to disestablish the callout, and then establishes a new callout in the
same manner as
.Fn timeout .
If there was already a pending callout and it was rescheduled, then
.Fn callout_reset
will return a non-zero value.
If the callout has an associated mutex, then that mutex must be
held when this function is called.
The function
.Fn callout_schedule
(re)schedules an existing callout for a new period of time;
it is equivalent to calling
.Fn callout_reset
with the
.Fa func
and
.Fa arg
parameters extracted from the callout structure (though possibly with
lower overhead).
.Pp
The functions
.Fn callout_reset_on
and
.Fn callout_schedule_on
are equivalent to
.Fn callout_reset
and
.Fn callout_schedule
but take an extra parameter specifying the target CPU for the callout.
.Pp
The function
.Fn callout_reset_sbt_on
allows to get higher time resolution, taking relative or absolute time
and precision instead of relative ticks count.
If specified time is in past, it will be silently converted to present
to run handler as soon as possible.
.Pp
The following
.Fa flags
may be specified:
.Bl -tag -width ".Dv C_DIRECT_EXEC"
.It Dv C_ABSOLUTE
Handle the
.Fa sbt
argument as absolute time of the event since boot, or relative time otherwise.
.It Dv C_DIRECT_EXEC
Run handler directly from hardware interrupt context instead of softclock swi.
It is faster, but puts more constraints on handlers.
Handlers may use only spin mutexes for locking, and they must be fast because
they run with absolute priority.
.It Fn C_PREL
Specifies relative event time precision as binary logarithm of time interval
divided by acceptable time deviation: 1 -- 1/2, 2 -- 1/4, etc.
Smaller value allows to aggregate more events in one timer interrupt to
reduce processing overhead and power consumption.
.El
.It Dv C_HARDCLOCK
Align the timeouts, if possible, to 
.Fn hardclock  
calls.
.El
.Pp
The functions
.Fn callout_reset_curcpu
and
.Fn callout_schedule_curcpu
are wrappers for
.Fn callout_reset_on
and
.Fn callout_schedule_on
using the current CPU as the target CPU.
.Pp
The macros
.Fn callout_pending ,
.Fn callout_active
and
.Fn callout_deactivate
provide access to the current state of the callout.
Careful use of these macros can avoid many of the race conditions
that are inherent in asynchronous timer facilities; see
.Sx "Avoiding Race Conditions"
below for further details.
The
.Fn callout_pending
macro checks whether a callout is
.Em pending ;
a callout is considered
.Em pending
when a timeout has been set but the time has not yet arrived.
Note that once the timeout time arrives and the callout subsystem
starts to process this callout,
.Fn callout_pending
will return
.Dv FALSE
even though the callout function may not have finished (or even begun)
executing.
The
.Fn callout_active
macro checks whether a callout is marked as
.Em active ,
and the
.Fn callout_deactivate
macro clears the callout's
.Em active
flag.
The callout subsystem marks a callout as
.Em active
when a timeout is set and it clears the
.Em active
flag in
.Fn callout_stop
and
.Fn callout_drain ,
but it
.Em does not
clear it when a callout expires normally via the execution of the
callout function.
.Ss "Avoiding Race Conditions"
The callout subsystem invokes callout functions from its own timer
context.
Without some kind of synchronization it is possible that a callout
function will be invoked concurrently with an attempt to stop or reset
the callout by another thread.
In particular, since callout functions typically acquire a mutex as
their first action, the callout function may have already been invoked,
but be blocked waiting for that mutex at the time that another thread
tries to reset or stop the callout.
.Pp
The callout subsystem provides a number of mechanisms to address these
synchronization concerns:
.Bl -enum -offset indent
.It
If the callout has an associated mutex that was specified using the
.Fn callout_init_mtx
function (or implicitly specified as the
.Va Giant
mutex using
.Fn callout_init
with
.Fa mpsafe
set to
.Dv FALSE ) ,
then this mutex is used to avoid the race conditions.
The associated mutex must be acquired by the caller before calling
.Fn callout_stop
or
.Fn callout_reset
and it is guaranteed that the callout will be correctly stopped
or reset as expected.
Note that it is still necessary to use
.Fn callout_drain
before destroying the callout or its associated mutex.
.It
The return value from
.Fn callout_stop
and
.Fn callout_reset
indicates whether or not the callout was removed.
If it is known that the callout was set and the callout function has
not yet executed, then a return value of
.Dv FALSE
indicates that the callout function is about to be called.
For example:
.Bd -literal -offset indent
if (sc->sc_flags & SCFLG_CALLOUT_RUNNING) {
	if (callout_stop(&sc->sc_callout)) {
		sc->sc_flags &= ~SCFLG_CALLOUT_RUNNING;
		/* successfully stopped */
	} else {
		/*
		 * callout has expired and callout
		 * function is about to be executed
		 */
	}
}
.Ed
.It
The
.Fn callout_pending ,
.Fn callout_active
and
.Fn callout_deactivate
macros can be used together to work around the race conditions.
When a callout's timeout is set, the callout subsystem marks the
callout as both
.Em active
and
.Em pending .
When the timeout time arrives, the callout subsystem begins processing
the callout by first clearing the
.Em pending
flag.
It then invokes the callout function without changing the
.Em active
flag, and does not clear the
.Em active
flag even after the callout function returns.
The mechanism described here requires the callout function itself to
clear the
.Em active
flag using the
.Fn callout_deactivate
macro.
The
.Fn callout_stop
and
.Fn callout_drain
functions always clear both the
.Em active
and
.Em pending
flags before returning.
.Pp
The callout function should first check the
.Em pending
flag and return without action if
.Fn callout_pending
returns
.Dv TRUE .
This indicates that the callout was rescheduled using
.Fn callout_reset
just before the callout function was invoked.
If
.Fn callout_active
returns
.Dv FALSE
then the callout function should also return without action.
This indicates that the callout has been stopped.
Finally, the callout function should call
.Fn callout_deactivate
to clear the
.Em active
flag.
For example:
.Bd -literal -offset indent
mtx_lock(&sc->sc_mtx);
if (callout_pending(&sc->sc_callout)) {
	/* callout was reset */
	mtx_unlock(&sc->sc_mtx);
	return;
}
if (!callout_active(&sc->sc_callout)) {
	/* callout was stopped */
	mtx_unlock(&sc->sc_mtx);
	return;
}
callout_deactivate(&sc->sc_callout);
/* rest of callout function */
.Ed
.Pp
Together with appropriate synchronization, such as the mutex used above,
this approach permits the
.Fn callout_stop
and
.Fn callout_reset
functions to be used at any time without races.
For example:
.Bd -literal -offset indent
mtx_lock(&sc->sc_mtx);
callout_stop(&sc->sc_callout);
/* The callout is effectively stopped now. */
.Ed
.Pp
If the callout is still pending then these functions operate normally,
but if processing of the callout has already begun then the tests in
the callout function cause it to return without further action.
Synchronization between the callout function and other code ensures that
stopping or resetting the callout will never be attempted while the
callout function is past the
.Fn callout_deactivate
call.
.Pp
The above technique additionally ensures that the
.Em active
flag always reflects whether the callout is effectively enabled or
disabled.
If
.Fn callout_active
returns false, then the callout is effectively disabled, since even if
the callout subsystem is actually just about to invoke the callout
function, the callout function will return without action.
.El
.Pp
There is one final race condition that must be considered when a
callout is being stopped for the last time.
In this case it may not be safe to let the callout function itself
detect that the callout was stopped, since it may need to access
data objects that have already been destroyed or recycled.
To ensure that the callout is completely finished, a call to
.Fn callout_drain
should be used.
.Sh RETURN VALUES
The
.Fn timeout
function returns a
.Ft struct callout_handle
that can be passed to
.Fn untimeout .
The
.Fn callout_stop
and
.Fn callout_drain
functions return non-zero if the callout was still pending when it was
called or zero otherwise.
.Sh HISTORY
The current timeout and untimeout routines are based on the work of
.An Adam M. Costello
and
.An George Varghese ,
published in a technical report entitled
.%T "Redesigning the BSD Callout and Timer Facilities"
and modified slightly for inclusion in
.Fx
by
.An Justin T. Gibbs .
The original work on the data structures used in this implementation
was published by
.An G. Varghese
and
.An A. Lauck
in the paper
.%T "Hashed and Hierarchical Timing Wheels: Data Structures for the Efficient Implementation of a Timer Facility"
in the
.%B "Proceedings of the 11th ACM Annual Symposium on Operating Systems Principles" .
The current implementation replaces the long standing
.Bx
linked list
callout mechanism which offered O(n) insertion and removal running time
but did not generate or require handles for untimeout operations.

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2014-08-16 20:02 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-16 17:56 Crash in makewhatis on FreeBSD's man9 manuals Ulrich Spörlein
2014-08-16 20:02 ` Ingo Schwarze

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).