Re: graceful restart under runit

["Dražen Kačar" <dave@fly.srk.fer.hr>]

Paul Jarc wrote:

> So, for zero-unavailability: there is a metaserver which listens on a

And then it's just a small matter of implementing the metaserver? :-)

It seems a bit complex to me. You'd have to implement a protocol for
starting a new metaserver version (which boils down to passing all
those file descriptors to the new metaserver)

Then you'd need to implement something to take care of metaserver crashes.
Probably a way for servers to pass listening sockets back to the
new metaserver.

Then servers would need a way to wait a bit if they want to restart while
the metaserver is being restarted.

Maybe a few more things as well. I suppose it's doable, but it seems like
a can of worms and races.

> While a server is handling connections, it would have to use
> select()/poll() to notice activity on either the listening socket or
> the filesystem connection;

And that isn't very nice.

> If the requestor exits, and no other requestors are around to pass the
> listening socket to, the metaserver could close it immediately, or
> could keep it open for a few seconds to see if a new requestor show
> up.  So quick, non-overlapping restarts would be transparent to the
> end clients.

How is it supposed to know that the requestor exited?

> To trigger the switchover, you wouldn't need any signals - just make
> runsv forget about the old process using Gerrit's patch.  When the new
> process starts up and connects to the filesystem socket, that will
> trigger everything else.

I meant to implement (when the time comes) something simpler. Either a
FIFO or a Unix domain socket[1] is used as a communications channel for
passing the listening socket, but without additional daemons.

The new server starts, acquires all resources necessary to run except the
listening socket and the PID file, then tries to connect to the file
system channel.

If there's no writer, it binds to the network socket, writes the PID file,
becomes the writer on the file system channel and starts doing its job.

If there is a writer, it's supposed to be an already running server
instance. Then the new server reads the PID file, signals the running
instance and blocks in read on the file system channel.

The running instance receives the signal, passes the listening socket,
performs whatever cleanup needs to be done[2] and then either exits or
waits for the current sessions to finish and then exits.

The new server reads the file descriptor, becomes the writer on the file
system channel, writes the new PID file and starts doing its job.

[1] FIFOs are nasty because O_RDONLY opens block if there are no writers
    and O_WRONLY opens block if there are no readers. O_NONBLOCK allows a
    reader to attach without the writer, but it doesn't allow a writer to
    attach when there is no reader. I'm not sure if the required
    mumbo-jumbo can be portably done with FIFOs (also, some OSs have bugs
    in this area IIRC). But if a FIFO isn't good enough, Unix domain
    socket should suffice.

[2] At least it needs to close the listening socket and the writing part
    of the file system channel to enable the new server to become a writer
    there.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave@fly.srk.fer.hr