Re: mdevd / udevd issues, and the issue of "reverse" dependencies

Re: mdevd / udevd issues, and the issue of "reverse" dependencies

[Laurent Bercot]

>This has obvious benefits, at least for now.  udevd does not have
>a readiness notification mechanism (polling for the existence of
>/run/udev/control surely does not count)

There is no fundamental reason why it doesn't. inotify works on tmpfs;
you don't need to poll for the existence of /run/udev/control, you can
inotifywait for its appearance.


>  and s6's fd-based mechanism does not integrate well with the shell

That is true, but I shamelessly chalk that up to the shell's
limitations: the shell can't create anonymous pipes outside of a
pipeline, and it can't make a pipeline without forking both the reader
and the writer. It's a terrible tool when you need semi-serious
plumbing work.


>  (Another issue, currently unsolved,
>is that mdevd does not have a "udevadm settle" equivalent, so that in
>theory we are not sure the basic devices are fully coldplugged when
>`mdevd-coldplug' exits.)

Funny you should mention that, it's the very problem I've been hitting
last week :)

It would be quite difficult to add a "udevadm settle" to mdevd. It
would require a communication channel with the outside (a fifodir?), and
heuristics to guesstimate when the coldplug starts and when it stops,
which I'm not sure could be made reliable in all cases. To be fully
reliable, the mechanism would need synchronization with the coldplugger,
and at this point we're well on our way to duplicating the bloat of
udevd.

However, depending on the kind of device you're waiting for, it may be
possible to avoid the race for that device. I needed to wait for a
network interface to appear after the coldplug, so I wrote a tool that
does just that: bcnm-waitif, in the bcnm package. (Documentation coming
soon.) So at least network interfaces are covered now.


>In the above, you have seen that I switched from `devd' depending on
>`devices' to the converse.  Suppose the problem of handling readiness in
>the shell could be done in a "magically simple" way, I would personally
>prefer the original way.

I don't understand why you'd prefer the original way. The natural and
normal way of proceeding is 1. start the daemon that processes the
uevents, 2. trigger the initial hardware scan that produces a big
batch of uevents. You don't need a temporary devd when you can just
start the real one; if the interfaces around the existing devd
implementations make it difficult to start properly, that's what should
be fixed.

--
Laurent

Re: mdevd / udevd issues, and the issue of "reverse" dependencies

[Casper Ti. Vector]

On Mon, Feb 10, 2020 at 08:02:44PM +0000, Laurent Bercot wrote:
> There is no fundamental reason why it doesn't. inotify works on tmpfs;
> you don't need to poll for the existence of /run/udev/control, you can
> inotifywait for its appearance.

Thanks.  I just did not take inotify into consideration.

> That is true, but I shamelessly chalk that up to the shell's
> limitations: the shell can't create anonymous pipes outside of a
> pipeline, and it can't make a pipeline without forking both the reader
> and the writer. It's a terrible tool when you need semi-serious
> plumbing work.

Now I daydream again about the scsh-like mini-language I want :)

> It would be quite difficult to add a "udevadm settle" to mdevd. [...]
> However, depending on the kind of device you're waiting for, it may be
> possible to avoid the race for that device. I needed to wait for a
> network interface to appear after the coldplug, so I wrote a tool that
> does just that: bcnm-waitif, in the bcnm package. (Documentation coming
> soon.) So at least network interfaces are covered now.

Here I am more concerned about the coldplugging of disks, so that
loggers other than the catch-all can be started.  I guess Alpine folks
would feel at home with similar requirements in `nlplug-findfs'...

> I don't understand why you'd prefer the original way. The natural and
> normal way of proceeding is 1. start the daemon that processes the
> uevents, 2. trigger the initial hardware scan that produces a big
> batch of uevents. You don't need a temporary devd when you can just
> start the real one; if the interfaces around the existing devd
> implementations make it difficult to start properly, that's what should
> be fixed.

Frankly I was just attempting to find an excuse to introduce my idea of
"reverse" dependencies.  The most important (and perhaps only) advantage
of the original way is that the devd has its own logger; but as we have
noticed, even the bloated udevd usually plays nice with the catch-all.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

mdevd / udevd issues, and the issue of "reverse" dependencies

[Casper Ti. Vector]

I recently added service definitions for mdevd into slew, during which
I switched from a `devd' longrun (which has a dedicated logger, and so
requires a R/W filesystem) that depends on a `devices' oneshot (starting
a temporary devd process to coldplug basic devices) plus a `devd'
longrun to a single `devd' longrun (which starts early, and just logs to
the catch-all logger) followed by a `devices' oneshot (which just do the
coldplug).

This has obvious benefits, at least for now.  udevd does not have
a readiness notification mechanism (polling for the existence of
/run/udev/control surely does not count), and s6's fd-based mechanism
does not integrate well with the shell (at least I have not come up
with a way that avoids the hassle of setting up a temporary fifo and
dancing with foreground and "background" processes that play with the
fifo).  So the `devices' oneshot is destined to be complex if "up" is
not to be identified with "ready".  (Another issue, currently unsolved,
is that mdevd does not have a "udevadm settle" equivalent, so that in
theory we are not sure the basic devices are fully coldplugged when
`mdevd-coldplug' exits.)

In the above, you have seen that I switched from `devd' depending on
`devices' to the converse.  Suppose the problem of handling readiness in
the shell could be done in a "magically simple" way, I would personally
prefer the original way.  However, I would like the switch between mdev,
mdevd, udev etc to be painless, so `devd' is not necessarily present;
instead, I want `devices.mdevd' / `devices.udevd' to automatically drag
`devd' into the dependency tree, even if it is `devd' that depends on
`devices' instead of the converse.  This request is admittedly based on
a fictional premise, but the supposed relation between `devices' and
`devd' is similar to that between `wpa_supplicant' and `wpa_cli', and
the latter is not fictional.

I do not find this kind of "reverse" dependencies trivially doable in
preprocessors (like slew's `prep').  The strightforward way to make
service `srv1' reverse-depend on `srv2' is to create a bundle that
contains both `srv1' and `srv2', and then to make all that depend
on `srv1' actually depend on the bundle; but how should we name the
services?  To minimise rewriting of config files, we would naturally
considering renaming the original `srv1' to something else and let the
bundle take the name.  However, the scripts in the original `srv1' may
change its behaviour based on the service name (eg. `getty.tty1' and
`dhcpc.eth0'), so some careful planning are needed to prevent the
renaming procedure from crashing with this mechanism.

Any thoughts on these issues?  Thank you in advance.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C