/dev/log: datagram, stream, both?

/dev/log: datagram, stream, both?

[Laurent Bercot]

  Hello,
  I just noticed that musl's syslog() uses a datagram socket.
  This will only work if the syslogd implementation has opened
/dev/log as a datagram socket too.

  Last time I checked, which was a certain number of years ago,
/dev/log was a SOCK_STREAM. And it was great, because it allowed
me to very easily whip up a superserver-based implementation of
syslogd in a few lines of code. I tested it, it worked, with all
the syslog() implementations I could find.

  But now, it appears to have changed, and my elegant approach
isn't working anymore. Gah!
  A Web search is pretty confusing: nobody seems to agree, there
doesn't seem to be an authority on this. So I'm asking here: is
/dev/log supposed to be a SOCK_DGRAM now? Is there a normative
body somewhere? anything else than "rsyslogd decides and libcs
have to adapt"? What are the reasons to use SOCK_DGRAM over
SOCK_STREAM?

  If anyone needed another reason not to use syslog: underspecified
protocol with a client in the libc but an external server. Wheee!

-- 
  Laurent

Re: /dev/log: datagram, stream, both?

[Rich Felker]

On Mon, Aug 10, 2015 at 11:04:51PM +0200, Laurent Bercot wrote:
> 
>  Hello,
>  I just noticed that musl's syslog() uses a datagram socket.
>  This will only work if the syslogd implementation has opened
> /dev/log as a datagram socket too.

AFAIK it was always SOCK_DGRAM.

>  Last time I checked, which was a certain number of years ago,
> /dev/log was a SOCK_STREAM. And it was great, because it allowed
> me to very easily whip up a superserver-based implementation of
> syslogd in a few lines of code. I tested it, it worked, with all
> the syslog() implementations I could find.
> 
>  But now, it appears to have changed, and my elegant approach
> isn't working anymore. Gah!
>  A Web search is pretty confusing: nobody seems to agree, there
> doesn't seem to be an authority on this. So I'm asking here: is
> /dev/log supposed to be a SOCK_DGRAM now? Is there a normative
> body somewhere? anything else than "rsyslogd decides and libcs
> have to adapt"? What are the reasons to use SOCK_DGRAM over
> SOCK_STREAM?

SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually
worked the way I intended it -- that the initial connect() binds the
address even if syslogd isn't listening yet and keeps it working even
if syslogd is restarted. Unforunately it doesn't work that way, which
means there's no way for chrooted processes to re-establish their
syslog connections if syslogd goes down.

Rich

Re: /dev/log: datagram, stream, both?

[Laurent Bercot]

On 11/08/2015 06:59, Rich Felker wrote:
> AFAIK it was always SOCK_DGRAM.

  It definitely wasn't, say, 6ish years ago. I have seen
glibc's and uClibc's syslog() work with SOCK_STREAM. I didn't
check the client code, maybe it tried both, but it *was*
succeeding in logging stuff when there was a server listening
on a /dev/log SOCK_STREAM.


> SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually
> worked the way I intended it -- that the initial connect() binds the
> address even if syslogd isn't listening yet and keeps it working even
> if syslogd is restarted. Unforunately it doesn't work that way, which
> means there's no way for chrooted processes to re-establish their
> syslog connections if syslogd goes down.

  And that's why a correct supervision architecture must perform
fd-holding on the fd outputting the logs. Which is only possible
if it knows that fd before the client start, i.e. the client logs
to stderr instead of syslog.

  Are there real, current advantages of SOCK_DGRAM - not only
hypothetical ones - and most importantly, is there a specification
anywhere ?

-- 
  Laurent

Re: /dev/log: datagram, stream, both?

[Rich Felker]

On Tue, Aug 11, 2015 at 08:02:43AM +0200, Laurent Bercot wrote:
> On 11/08/2015 06:59, Rich Felker wrote:
> >AFAIK it was always SOCK_DGRAM.
> 
>  It definitely wasn't, say, 6ish years ago. I have seen
> glibc's and uClibc's syslog() work with SOCK_STREAM. I didn't
> check the client code, maybe it tried both, but it *was*
> succeeding in logging stuff when there was a server listening
> on a /dev/log SOCK_STREAM.

Busybox 1.0 syslogd:

http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561

That's from 2004. We could go back even earlier but I don't know where
the syslogd was (or if it even had one) if we go further back.

> >SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually
> >worked the way I intended it -- that the initial connect() binds the
> >address even if syslogd isn't listening yet and keeps it working even
> >if syslogd is restarted. Unforunately it doesn't work that way, which
> >means there's no way for chrooted processes to re-establish their
> >syslog connections if syslogd goes down.
> 
>  And that's why a correct supervision architecture must perform
> fd-holding on the fd outputting the logs. Which is only possible
> if it knows that fd before the client start, i.e. the client logs
> to stderr instead of syslog.
> 
>  Are there real, current advantages of SOCK_DGRAM - not only
> hypothetical ones - and most importantly, is there a specification
> anywhere ?

I'm not sure.

Rich

Re: /dev/log: datagram, stream, both?

[Laurent Bercot]

On 11/08/2015 23:41, Rich Felker wrote:
> Busybox 1.0 syslogd:
>
> http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561
>
> That's from 2004. We could go back even earlier but I don't know where
> the syslogd was (or if it even had one) if we go further back.

  I'm not denying there were SOCK_DGRAM servers. What I'm saying is that
I had a SOCK_STREAM server and it worked, which means libcs had support
for SOCK_STREAM clients.

  And the glibc still does:
  http://repo.or.cz/w/glibc.git/blob/HEAD:/misc/syslog.c

  If there's no authoritative source asserting that /dev/log is a datagram
socket and nothing else, I'm submitting a feature request to support
SOCK_STREAM in musl's syslog(). I know it would horribly complicate the
code, but I have something that works with glibc and doesn't with musl,
and nobody wants that. :)

-- 
  Laurent

Re: /dev/log: datagram, stream, both?

[Rich Felker]

On Tue, Aug 11, 2015 at 11:53:38PM +0200, Laurent Bercot wrote:
> On 11/08/2015 23:41, Rich Felker wrote:
> >Busybox 1.0 syslogd:
> >
> >http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561
> >
> >That's from 2004. We could go back even earlier but I don't know where
> >the syslogd was (or if it even had one) if we go further back.
> 
>  I'm not denying there were SOCK_DGRAM servers. What I'm saying is that
> I had a SOCK_STREAM server and it worked, which means libcs had support
> for SOCK_STREAM clients.
> 
>  And the glibc still does:
>  http://repo.or.cz/w/glibc.git/blob/HEAD:/misc/syslog.c
> 
>  If there's no authoritative source asserting that /dev/log is a datagram
> socket and nothing else, I'm submitting a feature request to support
> SOCK_STREAM in musl's syslog(). I know it would horribly complicate the
> code, but I have something that works with glibc and doesn't with musl,
> and nobody wants that. :)

Obviously I'd rather not do this, but if there's a convincing argument
that it's needed for compatibility with real setups we could consider
it. Part of being a convincing argument would be showing that
SOCK_STREAM is a documented feature that's supposed to work rather
than historical baggage, but even then, since the current code works
with all widely used log daemons, I think we'd also want a strong
argument why stream is better than datagram for your setup.

If SOCK_STREAM support does end up being deemed a feature that should
be added, some careful consideration should be made to how the
fallback and reconnection logic should work. I suspect glibc does it
in a way that's less than ideal. I certainly would want to avoid any
robustness regressions for normal SOCK_DGRAM setups.

Rich

Re: /dev/log: datagram, stream, both?

[Laurent Bercot]

On 14/08/2015 05:10, Rich Felker wrote:
> Obviously I'd rather not do this, but if there's a convincing argument
> that it's needed for compatibility with real setups we could consider it.

  Well, the only thing I can say is: I have a SOCK_STREAM server, it's
working, people use it. As far as I know, there's a commercial embedded
box somewhere out there that uses it, if they finally managed to ship
that... thing.
  Of course, it wouldn't really matter for *that* embedded box, which
uses glibc. (Yeah.) But who knows, maybe they could be touched by
divine grace and recompile everything with musl. Miracles happen,
especially when they result in breaking things.
  And I very much intend to work on more embedded projects in the
future, with enough control to make musl mandatory.


> Part of being a convincing argument would be showing that
> SOCK_STREAM is a documented feature that's supposed to work rather
> than historical baggage

  Well, there's basically *no* documentation for that part of syslog
that I can find, and nothing is clear. So from my point of view,
anything is fair game; nothing even says that syslog() has to use
/dev/log as a communication point, except convention. That convention
is pretty strong, but that's all it is. The convention that syslog()
should use datagrams, however, is weak.


> but even then, since the current code works
> with all widely used log daemons, I think we'd also want a strong
> argument why stream is better than datagram for your setup.

  Stream is better than datagram because it can be handled by a
superserver. With stream, I can listen(), accept() a connection per
instance of openlog, and fork a private ucspilogd process that logs
one stream of data from a unique source: this makes for fewer lines
of code, and easier log processing, as if daemons were writing to
stderr. The client is authenticated once, at connection time.

  Datagram, on the other hand, enforces that there is only one instance
of the syslogd server, handling messages from all the syslog()s on the
machine. It enforces that its configuration will be global. Clients
are authenticated on every log message, so if the logger needs to
separate logs depending on client identity, its efficiency decreases
with the amount of data. Note that the reason why widespread setups
don't separate logs depending on client identity is that widespread
syslogd implementations make this difficult !

  With SOCK_STREAM, I can mitigate a large part of why syslog() is bad
and implement a complete syslog server with very little code in an
efficient way. With SOCK_DGRAM, I can just hang my hat and go home,
admit that there's no doing better than current syslogd implementations,
and get drunk when people start using journald on embedded boxes because
it's so much better (tm).


> If SOCK_STREAM support does end up being deemed a feature that should
> be added, some careful consideration should be made to how the
> fallback and reconnection logic should work.

  Design question first: why do you keep the datagram socket connected
instead of just using sendto()?

-- 
  Laurent