* /dev/log: datagram, stream, both? @ 2015-08-10 21:04 Laurent Bercot 2015-08-11 4:59 ` Rich Felker 0 siblings, 1 reply; 7+ messages in thread From: Laurent Bercot @ 2015-08-10 21:04 UTC (permalink / raw) To: musl Hello, I just noticed that musl's syslog() uses a datagram socket. This will only work if the syslogd implementation has opened /dev/log as a datagram socket too. Last time I checked, which was a certain number of years ago, /dev/log was a SOCK_STREAM. And it was great, because it allowed me to very easily whip up a superserver-based implementation of syslogd in a few lines of code. I tested it, it worked, with all the syslog() implementations I could find. But now, it appears to have changed, and my elegant approach isn't working anymore. Gah! A Web search is pretty confusing: nobody seems to agree, there doesn't seem to be an authority on this. So I'm asking here: is /dev/log supposed to be a SOCK_DGRAM now? Is there a normative body somewhere? anything else than "rsyslogd decides and libcs have to adapt"? What are the reasons to use SOCK_DGRAM over SOCK_STREAM? If anyone needed another reason not to use syslog: underspecified protocol with a client in the libc but an external server. Wheee! -- Laurent ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-10 21:04 /dev/log: datagram, stream, both? Laurent Bercot @ 2015-08-11 4:59 ` Rich Felker 2015-08-11 6:02 ` Laurent Bercot 0 siblings, 1 reply; 7+ messages in thread From: Rich Felker @ 2015-08-11 4:59 UTC (permalink / raw) To: musl On Mon, Aug 10, 2015 at 11:04:51PM +0200, Laurent Bercot wrote: > > Hello, > I just noticed that musl's syslog() uses a datagram socket. > This will only work if the syslogd implementation has opened > /dev/log as a datagram socket too. AFAIK it was always SOCK_DGRAM. > Last time I checked, which was a certain number of years ago, > /dev/log was a SOCK_STREAM. And it was great, because it allowed > me to very easily whip up a superserver-based implementation of > syslogd in a few lines of code. I tested it, it worked, with all > the syslog() implementations I could find. > > But now, it appears to have changed, and my elegant approach > isn't working anymore. Gah! > A Web search is pretty confusing: nobody seems to agree, there > doesn't seem to be an authority on this. So I'm asking here: is > /dev/log supposed to be a SOCK_DGRAM now? Is there a normative > body somewhere? anything else than "rsyslogd decides and libcs > have to adapt"? What are the reasons to use SOCK_DGRAM over > SOCK_STREAM? SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually worked the way I intended it -- that the initial connect() binds the address even if syslogd isn't listening yet and keeps it working even if syslogd is restarted. Unforunately it doesn't work that way, which means there's no way for chrooted processes to re-establish their syslog connections if syslogd goes down. Rich ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-11 4:59 ` Rich Felker @ 2015-08-11 6:02 ` Laurent Bercot 2015-08-11 21:41 ` Rich Felker 0 siblings, 1 reply; 7+ messages in thread From: Laurent Bercot @ 2015-08-11 6:02 UTC (permalink / raw) To: musl On 11/08/2015 06:59, Rich Felker wrote: > AFAIK it was always SOCK_DGRAM. It definitely wasn't, say, 6ish years ago. I have seen glibc's and uClibc's syslog() work with SOCK_STREAM. I didn't check the client code, maybe it tried both, but it *was* succeeding in logging stuff when there was a server listening on a /dev/log SOCK_STREAM. > SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually > worked the way I intended it -- that the initial connect() binds the > address even if syslogd isn't listening yet and keeps it working even > if syslogd is restarted. Unforunately it doesn't work that way, which > means there's no way for chrooted processes to re-establish their > syslog connections if syslogd goes down. And that's why a correct supervision architecture must perform fd-holding on the fd outputting the logs. Which is only possible if it knows that fd before the client start, i.e. the client logs to stderr instead of syslog. Are there real, current advantages of SOCK_DGRAM - not only hypothetical ones - and most importantly, is there a specification anywhere ? -- Laurent ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-11 6:02 ` Laurent Bercot @ 2015-08-11 21:41 ` Rich Felker 2015-08-11 21:53 ` Laurent Bercot 0 siblings, 1 reply; 7+ messages in thread From: Rich Felker @ 2015-08-11 21:41 UTC (permalink / raw) To: musl On Tue, Aug 11, 2015 at 08:02:43AM +0200, Laurent Bercot wrote: > On 11/08/2015 06:59, Rich Felker wrote: > >AFAIK it was always SOCK_DGRAM. > > It definitely wasn't, say, 6ish years ago. I have seen > glibc's and uClibc's syslog() work with SOCK_STREAM. I didn't > check the client code, maybe it tried both, but it *was* > succeeding in logging stuff when there was a server listening > on a /dev/log SOCK_STREAM. Busybox 1.0 syslogd: http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561 That's from 2004. We could go back even earlier but I don't know where the syslogd was (or if it even had one) if we go further back. > >SOCK_DGRAM would be a huge advantage over SOCK_STREAM if it actually > >worked the way I intended it -- that the initial connect() binds the > >address even if syslogd isn't listening yet and keeps it working even > >if syslogd is restarted. Unforunately it doesn't work that way, which > >means there's no way for chrooted processes to re-establish their > >syslog connections if syslogd goes down. > > And that's why a correct supervision architecture must perform > fd-holding on the fd outputting the logs. Which is only possible > if it knows that fd before the client start, i.e. the client logs > to stderr instead of syslog. > > Are there real, current advantages of SOCK_DGRAM - not only > hypothetical ones - and most importantly, is there a specification > anywhere ? I'm not sure. Rich ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-11 21:41 ` Rich Felker @ 2015-08-11 21:53 ` Laurent Bercot 2015-08-14 3:10 ` Rich Felker 0 siblings, 1 reply; 7+ messages in thread From: Laurent Bercot @ 2015-08-11 21:53 UTC (permalink / raw) To: musl On 11/08/2015 23:41, Rich Felker wrote: > Busybox 1.0 syslogd: > > http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561 > > That's from 2004. We could go back even earlier but I don't know where > the syslogd was (or if it even had one) if we go further back. I'm not denying there were SOCK_DGRAM servers. What I'm saying is that I had a SOCK_STREAM server and it worked, which means libcs had support for SOCK_STREAM clients. And the glibc still does: http://repo.or.cz/w/glibc.git/blob/HEAD:/misc/syslog.c If there's no authoritative source asserting that /dev/log is a datagram socket and nothing else, I'm submitting a feature request to support SOCK_STREAM in musl's syslog(). I know it would horribly complicate the code, but I have something that works with glibc and doesn't with musl, and nobody wants that. :) -- Laurent ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-11 21:53 ` Laurent Bercot @ 2015-08-14 3:10 ` Rich Felker 2015-08-14 10:08 ` Laurent Bercot 0 siblings, 1 reply; 7+ messages in thread From: Rich Felker @ 2015-08-14 3:10 UTC (permalink / raw) To: musl On Tue, Aug 11, 2015 at 11:53:38PM +0200, Laurent Bercot wrote: > On 11/08/2015 23:41, Rich Felker wrote: > >Busybox 1.0 syslogd: > > > >http://git.busybox.net/busybox/tree/sysklogd/syslogd.c?id=1_00#n561 > > > >That's from 2004. We could go back even earlier but I don't know where > >the syslogd was (or if it even had one) if we go further back. > > I'm not denying there were SOCK_DGRAM servers. What I'm saying is that > I had a SOCK_STREAM server and it worked, which means libcs had support > for SOCK_STREAM clients. > > And the glibc still does: > http://repo.or.cz/w/glibc.git/blob/HEAD:/misc/syslog.c > > If there's no authoritative source asserting that /dev/log is a datagram > socket and nothing else, I'm submitting a feature request to support > SOCK_STREAM in musl's syslog(). I know it would horribly complicate the > code, but I have something that works with glibc and doesn't with musl, > and nobody wants that. :) Obviously I'd rather not do this, but if there's a convincing argument that it's needed for compatibility with real setups we could consider it. Part of being a convincing argument would be showing that SOCK_STREAM is a documented feature that's supposed to work rather than historical baggage, but even then, since the current code works with all widely used log daemons, I think we'd also want a strong argument why stream is better than datagram for your setup. If SOCK_STREAM support does end up being deemed a feature that should be added, some careful consideration should be made to how the fallback and reconnection logic should work. I suspect glibc does it in a way that's less than ideal. I certainly would want to avoid any robustness regressions for normal SOCK_DGRAM setups. Rich ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: /dev/log: datagram, stream, both? 2015-08-14 3:10 ` Rich Felker @ 2015-08-14 10:08 ` Laurent Bercot 0 siblings, 0 replies; 7+ messages in thread From: Laurent Bercot @ 2015-08-14 10:08 UTC (permalink / raw) To: musl On 14/08/2015 05:10, Rich Felker wrote: > Obviously I'd rather not do this, but if there's a convincing argument > that it's needed for compatibility with real setups we could consider it. Well, the only thing I can say is: I have a SOCK_STREAM server, it's working, people use it. As far as I know, there's a commercial embedded box somewhere out there that uses it, if they finally managed to ship that... thing. Of course, it wouldn't really matter for *that* embedded box, which uses glibc. (Yeah.) But who knows, maybe they could be touched by divine grace and recompile everything with musl. Miracles happen, especially when they result in breaking things. And I very much intend to work on more embedded projects in the future, with enough control to make musl mandatory. > Part of being a convincing argument would be showing that > SOCK_STREAM is a documented feature that's supposed to work rather > than historical baggage Well, there's basically *no* documentation for that part of syslog that I can find, and nothing is clear. So from my point of view, anything is fair game; nothing even says that syslog() has to use /dev/log as a communication point, except convention. That convention is pretty strong, but that's all it is. The convention that syslog() should use datagrams, however, is weak. > but even then, since the current code works > with all widely used log daemons, I think we'd also want a strong > argument why stream is better than datagram for your setup. Stream is better than datagram because it can be handled by a superserver. With stream, I can listen(), accept() a connection per instance of openlog, and fork a private ucspilogd process that logs one stream of data from a unique source: this makes for fewer lines of code, and easier log processing, as if daemons were writing to stderr. The client is authenticated once, at connection time. Datagram, on the other hand, enforces that there is only one instance of the syslogd server, handling messages from all the syslog()s on the machine. It enforces that its configuration will be global. Clients are authenticated on every log message, so if the logger needs to separate logs depending on client identity, its efficiency decreases with the amount of data. Note that the reason why widespread setups don't separate logs depending on client identity is that widespread syslogd implementations make this difficult ! With SOCK_STREAM, I can mitigate a large part of why syslog() is bad and implement a complete syslog server with very little code in an efficient way. With SOCK_DGRAM, I can just hang my hat and go home, admit that there's no doing better than current syslogd implementations, and get drunk when people start using journald on embedded boxes because it's so much better (tm). > If SOCK_STREAM support does end up being deemed a feature that should > be added, some careful consideration should be made to how the > fallback and reconnection logic should work. Design question first: why do you keep the datagram socket connected instead of just using sendto()? -- Laurent ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-08-14 10:08 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-08-10 21:04 /dev/log: datagram, stream, both? Laurent Bercot 2015-08-11 4:59 ` Rich Felker 2015-08-11 6:02 ` Laurent Bercot 2015-08-11 21:41 ` Rich Felker 2015-08-11 21:53 ` Laurent Bercot 2015-08-14 3:10 ` Rich Felker 2015-08-14 10:08 ` Laurent Bercot
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).