* Linuxisms in s6
@ 2016-08-25 4:53 Jonathan de Boyne Pollard
2016-08-25 9:56 ` Jan Bramkamp
2016-08-25 15:46 ` Adrian Chadd
0 siblings, 2 replies; 13+ messages in thread
From: Jonathan de Boyne Pollard @ 2016-08-25 4:53 UTC (permalink / raw)
To: Adrian Chadd, Supervision, FreeBSD Hackers
http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
, Adrian Chadd:
> We're using s6 at work, and it works out mostly ok. Mostly once you
> get around the linuxisms, and the lack of sensible time code in it
> (its calculations for daemon run duration is based on system time, not
> wall clock, so if your box boots jan 1, 1970 then gets NTP, things
> are.. hilarious), and some of the arcane bits to get logging working
> right.
>
What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
Linuxisms, obviously. But what Linuxisms does s6 have?
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 4:53 Linuxisms in s6 Jonathan de Boyne Pollard
@ 2016-08-25 9:56 ` Jan Bramkamp
2016-08-25 15:46 ` Adrian Chadd
1 sibling, 0 replies; 13+ messages in thread
From: Jan Bramkamp @ 2016-08-25 9:56 UTC (permalink / raw)
To: Jonathan de Boyne Pollard, Adrian Chadd, Supervision, FreeBSD Hackers
On 25/08/16 06:53, Jonathan de Boyne Pollard wrote:
> http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
> , Adrian Chadd:
>
>> We're using s6 at work, and it works out mostly ok. Mostly once you
>> get around the linuxisms, and the lack of sensible time code in it
>> (its calculations for daemon run duration is based on system time, not
>> wall clock, so if your box boots jan 1, 1970 then gets NTP, things
>> are.. hilarious), and some of the arcane bits to get logging working
>> right.
>>
> What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
> Linuxisms, obviously. But what Linuxisms does s6 have?
>
The skalibs library used by s6 to calculate the deadlines should use
clock_gettime(CLOCK_MONOTONIC) on FreeBSD and as such shouldn't be
affected by changes to the wall clock.
I'm currently working on a FreeBSD only potential init replacement as
well just without the mandatory per service supervisor process. The new
kqueue EVFILT_PROCDESC filter type in FreeBSD 11 combined with pdfork()
should make it really easy to deal child processes in a single unified
kevent loop. Forking services could still be handled by a supervisor
using procctl(PROC_REAP_ACQUIRE).
At the moment I'm fighting with some corner cases in the file descriptor
passing code and redesigning the API to work without access to a
writable file system. My last API required a writable file system
because FreeBSD doesn't support listen()ing on unbound unix domain
seqpacket sockets and I don't want to require something like the Linux
/run tmpfs. Instead my new API uses socketpair() to create a connected
pair of anonymous unix domain sockets for each supervised process. Next
I have to find out if fexecve() works at least for fd 0, 1 and 2 without
a mounted fdescfs.
I want to implement the following features in a single process capable
of running as PID 1:
- Track service dependencies (want, require, bind, conflict)
- Store, Retrieve and close file descriptors.
- Spawn and supervise processes in a well defined environment.
- Reliable event notification with coalescing.
- Bootstrap the system with help from a default service.
With those features it should be able to wrap existing rc.d scripts
without resorting to polling.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 4:53 Linuxisms in s6 Jonathan de Boyne Pollard
2016-08-25 9:56 ` Jan Bramkamp
@ 2016-08-25 15:46 ` Adrian Chadd
2016-08-25 19:48 ` Lars Engels
` (2 more replies)
1 sibling, 3 replies; 13+ messages in thread
From: Adrian Chadd @ 2016-08-25 15:46 UTC (permalink / raw)
To: Jonathan de Boyne Pollard; +Cc: Supervision, FreeBSD Hackers
On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
<J.deBoynePollard-newsgroups@ntlworld.com> wrote:
> http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
> , Adrian Chadd:
>
>> We're using s6 at work, and it works out mostly ok. Mostly once you get
>> around the linuxisms, and the lack of sensible time code in it (its
>> calculations for daemon run duration is based on system time, not wall
>> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>> hilarious), and some of the arcane bits to get logging working right.
>>
> What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
> Linuxisms, obviously. But what Linuxisms does s6 have?
We just had a bunch of fun trying to get it to build right, and the
uptime stuff really threw us.
It's fine though, I found that s6 may start growing an IPC mechanism
so we could possibly do a launchd/jobd style service later (ie to run
things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
arbitrary messages, etc) so I may try incorporating it again. :)
-adrian
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 15:46 ` Adrian Chadd
@ 2016-08-25 19:48 ` Lars Engels
2016-08-25 21:08 ` Adrian Chadd
2016-08-27 13:38 ` Jonathan de Boyne Pollard
2016-08-27 16:51 ` Jonathan de Boyne Pollard
2 siblings, 1 reply; 13+ messages in thread
From: Lars Engels @ 2016-08-25 19:48 UTC (permalink / raw)
To: Adrian Chadd; +Cc: Supervision, Jonathan de Boyne Pollard, FreeBSD Hackers
[-- Attachment #1: Type: text/plain, Size: 1290 bytes --]
On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
> > http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
> > , Adrian Chadd:
> >
> >> We're using s6 at work, and it works out mostly ok. Mostly once you get
> >> around the linuxisms, and the lack of sensible time code in it (its
> >> calculations for daemon run duration is based on system time, not wall
> >> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
> >> hilarious), and some of the arcane bits to get logging working right.
> >>
> > What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
> > Linuxisms, obviously. But what Linuxisms does s6 have?
>
> We just had a bunch of fun trying to get it to build right, and the
> uptime stuff really threw us.
>
> It's fine though, I found that s6 may start growing an IPC mechanism
> so we could possibly do a launchd/jobd style service later (ie to run
> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
> arbitrary messages, etc) so I may try incorporating it again. :)
>
Can't this be done with devd?
[-- Attachment #2: Type: application/pgp-signature, Size: 603 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 19:48 ` Lars Engels
@ 2016-08-25 21:08 ` Adrian Chadd
2016-08-25 21:13 ` Warner Losh
2016-08-27 16:37 ` Jonathan de Boyne Pollard
0 siblings, 2 replies; 13+ messages in thread
From: Adrian Chadd @ 2016-08-25 21:08 UTC (permalink / raw)
To: Lars Engels; +Cc: Supervision, Jonathan de Boyne Pollard, FreeBSD Hackers
On 25 August 2016 at 12:48, Lars Engels <lars.engels@0x20.net> wrote:
> On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
>> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
>> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
>> > http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
>> > , Adrian Chadd:
>> >
>> >> We're using s6 at work, and it works out mostly ok. Mostly once you get
>> >> around the linuxisms, and the lack of sensible time code in it (its
>> >> calculations for daemon run duration is based on system time, not wall
>> >> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>> >> hilarious), and some of the arcane bits to get logging working right.
>> >>
>> > What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
>> > Linuxisms, obviously. But what Linuxisms does s6 have?
>>
>> We just had a bunch of fun trying to get it to build right, and the
>> uptime stuff really threw us.
>>
>> It's fine though, I found that s6 may start growing an IPC mechanism
>> so we could possibly do a launchd/jobd style service later (ie to run
>> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
>> arbitrary messages, etc) so I may try incorporating it again. :)
>>
>
> Can't this be done with devd?
Sure, but I'm looking for something more generic than just devd. Like,
notifications about events like "default route is up" can be done by
sniffing the rtsock, but notifications like "ntpdate has updated the
date, we can now do crypto services" doesn't happen there right now.
-adrian
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 21:08 ` Adrian Chadd
@ 2016-08-25 21:13 ` Warner Losh
2016-08-25 21:17 ` Adrian Chadd
2016-08-27 16:37 ` Jonathan de Boyne Pollard
1 sibling, 1 reply; 13+ messages in thread
From: Warner Losh @ 2016-08-25 21:13 UTC (permalink / raw)
To: Adrian Chadd
Cc: Supervision, Jonathan de Boyne Pollard, Lars Engels, FreeBSD Hackers
On Thu, Aug 25, 2016 at 3:08 PM, Adrian Chadd <adrian@freebsd.org> wrote:
> On 25 August 2016 at 12:48, Lars Engels <lars.engels@0x20.net> wrote:
>> On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
>>> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
>>> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
>>> > http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
>>> > , Adrian Chadd:
>>> >
>>> >> We're using s6 at work, and it works out mostly ok. Mostly once you get
>>> >> around the linuxisms, and the lack of sensible time code in it (its
>>> >> calculations for daemon run duration is based on system time, not wall
>>> >> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>>> >> hilarious), and some of the arcane bits to get logging working right.
>>> >>
>>> > What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
>>> > Linuxisms, obviously. But what Linuxisms does s6 have?
>>>
>>> We just had a bunch of fun trying to get it to build right, and the
>>> uptime stuff really threw us.
>>>
>>> It's fine though, I found that s6 may start growing an IPC mechanism
>>> so we could possibly do a launchd/jobd style service later (ie to run
>>> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
>>> arbitrary messages, etc) so I may try incorporating it again. :)
>>>
>>
>> Can't this be done with devd?
>
> Sure, but I'm looking for something more generic than just devd. Like,
> notifications about events like "default route is up" can be done by
> sniffing the rtsock, but notifications like "ntpdate has updated the
> date, we can now do crypto services" doesn't happen there right now.
devd was never intended to be limited to just device events from the
kernel. It has grown beyond that, and could easily grow to cope with
routing events and other notifications. No need to reinvent everything
for that.
What devd doesn't do is launchd / job control. That would be a whole
new kettle of fish for it, and one for which it may be ill suited. Though
viewed in the right way, it's all just a series of notifications: This service
is up, this is down, user wants to start this other one, etc, so maybe
it wouldn't so bad...
Warner
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 21:13 ` Warner Losh
@ 2016-08-25 21:17 ` Adrian Chadd
2016-08-26 10:44 ` Jan Bramkamp
0 siblings, 1 reply; 13+ messages in thread
From: Adrian Chadd @ 2016-08-25 21:17 UTC (permalink / raw)
To: Warner Losh
Cc: Supervision, Jonathan de Boyne Pollard, Lars Engels, FreeBSD Hackers
On 25 August 2016 at 14:13, Warner Losh <imp@bsdimp.com> wrote:
> On Thu, Aug 25, 2016 at 3:08 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>> On 25 August 2016 at 12:48, Lars Engels <lars.engels@0x20.net> wrote:
>>> On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
>>>> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
>>>> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
>>>> > http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
>>>> > , Adrian Chadd:
>>>> >
>>>> >> We're using s6 at work, and it works out mostly ok. Mostly once you get
>>>> >> around the linuxisms, and the lack of sensible time code in it (its
>>>> >> calculations for daemon run duration is based on system time, not wall
>>>> >> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>>>> >> hilarious), and some of the arcane bits to get logging working right.
>>>> >>
>>>> > What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
>>>> > Linuxisms, obviously. But what Linuxisms does s6 have?
>>>>
>>>> We just had a bunch of fun trying to get it to build right, and the
>>>> uptime stuff really threw us.
>>>>
>>>> It's fine though, I found that s6 may start growing an IPC mechanism
>>>> so we could possibly do a launchd/jobd style service later (ie to run
>>>> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
>>>> arbitrary messages, etc) so I may try incorporating it again. :)
>>>>
>>>
>>> Can't this be done with devd?
>>
>> Sure, but I'm looking for something more generic than just devd. Like,
>> notifications about events like "default route is up" can be done by
>> sniffing the rtsock, but notifications like "ntpdate has updated the
>> date, we can now do crypto services" doesn't happen there right now.
>
> devd was never intended to be limited to just device events from the
> kernel. It has grown beyond that, and could easily grow to cope with
> routing events and other notifications. No need to reinvent everything
> for that.
Right. I don't want to reinvent the wheel if it can be avoided.
But there are other things that want to produce and consume events.
eg, openvpn bringing up a VPN triggering possible ipfw rule changes.
Or openvpn coming down triggering other ipfw rule changes.
> What devd doesn't do is launchd / job control. That would be a whole
> new kettle of fish for it, and one for which it may be ill suited. Though
> viewed in the right way, it's all just a series of notifications: This service
> is up, this is down, user wants to start this other one, etc, so maybe
> it wouldn't so bad...
Well, ideally the jobd would sit on the message bus and take commands
to do things. Like dbus and udevd did in linux, before systemd
steamrolled over them. But then if I suggest we need a message bus
daemon up and going so arbitrary system pieces could talk to other
system pieces, I'll likely be shouted at.
But not by jkh. He'd likely be "YOURE ONLY JUST GETTING AROUND TO THIS
NOW?" and laugh a lot.
(jkh - please come to the next bafug so we can talk shop..)
-a
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 21:17 ` Adrian Chadd
@ 2016-08-26 10:44 ` Jan Bramkamp
2016-08-26 14:11 ` Warner Losh
0 siblings, 1 reply; 13+ messages in thread
From: Jan Bramkamp @ 2016-08-26 10:44 UTC (permalink / raw)
To: Adrian Chadd, Warner Losh
Cc: Supervision, Jonathan de Boyne Pollard, Lars Engels, FreeBSD Hackers
On 25/08/16 23:17, Adrian Chadd wrote:
> On 25 August 2016 at 14:13, Warner Losh <imp@bsdimp.com> wrote:
>> On Thu, Aug 25, 2016 at 3:08 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>>> On 25 August 2016 at 12:48, Lars Engels <lars.engels@0x20.net> wrote:
>>>> On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
>>>>> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
>>>>> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
>>>>>> http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
>>>>>> , Adrian Chadd:
>>>>>>
>>>>>>> We're using s6 at work, and it works out mostly ok. Mostly once you get
>>>>>>> around the linuxisms, and the lack of sensible time code in it (its
>>>>>>> calculations for daemon run duration is based on system time, not wall
>>>>>>> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>>>>>>> hilarious), and some of the arcane bits to get logging working right.
>>>>>>>
>>>>>> What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
>>>>>> Linuxisms, obviously. But what Linuxisms does s6 have?
>>>>>
>>>>> We just had a bunch of fun trying to get it to build right, and the
>>>>> uptime stuff really threw us.
>>>>>
>>>>> It's fine though, I found that s6 may start growing an IPC mechanism
>>>>> so we could possibly do a launchd/jobd style service later (ie to run
>>>>> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
>>>>> arbitrary messages, etc) so I may try incorporating it again. :)
>>>>>
>>>>
>>>> Can't this be done with devd?
>>>
>>> Sure, but I'm looking for something more generic than just devd. Like,
>>> notifications about events like "default route is up" can be done by
>>> sniffing the rtsock, but notifications like "ntpdate has updated the
>>> date, we can now do crypto services" doesn't happen there right now.
>>
>> devd was never intended to be limited to just device events from the
>> kernel. It has grown beyond that, and could easily grow to cope with
>> routing events and other notifications. No need to reinvent everything
>> for that.
>
> Right. I don't want to reinvent the wheel if it can be avoided.
Afaik devd is limited handling events reported by the kernel on
/etc/devctl. There is no way to inject events into arbitrary events from
userspace into devd (no ptrace hacks don't count).
> But there are other things that want to produce and consume events.
> eg, openvpn bringing up a VPN triggering possible ipfw rule changes.
> Or openvpn coming down triggering other ipfw rule changes.
FreeBSD offers several IPC APIs but non of them can implemented reliable
multicast as this would require an unbounded journal in stable memory.
For most use-cases reliable notification of the current state is enough.
Instead of reliably multicasting each message to each recipient just
send each observer the latest state of each observed value e.g. in your
OpenVPN example the IPFW wrapper doesn't care how many time the tunnel
flapped. The user just wants the right firewall configuration for his
current network environment. He doesn't want to replay every change on
the way.
In macOS has notifyd offers this service. The optimizations in notifyd
make it too large to just import its services into an init process, but
having a simpler reliable notification mechanism early would be useful.
>> What devd doesn't do is launchd / job control. That would be a whole
>> new kettle of fish for it, and one for which it may be ill suited. Though
>> viewed in the right way, it's all just a series of notifications: This service
>> is up, this is down, user wants to start this other one, etc, so maybe
>> it wouldn't so bad...
>
> Well, ideally the jobd would sit on the message bus and take commands
> to do things. Like dbus and udevd did in linux, before systemd
> steamrolled over them. But then if I suggest we need a message bus
> daemon up and going so arbitrary system pieces could talk to other
> system pieces, I'll likely be shouted at.
>
> But not by jkh. He'd likely be "YOURE ONLY JUST GETTING AROUND TO THIS
> NOW?" and laugh a lot.
>
> (jkh - please come to the next bafug so we can talk shop..)
Process spawning and supervision should be separate from the policy
engine(s) as the process supervision graph should be a tree rooted in
pid 1, but the user might want to run multiple rule/policy engines. An
init process should just offer the required mechanisms and nothing more.
Convenient policies can be implemented on top of those mechanisms.
For my own init system I'm still not sure if the init process should
track services and their dependencies at all or just processes and keep
the concept of services and dependencies in a service management process
instead of the init process.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-26 10:44 ` Jan Bramkamp
@ 2016-08-26 14:11 ` Warner Losh
0 siblings, 0 replies; 13+ messages in thread
From: Warner Losh @ 2016-08-26 14:11 UTC (permalink / raw)
To: Jan Bramkamp
Cc: Supervision, Adrian Chadd, Jonathan de Boyne Pollard,
Lars Engels, FreeBSD Hackers
On Fri, Aug 26, 2016 at 4:44 AM, Jan Bramkamp <crest@rlwinm.de> wrote:
> On 25/08/16 23:17, Adrian Chadd wrote:
>>
>> On 25 August 2016 at 14:13, Warner Losh <imp@bsdimp.com> wrote:
>>>
>>> On Thu, Aug 25, 2016 at 3:08 PM, Adrian Chadd <adrian@freebsd.org> wrote:
>>>>
>>>> On 25 August 2016 at 12:48, Lars Engels <lars.engels@0x20.net> wrote:
>>>>>
>>>>> On Thu, Aug 25, 2016 at 08:46:10AM -0700, Adrian Chadd wrote:
>>>>>>
>>>>>> On 24 August 2016 at 21:53, Jonathan de Boyne Pollard
>>>>>> <J.deBoynePollard-newsgroups@ntlworld.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955
>>>>>>> , Adrian Chadd:
>>>>>>>
>>>>>>>> We're using s6 at work, and it works out mostly ok. Mostly once you
>>>>>>>> get
>>>>>>>> around the linuxisms, and the lack of sensible time code in it (its
>>>>>>>> calculations for daemon run duration is based on system time, not
>>>>>>>> wall
>>>>>>>> clock, so if your box boots jan 1, 1970 then gets NTP, things are..
>>>>>>>> hilarious), and some of the arcane bits to get logging working
>>>>>>>> right.
>>>>>>>>
>>>>>>> What are these Linuxisms in s6? s6-linux-utils and s6-linux-init
>>>>>>> have
>>>>>>> Linuxisms, obviously. But what Linuxisms does s6 have?
>>>>>>
>>>>>>
>>>>>> We just had a bunch of fun trying to get it to build right, and the
>>>>>> uptime stuff really threw us.
>>>>>>
>>>>>> It's fine though, I found that s6 may start growing an IPC mechanism
>>>>>> so we could possibly do a launchd/jobd style service later (ie to run
>>>>>> things upon event changes, like ifup, ifdown, ifcreate, ifdestroy,
>>>>>> arbitrary messages, etc) so I may try incorporating it again. :)
>>>>>>
>>>>>
>>>>> Can't this be done with devd?
>>>>
>>>>
>>>> Sure, but I'm looking for something more generic than just devd. Like,
>>>> notifications about events like "default route is up" can be done by
>>>> sniffing the rtsock, but notifications like "ntpdate has updated the
>>>> date, we can now do crypto services" doesn't happen there right now.
>>>
>>>
>>> devd was never intended to be limited to just device events from the
>>> kernel. It has grown beyond that, and could easily grow to cope with
>>> routing events and other notifications. No need to reinvent everything
>>> for that.
>>
>>
>> Right. I don't want to reinvent the wheel if it can be avoided.
>
>
> Afaik devd is limited handling events reported by the kernel on /etc/devctl.
> There is no way to inject events into arbitrary events from userspace into
> devd (no ptrace hacks don't count).
It would be trivial to add though, especially since the author is easy to
work with :). It already fans out data to multiple listeners.
Warner
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 15:46 ` Adrian Chadd
2016-08-25 19:48 ` Lars Engels
@ 2016-08-27 13:38 ` Jonathan de Boyne Pollard
2016-08-27 16:51 ` Jonathan de Boyne Pollard
2 siblings, 0 replies; 13+ messages in thread
From: Jonathan de Boyne Pollard @ 2016-08-27 13:38 UTC (permalink / raw)
To: Adrian Chadd; +Cc: Supervision, FreeBSD Hackers
Jonathan de Boyne Pollard:
> What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have
> Linuxisms, obviously. But what Linuxisms does s6 have?
>
Adrian Chadd:
> We just had a bunch of fun trying to get it to build right, [...]
>
Such as what, specifically?
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 21:08 ` Adrian Chadd
2016-08-25 21:13 ` Warner Losh
@ 2016-08-27 16:37 ` Jonathan de Boyne Pollard
2016-08-28 1:58 ` Adrian Chadd
1 sibling, 1 reply; 13+ messages in thread
From: Jonathan de Boyne Pollard @ 2016-08-27 16:37 UTC (permalink / raw)
To: Adrian Chadd, Supervision, FreeBSD Hackers
Adrian Chadd:
> Sure, but I'm looking for something more generic than just devd. Like,
> notifications about events like "default route is up" can be done by
> sniffing the rtsock, but notifications like "ntpdate has updated the
> date, we can now do crypto services" doesn't happen there right now.
>
You're reinventing upstart. The lesson of upstart is that whilst the
event-driven paradigm looks like the bright shiny future, once one gets
down to the details it is a lot harder than it at first appears. I
strongly recommended learning about upstart, and especially learning the
problems that people hit with it, to anyone going down the same route.
The Debian systemd Hoo-Hah had some lengthy discussion of upstart.
(I regret not having bookmarked the discussion that I once came across,
where someone opined that xe preferred systemd to upstart because at a
Linux conference the systemd presentation had been exciting and had been
put forward as the wave of the future, where upstart had been presented
as old-school, traditional, and boring. Ironically, this person wasn't
aware that the designs are exactly the opposite of that. upstart has
the novel event-driven design where the system is configured with the
information that event A triggers programs P, Q, and R, and the system
starts by raising a "first event", that runs programs, that raise
further events, that run further programs. Whereas it is systemd that
has the conventional design, shared by Mewburn rc and others, of
starting from a goal, working through a dependency tree, and doing
topological sorts.)
The Debian people chose to improve a non-event-driven architecture
instead. It's a lesson to be learned from SMF, in fact. One can have a
lot more additional abstract targets, such as "/milestone/name-services"
and "/milestone/system-clock", and dependencies to and from them. The
world is not 2 to 4 run levels plus "DAEMON", "NETWORKING", and "$local-fs".
That said, something like this hypothetical "/milestone/system-clock" is
a milestone that would need to be reached *very* early on in the
bootstrap process. Fixing up the clock is something that both the nosh
system manager and systemd handle themselves directly, outwith of
service management. More on this in a moment.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-25 15:46 ` Adrian Chadd
2016-08-25 19:48 ` Lars Engels
2016-08-27 13:38 ` Jonathan de Boyne Pollard
@ 2016-08-27 16:51 ` Jonathan de Boyne Pollard
2 siblings, 0 replies; 13+ messages in thread
From: Jonathan de Boyne Pollard @ 2016-08-27 16:51 UTC (permalink / raw)
To: Adrian Chadd, Supervision, FreeBSD Hackers
Adrian Chadd:
> [...] the uptime stuff really threw us.
>
It's unfair to lay such system time problems at s6's door. Systems whose
system clock jumps 46 years during system bootstrap don't get to blame
s6 for mad time gaps that appear in logs and service start time
records. There is a *lot* of the Unix and Linux worlds that depends
from time being right. It's not just s6 that is affected by such
things. You note crypto. There are a lot of other things as well that
have unstated, sometimes undocumented, and sometimes surprising
dependencies upon system time being current.
Here's one such.
For quite a while, Linux distributions had rather an odd problem at
bootstrap. They'd repeatedly fsck volumes at every bootstrap when they
need not have. And this didn't affect U.S. or U.K. people, which is in
part why it persisted for so long.
* https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/63175
* https://bugs.archlinux.org/task/17438
* http://lwn.net/Articles/264498/
The problem was that people were erroneously running their real-time
clocks in local time rather than UTC, and this triggered an odd hidden
dependency upon having the right time in the system clock in countries
where local time was in advance of UTC. The Linux method for handling
RTCs erroneously running in local time is for the system bootstrap to
make a special settimeofday() call that effectively tells the kernel
what the UTC offset is for the RTC hardware. This could happen *after*
the fsck of the root volume, however. So whilst that fsck was
happening, the kernel was assuming that UTC was the local time that it
had taken from the RTC and initialized its system clock with. In
effect, as soon as the special settimeofday() call was executed, the
system clock would jump backwards by one or more hours, to what UTC
actually was.
But the ext2/3/4 filesystem format has last checked/mounted/written
timestamps in its superblock. Part of the checking to see whether a
full fsck is needed at bootstrap is comparing them to the current time.
If they are in the future by hours or more, something is clearly wrong,
thinks fsck, and it runs the full check. At bootstrap, when the initial
fsck (of at least the root volume and sometimes other volumes as well)
is run, the system clock is not UTC yet. Comedy results.
Both systemd and the nosh system manager have to ensure that they do the
special settimeofday() system call before they start off service
management and thus run mount/fsck services, or indeed anything else
that might have a closet dependency from not stepping the system time by
hours partway through bootstrap. The nosh system-manager's manual page
has a whole section on this subject.
FreeBSD/PC-BSD has a mechanism for correctly reading a RTC that is
erroneously in local time. One sets up the RTC's offset from UTC in the
machdep.adjkerntz variable in /boot/loader.conf{,.local} and the system
clock never has to jump by hours during bootstrap. I've yet to
experience a FreeBSD/PC-BSD system where the installer actually
configures this, though.
Interestingly, FreeBSD/PC-BSD also has a fallback mechanism that uses
the latest volume mount timestamp that it can find as the initial system
time when no hardware clock device registers at bootstrap. Presumably
you have a clock device that registers but it is not battery-backed,
your volumes don't preserve (or reset) their mount timestamps, or you
are encountering the comedy situation where FreeBSD/PC-BSD is setting
the system clock to 1970-01-01 because the last time around it mounted
the filesystems before the clock was corrected.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linuxisms in s6
2016-08-27 16:37 ` Jonathan de Boyne Pollard
@ 2016-08-28 1:58 ` Adrian Chadd
0 siblings, 0 replies; 13+ messages in thread
From: Adrian Chadd @ 2016-08-28 1:58 UTC (permalink / raw)
To: Jonathan de Boyne Pollard; +Cc: Supervision, FreeBSD Hackers
On 27 August 2016 at 09:37, Jonathan de Boyne Pollard
<J.deBoynePollard-newsgroups@ntlworld.com> wrote:
> Adrian Chadd:
>
>> Sure, but I'm looking for something more generic than just devd. Like,
>> notifications about events like "default route is up" can be done by
>> sniffing the rtsock, but notifications like "ntpdate has updated the date,
>> we can now do crypto services" doesn't happen there right now.
>>
> You're reinventing upstart. The lesson of upstart is that whilst the
> event-driven paradigm looks like the bright shiny future, once one gets down
> to the details it is a lot harder than it at first appears. I strongly
> recommended learning about upstart, and especially learning the problems
> that people hit with it, to anyone going down the same route. The Debian
> systemd Hoo-Hah had some lengthy discussion of upstart.
Oh yeah, I'm aware of the differences between systemd and upstart.
> (I regret not having bookmarked the discussion that I once came across,
> where someone opined that xe preferred systemd to upstart because at a Linux
> conference the systemd presentation had been exciting and had been put
> forward as the wave of the future, where upstart had been presented as
> old-school, traditional, and boring. Ironically, this person wasn't aware
> that the designs are exactly the opposite of that. upstart has the novel
> event-driven design where the system is configured with the information that
> event A triggers programs P, Q, and R, and the system starts by raising a
> "first event", that runs programs, that raise further events, that run
> further programs. Whereas it is systemd that has the conventional design,
> shared by Mewburn rc and others, of starting from a goal, working through a
> dependency tree, and doing topological sorts.)
For some environments (servers, desktop environments, etc) where most
of the dynamicness comes from "which user is logged in" and "maybe I
don't have my network hardware plugged in until later", sure, I can
see that the dependency tree model works great. Your aim is some
grander set of checkpoints, like "What do I need to run basic network
services", "What do i need up to run a desktop login environment",
etc.
But for things like "I'm a vpn server, and I need to speak to a vpn
server to provide access to my vpn clients, oh and I have firewall
rules that need applying based on which dynamic interfaces come/go" it
still maps to an event driven mechanism. Sure you can map that event
driven mechanism to a set of targets, but those targets may be
per-interface. Like, when a vpn client interface comes up, I have a
set of things that need to happen that depend upon which client. Same
with wireless clients associating. I may hit some situations (eg above
a certain threshold of associated clients) where I trigger events such
as "clean up old clients", "look at migrating clients to other access
points", etc.
If I can do this with s6, then cool - please let me know how and I'll
re-consider it. But regardless of that, I also do need some
generalised dbus style mechanism so all the pieces of the system can
talk to the other pieces of the system without having to .. well, wrap
it all in 'service' style shell scripts and calling ifconfig wlanX
list sta periodically from everywhere.
> The Debian people chose to improve a non-event-driven architecture instead.
> It's a lesson to be learned from SMF, in fact. One can have a lot more
> additional abstract targets, such as "/milestone/name-services" and
> "/milestone/system-clock", and dependencies to and from them. The world is
> not 2 to 4 run levels plus "DAEMON", "NETWORKING", and "$local-fs".
>
> That said, something like this hypothetical "/milestone/system-clock" is a
> milestone that would need to be reached *very* early on in the bootstrap
> process. Fixing up the clock is something that both the nosh system manager
> and systemd handle themselves directly, outwith of service management. More
> on this in a moment.
So, this is where it gets exciting in some of these appliances.
Sometimes there's no 'date/time' RTC hardware. Sometimes, you have to
present some UI to the user so they can enter a date/time, and some
services need to run before that, but some services (notably ntpdate)
won't work. So we can't, say, hold all network services back until we
have valid date/time or a bunch of the UI infrastructure won't be
there.
I can't expect the whole system to stop waiting for a system-clock
time to be valid. In fact, i ended up adding some stuff to our
appliance images that store the current clock value in a file every 15
minutes - the rootfs is read-only, so i can't just boot up from /its/
concept of "last mounted", as that filesystem is not modifiable.
Trouble is, that gets read from the system during boot, after FSes
have been mounted, etc. It's all terrible.
Thanks,
-adrian
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2016-08-28 1:58 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-25 4:53 Linuxisms in s6 Jonathan de Boyne Pollard
2016-08-25 9:56 ` Jan Bramkamp
2016-08-25 15:46 ` Adrian Chadd
2016-08-25 19:48 ` Lars Engels
2016-08-25 21:08 ` Adrian Chadd
2016-08-25 21:13 ` Warner Losh
2016-08-25 21:17 ` Adrian Chadd
2016-08-26 10:44 ` Jan Bramkamp
2016-08-26 14:11 ` Warner Losh
2016-08-27 16:37 ` Jonathan de Boyne Pollard
2016-08-28 1:58 ` Adrian Chadd
2016-08-27 13:38 ` Jonathan de Boyne Pollard
2016-08-27 16:51 ` Jonathan de Boyne Pollard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).