Per-user service managers

Per-user service managers

[Mike Buland]

Hello,

I wrote this little program that manages per-user runsv instances.
For each user in the "svusers" group it starts a service manager in
their ~/.sv directory.  The service manager runs as that user, so as
long as they can run the sv program, they can manage their own
services.

Per-user service managers run independently of user logins.

I've released this under the BSD license, and it's available on github.

https://github.com/eichlan/usersv

I hope you find this program useful, I know I have.

--Mike Buland

Re: Per-user service managers

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 2630 bytes --]

On Thu, 20 Oct 2011 10:26:17 -0600, Mike Buland <xagafinelle@gmail.com> wrote:
> I wrote this little program that manages per-user runsv instances.
> For each user in the "svusers" group it starts a service manager in
> their ~/.sv directory.  The service manager runs as that user, so as
> long as they can run the sv program, they can manage their own
> services.
>
> Per-user service managers run independently of user logins.
>
> I've released this under the BSD license, and it's available on github.
>
> https://github.com/eichlan/usersv

Hey, Mike.  Very cool!  I actually wrote basically the exact same thing
a while ago, but never got around to publishing it.  I think this sort
of thing can be very useful.  Thanks for sharing.

I see that your system uses a single process that spawns a runsvdir for
each user.  The problem I see with that is that it's hard to
individually control the user runsvdir processes.  If you do have an
idea about how to control (ie. start/stop/restart/etc.) the user
runsvdir processes I would be interested in hearing it.

The system I put together uses a separate runsv dir for each user, and
the entire system is basically encapsulated in the runsv run script (and
log/run script), which I've pasted in below.  The env dir for this
particular example is as follows:

HOME=/home/jrollins
LOG=/home/jrollins/.service.log
PATH=/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
RUNSVDIR=/home/jrollins/.service
USER=jrollins

Anyway, just thought you might be interested, and thanks again for
sharing your work.

jamie.


servo 0$ cat /etc/sv/runsvdir-jrollins/run
#!/bin/sh
exec 2>&1
USER=`head env/USER`
RUNSVDIR=`head env/RUNSVDIR`
GROUPS=$(groups "$USER" | cut -d: -f2 | tr ' ' ':')
echo "${USER}${GROUPS}"
until [ -d "$RUNSVDIR" ] ; do
    sleep 10
done
if [ -d log/main ] ; then
    exec chpst -u "${USER}${GROUPS}" -e env \
	runsvdir -P "$RUNSVDIR"
else
    exec chpst -u "${USER}${GROUPS}" -e env \
	runsvdir -P "$RUNSVDIR" 'log: ...........................................................................................................................................................................................................................................................................................................................................................................................................'
fi

servo 0$ cat /etc/sv/runsvdir-jrollins/log/run
#!/bin/sh
set -e
LOG=`readlink -f ./main`
USER=`head ../env/USER`
if ! [ -d "$LOG" ] ; then
    mkdir -p -m0750 "$LOG"
    chown "$USER":"$USER" "$LOG"
fi
exec chpst -u "$USER" svlogd -tt "$LOG"

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Per-user service managers

[Mike Buland]

[-- Attachment #1: Type: text/plain, Size: 3880 bytes --]

On Oct 20, 2011 11:40 AM, "Jameson Graef Rollins" <
jrollins@finestructure.net> wrote:
>
> On Thu, 20 Oct 2011 10:26:17 -0600, Mike Buland <xagafinelle@gmail.com>
wrote:
> > I wrote this little program that manages per-user runsv instances.
> > For each user in the "svusers" group it starts a service manager in
> > their ~/.sv directory.  The service manager runs as that user, so as
> > long as they can run the sv program, they can manage their own
> > services.
> >
> > Per-user service managers run independently of user logins.
> >
> > I've released this under the BSD license, and it's available on github.
> >
> > https://github.com/eichlan/usersv
>
> Hey, Mike.  Very cool!  I actually wrote basically the exact same thing
> a while ago, but never got around to publishing it.  I think this sort
> of thing can be very useful.  Thanks for sharing.
>
> I see that your system uses a single process that spawns a runsvdir for
> each user.  The problem I see with that is that it's hard to
> individually control the user runsvdir processes.  If you do have an
> idea about how to control (ie. start/stop/restart/etc.) the user
> runsvdir processes I would be interested in hearing it.

If I understand correctly you're referring to the directory containing the
services.  This program uses the .sv directory in each user's home
directory.

As for controlling the runsvdir processes, I've never had ocassion to stop
them once their started.  My program does track all of them, and it has been
my intention to shutdown the runsvdir process for a user when they are
removed from the svusers group.  However, since runsvdir is designed to run
forever and my intention was to provide reliable services for priveleged
users,  I'm not sure the restarting users runsvdir processes is necesarry.
That doesn't mean I wouldn't happily accept a patch that provides that
feature.

I think that's actually an interesting question, my original idea was based
around a grant/revoke model.  Providing more fine grain control of the
users' service managers never occured to me.

Thank you for your reply, it's always good to get feedback.

> The system I put together uses a separate runsv dir for each user, and
> the entire system is basically encapsulated in the runsv run script (and
> log/run script), which I've pasted in below.  The env dir for this
> particular example is as follows:
>
> HOME=/home/jrollins
> LOG=/home/jrollins/.service.log
> PATH=/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
> RUNSVDIR=/home/jrollins/.service
> USER=jrollins
>
> Anyway, just thought you might be interested, and thanks again for
> sharing your work.
>
> jamie.
>
>
> servo 0$ cat /etc/sv/runsvdir-jrollins/run
> #!/bin/sh
> exec 2>&1
> USER=`head env/USER`
> RUNSVDIR=`head env/RUNSVDIR`
> GROUPS=$(groups "$USER" | cut -d: -f2 | tr ' ' ':')
> echo "${USER}${GROUPS}"
> until [ -d "$RUNSVDIR" ] ; do
>    sleep 10
> done
> if [ -d log/main ] ; then
>    exec chpst -u "${USER}${GROUPS}" -e env \
>        runsvdir -P "$RUNSVDIR"
> else
>    exec chpst -u "${USER}${GROUPS}" -e env \
>        runsvdir -P "$RUNSVDIR" 'log:
...........................................................................................................................................................................................................................................................................................................................................................................................................'
> fi
>
> servo 0$ cat /etc/sv/runsvdir-jrollins/log/run
> #!/bin/sh
> set -e
> LOG=`readlink -f ./main`
> USER=`head ../env/USER`
> if ! [ -d "$LOG" ] ; then
>    mkdir -p -m0750 "$LOG"
>    chown "$USER":"$USER" "$LOG"
> fi
> exec chpst -u "$USER" svlogd -tt "$LOG"

Re: Per-user service managers

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

On Thu, 20 Oct 2011 12:11:38 -0600, Mike Buland <xagafinelle@gmail.com> wrote:
> As for controlling the runsvdir processes, I've never had ocassion to stop
> them once their started.  My program does track all of them, and it has been
> my intention to shutdown the runsvdir process for a user when they are
> removed from the svusers group.  However, since runsvdir is designed to run
> forever and my intention was to provide reliable services for priveleged
> users,  I'm not sure the restarting users runsvdir processes is necesarry.
> That doesn't mean I wouldn't happily accept a patch that provides that
> feature.

I think control of the user runsvdir processes is just as important as
control of other long-running processes that you usually intend to have
run forever (cron or apache or whatever).  That's why the sv interface
exists, after all, even though you don't usually intend to ever stop
cron, for example.  Ultimately there's always a need.  Stop the user's
service when they are removed from the system is a relevant example, for
instance.

I like to have a separate runsvdir process per user (as your system has)
but then to also have system-level runsv control over those user
runsvdir processes.  That's why I opted to go for separate runsv
directories per user.  I can then use a simple script to construct and
tear down those runsv directories as needed, and then use sv as root to
control the user directories from a system level.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Per-user service managers

[Mike Buland]

On Thu, Oct 20, 2011 at 1:16 PM, Jameson Graef Rollins
<jrollins@finestructure.net> wrote:
> On Thu, 20 Oct 2011 12:11:38 -0600, Mike Buland <xagafinelle@gmail.com> wrote:
>> As for controlling the runsvdir processes, I've never had ocassion to stop
>> them once their started.  My program does track all of them, and it has been
>> my intention to shutdown the runsvdir process for a user when they are
>> removed from the svusers group.  However, since runsvdir is designed to run
>> forever and my intention was to provide reliable services for priveleged
>> users,  I'm not sure the restarting users runsvdir processes is necesarry.
>> That doesn't mean I wouldn't happily accept a patch that provides that
>> feature.
>
> I think control of the user runsvdir processes is just as important as
> control of other long-running processes that you usually intend to have
> run forever (cron or apache or whatever).  That's why the sv interface
> exists, after all, even though you don't usually intend to ever stop
> cron, for example.  Ultimately there's always a need.  Stop the user's
> service when they are removed from the system is a relevant example, for
> instance.
>
> I like to have a separate runsvdir process per user (as your system has)
> but then to also have system-level runsv control over those user
> runsvdir processes.  That's why I opted to go for separate runsv
> directories per user.  I can then use a simple script to construct and
> tear down those runsv directories as needed, and then use sv as root to
> control the user directories from a system level.
>
> jamie.
>

I'm very sorry for the confusion.  I agree with you, it's a very good
idea.  I haven't needed that feature yet, and it hadn't occurred to
me, hence, it's not in this program yet.  Thank you for the feedback.
I didn't like managing the individual directories, so I made this.

Enjoy,
--Mike Buland

problem with mailing list and multipart/mime?

[Jameson Graef Rollins]

It seems that my multipart/signed messages sent to the list have had
their MIME structure destroyed somewhere along the way.  It looks like
the initial content boundary between the header and the body has been
stripped.

Has anyone else noticed this behavior?  I've never had this issue with
any other mailing list.  Would someone else mind sending at
multipart/signed message to the list?  I would be curious to see if the
same thing happens to anyone else.

jamie.

Re: problem with mailing list and multipart/mime?

[Uffe Jakobsen]

On 2011-10-21 18:44, Jameson Graef Rollins wrote:
 > It seems that my multipart/signed messages sent to the list have had
 > their MIME structure destroyed somewhere along the way.  It looks like
 > the initial content boundary between the header and the body has been
 > stripped.
 >
 > Has anyone else noticed this behavior?  I've never had this issue with
 > any other mailing list.

I can confirm your observations.

The two msgs sent by your earlier today was unreadable in Thunderbird 
3.1.15 (Ubuntu 11.04 64-bit)

I had to view the messags source (CTRL-U) from Thunderbird in order to 
read what you wrote. At the point I blamed Thunderbird - and gave it no 
more attention, I can add that I've never seem such problem before either.

/Uffe

Re: problem with mailing list and multipart/mime?

[Alex Efros]

[-- Attachment #1: Type: text/plain, Size: 45 bytes --]

Hi!

Test signed message.

-- 
			WBR, Alex.

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: problem with mailing list and multipart/mime?

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

On Sat, 22 Oct 2011 04:31:06 +0300, Alex Efros <powerman@powerman.name> wro=
te:
> Hi!
>=20
> Test signed message.

Thanks, Alex.  Your messages seems to have gone through fine.  Which
unfortunately means the problem is specific to me.

I'm going to try sending this message signed message but with a spoofed
user-agent and see what happens.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: problem with mailing list and multipart/mime?

[Alex Efros]

Hi!

I've received two copies of this email: one sent to me and second sent to
maillist. Copy sent to me works ok - correct signed message. Copy sent to
maillist is broken. So, looks like maillist bug.

To Laurent Bercot:
Broken message is multipart with two parts:

    ...
    Content-Type: multipart/signed; boundary="=-=-=";
	    micalg=pgp-sha256; protocol="application/pgp-signature"
    ...

    --=-=-=
    Content-Transfer-Encoding: quoted-printable

    ...

    --=-=-=
    Content-Type: application/pgp-signature

    ...
    --=-=-=--

My previous test message (which was processed correctly by maillist) looks
this way:

    ...
    Content-Type: multipart/signed; micalg=pgp-sha1;
	    protocol="application/pgp-signature"; boundary="V88s5gaDVPzZ0KCq"
    ...

    --V88s5gaDVPzZ0KCq
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline

    ...

    --V88s5gaDVPzZ0KCq
    Content-Type: application/pgp-signature

    ...

    --V88s5gaDVPzZ0KCq--


-- 
			WBR, Alex.

Re: problem with mailing list and multipart/mime?

[Laurent Bercot]

> I've received two copies of this email: one sent to me and second sent to
> maillist. Copy sent to me works ok - correct signed message. Copy sent to
> maillist is broken. So, looks like maillist bug.

 The mailing-list software *modifies* the message - namely, it adds a few
headers (such as List-Unsubscribe) and removes other headers (such as
Return-Path - for obvious reasons). So, it's normal that you cannot
verify the signature of a message after it goes through the list.

 The adding and removal of headers should be perfectly safe and I don't
think there's a bug in there. On the other hand, there *might* be a
problem with the handling of MIME types, because it's something
complex. The set of MIME types that are forbidden (i.e. if such a MIME
type is encountered in a multipart message, the corresponding part of
the message will be deleted) is the default set for the version of
ezmlm-idx I am using. It includes text/html; it does not include
text/plain, of course; and it does not include application/pgp-signature.

 Since messages are modified when going through the list, it makes no
sense to sign them, so I might as well add application/pgp-signature
to the list of removed MIME parts, and end this bit of confusion.

 If you think you have exhibited a real bug, such as a text/plain part
of a MIME message being removed, please send proof to me: a message
sent 1. to the list and 2. directly to me (using the address in the
"From:" header of this message, and replying to the confirmation
request), so I can compare the two and use it to investigate. But
ezmlm-idx has worked flawlessly for years, and the MIME filter has
being treating us pretty well, so I don't think there's a bug involved -
at most, a misconfiguration on my part is possible.

 Anyway I'll be away for the week-end, so we'll see this next Monday.

-- 
 Laurent

Re: problem with mailing list and multipart/mime?

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 3969 bytes --]

Hi, Laurent.  Thanks for the response.  Comments below.

On Sat, 22 Oct 2011 13:47:47 +0200, Laurent Bercot <ska-supervision@skarnet.org> wrote:
> The mailing-list software *modifies* the message - namely, it adds a
> few headers (such as List-Unsubscribe) and removes other headers (such
> as Return-Path - for obvious reasons). So, it's normal that you cannot
> verify the signature of a message after it goes through the list.

PGP/MIME signatures are made over the body (and possibly any
attachments) of the message, and specifically *not* the headers [0].
Signatures in fact can't be made over headers, since obviously the
headers change frequently during the delivery of a message, and if
signatures were made over the headers as well the signatures would
always break.

But in fact it has become clear that the problem is not with the
signatures.  Alex sent a PGP/MIME signed message to the list that
appeared to come through with no problems at all (that I can tell).  See
below for more information.

> The adding and removal of headers should be perfectly safe and I don't
> think there's a bug in there. On the other hand, there *might* be a
> problem with the handling of MIME types, because it's something
> complex. The set of MIME types that are forbidden (i.e. if such a MIME
> type is encountered in a multipart message, the corresponding part of
> the message will be deleted) is the default set for the version of
> ezmlm-idx I am using. It includes text/html; it does not include
> text/plain, of course; and it does not include
> application/pgp-signature.

The removal of MIME parts could definitely break signatures if the
signature was made over the removed part.
 
> Since messages are modified when going through the list, it makes no
> sense to sign them, so I might as well add application/pgp-signature
> to the list of removed MIME parts, and end this bit of confusion.

I disagree.  One of the main reason one signs messages is to verify that
they were not modified in transit.  But that said, I can understand that
a mailing list would want to scrub messages of potentially dangerous
parts.  However, OpenPGP signatures are not damaging, and as long as the
signatures cover only other benign parts, such as text/plain (as were my
messages), then there's no reason for the mailing list to mangle them.

I am subscribed to literally dozens of mailing lists, and I sign my
messages to all of them.  This is the only list I've ever encountered
where this is an issue.

> If you think you have exhibited a real bug, such as a text/plain part
> of a MIME message being removed, please send proof to me: a message
> sent 1. to the list and 2. directly to me (using the address in the
> "From:" header of this message, and replying to the confirmation
> request), so I can compare the two and use it to investigate. But
> ezmlm-idx has worked flawlessly for years, and the MIME filter has
> being treating us pretty well, so I don't think there's a bug involved
> - at most, a misconfiguration on my part is possible.

I actually think this is a real bug.  Even a misconfiguration should not
produce the effect that we're seeing.

I've attached two messages below: the original of a message I sent to
the list, and what I received back from the list.  The only problem is
that a single MIME content boundary has been removed, without removing
anything else in the message.  This of course breaks the MIME structure,
making the message unrenderable.

What's strange is that Alex's signed message sent to the list was *not*
mangled in the same way.  The only thing I can think of is that for some
reason the list software doesn't like the string used as the content
boundary in my message ("--=-=-="), but doesn't mind the one that Alex
used ("V88s5gaDVPzZ0KCq").

Anyway, thanks for looking into this, Laurent.  I hope you can find the
bug.  Let me know if there's anything else I can do to help.

jamie.

[0] http://tools.ietf.org/html/rfc3156


[-- Attachment #2: original message --]
[-- Type: message/rfc822, Size: 1891 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 114 bytes --]

This is another test signed message.  This message explicitly has a
"Content-Disposition: inline" header.

jamie.

[-- Attachment #2.1.2: Type: application/pgp-signature, Size: 835 bytes --]

[-- Attachment #3: message received from list --]
[-- Type: message/rfc822, Size: 3225 bytes --]

[-- Attachment #3.1.1: Type: text/plain, Size: 350 bytes --]

On Sat, 22 Oct 2011 04:31:06 +0300, Alex Efros <powerman@powerman.name> wro=
te:
> Hi!
>=20
> Test signed message.

Thanks, Alex.  Your messages seems to have gone through fine.  Which
unfortunately means the problem is specific to me.

I'm going to try sending this message signed message but with a spoofed
user-agent and see what happens.

jamie.

[-- Attachment #3.1.2: Type: application/pgp-signature, Size: 835 bytes --]

Re: problem with mailing list and multipart/mime?

[Jameson Graef Rollins]

Well, ironically, that message I just sent to the list shows that the
problem has nothing to do with multipart/signed messages, since that
message wasn't signed.  The message was just multipart/mixed.  And again
the MIME structure was destroyed by the list when a single content
boundary line ("--==-=-=") was removed from the message.  I hope that
helps clear things up.

jamie.

Re: problem with mailing list and multipart/mime?

[Laurent Bercot]

> Anyway, thanks for looking into this, Laurent.  I hope you can find the
> bug.  Let me know if there's anything else I can do to help.

 I'm sorry I haven't taken the time to look into this. Debugging other
people's code is what I do all day at work, and it's a painful, boring
and infuriating task - and I just can't get myself to do it again when
at home. ;)

 However, the skarnet.org server is going to migrate to another platform
soon (say, in about two or three months), and I'll use this opportunity
to upgrade the mailing-list management software to a somewhat less
ancient version. I hope the bug will be fixed in the upgraded version.

-- 
 Laurent

Re: problem with mailing list and multipart/mime?

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 114 bytes --]

This is another test signed message.  This message explicitly has a
"Content-Disposition: inline" header.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: problem with mailing list and multipart/mime?

[Laurent Bercot]

> Has anyone else noticed this behavior?  I've never had this issue with
> any other mailing list.  Would someone else mind sending at
> multipart/signed message to the list?  I would be curious to see if the
> same thing happens to anyone else.

 The mailing-list software destroys HTML parts of messages, it only
accepts plain text.
 It should accept multipart MIME messages, though, as long as there
is no HTML in it; if you send both, it should filter the HTML and
let the plain text get through.

 If you have sent plain text and it did not get through, please
confirm it; I will then investigate the issue.

-- 
 Laurent