[9fans] Backgrounding a task

[9fans] Backgrounding a task

[Chris McGee]

Hi All,

When I'm using Unix, there's a workflow that I use for long running commands that I'm hoping to find the equivalent in the Plan 9 way of doing things.

I will occasionally run a command, realize that it will take a long time to complete. I don't want to kill it. I'll just Ctrl-Z and bg to put it into the background using the shell. It's almost as if I had run it with '&' in the first place. I can then run other commands in the same working directory, environment and shell history.

Is there an equivalent to this workflow in Plan 9?

I realize that the whole job control system dates back to old single session terminals, which isn't a problem with Rio where you can draw new windows at will. Initially I thought, that you just drag that window to a corner somewhere and let it complete. But, if I draw a new window it won't be in the same working directory, have the same environment and namespace. Maybe there is a way to create a window that inherits these from an existing process?

Chris

Re: [9fans] Backgrounding a task

[Erik Quanstrom]

[-- Attachment #1: Type: text/html, Size: 51 bytes --]

Re: [9fans] Backgrounding a task

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

Spitballing here: in the new window do something like

cat /proc/123/ns | rc

Or first massage the ns then generate an output for rc.

On Fri, Oct 6, 2017, 4:34 PM Chris McGee <newton688@gmail.com> wrote:

> Hi All,
>
> When I'm using Unix, there's a workflow that I use for long running
> commands that I'm hoping to find the equivalent in the Plan 9 way of doing
> things.
>
> I will occasionally run a command, realize that it will take a long time
> to complete. I don't want to kill it. I'll just Ctrl-Z and bg to put it
> into the background using the shell. It's almost as if I had run it with
> '&' in the first place. I can then run other commands in the same working
> directory, environment and shell history.
>
> Is there an equivalent to this workflow in Plan 9?
>
> I realize that the whole job control system dates back to old single
> session terminals, which isn't a problem with Rio where you can draw new
> windows at will. Initially I thought, that you just drag that window to a
> corner somewhere and let it complete. But, if I draw a new window it won't
> be in the same working directory, have the same environment and namespace.
> Maybe there is a way to create a window that inherits these from an
> existing process?
>
> Chris
>

[-- Attachment #2: Type: text/html, Size: 1593 bytes --]

Re: [9fans] Backgrounding a task

[Chris McGee]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

Thanks for the tip! I'll give that a try.

Chris

> On Oct 7, 2017, at 12:04 AM, Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
> 
> Spitballing here: in the new window do something like
> 
> cat /proc/123/ns | rc 
> 
> Or first massage the ns then generate an output for rc.
> 
>> On Fri, Oct 6, 2017, 4:34 PM Chris McGee <newton688@gmail.com> wrote:
>> Hi All,
>> 
>> When I'm using Unix, there's a workflow that I use for long running commands that I'm hoping to find the equivalent in the Plan 9 way of doing things.
>> 
>> I will occasionally run a command, realize that it will take a long time to complete. I don't want to kill it. I'll just Ctrl-Z and bg to put it into the background using the shell. It's almost as if I had run it with '&' in the first place. I can then run other commands in the same working directory, environment and shell history.
>> 
>> Is there an equivalent to this workflow in Plan 9?
>> 
>> I realize that the whole job control system dates back to old single session terminals, which isn't a problem with Rio where you can draw new windows at will. Initially I thought, that you just drag that window to a corner somewhere and let it complete. But, if I draw a new window it won't be in the same working directory, have the same environment and namespace. Maybe there is a way to create a window that inherits these from an existing process?
>> 
>> Chris

[-- Attachment #2: Type: text/html, Size: 1974 bytes --]

[9fans] Backgrounding a task

[Yaroslav Kolomiiets]

?window -m cmd? will run the command in the same namespace, forked, but in new window.

?-m? is for ?mount?, an alternative way of communication with the window system to /dev/wctl which is default.

Yaroslav Kolomiiets

7 ????. 2017 ?. ? 15:21 Chris McGee <newton688 at gmail.com> ????:

Thanks for the tip! I'll give that a try.

Chris



? ??????????
> On Oct 7, 2017, at 12:04 AM, Skip Tavakkolian <skip.tavakkolian at gmail.com> wrote:
> 
> Spitballing here: in the new window do something like
> 
> cat /proc/123/ns | rc 
> 
> Or first massage the ns then generate an output for rc.
> 
>> On Fri, Oct 6, 2017, 4:34 PM Chris McGee <newton688 at gmail.com> wrote:
>> Hi All,
>> 
>> When I'm using Unix, there's a workflow that I use for long running commands that I'm hoping to find the equivalent in the Plan 9 way of doing things.
>> 
>> I will occasionally run a command, realize that it will take a long time to complete. I don't want to kill it. I'll just Ctrl-Z and bg to put it into the background using the shell. It's almost as if I had run it with '&' in the first place. I can then run other commands in the same working directory, environment and shell history.
>> 
>> Is there an equivalent to this workflow in Plan 9?
>> 
>> I realize that the whole job control system dates back to old single session terminals, which isn't a problem with Rio where you can draw new windows at will. Initially I thought, that you just drag that window to a corner somewhere and let it complete. But, if I draw a new window it won't be in the same working directory, have the same environment and namespace. Maybe there is a way to create a window that inherits these from an existing process?
>> 
>> Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.9fans.net/private/9fans/attachments/20171023/bf889239/attachment.html>

[9fans] Backgrounding a task

[Chris McGee]

Thanks,

I was thinking more about the case where I didn't know ahead of time that the command would take a long time. I have a sequence of steps I do in Unix to background the task using job control in the shell. I wasn't sure if there is some kind of plan 9 equivalent to the workflow, even if it is done in a different way.

So far, it looks like the closest equivalent is to draw a new window and inherit the namespace of the original one by reading the namespace from the proc. I wonder if there could be a Rio gesture to draw a new window inheriting the namespace of the window I pick by clicking.

Chris

> On Oct 23, 2017, at 12:20 PM, Yaroslav Kolomiiets <yk.9fans at icloud.com> wrote:
> 
> ?window -m cmd? will run the command in the same namespace, forked, but in new window.
> 
> ?-m? is for ?mount?, an alternative way of communication with the window system to /dev/wctl which is default.
> 
> Yaroslav Kolomiiets
> 
> 7 ????. 2017 ?. ? 15:21 Chris McGee <newton688 at gmail.com> ????:
> 
> Thanks for the tip! I'll give that a try.
> 
> Chris
> 
> 
> 
> ? ??????????
>> On Oct 7, 2017, at 12:04 AM, Skip Tavakkolian <skip.tavakkolian at gmail.com> wrote:
>> 
>> Spitballing here: in the new window do something like
>> 
>> cat /proc/123/ns | rc 
>> 
>> Or first massage the ns then generate an output for rc.
>> 
>>> On Fri, Oct 6, 2017, 4:34 PM Chris McGee <newton688 at gmail.com> wrote:
>>> Hi All,
>>> 
>>> When I'm using Unix, there's a workflow that I use for long running commands that I'm hoping to find the equivalent in the Plan 9 way of doing things.
>>> 
>>> I will occasionally run a command, realize that it will take a long time to complete. I don't want to kill it. I'll just Ctrl-Z and bg to put it into the background using the shell. It's almost as if I had run it with '&' in the first place. I can then run other commands in the same working directory, environment and shell history.
>>> 
>>> Is there an equivalent to this workflow in Plan 9?
>>> 
>>> I realize that the whole job control system dates back to old single session terminals, which isn't a problem with Rio where you can draw new windows at will. Initially I thought, that you just drag that window to a corner somewhere and let it complete. But, if I draw a new window it won't be in the same working directory, have the same environment and namespace. Maybe there is a way to create a window that inherits these from an existing process?
>>> 
>>> Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.9fans.net/private/9fans/attachments/20171023/7e8b3b4b/attachment.html>

[9fans] Backgrounding a task

[Giacomo Tesio]

Here it is: https://github.com/JehanneOS/jehanne/blob/master/sys/src/cmd/ns/clone.c

I'll leave the modifications to the plumber for another boring night...


Giacomo

2017-10-25 1:00 GMT+02:00 Chris McGee <newton688 at gmail.com>:
>
>> Think about multiple processes owned by multiple users running on a
>> cpu server.  Which processes should be allowed to join which
>> namespaces?
>>
>> Perhaps allowing only the hostowner to join namespaces for debugging
>> and administration purposes would be acceptable.
>
> Ah, right. What about only allowing a process to join another namespace if the user is the same?
>
>> This seems a contrived example.  Would you really spend HOURS working
>> on setting up a namespace by hand?  Surely you would instead be
>> working on a script that builds the namespace for you; make the
>> computer do the work.  Then when you mess up, you can modify the
>> script, create a new window, and try again.
>
> Yes, good point. That's the best way to do it. Also screen file can help add commands to the script post facto. The hours term isn't so much in the development of the environment but more to do with the amount of time you could be working in that shell instance. All the while not remembering all of the things you did to the namespace and environment in that time.
>
> Cheers,
> Chris
>

[9fans] Backgrounding a task

[Chris McGee]

> Think about multiple processes owned by multiple users running on a
> cpu server.  Which processes should be allowed to join which
> namespaces?
> 
> Perhaps allowing only the hostowner to join namespaces for debugging
> and administration purposes would be acceptable.

Ah, right. What about only allowing a process to join another namespace if the user is the same?

> This seems a contrived example.  Would you really spend HOURS working
> on setting up a namespace by hand?  Surely you would instead be
> working on a script that builds the namespace for you; make the
> computer do the work.  Then when you mess up, you can modify the
> script, create a new window, and try again.

Yes, good point. That's the best way to do it. Also screen file can help add commands to the script post facto. The hours term isn't so much in the development of the environment but more to do with the amount of time you could be working in that shell instance. All the while not remembering all of the things you did to the namespace and environment in that time.

Cheers,
Chris

[9fans] Backgrounding a task

[Giacomo Tesio]

Here it is: https://github.com/JehanneOS/jehanne/commit/320e6e6f35bfbc2e37dbd079c8d6a9124bd9ac6c

The simple test attached confirms that it works as expected:
https://github.com/JehanneOS/jehanne/blob/master/qa/kern/nsclone.c

Now it's just matter of modifying the plumber to use this facility and
add a ns/clone command that take a pid and a command to run so that

ns/clone 256 rc

would start a new rc in a copy of the name space of the process with pid 256.


Giacomo


2017-10-24 21:18 GMT+02:00 Giacomo Tesio <giacomo at tesio.it>:
> 2017-10-24 16:21 GMT+02:00 Alex Musolino <alex at musolino.id.au>:
>> Creating a child process is something that a process explicitly
>> controls and the RFNOTEG flag of rfork(2) allows a process to control
>> whether or not it shares its namespace with its children.  Allowing
>> other, unrelated processes to fiddle with your namespace is quite
>> different.
>>
>> Think about multiple processes owned by multiple users running on a
>> cpu server.  Which processes should be allowed to join which
>> namespaces?
>>
>> Perhaps allowing only the hostowner to join namespaces for debugging
>> and administration purposes would be acceptable.
>
> I like this idea a lot. I will give it a try in Jehanne.
>
> However I'm going to use a slightly different design: writing "clone"
> to /proc/$pid/ns will cause the current process to replace its own
> name space with a *copy* of that of $pid.
> If the owner of $pid is different from that of the current process or
> if $pid is not running on the same machine as the current process, the
> write will fail with an error.
>
> However any change to the name space after the clone does not impact
> the original process.
>
> As for the plumber, I will add a message that make the plumber clone
> the name space of a target process.
>
> This should address both use-cases without issues for the processes
> running in the original name space.
>
>
> Giacomo

[9fans] Backgrounding a task

[Giacomo Tesio]

2017-10-24 16:21 GMT+02:00 Alex Musolino <alex at musolino.id.au>:
> Creating a child process is something that a process explicitly
> controls and the RFNOTEG flag of rfork(2) allows a process to control
> whether or not it shares its namespace with its children.  Allowing
> other, unrelated processes to fiddle with your namespace is quite
> different.
>
> Think about multiple processes owned by multiple users running on a
> cpu server.  Which processes should be allowed to join which
> namespaces?
>
> Perhaps allowing only the hostowner to join namespaces for debugging
> and administration purposes would be acceptable.

I like this idea a lot. I will give it a try in Jehanne.

However I'm going to use a slightly different design: writing "clone"
to /proc/$pid/ns will cause the current process to replace its own
name space with a *copy* of that of $pid.
If the owner of $pid is different from that of the current process or
if $pid is not running on the same machine as the current process, the
write will fail with an error.

However any change to the name space after the clone does not impact
the original process.

As for the plumber, I will add a message that make the plumber clone
the name space of a target process.

This should address both use-cases without issues for the processes
running in the original name space.


Giacomo

Re: [9fans] Backgrounding a task

[Alex Musolino]

> The namespace join facility looks interesting. Do you have a patch
> somewhere for it?

I'll see what I can dig up though it wouldn't tbe erribly difficult to
reimplement.  You basically just need to modify the pgrp pointer of
the proc, adjusting ref counts as required.

>> Of course, a lot of the isolation that per-process namespaces give
>> you is suddenly undone by the introduction of this facility.
>
> I'm not sure if the lack of isolation is any different than what can
> be done with a child process that shares the namespace.  Is there a
> particular case that you are thinking?

Creating a child process is something that a process explicitly
controls and the RFNOTEG flag of rfork(2) allows a process to control
whether or not it shares its namespace with its children.  Allowing
other, unrelated processes to fiddle with your namespace is quite
different.

Think about multiple processes owned by multiple users running on a
cpu server.  Which processes should be allowed to join which
namespaces?

Perhaps allowing only the hostowner to join namespaces for debugging
and administration purposes would be acceptable.

>> At this point I'm not entirely convinced that it's worth the
>> trouble.
> 
> I think that it can be depending on how much time you have spent
> building up a namespace for a process.  Perhaps I have spent hours
> working on something slowly customizing the namespace mounting and
> binding things.  If I end up running a long running command that
> blocks and I want to work in parallel with it then I must remember
> everything that I have done and repeat in a new window.  It seems
> like something the computer should do for me or at least help me to do
> it.

This seems a contrived example.  Would you really spend HOURS working
on setting up a namespace by hand?  Surely you would instead be
working on a script that builds the namespace for you; make the
computer do the work.  Then when you mess up, you can modify the
script, create a new window, and try again.

One more thing to consider is the #σ device in 9front which seems to
address some of the problems that you might otherwise use nsjoin to
solve.

--
Cheers,
Alex Musolino

[9fans] Backgrounding a task

[Chris McGee]

The namespace join facility looks interesting. Do you have a patch somewhere for it?

> Of course, a lot of the isolation that per-process namespaces give you
> is suddenly undone by the introduction of this facility.  

I'm not sure if the lack of isolation is any different than what can be done with a child process that shares the namespace. Is there a particular case that you are thinking?

> At this
> point I'm not entirely convinced that it's worth the trouble.

I think that it can be depending on how much time you have spent building up a namespace for a process. Perhaps I have spent hours working on something slowly customizing the namespace mounting and binding things. If I end up running a long running command that blocks and I want to work in parallel with it then I must remember everything that I have done and repeat in a new window. It seems like something the computer should do for me or at least help me to do it.

Re: [9fans] Backgrounding a task

[Alex Musolino]

> So far, it looks like the closest equivalent is to draw a new window
> and inherit the namespace of the original one by reading the namespace
> from the proc.

The problem with /proc/$pid/ns is entries that can't be "replayed".
For example, the following command will not work:

	mount -b '#|/data' /mnt

Nor does it provide any real indication as to what exactly is hooked
up to the other end of the 9P pipe.

> I wonder if there could be a Rio gesture to draw a new window
> inheriting the namespace of the window I pick by clicking.

Rio can't access the namespaces of the processes running in its
windows.  The -m option to window(1) works by rforking a new process
with the RFNAMEG flag set and then doing the appropriate mounting and
binding to create and use a new rio window.

Some months ago I played with the idea of adding support for an
"nsjoin" command to the /proc/$pid/ctl file.  This allowed you to join
the namespace of another process a little something like this:

	term% ps | grep plumber
	alex            670    0:00   0:00      864K Pread    plumber
	alex            671    0:00   0:00      864K Rendez   plumber
	term% echo nsjoin 670 >/proc/$pid/ctl
	term%

Subsequent changes to the namespace would affect the plumber.  If you
wanted a copy, just do 'rfork n' after joining the namespace.

Of course, a lot of the isolation that per-process namespaces give you
is suddenly undone by the introduction of this facility.  At this
point I'm not entirely convinced that it's worth the trouble.

--
Cheers,
Alex Musolino