[9fans] hacking issue: memory resizing

[9fans] hacking issue: memory resizing

[Ronald G Minnich]

anybody willing to think about whether dynamic memory resizing would be 
doable in a Plan 9 kernel? This is both get bigger and get smaller.

newer machines have hot-plug memory, as do virtual machines.

ron

Re: [9fans] hacking issue: memory resizing

[Francisco Ballesteros]

Getting bigger should be easy, probably just populating
the allocators, perhaps using extra rmaps for the
extra mem and doing something similar with the allocators on top.

Getting smaller may require assigning pages  first from permanent
rmaps, to avoid relocating vm to the permanent ones.
But, is this part really needed?


On 9/7/05, Ronald G Minnich <rminnich@lanl.gov> wrote:
> anybody willing to think about whether dynamic memory resizing would be
> doable in a Plan 9 kernel? This is both get bigger and get smaller.
> 
> newer machines have hot-plug memory, as do virtual machines.
> 
> ron
>

Re: [9fans] hacking issue: memory resizing

[Ronald G Minnich]

Francisco Ballesteros wrote:

> Getting smaller may require assigning pages  first from permanent
> rmaps, to avoid relocating vm to the permanent ones.
> But, is this part really needed?

yes, getting smaller is needed as well.

ron

Re: [9fans] hacking issue: memory resizing

[Dave Lukes]

Ron,
'Scuse my ignorance,
but presumably you're thinking about some kind of tepid-swap
(OS migrates processes off the board and tells you when you can unplug it)
arrangement?

    Dave.

P.S. I haven't disturbed the demons in a while by mentioning Project 
Evil, so ...
<troll>Is there an open standard for hot-swap memory cards?</troll>

Ronald G Minnich wrote:

> Francisco Ballesteros wrote:
>
>> Getting smaller may require assigning pages  first from permanent
>> rmaps, to avoid relocating vm to the permanent ones.
>> But, is this part really needed?
>
>
> yes, getting smaller is needed as well.
>
> ron

Re: [9fans] hacking issue: memory resizing

[Ronald G Minnich]

Dave Lukes wrote:
> Ron,
> 'Scuse my ignorance,
> but presumably you're thinking about some kind of tepid-swap
> (OS migrates processes off the board and tells you when you can unplug it)
> arrangement?
yes, or more aggressive: OS remaps virtual memory and then removes that 
physical memory from the maps. I mean, if the DIMM with your kernel is 
bad, it's all over; but if it is some other DIMM, well, maybe it won't 
be too hard. Or maybe it will.

Needed for hot plug memory and xen.

Also, the memory rmap needs rewriting anyway; here's a chance to get 
this done.

Also, there's not enough kernel discussion on this list (as per the town 
meeting log). So I am trying to incite some.

ron

[9fans] hacking issue: memory resizing

[erik quanstrom]

this is a pretty interesting problem. i'd bet that nobody
envisioned doing things like this back when this code was 
written.

for a toehold on the problem, it looks like the cardbus
code (/sys/boot/pc/devpccard.c:828) and the functions
upamalloc() and upafree() in memory.c allow some kernel-level 
dynamic memory. (although upafree() doesn't do anything.)

am i wrong in this? what can i do with memory on a pccard?
how scary is it to eject a pccard with configured memory?
(i guess those are the same question.)

erik

Re: [9fans] hacking issue: memory resizing

[Ronald G Minnich]

erik quanstrom wrote:

> for a toehold on the problem, it looks like the cardbus
> code (/sys/boot/pc/devpccard.c:828) and the functions
> upamalloc() and upafree() in memory.c allow some kernel-level 
> dynamic memory. (although upafree() doesn't do anything.)
> 

My impression in general is that the device stuff in Plan 9 is not 
terribly dynamic -- not surprising, given the time in which the code was 
  written.

It would be cool if devices could be as dynamic as file systems.

Not an issue in my world, but it's getting to be very important in the 
rest of the world, where hotplug is the order of the day, even including 
hotplug CPUs and memory at this point. PCMCIA -- that's old news :-)

Anyway, I hope some smart people out there will take this as a challenge 
in the kernel space and start thinking about it. Plan 9 is not as 
advanced, as some other OSes, in the area of dynamic device management.

ron

Re: [9fans] hacking issue: memory resizing

[jmk]

Ron's reply sums it up nicely and caught me in mid-composition
in a similar vein, although I was using a lot of ALL CAPS to make
some points.

--jim

On Fri Sep  9 11:21:02 EDT 2005, rminnich@lanl.gov wrote:
> erik quanstrom wrote:
> 
> > for a toehold on the problem, it looks like the cardbus
> > code (/sys/boot/pc/devpccard.c:828) and the functions
> > upamalloc() and upafree() in memory.c allow some kernel-level 
> > dynamic memory. (although upafree() doesn't do anything.)
> > 
> 
> My impression in general is that the device stuff in Plan 9 is not 
> terribly dynamic -- not surprising, given the time in which the code was 
>   written.
> 
> It would be cool if devices could be as dynamic as file systems.
> 
> Not an issue in my world, but it's getting to be very important in the 
> rest of the world, where hotplug is the order of the day, even including 
> hotplug CPUs and memory at this point. PCMCIA -- that's old news :-)
> 
> Anyway, I hope some smart people out there will take this as a challenge 
> in the kernel space and start thinking about it. Plan 9 is not as 
> advanced, as some other OSes, in the area of dynamic device management.
> 
> ron

[9fans] hacking issue: memory resizing

[erik quanstrom]

i guess what i was getting at was: can some of the dynamic bits
like pccard and usb be flushed out and generalized to support
general hotplugging, or do we require a new framework?

erik

Re: [9fans] hacking issue: memory resizing

[jmk]

On Fri Sep  9 14:04:50 EDT 2005, quanstro@quanstro.net wrote:
> i guess what i was getting at was: can some of the dynamic bits
> like pccard and usb be flushed out and generalized to support
> general hotplugging, or do we require a new framework?
> 
> erik

it requires more drastic surgery than hacking on the
(well-intentioned) hacks. but that's just my opinion.

--jim

Re: [9fans] hacking issue: memory resizing

[Eric Van Hensbergen]

On 9/9/05, erik quanstrom <quanstro@quanstro.net> wrote:
> i guess what i was getting at was: can some of the dynamic bits
> like pccard and usb be flushed out and generalized to support
> general hotplugging, or do we require a new framework?
> 

IMHO the right answer is to take a step back and take some time to
think about the "right" way to support hot-pluggable hardware versus
"flushing things out".   I'm not saying that we'll need a whole new
framework, but I'd rather design a proper interface versus "patch"
deficiencies.

Some of the Plan B stuff may be pointers in the right direction since
they deal with file servers coming and going on a regular basis.  Its
just in the hot-plug case, the file server which is coming and going
is a physical device.

      -eric

Re: [9fans] hacking issue: memory resizing

[Ronald G Minnich]

Eric Van Hensbergen wrote:

> Some of the Plan B stuff may be pointers in the right direction since
> they deal with file servers coming and going on a regular basis.  Its
> just in the hot-plug case, the file server which is coming and going
> is a physical device.

also, there are pointers to remangle since it really is in the kernel. 
It would be nice if devices could be as easily popped into and out of 
existence as servers are now.

ron

Re: [9fans] hacking issue: memory resizing

[Tim Newsham]

On Fri, 9 Sep 2005, Ronald G Minnich wrote:
> My impression in general is that the device stuff in Plan 9 is not terribly 
> dynamic -- not surprising, given the time in which the code was  written.

This brings up something I've been wondering about lately.  Plan 9 was 
written for a particular environment (file server, cpu servers, many 
terminals).  It works well there but obviously lacks some features that a 
more "modern" environment might desire.  You hear it sometimes on this 
mailing list "I want to do so and so." "well, so and so doesn't really 
work well, remember the typical setup is a fileserver, cpu servers and 
terminals."

So the big question here is -- what would a more modern environment be, 
and what could plan9 (or a system like it) do to fit in better. Plan B 
looks like its addressing this issue in one sense (although perhaps not in 
the type of environment I would typically use computers in today).

I think this "hot swappable devices and services" thread is one instance 
of grappling with this larger issue.  What are the other big problems out 
there and what environment do people think is reasonable?  Seems like 
everyone has several fast machines, many of them mobile, often being 
disconnected from one network and attached to another (or used 
disconnected).  Often people want a filesystem and cpu server on their 
portable platform.  There are lots of small ultra-mobile platforms. Energy 
use is more a concern than it was before.

Tim Newsham
http://www.lava.net/~newsham/

Re: [9fans] hacking issue: memory resizing

[Francisco Ballesteros]

Hmm. It seems to me that the platform you are talking about is
exactly the kind of environment that I think about. What´s the
difference? I must be missing something.

 You have just files and services (including devices). You want your
laptop (or PDA) to work standalone, but you still want the stateless terminal
that Plan 9 provides (so you have to administer just one or a few machines).
To me, it seems  that replicating (sort of caching, like in coda) your FS into
your standalone machines is the easy way to do that. The only inconvenience
that I see in Plan 9 is that when the environment is too dynamic you have
to do many things by hand; but that can be fixed. Regarding aliens
(Linux, etc.),
if you can combine storage and services between all your systems, you end
up thinking that you have a single system.

> So the big question here is -- what would a more modern environment be,
> and what could plan9 (or a system like it) do to fit in better. Plan B
> looks like its addressing this issue in one sense (although perhaps not in
> the type of environment I would typically use computers in today).

> there and what environment do people think is reasonable?  Seems like
> everyone has several fast machines, many of them mobile, often being
> disconnected from one network and attached to another (or used
> disconnected).  Often people want a filesystem and cpu server on their
> portable platform.  There are lots of small ultra-mobile platforms. Energy
> use is more a concern than it was before.
> 
> Tim Newsham
> http://www.lava.net/~newsham/
>

Re: [9fans] hacking issue: memory resizing

[Brian L. Stuart]

In message <600308d605090913461de1965@mail.gmail.com>, Francisco Ballesteros wr
ites:
> You have just files and services (including devices). You want your
>laptop (or PDA) to work standalone, but you still want the stateless terminal
>that Plan 9 provides (so you have to administer just one or a few machines).
>To me, it seems  that replicating (sort of caching, like in coda) your FS into
>your standalone machines is the easy way to do that. The only inconvenience

As it turns out, I've been playing around with some ideas in
that direction.  I've got a rough prototype in Plan 9 and a
slightly less rough one in Linux using FUSE.  I didn't like
coda's special server code, so I took the approach of keeping
two trees synchronized.  That way you can connect to the file
server you're caching with 9P or NFS or SMB or...  It's been
proving handy allowing my laptop to cache stuff from work
and stuff from home at the same time.  Hopefully, I'll get
company permission to release the code soon.

BLS

Re: [9fans] hacking issue: memory resizing

[Francisco Ballesteros]

Are you using user-level programs called to synchronize?
We are doing the same. Look for repl in the Plan B man page.


On 9/10/05, Brian L. Stuart <blstuart@bellsouth.net> wrote:
> In message <600308d605090913461de1965@mail.gmail.com>, Francisco Ballesteros wr
> ites:
> > You have just files and services (including devices). You want your
> >laptop (or PDA) to work standalone, but you still want the stateless terminal
> >that Plan 9 provides (so you have to administer just one or a few machines).
> >To me, it seems  that replicating (sort of caching, like in coda) your FS into
> >your standalone machines is the easy way to do that. The only inconvenience
> 
> As it turns out, I've been playing around with some ideas in
> that direction.  I've got a rough prototype in Plan 9 and a
> slightly less rough one in Linux using FUSE.  I didn't like
> coda's special server code, so I took the approach of keeping
> two trees synchronized.  That way you can connect to the file
> server you're caching with 9P or NFS or SMB or...  It's been
> proving handy allowing my laptop to cache stuff from work
> and stuff from home at the same time.  Hopefully, I'll get
> company permission to release the code soon.
> 
> BLS
>

Re: [9fans] hacking issue: memory resizing

[Brian L. Stuart]

In message <600308d605091014412b6d1b71@mail.gmail.com>, Francisco Ballesteros w
rites:
>Are you using user-level programs called to synchronize?
>We are doing the same. Look for repl in the Plan B man page.

Yes.  It's a user-level file server in the spirit of cfs(4).
There are a few major differences, the first being the
persistance of the cache and a record of updates while
disconnected that is played back on reconnection.  The
other two differences were originally short-cuts to
simplify my life when working on what was originally
just an experiment.  One of them was to use files in
the local file system to store cache data and metadata
rather than using a raw partition.  The second was to
use ordinary system calls rather than pass the 9P requests
on to the file server directly.  This last one helps to
make it more flexible with respect to the server and
translated more easily to a Linux implementation.

To be honest, I haven't taken a look at the Plan B
design yet.  My only excuse is that it's on my to-do
list.

BLS