[9fans] fossil option -m

[9fans] fossil option -m

[arisawa]

Helo,

Plan 9 manual FOSSIL(4) says about fossil/fossil command option -m:
          -m      Allocate free-memory-percent percent of the avail-
                  able free RAM for buffers.  This overrides all other
                  memory sizing parameters, notably the -c option to
                  open.
How to give memory, for example 25%, to fossil in starting 9pcf or 9pccpuf kernel?
I think the value should be enabled to be given in plan9.ini, but I couldn't find in the manual PLAN9.INI(8).

Kenji Arisawa

Re: [9fans] fossil option -m

[erik quanstrom]

On Sat Aug 18 01:04:53 EDT 2012, arisawa@ar.aichi-u.ac.jp wrote:
> Helo,
>
> Plan 9 manual FOSSIL(4) says about fossil/fossil command option -m:
>           -m      Allocate free-memory-percent percent of the avail-
>                   able free RAM for buffers.  This overrides all other
>                   memory sizing parameters, notably the -c option to
>                   open.
> How to give memory, for example 25%, to fossil in starting 9pcf or 9pccpuf kernel?
> I think the value should be enabled to be given in plan9.ini, but I couldn't find in the manual PLAN9.INI(8).

if you're using standard plan 9 boot stuff, you'll need to hack boot/local.c
right under the /* start fossil */ comment, or fossil itself.  personally, i'd lean
toward adding an environment variable which could be put into plan9.ini
and be the equivalent of -m.

- erik

Re: [9fans] fossil option -m

[John Floren]

On Fri, Aug 17, 2012 at 10:30 PM, erik quanstrom <quanstro@quanstro.net> wrote:
> On Sat Aug 18 01:04:53 EDT 2012, arisawa@ar.aichi-u.ac.jp wrote:
>> Helo,
>>
>> Plan 9 manual FOSSIL(4) says about fossil/fossil command option -m:
>>           -m      Allocate free-memory-percent percent of the avail-
>>                   able free RAM for buffers.  This overrides all other
>>                   memory sizing parameters, notably the -c option to
>>                   open.
>> How to give memory, for example 25%, to fossil in starting 9pcf or 9pccpuf kernel?
>> I think the value should be enabled to be given in plan9.ini, but I couldn't find in the manual PLAN9.INI(8).
>
> if you're using standard plan 9 boot stuff, you'll need to hack boot/local.c
> right under the /* start fossil */ comment, or fossil itself.  personally, i'd lean
> toward adding an environment variable which could be put into plan9.ini
> and be the equivalent of -m.
>
> - erik
>

You can also use a boot.fs kernel, so you can define the startup in an
rc script. Very convenient, I used it to set up our
cpu/auth/fileserver to use a Coraid AoE device for Venti.

John

Re: [9fans] fossil option -m

[arisawa]

Thanks Erik and John,

It seems the option value -m is 20 by default.
/sys/src/9/boot/local.c in Plan9:
	run("/boot/fossil", "-m", "20", "-f", partition,
		"-c", "srv -A fboot", "-c", "srv -p fscons", nil);

I have looked contents in /boot/bootfs.paq of 9front.
I like the booting process: flexible, elegant and probably powerful.

Re: [9fans] fossil option -m

[Richard Miller]

> I have looked contents in /boot/bootfs.paq of 9front.
> I like the booting process: flexible, elegant and probably powerful.

Using a shell script to control booting has been around for a while:

--rwxrwxr-x M 12 sys sys 568 Feb  3  2005 /sys/lib/sysconfig/blast/boot

Re: [9fans] fossil option -m

[erik quanstrom]

> I have looked contents in /boot/bootfs.paq of 9front.
> I like the booting process: flexible, elegant and probably powerful.

it's another stage of bootloading which requires a number of things to be built-in
to the kernel.  in this sense, it's like initrd.

if we were to instead make the local file servers (fossil+venti, kfs, cwfs, etc.)
idiomatic enough in their arguments, it should be easy enough to drive them
with the same boot code, that needn't bloat the kernel, or require fancy
memory reclaim.

also,

> let's make it "fsmb".  that way we need not enumerate file servers.
> i've always found listing things in mb is easier to work out.

i take it back.  $fsname^mb is idiomatic enough to automate.
sorry for the noise.

- erik

Re: [9fans] fossil option -m

[cinap_lenrek]

term% cat /sys/src/9/port/bootfs.proto
$objtype
	bin
		9660srv
		awk
		aux
			kbdfs
		bind
		cat
		cfs
		cryptsetup
		dd
		disk
			fdisk
			kfs
			prep
		dossrv
		echo
		cwfs64x
		grep
		ip
			ipconfig
		ls
		mkdir
		mntgen
		mount
		mv
		hjfs
		rc
		rm
		sed
		sleep
		srv
		test
		unmount
		nusb
			usbd
			disk
			kb
		xd
rc
	lib
		rcmain
		local.rc 555 sys sys ../boot/local.rc
		net.rc 555 sys sys ../boot/net.rc
	bin
		fstype
		diskparts
		nusbrc 555 sys sys ../boot/nusbrc
		bootrc 555 sys sys ../boot/bootrc
tmp

the resulting bootfs.paq is 1.2 MB.

theres no reclaim. if init dies you get back to the bootrc
"bootargs" prompt.

for comparsion, drawing a 1024x768x32 rio window would be
like 3MB of precious kernel memory.

making fileservers idiomatic is a good idea. thats exactly
what i did with cwfs making it look like kfs. fossil/venti
is the only filesystem that requires special boot code.
(which does not matter in 9front as fossil was removed).

--
cinap

Re: [9fans] fossil option -m

[erik quanstrom]

> for comparsion, drawing a 1024x768x32 rio window would be
> like 3MB of precious kernel memory.

not an apples-to-apples comparison.  devdraw images are
reclaimable and compactable.  and does init paqfs exit or not?
that's another 10mb.  but memory use is really beside the point.

i think the rc booting travels a conceptually greater distance
between the kernel starting the first process and being fully
functional & configured kernel.  this is a cost that for me
outweighs the benefits.  and anyway, it's code either way.
the fact that it's rc and not c doesn't change the fact its built
into the kernel.

btw, is there any way to identify programs started from the init paq?
this is an additional 30 odd programs built into the kernel beyond
ipconfig and factotum.  (why aren't they part of the init paq?)
and i wonder if there couldn't be some confusion.

- erik

Re: [9fans] fossil option -m

[cinap_lenrek]

nope. paqfs doesnt exit. and its not wasting 10mb.

term% ps -a | grep paq
cinap_lenrek        5    0:00   0:00      188K Pread    paqfs

yes, bootrc basicly does the same stuff as the c based /boot/boot
did before. we found that the other 30 odd programs can come in handy.
especially when you try to boot a kernel on a odd machines where
theres no working disks (yet :-)).

the odd machines tend to be owned by people new to plan9 and
being short of patience for getting a pxe setup or attach
serial cables to the machine. but asking them to break into rc
and getting some readout of some ctl files works fine.

ipconfig *is* part of bootfs.paq.

term% ls -l /boot
--r-xr-xr-x / 0 cinap_lenrek cinap_lenrek   29306 Aug 16 10:57 /boot/boot
--r-xr-xr-x / 0 cinap_lenrek cinap_lenrek 1327174 Aug 16 10:57 /boot/bootfs.paq
--r-xr-xr-x / 0 cinap_lenrek cinap_lenrek  241755 Aug 16 10:57 /boot/factotum
--r-xr-xr-x / 0 cinap_lenrek cinap_lenrek   72165 Aug 16 10:57 /boot/paqfs

factotum is not build into the paq because libauth's auth_getkeys()
execs "/boot/factotum" to ask for passwords/keys. changing that would
break old binaries.

--
cinap

Re: [9fans] fossil option -m

[Richard Miller]

> yes, bootrc basicly does the same stuff as the c based /boot/boot
> did before. we found that the other 30 odd programs can come in handy.
> especially when you try to boot a kernel on a odd machines where
> theres no working disks (yet :-)).

Yes, I've found the same approach useful while developing an sd/mmc
card driver for the raspberry pi.  I started with this inital boot.rc:

#!/boot/rc -m /boot/rcmain

/boot/echo boot...
/boot/bind -c /boot /bin
bind '#p' /proc
bind '#d' /fd
bind -a '#P' /dev
bind -a '#t' /dev
bind -a '#S' /dev
echo -n rpi >/dev/hostowner
echo fdisk...
fdisk -p /dev/sdM0/data >/dev/sdM0/ctl
cat /dev/sdM0/ctl
dossrv -f/dev/sdM0/dos
mount -c /srv/dos /mnt
exec /boot/rc -m/boot/rcmain -i

And for sd driver testing there's no need for a paqfs - a handful
of useful programs can just be stuck into /boot directly:

bootdir
	boot.rc			boot
	/arm/bin/rc
	/rc/lib/rcmain
	/arm/bin/mount
	/arm/bin/unmount
	/arm/bin/bind
	/arm/bin/echo
	/arm/bin/cat
	/arm/bin/cmp
	/arm/bin/ls
	/arm/bin/ps
	/arm/bin/dd
	/arm/bin/xd
	/arm/bin/rm
	/arm/bin/mkdir
	/arm/bin/disk/fdisk
	/arm/bin/dossrv

But all this is now obsolete [see next message].

Re: [9fans] fossil option -m

[Richard Miller]

> personally, i'd lean
> toward adding an environment variable which could be put into plan9.ini
> and be the equivalent of -m.

How about
	*fossilpercent=NN
	*ventipercent=NN
	*kfspercent=NN
Any others?

I'll do a patch if nobody objects.  Needs a change to kfs to add '-m' option.

Re: [9fans] fossil option -m

[David du Colombier]

> How about
> 	*fossilpercent=NN
> 	*ventipercent=NN
> 	*kfspercent=NN
> Any others?
>
> I'll do a patch if nobody objects.  Needs a change to kfs to add '-m'
> option.


Yes. I think it's a good idea.
But maybe without the leading '*', since they
aren't kernel parameters.


--
David du Colombier

Re: [9fans] fossil option -m

[Richard Miller]

> But maybe without the leading '*', since they
> aren't kernel parameters.

Probably right, although it's never been clear to me why some things
have '*' and some don't.  For example etherN=XXXXX is used only by
the kernel, but it still gets passed in the normal '#e' environment
to processes.

On further thought, what about generalising a bit more: plan9.ini(8)
already has
        factotumopts=options
          Causes boot(8) to start factotum with the given options,

If we allowed XXXXopts=options, where XXXX is the name of any command
in /boot, my earlier suggestion would be just a particular case of this
with
	fossilopts=-m NN
	ventiopts=-m NN

Is this a simplification or a complication?

I'd actually prefer 'fossil-opts=-M NN' , but we already have the
precedent of factotumopts.

Re: [9fans] fossil option -m

[erik quanstrom]

> Probably right, although it's never been clear to me why some things
> have '*' and some don't.  For example etherN=XXXXX is used only by
> the kernel, but it still gets passed in the normal '#e' environment
> to processes.

i'd say that it's a bit of parsimony that we didn't get right.
quite similar in its dog-always-on-wrong-side-of-door nature
to bourne shell's "export".

> How about
> 	*fossilpercent=NN
> 	*ventipercent=NN
> 	*kfspercent=NN

let's make it "fsmb".  that way we need not enumerate file servers.
i've always found listing things in mb is easier to work out.

maybe i'm applying bad experiences with percentage-based partitions,
and *kernelpercent incorrectly.

(if i have *kernelpercent=9 or 10 and 3000mb of memory, how much
kernel memory do i get and why?  how much memory would i
get if i specified nothing?  i think the answer is 73.8125mb any
way for a cpu server, and 256mb for a terminal, using the distribution's
algorithms.)

- erik

Re: [9fans] fossil option -m

[David du Colombier]

> let's make it "fsmb".  that way we need not enumerate file servers.

Just wondering. How do you distinguish between Fossil
and Venti, when they are running on the same machine?
They have very different memory requirement.


--
David du Colombier

Re: [9fans] fossil option -m

[erik quanstrom]

On Sun Aug 19 05:06:03 EDT 2012, 0intro@gmail.com wrote:
> > let's make it "fsmb".  that way we need not enumerate file servers.
>
> Just wondering. How do you distinguish between Fossil
> and Venti, when they are running on the same machine?
> They have very different memory requirement.

i retracted this suggestion yesterday.  not only for the reason you
point out, but also because if we generalize the kfs, fossil code
to bootlocalfses, $($fsname)mb seems the obvious choice.

- erik

Re: [9fans] fossil option -m

[Bakul Shah]

On Aug 19, 2012, at 8:04 AM, erik quanstrom <quanstro@quanstro.net> wrote:

> On Sun Aug 19 05:06:03 EDT 2012, 0intro@gmail.com wrote:
>>> let's make it "fsmb".  that way we need not enumerate file servers.
>>
>> Just wondering. How do you distinguish between Fossil
>> and Venti, when they are running on the same machine?
>> They have very different memory requirement.
>
> i retracted this suggestion yesterday.  not only for the reason you
> point out, but also because if we generalize the kfs, fossil code
> to bootlocalfses, $($fsname)mb seems the obvious choice.

What's mb? Milli bits?

Re: [9fans] fossil option -m

[cinap_lenrek]

millerbytes :)

--
cinap

Re: [9fans] fossil option -m

[Richard Miller]

> millerbytes :)

No, he only barks occasionally.

Re: [9fans] fossil option -m

[erik quanstrom]

>
> What's mb? Milli bits?

micro baud.

- erik

Re: [9fans] fossil option -m

[erik quanstrom]

> You can also use a boot.fs kernel, so you can define the startup in an
> rc script. Very convenient, I used it to set up our
> cpu/auth/fileserver to use a Coraid AoE device for Venti.

you can do that with the usual scripts with 9atom.  it requires setting
"aoeif=<list>" and "aoedev=<list>".  see the manual pages.

- erik

Re: [9fans] fossil option -m

[erik quanstrom]

> > You can also use a boot.fs kernel, so you can define the startup in an
> > rc script. Very convenient, I used it to set up our
> > cpu/auth/fileserver to use a Coraid AoE device for Venti.
>
> you can do that with the usual scripts with 9atom.  it requires setting
> "aoeif=<list>" and "aoedev=<list>".  see the manual pages.

s:scripts:boot code:

- erik