[9fans] segfree() - more details?

[9fans] segfree() - more details?

[lucio]

The rather tantalising:

          Segfree tells the system that it may free any physical mem-
          ory within the span [va, va+len), but leaves that portion of
          the process's address space valid.  The system will not free
          any memory outside that span, and may not free all or even
          any of the specified memory.  If free'd memory is later ref-
          erenced, it will be initialized as appropriate for the seg-
          ment type.  For example data and text segments will be read
          from the executable file, and bss segments will be filled
          with zero bytes.

in segattach(2) suggests that there is some mechanism to associate
disk file portions with memory segments (that being what Unix's MMAP
does, roughly), but falls short of explaining how this association is
established.  I presume there is documentation for this elsewhere that
ought to be mentioned in the above.

Also, I'm too dense to grasp the exact intent of segfree(2) as implied
in the above description, but I'm sure once I understand it I'll be
able to make use of it in my efforts to make Plan 9 understand ELF
directly.  So if anyone can point me to the right place, I'd greatly
appreciate it.

I have Nemo's "commentary" that has proved invaluable to my
understanding of the Plan 9 kernel, but I always have to find it by
inspection because its filename is not very self-explanatory.  I guess
I ought to change that, but I'm the conservative type: does anyone
remember its name?

++L

Re: [9fans] segfree() - more details?

[Charles Forsyth]

>in segattach(2) suggests that there is some mechanism to associate
>disk file portions with memory segments (that being what Unix's MMAP
>does, roughly),

not really: it will read initial text and data from an image,
but that's it.  apparently if you segfree your data space it
will reinitialise it from the image, effectively resetting it,
but i'd be surprised if anything uses that.

Re: [9fans] segfree() - more details?

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 3725 bytes --]

I think russ has added this functionality to his kernel. The sourcecode of his
linuxemu had commented out lines that used special segname with a filename
to map elf-files into memory.

/n/sources/contrib/rsc/linuxemu/linuxemu.c:

/*
 * mmap, if it were handled by the kernel
 *
void*
_mmap(char *name)
{
	void *v;
	Dir *d;
	char *buf;
	vlong len;

	if((d = dirstat(name)) == nil || d->length == 0){
		free(d);
		return nil;
	}
	len = d->length;
	free(d);
	buf = malloc(strlen(name)+10);
	if(buf == nil)
		return nil;

	sprint(buf, "file!%s", name);
	v = (void*)segattach(0, buf, nil, len);
	print("v=%lx file=%s len=%ld\n", v, name, len);
	free(buf);
	if(v == (void*)-1)
		return nil;
	return v;
}

void*
_mmapfd(int fd)
{
	void *v;
	Dir d;
	char buf[30];

	if(dirfstat(fd, &d) < 0 || d.length == 0)
		return nil;
	snprint(buf, sizeof buf, "file!/fd/%d", fd);
	v = (void*)segattach(0, buf, nil, 4096);
	if(v == (void*)-1)
		return nil;
	return v;
}
*/

But you cant use it without increasing the maximum segment count per
process to work with dynamicly linked linux programs.

/*
 *  process memory segments - NSEG always last !
 */
enum
{
	SSEG, TSEG, DSEG, BSEG, ESEG, LSEG, SEG1, SEG2, SEG3, SEG4, NSEG
};

SSEG, TSEG, DSEG and BSEG are used already. That gets you
at maximum 6 more slots. Also, having a look at a typical linux
memory map, data and mmaped files are not separate continously
mapped but alternating.

term% acid -l linuxemu.acid 264
/proc/264/text:386 plan 9 executable

/sys/lib/acid/port
/sys/lib/acid/386
acid: umem(Current())
1 0x08048000-0x080dc000	0x00000000@/bin/bash
1 0x080dc000-0x080e2000	0x00093000@/bin/bash
1 0x080e2000-0x080e6000	*ANON*
1 0x080e6000-0x080fc000	0x00000000@/lib/ld-2.3.2.so
1 0x080fc000-0x080fd000	0x00015000@/lib/ld-2.3.2.so
1 0x080fd000-0x080fe000	*ANON*
1 0x08103000-0x08139000	0x00000000@/lib/libncurses.so.5.4
1 0x08139000-0x08142000	0x00035000@/lib/libncurses.so.5.4
1 0x08142000-0x08144000	0x00000000@/lib/libdl-2.3.2.so
1 0x08144000-0x08145000	0x00002000@/lib/libdl-2.3.2.so
1 0x08145000-0x0826d000	0x00000000@/lib/libc-2.3.2.so
1 0x0826d000-0x08275000	0x00127000@/lib/libc-2.3.2.so
1 0x08275000-0x08278000	*ANON*
1 0x08278000-0x0827d000	*ANON*
1 0x0827d000-0x08284000	0x00000000@/lib/libnss_compat-2.3.2.so
1 0x08284000-0x08285000	0x00006000@/lib/libnss_compat-2.3.2.so
1 0x08285000-0x08297000	0x00000000@/lib/libnsl-2.3.2.so
1 0x08297000-0x08298000	0x00011000@/lib/libnsl-2.3.2.so
1 0x08298000-0x0829a000	*ANON*
1 0x0829a000-0x082a2000	0x00000000@/lib/libnss_nis-2.3.2.so
1 0x082a2000-0x082a3000	0x00007000@/lib/libnss_nis-2.3.2.so
1 0x082a3000-0x082ab000	0x00000000@/lib/libnss_files-2.3.2.so
1 0x082ab000-0x082ac000	0x00008000@/lib/libnss_files-2.3.2.so
1 0x082ac000-0x082c1000	*ANON*
2 0xdefed000-0xdeffe000	*ANON*

So you would need separate memory segments in between too.

Linuxemu uses a fixed count of segments to emulate the linux address
space and just reads in mapped files at the right location.

term% cat /proc/264/segment
Stack     defff000 dffff000    1
Text   R  00001000 00020000    2
Shared    00020000 00029000    2
Shared    00029000 00054000    2
Bss       ceffe000 deffe000    1
Bss       08048000 082c1000    1
Shared    06000000 06001000    1

This works because here is just one fixed address mapping for the
first binary (that is where i create the segment) and all further
fixed mapping are relative to some other dynamic mapping so i just
expand the segment carefully.  The area where glibc maps stacks is
fixed size and works because plan9 overcommits memory for segments.
(you cant grow segments down)

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 3761 bytes --]

From: lucio@proxima.alt.za
To: 9fans@9fans.net
Subject: [9fans] segfree() - more details?
Date: Thu, 2 Apr 2009 11:13:18 +0200
Message-ID: <6c529a9785a1131ce815f5fa79593321@proxima.alt.za>

The rather tantalising:

          Segfree tells the system that it may free any physical mem-
          ory within the span [va, va+len), but leaves that portion of
          the process's address space valid.  The system will not free
          any memory outside that span, and may not free all or even
          any of the specified memory.  If free'd memory is later ref-
          erenced, it will be initialized as appropriate for the seg-
          ment type.  For example data and text segments will be read
          from the executable file, and bss segments will be filled
          with zero bytes.

in segattach(2) suggests that there is some mechanism to associate
disk file portions with memory segments (that being what Unix's MMAP
does, roughly), but falls short of explaining how this association is
established.  I presume there is documentation for this elsewhere that
ought to be mentioned in the above.

Also, I'm too dense to grasp the exact intent of segfree(2) as implied
in the above description, but I'm sure once I understand it I'll be
able to make use of it in my efforts to make Plan 9 understand ELF
directly.  So if anyone can point me to the right place, I'd greatly
appreciate it.

I have Nemo's "commentary" that has proved invaluable to my
understanding of the Plan 9 kernel, but I always have to find it by
inspection because its filename is not very self-explanatory.  I guess
I ought to change that, but I'm the conservative type: does anyone
remember its name?

++L

Re: [9fans] segfree() - more details?

[lucio]

> see port/fault.c to see what happens on a page fault in the text segment.

Yep, I remember trying to understand that part from a rather
superficial study of the sources.  Nemo's commentary will no doubt
prove itself invaluable once again, as soon as I track it down (time
to print a copy).  And thanks to Cinap for his comments, as usual very
much to the point.

Why I find all this so confusing after long years in the business
beats me.  Must be old age.  I guess I should be grateful it's not
Linux I am trying to make sense of.

++L

Re: [9fans] segfree() - more details?

[ron minnich]

On Thu, Apr 2, 2009 at 4:44 AM, Charles Forsyth <forsyth@terzarima.net> wrote:
>>OK, I believe you, but you're not telling me _how_ the "initial text
>>and data from an image" is specified.  And that is really the bit I
>>want to know about :-)
>
> it's set by exec.

see port/fault.c to see what happens on a page fault in the text segment.

ron

Re: [9fans] segfree() - more details?

[Charles Forsyth]

>OK, I believe you, but you're not telling me _how_ the "initial text
>and data from an image" is specified.  And that is really the bit I
>want to know about :-)

it's set by exec.

Re: [9fans] segfree() - more details?

[lucio]

>>in segattach(2) suggests that there is some mechanism to associate
>>disk file portions with memory segments (that being what Unix's MMAP
>>does, roughly),
>
> not really: it will read initial text and data from an image,
> but that's it.  apparently if you segfree your data space it
> will reinitialise it from the image, effectively resetting it,
> but i'd be surprised if anything uses that.

OK, I believe you, but you're not telling me _how_ the "initial text
and data from an image" is specified.  And that is really the bit I
want to know about :-)

++L