[9fans] Data segment vs BSS

[9fans] Data segment vs BSS

[Noah Evans]

Hey, quick 2l question. Is there any way to keep constant values in  
ROM? If possible I'd like to keep the data section entirely in ROM  
and only keep the BSS section in RAM. However, both sections seem to  
be inextricably linked.

What's the proper way to deal with this?

Noah

Re: [9fans] Data segment vs BSS

[jmk]

The normal Plan 9 binary format assumes the BSS starts immediately
after the data, there is no place in the header to specify otherwise.
So, you want to use a different header format, e.g. ELF or one of
the machine-specific boot formats, see /sys/src/cmd/?l/asm.c for
examples. There may be one there already that suits, or you can easily
add another for your particular environment.

But how do you separate out the constant data?

On Mon Jan 30 23:36:55 EST 2006, noah.evans@gmail.com wrote:
> Hey, quick 2l question. Is there any way to keep constant values in  
> ROM? If possible I'd like to keep the data section entirely in ROM  
> and only keep the BSS section in RAM. However, both sections seem to  
> be inextricably linked.
> 
> What's the proper way to deal with this?
> 
> Noah

Re: [9fans] Data segment vs BSS

[Noah Evans]

Hmmmmm... it will require a lot of work understanding the linker, but
if I end up running out of memory, I might see if I can get the const
keyword to throw  those variables in the text section and avoid the
problem altogether.

What do you think?

Noah



On 1/31/06, jmk@plan9.bell-labs.com <jmk@plan9.bell-labs.com> wrote:
> The normal Plan 9 binary format assumes the BSS starts immediately
> after the data, there is no place in the header to specify otherwise.
> So, you want to use a different header format, e.g. ELF or one of
> the machine-specific boot formats, see /sys/src/cmd/?l/asm.c for
> examples. There may be one there already that suits, or you can easily
> add another for your particular environment.
>
> But how do you separate out the constant data?
>
> On Mon Jan 30 23:36:55 EST 2006, noah.evans@gmail.com wrote:
> > Hey, quick 2l question. Is there any way to keep constant values in
> > ROM? If possible I'd like to keep the data section entirely in ROM
> > and only keep the BSS section in RAM. However, both sections seem to
> > be inextricably linked.
> >
> > What's the proper way to deal with this?
> >
> > Noah
>

Re: [9fans] Data segment vs BSS

[jmk]

The part of the loader for this is not dificult to understand.
Grep for HEADTYPE in the source, only asm.c and obj.c will likely
come up, and in each only one function: obj.c:main() sets up the
variables that say where the virtual addresses of the sections are
and asm.c:asmb() seeks around using those values for the different
header types and writes the stuff out.

Usually the data section has 2 types of data in it, constant data
and data that can be modiied by the programme, how are you planning
to separate those out?

Many years ago when I did the boot ROM for the Hobbit boards I just
copied the data to real memory so there was no issue like this. How
big is your data segment, big enough to worry about using up RAM?

On Tue Jan 31 00:36:11 EST 2006, noah.evans@gmail.com wrote:
> Hmmmmm... it will require a lot of work understanding the linker, but
> if I end up running out of memory, I might see if I can get the const
> keyword to throw  those variables in the text section and avoid the
> problem altogether.
> 
> What do you think?
> 
> Noah
> 
> 
> 
> On 1/31/06, jmk@plan9.bell-labs.com <jmk@plan9.bell-labs.com> wrote:
> > The normal Plan 9 binary format assumes the BSS starts immediately
> > after the data, there is no place in the header to specify otherwise.
> > So, you want to use a different header format, e.g. ELF or one of
> > the machine-specific boot formats, see /sys/src/cmd/?l/asm.c for
> > examples. There may be one there already that suits, or you can easily
> > add another for your particular environment.
> >
> > But how do you separate out the constant data?
> >
> > On Mon Jan 30 23:36:55 EST 2006, noah.evans@gmail.com wrote:
> > > Hey, quick 2l question. Is there any way to keep constant values in
> > > ROM? If possible I'd like to keep the data section entirely in ROM
> > > and only keep the BSS section in RAM. However, both sections seem to
> > > be inextricably linked.
> > >
> > > What's the proper way to deal with this?
> > >
> > > Noah
> >

Re: [9fans] Data segment vs BSS

[Ronald G Minnich]

jmk@plan9.bell-labs.com wrote:

> Many years ago when I did the boot ROM for the Hobbit boards I just
> copied the data to real memory so there was no issue like this. How
> big is your data segment, big enough to worry about using up RAM?

there is one other option. You can compress the data segment, and 
uncompress on loading. This will usually save a ton of space -- it's 
what we do on linuxbios, and the savings (surprise for me) really adds up.

ron

Re: [9fans] Data segment vs BSS

[Noah Evans]

Maybe I should be a little more specific. I have a machine(a game boy  
advance) that only has 256k of memory but has a 32mb for rom.  
Basically I want  to save every bit of RAM I can. Would your solution  
help?

Noah

On Jan 31, 2006, at 2:54 PM, Ronald G Minnich wrote:

> jmk@plan9.bell-labs.com wrote:
>
>> Many years ago when I did the boot ROM for the Hobbit boards I just
>> copied the data to real memory so there was no issue like this. How
>> big is your data segment, big enough to worry about using up RAM?
>
> there is one other option. You can compress the data segment, and  
> uncompress on loading. This will usually save a ton of space --  
> it's what we do on linuxbios, and the savings (surprise for me)  
> really adds up.
>
> ron

Re: [9fans] Data segment vs BSS

[Noah Evans]

Depending on how things end up I may get in there and start mucking  
around. Your advice is greatly appreciated.

The data section is pretty small now, something like 64 bytes(this is  
entirely a guesstimate based on the value of the pointer to edata, is  
bdata defined on 5l? I'm getting an undefined error if I try to use  
it), but I'm assuming as I get more and more of Inferno functioning  
this will get bigger.

What I'm thinking of doing regarding differentiating the immutable  
data is to use to symbol const to mark immutable data. Anything  
marked const would go in the text section, anything else would go to  
the data section.

But right now all of this is just speculation. I need to get the lcd  
before anything else.

Noah

On Jan 31, 2006, at 2:48 PM, jmk@plan9.bell-labs.com wrote:

> s is not dificult to understand.
> Grep for HEADTYPE in the source, only asm.c and obj.c will likely
> come up, and in each only one function: obj.c:main() sets up the
> variables that say where the virtual addresses of the sections are
> and asm.c:asmb() seeks around using those values for the different
> header types and writes the stuff out.
>
> Usually the data section has 2 types of data in it, constant data
> and data that can be modiied by the programme, how are you planning
> to separate those out?
>

Re: [9fans] Data segment vs BSS

[C H Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

that's the method i've used in a few bootstraps.
a few left the kernel text in ROM uncompressed,
to allow executing from there (usually with a significant speed penalty),
but the data was decompressed into RAM.

in the z[acl] suite for the 8-bit Atmel, we managed a small trick
to reduce the RAM bill, because the SRAM was much smaller than flash.
things such as radio parameters and audio processing tables were
fairly big, but constant, and we needed RAM space for the FFT variables.
the startup copies the initial contents of the writable data across
and clears the bss.  read-only values remain in program memory (flash).
the compiler would put any unmodified data declared static into program memory.
const was not needed: the compiler did the analysis.
it was harder than it sounds because the program memory requires different
instructions to read it, a little worse than plain `separate I&D addressing'.
the linker had a little more work to do, but
more important, the compiler had to ensure that no data was marked for program
memory until it had checked that it was never accessed indirectly (because
then the called function would use the wrong instruction).  again, const
would not have been accurate enough.  unfortunately, the
technique couldn't be used for strings, on that processor, because
functions such as str*, mem*, print, etc. are used with sources that can
be either readonly or writable.  on the ARM, though, the compiler could
put strings in the text segment easily enough; it might do that already,
but i can't remember and i'll need to check.

[-- Attachment #2: Type: message/rfc822, Size: 3218 bytes --]

From: Ronald G Minnich <rminnich@lanl.gov>
To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu>
Subject: Re: [9fans] Data segment vs BSS
Date: Mon, 30 Jan 2006 22:54:15 -0700
Message-ID: <43DEFB87.7040806@lanl.gov>

jmk@plan9.bell-labs.com wrote:

> Many years ago when I did the boot ROM for the Hobbit boards I just
> copied the data to real memory so there was no issue like this. How
> big is your data segment, big enough to worry about using up RAM?

there is one other option. You can compress the data segment, and 
uncompress on loading. This will usually save a ton of space -- it's 
what we do on linuxbios, and the savings (surprise for me) really adds up.

ron

Re: [9fans] Data segment vs BSS

[Charles Forsyth]

> more important, the compiler had to ensure that no data was marked for program
> memory until it had checked that it was never accessed indirectly (because
> then the called function would use the wrong instruction).  again, const

sorry, i meant `indirectly by something that might access both forms of data'.
for instance, it allowed a table's address to be taken by
	p = &table  (or plain p = table for an array)
and tracked that nothing was assigned through p, and that p
was not assigned elsewhere to point to writable data.

Re: [9fans] Data segment vs BSS

[Charles Forsyth]

> This will usually save a ton of space -- it's 
> what we do on linuxbios, and the savings (surprise for me) really adds up.

i was quite surprised how much SRAM on the atmel was occupied
by string literals, so traces and diagnostics quickly became terse;
still using letters not error numbers, though, so as not to
confuse data with disaster.   i suppose it wouldn't, however, be string literals
in linuxbios because gcc (i thought) tags them for rom.

Re: [9fans] Data segment vs BSS

[Charles Forsyth]

> on the ARM, though, the compiler could
> put strings in the text segment easily enough

i really meant the linker, but never mind.  it misses a trick and doesn't do it
by default.  the -t option enables it, but in the Inferno 5l that does thumb
as well, that sadly enables some debugging (`t'humb).   silly oversight.
fixed.

Re: [9fans] Data segment vs BSS

[Noah Evans]

Cool. Can you send me the fix via private mail?

After reading the 9fans responses, I have a better handle on the
problem. After a reading of a.out(10.6) I realized that I might have
to use a bootstrap loader.

Right now I'm just reading the rom directly from text(the GBA
typically runs all of it's programs from rom) and there's no
header(the gba needs a special rom header starting from 0x0, otherwise
it won't boot). It makes it to main just fine, but no header is
causing havoc with the data and bss sections.

How are edata and end defined by the linker? You can't redefine them
from your code, but given the current state of my code, if I could
just intuit and change where the program thinks the data and bss are,
it would be easy enough to memcpy them from rom where they need to go.

Regarding the program and memory, I'm looking at:

ksize igba.gba
359928t + 90992d + 54936b = 505856      igba.gba

which should halfway work.

Noah

On 1/31/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> > on the ARM, though, the compiler could
> > put strings in the text segment easily enough
>
> i really meant the linker, but never mind.  it misses a trick and doesn't do it
> by default.  the -t option enables it, but in the Inferno 5l that does thumb
> as well, that sadly enables some debugging (`t'humb).   silly oversight.
> fixed.
>
>

Re: [9fans] Data segment vs BSS

[Charles Forsyth]

> Cool. Can you send me the fix via private mail?

i've done that.  also how to declare and use edata, -D etc
to get the right effect.   i'd have sent it to the list but
we might as well check it by experiment before doing that,
since i might have got some points a little wrong,
such as who rounds what when, even though i've just
looked at the code.

Re: [9fans] Data segment vs BSS

[Ronald G Minnich]

Noah Evans wrote:
> Maybe I should be a little more specific. I have a machine(a game boy  
> advance) that only has 256k of memory but has a 32mb for rom.  Basically 
> I want  to save every bit of RAM I can. Would your solution  help?
> 

no. how much data will you save if it is rom, not ram. What are the 
addresses of rom and ram? contiguous?

ron

Re: [9fans] Data segment vs BSS

[Ronald G Minnich]

Charles Forsyth wrote:
>>This will usually save a ton of space -- it's 
>>what we do on linuxbios, and the savings (surprise for me) really adds up.
> 
> 
> i was quite surprised how much SRAM on the atmel was occupied
> by string literals, so traces and diagnostics quickly became terse;
> still using letters not error numbers, though, so as not to
> confuse data with disaster.   i suppose it wouldn't, however, be string literals
> in linuxbios because gcc (i thought) tags them for rom.
> 

actually, it is a lot of ldscript hackery that drives me nuts.

ron

Re: [9fans] Data segment vs BSS

[Noah Evans]

Nope unfortunately, RAM is:

#define	EWRAMZERO	0x02000000
#define	EWRAMTOP		0x0203FFFF

ROM starts at 0x08000000


On Feb 1, 2006, at 12:03 AM, Ronald G Minnich wrote:

> Noah Evans wrote:
>> Maybe I should be a little more specific. I have a machine(a game  
>> boy  advance) that only has 256k of memory but has a 32mb for  
>> rom.  Basically I want  to save every bit of RAM I can. Would your  
>> solution  help?
>
> no. how much data will you save if it is rom, not ram. What are the  
> addresses of rom and ram? contiguous?
>
> ron

Re: [9fans] Data segment vs BSS

[Ronald G Minnich]

Noah Evans wrote:
> Nope unfortunately, RAM is:
> 
> #define    EWRAMZERO    0x02000000
> #define    EWRAMTOP        0x0203FFFF
> 
> ROM starts at 0x08000000
> 


Is it really worth all the work for 64k :-)

ron

Re: [9fans] Data segment vs BSS

[Noah Evans]

It's big brother has 4meg, but no emulator :-( Have to start  
somewhere :-)

Noah

On Feb 1, 2006, at 1:05 AM, Ronald G Minnich wrote:

> Noah Evans wrote:
>> Nope unfortunately, RAM is:
>> #define    EWRAMZERO    0x02000000
>> #define    EWRAMTOP        0x0203FFFF
>> ROM starts at 0x08000000
>
>
> Is it really worth all the work for 64k :-)
>
> ron

Re: [9fans] Data segment vs BSS

[Russ Cox]

> Is it really worth all the work for 64k :-)

Your comment makes it sound like he is working real
hard to push 64k of data into the ROM, but that's not true.

His situation is the opposite: he has megabytes of ROM
but only 64k of data in which to work.  In that case it
may well be worth effort to get constant stuff into the
huge ROM and save the miniscule RAM for actual data.

Russ

Re: [9fans] Data segment vs BSS

[Paul Lalonde]

As someone who works on these consoles for a living, am I ever glad  
the modern ones have real amounts of RAM and disk media.
It's been a long time since I've had to worry about moving stuff to  
ROM.  I have vague recollections of those days, but I've blotted them  
out as best I can.

For oddness, though, over the last few days I've been writing object- 
file writing code - life is much better if you can adopt a binary  
file standard (even for your data, which is most of our case) that  
understands relocations natively, instead of some goobed up thing.   
But no-one's ELF libraries that I can find are anything except GPL  
viral.  And the platform has a dynamic loader, so ELF it has to be.

Paul

On 31-Jan-06, at 8:24 AM, Russ Cox wrote:

>> Is it really worth all the work for 64k :-)
>
> Your comment makes it sound like he is working real
> hard to push 64k of data into the ROM, but that's not true.
>
> His situation is the opposite: he has megabytes of ROM
> but only 64k of data in which to work.  In that case it
> may well be worth effort to get constant stuff into the
> huge ROM and save the miniscule RAM for actual data.
>
> Russ

Re: [9fans] Data segment vs BSS

[C H Forsyth]

> His situation is the opposite: he has megabytes of ROM
> but only 64k of data in which to work.  In that case it
> may well be worth effort to get constant stuff into the
> huge ROM and save the miniscule RAM for actual data.

for example,
in one (inferno) kernel i tried this morning, 54k was strings;
in another with a more typical set of modules, it was 128k!
usually it doesn't matter because text and data are in the
same physical space, but it does in his case.
it seems quite a bit at first, but a `strings' on it looks reasonable.

Re: [9fans] Data segment vs BSS

[Ronald G Minnich]

Russ Cox wrote:

> His situation is the opposite: he has megabytes of ROM
> but only 64k of data in which to work.  In that case it
> may well be worth effort to get constant stuff into the
> huge ROM and save the miniscule RAM for actual data.

arg, I misread the situation ... sorry.

ron