Re: [9fans] mmap and shared libraries

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.

I know one thing: every major operating system I have ever heard of
leverages shared libraries. Can all those people be wrong? I don't think
so.

> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.

Title: Dynamic Linking of Software Components
Author: Michael Franz
YoP: 1997

> ABSTRACT
>
> The concepts underlying dynamic linking reached maturity through modular
> operating systems and are being reintroduced in some of the most popular
> workstation and PC operating systems.

<http://portal.acm.org/citation.cfm?id=619016.620662&coll=GUIDE&dl=GUIDE&CFID=8901601&CFTOKEN=68907171>

The abstract says this is an overall survey of dynamic linking. I can't
access the full text and I wouldn't understand it even if I could. You are
the CS/CE person. Be the judge, too.

> So the compressed version is only 50% larger than acme. The point being?

That there's a lot of redundancy in the executable image principally
imposed by the binary format.

$ readelf -e /usr/local/bin/vim

Reveals over 170 KB of data only in .data, .rodata, and .bss sections.

$ objdump -s /usr/local/bin/vim

Displays tons of zeroes in all sections except .text. Even RLE could have
considerably reduced the image size.

> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.

I don't know. The size of Emacs executable has as much connection to that
question as I do, but it was _you_ who made the bad Emacs joke.

> Wrong for ELF.

About ELF:

> Defining data elements in the bss section is somewhat different from
> defining them in the data section. Instead of declaring specific data
> types, you just declare raw segments of memory that are reserved for
> whatever purpose you need them for.
>
> [...]
>
> One benefit to declaring data in the bss section is that the data is not
> included in the executable program. When data is defined in the data
> section, it must be included in the executable program, since it must be
> initialized with a specific value. Because the data areas declared in the
> bss section are not initialized with program data, the memory areas are
> reserved at runtime, and do not have to be included in the final program.

-- Professional Assembly Programming, Richard Blum, in the context of using
(g)as and ELF

Exactly what I said. I don't want to sound authoritative but when something
is right it is right and there's nothing you or I can do about it. So:
"right for ELF."

Executable image size can be less than memory size because there can be
_uninitialized_ data reserved in data segments, e.g. when you reserve a
buffer but don't care what it initially contains. However, _initialized_
data always and invariably gets written to disk--think of a whole 64K of
zeroes.

Example from a simple program for Flat Assembler with Win32 target:

section '.data' data readable writeable

	buffer_ptr	dd 00h                <---- a pointer reserved, and initialized

	buffer		db 00FFFFh dup (?)    <---- a buffer reserved, but not initialized


> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?

You know very well that the ratio is an inaccurate measure as the context
suggests. 1 to 5 is like your pinkie to all your fingers. Trying to
"context-switch" and turn a guesstimate into an exact number is only a
diversionary technique you have employed to evade an easily demonstrable
fact. Namely that Acme is a minimal text editor claiming to be an IDE while
Emacs is a behemoth with more features than you could count in a day, and
the following night--not that I believe Emacs is any better than Acme. Vim
leans more towards Emacs.

--On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
> <eris.discordia@gmail.com> wrote:
>>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
>>> about 1/2 M. Vim on DOS is 3x larger? impressive.
>>
>> My intent was, of course, to show your comparison is baseless. It seems
>> you still haven't realized that. You think Plan 9 is great?
>
> Actually, to keep it simple, I was trying to make a simple point, with
> a bit of humor intended, and not actually even directed at you,: that
> code built with shared libraries is not
> necessarily, or always, smaller than code built otherwise, for
> programs of similar capability. I don't have an emacs
> linked statically, however, so I took the nearest materials at hand.
> It depends on a lot of circumstances.
>
> There is the example of xclock which is small with X11 .so, and quite
> large otherwise. And there, for the case of a "clock", shared
> libraries clearly reduce disk footprint:
> xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..
>
> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.
>
> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.
>
>> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes
>
> So the compressed version is only 50% larger than acme. The point being?
>
>> because the bigger part of the uncompressed 1,585,152 bytes is redundant
>> filler required by the binary format specification.
>
> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.
>
>> The same happens for the
>> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
>> binary format and ELF both require the executable image to contain all
>> initialized but essentially redundant (i.e. zeroed) parts of the data
>> segement. Don't pretend you didn't know that.
>
> Wrong for ELF.
>
> Simple example from /bin/cat:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   LOAD           0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW
> 0x10000
>
> note that filesize is < memory size. I'll let you figure out how it works.
>
> This is the problem with your posts. You sound very authoritative, and
> I'm sure you figure you know all these bits,
> but you're wrong so often. It's confusing. It would be unfortunate
> were people to believe you are as right as you think you are.
>
>>
>> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
>> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
>> you have been unfair. No kidding.
>>
>
> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?
> Of course, you have to take into account that acme's extensions live
> in /bin or your number is bogus.
>
> ron
>

Re: [9fans] mmap and shared libraries

[Wes Kussmaul]

Eris Discordia wrote:
> I know one thing

I doubt that.

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

I think ericvh has a P.Enis, I know I do.

Actually maybe it was Ennis (that one's at home), but it's funny enough.

brucee

On Mon, Nov 10, 2008 at 12:37 AM, Dan Cross <crossd@gmail.com> wrote:
> On Sun, Nov 9, 2008 at 5:07 PM, Bruce Ellis <bruce.ellis@gmail.com> wrote:
>> [...] poor old Mr Peter Enis [...]
>
> Wow, this is really sad, but I *just* got that.
>
>        - Dan C.
>
>

Re: [9fans] mmap and shared libraries

[Dan Cross]

On Sun, Nov 9, 2008 at 5:07 PM, Bruce Ellis <bruce.ellis@gmail.com> wrote:
> [...] poor old Mr Peter Enis [...]

Wow, this is really sad, but I *just* got that.

        - Dan C.

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

Sharp, Noah. Shame you and all the 9fans had to get back to real work.

I'm trying to avoid it but I have to face reality (again) one day.

I'm actually starting to really appreciate Football (Soccer). Though
it seems inevitable if you like bars here.

I can't but think of poor old Mr Peter Enis when I see mail from Eris.
He had an office at the Labs in the nineties (building 6, floor 4 I
think). His door plate was very volatile.

brucee

On Sun, Nov 9, 2008 at 11:16 PM, Noah Evans <noah.evans@gmail.com> wrote:
> http://www.apa.org/journals/features/psp7761121.pdf
>
> On Sun, Nov 9, 2008 at 8:53 AM, Eris Discordia <eris.discordia@gmail.com>
> wrote:
>>> http://en.wikipedia.org/wiki/Psychological_projection
>>
>> http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement
>>
>> Bruce Ellis and Noah Evans, please populate the above stub.
>>
>> By the way, Bruce Ellis, do never ever recommend medication to anyone
>> _even
>> in jest_ without providing a visible disclaimer. That can get you caught.
>>
>> --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis
>> <bruce.ellis@gmail.com> wrote:
>>
>>> Noah is dangerously wise. Have you got rid of the smell of mackerel
>>> yet? Seriously Eris, you need a good hobby.
>>>
>>> And if I was your physician I would recommend medication. A SSRI or
>>> maybe a simple Benzo. Maybe twice a day, or when required.
>>>
>>> brucee
>>>
>>> On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans <noah.evans@gmail.com> wrote:
>>>>
>>>> http://en.wikipedia.org/wiki/Psychological_projection
>>>>
>>>> On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia
>>>> <eris.discordia@gmail.com> wrote:
>>>>>
>>>>> Little troll, thy baiting f'r fray--
>>>>> My thoughtless passage has flushed away
>>>>> Am not _I_ a troll like thee,
>>>>> Or art not _thou_ a Goddess like me?
>>>>>
>>>>> Practice your technique, little troll, while you have time to do
>>>>> mischief under the Goddess' nose!
>>>>>
>>>>> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
>>>>> <lyndon@orthanc.ca> wrote:
>>>>>
>>>>>>>> I know one thing: every major operating system I have ever heard of
>>>>>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>>>>>> think so.
>>>>>>
>>>>>> Eight billion Windows users can't be wrong. (Can they?)
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>>
>>
>>
>
>

Re: [9fans] mmap and shared libraries

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 1826 bytes --]

http://www.apa.org/journals/features/psp7761121.pdf

On Sun, Nov 9, 2008 at 8:53 AM, Eris Discordia <eris.discordia@gmail.com>
wrote:
>> http://en.wikipedia.org/wiki/Psychological_projection
>
> http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement
>
> Bruce Ellis and Noah Evans, please populate the above stub.
>
> By the way, Bruce Ellis, do never ever recommend medication to anyone
_even
> in jest_ without providing a visible disclaimer. That can get you caught.
>
> --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis
> <bruce.ellis@gmail.com> wrote:
>
>> Noah is dangerously wise. Have you got rid of the smell of mackerel
>> yet? Seriously Eris, you need a good hobby.
>>
>> And if I was your physician I would recommend medication. A SSRI or
>> maybe a simple Benzo. Maybe twice a day, or when required.
>>
>> brucee
>>
>> On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans <noah.evans@gmail.com> wrote:
>>>
>>> http://en.wikipedia.org/wiki/Psychological_projection
>>>
>>> On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia
>>> <eris.discordia@gmail.com> wrote:
>>>>
>>>> Little troll, thy baiting f'r fray--
>>>> My thoughtless passage has flushed away
>>>> Am not _I_ a troll like thee,
>>>> Or art not _thou_ a Goddess like me?
>>>>
>>>> Practice your technique, little troll, while you have time to do
>>>> mischief under the Goddess' nose!
>>>>
>>>> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
>>>> <lyndon@orthanc.ca> wrote:
>>>>
>>>>>>> I know one thing: every major operating system I have ever heard of
>>>>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>>>>> think so.
>>>>>
>>>>> Eight billion Windows users can't be wrong. (Can they?)
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 3018 bytes --]

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

Go and see a doctor then. What a silly guy, huh Noah?

Perhaps he doesn't know what "perhaps" means. He certainly doesn't
know jack about computer science.

Dr Bruce Ellis
Department of Conceptual Integrity
University of Thessaly

On Sun, Nov 9, 2008 at 4:53 PM, Eris Discordia <eris.discordia@gmail.com> wrote:
>> http://en.wikipedia.org/wiki/Psychological_projection
>
> http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement
>
> Bruce Ellis and Noah Evans, please populate the above stub.
>
> By the way, Bruce Ellis, do never ever recommend medication to anyone _even
> in jest_ without providing a visible disclaimer. That can get you caught.
>
> --On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis
> <bruce.ellis@gmail.com> wrote:
>
>> Noah is dangerously wise. Have you got rid of the smell of mackerel
>> yet? Seriously Eris, you need a good hobby.
>>
>> And if I was your physician I would recommend medication. A SSRI or
>> maybe a simple Benzo. Maybe twice a day, or when required.
>>
>> brucee
>>
>> On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans <noah.evans@gmail.com> wrote:
>>>
>>> http://en.wikipedia.org/wiki/Psychological_projection
>>>
>>> On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia
>>> <eris.discordia@gmail.com> wrote:
>>>>
>>>> Little troll, thy baiting f'r fray--
>>>> My thoughtless passage has flushed away
>>>> Am not _I_ a troll like thee,
>>>> Or art not _thou_ a Goddess like me?
>>>>
>>>> Practice your technique, little troll, while you have time to do
>>>> mischief under the Goddess' nose!
>>>>
>>>> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
>>>> <lyndon@orthanc.ca> wrote:
>>>>
>>>>>>> I know one thing: every major operating system I have ever heard of
>>>>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>>>>> think so.
>>>>>
>>>>> Eight billion Windows users can't be wrong. (Can they?)
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
>
>
>
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> http://en.wikipedia.org/wiki/Psychological_projection

http://en.wikipedia.org/wiki/Lack_of_aesthetical_refinement

Bruce Ellis and Noah Evans, please populate the above stub.

By the way, Bruce Ellis, do never ever recommend medication to anyone _even
in jest_ without providing a visible disclaimer. That can get you caught.

--On Sunday, November 09, 2008 12:12 AM +0200 Bruce Ellis
<bruce.ellis@gmail.com> wrote:

> Noah is dangerously wise. Have you got rid of the smell of mackerel
> yet? Seriously Eris, you need a good hobby.
>
> And if I was your physician I would recommend medication. A SSRI or
> maybe a simple Benzo. Maybe twice a day, or when required.
>
> brucee
>
> On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans <noah.evans@gmail.com> wrote:
>> http://en.wikipedia.org/wiki/Psychological_projection
>>
>> On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia
>> <eris.discordia@gmail.com> wrote:
>>> Little troll, thy baiting f'r fray--
>>> My thoughtless passage has flushed away
>>> Am not _I_ a troll like thee,
>>> Or art not _thou_ a Goddess like me?
>>>
>>> Practice your technique, little troll, while you have time to do
>>> mischief under the Goddess' nose!
>>>
>>> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
>>> <lyndon@orthanc.ca> wrote:
>>>
>>>>>> I know one thing: every major operating system I have ever heard of
>>>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>>>> think so.
>>>>
>>>> Eight billion Windows users can't be wrong. (Can they?)
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

Noah is dangerously wise. Have you got rid of the smell of mackerel
yet? Seriously Eris, you need a good hobby.

And if I was your physician I would recommend medication. A SSRI or
maybe a simple Benzo. Maybe twice a day, or when required.

brucee

On Sat, Nov 8, 2008 at 11:37 PM, Noah Evans <noah.evans@gmail.com> wrote:
> http://en.wikipedia.org/wiki/Psychological_projection
>
> On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia <eris.discordia@gmail.com> wrote:
>> Little troll, thy baiting f'r fray--
>> My thoughtless passage has flushed away
>> Am not _I_ a troll like thee,
>> Or art not _thou_ a Goddess like me?
>>
>> Practice your technique, little troll, while you have time to do mischief
>> under the Goddess' nose!
>>
>> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
>> <lyndon@orthanc.ca> wrote:
>>
>>>>> I know one thing: every major operating system I have ever heard of
>>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>>> think so.
>>>
>>> Eight billion Windows users can't be wrong. (Can they?)
>>>
>>
>>
>>
>>
>>
>>
>
>

Re: [9fans] mmap and shared libraries

[Noah Evans]

http://en.wikipedia.org/wiki/Psychological_projection

On Sat, Nov 8, 2008 at 9:21 AM, Eris Discordia <eris.discordia@gmail.com> wrote:
> Little troll, thy baiting f'r fray--
> My thoughtless passage has flushed away
> Am not _I_ a troll like thee,
> Or art not _thou_ a Goddess like me?
>
> Practice your technique, little troll, while you have time to do mischief
> under the Goddess' nose!
>
> --On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
> <lyndon@orthanc.ca> wrote:
>
>>>> I know one thing: every major operating system I have ever heard of
>>>> leverages shared libraries. Can all those people be wrong? I don't
>>>> think so.
>>
>> Eight billion Windows users can't be wrong. (Can they?)
>>
>
>
>
>
>
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

Little troll, thy baiting f'r fray--
My thoughtless passage has flushed away
Am not _I_ a troll like thee,
Or art not _thou_ a Goddess like me?

Practice your technique, little troll, while you have time to do mischief
under the Goddess' nose!

--On Friday, November 07, 2008 6:07 PM -0800 Lyndon Nerenberg
<lyndon@orthanc.ca> wrote:

>>> I know one thing: every major operating system I have ever heard of
>>> leverages shared libraries. Can all those people be wrong? I don't
>>> think so.
>
> Eight billion Windows users can't be wrong. (Can they?)
>

Re: [9fans] mmap and shared libraries

[Lyndon Nerenberg]

>> I know one thing: every major operating system I have ever heard of
>> leverages shared libraries. Can all those people be wrong? I don't think so.

Eight billion Windows users can't be wrong. (Can they?)

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

Wow what a lively thread. Lots of good information (thanks Ron, Rob et
al) and behavioural commentary (thanks Skip) but it seems to be of no
avail.

As I'm the only one awake in Volos at 10:30am and I have 40 cafes with
WiFi to myself I'll waste bandwidth and tell a little story with no
stated conclusions.

Three 9fans get into a taxi and need to get to Volos Bus Station. They
only speak English, Dutch, German, French, Italian, Spanish,
Portuguese, and Signing between them - no Greek. How do they get to
the bus station for the minimum fare (3 euros)? Sounds like a bad
joke, but it really happened. Answer at the end.

Fictional scenario: same thing with three Linux/BSD fans who only
speak English, Bash and Emacs. Their solution:

1) Whine at the taxi driver - cost 1 euro

2) Shout "why doesn't this cab run emacs?" - cost 1 euro

3) Download a language package, discover they have incompatible shared
libraries - cost 2 euros

4) Shout "mmap() me to the bus station" - cost 2 euros

5) Take out a phrase book and order a goat and a fishing village - cost 5 euros

6) Get dumped at a distant fishing village where they are savagely
beaten and dragged aboard a fishing boat as slaves - cost all their
worldly possessions

7) Get dumped back at Volos, because they can't fish without emacs,
wearing only shorts that smell of mackerel and one shoe between them -
time share the shoe and walk to the bus station - no charge

And the answer to the original question? The taxi driver spoke
Spanish. The 9fans tip 1 euro.

Sorry for wasting your time.

brucee

-- pages of replies -- elided

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Sorry, but "everyone does it" just doesn't hack it.

Common-sensically it does, but if you say it doesn't I concede.

> not what I asked.

It seems to answer your question by implying there's no question regarding
the use or disuse of shared libraries only regarding the strategy (and that
back in 1997). But I can't read beyond the abstract so I can't tell if it
really goes that far. Incidentally, it references some Wirth papers.

> you're kind of confusing how the file is created with what ELF wants,
> but that's up to you.

Perhaps, I'll try to read up on it.

> You need to understand the difference between sections and segments.
> From the point of view of the program loader, there is no such thing
> as a ..bss.

I believe I understand the difference. The loader may put .bss, .data,
.rodata, and some other data sections around the same place in memory and
decide about the order and relative placement of each. That doesn't change
anything about the binary format, however. Only when the image is being
loaded .bss grows to its true size. On the other hand, .data is more or
less the same in memory or on disk.

> But think of it this way: You can have a valid ELF executable that has
> no sections. Sections have nothing to do with running the program.

Initialized data has to be placed somewhere in the image, anyway.

> So, Wrong for ELF. Sorry. But it's an incredibly common mistake.

I'll try to learn more about it (and probably report back... okay, I won't).

> So I guess that acme has more features than vi, since it's extensions
> are "any executable"; you guess not.
> My claim is you don't actually understand acme.

That's a probable cause but only probable.

> Let's leave it at that. Except, uh, who claimed acme was an IDE? Just
> wondering. It wasn't me.

Well, leave it at that. By the way, the programmer's do-all is an IDE I
presume.

Finally, thanks for bearing with me and supplying things I didn't know.

--On Wednesday, November 05, 2008 5:25 PM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 4:45 PM, Eris Discordia <eris.discordia@gmail.com>
> wrote:
>
>> I know one thing: every major operating system I have ever heard of
>> leverages shared libraries. Can all those people be wrong? I don't think
>> so.
>
> I know one thing. Every major operating system in the late 1960s
> "knew" that card image files were the way to go. Could all those
> people be wrong?
>
> Yes.
>
> Sorry, but "everyone does it" just doesn't hack it.
>
>>
>>> FWIW, the whole issue of goodness/badness of shared libraries has
>>> never been systematically measured as far as I know -- in terms of
>>> performance cost, overhead, throughput, the usual suspects -- or at
>>> least
>>> I've never seen it done. It would be a lot of work to do it correctly.
>>> Might be interesting.
> =
>> The abstract says this is an overall survey of dynamic linking. I can't
>> access the full text and I wouldn't understand it even if I could. You
>> are the CS/CE person. Be the judge, too.
>
> not what I asked.
>
>>
>>> So the compressed version is only 50% larger than acme. The point being?
>>
>> That there's a lot of redundancy in the executable image principally
>> imposed by the binary format.
>>
>> $ readelf -e /usr/local/bin/vim
>>
>> Reveals over 170 KB of data only in .data, .rodata, and .bss sections.
>>
>> $ objdump -s /usr/local/bin/vim
>>
>> Displays tons of zeroes in all sections except .text. Even RLE could have
>> considerably reduced the image size.
>> ]
>
> you're kind of confusing how the file is created with what ELF wants,
> but that's up to you.
>
>>> Does the redundant litter cross the network when I have it mounted via
>>> a share and execute the program or not? That was the original
>>> discussion.
>>
>> I don't know. The size of Emacs executable has as much connection to that
>> question as I do, but it was _you_ who made the bad Emacs joke.
>
> I.e. You don't know. OK, I'll accept that.
>
>>
>>> Wrong for ELF.
>>
>> About ELF:
>>
>>> Defining data elements in the bss section is somewhat different from
>>> defining them in the data section. Instead of declaring specific data
>>> types, you just declare raw segments of memory that are reserved for
>>> whatever purpose you need them for.
>>>
>>> [...]
>>>
>>> One benefit to declaring data in the bss section is that the data is not
>>> included in the executable program. When data is defined in the data
>>> section, it must be included in the executable program, since it must be
>>> initialized with a specific value. Because the data areas declared in
>>> the bss section are not initialized with program data, the memory areas
>>> are reserved at runtime, and do not have to be included in the final
>>> program.
>>
>> -- Professional Assembly Programming, Richard Blum, in the context of
>> using (g)as and ELF
>>
>> Exactly what I said. I don't want to sound authoritative but when
>> something is right it is right and there's nothing you or I can do about
>> it. So: "right for ELF."
>
> I see your confusion, and it is common, and I have made this mistake
> too. ELF is like that :-)
> And readelf can make it worse.
>
> You need to understand the difference between sections and segments.
> From the point of view of the program loader, there is no such thing
> as a ..bss.
>
> From the point of view of the linker, there are .bss segments. But
> those don't have lots of meaning for a program loader. In fact program
> loaders that look at sections are broken.
>
> So you're going to need to go back to the standard, but most people
> are: I once worked with code that was created by a big company that
> mistakenly parsed the sections, not the segments, of an executable. It
> was a mess.
>
> But think of it this way: You can have a valid ELF executable that has
> no sections. Sections have nothing to do with running the program.
>
> So, Wrong for ELF. Sorry. But it's an incredibly common mistake.
>
>> Example from a simple program for Flat Assembler with Win32 target:
>>
>> section '.data' data readable writeable
>>
>>        buffer_ptr      dd 00h                <---- a pointer reserved,
>>        and initialized
>>
>>        buffer          db 00FFFFh dup (?)    <---- a buffer reserved, but
>> not initialized
>
> Trust me. I don't care about win32 targets.
>
>
>>
>>
>>> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
>>> in 5 and not 1 in 4.8, or 1 in 6?
>>
>> You know very well that the ratio is an inaccurate measure as the context
>> suggests. 1 to 5 is like your pinkie to all your fingers. Trying to
>> "context-switch" and turn a guesstimate into an exact number is only a
>> diversionary technique you have employed to evade an easily demonstrable
>> fact. Namely that Acme is a minimal text editor claiming to be an IDE
>> while Emacs is a behemoth with more features than you could count in a
>> day, and the following night--not that I believe Emacs is any better
>> than Acme. Vim leans more towards Emacs.
>
> So I guess that acme has more features than vi, since it's extensions
> are "any executable"; you guess not.
> My claim is you don't actually understand acme.
> Let's leave it at that. Except, uh, who claimed acme was an IDE? Just
> wondering. It wasn't me.
>
> ron
>

Re: [9fans] mmap and shared libraries

[Skip Tavakkolian]

> Sorry, but "everyone does it" just doesn't hack it.

also, "everyone does it" is an excuse that no one over the age 7
should use.  imitating blindly -- a.k.a monkey-see-monkey-do
(apologies to monkeys) -- seems to happen when we are unaware,
undisciplined, lazy or panicked.  it is how stampedes happen.  it's ok
if you're in a herd, can't see much and trying to avoid a predator;
not so for designing software.

a good example can be found in an episode of a UW TV program called
"behind the code", produced by MS and UW, where one of the original
developers of NT (hardware and drivers) recalls a kernel crash that
was caused by bad MS sample code that ended up in several driver by
different vendors.

Re: [9fans] mmap and shared libraries

[Eris Discordia]

Thanks for the information.

--On Wednesday, November 05, 2008 5:23 PM -0800 Rob Pike
<robpike@gmail.com> wrote:

> When Sun reported on their first implementation of shared libraries,
> the paper they presented (I think it was at Usenix) concluded that
> shared libraries made things bigger and slower, that they were a net
> loss, and in fact that they didn't save much disk space either.  The
> test case was Xlib, the benefit negative. But the customer expects us
> to do them so we'll ship it.
>
> So yes, every major operating system implements them but that does not
> mean they are a good idea.  Plan 9 was designed to ignore at least
> some of the received wisdom.
>
> -rob
>

Re: [9fans] mmap and shared libraries

[Roman Shaposhnik]

On Nov 5, 2008, at 2:13 PM, ron minnich wrote:
> On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
> <eris.discordia@gmail.com> wrote:
>>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan
>>> 9 is
>>> about 1/2 M. Vim on DOS is 3x larger? impressive.
>>
>> My intent was, of course, to show your comparison is baseless. It
>> seems you
>> still haven't realized that. You think Plan 9 is great?
>
> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.

Exactly! On Linux with ld.so accepting relocations (think non-PIC code)
in .text the appearance of some .so's being shared is highly
deceiving. Solaris is better in that regard, but the pressure from
userland community expecting their project to "just work" is mounting.

Thanks,
Roman.

Re: [9fans] mmap and shared libraries

[Noah Evans]

On Wed, Nov 5, 2008 at 6:48 PM, Eris Discordia <eris.discordia@gmail.com> wrote:

> I know one thing: every major operating system I have ever heard of
> leverages shared libraries. Can all those people be wrong? I don't think so.

http://en.wikipedia.org/wiki/Argumentum_ad_populum

Re: [9fans] mmap and shared libraries

[erik quanstrom]

> File size can be less than memory size when you have data reserved but not
> initialized. That happens in many cases, e.g. when you reserve a buffer.

>> One benefit to declaring data in the bss section is that the data is not
>> included in the executable program. When data is defined in the data
>> section, it must be included in the executable program, since it must be
>> initialized with a specific value. Because the data areas declared in the
>> bss section are not initialized with program data, the memory areas are
>> reserved at runtime, and do not have to be included in the final program.

i don't think uninitialized means what you think it does.  for the
benefit of those who might be confused by this largely purposeful
misdirection, the bss is initialized to zero.  the c standard
specifies that external declarations without explicit initialization
are initialized to zero.  this is intialized data that doesn't appear
in the executable.

because i'm pedantic, i'll point out that zero has always been a
specific value.

- erik

Re: [9fans] mmap and shared libraries

[ron minnich]

On Wed, Nov 5, 2008 at 4:45 PM, Eris Discordia <eris.discordia@gmail.com> wrote:

> I know one thing: every major operating system I have ever heard of
> leverages shared libraries. Can all those people be wrong? I don't think so.

I know one thing. Every major operating system in the late 1960s
"knew" that card image files were the way to go. Could all those
people be wrong?

Yes.

Sorry, but "everyone does it" just doesn't hack it.

>
>> FWIW, the whole issue of goodness/badness of shared libraries has
>> never been systematically measured as far as I know -- in terms of
>> performance cost, overhead, throughput, the usual suspects -- or at
>> least
>> I've never seen it done. It would be a lot of work to do it correctly.
>> Might be interesting.
=
> The abstract says this is an overall survey of dynamic linking. I can't
> access the full text and I wouldn't understand it even if I could. You are
> the CS/CE person. Be the judge, too.

not what I asked.

>
>> So the compressed version is only 50% larger than acme. The point being?
>
> That there's a lot of redundancy in the executable image principally imposed
> by the binary format.
>
> $ readelf -e /usr/local/bin/vim
>
> Reveals over 170 KB of data only in .data, .rodata, and .bss sections.
>
> $ objdump -s /usr/local/bin/vim
>
> Displays tons of zeroes in all sections except .text. Even RLE could have
> considerably reduced the image size.
>]

you're kind of confusing how the file is created with what ELF wants,
but that's up to you.

>> Does the redundant litter cross the network when I have it mounted via
>> a share and execute the program or not? That was the original
>> discussion.
>
> I don't know. The size of Emacs executable has as much connection to that
> question as I do, but it was _you_ who made the bad Emacs joke.

I.e. You don't know. OK, I'll accept that.

>
>> Wrong for ELF.
>
> About ELF:
>
>> Defining data elements in the bss section is somewhat different from
>> defining them in the data section. Instead of declaring specific data
>> types, you just declare raw segments of memory that are reserved for
>> whatever purpose you need them for.
>>
>> [...]
>>
>> One benefit to declaring data in the bss section is that the data is not
>> included in the executable program. When data is defined in the data
>> section, it must be included in the executable program, since it must be
>> initialized with a specific value. Because the data areas declared in the
>> bss section are not initialized with program data, the memory areas are
>> reserved at runtime, and do not have to be included in the final program.
>
> -- Professional Assembly Programming, Richard Blum, in the context of using
> (g)as and ELF
>
> Exactly what I said. I don't want to sound authoritative but when something
> is right it is right and there's nothing you or I can do about it. So:
> "right for ELF."

I see your confusion, and it is common, and I have made this mistake
too. ELF is like that :-)
And readelf can make it worse.

You need to understand the difference between sections and segments.
>From the point of view of the program loader, there is no such thing
as a ..bss.

>From the point of view of the linker, there are .bss segments. But
those don't have lots of meaning for a program loader. In fact program
loaders that look at sections are broken.

So you're going to need to go back to the standard, but most people
are: I once worked with code that was created by a big company that
mistakenly parsed the sections, not the segments, of an executable. It
was a mess.

But think of it this way: You can have a valid ELF executable that has
no sections. Sections have nothing to do with running the program.

So, Wrong for ELF. Sorry. But it's an incredibly common mistake.

> Example from a simple program for Flat Assembler with Win32 target:
>
> section '.data' data readable writeable
>
>        buffer_ptr      dd 00h                <---- a pointer reserved, and
> initialized
>
>        buffer          db 00FFFFh dup (?)    <---- a buffer reserved, but
> not initialized

Trust me. I don't care about win32 targets.


>
>
>> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
>> in 5 and not 1 in 4.8, or 1 in 6?
>
> You know very well that the ratio is an inaccurate measure as the context
> suggests. 1 to 5 is like your pinkie to all your fingers. Trying to
> "context-switch" and turn a guesstimate into an exact number is only a
> diversionary technique you have employed to evade an easily demonstrable
> fact. Namely that Acme is a minimal text editor claiming to be an IDE while
> Emacs is a behemoth with more features than you could count in a day, and
> the following night--not that I believe Emacs is any better than Acme. Vim
> leans more towards Emacs.

So I guess that acme has more features than vi, since it's extensions
are "any executable"; you guess not.
My claim is you don't actually understand acme.
Let's leave it at that. Except, uh, who claimed acme was an IDE? Just
wondering. It wasn't me.

ron

Re: [9fans] mmap and shared libraries

[Rob Pike]

When Sun reported on their first implementation of shared libraries,
the paper they presented (I think it was at Usenix) concluded that
shared libraries made things bigger and slower, that they were a net
loss, and in fact that they didn't save much disk space either.  The
test case was Xlib, the benefit negative. But the customer expects us
to do them so we'll ship it.

So yes, every major operating system implements them but that does not
mean they are a good idea.  Plan 9 was designed to ignore at least
some of the received wisdom.

-rob

Re: [9fans] mmap and shared libraries

[erik quanstrom]

> I know one thing. Every major operating system in the late 1960s
> "knew" that card image files were the way to go. Could all those
> people be wrong?
>
> Yes.
>
> Sorry, but "everyone does it" just doesn't hack it.

it's the chewbacca proof.

- erik

Re: [9fans] mmap and shared libraries

[andrey mirtchovski]

On Wed, Nov 5, 2008 at 5:53 PM, Eris Discordia <eris.discordia@gmail.com> wrote:
> Please forgive the repeated messages. It didn't appear in my mail client's
> Sent view after I hit send. Thought it might have been lost so I re-wrote
> it.

i expect you to start  littering the web forums and mailing lists
dedicated to your mail client software immediately. after all it did
cause you discomfort, didn't it?

let us know how it goes.

Re: [9fans] mmap and shared libraries

[Eris Discordia]

Please forgive the repeated messages. It didn't appear in my mail client's
Sent view after I hit send. Thought it might have been lost so I re-wrote
it.

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.

I know one thing: every major operating system I have ever heard of
leverages shared libraries. Can all those people be wrong? I don't think
so.

> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.

Title: Dynamic Linking of Software Components
Author: Michael Franz
YoP: 1997

> ABSTRACT
>
> The concepts underlying dynamic linking reached maturity through modular
> operating systems and are being reintroduced in some of the most popular
> workstation and PC operating systems.

<http://portal.acm.org/citation.cfm?id=619016.620662&coll=GUIDE&dl=GUIDE&CFID=8901601&CFTOKEN=68907171>

The abstract says this is an overall survey of dynamic linking. I can't
access the full text and I wouldn't understand it even if I could. You are
the CS/CE person. Be the judge, too.

> So the compressed version is only 50% larger than acme. The point being?

That there's a lot of redundancy in the executable image principally
imposed by the binary format.

$ readelf -e /usr/local/bin/vim

Reveals over 170 KB of data only in .data, .rodata, and .bss sections.

$ objdump -s /usr/local/bin/vim

Displays tons of zeroes in all sections except .text. Even RLE could have
considerably reduced the image size.

> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.

I don't know. The size of Emacs executable has as much connection to that
question as I do, but it was _you_ who made the bad Emacs joke.

> Wrong for ELF.

About ELF:

> Defining data elements in the bss section is somewhat different from
> defining them in the data section. Instead of declaring specific data
> types, you just declare raw segments of memory that are reserved for
> whatever purpose you need them for.
>
> [...]
>
> One benefit to declaring data in the bss section is that the data is not
> included in the executable program. When data is defined in the data
> section, it must be included in the executable program, since it must be
> initialized with a specific value. Because the data areas declared in the
> bss section are not initialized with program data, the memory areas are
> reserved at runtime, and do not have to be included in the final program.

-- Professional Assembly Programming, Richard Blum, in the context of using
(g)as and ELF

Exactly what I said. I don't want to sound authoritative but when something
is right it is right and there's nothing you or I can do about it.

Executable image size can be less than memory size because there can be
_uninitialized_ data reserved in data segments, e.g. when you reserve a
buffer but don't care what it initially contains. However, _initialized_
data always and invariably gets written to disk--think of a whole 64K of
zeroes.

Example from a simple program for Flat Assembler with Win32 target:

section '.data' data readable writeable

	buffer_ptr	dd 00h                <---- a pointer reserved, and initialized

	buffer		db 00FFFFh dup (?)    <---- a buffer reserved, but not initialized


> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?

You know very well that the ratio is an inaccurate measure as the context
suggests. 1 to 5 is like your pinkie to all your fingers. Trying to
"context-switch" and turn a guesstimate into an exact number is only a
diversionary technique you have employed to evade an easily demonstrable
fact. Namely that Acme is a minimal text editor claiming to be an IDE while
Emacs is a behemoth with more features than you could count in a day, and
the following night--not that I believe Emacs is any better than Acme. Vim
leans more towards Emacs.

--On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
> <eris.discordia@gmail.com> wrote:
>>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
>>> about 1/2 M. Vim on DOS is 3x larger? impressive.
>>
>> My intent was, of course, to show your comparison is baseless. It seems
>> you still haven't realized that. You think Plan 9 is great?
>
> Actually, to keep it simple, I was trying to make a simple point, with
> a bit of humor intended, and not actually even directed at you,: that
> code built with shared libraries is not
> necessarily, or always, smaller than code built otherwise, for
> programs of similar capability. I don't have an emacs
> linked statically, however, so I took the nearest materials at hand.
> It depends on a lot of circumstances.
>
> There is the example of xclock which is small with X11 .so, and quite
> large otherwise. And there, for the case of a "clock", shared
> libraries clearly reduce disk footprint:
> xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..
>
> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.
>
> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.
>
>> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes
>
> So the compressed version is only 50% larger than acme. The point being?
>
>> because the bigger part of the uncompressed 1,585,152 bytes is redundant
>> filler required by the binary format specification.
>
> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.
>
>> The same happens for the
>> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
>> binary format and ELF both require the executable image to contain all
>> initialized but essentially redundant (i.e. zeroed) parts of the data
>> segement. Don't pretend you didn't know that.
>
> Wrong for ELF.
>
> Simple example from /bin/cat:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   LOAD           0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW
> 0x10000
>
> note that filesize is < memory size. I'll let you figure out how it works.
>
> This is the problem with your posts. You sound very authoritative, and
> I'm sure you figure you know all these bits,
> but you're wrong so often. It's confusing. It would be unfortunate
> were people to believe you are as right as you think you are.
>
>>
>> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
>> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
>> you have been unfair. No kidding.
>>
>
> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?
> Of course, you have to take into account that acme's extensions live
> in /bin or your number is bogus.
>
> ron
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.

I know one thing: shared libraries are employed on every major operating
system I have ever heard of. Can all these people be wrong? I don't think
so.

> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.

Title: Dynamic Linking of Software Components
Author: Michael Franz
YoP: 1997

> ABSTRACT
>
> The concepts underlying dynamic linking reached maturity through modular
> operating systems and are being reintroduced in some of the most popular
> workstation and PC operating systems. [...]

<http://portal.acm.org/citation.cfm?id=619016.620662&coll=GUIDE&dl=GUIDE&CFID=8901601&CFTOKEN=68907171>

The abstract says it is an overall survey of dynamic linking. I don't have
access to the full text and even if I had I wouldn't understand a word of
it. You are the CS/CE person. Be the judge, too.

> So the compressed version is only 50% larger than acme. The point being?

That there's a lot of redundancy in the executable principally imposed by
the binary format.

In case of the FreeBSD vim executable:

$ readelf -e /usr/local/bin/vim

Reveals 171,365 bytes of redundant data only in .data, .rodata, and .bss
sections.

$ objdump -s /usr/local/bin/vim

Will display tons of zeroes in all other sections except .text. Even RLE
could significantly reduce the image size.

> note that filesize is < memory size. I'll let you figure out how it works.

I'm not that intelligent but didn't I say it was _initialized_ data that
had to be written to executable image?

Example from a simple program I wrote to be assembled by Flat Assembler for
Win32:

--------------------------------------------------------------------------------
section '.data' data readable writeable
.
.
.
  buffer_ptr	dd 00h           <----- pointer reserved, and initialized
.
.
.
  buffer	db 00FFFFh dup (?)     <----- buffer reserved, not initialized
.
.
.
--------------------------------------------------------------------------------

File size can be less than memory size when you have data reserved but not
initialized. That happens in many cases, e.g. when you reserve a buffer.

As for ELF:

> Defining data elements in the bss section is somewhat different from
> defining them in the data section. Instead of declaring specific data
> types, you just declare raw segments of memory that  are reserved for
> whatever purpose you need them for.
>
> [...]
>
> One benefit to declaring data in the bss section is that the data is not
> included in the executable program. When data is defined in the data
> section, it must be included in the executable program, since it must be
> initialized with a specific value. Because the data areas declared in the
> bss section are not initialized with program data, the memory areas are
> reserved at runtime, and do not have to be included in the final program.

-- Richard Blum, Professional Assembly Language, in the context of using
(g)as and ELF format

I don't want to sound authoritative but when something is right it is right
and there's nothing you or I can do about it.

> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.

I don't know. The size of Emacs executable has as much connection to that
question as I do, but it was _you_ who made the bad Emacs joke.

> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?

You know very well that the ratio is just an inaccurate measure as the
context suggests. 1 to 5 means like your pinkie compared to your entire
hand. Trying to "context-switch" and move from an inaccurate measure to an
exact number is just a diversionary technique you have employed for
countering an easily demonstrable fact. Namely that Acme is a minimal text
editor claiming to be an IDE while Emacs is a behemoth with more features
than you could count through a day, and the following night--not that I
believe Emacs is any better than Acme.

--On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
> <eris.discordia@gmail.com> wrote:
>>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
>>> about 1/2 M. Vim on DOS is 3x larger? impressive.
>>
>> My intent was, of course, to show your comparison is baseless. It seems
>> you still haven't realized that. You think Plan 9 is great?
>
> Actually, to keep it simple, I was trying to make a simple point, with
> a bit of humor intended, and not actually even directed at you,: that
> code built with shared libraries is not
> necessarily, or always, smaller than code built otherwise, for
> programs of similar capability. I don't have an emacs
> linked statically, however, so I took the nearest materials at hand.
> It depends on a lot of circumstances.
>
> There is the example of xclock which is small with X11 .so, and quite
> large otherwise. And there, for the case of a "clock", shared
> libraries clearly reduce disk footprint:
> xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..
>
> Sadly, the picture changes at run time: clock on plan 9 is 128k in
> memory, xclock is 4.2M RSS and 10M VSZ.
> Sic transit gloria .so. Of course, then we hear that "well, all that
> is shared". Hmm. Prove it.
>
> FWIW, the whole issue of goodness/badness of shared libraries has
> never been systematically measured as far as I know -- in terms of
> performance cost, overhead, throughput, the usual suspects -- or at
> least
> I've never seen it done. It would be a lot of work to do it correctly.
> Might be interesting.
>
>> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes
>
> So the compressed version is only 50% larger than acme. The point being?
>
>> because the bigger part of the uncompressed 1,585,152 bytes is redundant
>> filler required by the binary format specification.
>
> Does the redundant litter cross the network when I have it mounted via
> a share and execute the program or not? That was the original
> discussion.
>
>> The same happens for the
>> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
>> binary format and ELF both require the executable image to contain all
>> initialized but essentially redundant (i.e. zeroed) parts of the data
>> segement. Don't pretend you didn't know that.
>
> Wrong for ELF.
>
> Simple example from /bin/cat:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   LOAD           0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW
> 0x10000
>
> note that filesize is < memory size. I'll let you figure out how it works.
>
> This is the problem with your posts. You sound very authoritative, and
> I'm sure you figure you know all these bits,
> but you're wrong so often. It's confusing. It would be unfortunate
> were people to believe you are as right as you think you are.
>
>>
>> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
>> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
>> you have been unfair. No kidding.
>>
>
> Ah! Now we're into feature comparisons! I'm game. How did you get to 1
> in 5 and not 1 in 4.8, or 1 in 6?
> Of course, you have to take into account that acme's extensions live
> in /bin or your number is bogus.
>
> ron
>

Re: [9fans] mmap and shared libraries

[john]

>> less is more.
>
> If you say so, sir, it must be true. Is it also true that the less I
> understand of your comment the more meaningful it becomes?
>

Judging by your posts, the less you know, the more meaningful you
consider your opinion, so I'd agree with you here.

John

Re: [9fans] mmap and shared libraries

[Skip Tavakkolian]

> Is it also true that the less I
> understand

it's as if Choate's twin has come to visit.

Re: [9fans] mmap and shared libraries

[ron minnich]

On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
<eris.discordia@gmail.com> wrote:
>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
>> about 1/2 M. Vim on DOS is 3x larger? impressive.
>
> My intent was, of course, to show your comparison is baseless. It seems you
> still haven't realized that. You think Plan 9 is great?

Actually, to keep it simple, I was trying to make a simple point, with
a bit of humor intended, and not actually even directed at you,: that
code built with shared libraries is not
necessarily, or always, smaller than code built otherwise, for
programs of similar capability. I don't have an emacs
linked statically, however, so I took the nearest materials at hand.
It depends on a lot of circumstances.

There is the example of xclock which is small with X11 .so, and quite
large otherwise. And there, for the case of a "clock", shared
libraries clearly reduce disk footprint:
xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..

Sadly, the picture changes at run time: clock on plan 9 is 128k in
memory, xclock is 4.2M RSS and 10M VSZ.
Sic transit gloria .so. Of course, then we hear that "well, all that
is shared". Hmm. Prove it.

FWIW, the whole issue of goodness/badness of shared libraries has
never been systematically measured as far as I know -- in terms of
performance cost, overhead, throughput, the usual suspects -- or at
least
I've never seen it done. It would be a lot of work to do it correctly.
Might be interesting.

> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes

So the compressed version is only 50% larger than acme. The point being?

> because the bigger part of the uncompressed 1,585,152 bytes is redundant
> filler required by the binary format specification.

Does the redundant litter cross the network when I have it mounted via
a share and execute the program or not? That was the original
discussion.

> The same happens for the
> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
> binary format and ELF both require the executable image to contain all
> initialized but essentially redundant (i.e. zeroed) parts of the data
> segement. Don't pretend you didn't know that.

Wrong for ELF.

Simple example from /bin/cat:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW  0x10000

note that filesize is < memory size. I'll let you figure out how it works.

This is the problem with your posts. You sound very authoritative, and
I'm sure you figure you know all these bits,
but you're wrong so often. It's confusing. It would be unfortunate
were people to believe you are as right as you think you are.

>
> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So you
> have been unfair. No kidding.
>

Ah! Now we're into feature comparisons! I'm game. How did you get to 1
in 5 and not 1 in 4.8, or 1 in 6?
Of course, you have to take into account that acme's extensions live
in /bin or your number is bogus.

ron

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> less is more.

If you say so, sir, it must be true. Is it also true that the less I
understand of your comment the more meaningful it becomes?

--On Wednesday, November 05, 2008 1:12 PM -0800 Rob Pike
<robpike@gmail.com> wrote:

> less is more.
>
> -rob
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Eris, did you just post the following to slashdot? s/OpenBSD/Plan
> 9/;s/Theo/9whacko/ and we've got your entire posting history on this
> mailing list. the similarity is uncanny.

I don't think the question is serious but I'm very bad at detecting
sarcasm--too many false positives. So I'll provide an answer: no.

By the way, comparing Theo de Raadt to 9people is really mean... towards
the 9people. He _is_ a tyrant, even though absolutely apt. The 9people are
far more tolerant and well-behaved. Comparing OpenBSD to Plan 9 is also
really mean... towards OpenBSD.

--On Wednesday, November 05, 2008 2:09 PM -0700 andrey mirtchovski
<mirtchovski@gmail.com> wrote:

> Eris, did you just post the following to slashdot? s/OpenBSD/Plan
> 9/;s/Theo/9whacko/ and we've got your entire posting history on this
> mailing list. the similarity is uncanny.
>
> "Yeah. I'd really like to like OpenBSD. Technically, it's superb. It's
> smooth, polished, well documented --- it's got a level of consistency
> that most Linux distros can only hope to dream of. The kernel is well
> designed and fast, with excellent hardware support. System setup is
> consistent and well-thought out. Above all, it doesn't confuse
> easy-to-use with easy-to-learn --- everything is as simple as possible
> without oversimplifying, which makes it a joy to admin.
>
> But then, every time I try to use it, I run up against the OpenBSD
> developers, who are an arrogant bunch of elitist assholes. In a couple
> of years, on and off, I think I've seen Theo make a civil reply to
> someone *once*. Maybe twice. No, I'm not kidding. When you see someone
> ask what looks to my untutored eye a reasonable question about VMs,
> and the head developer replies publicly with the words 'You are full
> of shit' and nothing else (apart from a complete copy of the original
> message, no snipping), there is something very wrong. Most of the
> other devs are nearly as bad, and of course there are hordes of
> groupies who assume that if the people in charge are okay with
> personal abuse, then it's alright for them, too.
>
> Despite this, the actual operating system is definitely worth checking
> out if you're interested in what a well-designed Unix actually looks
> like. Linux can learn a lot from it."
>

Re: [9fans] mmap and shared libraries

[Rob Pike]

less is more.

-rob

Re: [9fans] mmap and shared libraries

[andrey mirtchovski]

Eris, did you just post the following to slashdot? s/OpenBSD/Plan
9/;s/Theo/9whacko/ and we've got your entire posting history on this
mailing list. the similarity is uncanny.

"Yeah. I'd really like to like OpenBSD. Technically, it's superb. It's
smooth, polished, well documented --- it's got a level of consistency
that most Linux distros can only hope to dream of. The kernel is well
designed and fast, with excellent hardware support. System setup is
consistent and well-thought out. Above all, it doesn't confuse
easy-to-use with easy-to-learn --- everything is as simple as possible
without oversimplifying, which makes it a joy to admin.

But then, every time I try to use it, I run up against the OpenBSD
developers, who are an arrogant bunch of elitist assholes. In a couple
of years, on and off, I think I've seen Theo make a civil reply to
someone *once*. Maybe twice. No, I'm not kidding. When you see someone
ask what looks to my untutored eye a reasonable question about VMs,
and the head developer replies publicly with the words 'You are full
of shit' and nothing else (apart from a complete copy of the original
message, no snipping), there is something very wrong. Most of the
other devs are nearly as bad, and of course there are hordes of
groupies who assume that if the people in charge are okay with
personal abuse, then it's alright for them, too.

Despite this, the actual operating system is definitely worth checking
out if you're interested in what a well-designed Unix actually looks
like. Linux can learn a lot from it."

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
> about 1/2 M. Vim on DOS is 3x larger? impressive.

My intent was, of course, to show your comparison is baseless. It seems you
still haven't realized that. You think Plan 9 is great? Sure you know a lot
more about it than I do so I think you are entitled to your opinion, but
drawing a baseless analogy and ridiculing other OS's--as is common on this
mailing list--won't help your cause.

I didn't post a listing for a DOS executable. Vim running under cmd.exe
(vim.exe) is a normal 32-bit Windows process, only with output to Windows
"console." The little DOS in every popular Windows ceased to exist like 8
years ago.

The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes
because the bigger part of the uncompressed 1,585,152 bytes is redundant
filler required by the binary format specification. The same happens for
the vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows
PE binary format and ELF both require the executable image to contain all
initialized but essentially redundant (i.e. zeroed) parts of the data
segement. Don't pretend you didn't know that.

Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
you have been unfair. No kidding.

--On Wednesday, November 05, 2008 10:15 AM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia <eris.discordia@gmail.com>
> wrote:
>> [root@host ~/clms]# ls -l `which vim`
>> -rwxr-xr-x  1 root  wheel  1221212 Oct 15  2006 /usr/local/bin/vim
>>
>> ------------------------------------------------------------------------
>> --------
>>
>> C:\Program Files (x86)\Vim\vim71>dir gvim.exe
>> Volume in drive C has no label.
>> Volume Serial Number is B001-B4A3
>>
>> Directory of C:\Program Files (x86)\Vim\vim71
>>
>> 05/12/2007  12:19 PM         1,585,152 gvim.exe
>>              1 File(s)      1,585,152 bytes
>>              0 Dir(s)   5,075,197,952 bytes free
>>
>> C:\Program Files (x86)\Vim\vim71>dir vim.exe
>> Volume in drive C has no label.
>> Volume Serial Number is B001-B4A3
>>
>> Directory of C:\Program Files (x86)\Vim\vim71
>>
>> 05/12/2007  12:14 PM         1,372,160 vim.exe
>>              1 File(s)      1,372,160 bytes
>>              0 Dir(s)   5,075,197,952 bytes free
>>
>> C:\Program Files (x86)\Vim\vim71>
>>
>> ------------------------------------------------------------------------
>> --------
>>
>> So what? (and the two latter were _Windows_ binaries).
>>
>>> I realize that is utterly unfair. Sort of.
>>
>> Nice of you to realize that. Sort of.
>>
>
> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
> about 1/2 M. Vim on DOS is 3x larger? impressive.
>
> Ron
>

Re: [9fans] mmap and shared libraries

[Rob Pike]

don't forgot that plan 9 binaries are fully linked while most other
systems pull in more code through dynamic linking when the binary is
executed.

-rob

Re: [9fans] mmap and shared libraries

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]

On Wed, Nov 5, 2008 at 10:15 AM, ron minnich <rminnich@gmail.com> wrote:

> On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia <eris.discordia@gmail.com>
> wrote:
> > [root@host ~/clms]# ls -l `which vim`
> > -rwxr-xr-x  1 root  wheel  1221212 Oct 15  2006 /usr/local/bin/vim
> >
> >
> --------------------------------------------------------------------------------
> >
> > C:\Program Files (x86)\Vim\vim71>dir gvim.exe
> > Volume in drive C has no label.
> > Volume Serial Number is B001-B4A3
> >
> > Directory of C:\Program Files (x86)\Vim\vim71
> >
> > 05/12/2007  12:19 PM         1,585,152 gvim.exe
> >              1 File(s)      1,585,152 bytes
> >              0 Dir(s)   5,075,197,952 bytes free
> >
> > C:\Program Files (x86)\Vim\vim71>dir vim.exe
> > Volume in drive C has no label.
> > Volume Serial Number is B001-B4A3
> >
> > Directory of C:\Program Files (x86)\Vim\vim71
> >
> > 05/12/2007  12:14 PM         1,372,160 vim.exe
> >              1 File(s)      1,372,160 bytes
> >              0 Dir(s)   5,075,197,952 bytes free
> >
> > C:\Program Files (x86)\Vim\vim71>
> >
> >
> --------------------------------------------------------------------------------
> >
> > So what? (and the two latter were _Windows_ binaries).
> >
> >> I realize that is utterly unfair. Sort of.
> >
> > Nice of you to realize that. Sort of.
> >
>
> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
> about 1/2 M. Vim on DOS is 3x larger? impressive.
>
> Ron
>
> How much of that is library support code to make it work? :-)

[-- Attachment #2: Type: text/html, Size: 2388 bytes --]

Re: [9fans] mmap and shared libraries

[ron minnich]

On Wed, Nov 5, 2008 at 2:57 AM, Eris Discordia <eris.discordia@gmail.com> wrote:
> [root@host ~/clms]# ls -l `which vim`
> -rwxr-xr-x  1 root  wheel  1221212 Oct 15  2006 /usr/local/bin/vim
>
> --------------------------------------------------------------------------------
>
> C:\Program Files (x86)\Vim\vim71>dir gvim.exe
> Volume in drive C has no label.
> Volume Serial Number is B001-B4A3
>
> Directory of C:\Program Files (x86)\Vim\vim71
>
> 05/12/2007  12:19 PM         1,585,152 gvim.exe
>              1 File(s)      1,585,152 bytes
>              0 Dir(s)   5,075,197,952 bytes free
>
> C:\Program Files (x86)\Vim\vim71>dir vim.exe
> Volume in drive C has no label.
> Volume Serial Number is B001-B4A3
>
> Directory of C:\Program Files (x86)\Vim\vim71
>
> 05/12/2007  12:14 PM         1,372,160 vim.exe
>              1 File(s)      1,372,160 bytes
>              0 Dir(s)   5,075,197,952 bytes free
>
> C:\Program Files (x86)\Vim\vim71>
>
> --------------------------------------------------------------------------------
>
> So what? (and the two latter were _Windows_ binaries).
>
>> I realize that is utterly unfair. Sort of.
>
> Nice of you to realize that. Sort of.
>

yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
about 1/2 M. Vim on DOS is 3x larger? impressive.

Ron

Re: [9fans] mmap and shared libraries

[roger peppe]

On Wed, Nov 5, 2008 at 4:34 PM, Abhishek Kulkarni <abbyzcool@gmail.com> wrote:
> term% cat pipeto.eris
> /bin/upas/filter -h $1 $2 'From: "Eris Discordia"' /dev/null

personally, i think eris makes a lot of reasonable points.

Re: [9fans] mmap and shared libraries

[Abhishek Kulkarni]

you may take this to comp.os.ms-windows.advocacy if you want.

term% cat pipeto.eris
/bin/upas/filter -h $1 $2 'From: "Eris Discordia"' /dev/null

>
> > I realize that is utterly unfair. Sort of.
>
> Nice of you to realize that. Sort of.
>
> --On Tuesday, November 04, 2008 9:21 PM -0800 ron minnich
>
> <rminn...@gmail.com> wrote:
> > On Tue, Nov 4, 2008 at 9:17 PM, Roman Shaposhnik <r...@sun.com> wrote:
>
> >> A standalone statically linked binary is going to be considerable larger
> >> while
> >> in flight over data links.
>
> > ah yes well maybe.
>
> > rminn...@login3.surveyor:~/mtelnet> ls -l `which emacs`
> > -rwxr-xr-t 1 root root 4587528 2006-06-17 06:59 /usr/bin/emacs
> > rminn...@login3.surveyor:~/mtelnet> ls -l `which acme`
> > -rwxr-xr-x 1 rminnich users 1361257 2008-11-04 12:39
> > /home/rminnich/plan9/bin/acme
> > rminn...@login3.surveyor:~/mtelnet>
>
> > I realize that is utterly unfair. Sort of.
>
> > ron
>
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Why do gods that walk the earth invariably act like spoilt brats? Ah,
> hang on ...

Prolly because a god is only a human's conceited ego. Oh, wait...

--On Wednesday, November 05, 2008 10:55 AM +0000 Robert Raschke
<rtrlists@googlemail.com> wrote:

> On Wed, Nov 5, 2008 at 10:33 AM, Eris Discordia
> <eris.discordia@gmail.com> wrote:
>>> Man's got to know his limitations.
>>
>> Yes, _man_ has got to. That doesn't apply to deities :-P
>
> Why do gods that walk the earth invariably act like spoilt brats? Ah,
> hang on ...
>

Re: [9fans] mmap and shared libraries

[Eris Discordia]

[root@host ~/clms]# ls -l `which vim`
-rwxr-xr-x  1 root  wheel  1221212 Oct 15  2006 /usr/local/bin/vim

--------------------------------------------------------------------------------

C:\Program Files (x86)\Vim\vim71>dir gvim.exe
 Volume in drive C has no label.
 Volume Serial Number is B001-B4A3

 Directory of C:\Program Files (x86)\Vim\vim71

05/12/2007  12:19 PM         1,585,152 gvim.exe
               1 File(s)      1,585,152 bytes
               0 Dir(s)   5,075,197,952 bytes free

C:\Program Files (x86)\Vim\vim71>dir vim.exe
 Volume in drive C has no label.
 Volume Serial Number is B001-B4A3

 Directory of C:\Program Files (x86)\Vim\vim71

05/12/2007  12:14 PM         1,372,160 vim.exe
               1 File(s)      1,372,160 bytes
               0 Dir(s)   5,075,197,952 bytes free

C:\Program Files (x86)\Vim\vim71>

--------------------------------------------------------------------------------

So what? (and the two latter were _Windows_ binaries).

> I realize that is utterly unfair. Sort of.

Nice of you to realize that. Sort of.

--On Tuesday, November 04, 2008 9:21 PM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Tue, Nov 4, 2008 at 9:17 PM, Roman Shaposhnik <rvs@sun.com> wrote:
>
>> A standalone statically linked binary is going to be considerable larger
>> while
>> in flight over data links.
>>
>
> ah yes well maybe.
>
> rminnich@login3.surveyor:~/mtelnet> ls -l `which emacs`
> -rwxr-xr-t 1 root root 4587528 2006-06-17 06:59 /usr/bin/emacs
> rminnich@login3.surveyor:~/mtelnet> ls -l `which acme`
> -rwxr-xr-x 1 rminnich users 1361257 2008-11-04 12:39
> /home/rminnich/plan9/bin/acme
> rminnich@login3.surveyor:~/mtelnet>
>
> I realize that is utterly unfair. Sort of.
>
>
>
> ron
>

Re: [9fans] mmap and shared libraries

[Robert Raschke]

On Wed, Nov 5, 2008 at 10:33 AM, Eris Discordia
<eris.discordia@gmail.com> wrote:
>> Man's got to know his limitations.
>
> Yes, _man_ has got to. That doesn't apply to deities :-P

Why do gods that walk the earth invariably act like spoilt brats? Ah,
hang on ...

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> Man's got to know his limitations.

Yes, _man_ has got to. That doesn't apply to deities :-P

--On Tuesday, November 04, 2008 7:59 AM -0800 ron minnich
<rminnich@gmail.com> wrote:

> On Tue, Nov 4, 2008 at 7:06 AM, Eris Discordia <eris.discordia@gmail.com>
> wrote:
>
>> Attempts to live boot Plan 9 on the same machine fail because
>> some 9wacko believes CD-ROM drives must be secondary master or
>> something--and I won't move a jumper to suit a 9wacko's whim; not that
>> I've ever been asked to do that.
>>
>
> yep, it's a hackers system. You get to fix it or whine about it, your
> call. Although I think you've made your preferences clear.
>
> Man's got to know his limitations.
>
> ron
>

Re: [9fans] mmap and shared libraries

[Lyndon Nerenberg]

> And what is a 9whacko?

Someone without a mirror? And not enough time to think?

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

And what is a 9whacko? I didn't see any at IWP9 but I didn't have a mirror.

brucee

On Wed, Nov 5, 2008 at 10:32 AM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
>> And who is this Eris dick?
>
> Just a simple first year regexp assignment.
>
>
>

Re: [9fans] mmap and shared libraries

[Lyndon Nerenberg]

>And who is this Eris dick?

Just a simple first year regexp assignment.

Re: [9fans] mmap and shared libraries

[Bruce Ellis]

grep whining /sys/games/lib/fortunes

Why do I have to send this mail every year or so? And who is this Eris dick?

brucee

On Tue, Nov 4, 2008 at 5:59 PM, ron minnich <rminnich@gmail.com> wrote:
> On Tue, Nov 4, 2008 at 7:06 AM, Eris Discordia <eris.discordia@gmail.com> wrote:
>
>> Attempts to live boot Plan 9 on the same machine fail because
>> some 9wacko believes CD-ROM drives must be secondary master or
>> something--and I won't move a jumper to suit a 9wacko's whim; not that I've
>> ever been asked to do that.
>>
>
> yep, it's a hackers system. You get to fix it or whine about it, your
> call. Although I think you've made your preferences clear.
>
> Man's got to know his limitations.
>
> ron
>
>

Re: [9fans] mmap and shared libraries

[Lyndon Nerenberg]

>> A standalone statically linked binary is going to be considerable larger
>> while
>> in flight over data links.

But that static binary only flies once, geting sucked into memory
with a (mostly) simple bcopy equiv at process launch time. Shared
memory regimes thrash the living daylights out of MMUs and PTLBs, on each
and every context switch.

Re: [9fans] mmap and shared libraries

[ron minnich]

On Tue, Nov 4, 2008 at 9:17 PM, Roman Shaposhnik <rvs@sun.com> wrote:

> A standalone statically linked binary is going to be considerable larger
> while
> in flight over data links.
>

ah yes well maybe.

rminnich@login3.surveyor:~/mtelnet> ls -l `which emacs`
-rwxr-xr-t 1 root root 4587528 2006-06-17 06:59 /usr/bin/emacs
rminnich@login3.surveyor:~/mtelnet> ls -l `which acme`
-rwxr-xr-x 1 rminnich users 1361257 2008-11-04 12:39
/home/rminnich/plan9/bin/acme
rminnich@login3.surveyor:~/mtelnet>

I realize that is utterly unfair. Sort of.



ron

Re: [9fans] mmap and shared libraries

[Roman Shaposhnik]

On Nov 3, 2008, at 5:16 AM, dave.l@mac.com wrote:
> A thought ...
>
> Shared libraries do 2 possibly useful things:
> 1) save space
> 2) stop you having to re-link when a new library is released.
>
> Now 2) doesn't really happen anyway, due to .so versioning hell,
> so we're left with 1) ...
>
> I know it's kind-of hacky and unstructured (how do you know the
> venti block size?),
> but could you not block-align the library members in the .a,
> so venti would do the sharing on disk for you?
>
> As a further step, you could also page align the text segments of
> the library members,
> then implement a venti-like scheme for read-only pages in memory,
> so that when the executable loads, it gets any common library pages
> shared.

I was thinking about exactly the same approach when I was
contemplating the
evils of dynamic linking. It seems that if you link statically and you
use FS
to help you coalesce identical blocks (on storage devices and in RAM)
you get
all of the potential benefits of a dynamically linked binary except
one -- distribution.
A standalone statically linked binary is going to be considerable
larger while
in flight over data links.

> P.S. Sorry not to make Volos: where are the rest of the photos?

Yeah -- where are they?!?! ;-)

Thanks,
Roman.

Re: [9fans] mmap and shared libraries

[ron minnich]

On Tue, Nov 4, 2008 at 7:06 AM, Eris Discordia <eris.discordia@gmail.com> wrote:

> Attempts to live boot Plan 9 on the same machine fail because
> some 9wacko believes CD-ROM drives must be secondary master or
> something--and I won't move a jumper to suit a 9wacko's whim; not that I've
> ever been asked to do that.
>

yep, it's a hackers system. You get to fix it or whine about it, your
call. Although I think you've made your preferences clear.

Man's got to know his limitations.

ron

Re: [9fans] mmap and shared libraries

[Eris Discordia]

> I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of
> memory FreeBSD is paging nightmare, despite it's wonderfully complex
> shared library environment.

You're wrong. Case in point: my FreeBSD 6.2-RELEASE installation on a 233
MHz PII (one of those Slot 1 processors) with 128 MB of RAM runs just fine
and serves well the purpose of 24x7 downloading and serving FTP; home
scale, of course. Attempts to live boot Plan 9 on the same machine fail
because some 9wacko believes CD-ROM drives must be secondary master or
something--and I won't move a jumper to suit a 9wacko's whim; not that I've
ever been asked to do that.

--On Monday, November 03, 2008 5:23 PM -0800 Lyndon Nerenberg
<lyndon@orthanc.ca> wrote:

>> A thought ...
>>
>> Shared libraries do 2 possibly useful things:
>> 1) save space
>> 2) stop you having to re-link when a new library is released.
>>
>> Now 2) doesn't really happen anyway, due to .so versioning hell,
>> so we're left with 1) ...
>
> I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of
> memory FreeBSD is paging nightmare, despite it's wonderfully complex
> shared library environment. Oh, and Solaris won't even boot with less 512
> MB these days.
>
> There is no space shortage on Plan 9 that's looking for a solution.
>

Re: [9fans] mmap and shared libraries

[michael block]

On Mon, Nov 3, 2008 at 07:16,  <dave.l@mac.com> wrote:
> A thought ...
>
> Shared libraries do 2 possibly useful things:
> 1) save space
> 2) stop you having to re-link when a new library is released.

i can see how relinks are painful with gnu-style build systems where
you need to run ./configure and recurse thru endless Makefiles. but
under the much cleaner plan 9 source tree, such things take seconds.
plan 9 doesn't really have that many libraries either. my (probably
naïve) impression is that much of what can be done with dynamically
linked libraries can be done with pipes or layered filesystems

also linking at loadtime or runtime slows things down. you might not
notice interactively, but in shell scripts you certainly would. that
is one of the reasons why the unix world uses shell scripts to a
lesser extent. there are other advantages to static linking, but they
slip my mind for now

losing the advantages of static linking in order to duplicate
functionality that can be gotten from pipes and filesystems doesn't
seem like such a good tradeoff to me

-- m

Re: [9fans] mmap and shared libraries

[Lyndon Nerenberg]

> A thought ...
>
> Shared libraries do 2 possibly useful things:
> 1) save space
> 2) stop you having to re-link when a new library is released.
>
> Now 2) doesn't really happen anyway, due to .so versioning hell,
> so we're left with 1) ...

I can run Plan 9 quite nicely in 128 MB of RAM. In the same amount of
memory FreeBSD is paging nightmare, despite it's wonderfully complex
shared library environment. Oh, and Solaris won't even boot with less 512
MB these days.

There is no space shortage on Plan 9 that's looking for a solution.

Re: [9fans] mmap and shared libraries

[Kernel Panic]

dave.l@mac.com wrote:
> A thought ...
>
> Shared libraries do 2 possibly useful things:
> 1) save space
> 2) stop you having to re-link when a new library is released.
>
> Now 2) doesn't really happen anyway, due to .so versioning hell,
> so we're left with 1) ...
>
> I know it's kind-of hacky and unstructured (how do you know the venti
> block size?),
> but could you not block-align the library members in the .a,
> so venti would do the sharing on disk for you?

Look at this:
http://gsoc.cat-v.org/people/mjl/blog//2007-08-06-1_Rabin_fingerprints

> As a further step, you could also page align the text segments of the
> library members,
> then implement a venti-like scheme for read-only pages in memory,
> so that when the executable loads, it gets any common library pages
> shared.
>
> Dave.
>
> P.S. Sorry not to make Volos: where are the rest of the photos?
>

Re: [9fans] mmap and shared libraries

[dave.l]

A thought ...

Shared libraries do 2 possibly useful things:
1) save space
2) stop you having to re-link when a new library is released.

Now 2) doesn't really happen anyway, due to .so versioning hell,
so we're left with 1) ...

I know it's kind-of hacky and unstructured (how do you know the venti
block size?),
but could you not block-align the library members in the .a,
so venti would do the sharing on disk for you?

As a further step, you could also page align the text segments of the
library members,
then implement a venti-like scheme for read-only pages in memory,
so that when the executable loads, it gets any common library pages
shared.

Dave.

P.S. Sorry not to make Volos: where are the rest of the photos?