9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* Re: [9fans] Virtual memory & paging
@ 2002-02-04 10:38 geoff
  2002-02-04 11:16 ` Boyd Roberts
                   ` (2 more replies)
  0 siblings, 3 replies; 30+ messages in thread
From: geoff @ 2002-02-04 10:38 UTC (permalink / raw)
  To: 9fans

There isn't a copy of the entire C library in every binary.  There is
a copy of each library routine called, directly or indirectly, by the
program in question.

Sharing of instructions is done at the granularity of process text
segments, as in V6 or V7 Unix.  The text segment of a process that
forks is shared between parent and child by page mapping.  Also,
running (via exec) a program multiple times concurrently causes the
(pure) text segment to be shared by page mapping across those
processes.  So all copies of rc and on a machine should share a text
segment.

Given that degree of sharing, the low cost of RAM, and the increase in
OS complexity, slowness and insecurity in the implementations of
dynamic libraries that I've seen, I don't see a need for dynamic
libraries.  (Remember that the real impetus for adding them to Unix
was X11 and its big and badly-factored libraries, which most of us
aren't blessed with.)  My terminal has 115 processes; all but 4 of
them share their text segment with at least one other process, usually
more.  74 of them are instances of rio, Mail, rc, acme, listen,
plumber and samterm.  A CPU server has 141 processes; all but 2 share
text.  80 of them are listen, another 21 are rc, exportfs, kfs, dns
and consolefs.  A quick sampling suggests that Plan 9 programs are
typically smaller than FreeBSD/386 programs even with shared
libraries.  Here are some FreeBSD sizes:

: unix; size /bin/cat /bin/ed /usr/bin/awk /usr/X11/bin/sam
   text    data     bss     dec     hex filename
  54188    4324    9760   68272   10ab0 /bin/cat
 122835    8772   81920  213527   34217 /bin/ed
 135761    4772   15756  156289   26281 /usr/bin/awk
  52525    1412   53448  107385   1a379 /usr/X11/bin/sam

Of those, awk and sam use shared libraries.  The corresponding Plan 9
sizes are:

; cd /bin; size cat ed awk sam
15996t + 2208d + 944b = 19148	cat
45964t + 4212d + 41232b = 91408	ed
114731t + 35660d + 12040b = 162431	awk
86574t + 7800d + 66240b = 160614	sam

and the Plan 9 programs cope with Unicode and UTF.



^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-15 14:32 Richard Uhtenwoldt
  2002-02-16 21:56 ` Ronald G Minnich
  0 siblings, 1 reply; 30+ messages in thread
From: Richard Uhtenwoldt @ 2002-02-15 14:32 UTC (permalink / raw)
  To: Ronald G Minnich; +Cc: 9fans

Geoff Collyer:

> > Given that degree of sharing, the low cost of RAM, and the increase in
> > OS complexity, slowness and insecurity in the implementations of
> > dynamic libraries that I've seen, I don't see a need for dynamic
> > libraries.

Ron Minnich

>and in the worst case you'll end up where GNU is now, with symbol
>versioning, 21 different versions of opendir in glibc, and so on an so on.
>Watching a simple 'ls' do hundreds of symbol fixups is really
>enlightening. Especially when so many of them are for the same symbol with
>slight variations on the name. My simple, formerly working, libc-based
>private name spaces are still totally hosed due to this nonsense.

Totally hosed because Linux moved from libc5 to glibc?

If so, you should have tried to fight the move.  Very little chance you
could have influenced *all* Linux users to stay with libc5, but you
easily could have influenced *me* to do so --you would have had to write
only one paragraph-- and probably a few hundred other Linux users, at
least for a year or 2 or 3, till the apps start not working with libc5.

I didn't know about your work on private namespaces for Linux until
about 3-6 months ago.  Can't recall exactly when.  I think I saw an
announce on Slashdot and short afterward you mentioned it here.

I subscribed to 9fans to find out about 9-coolness that gets added
to Linux and *BSD.
Where on the Net could I have found out earlier about your Linux work?


^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-05 10:57 geoff
  2002-02-05 11:37 ` Boyd Roberts
  2002-02-05 14:01 ` david presotto
  0 siblings, 2 replies; 30+ messages in thread
From: geoff @ 2002-02-05 10:57 UTC (permalink / raw)
  To: 9fans

I didn't say that "we can't write an implementation without bugs",
merely that the existing implementations I'm familiar with have
made the system slower (e.g., adding shared libraries in SunOS 4.0
hugely slowed down the fork system call of even the smallest possible process),
added complexity to the system and for the users (more kernel code,
two sets of libraries [shared and unshared], static vs. dynamic link
decisions, run-time loader paths, extra process start-up processing),
usually coped poorly with set-id programs, and the extra complexity made
the system more fragile in single-user mode (you need all the right
shared libraries on the root filesystem) and when manipulating libraries
generally.  So I don't believe I've seen an existence proof that shared
libraries can do more good than harm; maybe they can, but given the
evidence to date, I'm skeptical.  And given that we (outside the Labs anyway)
don't have the problem that made people add shared libraries to Unix, X,
and that our binaries are at least competitive in size with systems that do
have shared libraries, and that one can just add RAM cheaply to avoid swapping
anyway, why bother?  Adding unneeded goo like shared libraries to a system
just makes it bigger, at minimum.  Depending upon the quality of the implementation,
it may also make it slower, more complex and fragile, and create new security holes.

Such a deal.


^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-04 11:03 Fco.J.Ballesteros
  0 siblings, 0 replies; 30+ messages in thread
From: Fco.J.Ballesteros @ 2002-02-04 11:03 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 68 bytes --]

It still uses to be made; at least that's what I tell my students.

[-- Attachment #2: Type: message/rfc822, Size: 1557 bytes --]

From: forsyth@caldo.demon.co.uk
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Virtual memory & paging
Date: Mon, 4 Feb 2002 10:59:30 0000
Message-ID: <20020204110139.588D119999@mail.cse.psu.edu>

	I don't know that anyone else makes this distinction, but to me
	virtual memory is a technique an operating system can use to manage
	user memory, while paging is a technique for coping with a shortfall
	in physical memory.  VM manages memory by using page faults to fill in

it's a distinction that certainly used to be made when teaching operating systems.

^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-04 10:59 forsyth
  0 siblings, 0 replies; 30+ messages in thread
From: forsyth @ 2002-02-04 10:59 UTC (permalink / raw)
  To: 9fans

	I don't know that anyone else makes this distinction, but to me
	virtual memory is a technique an operating system can use to manage
	user memory, while paging is a technique for coping with a shortfall
	in physical memory.  VM manages memory by using page faults to fill in

it's a distinction that certainly used to be made when teaching operating systems.



^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-04 10:30 forsyth
  0 siblings, 0 replies; 30+ messages in thread
From: forsyth @ 2002-02-04 10:30 UTC (permalink / raw)
  To: 9fans

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

there isn't a copy of the library in every executable because
the linker includes from the library only the functions the
program uses (directly or indirectly).   this is typically
smaller than the whole library.  there would be scope for some sharing
but the gain dynamically is typically small: not all the
executables in /386/bin are in use simultaneously.   after all, it's hard
to buy a machine with less than 64mbytes these days
and with plan 9 running rather a lot of that is free space, unless
you've got big memory applications such as ghostscript in which case
it's the application rather than the library that consumes the space.
i suppose if you'd got an enormous application library shared
by several applications in use at once you'd see more need
for library sharing.  in plan 9, that shared functionality might
be better represented as a file server (though not always).
dynamic linking might be of more interest (in Plan 9)
for selection amongst a set of implementations at run-time.


[-- Attachment #2: Type: message/rfc822, Size: 2323 bytes --]

To: 9fans@cse.psu.edu
Subject: Re: [9fans] Virtual memory & paging
Date: Sun, 3 Feb 2002 22:21:45 -0800
Message-ID: <20020203222145.A22998@ohio.river.org>

On Sun, Feb 03, 2002 at 03:08:39PM -0800, geoff@collyer.net wrote:
> Even if your physical memory were as large as your virtual address
> space, an obvious advantage to mapping addresses is [...]

> Another potential use of mapping is to avoid copying data by remapping
> pages.

Plan 9 has no DLLs.  It has a C runtime library.  I'm sure it's smaller
than the GNU C library, but is there a copy of the libary in every
executable in /bin?  

is there a copy of it in every
process's memory or is it shared among all the processses?

if so, is memory mapping how the sharing is effected?

(I know this is undergrad material.  Let me know if I should stop
asking elementary questions here.)

^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-03 23:08 geoff
  2002-02-03 20:26 ` Andrew Simmons
  2002-02-04  6:21 ` Richard Uhtenwoldt
  0 siblings, 2 replies; 30+ messages in thread
From: geoff @ 2002-02-03 23:08 UTC (permalink / raw)
  To: 9fans

Even if your physical memory were as large as your virtual address
space, an obvious advantage to mapping addresses is to permit programs
linked to start at a fixed address (usually the second page of the
address space) to be run without first running a loading pass to
relocate all the addresses in the entire program.

In addition, memory management units normally provide varying forms of
protection for pages (e.g., instructions are usually made read- or
execute-only) and checking for out-of-bounds (unmapped) addresses.  On
many machines, notably the PC, a read from an address with no
corresponding memory (i.e., out-of-bounds) yields garbage; with memory
management on, many such references can be caught.  The MMU can thus
be used to ensure that a program doesn't access data in other programs
(including the kernel) without their consent.

Another potential use of mapping is to avoid copying data by remapping
pages.

I would turn the question around: when would you want a one-to-one
mapping of virtual to physical addresses?  There are a few such cases:
if memory serves, the old VAX 750 Plan 9 file server ran with the MMU
off because it runs no user-mode processes and it was found that
running with the MMU off made the machine run much faster.



^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-03 21:53 presotto
  2002-02-03 22:36 ` Andrew Simmons
  0 siblings, 1 reply; 30+ messages in thread
From: presotto @ 2002-02-03 21:53 UTC (permalink / raw)
  To: 9fans

Virtual memory covers any memory that isn't laid out
exactly as it seems to an executing program, i.e., that
there exists some mapping between addresses as seen by the
execution unit and those on the bus to physical memory.

The mapping can happen a number of ways.  The two more
popular are via paging hardware or segmentation hardware.
Pages are typically fixed size and segments variable size.
 From the PDP-11 to the MIPS there have been enough flavors
that the distinction is a bit blurred.

If one is willing to move data between a backing store and
main memory in addition to chaging the mappings, one can
make the main memory appear larger than it really is.  This
is usually termed demand paging or demand segmentation.

An earlier technique, still in use in addition to demand paging/
segmentation is swapping.  If many processes are running on
a particular system, it may desirable to swap in main memory
one for another to give the illusion of a main memory that can
hold all of them.

And of course there is the even earlier technique of overlays.
Here one figures out the execution tree of routines in a
program.  One might note that routine A will never be an
ancestor of routine B or vice versa.  THerefore, there's no
requirement that they live in memory simultaneously and can
occupy the same memory locations, just at different times.


^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-03 21:21 rob pike
  2002-02-04 21:46 ` skipt
  0 siblings, 1 reply; 30+ messages in thread
From: rob pike @ 2002-02-03 21:21 UTC (permalink / raw)
  To: 9fans

I don't know that anyone else makes this distinction, but to me
virtual memory is a technique an operating system can use to manage
user memory, while paging is a technique for coping with a shortfall
in physical memory.  VM manages memory by using page faults to fill in
the user address space; paging is just swapping a page at a time.  The
relationship between these notions is primarily that both permit the
process to execute when it is not entirely resident.

By these definitions, it is possible (easy even) to build a system
with virtual memory but no paging.  Even I can do it.

-rob



^ permalink raw reply	[flat|nested] 30+ messages in thread
* Re: [9fans] Virtual memory & paging
@ 2002-02-03 21:12 andrey mirtchovski
  0 siblings, 0 replies; 30+ messages in thread
From: andrey mirtchovski @ 2002-02-03 21:12 UTC (permalink / raw)
  To: 9fans

virtual memory as a technique could be implemented using both 'demand paging' and 'demand segmentation' the most obvious difference between a page and a segment is that different segments could be of different sizes while pages have the same size (or set of sizes)..

hmm.. i just found the relevant textbook -- Operating Systems Concepts, 5th ed, (Silberschatz, Galvin) page 291



^ permalink raw reply	[flat|nested] 30+ messages in thread
* [9fans] Virtual memory & paging
@ 2002-02-03 21:01 Andrew Simmons
  0 siblings, 0 replies; 30+ messages in thread
From: Andrew Simmons @ 2002-02-03 21:01 UTC (permalink / raw)
  To: 9fans

I seem to remember some one saying here a while back that virtual memory
and paging were quite distinct, whereas I'd always vaguely assumed they
were the same thing. Could some one explain the difference to me, or point
me in the direction of enlightenment? The issue came up in a friend's job
interview recently, and he wrongly assumed the same as me.

Thanks



^ permalink raw reply	[flat|nested] 30+ messages in thread

end of thread, other threads:[~2002-03-30  5:46 UTC | newest]

Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-02-04 10:38 [9fans] Virtual memory & paging geoff
2002-02-04 11:16 ` Boyd Roberts
2002-02-04 11:45 ` Boyd Roberts
2002-02-04 17:10   ` Andrew Simmons
2002-02-05 11:17     ` Boyd Roberts
2002-03-30  4:26   ` Richard Maxwell Underwood
2002-03-30  5:46     ` [9fans] Uncluttering Richard Maxwell Underwood
2002-02-05  9:53 ` [9fans] Virtual memory & paging Thomas Bushnell, BSG
2002-02-05 16:06   ` Ronald G Minnich
  -- strict thread matches above, loose matches on Subject: below --
2002-02-15 14:32 Richard Uhtenwoldt
2002-02-16 21:56 ` Ronald G Minnich
2002-02-05 10:57 geoff
2002-02-05 11:37 ` Boyd Roberts
2002-02-05 14:01 ` david presotto
2002-02-04 11:03 Fco.J.Ballesteros
2002-02-04 10:59 forsyth
2002-02-04 10:30 forsyth
2002-02-03 23:08 geoff
2002-02-03 20:26 ` Andrew Simmons
2002-02-04 16:15   ` Ronald G Minnich
2002-02-04  6:21 ` Richard Uhtenwoldt
2002-02-03 21:53 presotto
2002-02-03 22:36 ` Andrew Simmons
2002-02-03 21:21 rob pike
2002-02-04 21:46 ` skipt
2002-02-04 22:11   ` Ronald G Minnich
     [not found]   ` <Pine.LNX.4.33.0202041510540.4327-100000@snaresland.acl.lan l.gov>
2002-02-05  1:30     ` skipt
2002-02-05 15:32       ` Ronald G Minnich
2002-02-03 21:12 andrey mirtchovski
2002-02-03 21:01 Andrew Simmons

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).