Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Rudi Blom]

>Date: Sun, 30 Dec 2018 14:24:55 -0500
>From: Paul Winalski <paul.winalski@gmail.com>
>To: Norman Wilson <norman@oclsc.org>
>Cc: tuhs@tuhs.org
>Subject: Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?
>Message-ID:
>        <CABH=_VTqZzNXPNecFCVZeqfMTnoJiWHbXZz->BriRGtxBY0J10Q@mail.gmail.com>
>Content-Type: text/plain; charset="UTF-8"
>
>On 12/30/18, Norman Wilson <norman@oclsc.org> wrote:
>
>> <snip>
>>
>> Nowadays ranlib is no longer a separate program: ar
>> recognizes object files and maintains an index if any are
>> present.  I never especially liked that; ar is in.
>> principle a general tool so why should it have a special
>> case for one type of file?  But in practice I don't know
>> anyone who uses ar for anything except libraries any more
>> (everyone uses tar for the general case, since .it does a.
>> better job).
>
>As you say, nobody these days uses ar for anything except object
>module libraries.  And just about anything you do that modifies an ar
>library will require re-running ranlib afterwards.  So as a
>convenience and as a way to avoid cockpit errors, it makes sense to
>merge the ranlib function into ar.  MacOS still uses an independent
>ranlib, and it's a pain in the butt to have to remember to run ranlib
>after each time you modify an archive.
>
Maybe not on some of the older, more (resource) restricted systems,
but now normally wouldn't modifying an archive be part of
definitions/rules in a makefile and as such wouldn't the makefile
include using ranlib if an archive was modified ?

uncle rubl

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Doug McIlroy]

> But wasn't it tsort that did the heavy lifting to get things in order?

An amusing notion. Having written tsort, I can assure you it couldn't
lift anything heavy--it used the most naive quadratic algorithm. But
it was good enough for libc.

Doug

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Will Senn]

On Dec 30, 2018, at 8:28 PM, Doug McIlroy <doug@cs.dartmouth.edu> wrote:

>> But wasn't it tsort that did the heavy lifting to get things in order?
> 
> An amusing notion. Having written tsort, I can assure you it couldn't
> lift anything heavy--it used the most naive quadratic algorithm. But
> it was good enough for libc.
> 
> Doug

I don’t see tsort anywhere on the system, is it v7 or an addon to v6? 

Thanks,

Will

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[ron]

Not in v6.   Showed up in V7.


> -----Original Message-----
> From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Will Senn
> Sent: Sunday, December 30, 2018 10:35 PM
> To: Doug McIlroy <doug@cs.dartmouth.edu>
> Cc: tuhs@tuhs.org
> Subject: Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?
> 
> 
> 
> On Dec 30, 2018, at 8:28 PM, Doug McIlroy <doug@cs.dartmouth.edu>
> wrote:
> 
> >> But wasn't it tsort that did the heavy lifting to get things in order?
> >
> > An amusing notion. Having written tsort, I can assure you it couldn't
> > lift anything heavy--it used the most naive quadratic algorithm. But
> > it was good enough for libc.
> >
> > Doug
> 
> I don’t see tsort anywhere on the system, is it v7 or an addon to v6?
> 
> Thanks,
> 
> Will

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Norman Wilson]

Warner Losh:

  But wasn't it tsort that did the heavy lifting to get things in order?

  ar c foo.a `tsort *.o`

  Ranlib just made it fast by adding an index..

====

There's a little more than that to ranlib.

Without ranlib, ld made a single pass through each library,
loading the modules that resolved unresolved symbols.  If
a module itself had unresolved symbols (as many do) and
some of those symbols were defined in a module ld had
already passed, you were out of luck, unless you explicitly
told ld to run a second pass, e.g. cc x.o y.o -la -la.
Hence the importance of explicit ordering when building
the library archive, and the usefulness of tsort.

Ranlib makes a list of all the .o file in this archive and
the global symbols defined or used by each module, and
places the list first in the archive.  If ld is (as it
was) changed to recognize the index, it can then make a
list of all the object files needed from this archive, even
if needed only by some file it will load from the same
archive, then collect all required modules in a single pass.

Ld could all along have just made two passes through the
library, one to assemble the same list ranlib did in advance,
a second to load the files.  (Or perhaps a first pass to
load what it knows it needs and assemble the list, and a
second only if necessary.)  Presumably it didn't both to
make ld simpler and because disk I/O was much slower back
then (especially on a heavily-loaded time-sharing system,
something far less common today).  I suspect it would work
fine just to do it that way today.

Nowadays ranlib is no longer a separate program: ar
recognizes object files and maintains an index if any are
present.  I never especially liked that; ar is in
principle a general tool so why should it have a special
case for one type of file?  But in practice I don't know
anyone who uses ar for anything except libraries any more
(everyone uses tar for the general case, since it does a
better job).  Were I to wave flags over the matter I'd
rather push to ditch ar entirely save for compatibility
with the past, move to using tar rather than ar for object
libraries, and let ld do two passes when necessary rather
than requiring that libraries be specially prepared.  As
I say, I think modern systems are fast enough that that
would work fine.

Norman Wilson
Toronto ON

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Paul Winalski]

On 12/30/18, Norman Wilson <norman@oclsc.org> wrote:
>
> Ld could all along have just made two passes through the
> library, one to assemble the same list ranlib did in advance,
> a second to load the files.  (Or perhaps a first pass to
> load what it knows it needs and assemble the list, and a
> second only if necessary.)

That would avoid the O(N**2) situation I described in a reply to this thread.

>  Presumably it didn't both to
> make ld simpler and because disk I/O was much slower back
> then (especially on a heavily-loaded time-sharing system,
> something far less common today).  I suspect it would work
> fine just to do it that way today.

Probably not.  For some reason linkers have always been notoriously
slow when compared to other parts of the compilation toolchain.  I
suspect it's because of all the I/O involved.

> Nowadays ranlib is no longer a separate program: ar
> recognizes object files and maintains an index if any are
> present.  I never especially liked that; ar is in
> principle a general tool so why should it have a special
> case for one type of file?  But in practice I don't know
> anyone who uses ar for anything except libraries any more
> (everyone uses tar for the general case, since it does a
> better job).

As you say, nobody these days uses ar for anything except object
module libraries.  And just about anything you do that modifies an ar
library will require re-running ranlib afterwards.  So as a
convenience and as a way to avoid cockpit errors, it makes sense to
merge the ranlib function into ar.  MacOS still uses an independent
ranlib, and it's a pain in the butt to have to remember to run ranlib
after each time you modify an archive.


>  Were I to wave flags over the matter I'd
> rather push to ditch ar entirely save for compatibility
> with the past, move to using tar rather than ar for object
> libraries, and let ld do two passes when necessary rather
> than requiring that libraries be specially prepared.  As
> I say, I think modern systems are fast enough that that
> would work fine.

The tar file format isn't as compact as ar's, since it operates on
fixed-size blocks.  Nowadays that wouldn't be a problem, but it
certainly was when disks were small, given that object modules very
often would be significantly smaller than a ranlib block.  Making two
passes over the entire library might be OK if the file system caches
file contents, but it would still require at least one complete scan
of the library, whereas if you have an index of global symbols, you
just have to process that index (and, of course, the modules you
finally decide to load).  As I said earlier, linkers are slow enough
as is--we don't need anything to make them less efficient.

-Paul W.

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[arnold]

Norman Wilson <norman@oclsc.org> wrote:

> Nowadays ranlib is no longer a separate program: ar
> recognizes object files and maintains an index if any are
> present.  I never especially liked that; ar is in
> principle a general tool so why should it have a special
> case for one type of file?

I'll agree with that. My first exposure to ar was from the version
in Software Tools. I was suprised to learn at some point that it
understood object file formats.

>  But in practice I don't know
> anyone who uses ar for anything except libraries any more

I know one person. Brian Kernighan maintains an archive of his awk test
suite using ar.  Or rather, he did until recently, when I got him to
move to using tar a few months ago. :-)

Arnold

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Tony Finch]

arnold@skeeve.com <arnold@skeeve.com> wrote:
> Norman Wilson <norman@oclsc.org> wrote:
>
> > But in practice I don't know anyone who uses ar for anything except
> > libraries any more
>
> I know one person. Brian Kernighan maintains an archive of his awk test
> suite using ar.  Or rather, he did until recently, when I got him to
> move to using tar a few months ago. :-)

It's also handy for fiddling around with Debian packages:

$ ar t /var/cache/apt/archives/dpkg_1.18.25_amd64.deb
debian-binary
control.tar.gz
data.tar.xz
$

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
Rattray Head to Berwick upon Tweed: West or northwest 3 or 4, becoming
variable 3 or less, becoming west or southwest 3 or 4 later. Rough at first
near Rattray Head and Berwick-upon-Tweed, otherwise slight or moderate.
Showers at first. Good.

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Noel Chiappa]

    > From: Warren Toomey

    > I just tried it here. I had to do:
    > ...
    > ar r ../lib1 *.o
    > ...
    > to get them to rebuild. Otherwise, I had empty libraries.

Duhh. I never noticed the missing "*.o"!

I wonder how that one slipped through? Looking at 'run', it really does look
like it was used to prepare the systems on the distribution tape. So probably
the libraries just happened to already hold the latest and greatest, so that
error had no effect.


The thing with needing to order the library contents properly to cause all the
modules to get loaded is, I reckon, the reason why 'ar' has those arguments to
specify where in the archive a given file goes.

	Noel

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Noel Chiappa]

    > From: Will Senn

    > I whacked /usr/sys/lib1 and lib2 'accidentally' meaning I logged in as
    > bin changed to /usr/sys and typed rm lib1 and rm lib2 :).

Doesn't sound very accidental... :-)

    > sh run as bin doesn't do it.

Odd. 'run' in /usr/sys on my V6 machine (not that I use that, mind) says:
  
  chdir ken
  cc -c -O *.c
  ar r ../lib1
  rm *.o
  chdir ../dmr
  cc -c -O *.c
  ar r ../lib2
  rm *.o        

which should regenerate them - sort of. I suspect you really meant 'doing sh
run creates a lib1 and lib2, but then I get errors from the ld phase with
missing symbols'. Yes?

If so, the thing is that the V6 linker won't pull in an object module from a
library unless a global in it satisfies an already existing (i.e. in the
linking process) undefined global. (I don't know if this is true of later
linkers; never used 'em.) In other words, when loading a multi-module system,
the module with 'main' has to be first, and then the rest in an order such
that each one holds a previously-undefined symbol.

So the order of the object modules you'll get in lib? from the above, if you
precede them with 'rm lib?', is probably not the right order. (The above shell
script assumes they already exist, with the modules in the right order, so the
above just replaces them with the newly compiled versions...)

    > So, what magic incantation is required to rebuild them.

Here's the ordering in lib1:

  main.o
  alloc.o
  iget.o
  prf.o
  rdwri.o
  slp.o
  subr.o
  text.o
  trap.o
  sig.o
  sysent.o
  clock.o
  fio.o
  malloc.o
  nami.o
  pipe.o
  sys1.o
  sys2.o
  sys3.o
  sys4.o   

Other orders would work too (e.g. you could move sys?.o up just after sysent.o
and it should work).

My lib2 is somewhat odd, so I hesitate to list it, but since most modules in
dmr are pulled in from entries in c.c, almost any order will work, I think.

    Noel

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Will Senn]

Ah! Thanks Noel. Yes, the two files are created and then a slew of linker errors.  What you say makes sense. I can just compare with another v6 instance that I have that is similar to replicate the delivered order. Now I’m off to trying it out.


Will


Sent from my iPhone

On Dec 28, 2018, at 7:09 PM, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:

>> From: Will Senn
> 
>> I whacked /usr/sys/lib1 and lib2 'accidentally' meaning I logged in as
>> bin changed to /usr/sys and typed rm lib1 and rm lib2 :).
> 
> Doesn't sound very accidental... :-)
> 
>> sh run as bin doesn't do it.
> 
> Odd. 'run' in /usr/sys on my V6 machine (not that I use that, mind) says:
> 
>  chdir ken
>  cc -c -O *.c
>  ar r ../lib1
>  rm *.o
>  chdir ../dmr
>  cc -c -O *.c
>  ar r ../lib2
>  rm *.o        
> 
> which should regenerate them - sort of. I suspect you really meant 'doing sh
> run creates a lib1 and lib2, but then I get errors from the ld phase with
> missing symbols'. Yes?
> 
> If so, the thing is that the V6 linker won't pull in an object module from a
> library unless a global in it satisfies an already existing (i.e. in the
> linking process) undefined global. (I don't know if this is true of later
> linkers; never used 'em.) In other words, when loading a multi-module system,
> the module with 'main' has to be first, and then the rest in an order such
> that each one holds a previously-undefined symbol.
> 
> So the order of the object modules you'll get in lib? from the above, if you
> precede them with 'rm lib?', is probably not the right order. (The above shell
> script assumes they already exist, with the modules in the right order, so the
> above just replaces them with the newly compiled versions...)
> 
>> So, what magic incantation is required to rebuild them.
> 
> Here's the ordering in lib1:
> 
>  main.o
>  alloc.o
>  iget.o
>  prf.o
>  rdwri.o
>  slp.o
>  subr.o
>  text.o
>  trap.o
>  sig.o
>  sysent.o
>  clock.o
>  fio.o
>  malloc.o
>  nami.o
>  pipe.o
>  sys1.o
>  sys2.o
>  sys3.o
>  sys4.o   
> 
> Other orders would work too (e.g. you could move sys?.o up just after sysent.o
> and it should work).
> 
> My lib2 is somewhat odd, so I hesitate to list it, but since most modules in
> dmr are pulled in from entries in c.c, almost any order will work, I think.
> 
>    Noel

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Warren Toomey]

On Fri, Dec 28, 2018 at 08:09:00PM -0500, Noel Chiappa wrote:
> Odd. 'run' in /usr/sys on my V6 machine (not that I use that, mind) says:
>   
>   chdir ken
>   cc -c -O *.c
>   ar r ../lib1
>   rm *.o
>   chdir ../dmr
>   cc -c -O *.c
>   ar r ../lib2
>   rm *.o        
> 
> which should regenerate them - sort of.

I just tried it here. I had to do:
	chdir ken; ...
	ar r ../lib1 *.o
	chdir ../dmr; ...
	ar r ../lib2 *.o

to get them to rebuild. Otherwise, I had empty libraries.

Cheers, Warren

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Warren Toomey]

> On 12/28/18 7:35 PM, Warren Toomey wrote:
> > I just tried it here. I had to do:
> > 	chdir ken; ...
> > 	ar r ../lib1 *.o
> > 	chdir ../dmr; ...
> > 	ar r ../lib2 *.o

On Fri, Dec 28, 2018 at 08:02:55PM -0600, Will Senn wrote:
> I wound up doing:
> chdir ken
> cc -c -O *.c
> ar r ../lib1 main.o
> ar r ../lib1 alloc.o
> ar r ../lib1 iget.o
> ar r ../lib1 prf.o
> ar r ../lib1 rdwri.o
> ar r ../lib1 slp.o
> ar r ../lib1 subr.o
> ar r ../lib1 text.o
> ar r ../lib1 trap.o
> ar r ../lib1 sig.o
> ar r ../lib1 sysent.o
> ar r ../lib1 clock.o
> ar r ../lib1 fio.o
> ar r ../lib1 malloc.o
> ar r ../lib1 nami.o
> ar r ../lib1 pipe.o
> ar r ../lib1 sys1.o
> ar r ../lib1 sys2.o
> ar r ../lib1 sys3.o
> ar r ../lib1 sys4.o
> 
> rm *.o
> 
> chdir ../dmr
> cc -c -O *.c
> 
> ar r ../lib2 bio.o
> ar r ../lib2 tty.o
> ar r ../lib2 dc.o
> ar r ../lib2 dn.o
> ar r ../lib2 dp.o
> ar r ../lib2 kl.o
> ar r ../lib2 mem.o
> ar r ../lib2 pc.o
> ar r ../lib2 rf.o
> ar r ../lib2 rk.o
> ar r ../lib2 tc.o
> ar r ../lib2 tm.o
> ar r ../lib2 partab.o
> ar r ../lib2 rp.o
> ar r ../lib2 lp.o
> ar r ../lib2 dhdm.o
> ar r ../lib2 dh.o
> ar r ../lib2 dhfdm.o
> ar r ../lib2 sys.o
> ar r ../lib2 hp.o
> ar r ../lib2 ht.o
> ar r ../lib2 hs.o
> rm *.o
> 
> Then I continued with the system build and it worked and my changes were
> there!
> Will

Yes, order will be important, I forgot. There's no ranlib in v6 :-)

Cheers, Warren

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 960 bytes --]

On Fri, Dec 28, 2018 at 11:59 PM Warren Toomey <wkt@tuhs.org> wrote:

> Yes, order will be important, I forgot. There's no ranlib in v6 :-)
>

Good point.    I've forgotten as to where and when did ranlib appear in the
dev stream?   Was it research, UCB or somewhere else like on the Harvard
Tape?

Just now, I took a quick peak at the 1BSD archive on TUHS.org but the
subdirtectories are all packed up as v6 ar archives (cont.a files) - *i.e.*
when somebody converted the BSD stp tape to a tar image they just wrote the
archive and then rewrote it as a compressed tar ball.  So I will take a
little more work to unpack them, ensure the dates are 1978 based. (which
I'll do at some point and offer them back to Warren).

But I do remember when ranlib showing up it was such a win for fixing C
compiler (well linkage) errors.  I could have sworn, we had it was before
V7, so maybe it came with the Typesetter C or UNIX/TS stuff.

Clem
ᐧ

[-- Attachment #2: Type: text/html, Size: 2194 bytes --]

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

On Sat, Dec 29, 2018, 11:27 AM Clem Cole <clemc@ccc.com wrote:

>
>
> On Fri, Dec 28, 2018 at 11:59 PM Warren Toomey <wkt@tuhs.org> wrote:
>
>> Yes, order will be important, I forgot. There's no ranlib in v6 :-)
>>
>
> Good point.    I've forgotten as to where and when did ranlib appear in
> the dev stream?   Was it research, UCB or somewhere else like on the
> Harvard Tape?
>
> Just now, I took a quick peak at the 1BSD archive on TUHS.org but the
> subdirtectories are all packed up as v6 ar archives (cont.a files) -
> *i.e.* when somebody converted the BSD stp tape to a tar image they just
> wrote the archive and then rewrote it as a compressed tar ball.  So I will
> take a little more work to unpack them, ensure the dates are 1978 based.
> (which I'll do at some point and offer them back to Warren).
>
> But I do remember when ranlib showing up it was such a win for fixing C
> compiler (well linkage) errors.  I could have sworn, we had it was before
> V7, so maybe it came with the Typesetter C or UNIX/TS stuff.
>

But wasn't it tsort that did the heavy lifting to get things in order?

ar c foo.a `tsort *.o`

Ranlib just made it fast by adding an index..

Warner

> ᐧ
>

[-- Attachment #2: Type: text/html, Size: 2869 bytes --]

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Clem cole]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

Yes.    

Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite. 

> On Dec 29, 2018, at 11:49 AM, Warner Losh <imp@bsdimp.com> wrote:
> 
> 
> 
>> On Sat, Dec 29, 2018, 11:27 AM Clem Cole <clemc@ccc.com wrote:
>> 
>> 
>>> On Fri, Dec 28, 2018 at 11:59 PM Warren Toomey <wkt@tuhs.org> wrote:
>>> Yes, order will be important, I forgot. There's no ranlib in v6 :-)
>> 
>> Good point.    I've forgotten as to where and when did ranlib appear in the dev stream?   Was it research, UCB or somewhere else like on the Harvard Tape?
>> 
>> Just now, I took a quick peak at the 1BSD archive on TUHS.org but the subdirtectories are all packed up as v6 ar archives (cont.a files) - i.e. when somebody converted the BSD stp tape to a tar image they just wrote the archive and then rewrote it as a compressed tar ball.  So I will take a little more work to unpack them, ensure the dates are 1978 based. (which I'll do at some point and offer them back to Warren).
>> 
>> But I do remember when ranlib showing up it was such a win for fixing C compiler (well linkage) errors.  I could have sworn, we had it was before V7, so maybe it came with the Typesetter C or UNIX/TS stuff.
> 
> 
> But wasn't it tsort that did the heavy lifting to get things in order?
> 
> ar c foo.a `tsort *.o`
> 
> Ranlib just made it fast by adding an index..
> 
> Warner
>> ᐧ

[-- Attachment #2: Type: text/html, Size: 3375 bytes --]

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Paul Winalski]

On 12/28/18, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
>
> If so, the thing is that the V6 linker won't pull in an object module from
> a
> library unless a global in it satisfies an already existing (i.e. in the
> linking process) undefined global. (I don't know if this is true of later
> linkers; never used 'em.)

I think this has been pretty much universal behavior for all linkers
on all OSes since the 1960s.  It continues to be true today.

Sometimes one runs into a situation where a module loaded from lib1.a
has an undefined symbol that causes a module from lib2.a to be loaded,
and that module in turn has an undefined symbol that is defined in
lib1.a.  In that case, you have to cause the linker to scan lib1.a
twice:

    ld main.o lib1.a lib2.a lib1.a

-Paul W.

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[ron]

Yes, it pretty much has to be this way as it is a single pass process and the ar format is simple.
The later libraries have a symbol "dictionary" added to the beginning.   Before that you manually arranged the order of the relocatables in the library and later automated tools to put them in dependency order (ranlib, etc...) came about.


> -----Original Message-----
> From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Paul Winalski
> Sent: Sunday, December 30, 2018 1:34 PM
> To: Noel Chiappa <jnc@mercury.lcs.mit.edu>
> Cc: tuhs@minnie.tuhs.org
> Subject: Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?
> 
> On 12/28/18, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> >
> > If so, the thing is that the V6 linker won't pull in an object module
> > from a library unless a global in it satisfies an already existing
> > (i.e. in the linking process) undefined global. (I don't know if this
> > is true of later linkers; never used 'em.)
> 
> I think this has been pretty much universal behavior for all linkers on all OSes
> since the 1960s.  It continues to be true today.
> 
> Sometimes one runs into a situation where a module loaded from lib1.a has
> an undefined symbol that causes a module from lib2.a to be loaded, and that
> module in turn has an undefined symbol that is defined in lib1.a.  In that
> case, you have to cause the linker to scan lib1.a
> twice:
> 
>     ld main.o lib1.a lib2.a lib1.a
> 
> -Paul W.

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1734 bytes --]

On Sun, Dec 30, 2018, 1:49 PM <ron@ronnatalie.com wrote:

> Yes, it pretty much has to be this way as it is a single pass process and
> the ar format is simple.
> The later libraries have a symbol "dictionary" added to the beginning.
>  Before that you manually arranged the order of the relocatables in the
> library and later automated tools to put them in dependency order (ranlib,
> etc...) came about.
>


Tsort/lorder put them in dependency order. Ranlib gave the index of symbols
as the first or last archive member... in time, the indexing moved up into
ar....

Warner


> > -----Original Message-----
> > From: TUHS <tuhs-bounces@minnie.tuhs.org> On Behalf Of Paul Winalski
> > Sent: Sunday, December 30, 2018 1:34 PM
> > To: Noel Chiappa <jnc@mercury.lcs.mit.edu>
> > Cc: tuhs@minnie.tuhs.org
> > Subject: Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?
> >
> > On 12/28/18, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> > >
> > > If so, the thing is that the V6 linker won't pull in an object module
> > > from a library unless a global in it satisfies an already existing
> > > (i.e. in the linking process) undefined global. (I don't know if this
> > > is true of later linkers; never used 'em.)
> >
> > I think this has been pretty much universal behavior for all linkers on
> all OSes
> > since the 1960s.  It continues to be true today.
> >
> > Sometimes one runs into a situation where a module loaded from lib1.a has
> > an undefined symbol that causes a module from lib2.a to be loaded, and
> that
> > module in turn has an undefined symbol that is defined in lib1.a.  In
> that
> > case, you have to cause the linker to scan lib1.a
> > twice:
> >
> >     ld main.o lib1.a lib2.a lib1.a
> >
> > -Paul W.
>
>

[-- Attachment #2: Type: text/html, Size: 2835 bytes --]

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Paul Winalski]

On 12/30/18, ron@ronnatalie.com <ron@ronnatalie.com> wrote:
> Yes, it pretty much has to be this way as it is a single pass process and
> the ar format is simple.

Early ld only made a single pass over each library on the command
line?  Interesting.  Most linkers (all modern ones?) make repeated
passes over a library until a pass doesn't resolve any symbols; only
then does the linker go on to the next library or object module on the
command line.  A single pass link makes some sense, though, on a
resource-constrained system--it's easier to implement, and a
multi-pass link has O(N**2) worst-case behavior.

ranlib greatly speeds up the link process, but early implementations
of ranlib had a problem (I would call it a bug):  they indexed common
symbols as well as globals.  Common symbol semantics require that if
an undefined external resolves against a common symbol, that in itself
is not enough to cause the object module to be loaded.  So even with
ranlib it's necessary for the linker to process the symbol table of
each module that gets a hit when the ranlib index is processed, and to
ignore instances where the resolution is against a common symbol.  ld
had a bug in this area that caused common symbol sizes to be maximized
improperly, and one of the stdio modules had a dependency on this bug.
Later versions of UNIX didn't have this bug; I don't know whether it
was ld or the stdio module that was fixed (I suspect the latter).

-Paul W.

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Donald ODona]

At 30 Dec 2018 18:35:22 +0000 (+00:00) from Paul Winalski <paul.winalski@gmail.com>:
> 
> I think this has been pretty much universal behavior for all linkers
> on all OSes since the 1960s.  It continues to be true today.
> 
> Sometimes one runs into a situation where a module loaded from lib1.a
> has an undefined symbol that causes a module from lib2.a to be loaded,
> and that module in turn has an undefined symbol that is defined in
> lib1.a.  In that case, you have to cause the linker to scan lib1.a
> twice:
> 
>     ld main.o lib1.a lib2.a lib1.a
> 
thats not true, because the M$ and borland linkers of the 80ths under MSDOS(sic!) already processed indexed libraries. Therefore the multiple inclusion of libraries wans'nt needed. 

Re: [TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Paul Winalski]

On 12/30/18, Donald ODona <mutiny.mutiny@india.com> wrote:
>
>
> At 30 Dec 2018 18:35:22 +0000 (+00:00) from Paul Winalski
> <paul.winalski@gmail.com>:
>>
>> Sometimes one runs into a situation where a module loaded from lib1.a
>> has an undefined symbol that causes a module from lib2.a to be loaded,
>> and that module in turn has an undefined symbol that is defined in
>> lib1.a.  In that case, you have to cause the linker to scan lib1.a
>> twice:
>>
>>     ld main.o lib1.a lib2.a lib1.a
>>
> thats not true, because the M$ and borland linkers of the 80ths under
> MSDOS(sic!) already processed indexed libraries. Therefore the multiple
> inclusion of libraries wans'nt needed.
>
No, you still need multiple inclusion in that situation.  Consider
this situation:

main.o has an undefined reference to s1
m1.o in lib1.a defines s1 and has an undefined reference to s2
m3.o in lib1.a defines s3
m2.o in lib2.a defines s2 and has an undefined reference to s3

The command line is:  ld main.o lib1.a lib2.a

The linker makes one or more passes over the index of lib1.a until no
more symbols are resolved.  The linker loads m1.o.  It does not load
m3.o because m3.o doesn't resolve any outstanding undefined
references.

The linker still has s2 undefined.  It now makes one or more passes
over the index of lib2.a until no more symbols are resolved.  In the
process it loads m2.o to resolve s2 and adds s3 to the list of
unresolved symbols.

After lib2.a is processed, s3 is still undefined, and ld has nothing
left to process.  ld will issue an "unresolved symbol" error message
for s3.  You need to specify lib1.a a second time to cause ld to scan
it again.

-Paul W.

[TUHS] Deleted lib1 and lib2 in v6, recoverable?

[Will Senn]

So... I whacked /usr/sys/lib1 and lib2 ‘accidentally’ meaning I logged in as bin changed to /usr/sys and typed rm lib1 and rm lib2 :). Now, I was thinking at the time that I could regenerate them... this seems like a possibility, but I can’t seem to get them back. 

sh run as bin doesn’t do it.

So, what magic incantation is required to rebuild them.

What motivated the exploration was a desire to modify main.c and see those changes manifest.

Help.

Thanks,

Will

Sent from my iPhone