The Unix Heritage Society mailing list
 help / color / Atom feed
* [TUHS] History of symbol preemption
@ 2020-01-13 19:58 Paul Winalski
  2020-01-13 20:46 ` Rico Pajarola
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Paul Winalski @ 2020-01-13 19:58 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

The Executable and Linkable Format (ELF) is the modern standard for
object files in Unix and Unix-like OSes (e.g., Linux), and even for
OpenVMS.  LInux, AIX and probably other implementations of ELF have a
feature in the runtime loader called symbol preemption.  When loading
a shared library, the runtime loader examines the library's symbol
table.  If there is a global symbol with default visibility, and a
value for that symbol has already been loaded, all references to the
symbol in the library being loaded are rebound to the existing
definition.  The existing value thus preempts the definition in the
library.

I'm curious about the history of symbol preemption.  It does not exist
in other implementations of shared libraries, such as IBM OS/370 and
its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently
was designed in the mid-1990s.  I have found a copy of the System V
Application Binary Interface from April 2001 that describes symbol
preemption in the section on the ELF symbol table.

When was symbol preemption when loading shared objects first
implemented in Unix?  Are there versions of Unix that don't do symbol
preemption?

-Paul W.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 19:58 [TUHS] History of symbol preemption Paul Winalski
@ 2020-01-13 20:46 ` Rico Pajarola
  2020-01-13 21:04   ` Clem Cole
  2020-01-14  2:53 ` Rob Gingell
  2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
  2 siblings, 1 reply; 15+ messages in thread
From: Rico Pajarola @ 2020-01-13 20:46 UTC (permalink / raw)
  To: Paul Winalski; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

This seems to have originated with SunOS 4. I believe a good proxy for
finding anything that inherited from or was inspired by this is a linker
that recognizes LD_PRELOAD. I wonder if there are other independent
implementations in the Unix space.


On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski <paul.winalski@gmail.com>
wrote:

> The Executable and Linkable Format (ELF) is the modern standard for
> object files in Unix and Unix-like OSes (e.g., Linux), and even for
> OpenVMS.  LInux, AIX and probably other implementations of ELF have a
> feature in the runtime loader called symbol preemption.  When loading
> a shared library, the runtime loader examines the library's symbol
> table.  If there is a global symbol with default visibility, and a
> value for that symbol has already been loaded, all references to the
> symbol in the library being loaded are rebound to the existing
> definition.  The existing value thus preempts the definition in the
> library.
>
> I'm curious about the history of symbol preemption.  It does not exist
> in other implementations of shared libraries, such as IBM OS/370 and
> its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently
> was designed in the mid-1990s.  I have found a copy of the System V
> Application Binary Interface from April 2001 that describes symbol
> preemption in the section on the ELF symbol table.
>
> When was symbol preemption when loading shared objects first
> implemented in Unix?  Are there versions of Unix that don't do symbol
> preemption?
>
> -Paul W.
>

[-- Attachment #2: Type: text/html, Size: 1930 bytes --]

<div dir="ltr">This seems to have originated with SunOS 4. I believe a good proxy for finding anything that inherited from or was inspired by this is a linker that recognizes LD_PRELOAD. I wonder if there are other independent implementations in the Unix space.<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski &lt;<a href="mailto:paul.winalski@gmail.com">paul.winalski@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The Executable and Linkable Format (ELF) is the modern standard for<br>
object files in Unix and Unix-like OSes (e.g., Linux), and even for<br>
OpenVMS.  LInux, AIX and probably other implementations of ELF have a<br>
feature in the runtime loader called symbol preemption.  When loading<br>
a shared library, the runtime loader examines the library&#39;s symbol<br>
table.  If there is a global symbol with default visibility, and a<br>
value for that symbol has already been loaded, all references to the<br>
symbol in the library being loaded are rebound to the existing<br>
definition.  The existing value thus preempts the definition in the<br>
library.<br>
<br>
I&#39;m curious about the history of symbol preemption.  It does not exist<br>
in other implementations of shared libraries, such as IBM OS/370 and<br>
its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently<br>
was designed in the mid-1990s.  I have found a copy of the System V<br>
Application Binary Interface from April 2001 that describes symbol<br>
preemption in the section on the ELF symbol table.<br>
<br>
When was symbol preemption when loading shared objects first<br>
implemented in Unix?  Are there versions of Unix that don&#39;t do symbol<br>
preemption?<br>
<br>
-Paul W.<br>
</blockquote></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 20:46 ` Rico Pajarola
@ 2020-01-13 21:04   ` Clem Cole
  2020-01-13 21:40     ` Rico Pajarola
  2020-01-13 21:42     ` Paul Winalski
  0 siblings, 2 replies; 15+ messages in thread
From: Clem Cole @ 2020-01-13 21:04 UTC (permalink / raw)
  To: Rico Pajarola; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2859 bytes --]

@ Rico I'm failing sure ELF came from AT&T Summit, not Sun.
@ Steve Johnson were you the manager when was created or were you folks
still using COFF?

Anyway... There were issues with COFF WRT being
architecture-independent and supporting dynamic loading well.  Steve Rago
would also be a good person to ask if you want some of the details.  At one
point there was a COFF2 document, but it may have been System Vx licenses
only.   Also, one of the issues was that AT&T had officially tied up COFF
as a proprietary format -- all part of the 'consider it standard' trying to
force their lunch down all the other UNIX systems throat which was not
having it.   As a result, CMU's MachO was about to become the default
format (OSF and Apple were already using it for that reason), and Unix
International stepped in and convinced AT&T to released the ELF documents
(I was on the UI technical board at that point).  I'm not sure how/why OSF
decided to back off, maybe because after ELF became public it got supported
by GCC.

Now my memory is a little hazy... I think OSF/1-386 used MachO originally,
but I've forgotten.   Switching the kernel to use ELF was one of the
differences between OSF1 and Tru64 IIRC.

On Mon, Jan 13, 2020 at 3:47 PM Rico Pajarola <rp@servium.ch> wrote:

> This seems to have originated with SunOS 4. I believe a good proxy for
> finding anything that inherited from or was inspired by this is a linker
> that recognizes LD_PRELOAD. I wonder if there are other independent
> implementations in the Unix space.
>
>
> On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski <paul.winalski@gmail.com>
> wrote:
>
>> The Executable and Linkable Format (ELF) is the modern standard for
>> object files in Unix and Unix-like OSes (e.g., Linux), and even for
>> OpenVMS.  LInux, AIX and probably other implementations of ELF have a
>> feature in the runtime loader called symbol preemption.  When loading
>> a shared library, the runtime loader examines the library's symbol
>> table.  If there is a global symbol with default visibility, and a
>> value for that symbol has already been loaded, all references to the
>> symbol in the library being loaded are rebound to the existing
>> definition.  The existing value thus preempts the definition in the
>> library.
>>
>> I'm curious about the history of symbol preemption.  It does not exist
>> in other implementations of shared libraries, such as IBM OS/370 and
>> its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently
>> was designed in the mid-1990s.  I have found a copy of the System V
>> Application Binary Interface from April 2001 that describes symbol
>> preemption in the section on the ELF symbol table.
>>
>> When was symbol preemption when loading shared objects first
>> implemented in Unix?  Are there versions of Unix that don't do symbol
>> preemption?
>>
>> -Paul W.
>>
>

[-- Attachment #2: Type: text/html, Size: 4028 bytes --]

<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">@ Rico I&#39;m failing sure ELF came from AT&amp;T Summit, not Sun. </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">@ Steve Johnson were you the manager when was created or were you folks still using COFF?</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Anyway... There were issues with COFF WRT being architecture-independent and supporting dynamic loading well.  Steve Rago would also be a good person to ask if you want some of the details.  At one point there was a COFF2 document, but it may have been System Vx licenses only.   Also, one of the issues was that AT&amp;T had officially tied up COFF as a proprietary format -- all part of the &#39;consider it standard&#39; trying to force their lunch down all the other UNIX systems throat which was not having it.   As a result, CMU&#39;s MachO was about to become the default format (OSF and Apple were already using it for that reason), and Unix International stepped in and convinced AT&amp;T to released the ELF documents (I was on the UI technical board at that point).  I&#39;m not sure how/why OSF decided to back off, maybe because after ELF became public it got supported by GCC.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Now my memory is a little hazy... I think OSF/1-386 used MachO originally, but I&#39;ve forgotten.   Switching the kernel to use ELF was one of the differences between OSF1 and Tru64 IIRC.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 3:47 PM Rico Pajarola &lt;<a href="mailto:rp@servium.ch">rp@servium.ch</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">This seems to have originated with SunOS 4. I believe a good proxy for finding anything that inherited from or was inspired by this is a linker that recognizes LD_PRELOAD. I wonder if there are other independent implementations in the Unix space.<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski &lt;<a href="mailto:paul.winalski@gmail.com" target="_blank">paul.winalski@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The Executable and Linkable Format (ELF) is the modern standard for<br>
object files in Unix and Unix-like OSes (e.g., Linux), and even for<br>
OpenVMS.  LInux, AIX and probably other implementations of ELF have a<br>
feature in the runtime loader called symbol preemption.  When loading<br>
a shared library, the runtime loader examines the library&#39;s symbol<br>
table.  If there is a global symbol with default visibility, and a<br>
value for that symbol has already been loaded, all references to the<br>
symbol in the library being loaded are rebound to the existing<br>
definition.  The existing value thus preempts the definition in the<br>
library.<br>
<br>
I&#39;m curious about the history of symbol preemption.  It does not exist<br>
in other implementations of shared libraries, such as IBM OS/370 and<br>
its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently<br>
was designed in the mid-1990s.  I have found a copy of the System V<br>
Application Binary Interface from April 2001 that describes symbol<br>
preemption in the section on the ELF symbol table.<br>
<br>
When was symbol preemption when loading shared objects first<br>
implemented in Unix?  Are there versions of Unix that don&#39;t do symbol<br>
preemption?<br>
<br>
-Paul W.<br>
</blockquote></div>
</blockquote></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:04   ` Clem Cole
@ 2020-01-13 21:40     ` Rico Pajarola
  2020-01-13 21:44       ` Paul Winalski
  2020-01-13 21:42     ` Paul Winalski
  1 sibling, 1 reply; 15+ messages in thread
From: Rico Pajarola @ 2020-01-13 21:40 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 3082 bytes --]

On Mon, Jan 13, 2020 at 1:04 PM Clem Cole <clemc@ccc.com> wrote:

> @ Rico I'm failing sure ELF came from AT&T Summit, not Sun.
>
yes, but unless my memory is playing tricks, SunOS a.out had this feature.


> @ Steve Johnson were you the manager when was created or were you folks
> still using COFF?
>
> Anyway... There were issues with COFF WRT being
> architecture-independent and supporting dynamic loading well.  Steve Rago
> would also be a good person to ask if you want some of the details.  At one
> point there was a COFF2 document, but it may have been System Vx licenses
> only.   Also, one of the issues was that AT&T had officially tied up COFF
> as a proprietary format -- all part of the 'consider it standard' trying to
> force their lunch down all the other UNIX systems throat which was not
> having it.   As a result, CMU's MachO was about to become the default
> format (OSF and Apple were already using it for that reason), and Unix
> International stepped in and convinced AT&T to released the ELF documents
> (I was on the UI technical board at that point).  I'm not sure how/why OSF
> decided to back off, maybe because after ELF became public it got supported
> by GCC.
>
> Now my memory is a little hazy... I think OSF/1-386 used MachO originally,
> but I've forgotten.   Switching the kernel to use ELF was one of the
> differences between OSF1 and Tru64 IIRC.
>
> On Mon, Jan 13, 2020 at 3:47 PM Rico Pajarola <rp@servium.ch> wrote:
>
>> This seems to have originated with SunOS 4. I believe a good proxy for
>> finding anything that inherited from or was inspired by this is a linker
>> that recognizes LD_PRELOAD. I wonder if there are other independent
>> implementations in the Unix space.
>>
>>
>> On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski <paul.winalski@gmail.com>
>> wrote:
>>
>>> The Executable and Linkable Format (ELF) is the modern standard for
>>> object files in Unix and Unix-like OSes (e.g., Linux), and even for
>>> OpenVMS.  LInux, AIX and probably other implementations of ELF have a
>>> feature in the runtime loader called symbol preemption.  When loading
>>> a shared library, the runtime loader examines the library's symbol
>>> table.  If there is a global symbol with default visibility, and a
>>> value for that symbol has already been loaded, all references to the
>>> symbol in the library being loaded are rebound to the existing
>>> definition.  The existing value thus preempts the definition in the
>>> library.
>>>
>>> I'm curious about the history of symbol preemption.  It does not exist
>>> in other implementations of shared libraries, such as IBM OS/370 and
>>> its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently
>>> was designed in the mid-1990s.  I have found a copy of the System V
>>> Application Binary Interface from April 2001 that describes symbol
>>> preemption in the section on the ELF symbol table.
>>>
>>> When was symbol preemption when loading shared objects first
>>> implemented in Unix?  Are there versions of Unix that don't do symbol
>>> preemption?
>>>
>>> -Paul W.
>>>
>>

[-- Attachment #2: Type: text/html, Size: 4531 bytes --]

<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 1:04 PM Clem Cole &lt;<a href="mailto:clemc@ccc.com">clemc@ccc.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif">@ Rico I&#39;m failing sure ELF came from AT&amp;T Summit, not Sun. </div></div></blockquote><div>yes, but unless my memory is playing tricks, SunOS a.out had this feature.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif">@ Steve Johnson were you the manager when was created or were you folks still using COFF?</div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">Anyway... There were issues with COFF WRT being architecture-independent and supporting dynamic loading well.  Steve Rago would also be a good person to ask if you want some of the details.  At one point there was a COFF2 document, but it may have been System Vx licenses only.   Also, one of the issues was that AT&amp;T had officially tied up COFF as a proprietary format -- all part of the &#39;consider it standard&#39; trying to force their lunch down all the other UNIX systems throat which was not having it.   As a result, CMU&#39;s MachO was about to become the default format (OSF and Apple were already using it for that reason), and Unix International stepped in and convinced AT&amp;T to released the ELF documents (I was on the UI technical board at that point).  I&#39;m not sure how/why OSF decided to back off, maybe because after ELF became public it got supported by GCC.</div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">Now my memory is a little hazy... I think OSF/1-386 used MachO originally, but I&#39;ve forgotten.   Switching the kernel to use ELF was one of the differences between OSF1 and Tru64 IIRC.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 3:47 PM Rico Pajarola &lt;<a href="mailto:rp@servium.ch" target="_blank">rp@servium.ch</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">This seems to have originated with SunOS 4. I believe a good proxy for finding anything that inherited from or was inspired by this is a linker that recognizes LD_PRELOAD. I wonder if there are other independent implementations in the Unix space.<div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 11:59 AM Paul Winalski &lt;<a href="mailto:paul.winalski@gmail.com" target="_blank">paul.winalski@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The Executable and Linkable Format (ELF) is the modern standard for<br>
object files in Unix and Unix-like OSes (e.g., Linux), and even for<br>
OpenVMS.  LInux, AIX and probably other implementations of ELF have a<br>
feature in the runtime loader called symbol preemption.  When loading<br>
a shared library, the runtime loader examines the library&#39;s symbol<br>
table.  If there is a global symbol with default visibility, and a<br>
value for that symbol has already been loaded, all references to the<br>
symbol in the library being loaded are rebound to the existing<br>
definition.  The existing value thus preempts the definition in the<br>
library.<br>
<br>
I&#39;m curious about the history of symbol preemption.  It does not exist<br>
in other implementations of shared libraries, such as IBM OS/370 and<br>
its descendants, OpenVMS, and Microsoft Windows NT.  ELF apparently<br>
was designed in the mid-1990s.  I have found a copy of the System V<br>
Application Binary Interface from April 2001 that describes symbol<br>
preemption in the section on the ELF symbol table.<br>
<br>
When was symbol preemption when loading shared objects first<br>
implemented in Unix?  Are there versions of Unix that don&#39;t do symbol<br>
preemption?<br>
<br>
-Paul W.<br>
</blockquote></div>
</blockquote></div>
</blockquote></div></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:04   ` Clem Cole
  2020-01-13 21:40     ` Rico Pajarola
@ 2020-01-13 21:42     ` Paul Winalski
  2020-01-13 22:53       ` Henry Bent
  1 sibling, 1 reply; 15+ messages in thread
From: Paul Winalski @ 2020-01-13 21:42 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society, Rico Pajarola

On 1/13/20, Clem Cole <clemc@ccc.com> wrote:
>
> As a result, CMU's MachO was about to become the default
> format (OSF and Apple were already using it for that reason), and Unix
> International stepped in and convinced AT&T to released the ELF documents
> (I was on the UI technical board at that point).  I'm not sure how/why OSF
> decided to back off, maybe because after ELF became public it got supported
> by GCC.

Mach-O was decidedly a step up from a.out in terms of flexibility and
extensibility, but it is inferior to both COFF and ELF, IMO.  Mach-O
(at least the Apple OS X implementation) handles global symbols and
relocations in a clumsy and baroque way.  Of the three
object/executable formats, ELF is by far the cleanest and most
flexible, and that's possibly why OSF went to it once it became
available.

Microsoft went with COFF, sort-of, in Windows.  But PECOFF (Portable
and Extensible Common Object File Format) is different enough from
vanilla COFF that when I implemented support for it in GEM I found to
better to write an entire new module for PECOFF rather than put
conditional code in the existing COFF-handling code.  I think this was
another of MS's "embrace and extend" gambits to keep control over
their development environment.

> Now my memory is a little hazy... I think OSF/1-386 used MachO originally,
> but I've forgotten.   Switching the kernel to use ELF was one of the
> differences between OSF1 and Tru64 IIRC.

GEM never had to support Mach-O on any of its target platforms.  DEC's
Unix on MIPS used COFF.  Tru64 used ELF exclusively.  I don't know
what Apple used for object files before OS X.  IIRC NeXT was based on
the CMU MACH microkernel and hence used Mach-O.  OS X is
FreeBSD-based, but it uses Mach-O.

Tru64 has symbol preemption.  The Tru64 C/C++ compiler by default
turns off symbol preemption by using PROTECTED visibility for global
symbols, but it does have a --preempt_symbol option.  Using that
option can have disastrous effects on optimization.

Was symbol visibility (default/protected/internal) part of the
original ELF spec from the 1990s, or was it added later?

-Paul W.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:40     ` Rico Pajarola
@ 2020-01-13 21:44       ` Paul Winalski
  2020-01-13 21:45         ` Rico Pajarola
  0 siblings, 1 reply; 15+ messages in thread
From: Paul Winalski @ 2020-01-13 21:44 UTC (permalink / raw)
  To: Rico Pajarola; +Cc: The Eunuchs Hysterical Society

On 1/13/20, Rico Pajarola <rp@servium.ch> wrote:
>
> yes, but unless my memory is playing tricks, SunOS a.out had this feature.

Do you mean support for shared libraries, or symbol preemption?

-Paul W.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:44       ` Paul Winalski
@ 2020-01-13 21:45         ` Rico Pajarola
  2020-01-13 22:20           ` Larry McVoy
  0 siblings, 1 reply; 15+ messages in thread
From: Rico Pajarola @ 2020-01-13 21:45 UTC (permalink / raw)
  To: Paul Winalski; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

On Mon, Jan 13, 2020 at 1:44 PM Paul Winalski <paul.winalski@gmail.com>
wrote:

> On 1/13/20, Rico Pajarola <rp@servium.ch> wrote:
> >
> > yes, but unless my memory is playing tricks, SunOS a.out had this
> feature.
>
> Do you mean support for shared libraries, or symbol preemption?
>
symbol preemption


> -Paul W.
>

[-- Attachment #2: Type: text/html, Size: 851 bytes --]

<div dir="ltr"><div dir="ltr">On Mon, Jan 13, 2020 at 1:44 PM Paul Winalski &lt;<a href="mailto:paul.winalski@gmail.com">paul.winalski@gmail.com</a>&gt; wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 1/13/20, Rico Pajarola &lt;<a href="mailto:rp@servium.ch" target="_blank">rp@servium.ch</a>&gt; wrote:<br>
&gt;<br>
&gt; yes, but unless my memory is playing tricks, SunOS a.out had this feature.<br>
<br>
Do you mean support for shared libraries, or symbol preemption?<br></blockquote>symbol preemption</div><div class="gmail_quote"><div></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-Paul W.<br>
</blockquote></div></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:45         ` Rico Pajarola
@ 2020-01-13 22:20           ` Larry McVoy
  0 siblings, 0 replies; 15+ messages in thread
From: Larry McVoy @ 2020-01-13 22:20 UTC (permalink / raw)
  To: Rico Pajarola; +Cc: The Eunuchs Hysterical Society

On Mon, Jan 13, 2020 at 01:45:48PM -0800, Rico Pajarola wrote:
> On Mon, Jan 13, 2020 at 1:44 PM Paul Winalski <paul.winalski@gmail.com>
> wrote:
> 
> > On 1/13/20, Rico Pajarola <rp@servium.ch> wrote:
> > >
> > > yes, but unless my memory is playing tricks, SunOS a.out had this
> > feature.
> >
> > Do you mean support for shared libraries, or symbol preemption?
> >
> symbol preemption

My memory of this is it was dns lookups that wanted something like this.
SunOS really wanted the world to love YP but the world wanted Sun to
support DNS.  You could, and people did, make a shared lib that overloaded
the default gethostbyname(3) that wanted to do YP, the lib would provide
a DNS based one instead.

That's where I first learned of this stuff.  Gingell would know if you
want to know for sure.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 21:42     ` Paul Winalski
@ 2020-01-13 22:53       ` Henry Bent
  2020-01-14  0:31         ` Clem Cole
  0 siblings, 1 reply; 15+ messages in thread
From: Henry Bent @ 2020-01-13 22:53 UTC (permalink / raw)
  To: Paul Winalski; +Cc: The Eunuchs Hysterical Society, Rico Pajarola

[-- Attachment #1: Type: text/plain, Size: 1348 bytes --]

On Mon, 13 Jan 2020 at 16:43, Paul Winalski <paul.winalski@gmail.com> wrote:

>
> > Now my memory is a little hazy... I think OSF/1-386 used MachO
> originally,
> > but I've forgotten.   Switching the kernel to use ELF was one of the
> > differences between OSF1 and Tru64 IIRC.
>
> GEM never had to support Mach-O on any of its target platforms.  DEC's
> Unix on MIPS used COFF.  Tru64 used ELF exclusively.  I don't know
> what Apple used for object files before OS X.  IIRC NeXT was based on
> the CMU MACH microkernel and hence used Mach-O.  OS X is
> FreeBSD-based, but it uses Mach-O.
>

OSF/1 on MIPS used ECOFF by default, but at least some versions could also
create and run ELF executables.  That was all early to mid 1992, I
believe.  I don't have my DECstation up right now to check but I'm sure
that the OSF/1 2.0 beta can do it, and I wouldn't be surprised if the
versions of 1.0 with the v3.0 compiler could also do it.  I remember trying
to do ELF shared libraries but I think that support wasn't ready yet, which
is a shame because the ECOFF shared libraries on that platform are not fun
to deal with.  Not as bad as SGI's ECOFF shared libraries on IRIX 4
though.  I'm not sure if anyone outside of SGI ever bothered to put in the
work required to make one.

Wasn't OSF's original intent to use the OSF/ROSE object format?

-Henry

[-- Attachment #2: Type: text/html, Size: 1783 bytes --]

<div dir="ltr"><div dir="ltr">On Mon, 13 Jan 2020 at 16:43, Paul Winalski &lt;<a href="mailto:paul.winalski@gmail.com">paul.winalski@gmail.com</a>&gt; wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
&gt; Now my memory is a little hazy... I think OSF/1-386 used MachO originally,<br>
&gt; but I&#39;ve forgotten.   Switching the kernel to use ELF was one of the<br>
&gt; differences between OSF1 and Tru64 IIRC.<br>
<br>
GEM never had to support Mach-O on any of its target platforms.  DEC&#39;s<br>
Unix on MIPS used COFF.  Tru64 used ELF exclusively.  I don&#39;t know<br>
what Apple used for object files before OS X.  IIRC NeXT was based on<br>
the CMU MACH microkernel and hence used Mach-O.  OS X is<br>
FreeBSD-based, but it uses Mach-O.<br>
</blockquote><div><br></div><div>OSF/1 on MIPS used ECOFF by default, but at least some versions could also create and run ELF executables.  That was all early to mid 1992, I believe.  I don&#39;t have my DECstation up right now to check but I&#39;m sure that the OSF/1 2.0 beta can do it, and I wouldn&#39;t be surprised if the versions of 1.0 with the v3.0 compiler could also do it.  I remember trying to do ELF shared libraries but I think that support wasn&#39;t ready yet, which is a shame because the ECOFF shared libraries on that platform are not fun to deal with.  Not as bad as SGI&#39;s ECOFF shared libraries on IRIX 4 though.  I&#39;m not sure if anyone outside of SGI ever bothered to put in the work required to make one.<br></div><div><br></div><div>Wasn&#39;t OSF&#39;s original intent to use the OSF/ROSE object format?</div><div><br></div><div>-Henry<br></div></div></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 22:53       ` Henry Bent
@ 2020-01-14  0:31         ` Clem Cole
  0 siblings, 0 replies; 15+ messages in thread
From: Clem Cole @ 2020-01-14  0:31 UTC (permalink / raw)
  To: Henry Bent; +Cc: The Eunuchs Hysterical Society, Rico Pajarola

[-- Attachment #1: Type: text/plain, Size: 252 bytes --]

On Mon, Jan 13, 2020 at 5:54 PM Henry Bent <henry.r.bent@gmail.com> wrote:

> Wasn't OSF's original intent to use the OSF/ROSE object format?
>
Yes and when I said OSF/1 I meant the version from OSF itself which was on
the 386,  MIPS and an Encore box

[-- Attachment #2: Type: text/html, Size: 781 bytes --]

<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 13, 2020 at 5:54 PM Henry Bent &lt;<a href="mailto:henry.r.bent@gmail.com">henry.r.bent@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Wasn&#39;t OSF&#39;s original intent to use the OSF/ROSE object format?</div></div></blockquote><div><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">Yes and when I said OSF/1 I meant the version from OSF itself which was on the 386,  MIPS and an Encore box</span> </div></div></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] History of symbol preemption
  2020-01-13 19:58 [TUHS] History of symbol preemption Paul Winalski
  2020-01-13 20:46 ` Rico Pajarola
@ 2020-01-14  2:53 ` Rob Gingell
  2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
  2 siblings, 0 replies; 15+ messages in thread
From: Rob Gingell @ 2020-01-14  2:53 UTC (permalink / raw)
  To: Paul Winalski, The Eunuchs Hysterical Society

On 1/13/2020 11:58 AM, Paul Winalski wrote:
> When was symbol preemption when loading shared objects first
> implemented in Unix?  Are there versions of Unix that don't do symbol
> preemption?

The behavior described was in dynamic linking as introduced in SunOS 
4.0. There had been a couple of earlier shared library implementations, 
notably System V Shared Libraries but I couldn't speak to whether any of 
them had the behavior.

The reason the behavior is present is to (try to) preserve the behavior 
of an "ld" command line with respect to how the order of library 
specification impacted which definition of a given symbol was used to 
resolve references to it. With archive libraries the ordering could be 
used to effect interposition in sometimes obscure and questionable ways 
especially when archives were unordered. With shared objects, given that 
the whole object is always present, the usual intent of the ordering of 
libraries with respect to interposition is preserved by having the first 
definition be used to resolve all references.

(And the reason "ld"'s behavior was relevant is that the idea was to 
make link editing an ongoing process as code assembled rather than a one 
time thing. So yes the run-time link-editor does it but that's because 
it's conceptually the functionality of the static link-editor continuing 
its work to knit together the still-assembling program.)

The evolution of link editors in Solaris eventually resulted in the 
ability to effect more granular scoping and binding operations, 
including shared objects whose references to internal symbols could not 
be interposed upon and thus perhaps reflected the behavior of other 
systems mentioned in the original question.

But the interposition / preemption behavior was the one which, at the 
time dynamic linking was introduced, best satisfied the "Principle of 
Least Surprise".

On 1/13/2020 12:46 PM, Rico Pajarola wrote:
> I believe a good proxy for 
> finding anything that inherited from or was inspired by this is a linker 
> that recognizes LD_PRELOAD.

On 1/13/2020 2:20 PM, Larry McVoy wrote:
> You could, and people did, make a shared lib that overloaded
> the default gethostbyname(3) that wanted to do YP, the lib would provide
> a DNS based one instead.

Handling interposition in this way turned it into an explicit feature. 
Once multiple definitions for a symbol could exist in a program (or, 
definitions not yet referenced), devices to permit the navigation of the 
interposition like LD_PRELOAD, or the Name Service Switch, or Pluggable 
Authentication Modules became things that could be built around the 
functionality.

On 1/13/2020 1:04 PM, Clem Cole wrote:
> @ Rico I'm failing sure ELF came from AT&T Summit, not Sun.

Yes, SunOS 4.0 rode the a.out object file format. ELF originated at 
Summit and as others have noted was a vast improvement.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [TUHS] two AIX items [was Re:  History of symbol preemption
  2020-01-13 19:58 [TUHS] History of symbol preemption Paul Winalski
  2020-01-13 20:46 ` Rico Pajarola
  2020-01-14  2:53 ` Rob Gingell
@ 2020-01-14 19:21 ` " Charles H Sauer
  2020-01-14 20:31   ` Clem Cole
                     ` (2 more replies)
  2 siblings, 3 replies; 15+ messages in thread
From: Charles H Sauer @ 2020-01-14 19:21 UTC (permalink / raw)
  To: tuhs



On 1/13/2020 1:58 PM, Paul Winalski wrote:

> ...  LInux, AIX and probably other implementations of ELF have a
> feature in the runtime loader called symbol preemption.  When loading

As far as I know, AIX has never used ELF.

On the RT, AIX 1 & 2 used a.out enhanced for basic shared library 
support, designed by Larry Loucks with help from ISC, probably John 
Levine, if I recall correctly.

Starting with AIX 3, AIX used an extended COFF. See Auslander et al, 
"Dynamic Linking and Loading in the AIX System", SA23-2619 RISC 
System/6000 Technology p. 151. (I don't have/know of PDF of SA23-2619.) 
See, also, https://en.wikipedia.org/wiki/XCOFF.

It seems likely that AIX XCOFF supported symbol preemption, I haven't 
tried to determine one way or the other.

AIX on 386 & 370 probably used a.out. Clem probably knows.

An IBM retirees group on Facebook led me to IBM AIX Enhancements and 
Modernization at http://www.redbooks.ibm.com/abstracts/sg248453.html, 
just available yesterday. I tend to think of AIX as abandoned by IBM in 
favor of Linux, but, of course, that isn't really true. I've downloaded 
the PDF, but not looked inside.

Charlie

-- 
voice: +1.512.784.7526       e-mail: sauer@technologists.com
fax: +1.512.346.5240         Web: https://technologists.com/sauer/
Facebook/Google/Skype/Twitter: CharlesHSauer

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] two AIX items [was Re: History of symbol preemption
  2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
@ 2020-01-14 20:31   ` Clem Cole
  2020-01-14 23:22   ` Kevin Bowling
  2020-01-15 16:41   ` Paul Winalski
  2 siblings, 0 replies; 15+ messages in thread
From: Clem Cole @ 2020-01-14 20:31 UTC (permalink / raw)
  To: Charles H Sauer; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 357 bytes --]

On Tue, Jan 14, 2020 at 2:29 PM Charles H Sauer <sauer@technologists.com>
wrote:

> AIX on 386 & 370 probably used a.out. Clem probably knows.
>
I don't remember for sure, but I think that's right.  It might have used
COFF, given the time frame but I don't think so since it was originally
targetted for University/Research sites which were a.out oriented.

[-- Attachment #2: Type: text/html, Size: 855 bytes --]

<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 14, 2020 at 2:29 PM Charles H Sauer &lt;<a href="mailto:sauer@technologists.com">sauer@technologists.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">AIX on 386 &amp; 370 probably used a.out. Clem probably knows.<br></blockquote><div><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">I don&#39;t remember for sure, but I think that&#39;s right.  It might have used COFF, given the time frame but I don&#39;t think so since it was originally targetted for University/Research sites which were a.out oriented.</span></div></div></div>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] two AIX items [was Re: History of symbol preemption
  2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
  2020-01-14 20:31   ` Clem Cole
@ 2020-01-14 23:22   ` Kevin Bowling
  2020-01-15 16:41   ` Paul Winalski
  2 siblings, 0 replies; 15+ messages in thread
From: Kevin Bowling @ 2020-01-14 23:22 UTC (permalink / raw)
  To: Charles H Sauer; +Cc: TUHS main list

I've got a bunch of paper AIX internals documentation outside the
usual, most interesting from memory is one describing the 4.1 work
where it gained SMP support, pthreads, and a bunch of other things.  I
need to get some kind of scanning setup if you are interested.

On Tue, Jan 14, 2020 at 12:29 PM Charles H Sauer
<sauer@technologists.com> wrote:
>
>
>
> On 1/13/2020 1:58 PM, Paul Winalski wrote:
>
> > ...  LInux, AIX and probably other implementations of ELF have a
> > feature in the runtime loader called symbol preemption.  When loading
>
> As far as I know, AIX has never used ELF.
>
> On the RT, AIX 1 & 2 used a.out enhanced for basic shared library
> support, designed by Larry Loucks with help from ISC, probably John
> Levine, if I recall correctly.
>
> Starting with AIX 3, AIX used an extended COFF. See Auslander et al,
> "Dynamic Linking and Loading in the AIX System", SA23-2619 RISC
> System/6000 Technology p. 151. (I don't have/know of PDF of SA23-2619.)
> See, also, https://en.wikipedia.org/wiki/XCOFF.
>
> It seems likely that AIX XCOFF supported symbol preemption, I haven't
> tried to determine one way or the other.
>
> AIX on 386 & 370 probably used a.out. Clem probably knows.
>
> An IBM retirees group on Facebook led me to IBM AIX Enhancements and
> Modernization at http://www.redbooks.ibm.com/abstracts/sg248453.html,
> just available yesterday. I tend to think of AIX as abandoned by IBM in
> favor of Linux, but, of course, that isn't really true. I've downloaded
> the PDF, but not looked inside.
>
> Charlie
>
> --
> voice: +1.512.784.7526       e-mail: sauer@technologists.com
> fax: +1.512.346.5240         Web: https://technologists.com/sauer/
> Facebook/Google/Skype/Twitter: CharlesHSauer

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [TUHS] two AIX items [was Re: History of symbol preemption
  2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
  2020-01-14 20:31   ` Clem Cole
  2020-01-14 23:22   ` Kevin Bowling
@ 2020-01-15 16:41   ` Paul Winalski
  2 siblings, 0 replies; 15+ messages in thread
From: Paul Winalski @ 2020-01-15 16:41 UTC (permalink / raw)
  To: Charles H Sauer; +Cc: tuhs

On 1/14/20, Charles H Sauer <sauer@technologists.com> wrote:
>
> As far as I know, AIX has never used ELF.

You're right.  My mistake.

-Paul W.

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, back to index

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-13 19:58 [TUHS] History of symbol preemption Paul Winalski
2020-01-13 20:46 ` Rico Pajarola
2020-01-13 21:04   ` Clem Cole
2020-01-13 21:40     ` Rico Pajarola
2020-01-13 21:44       ` Paul Winalski
2020-01-13 21:45         ` Rico Pajarola
2020-01-13 22:20           ` Larry McVoy
2020-01-13 21:42     ` Paul Winalski
2020-01-13 22:53       ` Henry Bent
2020-01-14  0:31         ` Clem Cole
2020-01-14  2:53 ` Rob Gingell
2020-01-14 19:21 ` [TUHS] two AIX items [was " Charles H Sauer
2020-01-14 20:31   ` Clem Cole
2020-01-14 23:22   ` Kevin Bowling
2020-01-15 16:41   ` Paul Winalski

The Unix Heritage Society mailing list

Archives are clonable: git clone --mirror http://inbox.vuxu.org/tuhs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.tuhs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git