The Unix Heritage Society mailing list
 help / color / Atom feed
* [TUHS] buffer overflow (Re:  Happy birthday Morris worm
@ 2019-11-12 22:39 Norman Wilson
  2019-11-13  1:43 ` John P. Linderman
  0 siblings, 1 reply; 21+ messages in thread
From: Norman Wilson @ 2019-11-12 22:39 UTC (permalink / raw)
  To: tuhs

Bakul Shah:

  Unfortunately strcpy & other buffer overflow friendly
  functions are still present in the C standard (I am looking at
  n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?

====

If you mean `can C be made proof against careless programmers,'
no.  You could try but the result wouldn't be C.  And Flon's
Dictum applies anyway, as always.

It's perfectly possible to program in C without overflowing
fixed buffers, just as it's perfectly possible to program in
C without dereferencing a NULL or garbage pointer.  I don't
claim to be perfect, but before the rtm worm rubbed my nose
in such problems, I was often sloppy about them, and afterward
I was very much aware of them and paid attention.

That's all I ask: we need to pay attention.  It's not about
tools, it's about brains and craftmanship and caring more
about quality than about feature count or shiny surfaces
or pushing the product out the door.

Which is a good bit of what was attractive about UNIX in
the first place--that both its ideas and its implementation
were straightforward and comprehensible and made with some
care.  (Never mind that it wasn't perfect either.)

Too bad software in general and UNIX descendants in particular
seem to have left all that behind.

Norman Wilson
Toronto ON

PS: if you find this depressing, cheer yourself up by watching
the LCM video showing off UNICS on the PDP-7.  I just did, and
it did.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 22:39 [TUHS] buffer overflow (Re: Happy birthday Morris worm Norman Wilson
@ 2019-11-13  1:43 ` John P. Linderman
  2019-11-21 13:10   ` William Cheswick
  0 siblings, 1 reply; 21+ messages in thread
From: John P. Linderman @ 2019-11-13  1:43 UTC (permalink / raw)
  To: Norman Wilson; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

Lorinda Cherry told me that that RTM (senior) used to test people's
programs by feeding them to themselves as input, a.out < a.out. It helped
cure people of the assumption that a program would only see "reasonable"
inputs.

On Tue, Nov 12, 2019 at 5:40 PM Norman Wilson <norman@oclsc.org> wrote:

> Bakul Shah:
>
>   Unfortunately strcpy & other buffer overflow friendly
>   functions are still present in the C standard (I am looking at
>   n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?
>
> ====
>
> If you mean `can C be made proof against careless programmers,'
> no.  You could try but the result wouldn't be C.  And Flon's
> Dictum applies anyway, as always.
>
> It's perfectly possible to program in C without overflowing
> fixed buffers, just as it's perfectly possible to program in
> C without dereferencing a NULL or garbage pointer.  I don't
> claim to be perfect, but before the rtm worm rubbed my nose
> in such problems, I was often sloppy about them, and afterward
> I was very much aware of them and paid attention.
>
> That's all I ask: we need to pay attention.  It's not about
> tools, it's about brains and craftmanship and caring more
> about quality than about feature count or shiny surfaces
> or pushing the product out the door.
>
> Which is a good bit of what was attractive about UNIX in
> the first place--that both its ideas and its implementation
> were straightforward and comprehensible and made with some
> care.  (Never mind that it wasn't perfect either.)
>
> Too bad software in general and UNIX descendants in particular
> seem to have left all that behind.
>
> Norman Wilson
> Toronto ON
>
> PS: if you find this depressing, cheer yourself up by watching
> the LCM video showing off UNICS on the PDP-7.  I just did, and
> it did.
>

[-- Attachment #2: Type: text/html, Size: 2309 bytes --]

<div dir="ltr"><div class="gmail_default" style="font-size:small">Lorinda Cherry told me that that RTM (senior) used to test people&#39;s programs by feeding them to themselves as input, a.out &lt; a.out. It helped cure people of the assumption that a program would only see &quot;reasonable&quot; inputs.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 12, 2019 at 5:40 PM Norman Wilson &lt;<a href="mailto:norman@oclsc.org">norman@oclsc.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Bakul Shah:<br>
<br>
  Unfortunately strcpy &amp; other buffer overflow friendly<br>
  functions are still present in the C standard (I am looking at<br>
  n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?<br>
<br>
====<br>
<br>
If you mean `can C be made proof against careless programmers,&#39;<br>
no.  You could try but the result wouldn&#39;t be C.  And Flon&#39;s<br>
Dictum applies anyway, as always.<br>
<br>
It&#39;s perfectly possible to program in C without overflowing<br>
fixed buffers, just as it&#39;s perfectly possible to program in<br>
C without dereferencing a NULL or garbage pointer.  I don&#39;t<br>
claim to be perfect, but before the rtm worm rubbed my nose<br>
in such problems, I was often sloppy about them, and afterward<br>
I was very much aware of them and paid attention.<br>
<br>
That&#39;s all I ask: we need to pay attention.  It&#39;s not about<br>
tools, it&#39;s about brains and craftmanship and caring more<br>
about quality than about feature count or shiny surfaces<br>
or pushing the product out the door.<br>
<br>
Which is a good bit of what was attractive about UNIX in<br>
the first place--that both its ideas and its implementation<br>
were straightforward and comprehensible and made with some<br>
care.  (Never mind that it wasn&#39;t perfect either.)<br>
<br>
Too bad software in general and UNIX descendants in particular<br>
seem to have left all that behind.<br>
<br>
Norman Wilson<br>
Toronto ON<br>
<br>
PS: if you find this depressing, cheer yourself up by watching<br>
the LCM video showing off UNICS on the PDP-7.  I just did, and<br>
it did.<br>
</blockquote></div>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-13  1:43 ` John P. Linderman
@ 2019-11-21 13:10   ` William Cheswick
  2019-11-21 18:04     ` Steve Johnson
  0 siblings, 1 reply; 21+ messages in thread
From: William Cheswick @ 2019-11-21 13:10 UTC (permalink / raw)
  Cc: The Eunuchs Hysterical Society

BTW, Bob Morris was not a Senior, and had no middle name.  It is my understanding that he inserted an “h” when a middle initial was demanded at the Labs, hence “rhm”.

His son is Robert Tappan Morris, Labs login and general tag “rtm," is not a junior.

(I had to fight spelling correction on this Mac so much for this post that I have turned it off.)

> On Nov 12, 2019, at 8:43 PM, John P. Linderman <jpl.jpl@gmail.com> wrote:
> 
> Lorinda Cherry told me that that RTM (senior) used to test people's programs by feeding them to themselves as input, a.out < a.out. It helped cure people of the assumption that a program would only see "reasonable" inputs.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-21 13:10   ` William Cheswick
@ 2019-11-21 18:04     ` Steve Johnson
  2019-11-21 21:51       ` John P. Linderman
  0 siblings, 1 reply; 21+ messages in thread
From: Steve Johnson @ 2019-11-21 18:04 UTC (permalink / raw)
  To: William Cheswick; +Cc: The Eunuchs Hysterical Society

Not everyone at the labs had a three-letter login.  Bjarne Stroustrup had the login bs, despite several gentle suggestions from myself and others that he add a middle initial...
Steve

Sent from my iPhone

> On Nov 21, 2019, at 5:18 AM, William Cheswick <ches@cheswick.com> wrote:
> 
> BTW, Bob Morris was not a Senior, and had no middle name.  It is my understanding that he inserted an “h” when a middle initial was demanded at the Labs, hence “rhm”.
> 
> His son is Robert Tappan Morris, Labs login and general tag “rtm," is not a junior.
> 
> (I had to fight spelling correction on this Mac so much for this post that I have turned it off.)
> 
>> On Nov 12, 2019, at 8:43 PM, John P. Linderman <jpl.jpl@gmail.com> wrote:
>> 
>> Lorinda Cherry told me that that RTM (senior) used to test people's programs by feeding them to themselves as input, a.out < a.out. It helped cure people of the assumption that a program would only see "reasonable" inputs.
> 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-21 18:04     ` Steve Johnson
@ 2019-11-21 21:51       ` John P. Linderman
  0 siblings, 0 replies; 21+ messages in thread
From: John P. Linderman @ 2019-11-21 21:51 UTC (permalink / raw)
  To: Steve Johnson; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1263 bytes --]

Perhaps I should have said "The Big Enchilada", although I have only
second-hand information that that uniquely identified Bob Morris.

On Thu, Nov 21, 2019 at 1:13 PM Steve Johnson <scj@yaccman.com> wrote:

> Not everyone at the labs had a three-letter login.  Bjarne Stroustrup had
> the login bs, despite several gentle suggestions from myself and others
> that he add a middle initial...
> Steve
>
> Sent from my iPhone
>
> > On Nov 21, 2019, at 5:18 AM, William Cheswick <ches@cheswick.com> wrote:
> >
> > BTW, Bob Morris was not a Senior, and had no middle name.  It is my
> understanding that he inserted an “h” when a middle initial was demanded at
> the Labs, hence “rhm”.
> >
> > His son is Robert Tappan Morris, Labs login and general tag “rtm," is
> not a junior.
> >
> > (I had to fight spelling correction on this Mac so much for this post
> that I have turned it off.)
> >
> >> On Nov 12, 2019, at 8:43 PM, John P. Linderman <jpl.jpl@gmail.com>
> wrote:
> >>
> >> Lorinda Cherry told me that that RTM (senior) used to test people's
> programs by feeding them to themselves as input, a.out < a.out. It helped
> cure people of the assumption that a program would only see "reasonable"
> inputs.
> >
>
>

[-- Attachment #2: Type: text/html, Size: 1803 bytes --]

<div dir="ltr"><div class="gmail_default" style="font-size:small">Perhaps I should have said &quot;The Big Enchilada&quot;, although I have only second-hand information that that uniquely identified Bob Morris.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2019 at 1:13 PM Steve Johnson &lt;<a href="mailto:scj@yaccman.com">scj@yaccman.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Not everyone at the labs had a three-letter login.  Bjarne Stroustrup had the login bs, despite several gentle suggestions from myself and others that he add a middle initial...<br>
Steve<br>
<br>
Sent from my iPhone<br>
<br>
&gt; On Nov 21, 2019, at 5:18 AM, William Cheswick &lt;<a href="mailto:ches@cheswick.com" target="_blank">ches@cheswick.com</a>&gt; wrote:<br>
&gt; <br>
&gt; BTW, Bob Morris was not a Senior, and had no middle name.  It is my understanding that he inserted an “h” when a middle initial was demanded at the Labs, hence “rhm”.<br>
&gt; <br>
&gt; His son is Robert Tappan Morris, Labs login and general tag “rtm,&quot; is not a junior.<br>
&gt; <br>
&gt; (I had to fight spelling correction on this Mac so much for this post that I have turned it off.)<br>
&gt; <br>
&gt;&gt; On Nov 12, 2019, at 8:43 PM, John P. Linderman &lt;<a href="mailto:jpl.jpl@gmail.com" target="_blank">jpl.jpl@gmail.com</a>&gt; wrote:<br>
&gt;&gt; <br>
&gt;&gt; Lorinda Cherry told me that that RTM (senior) used to test people&#39;s programs by feeding them to themselves as input, a.out &lt; a.out. It helped cure people of the assumption that a program would only see &quot;reasonable&quot; inputs.<br>
&gt; <br>
<br>
</blockquote></div>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-21 20:38         ` Warner Losh
  2019-11-21 21:04           ` Clem Cole
@ 2019-11-21 22:06           ` Dave Horsfall
  1 sibling, 0 replies; 21+ messages in thread
From: Dave Horsfall @ 2019-11-21 22:06 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

On Thu, 21 Nov 2019, Warner Losh wrote:

> A few things actually broke when FreeBSD removed it. Apart from 
> 'wrappers' that needed it for various reasons, it was only a few 
> programs in our 'ports' package that needed to be corrected.

FreeBSD prints a stern warning (both at compilation and execution) if you 
use it; I quickly fixed all my programs when I saw them :-)

> Most people have moved on with the 20 years of warnings when it was 
> used... Sadly only most...

Yep :-(

-- Dave

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-21 20:02       ` Dave Horsfall
  2019-11-21 20:38         ` Warner Losh
@ 2019-11-21 21:48         ` Steffen Nurpmeso
  1 sibling, 0 replies; 21+ messages in thread
From: Steffen Nurpmeso @ 2019-11-21 21:48 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

Dave Horsfall wrote in <alpine.BSF.2.21.9999.1911220658250.33542@aneurin\
.horsfall.org>:
 |On Tue, 19 Nov 2019, Tony Finch wrote:
 |> Amusingly POSIX says the C standard takes precedence wrt the details of 
 |> gets() (and other library functions) and C18 abolished gets(). I'm 
 |> slightly surprised that the POSIX committee didn't see that coming and 
 |> include the change in the 2018 edition...

This week (on the 19th, to be exact) Geoff Clare made official the
desire to align a forthcoming POSIX Issue 8 with ISO C17.
Current POSIX (1003.1(2016)/Issue7+TC2) still aligns with C99.

And, compared to C99, the POSIX wording causes sympathy

d37021   APPLICATION USAGE
37022         Reading a line that overflows the array pointed to by s results in undefined behavior. The use of
37023         fgets( ) is recommended.
37024               Since the user cannot specify the length of the buffer passed to gets( ), use of this function is
37025               discouraged. The length of the string read is unlimited. It is possible to overflow this buffer in
37026               such a way as to cause applications to fail, or possible system security violations.
37027               Applications should use the fgets( ) function instead of the obsolescent gets( ) function.
37028   RATIONALE
37029         The standard developers decided to mark the gets( ) function as obsolescent even though it is in
37030         the ISO C standard due to the possibility of buffer overflow.
37031   FUTURE DIRECTIONS
37032         The gets( ) function may be removed in a future version.

 |Didn't know that gets() had finally been abolished; it's possibly the most 
 |unsafe function (OK, macro) on the planet.  I've long been tempted to 
 |remove gets() and see what breaks...

It seems to have been removed in C 2011, except get_s(), which is
still present in the C 2017 draft that i have.  (I have never used
any of the _s() functions.)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-21 20:38         ` Warner Losh
@ 2019-11-21 21:04           ` Clem Cole
  2019-11-21 22:06           ` Dave Horsfall
  1 sibling, 0 replies; 21+ messages in thread
From: Clem Cole @ 2019-11-21 21:04 UTC (permalink / raw)
  To: Warner Losh; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

On Thu, Nov 21, 2019 at 3:39 PM Warner Losh <imp@bsdimp.com> wrote:

> A few things actually broke when FreeBSD removed it. Apart from 'wrappers'
> that needed it for various reasons, it was only a few programs in our
> 'ports' package that needed to be corrected.
>
> Most people have moved on with the 20 years of warnings when it was
> used... Sadly only most...
>
> Warner
>
That is encouraging to hear.

[-- Attachment #2: Type: text/html, Size: 992 bytes --]

<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2019 at 3:39 PM Warner Losh &lt;<a href="mailto:imp@bsdimp.com">imp@bsdimp.com</a>&gt; wrote:</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>A few things actually broke when FreeBSD removed it. Apart from &#39;wrappers&#39; that needed it for various reasons, it was only a few programs in our &#39;ports&#39; package that needed to be corrected.</div><div><br></div><div>Most people have moved on with the 20 years of warnings when it was used... Sadly only most...</div><div><br></div><div>Warner </div></div></div></blockquote><div><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">That is encouraging to hear.</span> </div></div></div>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-21 20:02       ` Dave Horsfall
@ 2019-11-21 20:38         ` Warner Losh
  2019-11-21 21:04           ` Clem Cole
  2019-11-21 22:06           ` Dave Horsfall
  2019-11-21 21:48         ` Steffen Nurpmeso
  1 sibling, 2 replies; 21+ messages in thread
From: Warner Losh @ 2019-11-21 20:38 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Thu, Nov 21, 2019 at 1:02 PM Dave Horsfall <dave@horsfall.org> wrote:

> On Tue, 19 Nov 2019, Tony Finch wrote:
>
> > Amusingly POSIX says the C standard takes precedence wrt the details of
> > gets() (and other library functions) and C18 abolished gets(). I'm
> > slightly surprised that the POSIX committee didn't see that coming and
> > include the change in the 2018 edition...
>
> Didn't know that gets() had finally been abolished; it's possibly the most
> unsafe function (OK, macro) on the planet.  I've long been tempted to
> remove gets() and see what breaks...
>

A few things actually broke when FreeBSD removed it. Apart from 'wrappers'
that needed it for various reasons, it was only a few programs in our
'ports' package that needed to be corrected.

Most people have moved on with the 20 years of warnings when it was used...
Sadly only most...

Warner

[-- Attachment #2: Type: text/html, Size: 1333 bytes --]

<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2019 at 1:02 PM Dave Horsfall &lt;<a href="mailto:dave@horsfall.org">dave@horsfall.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, 19 Nov 2019, Tony Finch wrote:<br>
<br>
&gt; Amusingly POSIX says the C standard takes precedence wrt the details of <br>
&gt; gets() (and other library functions) and C18 abolished gets(). I&#39;m <br>
&gt; slightly surprised that the POSIX committee didn&#39;t see that coming and <br>
&gt; include the change in the 2018 edition...<br>
<br>
Didn&#39;t know that gets() had finally been abolished; it&#39;s possibly the most <br>
unsafe function (OK, macro) on the planet.  I&#39;ve long been tempted to <br>
remove gets() and see what breaks...<br></blockquote><div><br></div><div>A few things actually broke when FreeBSD removed it. Apart from &#39;wrappers&#39; that needed it for various reasons, it was only a few programs in our &#39;ports&#39; package that needed to be corrected.</div><div><br></div><div>Most people have moved on with the 20 years of warnings when it was used... Sadly only most...</div><div><br></div><div>Warner </div></div></div>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
       [not found]     ` <alpine.DEB.2.20.1911191443530.10845@grey.csi.cam.ac.uk>
@ 2019-11-21 20:02       ` Dave Horsfall
  2019-11-21 20:38         ` Warner Losh
  2019-11-21 21:48         ` Steffen Nurpmeso
  0 siblings, 2 replies; 21+ messages in thread
From: Dave Horsfall @ 2019-11-21 20:02 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 19 Nov 2019, Tony Finch wrote:

> Amusingly POSIX says the C standard takes precedence wrt the details of 
> gets() (and other library functions) and C18 abolished gets(). I'm 
> slightly surprised that the POSIX committee didn't see that coming and 
> include the change in the 2018 edition...

Didn't know that gets() had finally been abolished; it's possibly the most 
unsafe function (OK, macro) on the planet.  I've long been tempted to 
remove gets() and see what breaks...

-- Dave

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 23:45       ` Jon Steinhart
  2019-11-13  0:38         ` Warren Toomey
@ 2019-11-13  1:09         ` Arthur Krewat
  1 sibling, 0 replies; 21+ messages in thread
From: Arthur Krewat @ 2019-11-13  1:09 UTC (permalink / raw)
  To: tuhs

On 11/12/2019 6:45 PM, Jon Steinhart wrote:
> Use that energy to get out there and teach
> people to be better and more careful programmers.

If I was religious, you'd get an "Amen, brother!".

But since I'm not, a simple "exactly" will suffice ;)

ak

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 23:45       ` Jon Steinhart
@ 2019-11-13  0:38         ` Warren Toomey
  2019-11-13  1:09         ` Arthur Krewat
  1 sibling, 0 replies; 21+ messages in thread
From: Warren Toomey @ 2019-11-13  0:38 UTC (permalink / raw)
  To: tuhs

On Tue, Nov 12, 2019 at 03:45:23PM -0800, Jon Steinhart wrote:
> So let's not rehash our favorite arguments about strings until Warren
> shuts down the discussion.  Use that energy to get out there and teach
> people to be better and more careful programmers.

I agree on both points, and it's a good time to ask for discussion on
strings and worms to be moved over to the COFF list!

Thanks, Warren

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-12 22:41     ` Robert Clausecker
  2019-11-12 22:49       ` Arthur Krewat
  2019-11-12 23:45       ` Jon Steinhart
@ 2019-11-13  0:24       ` Larry McVoy
  2 siblings, 0 replies; 21+ messages in thread
From: Larry McVoy @ 2019-11-13  0:24 UTC (permalink / raw)
  To: Robert Clausecker; +Cc: tuhs

On Tue, Nov 12, 2019 at 11:41:51PM +0100, Robert Clausecker wrote:
> Oh please no.  One of the things we've hopefully all learned from Pascal
> is that length-prefixed strings suck because you can't perform anything
> useful without copying the entire string.  Rob Pike and friends showed
> how to get strings and vectors right in the Go language where you have a
> builtin slice type which is essentially a structure
> 
>     struct slice(type) {
>             type *data;
>             size_t len, cap;
>     };
> 
> where data points to a buffer, len is the length of meaningful data in
> that buffer and cap is the total buffer size.  

We did something similar in BitKeeper but we added a spicy little twist.
We encoded len and cap in one word by making cap increase in powers of
2 only (which means you need log(n) bits for cap).  So it was a data
structure that scaled both up and down.

We used it everywhere in BitKeeper, it was super handy.

http://repos.bkbits.net/bk/dev/src/libc/utils/lines.c?PAGE=anno&REV=56cf7e34BTkDFx47E54DPNG51B2uCA

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 22:41     ` Robert Clausecker
  2019-11-12 22:49       ` Arthur Krewat
@ 2019-11-12 23:45       ` Jon Steinhart
  2019-11-13  0:38         ` Warren Toomey
  2019-11-13  1:09         ` Arthur Krewat
  2019-11-13  0:24       ` Larry McVoy
  2 siblings, 2 replies; 21+ messages in thread
From: Jon Steinhart @ 2019-11-12 23:45 UTC (permalink / raw)
  To: tuhs

[-- Warning: decoded text below may be mangled --]
[-- Attachment #0: Type: text/plain; charset="UTF-8", Size: 3612 bytes --]

Robert Clausecker writes:
> Oh please no.  One of the things we've hopefully all learned from Pascal
> is that length-prefixed strings suck because you can't perform anything
> useful without copying the entire string.  Rob Pike and friends showed
> how to get strings and vectors right in the Go language where you have a
> builtin slice type which is essentially a structure
>
>     struct slice(type) {
>             type *data;
>             size_t len, cap;
>     };
>
> where data points to a buffer, len is the length of meaningful data in
> that buffer and cap is the total buffer size.  This allows the language
> to take subslices and to append to existing slices without requiring
> copies in most cases.  If a copy is necessary, the runtime can allocate
> a slightly larger buffer in advance to allow for appending in amortised
> linear time.
>
> Overall, much more versatile than Pascal strings.
>
> But let's get back to the topic, after all I promised not to flame as
> much as Jörg did.
>
> Yours,
> Robert Clausecker
>
> On Tue, Nov 12, 2019 at 02:14:18PM -0800, Larry McVoy wrote:
> > On Tue, Nov 12, 2019 at 02:10:46PM -0800, Bakul Shah wrote:
> > > On Tue, 12 Nov 2019 15:56:15 -0500 Norman Wilson <norman@oclsc.org> wrote:
> > > >
> > > > My longer-term reaction was to completely drop my sloppy
> > > > old habit (common in those days not just in my code but in
> > > > that of many others) of ignoring possible buffer overflows.
> > > > I find it mind-boggling that people still make that mistake;
> > > > it has been literal decades since the lesson was rubbed in
> > > > our community's collective noses.  I am very disappointed
> > > > that programming education seems not to care enough about
> > > > this sort of thing, even today.
> > > 
> > > Unfortunately strcpy & other buffer overflow friendly
> > > functions are still present in the C standard (I am looking at
> > > n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?
> > 
> > Someone needs to do Strcpy() etc that have the length in the 
> > first bytes[s] of the string.
> > -- 
> > ---
> > Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

OK, been biting my tongue here.  I, for one, appreciate the fact that
it's still legal to purchase and use power tools.  I like being able
to use my chainsaw to cut up a tree without having to measure it first.
And I don't have a variety of tree here that contains its length at the
base.  Sure, things can go wrong; I have a scar on one leg to prove it.
But that just shows that I need to be more careful, not that I need to
try logging with a safety razor.

C strings were great at the time, and are still great, especially in
very small systems.  Go may be nicer in larger systems but I probably
wouldn't want the overhead in a small embedded system.  The length at
the start of a string has its issues too; do you want to consume 8
bytes at the start of each string on a 64-bit machine, or have strings
that can support different lengths and have to deal with converting
between them?

Programming isn't a good place for careless people.  I recognize that
what passes for "software technology" these days is coming up with
mechanisms that minimize the damage that can be done by unskilled
people.  But it's never gonna work.  Sure, you can replace pointer
problems with reference problems and so on, but that doesn't really
solve anything.

So let's not rehash our favorite arguments about strings until Warren
shuts down the discussion.  Use that energy to get out there and teach
people to be better and more careful programmers.

Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 23:22     ` Warner Losh
@ 2019-11-12 23:27       ` Arthur Krewat
  0 siblings, 0 replies; 21+ messages in thread
From: Arthur Krewat @ 2019-11-12 23:27 UTC (permalink / raw)
  To: tuhs


On 11/12/2019 6:22 PM, Warner Losh wrote:
> strncpy has two issues. First, it doesn't guarantee NUL termination. 
> Second, it always writes N bytes. It's for a fixed width data field, 
> not a variable length string whose buffer size is known. strlcpy is 
> much better, but still has some issues...
Maybe he meant strcpy_s()

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 22:54   ` Dave Horsfall
@ 2019-11-12 23:22     ` Warner Losh
  2019-11-12 23:27       ` Arthur Krewat
       [not found]     ` <alpine.DEB.2.20.1911191443530.10845@grey.csi.cam.ac.uk>
  1 sibling, 1 reply; 21+ messages in thread
From: Warner Losh @ 2019-11-12 23:22 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On Tue, Nov 12, 2019 at 3:54 PM Dave Horsfall <dave@horsfall.org> wrote:

> On Tue, 12 Nov 2019, Bakul Shah wrote:
>
> > Unfortunately strcpy & other buffer overflow friendly functions are
> > still present in the C standard (I am looking at n2434.pdf, draft of
> > Sept 25, 2019). Is C really not fixable?
>
> No; POSIX requires all sorts of broken functions be present, otherwise it
> is not compliant; heck, last I looked it even requires gets().  And let's
> not even mention pointers...  We are our own worst enemy.[*]
>

POSIX can't even recognize that leap seconds exist :(


> All is not lost, though; use strncpy() instead of strcpy() etc.  These
> days my first choice is Perl, despite it being bloated (I only use C if
> it's trivial or I need the speed).  I must look at Ruby, though...
>

strncpy has two issues. First, it doesn't guarantee NUL termination.
Second, it always writes N bytes. It's for a fixed width data field, not a
variable length string whose buffer size is known. strlcpy is much better,
but still has some issues...

Warner

[-- Attachment #2: Type: text/html, Size: 1654 bytes --]

<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 12, 2019 at 3:54 PM Dave Horsfall &lt;<a href="mailto:dave@horsfall.org">dave@horsfall.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, 12 Nov 2019, Bakul Shah wrote:<br>
<br>
&gt; Unfortunately strcpy &amp; other buffer overflow friendly functions are <br>
&gt; still present in the C standard (I am looking at n2434.pdf, draft of <br>
&gt; Sept 25, 2019). Is C really not fixable?<br>
<br>
No; POSIX requires all sorts of broken functions be present, otherwise it <br>
is not compliant; heck, last I looked it even requires gets().  And let&#39;s <br>
not even mention pointers...  We are our own worst enemy.[*]<br></blockquote><div><br></div><div>POSIX can&#39;t even recognize that leap seconds exist :(</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
All is not lost, though; use strncpy() instead of strcpy() etc.  These <br>
days my first choice is Perl, despite it being bloated (I only use C if <br>
it&#39;s trivial or I need the speed).  I must look at Ruby, though...<br></blockquote><div><br></div><div>strncpy has two issues. First, it doesn&#39;t guarantee NUL termination. Second, it always writes N bytes. It&#39;s for a fixed width data field, not a variable length string whose buffer size is known. strlcpy is much better, but still has some issues...</div><div><br></div><div>Warner</div></div></div>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-12 22:10 ` [TUHS] buffer overflow (Re: " Bakul Shah
  2019-11-12 22:14   ` Larry McVoy
@ 2019-11-12 22:54   ` Dave Horsfall
  2019-11-12 23:22     ` Warner Losh
       [not found]     ` <alpine.DEB.2.20.1911191443530.10845@grey.csi.cam.ac.uk>
  1 sibling, 2 replies; 21+ messages in thread
From: Dave Horsfall @ 2019-11-12 22:54 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 12 Nov 2019, Bakul Shah wrote:

> Unfortunately strcpy & other buffer overflow friendly functions are 
> still present in the C standard (I am looking at n2434.pdf, draft of 
> Sept 25, 2019). Is C really not fixable?

No; POSIX requires all sorts of broken functions be present, otherwise it 
is not compliant; heck, last I looked it even requires gets().  And let's 
not even mention pointers...  We are our own worst enemy.[*]

All is not lost, though; use strncpy() instead of strcpy() etc.  These 
days my first choice is Perl, despite it being bloated (I only use C if 
it's trivial or I need the speed).  I must look at Ruby, though...

[*]
Of if you were a Pogo fan, "We have met the enemy, and he is us".

-- Dave

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re: Happy birthday Morris worm
  2019-11-12 22:41     ` Robert Clausecker
@ 2019-11-12 22:49       ` Arthur Krewat
  2019-11-12 23:45       ` Jon Steinhart
  2019-11-13  0:24       ` Larry McVoy
  2 siblings, 0 replies; 21+ messages in thread
From: Arthur Krewat @ 2019-11-12 22:49 UTC (permalink / raw)
  To: tuhs



On 11/12/2019 5:41 PM, Robert Clausecker wrote:
> Oh please no.  One of the things we've hopefully all learned from Pascal
> is that length-prefixed strings suck because you can't perform anything
> useful without copying the entire string.  Rob Pike and friends showed
> how to get strings and vectors right in the Go language where you have a
> builtin slice type which is essentially a structure
>
>      struct slice(type) {
>              type *data;
>              size_t len, cap;
>      };

And none of that stops some programmer from doing slice.cap=255 - or is 
it read-only? ;)



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-12 22:14   ` Larry McVoy
@ 2019-11-12 22:41     ` Robert Clausecker
  2019-11-12 22:49       ` Arthur Krewat
                         ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Robert Clausecker @ 2019-11-12 22:41 UTC (permalink / raw)
  To: tuhs

Oh please no.  One of the things we've hopefully all learned from Pascal
is that length-prefixed strings suck because you can't perform anything
useful without copying the entire string.  Rob Pike and friends showed
how to get strings and vectors right in the Go language where you have a
builtin slice type which is essentially a structure

    struct slice(type) {
            type *data;
            size_t len, cap;
    };

where data points to a buffer, len is the length of meaningful data in
that buffer and cap is the total buffer size.  This allows the language
to take subslices and to append to existing slices without requiring
copies in most cases.  If a copy is necessary, the runtime can allocate
a slightly larger buffer in advance to allow for appending in amortised
linear time.

Overall, much more versatile than Pascal strings.

But let's get back to the topic, after all I promised not to flame as
much as Jörg did.

Yours,
Robert Clausecker

On Tue, Nov 12, 2019 at 02:14:18PM -0800, Larry McVoy wrote:
> On Tue, Nov 12, 2019 at 02:10:46PM -0800, Bakul Shah wrote:
> > On Tue, 12 Nov 2019 15:56:15 -0500 Norman Wilson <norman@oclsc.org> wrote:
> > >
> > > My longer-term reaction was to completely drop my sloppy
> > > old habit (common in those days not just in my code but in
> > > that of many others) of ignoring possible buffer overflows.
> > > I find it mind-boggling that people still make that mistake;
> > > it has been literal decades since the lesson was rubbed in
> > > our community's collective noses.  I am very disappointed
> > > that programming education seems not to care enough about
> > > this sort of thing, even today.
> > 
> > Unfortunately strcpy & other buffer overflow friendly
> > functions are still present in the C standard (I am looking at
> > n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?
> 
> Someone needs to do Strcpy() etc that have the length in the 
> first bytes[s] of the string.
> -- 
> ---
> Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

-- 
()  ascii ribbon campaign - for an 8-bit clean world 
/\  - against html email  - against proprietary attachments

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-12 22:10 ` [TUHS] buffer overflow (Re: " Bakul Shah
@ 2019-11-12 22:14   ` Larry McVoy
  2019-11-12 22:41     ` Robert Clausecker
  2019-11-12 22:54   ` Dave Horsfall
  1 sibling, 1 reply; 21+ messages in thread
From: Larry McVoy @ 2019-11-12 22:14 UTC (permalink / raw)
  To: Bakul Shah; +Cc: tuhs

On Tue, Nov 12, 2019 at 02:10:46PM -0800, Bakul Shah wrote:
> On Tue, 12 Nov 2019 15:56:15 -0500 Norman Wilson <norman@oclsc.org> wrote:
> >
> > My longer-term reaction was to completely drop my sloppy
> > old habit (common in those days not just in my code but in
> > that of many others) of ignoring possible buffer overflows.
> > I find it mind-boggling that people still make that mistake;
> > it has been literal decades since the lesson was rubbed in
> > our community's collective noses.  I am very disappointed
> > that programming education seems not to care enough about
> > this sort of thing, even today.
> 
> Unfortunately strcpy & other buffer overflow friendly
> functions are still present in the C standard (I am looking at
> n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?

Someone needs to do Strcpy() etc that have the length in the 
first bytes[s] of the string.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [TUHS] buffer overflow (Re:  Happy birthday Morris worm
  2019-11-12 20:56 [TUHS] " Norman Wilson
@ 2019-11-12 22:10 ` " Bakul Shah
  2019-11-12 22:14   ` Larry McVoy
  2019-11-12 22:54   ` Dave Horsfall
  0 siblings, 2 replies; 21+ messages in thread
From: Bakul Shah @ 2019-11-12 22:10 UTC (permalink / raw)
  To: tuhs

On Tue, 12 Nov 2019 15:56:15 -0500 Norman Wilson <norman@oclsc.org> wrote:
>
> My longer-term reaction was to completely drop my sloppy
> old habit (common in those days not just in my code but in
> that of many others) of ignoring possible buffer overflows.
> I find it mind-boggling that people still make that mistake;
> it has been literal decades since the lesson was rubbed in
> our community's collective noses.  I am very disappointed
> that programming education seems not to care enough about
> this sort of thing, even today.

Unfortunately strcpy & other buffer overflow friendly
functions are still present in the C standard (I am looking at
n2434.pdf, draft of Sept 25, 2019). Is C really not fixable?

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, back to index

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-12 22:39 [TUHS] buffer overflow (Re: Happy birthday Morris worm Norman Wilson
2019-11-13  1:43 ` John P. Linderman
2019-11-21 13:10   ` William Cheswick
2019-11-21 18:04     ` Steve Johnson
2019-11-21 21:51       ` John P. Linderman
  -- strict thread matches above, loose matches on Subject: below --
2019-11-12 20:56 [TUHS] " Norman Wilson
2019-11-12 22:10 ` [TUHS] buffer overflow (Re: " Bakul Shah
2019-11-12 22:14   ` Larry McVoy
2019-11-12 22:41     ` Robert Clausecker
2019-11-12 22:49       ` Arthur Krewat
2019-11-12 23:45       ` Jon Steinhart
2019-11-13  0:38         ` Warren Toomey
2019-11-13  1:09         ` Arthur Krewat
2019-11-13  0:24       ` Larry McVoy
2019-11-12 22:54   ` Dave Horsfall
2019-11-12 23:22     ` Warner Losh
2019-11-12 23:27       ` Arthur Krewat
     [not found]     ` <alpine.DEB.2.20.1911191443530.10845@grey.csi.cam.ac.uk>
2019-11-21 20:02       ` Dave Horsfall
2019-11-21 20:38         ` Warner Losh
2019-11-21 21:04           ` Clem Cole
2019-11-21 22:06           ` Dave Horsfall
2019-11-21 21:48         ` Steffen Nurpmeso

The Unix Heritage Society mailing list

Archives are clonable: git clone --mirror http://inbox.vuxu.org/tuhs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.tuhs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git