At 2024-07-03T08:59:11-0600, Marc Rochkind wrote:
> Steve Jenkin suggests: "Developers of Initial Unix arguably were
> 10x-100x more productive than IBM OS/360..."
> 
> Indeed, this is part of accepted UNIX lore.

That claim reminds me of a more general one.  Applied to software
development writ large, it seems to be lore, not a reproducible
scientific result.

I refer of course to Sackman, Erickson, and Grant's 1968 CACM paper
which documented a DARPA experiment that found a productivity range of
28:1 in their sample of programmers (with veterans of 7 years'
experience pitted against "trainees").  Naturally enough, plenty of
people who make claims about variance in programmer productivity are
unaware of this paper's existence; it's not actually relevant to them as
a source of knowledge.

https://web.archive.org/web/20120204023412/http://dustyvolumes.com/archives/497

Thomas Dickey, better known today as the maintainer of ncurses, xterm,
lynx, and mawk (all for 30 years or more, and among other projects),
published a critique of this study in 1981.

https://web.archive.org/web/20120204023555/http://dustyvolumes.com/archives/498

Bill Curtis published a critique of the Sackler et al. paper in 1988.

I quote (via Dickey):

"Sackman's ... message that substantial performance differences do exist
among programmers remains valid.  Detecting a 20+:1 range ratio depends
upon having one brilliant and one horrid performance in a sample.
However the range ratio is not a particularly stable measure of
performance variability among programmers.  The dispersions of such data
as appear in Table I are better represented by such measures as the
standard deviation or semiinterquartile range."

https://invisible-island.net/personal/paperstuff.html

We have likely all observed how this 28:1 ratio has bloated in retelling
over time, like the length of a fish catch, to 100:1 or even 1000:1.
Similarly we're all familiar with the common practice of presenting the
mean and sometimes the range of some data sample to support one's
argument, without mentioning the median or mode, let alone the variance
(or the standard deviation).  (If a member of one's audience is familiar
with non-Gaussian distributions and inquires whether one's sample may be
better characterized by one, you invite them to disengage from the
discussion.)

I assert that this "productivity gap" is a myth, and that it persists
because it serves the purposes of diverse audiences who adopt it with
motivated reasoning.

1.  Immature Unix enthusiasts like to reassure themselves, and others
    nearby, of their inherent superiority to rival programmers.

2.  Managers like to contrive reasons for (not) promoting individual
    contributors.  It's easy to cite this productivity "statistic" and
    then suggest, without indicating anything concrete, that an employee
    is either a rock star or a mediocrity.

3.  Directors in organizations like not having to further justify a
    "stack-rank and cut" approach to reducing salary and benefits as a
    proportion of operational expenditures.

    https://en.wikipedia.org/wiki/Vitality_curve

4.  Business culture in general is deeply wedded to the idea that
    individual productivity, merit, or capacity for "wealth creation" is
    variable by several orders of magnitude, because this claim
    "justifies" variance in compensation over a similarly large range,
    even among college-educated professionals in an organization,
    setting aside those members of staff whose collars shade more toward
    blue.  (Outsourcing is useful in increasing opacity, segregating
    workers, and setting them up to have conflicting interests.)

    If people start applying their capacity for critical thought to the
    proposition that the CEO is 40,000 times more productive than a
    "Software Engineer II", nothing good will happen.

_Is_ "productivity" among programmers, however defined and measured,
nonuniform?  Likely yes.  Has our industry studied the question in a
serious way, applying rigorous experimental design and statistical
analysis?  Perhaps not.

And if we did, would any of the people making this claim read or
comprehend the research if it didn't support their biases?

You already know the answer.

We utter myths about falsifiable propositions not because we care about
their truth values, but precisely because we don't.

Regards,
Branden