[TUHS] Re: SCCS roach motel

[TUHS] Re: SCCS roach motel

[Norman Wilson]

Rob Pike:

  According to the Unix room fortunes file, the actual quote is

  SCCS: the source-code motel -- your code checks in but it never checks out. Ken Thompson

====

As a Unix-room-culture aside: I believe this quote was what
inspired Andrew Hume to call his backup system the File Motel.

Norman Wilson
Toronto ON

[TUHS] Re: SCCS roach motel

[andrew]

indeed, it was.

> On Dec 13, 2024, at 2:33 PM, Norman Wilson <norman@oclsc.org> wrote:
> 
> Rob Pike:
> 
>  According to the Unix room fortunes file, the actual quote is
> 
>  SCCS: the source-code motel -- your code checks in but it never checks out. Ken Thompson
> 
> ====
> 
> As a Unix-room-culture aside: I believe this quote was what
> inspired Andrew Hume to call his backup system the File Motel.
> 
> Norman Wilson
> Toronto ON

[TUHS] Re: SCCS roach motel

[Norman Wilson]

This is verging on COFF material, and I won't mind if someone
moves the discussion thither:

Clem Cole:

  As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred
  choice.

====

I have to admit SCCS is one of the many pieces of software
I tried for a week or two > 40 years ago and abandoned because
I couldn't stand it.  I don't think I ever tried RCS, because
I could see it didn't what I saw as the underlying problems.
CVS likewise.  Subversion was the earliest version-control
system that felt usable to me.

What bugged me so much?  The earlier systems were focussed
entirely (or for CVS, almost entirely) on individual files.
There was no way to link changes that affected more than one
file:
-- SCCS and RCS don't (so far as I can tell) understand any
relation between files at all.  When I'm working on a real
project of any size, there is almost always more than one
file, and there are often changes that affect multiple files
at once: if an interface changes, the changes to definitions
and uses really all should be a single commit, logged in one
go and reversible as a single coordinated operation; if I
refactor code, moving it between files and perhaps adding
files or removing them, likewise.  It is possible to check
in a bunch of files at once and reuse the log message, but
I couldn't find a way to make it a true coordinated single
commit; and neither SCCS nor RCS has a way I could find to
record, as a structured commit, that files are added or
removed from a directory or directory tree.
-- CVS can track adds and deletes and can bundle changes
into a single commit on the command line, but the changes
are still stored individually per file.
-- None of the systems even tries to track file renames,
something I also do occasionally as programs and especially
documentation evolves.

It wasn't until I discovered Subversion that I started using
version control regularly in my own work.

Ironically, a few years after I began using svn for myself,
I ended up working in places that had great compost heaps
of RCS and CVS.  This would have made sense in the 1990s,
but it was in the 2010s, and continues in the 2020s.

I now use hg for my personal work, and have been attempting
to drag my workplace into using git if only so new hires
don't have to learn clumsy outdated stuff just to do their
jobs.  I expect to retire (not for this reason) before I
really succeed, though.

Norman Wilson
Toronto ON

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote:
> This is verging on COFF material, and I won't mind if someone
> moves the discussion thither:
> 
> Clem Cole:
> 
>   As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred
>   choice.
> 
> ====
> 
> I have to admit SCCS is one of the many pieces of software
> I tried for a week or two > 40 years ago and abandoned because
> I couldn't stand it.  I don't think I ever tried RCS, because
> I could see it didn't what I saw as the underlying problems.
> CVS likewise.  Subversion was the earliest version-control
> system that felt usable to me.
> 
> What bugged me so much?  The earlier systems were focussed
> entirely (or for CVS, almost entirely) on individual files.
> There was no way to link changes that affected more than one
> file:

That was the problem that BitKeeper solved.  There was an extra step,
bk commit, that glued all the files together in an atomic commit.
That and each commit was like a CVS tag, you can roll the history back
to any commit, no tags are needed.  That's because while you think of a
revision as 1.5 or whatever, and BitKeeper had that interface, the real
name is a a provably unique key made up of

    user@host|path/to/file.c|time_t|sccs_cksum

We called those "keys" and you could use a key any place you could use
a revision.

I'll spare you how we made them unique, but I can tell you that in two
decades of BK use on every continent other than the artic, we've never
had a key collision.  Does require that you use DNS.

BTW, the CVS/SCCS/RCS importers guessed at commit boundaries and made
those systems yield commits.  We looked at author, check in comments,
and time stamps, same author, same comments and within a short window,
that's the same commit.

--lm

[TUHS] Re: SCCS roach motel

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 2197 bytes --]

On Fri, Dec 13, 2024, 4:19 PM Larry McVoy <lm@mcvoy.com> wrote:

> On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote:
> > This is verging on COFF material, and I won't mind if someone
> > moves the discussion thither:
> >
> > Clem Cole:
> >
> >   As a satisfied user of SCCS (and later Bitkeeper), it's still my
> preferred
> >   choice.
> >
> > ====
> >
> > I have to admit SCCS is one of the many pieces of software
> > I tried for a week or two > 40 years ago and abandoned because
> > I couldn't stand it.  I don't think I ever tried RCS, because
> > I could see it didn't what I saw as the underlying problems.
> > CVS likewise.  Subversion was the earliest version-control
> > system that felt usable to me.
> >
> > What bugged me so much?  The earlier systems were focussed
> > entirely (or for CVS, almost entirely) on individual files.
> > There was no way to link changes that affected more than one
> > file:
>
> That was the problem that BitKeeper solved.  There was an extra step,
> bk commit, that glued all the files together in an atomic commit.
> That and each commit was like a CVS tag, you can roll the history back
> to any commit, no tags are needed.  That's because while you think of a
> revision as 1.5 or whatever, and BitKeeper had that interface, the real
> name is a a provably unique key made up of
>
>     user@host|path/to/file.c|time_t|sccs_cksum
>
> We called those "keys" and you could use a key any place you could use
> a revision.
>

That's a nice feature... Too bad we don't have it for the historic SCCS
trees.

Also, the historical SCCS trees lack metatdata about file renames (which
were done by moving the ,s files)...

Warner

I'll spare you how we made them unique, but I can tell you that in two
> decades of BK use on every continent other than the artic, we've never
> had a key collision.  Does require that you use DNS.
>
> BTW, the CVS/SCCS/RCS importers guessed at commit boundaries and made
> those systems yield commits.  We looked at author, check in comments,
> and time stamps, same author, same comments and within a short window,
> that's the same commit.
>
> --lm
>

[-- Attachment #2: Type: text/html, Size: 3058 bytes --]

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 04:38:49PM -0700, Warner Losh wrote:
> On Fri, Dec 13, 2024, 4:19???PM Larry McVoy <lm@mcvoy.com> wrote:
> 
> > On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote:
> > > This is verging on COFF material, and I won't mind if someone
> > > moves the discussion thither:
> > >
> > > Clem Cole:
> > >
> > >   As a satisfied user of SCCS (and later Bitkeeper), it's still my
> > preferred
> > >   choice.
> > >
> > > ====
> > >
> > > I have to admit SCCS is one of the many pieces of software
> > > I tried for a week or two > 40 years ago and abandoned because
> > > I couldn't stand it.  I don't think I ever tried RCS, because
> > > I could see it didn't what I saw as the underlying problems.
> > > CVS likewise.  Subversion was the earliest version-control
> > > system that felt usable to me.
> > >
> > > What bugged me so much?  The earlier systems were focussed
> > > entirely (or for CVS, almost entirely) on individual files.
> > > There was no way to link changes that affected more than one
> > > file:
> >
> > That was the problem that BitKeeper solved.  There was an extra step,
> > bk commit, that glued all the files together in an atomic commit.
> > That and each commit was like a CVS tag, you can roll the history back
> > to any commit, no tags are needed.  That's because while you think of a
> > revision as 1.5 or whatever, and BitKeeper had that interface, the real
> > name is a a provably unique key made up of
> >
> >     user@host|path/to/file.c|time_t|sccs_cksum
> >
> > We called those "keys" and you could use a key any place you could use
> > a revision.
> >
> 
> That's a nice feature... Too bad we don't have it for the historic SCCS
> trees.

SCCS doesn't version pathnames so you have to fake it.  You could look at
BitKeeper's import -tSCCS to see what we did, I would think we would have
done something sensible.

> Also, the historical SCCS trees lack metatdata about file renames (which
> were done by moving the ,s files)...

See above.  BitKeeper versioned not only path names (they are an attribute
of each delta) but also file types: symlink, regular, not sure if we ever
did hard links.
-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

IEEE Transactions on Software Engineering has asked me to write a
retrospective on the influence of SCCS over the last 50 years, as my SCCS
paper was published in 1975. They consider it one of the most influential
papers from TSE's first decade.

There's a funny quote from Ken Thompson that circulates from time-to-time:

"SCCS, the source motel! Programs check in and never check out!"

But nobody seems to know what it means exactly. As part of my research, I
asked Ken what the quote meant, sunce I wanted to include it. He explained
that it refers to SCCS storing binary data in its repository file,
preventing UNIX text tools from operating on the file.

Of course, this is only one of SCCS's many weaknesses. If you have anything
funny about any of the others, post it here. I already have all the boring
usual stuff (e.g., long-term locks, file-oriented, no merging).

Marc Rochkind
mrochkind.com

[-- Attachment #2: Type: text/html, Size: 1187 bytes --]

[TUHS] Re: SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

On Fri, Dec 13, 2024 at 10:03 AM Mark Seiden <mseiden@gmail.com> wrote:

> (I know there is a special place in hell for those who explain a joke,
> but, you asked…)
>
> it’s just an allusion to the Black Flag Roach Motel product (still being
> produced)
> which has a trademark on the phrase “Roaches Check in… But they Don’t
> Check Out”.
>
> Yeah, I knew that much. My question to Ken was about what this was saying
about SCCS.

Marc

[-- Attachment #2: Type: text/html, Size: 920 bytes --]

[TUHS] Re: SCCS roach motel

[Dan Cross]

On Fri, Dec 13, 2024 at 3:28 PM Marc Rochkind <mrochkind@gmail.com> wrote:
> On Fri, Dec 13, 2024 at 10:03 AM Mark Seiden <mseiden@gmail.com> wrote:
>> (I know there is a special place in hell for those who explain a joke, but, you asked…)
>>
>> it’s just an allusion to the Black Flag Roach Motel product (still being produced)
>> which has a trademark on the phrase “Roaches Check in… But they Don’t Check Out”.
>
> Yeah, I knew that much. My question to Ken was about what this was saying about SCCS.

I had heard a story once, that based on what you wrote today, I now
think is apocryphal and never actually happened.

The story was that there was a bug in early SCCS where a source file
had to be checked in twice before it could be (successfully) checked
out. The bug was such that if one only checked it in once and tried to
check it out, it would truncate the file. "Programmers using it just
learned to check in twice."

I never used SCCS extensively, and certainly never observed that
behavior, so chucked it up to the bug having been long fixed. As I
mentioned above, I doubt it was ever there to begin with. A fun story,
though.

        - Dan C.

[TUHS] Re: SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

On Fri, Dec 13, 2024 at 2:10 PM Dan Cross <crossd@gmail.com> wrote:

>
> I had heard a story once, that based on what you wrote today, I now
> think is apocryphal and never actually happened.
>
> The story was that there was a bug in early SCCS where a source file
> had to be checked in twice before it could be (successfully) checked
> out. The bug was such that if one only checked it in once and tried to
> check it out, it would truncate the file. "Programmers using it just
> learned to check in twice."
>
> I never used SCCS extensively, and certainly never observed that
> behavior, so chucked it up to the bug having been long fixed. As I
> mentioned above, I doubt it was ever there to begin with. A fun story,
> though.
>
>         - Dan C.
>

This is completely ridiculous. No program I wrote ever had one of those bug
things.

Marc

[-- Attachment #2: Type: text/html, Size: 1296 bytes --]

[TUHS] Re: SCCS roach motel

[Dan Cross]

On Fri, Dec 13, 2024 at 8:12 PM Marc Rochkind <mrochkind@gmail.com> wrote:
> On Fri, Dec 13, 2024 at 2:10 PM Dan Cross <crossd@gmail.com> wrote:
>> I had heard a story once, that based on what you wrote today, I now
>> think is apocryphal and never actually happened.
>>
>> The story was that there was a bug in early SCCS where a source file
>> had to be checked in twice before it could be (successfully) checked
>> out. The bug was such that if one only checked it in once and tried to
>> check it out, it would truncate the file. "Programmers using it just
>> learned to check in twice."
>>
>> I never used SCCS extensively, and certainly never observed that
>> behavior, so chucked it up to the bug having been long fixed. As I
>> mentioned above, I doubt it was ever there to begin with. A fun story,
>> though.
>
> This is completely ridiculous. No program I wrote ever had one of those bug things.

Oh dear, I do hope no offense was taken.  As I said, I firmly believe
the story was apocryphal.

The identity of the person from whom I heard it is probably known to
at least some of the USENIX old school crew; I rather thought at the
time they were under the impression that SCCS was the result of a
large team, or perhaps some kind of fork.

Anyway, I don't mean to belabor the point. Please accept my apologies
if the story rubbed you the wrong way.

        - Dan C.

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 08:27:54PM -0500, Dan Cross wrote:
> On Fri, Dec 13, 2024 at 8:12???PM Marc Rochkind <mrochkind@gmail.com> wrote:
> > On Fri, Dec 13, 2024 at 2:10???PM Dan Cross <crossd@gmail.com> wrote:
> >> I had heard a story once, that based on what you wrote today, I now
> >> think is apocryphal and never actually happened.
> >>
> >> The story was that there was a bug in early SCCS where a source file
> >> had to be checked in twice before it could be (successfully) checked
> >> out. The bug was such that if one only checked it in once and tried to
> >> check it out, it would truncate the file. "Programmers using it just
> >> learned to check in twice."
> >>
> >> I never used SCCS extensively, and certainly never observed that
> >> behavior, so chucked it up to the bug having been long fixed. As I
> >> mentioned above, I doubt it was ever there to begin with. A fun story,
> >> though.
> >
> > This is completely ridiculous. No program I wrote ever had one of those bug things.
> 
> Oh dear, I do hope no offense was taken.  As I said, I firmly believe
> the story was apocryphal.
> 
> The identity of the person from whom I heard it is probably known to
> at least some of the USENIX old school crew; I rather thought at the
> time they were under the impression that SCCS was the result of a
> large team, or perhaps some kind of fork.
> 
> Anyway, I don't mean to belabor the point. Please accept my apologies
> if the story rubbed you the wrong way.

I'm 90% sure Marc was joking, I got it.

[TUHS] Re: SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 1749 bytes --]

That was a joke, Dan.

I've been programming for 58 years, and nobody has put in more bugs than
me, I'm sure.

Marc

On Fri, Dec 13, 2024 at 6:28 PM Dan Cross <crossd@gmail.com> wrote:

> On Fri, Dec 13, 2024 at 8:12 PM Marc Rochkind <mrochkind@gmail.com> wrote:
> > On Fri, Dec 13, 2024 at 2:10 PM Dan Cross <crossd@gmail.com> wrote:
> >> I had heard a story once, that based on what you wrote today, I now
> >> think is apocryphal and never actually happened.
> >>
> >> The story was that there was a bug in early SCCS where a source file
> >> had to be checked in twice before it could be (successfully) checked
> >> out. The bug was such that if one only checked it in once and tried to
> >> check it out, it would truncate the file. "Programmers using it just
> >> learned to check in twice."
> >>
> >> I never used SCCS extensively, and certainly never observed that
> >> behavior, so chucked it up to the bug having been long fixed. As I
> >> mentioned above, I doubt it was ever there to begin with. A fun story,
> >> though.
> >
> > This is completely ridiculous. No program I wrote ever had one of those
> bug things.
>
> Oh dear, I do hope no offense was taken.  As I said, I firmly believe
> the story was apocryphal.
>
> The identity of the person from whom I heard it is probably known to
> at least some of the USENIX old school crew; I rather thought at the
> time they were under the impression that SCCS was the result of a
> large team, or perhaps some kind of fork.
>
> Anyway, I don't mean to belabor the point. Please accept my apologies
> if the story rubbed you the wrong way.
>
>         - Dan C.
>


-- 
*My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*

[-- Attachment #2: Type: text/html, Size: 2584 bytes --]

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 06:11:57PM -0700, Marc Rochkind wrote:
> On Fri, Dec 13, 2024 at 2:10???PM Dan Cross <crossd@gmail.com> wrote:
> 
> >
> > I had heard a story once, that based on what you wrote today, I now
> > think is apocryphal and never actually happened.
> >
> > The story was that there was a bug in early SCCS where a source file
> > had to be checked in twice before it could be (successfully) checked
> > out. The bug was such that if one only checked it in once and tried to
> > check it out, it would truncate the file. "Programmers using it just
> > learned to check in twice."
> >
> > I never used SCCS extensively, and certainly never observed that
> > behavior, so chucked it up to the bug having been long fixed. As I
> > mentioned above, I doubt it was ever there to begin with. A fun story,
> > though.
> >
> >         - Dan C.
> >
> 
> This is completely ridiculous. No program I wrote ever had one of those bug
> things.

Heh.  Me too, bug?  What's that?

BTW, this thread, Marc, has had me wandering into the BitKeeper code
over and over, and while I'm done with coding, it was super pleasant to
go look at that code.  We didn't win but we wrote really nice C.  I'm
proud of that, it's an old guy thing, it's nice to look back and see
we did a good job.

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
> IEEE Transactions on Software Engineering has asked me to write a
> retrospective on the influence of SCCS over the last 50 years, as my SCCS
> paper was published in 1975. They consider it one of the most influential
> papers from TSE's first decade.
> 
> There's a funny quote from Ken Thompson that circulates from time-to-time:
> 
> "SCCS, the source motel! Programs check in and never check out!"
> 
> But nobody seems to know what it means exactly. As part of my research, I
> asked Ken what the quote meant, sunce I wanted to include it. He explained
> that it refers to SCCS storing binary data in its repository file,
> preventing UNIX text tools from operating on the file.
> 
> Of course, this is only one of SCCS's many weaknesses. If you have anything
> funny about any of the others, post it here. I already have all the boring
> usual stuff (e.g., long-term locks, file-oriented, no merging).

Warning, I know more about SCCS than the average person, I've
reimplemented it from scratch and then built BitKeeper on top of an
extended SCCS file format.  So lots of info coming.  Rick Smith and
Wayne Scott know as much as I do, Rick knows more, he joined me and
promptly started fixing my SCCS implementation.  So far as I know, 
there is not a more knowledgable person that Rick when it comes to
weave file formats.

SCCS's strength is the weave format.  It's largely not understood, even
by other people working in source management.  Here's the benefit of
that weave (if people use it, which, other than BitKeeper, they don't,
looking at you, Clearcase, you had a weave and didn't use it): SCCS can
pass merge data by reference, everyone else copies the data.

SCCS is a set based system.   Each node has a revision number, like 1.5,
but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
like 1.5.1.1, it is _impossible_ to build the history graph from the 
revisions, you can in simple graphs but as soon as you branch from a
branch, all bets are off.

The graph is built from what BitKeeper called serial numbers.  Each node
in the graph has at least 2 serials, one that names that node in the
graph, and one that is the parent.

So if I have a file with 5 revisions in straight line history, the
internal stuff will look something like

rev	me	parent
1.5	5	4
1.4	4	3
1.3	3	2
1.2	2	1
1.1	1	0

So what's the set?  Pretty simple for straight line history, you walk
the history from the rev that you want, adding the "me" serial and
going to the parent, repeat until the parent is 0.

Suppose you branch from rev 1.3.

rev	me	parent
1.3.1.1	6	3
1.5	5	4
1.4	4	3
1.3	3	2
...

See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
and 4.  If you understand that, you are starting to understand the set
and how it is constructed.

Did you know you can have an argument in the revision history without
adding anything to the data part?  SCCS has the ability to include 
and/or exclude serials as part of a delta.  Lets say Marc looked at
my 1.5 and thought it was garbage.  He can exclude it from the 
set like so:

rev	me	parent	include	exclude
1.6	7	5	0	5
1.3.1.1	6	3
1.5	5	4
1.4	4	3
1.3	3	2
...

That doesn't change the data part of the file AT ALL, it's just saying
Marc doesn't want anyone to see the 1.5 changes.

To understand that, you need to know how SCCS checks out a file.  And
you need to know how the data is stored.  Which is in a weave.  RCS,
and pretty much everything that followed it, doesn't use a weave at
all.  RCS stores the most recent version of the file as a complete 
copy of the checked out file.  Then each delta working backwards up
the trunk is a patch, what diff produces.

Think about what that means for working on a branch.  You have to start
with the most recent version of the file, apply backward patches to go
to earlier versions all the way back to the branch point, then apply
forward patches to work your way to tip of the branch.  Ask Dave Miller
how pleasant it is to work on gcc on a branch.  It's crazy slow and
painful.

So how does SCCS do it?  Lets say the first version of a file is

1
2
3
4
5

The data portion of the history file will look like:

^AI 1
1
2
3
4
5
^AE 1

SCCS used ^A at the beginning of a line to mean "this is metadata for
SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
with a ^AE which means end.  The number after is the serial.  So that
weave says "If serial 1 is in your set, everything after ^AI 1 is part
of that set until you hit the matching ^AE 1.

Lets say the 2nd version is

1
2
serial 2 added this
3
4

Notice that serial 2 deleted what was line 5.

^AI 1
1
2
^AI 2
serial 2 added this
^AE 2
3
4
^AD 2
5
^AE 2
^AE 1

So now we can start to see how you walk the weave.  If I'm trying to
check out 1.1 aka serial 1, I build a set that has only '1' in the set.
I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
which means print each data line.  I hit ^AI 2, that's not in my set,
so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
2 is NOT in my set, so I stay in print mode.  Etc.

Let's make a branch, 1.1.1.1, with lots of data.

1
2
3
branch line 1
branch line 2
...
branch line 10000
4
5

^AI 1
1
2
^AI 2
serial 2 added this
^AE 2
3
^AI 3
branch line 1
branch line 2
...
branch line 10000
^AE 3
4
^AD 2
5
^AE 2
^AE 1

So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
print anything inserted by either of those, delete anything deleted
by those, skip anything inserted by anything not in the set, skip any
deletes by anything not in the set.

The delta table looks like this, notice I've added an author column:

rev	me	parent	include	exclude	author
1.1.1.1	3	1			lm
1.2	2	1			lm
1.1	1	0			lm

If you followed all that, you can see how SCCS can merge by reference.
Lets say Clem decides to merge my branch onto the trunk. The delta table
will get a new entry:

rev	me	parent	include	exclude	author
1.3	4	2	3		clem
1.1.1.1	3	1			lm
1.2	2	1			lm
1.1	1	0			lm

The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3 way
merge, it will find the lines "3" and "5" as anchor points in both versions,
so it is a simple insert with no new data added to the weave.

Here's some magic that *everyone* else gets wrong when they pass by value:
In a system that passes by value (copies) the data, the merge done by clem
would have an annotated listing like so:

lm	1
lm	2
lm	3
clem	branch line 1
clem	branch line 2
clem	...
clem	branch line 10000
lm	4
lm	5

Since it copied the data, it looks like Clem wrote it but he didn't, he
just automerged it.  In SCCS/BitKeeper it would look like:

lm	1
lm	2
lm	3
lm	branch line 1
lm	branch line 2
lm	...
lm	branch line 10000
lm	4
lm	5

which is correct, all of those lines were authored by one person.  The only
time the merger should show up as an author is if there was a conflict, 
however the merger resolved that conflict is new work and should be
authored by the merger.

What BitKeeper did, that was a profound step forward, was make the idea
of a repository a formal thing and introduced the concept of changesets
that keeps track of all this stuff at the repository level.  So it does
all this stuff at the file level but you don't have to do that low level
work.  You could think of SCCS as assembly and BitKeeper as more like C,
it upleveled things to the point that humans can manage the repository
easily.

Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
questions?  It should be obvious that I *love* SCCS, it's a dramatically
better file format than a patch based one, you can get *any* version of
the file in constant time, authorship can be preserved across versions,
it's pretty brilliant and I consider myself blessed to be posting this
in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
to the RCS guy, who got a PhD for RCS (give me a break) and did the
world a huge disservice by bad mouthing SCCS so he could promote RCS.

--lm

[TUHS] Re: SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 10347 bytes --]

Larry, thanks for this. I had read some things you've written about the
weave before, but not with this level of detail. Sounds weird, but I didn't
really appreciate the implications of the weave even though I'm the guy who
thought it up. I did understand the importance of not copying data if you
can reference it, which is a principle of database design (normal forms,
etc).

In my paper, I can add a little more about the weave and its advantages.
Aside from this TUHS post, is there something I can put in the References
that people can find?

Question: Is this right, that TeamWare was literally layered on top of AT&T
SCCS, but BitKeeper was layered on your implementation of SCCS? Or, was it
more complicated than that?

Was your implementation of SCCS ever released by itself?

Marc

On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
> > IEEE Transactions on Software Engineering has asked me to write a
> > retrospective on the influence of SCCS over the last 50 years, as my SCCS
> > paper was published in 1975. They consider it one of the most influential
> > papers from TSE's first decade.
> >
> > There's a funny quote from Ken Thompson that circulates from
> time-to-time:
> >
> > "SCCS, the source motel! Programs check in and never check out!"
> >
> > But nobody seems to know what it means exactly. As part of my research, I
> > asked Ken what the quote meant, sunce I wanted to include it. He
> explained
> > that it refers to SCCS storing binary data in its repository file,
> > preventing UNIX text tools from operating on the file.
> >
> > Of course, this is only one of SCCS's many weaknesses. If you have
> anything
> > funny about any of the others, post it here. I already have all the
> boring
> > usual stuff (e.g., long-term locks, file-oriented, no merging).
>
> Warning, I know more about SCCS than the average person, I've
> reimplemented it from scratch and then built BitKeeper on top of an
> extended SCCS file format.  So lots of info coming.  Rick Smith and
> Wayne Scott know as much as I do, Rick knows more, he joined me and
> promptly started fixing my SCCS implementation.  So far as I know,
> there is not a more knowledgable person that Rick when it comes to
> weave file formats.
>
> SCCS's strength is the weave format.  It's largely not understood, even
> by other people working in source management.  Here's the benefit of
> that weave (if people use it, which, other than BitKeeper, they don't,
> looking at you, Clearcase, you had a weave and didn't use it): SCCS can
> pass merge data by reference, everyone else copies the data.
>
> SCCS is a set based system.   Each node has a revision number, like 1.5,
> but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
> like 1.5.1.1, it is _impossible_ to build the history graph from the
> revisions, you can in simple graphs but as soon as you branch from a
> branch, all bets are off.
>
> The graph is built from what BitKeeper called serial numbers.  Each node
> in the graph has at least 2 serials, one that names that node in the
> graph, and one that is the parent.
>
> So if I have a file with 5 revisions in straight line history, the
> internal stuff will look something like
>
> rev     me      parent
> 1.5     5       4
> 1.4     4       3
> 1.3     3       2
> 1.2     2       1
> 1.1     1       0
>
> So what's the set?  Pretty simple for straight line history, you walk
> the history from the rev that you want, adding the "me" serial and
> going to the parent, repeat until the parent is 0.
>
> Suppose you branch from rev 1.3.
>
> rev     me      parent
> 1.3.1.1 6       3
> 1.5     5       4
> 1.4     4       3
> 1.3     3       2
> ...
>
> See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
> and 4.  If you understand that, you are starting to understand the set
> and how it is constructed.
>
> Did you know you can have an argument in the revision history without
> adding anything to the data part?  SCCS has the ability to include
> and/or exclude serials as part of a delta.  Lets say Marc looked at
> my 1.5 and thought it was garbage.  He can exclude it from the
> set like so:
>
> rev     me      parent  include exclude
> 1.6     7       5       0       5
> 1.3.1.1 6       3
> 1.5     5       4
> 1.4     4       3
> 1.3     3       2
> ...
>
> That doesn't change the data part of the file AT ALL, it's just saying
> Marc doesn't want anyone to see the 1.5 changes.
>
> To understand that, you need to know how SCCS checks out a file.  And
> you need to know how the data is stored.  Which is in a weave.  RCS,
> and pretty much everything that followed it, doesn't use a weave at
> all.  RCS stores the most recent version of the file as a complete
> copy of the checked out file.  Then each delta working backwards up
> the trunk is a patch, what diff produces.
>
> Think about what that means for working on a branch.  You have to start
> with the most recent version of the file, apply backward patches to go
> to earlier versions all the way back to the branch point, then apply
> forward patches to work your way to tip of the branch.  Ask Dave Miller
> how pleasant it is to work on gcc on a branch.  It's crazy slow and
> painful.
>
> So how does SCCS do it?  Lets say the first version of a file is
>
> 1
> 2
> 3
> 4
> 5
>
> The data portion of the history file will look like:
>
> ^AI 1
> 1
> 2
> 3
> 4
> 5
> ^AE 1
>
> SCCS used ^A at the beginning of a line to mean "this is metadata for
> SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
> with a ^AE which means end.  The number after is the serial.  So that
> weave says "If serial 1 is in your set, everything after ^AI 1 is part
> of that set until you hit the matching ^AE 1.
>
> Lets say the 2nd version is
>
> 1
> 2
> serial 2 added this
> 3
> 4
>
> Notice that serial 2 deleted what was line 5.
>
> ^AI 1
> 1
> 2
> ^AI 2
> serial 2 added this
> ^AE 2
> 3
> 4
> ^AD 2
> 5
> ^AE 2
> ^AE 1
>
> So now we can start to see how you walk the weave.  If I'm trying to
> check out 1.1 aka serial 1, I build a set that has only '1' in the set.
> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
> which means print each data line.  I hit ^AI 2, that's not in my set,
> so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
> I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
> 2 is NOT in my set, so I stay in print mode.  Etc.
>
> Let's make a branch, 1.1.1.1, with lots of data.
>
> 1
> 2
> 3
> branch line 1
> branch line 2
> ...
> branch line 10000
> 4
> 5
>
> ^AI 1
> 1
> 2
> ^AI 2
> serial 2 added this
> ^AE 2
> 3
> ^AI 3
> branch line 1
> branch line 2
> ...
> branch line 10000
> ^AE 3
> 4
> ^AD 2
> 5
> ^AE 2
> ^AE 1
>
> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
> print anything inserted by either of those, delete anything deleted
> by those, skip anything inserted by anything not in the set, skip any
> deletes by anything not in the set.
>
> The delta table looks like this, notice I've added an author column:
>
> rev     me      parent  include exclude author
> 1.1.1.1 3       1                       lm
> 1.2     2       1                       lm
> 1.1     1       0                       lm
>
> If you followed all that, you can see how SCCS can merge by reference.
> Lets say Clem decides to merge my branch onto the trunk. The delta table
> will get a new entry:
>
> rev     me      parent  include exclude author
> 1.3     4       2       3               clem
> 1.1.1.1 3       1                       lm
> 1.2     2       1                       lm
> 1.1     1       0                       lm
>
> The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3 way
> merge, it will find the lines "3" and "5" as anchor points in both
> versions,
> so it is a simple insert with no new data added to the weave.
>
> Here's some magic that *everyone* else gets wrong when they pass by value:
> In a system that passes by value (copies) the data, the merge done by clem
> would have an annotated listing like so:
>
> lm      1
> lm      2
> lm      3
> clem    branch line 1
> clem    branch line 2
> clem    ...
> clem    branch line 10000
> lm      4
> lm      5
>
> Since it copied the data, it looks like Clem wrote it but he didn't, he
> just automerged it.  In SCCS/BitKeeper it would look like:
>
> lm      1
> lm      2
> lm      3
> lm      branch line 1
> lm      branch line 2
> lm      ...
> lm      branch line 10000
> lm      4
> lm      5
>
> which is correct, all of those lines were authored by one person.  The only
> time the merger should show up as an author is if there was a conflict,
> however the merger resolved that conflict is new work and should be
> authored by the merger.
>
> What BitKeeper did, that was a profound step forward, was make the idea
> of a repository a formal thing and introduced the concept of changesets
> that keeps track of all this stuff at the repository level.  So it does
> all this stuff at the file level but you don't have to do that low level
> work.  You could think of SCCS as assembly and BitKeeper as more like C,
> it upleveled things to the point that humans can manage the repository
> easily.
>
> Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
> questions?  It should be obvious that I *love* SCCS, it's a dramatically
> better file format than a patch based one, you can get *any* version of
> the file in constant time, authorship can be preserved across versions,
> it's pretty brilliant and I consider myself blessed to be posting this
> in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
> to the RCS guy, who got a PhD for RCS (give me a break) and did the
> world a huge disservice by bad mouthing SCCS so he could promote RCS.
>
> --lm
>


-- 
*My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*

[-- Attachment #2: Type: text/html, Size: 12092 bytes --]

[TUHS] Re: SCCS roach motel

[Marc Rochkind]

[-- Attachment #1: Type: text/plain, Size: 10947 bytes --]

Larry, I found this:

https://www.bitkeeper.org/src-notes/SCCSWEAVE.html

A good reference?

Marc

On Fri, Dec 13, 2024 at 11:32 AM Marc Rochkind <mrochkind@gmail.com> wrote:

> Larry, thanks for this. I had read some things you've written about the
> weave before, but not with this level of detail. Sounds weird, but I didn't
> really appreciate the implications of the weave even though I'm the guy who
> thought it up. I did understand the importance of not copying data if you
> can reference it, which is a principle of database design (normal forms,
> etc).
>
> In my paper, I can add a little more about the weave and its advantages.
> Aside from this TUHS post, is there something I can put in the References
> that people can find?
>
> Question: Is this right, that TeamWare was literally layered on top of
> AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or,
> was it more complicated than that?
>
> Was your implementation of SCCS ever released by itself?
>
> Marc
>
> On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
>> > IEEE Transactions on Software Engineering has asked me to write a
>> > retrospective on the influence of SCCS over the last 50 years, as my
>> SCCS
>> > paper was published in 1975. They consider it one of the most
>> influential
>> > papers from TSE's first decade.
>> >
>> > There's a funny quote from Ken Thompson that circulates from
>> time-to-time:
>> >
>> > "SCCS, the source motel! Programs check in and never check out!"
>> >
>> > But nobody seems to know what it means exactly. As part of my research,
>> I
>> > asked Ken what the quote meant, sunce I wanted to include it. He
>> explained
>> > that it refers to SCCS storing binary data in its repository file,
>> > preventing UNIX text tools from operating on the file.
>> >
>> > Of course, this is only one of SCCS's many weaknesses. If you have
>> anything
>> > funny about any of the others, post it here. I already have all the
>> boring
>> > usual stuff (e.g., long-term locks, file-oriented, no merging).
>>
>> Warning, I know more about SCCS than the average person, I've
>> reimplemented it from scratch and then built BitKeeper on top of an
>> extended SCCS file format.  So lots of info coming.  Rick Smith and
>> Wayne Scott know as much as I do, Rick knows more, he joined me and
>> promptly started fixing my SCCS implementation.  So far as I know,
>> there is not a more knowledgable person that Rick when it comes to
>> weave file formats.
>>
>> SCCS's strength is the weave format.  It's largely not understood, even
>> by other people working in source management.  Here's the benefit of
>> that weave (if people use it, which, other than BitKeeper, they don't,
>> looking at you, Clearcase, you had a weave and didn't use it): SCCS can
>> pass merge data by reference, everyone else copies the data.
>>
>> SCCS is a set based system.   Each node has a revision number, like 1.5,
>> but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
>> like 1.5.1.1, it is _impossible_ to build the history graph from the
>> revisions, you can in simple graphs but as soon as you branch from a
>> branch, all bets are off.
>>
>> The graph is built from what BitKeeper called serial numbers.  Each node
>> in the graph has at least 2 serials, one that names that node in the
>> graph, and one that is the parent.
>>
>> So if I have a file with 5 revisions in straight line history, the
>> internal stuff will look something like
>>
>> rev     me      parent
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> 1.2     2       1
>> 1.1     1       0
>>
>> So what's the set?  Pretty simple for straight line history, you walk
>> the history from the rev that you want, adding the "me" serial and
>> going to the parent, repeat until the parent is 0.
>>
>> Suppose you branch from rev 1.3.
>>
>> rev     me      parent
>> 1.3.1.1 6       3
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> ...
>>
>> See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
>> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
>> and 4.  If you understand that, you are starting to understand the set
>> and how it is constructed.
>>
>> Did you know you can have an argument in the revision history without
>> adding anything to the data part?  SCCS has the ability to include
>> and/or exclude serials as part of a delta.  Lets say Marc looked at
>> my 1.5 and thought it was garbage.  He can exclude it from the
>> set like so:
>>
>> rev     me      parent  include exclude
>> 1.6     7       5       0       5
>> 1.3.1.1 6       3
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> ...
>>
>> That doesn't change the data part of the file AT ALL, it's just saying
>> Marc doesn't want anyone to see the 1.5 changes.
>>
>> To understand that, you need to know how SCCS checks out a file.  And
>> you need to know how the data is stored.  Which is in a weave.  RCS,
>> and pretty much everything that followed it, doesn't use a weave at
>> all.  RCS stores the most recent version of the file as a complete
>> copy of the checked out file.  Then each delta working backwards up
>> the trunk is a patch, what diff produces.
>>
>> Think about what that means for working on a branch.  You have to start
>> with the most recent version of the file, apply backward patches to go
>> to earlier versions all the way back to the branch point, then apply
>> forward patches to work your way to tip of the branch.  Ask Dave Miller
>> how pleasant it is to work on gcc on a branch.  It's crazy slow and
>> painful.
>>
>> So how does SCCS do it?  Lets say the first version of a file is
>>
>> 1
>> 2
>> 3
>> 4
>> 5
>>
>> The data portion of the history file will look like:
>>
>> ^AI 1
>> 1
>> 2
>> 3
>> 4
>> 5
>> ^AE 1
>>
>> SCCS used ^A at the beginning of a line to mean "this is metadata for
>> SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
>> with a ^AE which means end.  The number after is the serial.  So that
>> weave says "If serial 1 is in your set, everything after ^AI 1 is part
>> of that set until you hit the matching ^AE 1.
>>
>> Lets say the 2nd version is
>>
>> 1
>> 2
>> serial 2 added this
>> 3
>> 4
>>
>> Notice that serial 2 deleted what was line 5.
>>
>> ^AI 1
>> 1
>> 2
>> ^AI 2
>> serial 2 added this
>> ^AE 2
>> 3
>> 4
>> ^AD 2
>> 5
>> ^AE 2
>> ^AE 1
>>
>> So now we can start to see how you walk the weave.  If I'm trying to
>> check out 1.1 aka serial 1, I build a set that has only '1' in the set.
>> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
>> which means print each data line.  I hit ^AI 2, that's not in my set,
>> so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
>> I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
>> 2 is NOT in my set, so I stay in print mode.  Etc.
>>
>> Let's make a branch, 1.1.1.1, with lots of data.
>>
>> 1
>> 2
>> 3
>> branch line 1
>> branch line 2
>> ...
>> branch line 10000
>> 4
>> 5
>>
>> ^AI 1
>> 1
>> 2
>> ^AI 2
>> serial 2 added this
>> ^AE 2
>> 3
>> ^AI 3
>> branch line 1
>> branch line 2
>> ...
>> branch line 10000
>> ^AE 3
>> 4
>> ^AD 2
>> 5
>> ^AE 2
>> ^AE 1
>>
>> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
>> print anything inserted by either of those, delete anything deleted
>> by those, skip anything inserted by anything not in the set, skip any
>> deletes by anything not in the set.
>>
>> The delta table looks like this, notice I've added an author column:
>>
>> rev     me      parent  include exclude author
>> 1.1.1.1 3       1                       lm
>> 1.2     2       1                       lm
>> 1.1     1       0                       lm
>>
>> If you followed all that, you can see how SCCS can merge by reference.
>> Lets say Clem decides to merge my branch onto the trunk. The delta table
>> will get a new entry:
>>
>> rev     me      parent  include exclude author
>> 1.3     4       2       3               clem
>> 1.1.1.1 3       1                       lm
>> 1.2     2       1                       lm
>> 1.1     1       0                       lm
>>
>> The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3
>> way
>> merge, it will find the lines "3" and "5" as anchor points in both
>> versions,
>> so it is a simple insert with no new data added to the weave.
>>
>> Here's some magic that *everyone* else gets wrong when they pass by value:
>> In a system that passes by value (copies) the data, the merge done by clem
>> would have an annotated listing like so:
>>
>> lm      1
>> lm      2
>> lm      3
>> clem    branch line 1
>> clem    branch line 2
>> clem    ...
>> clem    branch line 10000
>> lm      4
>> lm      5
>>
>> Since it copied the data, it looks like Clem wrote it but he didn't, he
>> just automerged it.  In SCCS/BitKeeper it would look like:
>>
>> lm      1
>> lm      2
>> lm      3
>> lm      branch line 1
>> lm      branch line 2
>> lm      ...
>> lm      branch line 10000
>> lm      4
>> lm      5
>>
>> which is correct, all of those lines were authored by one person.  The
>> only
>> time the merger should show up as an author is if there was a conflict,
>> however the merger resolved that conflict is new work and should be
>> authored by the merger.
>>
>> What BitKeeper did, that was a profound step forward, was make the idea
>> of a repository a formal thing and introduced the concept of changesets
>> that keeps track of all this stuff at the repository level.  So it does
>> all this stuff at the file level but you don't have to do that low level
>> work.  You could think of SCCS as assembly and BitKeeper as more like C,
>> it upleveled things to the point that humans can manage the repository
>> easily.
>>
>> Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
>> questions?  It should be obvious that I *love* SCCS, it's a dramatically
>> better file format than a patch based one, you can get *any* version of
>> the file in constant time, authorship can be preserved across versions,
>> it's pretty brilliant and I consider myself blessed to be posting this
>> in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
>> to the RCS guy, who got a PhD for RCS (give me a break) and did the
>> world a huge disservice by bad mouthing SCCS so he could promote RCS.
>>
>> --lm
>>
>
>
> --
> *My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*
>


-- 
*My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*

[-- Attachment #2: Type: text/html, Size: 12979 bytes --]

[TUHS] Re: SCCS roach motel

[Larry McVoy]

Yes, but note that when you see "^AcK" for varying values of "K", those
are BitKeeper extensions to the file format.  We figured out that SCCS
treats "^Ac " the same whether there is a space or a character there.
It was a way for us to add stuff to the file format where it would look
weird when you printed the comments, but SCCS could, at least, read the
BK file format.

All of src-notes/<thing> was true at some point but has almost certainly
rotted.  We wrote those as a way to get developers started in the right
direction.  The source code is the real reference.

On Fri, Dec 13, 2024 at 11:39:03AM -0700, Marc Rochkind wrote:
> Larry, I found this:
> 
> https://www.bitkeeper.org/src-notes/SCCSWEAVE.html
> 
> A good reference?
> 
> Marc
> 
> On Fri, Dec 13, 2024 at 11:32???AM Marc Rochkind <mrochkind@gmail.com> wrote:
> 
> > Larry, thanks for this. I had read some things you've written about the
> > weave before, but not with this level of detail. Sounds weird, but I didn't
> > really appreciate the implications of the weave even though I'm the guy who
> > thought it up. I did understand the importance of not copying data if you
> > can reference it, which is a principle of database design (normal forms,
> > etc).
> >
> > In my paper, I can add a little more about the weave and its advantages.
> > Aside from this TUHS post, is there something I can put in the References
> > that people can find?
> >
> > Question: Is this right, that TeamWare was literally layered on top of
> > AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or,
> > was it more complicated than that?
> >
> > Was your implementation of SCCS ever released by itself?
> >
> > Marc
> >
> > On Fri, Dec 13, 2024 at 11:06???AM Larry McVoy <lm@mcvoy.com> wrote:
> >
> >> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
> >> > IEEE Transactions on Software Engineering has asked me to write a
> >> > retrospective on the influence of SCCS over the last 50 years, as my
> >> SCCS
> >> > paper was published in 1975. They consider it one of the most
> >> influential
> >> > papers from TSE's first decade.
> >> >
> >> > There's a funny quote from Ken Thompson that circulates from
> >> time-to-time:
> >> >
> >> > "SCCS, the source motel! Programs check in and never check out!"
> >> >
> >> > But nobody seems to know what it means exactly. As part of my research,
> >> I
> >> > asked Ken what the quote meant, sunce I wanted to include it. He
> >> explained
> >> > that it refers to SCCS storing binary data in its repository file,
> >> > preventing UNIX text tools from operating on the file.
> >> >
> >> > Of course, this is only one of SCCS's many weaknesses. If you have
> >> anything
> >> > funny about any of the others, post it here. I already have all the
> >> boring
> >> > usual stuff (e.g., long-term locks, file-oriented, no merging).
> >>
> >> Warning, I know more about SCCS than the average person, I've
> >> reimplemented it from scratch and then built BitKeeper on top of an
> >> extended SCCS file format.  So lots of info coming.  Rick Smith and
> >> Wayne Scott know as much as I do, Rick knows more, he joined me and
> >> promptly started fixing my SCCS implementation.  So far as I know,
> >> there is not a more knowledgable person that Rick when it comes to
> >> weave file formats.
> >>
> >> SCCS's strength is the weave format.  It's largely not understood, even
> >> by other people working in source management.  Here's the benefit of
> >> that weave (if people use it, which, other than BitKeeper, they don't,
> >> looking at you, Clearcase, you had a weave and didn't use it): SCCS can
> >> pass merge data by reference, everyone else copies the data.
> >>
> >> SCCS is a set based system.   Each node has a revision number, like 1.5,
> >> but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
> >> like 1.5.1.1, it is _impossible_ to build the history graph from the
> >> revisions, you can in simple graphs but as soon as you branch from a
> >> branch, all bets are off.
> >>
> >> The graph is built from what BitKeeper called serial numbers.  Each node
> >> in the graph has at least 2 serials, one that names that node in the
> >> graph, and one that is the parent.
> >>
> >> So if I have a file with 5 revisions in straight line history, the
> >> internal stuff will look something like
> >>
> >> rev     me      parent
> >> 1.5     5       4
> >> 1.4     4       3
> >> 1.3     3       2
> >> 1.2     2       1
> >> 1.1     1       0
> >>
> >> So what's the set?  Pretty simple for straight line history, you walk
> >> the history from the rev that you want, adding the "me" serial and
> >> going to the parent, repeat until the parent is 0.
> >>
> >> Suppose you branch from rev 1.3.
> >>
> >> rev     me      parent
> >> 1.3.1.1 6       3
> >> 1.5     5       4
> >> 1.4     4       3
> >> 1.3     3       2
> >> ...
> >>
> >> See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
> >> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
> >> and 4.  If you understand that, you are starting to understand the set
> >> and how it is constructed.
> >>
> >> Did you know you can have an argument in the revision history without
> >> adding anything to the data part?  SCCS has the ability to include
> >> and/or exclude serials as part of a delta.  Lets say Marc looked at
> >> my 1.5 and thought it was garbage.  He can exclude it from the
> >> set like so:
> >>
> >> rev     me      parent  include exclude
> >> 1.6     7       5       0       5
> >> 1.3.1.1 6       3
> >> 1.5     5       4
> >> 1.4     4       3
> >> 1.3     3       2
> >> ...
> >>
> >> That doesn't change the data part of the file AT ALL, it's just saying
> >> Marc doesn't want anyone to see the 1.5 changes.
> >>
> >> To understand that, you need to know how SCCS checks out a file.  And
> >> you need to know how the data is stored.  Which is in a weave.  RCS,
> >> and pretty much everything that followed it, doesn't use a weave at
> >> all.  RCS stores the most recent version of the file as a complete
> >> copy of the checked out file.  Then each delta working backwards up
> >> the trunk is a patch, what diff produces.
> >>
> >> Think about what that means for working on a branch.  You have to start
> >> with the most recent version of the file, apply backward patches to go
> >> to earlier versions all the way back to the branch point, then apply
> >> forward patches to work your way to tip of the branch.  Ask Dave Miller
> >> how pleasant it is to work on gcc on a branch.  It's crazy slow and
> >> painful.
> >>
> >> So how does SCCS do it?  Lets say the first version of a file is
> >>
> >> 1
> >> 2
> >> 3
> >> 4
> >> 5
> >>
> >> The data portion of the history file will look like:
> >>
> >> ^AI 1
> >> 1
> >> 2
> >> 3
> >> 4
> >> 5
> >> ^AE 1
> >>
> >> SCCS used ^A at the beginning of a line to mean "this is metadata for
> >> SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
> >> with a ^AE which means end.  The number after is the serial.  So that
> >> weave says "If serial 1 is in your set, everything after ^AI 1 is part
> >> of that set until you hit the matching ^AE 1.
> >>
> >> Lets say the 2nd version is
> >>
> >> 1
> >> 2
> >> serial 2 added this
> >> 3
> >> 4
> >>
> >> Notice that serial 2 deleted what was line 5.
> >>
> >> ^AI 1
> >> 1
> >> 2
> >> ^AI 2
> >> serial 2 added this
> >> ^AE 2
> >> 3
> >> 4
> >> ^AD 2
> >> 5
> >> ^AE 2
> >> ^AE 1
> >>
> >> So now we can start to see how you walk the weave.  If I'm trying to
> >> check out 1.1 aka serial 1, I build a set that has only '1' in the set.
> >> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
> >> which means print each data line.  I hit ^AI 2, that's not in my set,
> >> so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
> >> I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
> >> 2 is NOT in my set, so I stay in print mode.  Etc.
> >>
> >> Let's make a branch, 1.1.1.1, with lots of data.
> >>
> >> 1
> >> 2
> >> 3
> >> branch line 1
> >> branch line 2
> >> ...
> >> branch line 10000
> >> 4
> >> 5
> >>
> >> ^AI 1
> >> 1
> >> 2
> >> ^AI 2
> >> serial 2 added this
> >> ^AE 2
> >> 3
> >> ^AI 3
> >> branch line 1
> >> branch line 2
> >> ...
> >> branch line 10000
> >> ^AE 3
> >> 4
> >> ^AD 2
> >> 5
> >> ^AE 2
> >> ^AE 1
> >>
> >> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
> >> print anything inserted by either of those, delete anything deleted
> >> by those, skip anything inserted by anything not in the set, skip any
> >> deletes by anything not in the set.
> >>
> >> The delta table looks like this, notice I've added an author column:
> >>
> >> rev     me      parent  include exclude author
> >> 1.1.1.1 3       1                       lm
> >> 1.2     2       1                       lm
> >> 1.1     1       0                       lm
> >>
> >> If you followed all that, you can see how SCCS can merge by reference.
> >> Lets say Clem decides to merge my branch onto the trunk. The delta table
> >> will get a new entry:
> >>
> >> rev     me      parent  include exclude author
> >> 1.3     4       2       3               clem
> >> 1.1.1.1 3       1                       lm
> >> 1.2     2       1                       lm
> >> 1.1     1       0                       lm
> >>
> >> The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3
> >> way
> >> merge, it will find the lines "3" and "5" as anchor points in both
> >> versions,
> >> so it is a simple insert with no new data added to the weave.
> >>
> >> Here's some magic that *everyone* else gets wrong when they pass by value:
> >> In a system that passes by value (copies) the data, the merge done by clem
> >> would have an annotated listing like so:
> >>
> >> lm      1
> >> lm      2
> >> lm      3
> >> clem    branch line 1
> >> clem    branch line 2
> >> clem    ...
> >> clem    branch line 10000
> >> lm      4
> >> lm      5
> >>
> >> Since it copied the data, it looks like Clem wrote it but he didn't, he
> >> just automerged it.  In SCCS/BitKeeper it would look like:
> >>
> >> lm      1
> >> lm      2
> >> lm      3
> >> lm      branch line 1
> >> lm      branch line 2
> >> lm      ...
> >> lm      branch line 10000
> >> lm      4
> >> lm      5
> >>
> >> which is correct, all of those lines were authored by one person.  The
> >> only
> >> time the merger should show up as an author is if there was a conflict,
> >> however the merger resolved that conflict is new work and should be
> >> authored by the merger.
> >>
> >> What BitKeeper did, that was a profound step forward, was make the idea
> >> of a repository a formal thing and introduced the concept of changesets
> >> that keeps track of all this stuff at the repository level.  So it does
> >> all this stuff at the file level but you don't have to do that low level
> >> work.  You could think of SCCS as assembly and BitKeeper as more like C,
> >> it upleveled things to the point that humans can manage the repository
> >> easily.
> >>
> >> Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
> >> questions?  It should be obvious that I *love* SCCS, it's a dramatically
> >> better file format than a patch based one, you can get *any* version of
> >> the file in constant time, authorship can be preserved across versions,
> >> it's pretty brilliant and I consider myself blessed to be posting this
> >> in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
> >> to the RCS guy, who got a PhD for RCS (give me a break) and did the
> >> world a huge disservice by bad mouthing SCCS so he could promote RCS.
> >>
> >> --lm
> >>
> >
> >
> > --
> > *My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*
> >
> 
> 
> -- 
> *My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote:
> Larry, thanks for this. I had read some things you've written about the
> weave before, but not with this level of detail. Sounds weird, but I didn't
> really appreciate the implications of the weave even though I'm the guy who
> thought it up. I did understand the importance of not copying data if you
> can reference it, which is a principle of database design (normal forms,
> etc).
> 
> In my paper, I can add a little more about the weave and its advantages.
> Aside from this TUHS post, is there something I can put in the References
> that people can find?
> 
> Question: Is this right, that TeamWare was literally layered on top of AT&T
> SCCS, but BitKeeper was layered on your implementation of SCCS? Or, was it
> more complicated than that?

Yeah, I wrote TeamWare as well though the source management group took all
my perl4 and rewrote it in C++ (and regretted it).  The only C program,
smoosh.c, which zippered together 2 SCCS files that had at least 1.1 in
common, they checked in without preserving my authorship.  Yes, douche
move, but they hated me because I was a kernel engineer, doing well at
that, and I wrote NSElite in my spare time.  NSElite was what they took
and turned into TeamWare.  The sad thing, for them, is that if they had
let me keep going I would have evolved NSElite into BitKeeper, I was
already thinking of changesets back them.  They weren't the brightest
people, they could copy what I did and that was about it.

> Was your implementation of SCCS ever released by itself?

See if the wayback machine has BitSCCS somewhere.  It was pretty early,
before Rick showed up to fix my screwups.  He did point out that my
weave implementation was the only one written such that I could have
N serial sets in my hand, and do one pass through the weave and get 
N different checked out files.  I don't think we ever used that but 
if we did it would be in smerge.c.

> Marc
> 
> On Fri, Dec 13, 2024 at 11:06???AM Larry McVoy <lm@mcvoy.com> wrote:
> 
> > On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
> > > IEEE Transactions on Software Engineering has asked me to write a
> > > retrospective on the influence of SCCS over the last 50 years, as my SCCS
> > > paper was published in 1975. They consider it one of the most influential
> > > papers from TSE's first decade.
> > >
> > > There's a funny quote from Ken Thompson that circulates from
> > time-to-time:
> > >
> > > "SCCS, the source motel! Programs check in and never check out!"
> > >
> > > But nobody seems to know what it means exactly. As part of my research, I
> > > asked Ken what the quote meant, sunce I wanted to include it. He
> > explained
> > > that it refers to SCCS storing binary data in its repository file,
> > > preventing UNIX text tools from operating on the file.
> > >
> > > Of course, this is only one of SCCS's many weaknesses. If you have
> > anything
> > > funny about any of the others, post it here. I already have all the
> > boring
> > > usual stuff (e.g., long-term locks, file-oriented, no merging).
> >
> > Warning, I know more about SCCS than the average person, I've
> > reimplemented it from scratch and then built BitKeeper on top of an
> > extended SCCS file format.  So lots of info coming.  Rick Smith and
> > Wayne Scott know as much as I do, Rick knows more, he joined me and
> > promptly started fixing my SCCS implementation.  So far as I know,
> > there is not a more knowledgable person that Rick when it comes to
> > weave file formats.
> >
> > SCCS's strength is the weave format.  It's largely not understood, even
> > by other people working in source management.  Here's the benefit of
> > that weave (if people use it, which, other than BitKeeper, they don't,
> > looking at you, Clearcase, you had a weave and didn't use it): SCCS can
> > pass merge data by reference, everyone else copies the data.
> >
> > SCCS is a set based system.   Each node has a revision number, like 1.5,
> > but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
> > like 1.5.1.1, it is _impossible_ to build the history graph from the
> > revisions, you can in simple graphs but as soon as you branch from a
> > branch, all bets are off.
> >
> > The graph is built from what BitKeeper called serial numbers.  Each node
> > in the graph has at least 2 serials, one that names that node in the
> > graph, and one that is the parent.
> >
> > So if I have a file with 5 revisions in straight line history, the
> > internal stuff will look something like
> >
> > rev     me      parent
> > 1.5     5       4
> > 1.4     4       3
> > 1.3     3       2
> > 1.2     2       1
> > 1.1     1       0
> >
> > So what's the set?  Pretty simple for straight line history, you walk
> > the history from the rev that you want, adding the "me" serial and
> > going to the parent, repeat until the parent is 0.
> >
> > Suppose you branch from rev 1.3.
> >
> > rev     me      parent
> > 1.3.1.1 6       3
> > 1.5     5       4
> > 1.4     4       3
> > 1.3     3       2
> > ...
> >
> > See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
> > for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
> > and 4.  If you understand that, you are starting to understand the set
> > and how it is constructed.
> >
> > Did you know you can have an argument in the revision history without
> > adding anything to the data part?  SCCS has the ability to include
> > and/or exclude serials as part of a delta.  Lets say Marc looked at
> > my 1.5 and thought it was garbage.  He can exclude it from the
> > set like so:
> >
> > rev     me      parent  include exclude
> > 1.6     7       5       0       5
> > 1.3.1.1 6       3
> > 1.5     5       4
> > 1.4     4       3
> > 1.3     3       2
> > ...
> >
> > That doesn't change the data part of the file AT ALL, it's just saying
> > Marc doesn't want anyone to see the 1.5 changes.
> >
> > To understand that, you need to know how SCCS checks out a file.  And
> > you need to know how the data is stored.  Which is in a weave.  RCS,
> > and pretty much everything that followed it, doesn't use a weave at
> > all.  RCS stores the most recent version of the file as a complete
> > copy of the checked out file.  Then each delta working backwards up
> > the trunk is a patch, what diff produces.
> >
> > Think about what that means for working on a branch.  You have to start
> > with the most recent version of the file, apply backward patches to go
> > to earlier versions all the way back to the branch point, then apply
> > forward patches to work your way to tip of the branch.  Ask Dave Miller
> > how pleasant it is to work on gcc on a branch.  It's crazy slow and
> > painful.
> >
> > So how does SCCS do it?  Lets say the first version of a file is
> >
> > 1
> > 2
> > 3
> > 4
> > 5
> >
> > The data portion of the history file will look like:
> >
> > ^AI 1
> > 1
> > 2
> > 3
> > 4
> > 5
> > ^AE 1
> >
> > SCCS used ^A at the beginning of a line to mean "this is metadata for
> > SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
> > with a ^AE which means end.  The number after is the serial.  So that
> > weave says "If serial 1 is in your set, everything after ^AI 1 is part
> > of that set until you hit the matching ^AE 1.
> >
> > Lets say the 2nd version is
> >
> > 1
> > 2
> > serial 2 added this
> > 3
> > 4
> >
> > Notice that serial 2 deleted what was line 5.
> >
> > ^AI 1
> > 1
> > 2
> > ^AI 2
> > serial 2 added this
> > ^AE 2
> > 3
> > 4
> > ^AD 2
> > 5
> > ^AE 2
> > ^AE 1
> >
> > So now we can start to see how you walk the weave.  If I'm trying to
> > check out 1.1 aka serial 1, I build a set that has only '1' in the set.
> > I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
> > which means print each data line.  I hit ^AI 2, that's not in my set,
> > so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
> > I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
> > 2 is NOT in my set, so I stay in print mode.  Etc.
> >
> > Let's make a branch, 1.1.1.1, with lots of data.
> >
> > 1
> > 2
> > 3
> > branch line 1
> > branch line 2
> > ...
> > branch line 10000
> > 4
> > 5
> >
> > ^AI 1
> > 1
> > 2
> > ^AI 2
> > serial 2 added this
> > ^AE 2
> > 3
> > ^AI 3
> > branch line 1
> > branch line 2
> > ...
> > branch line 10000
> > ^AE 3
> > 4
> > ^AD 2
> > 5
> > ^AE 2
> > ^AE 1
> >
> > So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
> > print anything inserted by either of those, delete anything deleted
> > by those, skip anything inserted by anything not in the set, skip any
> > deletes by anything not in the set.
> >
> > The delta table looks like this, notice I've added an author column:
> >
> > rev     me      parent  include exclude author
> > 1.1.1.1 3       1                       lm
> > 1.2     2       1                       lm
> > 1.1     1       0                       lm
> >
> > If you followed all that, you can see how SCCS can merge by reference.
> > Lets say Clem decides to merge my branch onto the trunk. The delta table
> > will get a new entry:
> >
> > rev     me      parent  include exclude author
> > 1.3     4       2       3               clem
> > 1.1.1.1 3       1                       lm
> > 1.2     2       1                       lm
> > 1.1     1       0                       lm
> >
> > The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3 way
> > merge, it will find the lines "3" and "5" as anchor points in both
> > versions,
> > so it is a simple insert with no new data added to the weave.
> >
> > Here's some magic that *everyone* else gets wrong when they pass by value:
> > In a system that passes by value (copies) the data, the merge done by clem
> > would have an annotated listing like so:
> >
> > lm      1
> > lm      2
> > lm      3
> > clem    branch line 1
> > clem    branch line 2
> > clem    ...
> > clem    branch line 10000
> > lm      4
> > lm      5
> >
> > Since it copied the data, it looks like Clem wrote it but he didn't, he
> > just automerged it.  In SCCS/BitKeeper it would look like:
> >
> > lm      1
> > lm      2
> > lm      3
> > lm      branch line 1
> > lm      branch line 2
> > lm      ...
> > lm      branch line 10000
> > lm      4
> > lm      5
> >
> > which is correct, all of those lines were authored by one person.  The only
> > time the merger should show up as an author is if there was a conflict,
> > however the merger resolved that conflict is new work and should be
> > authored by the merger.
> >
> > What BitKeeper did, that was a profound step forward, was make the idea
> > of a repository a formal thing and introduced the concept of changesets
> > that keeps track of all this stuff at the repository level.  So it does
> > all this stuff at the file level but you don't have to do that low level
> > work.  You could think of SCCS as assembly and BitKeeper as more like C,
> > it upleveled things to the point that humans can manage the repository
> > easily.
> >
> > Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
> > questions?  It should be obvious that I *love* SCCS, it's a dramatically
> > better file format than a patch based one, you can get *any* version of
> > the file in constant time, authorship can be preserved across versions,
> > it's pretty brilliant and I consider myself blessed to be posting this
> > in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
> > to the RCS guy, who got a PhD for RCS (give me a break) and did the
> > world a huge disservice by bad mouthing SCCS so he could promote RCS.
> >
> > --lm
> >
> 
> 
> -- 
> *My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*

-- 
---
Larry McVoy           Retired to fishing          http://www.mcvoy.com/lm/boat

[TUHS] Re: SCCS roach motel

[Henry Bent]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

On Fri, 13 Dec 2024 at 14:03, Larry McVoy <lm@mcvoy.com> wrote:

> On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote:
>
> > Was your implementation of SCCS ever released by itself?
>
> See if the wayback machine has BitSCCS somewhere.  It was pretty early,
> before Rick showed up to fix my screwups.  He did point out that my
> weave implementation was the only one written such that I could have
> N serial sets in my hand, and do one pass through the weave and get
> N different checked out files.  I don't think we ever used that but
> if we did it would be in smerge.c.
>

There are many preservations of http://www.bitmover.com/bitsccs/, but since
the BitSCCS sources were distributed via FTP the wayback machine doesn't
have the actual sources.

I did a bit of searching and found
ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz
; I don't know where that falls in the product's lifetime.

-Henry

[-- Attachment #2: Type: text/html, Size: 1556 bytes --]

[TUHS] Re: SCCS roach motel

[arnold]

Henry Bent <henry.r.bent@gmail.com> wrote:

> On Fri, 13 Dec 2024 at 14:03, Larry McVoy <lm@mcvoy.com> wrote:
>
> > On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote:
> >
> > > Was your implementation of SCCS ever released by itself?
> >
> > See if the wayback machine has BitSCCS somewhere.  It was pretty early,
> > before Rick showed up to fix my screwups.  He did point out that my
> > weave implementation was the only one written such that I could have
> > N serial sets in my hand, and do one pass through the weave and get
> > N different checked out files.  I don't think we ever used that but
> > if we did it would be in smerge.c.
> >
>
> There are many preservations of http://www.bitmover.com/bitsccs/, but since
> the BitSCCS sources were distributed via FTP the wayback machine doesn't
> have the actual sources.
>
> I did a bit of searching and found
> ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz
> ; I don't know where that falls in the product's lifetime.
>
> -Henry

Larry,

What about GNU CSSC? (https://ftp.gnu.org/gnu/cssc/)? Isn't that a
reimplementation of SCCS?

Hmm, the README and THANKS file in it provide some info. It's not
a full clone, there are some missing features.

The last release is from 2019.

Arnold

[TUHS] Re: SCCS roach motel

[Larry McVoy]

On Sat, Dec 14, 2024 at 11:29:45AM -0700, arnold@skeeve.com wrote:
> Henry Bent <henry.r.bent@gmail.com> wrote:
> 
> > On Fri, 13 Dec 2024 at 14:03, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > > On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote:
> > >
> > > > Was your implementation of SCCS ever released by itself?
> > >
> > > See if the wayback machine has BitSCCS somewhere.  It was pretty early,
> > > before Rick showed up to fix my screwups.  He did point out that my
> > > weave implementation was the only one written such that I could have
> > > N serial sets in my hand, and do one pass through the weave and get
> > > N different checked out files.  I don't think we ever used that but
> > > if we did it would be in smerge.c.
> > >
> >
> > There are many preservations of http://www.bitmover.com/bitsccs/, but since
> > the BitSCCS sources were distributed via FTP the wayback machine doesn't
> > have the actual sources.
> >
> > I did a bit of searching and found
> > ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz
> > ; I don't know where that falls in the product's lifetime.
> >
> > -Henry
> 
> Larry,
> 
> What about GNU CSSC? (https://ftp.gnu.org/gnu/cssc/)? Isn't that a
> reimplementation of SCCS?

It is, it's C++ and really slow.  I looked at it to see if I could skip
reimplementing SCCS and decided I couldn't.

The problem with SCM in general, is it isn't sexy so it doesn't attract
the best people.  I was a kernel engineer who got distracted by SCM.
I worked on performance, the Sun SCM group had tried to clone clearclase,
it was so slow that my friends were leaving Sun.  I looked at what it
was doing and decided it was unfixable.  I realized it was using SCCS
under the covers and I wrote NSElite (the slow thing was "NSE") and
did the magic such that I could clone/pull/push from my stuff to NSE.
The kernel group promptly dumped NSE and moved to NSElite.

Sun wanted me to go to the SCM group so that everything was politically
correct, I looked at the caliber of people there and declined, it was
a definite step down from working on the kernel.

BK is what you get when you let a kernel engineer do SCM.  

[TUHS] Re: SCCS roach motel

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 11599 bytes --]

@Marc and Larry

As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred
choice. To this day, I have looked up directions to do simple things in Git
that were so natural for me in SCCS. I don't think it's the old dog
syndrome, either. SCCS was hardly perfect, but it solved a problem very
well.   Eric's sccs(1) front end for it from UCB cleaned up a few of the
rough edges and experience taught us a little about care and feeding.
Truth is I still use for small projects.  It's easier to set up and it just
protected me against myself.

As a side note, it also exposed/demonstrated my real dislike for NFS early
on when we started to see ZERO filled blocks in SCCS files (stateless just
sucks).

So thank you both. I have no idea how many times you saved my team and me
time and bailed us out.
ᐧ

On Fri, Dec 13, 2024 at 1:33 PM Marc Rochkind <mrochkind@gmail.com> wrote:

> Larry, thanks for this. I had read some things you've written about the
> weave before, but not with this level of detail. Sounds weird, but I didn't
> really appreciate the implications of the weave even though I'm the guy who
> thought it up. I did understand the importance of not copying data if you
> can reference it, which is a principle of database design (normal forms,
> etc).
>
> In my paper, I can add a little more about the weave and its advantages.
> Aside from this TUHS post, is there something I can put in the References
> that people can find?
>
> Question: Is this right, that TeamWare was literally layered on top of
> AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or,
> was it more complicated than that?
>
> Was your implementation of SCCS ever released by itself?
>
> Marc
>
> On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy <lm@mcvoy.com> wrote:
>
>> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote:
>> > IEEE Transactions on Software Engineering has asked me to write a
>> > retrospective on the influence of SCCS over the last 50 years, as my
>> SCCS
>> > paper was published in 1975. They consider it one of the most
>> influential
>> > papers from TSE's first decade.
>> >
>> > There's a funny quote from Ken Thompson that circulates from
>> time-to-time:
>> >
>> > "SCCS, the source motel! Programs check in and never check out!"
>> >
>> > But nobody seems to know what it means exactly. As part of my research,
>> I
>> > asked Ken what the quote meant, sunce I wanted to include it. He
>> explained
>> > that it refers to SCCS storing binary data in its repository file,
>> > preventing UNIX text tools from operating on the file.
>> >
>> > Of course, this is only one of SCCS's many weaknesses. If you have
>> anything
>> > funny about any of the others, post it here. I already have all the
>> boring
>> > usual stuff (e.g., long-term locks, file-oriented, no merging).
>>
>> Warning, I know more about SCCS than the average person, I've
>> reimplemented it from scratch and then built BitKeeper on top of an
>> extended SCCS file format.  So lots of info coming.  Rick Smith and
>> Wayne Scott know as much as I do, Rick knows more, he joined me and
>> promptly started fixing my SCCS implementation.  So far as I know,
>> there is not a more knowledgable person that Rick when it comes to
>> weave file formats.
>>
>> SCCS's strength is the weave format.  It's largely not understood, even
>> by other people working in source management.  Here's the benefit of
>> that weave (if people use it, which, other than BitKeeper, they don't,
>> looking at you, Clearcase, you had a weave and didn't use it): SCCS can
>> pass merge data by reference, everyone else copies the data.
>>
>> SCCS is a set based system.   Each node has a revision number, like 1.5,
>> but because SCCS, unlike RCS, limited the revisions to at most 4 fields,
>> like 1.5.1.1, it is _impossible_ to build the history graph from the
>> revisions, you can in simple graphs but as soon as you branch from a
>> branch, all bets are off.
>>
>> The graph is built from what BitKeeper called serial numbers.  Each node
>> in the graph has at least 2 serials, one that names that node in the
>> graph, and one that is the parent.
>>
>> So if I have a file with 5 revisions in straight line history, the
>> internal stuff will look something like
>>
>> rev     me      parent
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> 1.2     2       1
>> 1.1     1       0
>>
>> So what's the set?  Pretty simple for straight line history, you walk
>> the history from the rev that you want, adding the "me" serial and
>> going to the parent, repeat until the parent is 0.
>>
>> Suppose you branch from rev 1.3.
>>
>> rev     me      parent
>> 1.3.1.1 6       3
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> ...
>>
>> See that 1.3.1.1 is me: 6 and parent: 3.  So if I were building the set
>> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5
>> and 4.  If you understand that, you are starting to understand the set
>> and how it is constructed.
>>
>> Did you know you can have an argument in the revision history without
>> adding anything to the data part?  SCCS has the ability to include
>> and/or exclude serials as part of a delta.  Lets say Marc looked at
>> my 1.5 and thought it was garbage.  He can exclude it from the
>> set like so:
>>
>> rev     me      parent  include exclude
>> 1.6     7       5       0       5
>> 1.3.1.1 6       3
>> 1.5     5       4
>> 1.4     4       3
>> 1.3     3       2
>> ...
>>
>> That doesn't change the data part of the file AT ALL, it's just saying
>> Marc doesn't want anyone to see the 1.5 changes.
>>
>> To understand that, you need to know how SCCS checks out a file.  And
>> you need to know how the data is stored.  Which is in a weave.  RCS,
>> and pretty much everything that followed it, doesn't use a weave at
>> all.  RCS stores the most recent version of the file as a complete
>> copy of the checked out file.  Then each delta working backwards up
>> the trunk is a patch, what diff produces.
>>
>> Think about what that means for working on a branch.  You have to start
>> with the most recent version of the file, apply backward patches to go
>> to earlier versions all the way back to the branch point, then apply
>> forward patches to work your way to tip of the branch.  Ask Dave Miller
>> how pleasant it is to work on gcc on a branch.  It's crazy slow and
>> painful.
>>
>> So how does SCCS do it?  Lets say the first version of a file is
>>
>> 1
>> 2
>> 3
>> 4
>> 5
>>
>> The data portion of the history file will look like:
>>
>> ^AI 1
>> 1
>> 2
>> 3
>> 4
>> 5
>> ^AE 1
>>
>> SCCS used ^A at the beginning of a line to mean "this is metadata for
>> SCCS".  ^AI is an insert, ^AD is a delete, and insert/delete are paired
>> with a ^AE which means end.  The number after is the serial.  So that
>> weave says "If serial 1 is in your set, everything after ^AI 1 is part
>> of that set until you hit the matching ^AE 1.
>>
>> Lets say the 2nd version is
>>
>> 1
>> 2
>> serial 2 added this
>> 3
>> 4
>>
>> Notice that serial 2 deleted what was line 5.
>>
>> ^AI 1
>> 1
>> 2
>> ^AI 2
>> serial 2 added this
>> ^AE 2
>> 3
>> 4
>> ^AD 2
>> 5
>> ^AE 2
>> ^AE 1
>>
>> So now we can start to see how you walk the weave.  If I'm trying to
>> check out 1.1 aka serial 1, I build a set that has only '1' in the set.
>> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode,
>> which means print each data line.  I hit ^AI 2, that's not in my set,
>> so I'm now in skip mode.  And I skip the stuff inserted by serial 2.
>> I see the ^AE 2 and I revert back to print mode.  I get to ^AD 2,
>> 2 is NOT in my set, so I stay in print mode.  Etc.
>>
>> Let's make a branch, 1.1.1.1, with lots of data.
>>
>> 1
>> 2
>> 3
>> branch line 1
>> branch line 2
>> ...
>> branch line 10000
>> 4
>> 5
>>
>> ^AI 1
>> 1
>> 2
>> ^AI 2
>> serial 2 added this
>> ^AE 2
>> 3
>> ^AI 3
>> branch line 1
>> branch line 2
>> ...
>> branch line 10000
>> ^AE 3
>> 4
>> ^AD 2
>> 5
>> ^AE 2
>> ^AE 1
>>
>> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll
>> print anything inserted by either of those, delete anything deleted
>> by those, skip anything inserted by anything not in the set, skip any
>> deletes by anything not in the set.
>>
>> The delta table looks like this, notice I've added an author column:
>>
>> rev     me      parent  include exclude author
>> 1.1.1.1 3       1                       lm
>> 1.2     2       1                       lm
>> 1.1     1       0                       lm
>>
>> If you followed all that, you can see how SCCS can merge by reference.
>> Lets say Clem decides to merge my branch onto the trunk. The delta table
>> will get a new entry:
>>
>> rev     me      parent  include exclude author
>> 1.3     4       2       3               clem
>> 1.1.1.1 3       1                       lm
>> 1.2     2       1                       lm
>> 1.1     1       0                       lm
>>
>> The weave DOES NOT CHANGE.  That's the pass by reference.  You do the 3
>> way
>> merge, it will find the lines "3" and "5" as anchor points in both
>> versions,
>> so it is a simple insert with no new data added to the weave.
>>
>> Here's some magic that *everyone* else gets wrong when they pass by value:
>> In a system that passes by value (copies) the data, the merge done by clem
>> would have an annotated listing like so:
>>
>> lm      1
>> lm      2
>> lm      3
>> clem    branch line 1
>> clem    branch line 2
>> clem    ...
>> clem    branch line 10000
>> lm      4
>> lm      5
>>
>> Since it copied the data, it looks like Clem wrote it but he didn't, he
>> just automerged it.  In SCCS/BitKeeper it would look like:
>>
>> lm      1
>> lm      2
>> lm      3
>> lm      branch line 1
>> lm      branch line 2
>> lm      ...
>> lm      branch line 10000
>> lm      4
>> lm      5
>>
>> which is correct, all of those lines were authored by one person.  The
>> only
>> time the merger should show up as an author is if there was a conflict,
>> however the merger resolved that conflict is new work and should be
>> authored by the merger.
>>
>> What BitKeeper did, that was a profound step forward, was make the idea
>> of a repository a formal thing and introduced the concept of changesets
>> that keeps track of all this stuff at the repository level.  So it does
>> all this stuff at the file level but you don't have to do that low level
>> work.  You could think of SCCS as assembly and BitKeeper as more like C,
>> it upleveled things to the point that humans can manage the repository
>> easily.
>>
>> Whew.  That's a butt load of info.  Perhaps better for COFF?  Any
>> questions?  It should be obvious that I *love* SCCS, it's a dramatically
>> better file format than a patch based one, you can get *any* version of
>> the file in constant time, authorship can be preserved across versions,
>> it's pretty brilliant and I consider myself blessed to be posting this
>> in response to SCCS's creator.  Hats off to Marc.  And big boo, hiss,
>> to the RCS guy, who got a PhD for RCS (give me a break) and did the
>> world a huge disservice by bad mouthing SCCS so he could promote RCS.
>>
>> --lm
>>
>
>
> --
> *My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*
>

[-- Attachment #2: Type: text/html, Size: 14178 bytes --]

[TUHS] Re: SCCS roach motel

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

According to the Unix room fortunes file, the actual quote is

SCCS: the source-code motel -- your code checks in but it never checks out.
Ken Thompson

On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind <mrochkind@gmail.com> wrote:

> IEEE Transactions on Software Engineering has asked me to write a
> retrospective on the influence of SCCS over the last 50 years, as my SCCS
> paper was published in 1975. They consider it one of the most influential
> papers from TSE's first decade.
>
> There's a funny quote from Ken Thompson that circulates from time-to-time:
>
> "SCCS, the source motel! Programs check in and never check out!"
>
> But nobody seems to know what it means exactly. As part of my research, I
> asked Ken what the quote meant, sunce I wanted to include it. He explained
> that it refers to SCCS storing binary data in its repository file,
> preventing UNIX text tools from operating on the file.
>
> Of course, this is only one of SCCS's many weaknesses. If you have
> anything funny about any of the others, post it here. I already have all
> the boring usual stuff (e.g., long-term locks, file-oriented, no merging).
>
> Marc Rochkind
> mrochkind.com
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1887 bytes --]

[TUHS] Re: SCCS roach motel

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1936 bytes --]

To answer your actual question, it is of course a riff on a TV ad for a
cockroach trap in the 1980s. The sentiment of the quote, as I saw it (it's
possible I was the one who added it to the fortunes file after ken saw the
SCCS burble at the top of some file from USG and laughed), was primarily a
reaction to the taint it added to the previously annotation-free top of the
file. It was also a response to the march of corporate code management
stepping into the research world, or perhaps the hacker world. It's a
philosophical thing, a feeling, not an argument.

It all seems so quaint now.

-rob


On Sat, Dec 14, 2024 at 8:22 AM Rob Pike <robpike@gmail.com> wrote:

> According to the Unix room fortunes file, the actual quote is
>
> SCCS: the source-code motel -- your code checks in but it never checks out.
> Ken Thompson
>
> On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind <mrochkind@gmail.com> wrote:
>
>> IEEE Transactions on Software Engineering has asked me to write a
>> retrospective on the influence of SCCS over the last 50 years, as my SCCS
>> paper was published in 1975. They consider it one of the most influential
>> papers from TSE's first decade.
>>
>> There's a funny quote from Ken Thompson that circulates from time-to-time:
>>
>> "SCCS, the source motel! Programs check in and never check out!"
>>
>> But nobody seems to know what it means exactly. As part of my research, I
>> asked Ken what the quote meant, sunce I wanted to include it. He explained
>> that it refers to SCCS storing binary data in its repository file,
>> preventing UNIX text tools from operating on the file.
>>
>> Of course, this is only one of SCCS's many weaknesses. If you have
>> anything funny about any of the others, post it here. I already have all
>> the boring usual stuff (e.g., long-term locks, file-oriented, no merging).
>>
>> Marc Rochkind
>> mrochkind.com
>>
>>
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 3286 bytes --]

[TUHS] Re: SCCS roach motel

[Aron Insinga]

[-- Attachment #1: Type: text/plain, Size: 2621 bytes --]

The product and its advertising tag line still exist:

https://www.homedepot.com/p/Black-Flag-Roach-Motel-Insect-Glue-Traps-2-Count-HG-11020-1/204237338

https://images.thdstatic.com/productImages/e8dd1acb-7a49-40ce-8ae2-c582c8bda9a2/svn/brown-black-flag-insect-traps-hg-11020-1-64_1000.jpg

Note the red rectangle-with-rounded-corners under the large yellow word 
'Motel' on the front of the box.

- Aron


On 12/13/24 16:27, Rob Pike wrote:
> To answer your actual question, it is of course a riff on a TV ad for 
> a cockroach trap in the 1980s. The sentiment of the quote, as I saw it 
> (it's possible I was the one who added it to the fortunes file after 
> ken saw the SCCS burble at the top of some file from USG and laughed), 
> was primarily a reaction to the taint it added to the previously 
> annotation-free top of the file. It was also a response to the march 
> of corporate code management stepping into the research world, or 
> perhaps the hacker world. It's a philosophical thing, a feeling, not 
> an argument.
>
> It all seems so quaint now.
>
> -rob
>
>
> On Sat, Dec 14, 2024 at 8:22 AM Rob Pike <robpike@gmail.com> wrote:
>
>     According to the Unix room fortunes file, the actual quote is
>
>     SCCS: the source-code motel -- your code checks in but it never
>     checks out.
>     Ken Thompson
>
>     On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind
>     <mrochkind@gmail.com> wrote:
>
>         IEEE Transactions on Software Engineering has asked me to
>         write a retrospective on the influence of SCCS over the last
>         50 years, as my SCCS paper was published in 1975. They
>         consider it one of the most influential papers from TSE's
>         first decade.
>
>         There's a funny quote from Ken Thompson that circulates from
>         time-to-time:
>
>         "SCCS, the source motel! Programs check in and never check out!"
>
>         But nobody seems to know what it means exactly. As part of my
>         research, I asked Ken what the quote meant, sunce I wanted to
>         include it. He explained that it refers to SCCS storing binary
>         data in its repository file, preventing UNIX text tools from
>         operating on the file.
>
>         Of course, this is only one of SCCS's many weaknesses. If you
>         have anything funny about any of the others, post it here. I
>         already have all the boring usual stuff (e.g., long-term
>         locks, file-oriented, no merging).
>
>         Marc Rochkind
>         mrochkind.com <http://mrochkind.com>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 6076 bytes --]

[TUHS] Re: SCCS roach motel

[Aron Insinga]

[-- Attachment #1: Type: text/plain, Size: 2894 bytes --]

p.s. I should have looked at more hits before sending that.  Here's the 
ad (or one of them):

https://www.youtube.com/watch?v=jKhGHxO-woc

- Aron

On 12/13/24 16:37, Aron Insinga wrote:
> The product and its advertising tag line still exist:
>
> https://www.homedepot.com/p/Black-Flag-Roach-Motel-Insect-Glue-Traps-2-Count-HG-11020-1/204237338
>
> https://images.thdstatic.com/productImages/e8dd1acb-7a49-40ce-8ae2-c582c8bda9a2/svn/brown-black-flag-insect-traps-hg-11020-1-64_1000.jpg
>
> Note the red rectangle-with-rounded-corners under the large yellow 
> word 'Motel' on the front of the box.
>
> - Aron
>
>
> On 12/13/24 16:27, Rob Pike wrote:
>> To answer your actual question, it is of course a riff on a TV ad for 
>> a cockroach trap in the 1980s. The sentiment of the quote, as I saw 
>> it (it's possible I was the one who added it to the fortunes file 
>> after ken saw the SCCS burble at the top of some file from USG and 
>> laughed), was primarily a reaction to the taint it added to the 
>> previously annotation-free top of the file. It was also a response to 
>> the march of corporate code management stepping into the research 
>> world, or perhaps the hacker world. It's a philosophical thing, a 
>> feeling, not an argument.
>>
>> It all seems so quaint now.
>>
>> -rob
>>
>>
>> On Sat, Dec 14, 2024 at 8:22 AM Rob Pike <robpike@gmail.com> wrote:
>>
>>     According to the Unix room fortunes file, the actual quote is
>>
>>     SCCS: the source-code motel -- your code checks in but it never
>>     checks out.
>>     Ken Thompson
>>
>>     On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind
>>     <mrochkind@gmail.com> wrote:
>>
>>         IEEE Transactions on Software Engineering has asked me to
>>         write a retrospective on the influence of SCCS over the last
>>         50 years, as my SCCS paper was published in 1975. They
>>         consider it one of the most influential papers from TSE's
>>         first decade.
>>
>>         There's a funny quote from Ken Thompson that circulates from
>>         time-to-time:
>>
>>         "SCCS, the source motel! Programs check in and never check out!"
>>
>>         But nobody seems to know what it means exactly. As part of my
>>         research, I asked Ken what the quote meant, sunce I wanted to
>>         include it. He explained that it refers to SCCS storing
>>         binary data in its repository file, preventing UNIX text
>>         tools from operating on the file.
>>
>>         Of course, this is only one of SCCS's many weaknesses. If you
>>         have anything funny about any of the others, post it here. I
>>         already have all the boring usual stuff (e.g., long-term
>>         locks, file-oriented, no merging).
>>
>>         Marc Rochkind
>>         mrochkind.com <http://mrochkind.com>
>>
>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 7054 bytes --]

[TUHS] Re: SCCS roach motel

[Luther Johnson]

[-- Attachment #1: Type: text/plain, Size: 1580 bytes --]

When I just read your quote, this joke came to mind:

What's the difference between a public library and a software library ?

At the public library, everyone wants to check things out, but no-one
ever wants to return anything. In a source code library, everyone wants
to put something in the library, but programmers rarely want to take
anything in it out, and use it.

Maybe it doesn't seem as funny as the first time I heard it, but I think
it's still true. Maybe a similar thought is behind the 'source motel' quote.

On 12/13/2024 09:52 AM, Marc Rochkind wrote:
> IEEE Transactions on Software Engineering has asked me to write a
> retrospective on the influence of SCCS over the last 50 years, as my
> SCCS paper was published in 1975. They consider it one of the most
> influential papers from TSE's first decade.
>
> There's a funny quote from Ken Thompson that circulates from time-to-time:
>
> "SCCS, the source motel! Programs check in and never check out!"
>
> But nobody seems to know what it means exactly. As part of my
> research, I asked Ken what the quote meant, sunce I wanted to include
> it. He explained that it refers to SCCS storing binary data in its
> repository file, preventing UNIX text tools from operating on the file.
>
> Of course, this is only one of SCCS's many weaknesses. If you have
> anything funny about any of the others, post it here. I already have
> all the boring usual stuff (e.g., long-term locks, file-oriented, no
> merging).
>
> Marc Rochkind
> mrochkind.com <http://mrochkind.com>
>
>
>
>


[-- Attachment #2: Type: text/html, Size: 2625 bytes --]