The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Favorite unix design principles?
@ 2021-01-25 11:10 Tyler Adams
  2021-01-25 12:32 ` Steve Nickolas
  0 siblings, 1 reply; 70+ messages in thread
From: Tyler Adams @ 2021-01-25 11:10 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

I'm writing about my 5 favorite unix design principles on my blog this
week, and it got me wondering what others' favorite unix design principles
are? For reference, mine are:

- Rule of Separation (from TAOUP <http://catb.org/~esr/writings/taoup/html/>
)
- Let the Machine Do the Dirty Work (from Elements of Programming Style)
- Rule of Silence (from TAOUP <http://catb.org/~esr/writings/taoup/html/>)
- Data Dominates (Rob Pike #5)
- The SPOT (Single Point of Truth) Rule (from TAOUP
<http://catb.org/~esr/writings/taoup/html/>)

 Tyler

[-- Attachment #2: Type: text/html, Size: 815 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-25 11:10 [TUHS] Favorite unix design principles? Tyler Adams
@ 2021-01-25 12:32 ` Steve Nickolas
  2021-01-26  2:06   ` M Douglas McIlroy
  0 siblings, 1 reply; 70+ messages in thread
From: Steve Nickolas @ 2021-01-25 12:32 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Mon, 25 Jan 2021, Tyler Adams wrote:

> I'm writing about my 5 favorite unix design principles on my blog this
> week, and it got me wondering what others' favorite unix design principles
> are? For reference, mine are:
>
> - Rule of Separation (from TAOUP <http://catb.org/~esr/writings/taoup/html/>
> )
> - Let the Machine Do the Dirty Work (from Elements of Programming Style)
> - Rule of Silence (from TAOUP <http://catb.org/~esr/writings/taoup/html/>)
> - Data Dominates (Rob Pike #5)
> - The SPOT (Single Point of Truth) Rule (from TAOUP
> <http://catb.org/~esr/writings/taoup/html/>)
>
> Tyler
>

1. Pipes
2. Text as the preferred format for input and output
3. 'Most everything as a file
4. The idea of simple tools that are optimized for a single task
5. A powerful scripting language built into the system that, combined with 
1-4, makes writing new tools heaps easier.

-uso.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-25 12:32 ` Steve Nickolas
@ 2021-01-26  2:06   ` M Douglas McIlroy
  2021-01-26  2:53     ` Steve Nickolas
  2021-01-26 10:22     ` Tyler Adams
  0 siblings, 2 replies; 70+ messages in thread
From: M Douglas McIlroy @ 2021-01-26  2:06 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

It might be interesting to compare your final list with the two lists in
the 1978 special issue of the BSTJ--one in the Foreword, the other in the
revised version of the Ritchi/Thompson article from the CACM. How have
perceptions or values changed over time?

Doug


On Mon, Jan 25, 2021 at 7:32 AM Steve Nickolas <usotsuki@buric.co> wrote:

> On Mon, 25 Jan 2021, Tyler Adams wrote:
>
> > I'm writing about my 5 favorite unix design principles on my blog this
> > week, and it got me wondering what others' favorite unix design
> principles
> > are? For reference, mine are:
> >
> > - Rule of Separation (from TAOUP <
> http://catb.org/~esr/writings/taoup/html/>
> > )
> > - Let the Machine Do the Dirty Work (from Elements of Programming Style)
> > - Rule of Silence (from TAOUP <http://catb.org/~esr/writings/taoup/html/
> >)
> > - Data Dominates (Rob Pike #5)
> > - The SPOT (Single Point of Truth) Rule (from TAOUP
> > <http://catb.org/~esr/writings/taoup/html/>)
> >
> > Tyler
> >
>
> 1. Pipes
> 2. Text as the preferred format for input and output
> 3. 'Most everything as a file
> 4. The idea of simple tools that are optimized for a single task
> 5. A powerful scripting language built into the system that, combined with
> 1-4, makes writing new tools heaps easier.
>
> -uso.
>

[-- Attachment #2: Type: text/html, Size: 2115 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26  2:06   ` M Douglas McIlroy
@ 2021-01-26  2:53     ` Steve Nickolas
  2021-01-26 10:22     ` Tyler Adams
  1 sibling, 0 replies; 70+ messages in thread
From: Steve Nickolas @ 2021-01-26  2:53 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: The Eunuchs Hysterical Society

On Mon, 25 Jan 2021, M Douglas McIlroy wrote:

> It might be interesting to compare your final list with the two lists in
> the 1978 special issue of the BSTJ--one in the Foreword, the other in the
> revised version of the Ritchi/Thompson article from the CACM. How have
> perceptions or values changed over time?
>
> Doug

Funny thing is I came into *x (Solaris, and then Linux) relatively late - 
ca. 1997, after I had been using MS-DOS and ProDOS for years, which were 
the operating systems I compared it to.

It was because MS-DOS had adopted ideas from Xenix that *x was easier to 
acclimate to than other OSes I'd used.

-uso.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26  2:06   ` M Douglas McIlroy
  2021-01-26  2:53     ` Steve Nickolas
@ 2021-01-26 10:22     ` Tyler Adams
  2021-01-26 12:26       ` John P. Linderman
                         ` (2 more replies)
  1 sibling, 3 replies; 70+ messages in thread
From: Tyler Adams @ 2021-01-26 10:22 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1778 bytes --]

Looking at the 1978 list, the last one really stands out:

"Use tools in preference to unskilled help to lighten a programming task"
-- The concept of unskilled help for a programming task...doesn't really
exist in 2020. The only special case is doing unskilled labor yourself.
What unskilled tasks did people used to do back in the day?

Tyler


On Tue, Jan 26, 2021 at 4:07 AM M Douglas McIlroy <
m.douglas.mcilroy@dartmouth.edu> wrote:

> It might be interesting to compare your final list with the two lists in
> the 1978 special issue of the BSTJ--one in the Foreword, the other in the
> revised version of the Ritchi/Thompson article from the CACM. How have
> perceptions or values changed over time?
>
> Doug
>
>
> On Mon, Jan 25, 2021 at 7:32 AM Steve Nickolas <usotsuki@buric.co> wrote:
>
>> On Mon, 25 Jan 2021, Tyler Adams wrote:
>>
>> > I'm writing about my 5 favorite unix design principles on my blog this
>> > week, and it got me wondering what others' favorite unix design
>> principles
>> > are? For reference, mine are:
>> >
>> > - Rule of Separation (from TAOUP <
>> http://catb.org/~esr/writings/taoup/html/>
>> > )
>> > - Let the Machine Do the Dirty Work (from Elements of Programming Style)
>> > - Rule of Silence (from TAOUP <
>> http://catb.org/~esr/writings/taoup/html/>)
>> > - Data Dominates (Rob Pike #5)
>> > - The SPOT (Single Point of Truth) Rule (from TAOUP
>> > <http://catb.org/~esr/writings/taoup/html/>)
>> >
>> > Tyler
>> >
>>
>> 1. Pipes
>> 2. Text as the preferred format for input and output
>> 3. 'Most everything as a file
>> 4. The idea of simple tools that are optimized for a single task
>> 5. A powerful scripting language built into the system that, combined
>> with
>> 1-4, makes writing new tools heaps easier.
>>
>> -uso.
>>
>

[-- Attachment #2: Type: text/html, Size: 3056 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26 10:22     ` Tyler Adams
@ 2021-01-26 12:26       ` John P. Linderman
  2021-01-26 15:23       ` Clem Cole
       [not found]       ` <CAKH6PiXKjksEpQOMMMQTbcsMvX2thz3WzqjoRWJAsXnZ4Eq_iQ@mail.gmail.com>
  2 siblings, 0 replies; 70+ messages in thread
From: John P. Linderman @ 2021-01-26 12:26 UTC (permalink / raw)
  To: Tyler Adams; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

On Tue, Jan 26, 2021 at 5:24 AM Tyler Adams <coppero1237@gmail.com> wrote:

> Looking at the 1978 list, the last one really stands out:
>
> "Use tools in preference to unskilled help to lighten a programming task"
> -- The concept of unskilled help for a programming task...doesn't really
> exist in 2020. The only special case is doing unskilled labor yourself.
> What unskilled tasks did people used to do back in the day?
>
> Tyler
>
>>
>>>
Drifting far off topic, but when I was programming part time for the MIT
administration in the late 60's, we wrote a small language for generating
reports from tape input. It did things like reading a record ahead, so it
could detect the last and first records having some (presumably sorted)
value, making it easy to do headers and summaries. Many report requests
from assorted administrative offices could be handled by our operators, who
had no formal programming experience. Occasionally they'd get bitten by a
request like "Give me a count of freshman and sophomores in course 18" and
they'd do the equivalent of "class == freshmen && class == sophomore",
paralleling the spoken request, and then wonder why nothing got selected.
All in all, though, it took a huge load off the skilled programmers, and
reduced the time to respond to simple requests. When I joined the Labs in
1973, we had no operators, no administrative report requests, and most
co-workers were proficient programmers. But the administrative work
environment at MIT was more representative of "the real world" than the
Labs.

[-- Attachment #2: Type: text/html, Size: 2440 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26 10:22     ` Tyler Adams
  2021-01-26 12:26       ` John P. Linderman
@ 2021-01-26 15:23       ` Clem Cole
  2021-01-26 16:00         ` Niklas Karlsson
       [not found]       ` <CAKH6PiXKjksEpQOMMMQTbcsMvX2thz3WzqjoRWJAsXnZ4Eq_iQ@mail.gmail.com>
  2 siblings, 1 reply; 70+ messages in thread
From: Clem Cole @ 2021-01-26 15:23 UTC (permalink / raw)
  To: Tyler Adams; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

On Tue, Jan 26, 2021 at 5:23 AM Tyler Adams <coppero1237@gmail.com> wrote:

> Looking at the 1978 list, the last one really stands out:
>
> "Use tools in preference to unskilled help to lighten a programming task"
> -- The concept of unskilled help for a programming task...doesn't really
> exist in 2020. The only special case is doing unskilled labor yourself.
> What unskilled tasks did people used to do back in the day?
>
> Tyler
>
I've often wondered if this is the source of the infamous putdown: When you
operate in such or such manner,* "you could/should be replaced with a shell
script."*
ᐧ

[-- Attachment #2: Type: text/html, Size: 1568 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26 15:23       ` Clem Cole
@ 2021-01-26 16:00         ` Niklas Karlsson
  2021-01-26 16:13           ` Adam Thornton
  0 siblings, 1 reply; 70+ messages in thread
From: Niklas Karlsson @ 2021-01-26 16:00 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

Den tis 26 jan. 2021 kl 16:24 skrev Clem Cole <clemc@ccc.com>:

>
>
> On Tue, Jan 26, 2021 at 5:23 AM Tyler Adams <coppero1237@gmail.com> wrote:
>
>> Looking at the 1978 list, the last one really stands out:
>>
>> "Use tools in preference to unskilled help to lighten a programming task"
>> -- The concept of unskilled help for a programming task...doesn't really
>> exist in 2020. The only special case is doing unskilled labor yourself.
>> What unskilled tasks did people used to do back in the day?
>>
>> Tyler
>>
> I've often wondered if this is the source of the infamous putdown: When
> you operate in such or such manner,* "you could/should be replaced with a
> shell script."*
>

The version I've heard is even "... a very small shell script".

Niklas

> ᐧ
>

[-- Attachment #2: Type: text/html, Size: 2205 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-26 16:00         ` Niklas Karlsson
@ 2021-01-26 16:13           ` Adam Thornton
  0 siblings, 0 replies; 70+ messages in thread
From: Adam Thornton @ 2021-01-26 16:13 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]

To be fair I've made a decent career spanning five calendar decades so far
out of replacing the thing I started out doing with a shell script (in
recent decades, more often Perl, then Python), so it's not the _worst_ job
advice....

On Tue, Jan 26, 2021 at 9:01 AM Niklas Karlsson <nikke.karlsson@gmail.com>
wrote:

>
>
> Den tis 26 jan. 2021 kl 16:24 skrev Clem Cole <clemc@ccc.com>:
>
>>
>>
>> On Tue, Jan 26, 2021 at 5:23 AM Tyler Adams <coppero1237@gmail.com>
>> wrote:
>>
>>> Looking at the 1978 list, the last one really stands out:
>>>
>>> "Use tools in preference to unskilled help to lighten a programming
>>> task" -- The concept of unskilled help for a programming task...doesn't
>>> really exist in 2020. The only special case is doing unskilled labor
>>> yourself. What unskilled tasks did people used to do back in the day?
>>>
>>> Tyler
>>>
>> I've often wondered if this is the source of the infamous putdown: When
>> you operate in such or such manner,* "you could/should be replaced with
>> a shell script."*
>>
>
> The version I've heard is even "... a very small shell script".
>
> Niklas
>
>> ᐧ
>>
>

[-- Attachment #2: Type: text/html, Size: 2833 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
       [not found]       ` <CAKH6PiXKjksEpQOMMMQTbcsMvX2thz3WzqjoRWJAsXnZ4Eq_iQ@mail.gmail.com>
@ 2021-01-30 19:01         ` Tyler Adams
  2021-01-30 19:50           ` Jon Steinhart
  0 siblings, 1 reply; 70+ messages in thread
From: Tyler Adams @ 2021-01-30 19:01 UTC (permalink / raw)
  To: M Douglas McIlroy, The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 3115 bytes --]

For sure, I've seen at least two interesting changes:
- market forces have pushed fast iteration and fast prototyping into the
mainstream in the form of Silicon valley "fail fast" culture and the
"agile" culture. This, over the disastrous "waterfall" style, has led to a
momentous improvement in overall productivity improvements.
- As coders get pulled away from the machine and performance is less and
less in coders hands, engineers aren't sucked into (premature) optimization
as much.

 Tyler


On Sat, Jan 30, 2021 at 6:10 AM M Douglas McIlroy <
m.douglas.mcilroy@dartmouth.edu> wrote:

> Have you spotted an evolutionary trend toward better, more productive
> programmers? Or has programmer productivity risen across the board due to
> better tools?  Arguably what's happened is that principle has been
> self-obsoleting, for we have cut back on the demand for unskilled (i.e.
> less capable) programmers. A broad moral principle  may be in play:
> programmers should work to put themselves out of business, i.e. it is wrong
> to be doing the same kind of work (or working in the same way) tomorrowas
> yesterday.
>
> Doug
>
>
> On Tue, Jan 26, 2021 at 5:23 AM Tyler Adams <coppero1237@gmail.com> wrote:
>
>> Looking at the 1978 list, the last one really stands out:
>>
>> "Use tools in preference to unskilled help to lighten a programming task"
>> -- The concept of unskilled help for a programming task...doesn't really
>> exist in 2020. The only special case is doing unskilled labor yourself.
>> What unskilled tasks did people used to do back in the day?
>>
>> Tyler
>>
>>
>> On Tue, Jan 26, 2021 at 4:07 AM M Douglas McIlroy <
>> m.douglas.mcilroy@dartmouth.edu> wrote:
>>
>>> It might be interesting to compare your final list with the two lists in
>>> the 1978 special issue of the BSTJ--one in the Foreword, the other in the
>>> revised version of the Ritchi/Thompson article from the CACM. How have
>>> perceptions or values changed over time?
>>>
>>> Doug
>>>
>>>
>>> On Mon, Jan 25, 2021 at 7:32 AM Steve Nickolas <usotsuki@buric.co>
>>> wrote:
>>>
>>>> On Mon, 25 Jan 2021, Tyler Adams wrote:
>>>>
>>>> > I'm writing about my 5 favorite unix design principles on my blog this
>>>> > week, and it got me wondering what others' favorite unix design
>>>> principles
>>>> > are? For reference, mine are:
>>>> >
>>>> > - Rule of Separation (from TAOUP <
>>>> http://catb.org/~esr/writings/taoup/html/>
>>>> > )
>>>> > - Let the Machine Do the Dirty Work (from Elements of Programming
>>>> Style)
>>>> > - Rule of Silence (from TAOUP <
>>>> http://catb.org/~esr/writings/taoup/html/>)
>>>> > - Data Dominates (Rob Pike #5)
>>>> > - The SPOT (Single Point of Truth) Rule (from TAOUP
>>>> > <http://catb.org/~esr/writings/taoup/html/>)
>>>> >
>>>> > Tyler
>>>> >
>>>>
>>>> 1. Pipes
>>>> 2. Text as the preferred format for input and output
>>>> 3. 'Most everything as a file
>>>> 4. The idea of simple tools that are optimized for a single task
>>>> 5. A powerful scripting language built into the system that, combined
>>>> with
>>>> 1-4, makes writing new tools heaps easier.
>>>>
>>>> -uso.
>>>>
>>>

[-- Attachment #2: Type: text/html, Size: 5101 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 19:01         ` Tyler Adams
@ 2021-01-30 19:50           ` Jon Steinhart
  2021-01-30 20:06             ` Tyler Adams
  0 siblings, 1 reply; 70+ messages in thread
From: Jon Steinhart @ 2021-01-30 19:50 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Tyler Adams writes:
>
> For sure, I've seen at least two interesting changes:
> - market forces have pushed fast iteration and fast prototyping into the
> mainstream in the form of Silicon valley "fail fast" culture and the
> "agile" culture. This, over the disastrous "waterfall" style, has led to a
> momentous improvement in overall productivity improvements.
> - As coders get pulled away from the machine and performance is less and
> less in coders hands, engineers aren't sucked into (premature) optimization
> as much.

It's interesting in more than one way.

The "fail fast" culture seems to result in a lot more failure than I find
acceptable.

As performance is less in coders hands, performance is getting worse.  I
haven't seen less premature optimization, I've just seen more premature
optimization that didn't optimize anything.

My take is that the above changes have resulted in less reliable products
with poor performance being delivered more quickly.  I'm just kind of weird
in that I'd prefer better products delivered more slowly.  Especially since
much of what counts as a product these days is just churn to keep people
buying, not to provide things that are actually useful.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 19:50           ` Jon Steinhart
@ 2021-01-30 20:06             ` Tyler Adams
  2021-01-30 21:28               ` Clem Cole
  0 siblings, 1 reply; 70+ messages in thread
From: Tyler Adams @ 2021-01-30 20:06 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1479 bytes --]

Really? Except for one particularly incompetent team, I cannot recall
working with nor reviewing code that sacrificed clarity for performance.

 Tyler


On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon@fourwinds.com> wrote:

> Tyler Adams writes:
> >
> > For sure, I've seen at least two interesting changes:
> > - market forces have pushed fast iteration and fast prototyping into the
> > mainstream in the form of Silicon valley "fail fast" culture and the
> > "agile" culture. This, over the disastrous "waterfall" style, has led to
> a
> > momentous improvement in overall productivity improvements.
> > - As coders get pulled away from the machine and performance is less and
> > less in coders hands, engineers aren't sucked into (premature)
> optimization
> > as much.
>
> It's interesting in more than one way.
>
> The "fail fast" culture seems to result in a lot more failure than I find
> acceptable.
>
> As performance is less in coders hands, performance is getting worse.  I
> haven't seen less premature optimization, I've just seen more premature
> optimization that didn't optimize anything.
>
> My take is that the above changes have resulted in less reliable products
> with poor performance being delivered more quickly.  I'm just kind of weird
> in that I'd prefer better products delivered more slowly.  Especially since
> much of what counts as a product these days is just churn to keep people
> buying, not to provide things that are actually useful.
>

[-- Attachment #2: Type: text/html, Size: 2065 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 20:06             ` Tyler Adams
@ 2021-01-30 21:28               ` Clem Cole
  2021-01-30 21:42                 ` Dave Horsfall
                                   ` (3 more replies)
  0 siblings, 4 replies; 70+ messages in thread
From: Clem Cole @ 2021-01-30 21:28 UTC (permalink / raw)
  To: Tyler Adams; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 3378 bytes --]

Tyler - I'm with Jon on this.  I'll pick on Apple here.  It used to be a
huge difference between MSFT SW and MacOS was that the systems folks at
Apple really tested the system and the result was that Mac OS with way
really stable.   My system never panic'ed except when I ran Windows under
parallels.   After 3-4 years ago, that stopped being true.  Crashes occur,
just like Windows BSD.   It's not unusual for my Mac to panic just letting
it run overnight - which is just backups and the like.  Yes, I have a
multiple monitors, a zillion windows open etc..


I come downstairs and the screen is blank (it should be, I have it turn off
after no activity), but I move the mouse or try to type something  --
nothing wakes the system up again.  I've chased it to Mac OS running out of
memory and not gracefully handling the lowe memory situation.  Sad, I have
16G of RAM a 1T SSD and many TB of memory on Thunderbolt 3 connectivity.

Look I grew up with a 256K byte RAM Unix V6 system on an 11/34, 3 RK05s and
an RK07 for storage.  We swapped.  Yeah, I never ran a window manager, but
he had a number of 9600K terminals on DH11's and we were happy.  You could
see it swapping like mad, but that system never crashed.  It just ran and
ran and ran.

IMO, this is what I think Jon is referring.   Those systems were stable
because we tested them and found and fixed the issue.   These days, Apple
no longer cares about Mac OS because iOS is where they now put their
effort, although I'm not super impressed there either, but I also don't
push it like I do Mac OS.  Sad really.   If I could get the day-2-day
applications that I need to work on FreeBSD, I suspect I would be there in
a heartbeat.

Clem
ᐧ

On Sat, Jan 30, 2021 at 3:07 PM Tyler Adams <coppero1237@gmail.com> wrote:

> Really? Except for one particularly incompetent team, I cannot recall
> working with nor reviewing code that sacrificed clarity for performance.
>
>  Tyler
>
>
> On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon@fourwinds.com> wrote:
>
>> Tyler Adams writes:
>> >
>> > For sure, I've seen at least two interesting changes:
>> > - market forces have pushed fast iteration and fast prototyping into the
>> > mainstream in the form of Silicon valley "fail fast" culture and the
>> > "agile" culture. This, over the disastrous "waterfall" style, has led
>> to a
>> > momentous improvement in overall productivity improvements.
>> > - As coders get pulled away from the machine and performance is less and
>> > less in coders hands, engineers aren't sucked into (premature)
>> optimization
>> > as much.
>>
>> It's interesting in more than one way.
>>
>> The "fail fast" culture seems to result in a lot more failure than I find
>> acceptable.
>>
>> As performance is less in coders hands, performance is getting worse.  I
>> haven't seen less premature optimization, I've just seen more premature
>> optimization that didn't optimize anything.
>>
>> My take is that the above changes have resulted in less reliable products
>> with poor performance being delivered more quickly.  I'm just kind of
>> weird
>> in that I'd prefer better products delivered more slowly.  Especially
>> since
>> much of what counts as a product these days is just churn to keep people
>> buying, not to provide things that are actually useful.
>>
>

[-- Attachment #2: Type: text/html, Size: 5257 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 21:28               ` Clem Cole
@ 2021-01-30 21:42                 ` Dave Horsfall
  2021-01-30 21:45                 ` Tyler Adams
                                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 70+ messages in thread
From: Dave Horsfall @ 2021-01-30 21:42 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On Sat, 30 Jan 2021, Clem Cole wrote:

> Look I grew up with a 256K byte RAM Unix V6 system on an 11/34, 3 RK05s 
> and an RK07 for storage.  We swapped.  Yeah, I never ran a window 
> manager, but he had a number of 9600K terminals on DH11's and we were 
> happy.  You could see it swapping like mad, but that system never 
> crashed.  It just ran and ran and ran.

You had *three* RK05s, an RK07, *and* a DH11?  We had just two, and a 
crappy DJ-11 on our /40s.  Pretty much stable; any bugs were fixed on the 
spot because we had the source code after all.

(They were configured that way to run RSX-11D, then that issue of the BSTJ 
appeared; the rest is history.)

-- Dave

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 21:28               ` Clem Cole
  2021-01-30 21:42                 ` Dave Horsfall
@ 2021-01-30 21:45                 ` Tyler Adams
  2021-01-30 22:31                   ` Larry McVoy
  2021-01-30 22:28                 ` Larry McVoy
  2021-01-30 23:09                 ` [TUHS] Favorite unix design principles? John Cowan
  3 siblings, 1 reply; 70+ messages in thread
From: Tyler Adams @ 2021-01-30 21:45 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 4079 bytes --]

Fair points about macOS, I hated using my mac in 2018 (thankfully I've
lived in linux bliss since), but I wouldn't say macOS is representative of
the software industry as a whole. Especially as you said iOS is Apple's
real cash cow and apple's been focusing on better *mac hardware *and from
what I hear that M1 delivers*.*

The modern software industry is mostly over the browser (or an app), and
honestly almost every site I use these days is pretty fast and stable.
Unless it has too many ads.

 Tyler


On Sat, Jan 30, 2021 at 11:28 PM Clem Cole <clemc@ccc.com> wrote:

> Tyler - I'm with Jon on this.  I'll pick on Apple here.  It used to be a
> huge difference between MSFT SW and MacOS was that the systems folks at
> Apple really tested the system and the result was that Mac OS with way
> really stable.   My system never panic'ed except when I ran Windows under
> parallels.   After 3-4 years ago, that stopped being true.  Crashes occur,
> just like Windows BSD.   It's not unusual for my Mac to panic just letting
> it run overnight - which is just backups and the like.  Yes, I have a
> multiple monitors, a zillion windows open etc..
>
>
> I come downstairs and the screen is blank (it should be, I have it turn
> off after no activity), but I move the mouse or try to type something  --
> nothing wakes the system up again.  I've chased it to Mac OS running out of
> memory and not gracefully handling the lowe memory situation.  Sad, I have
> 16G of RAM a 1T SSD and many TB of memory on Thunderbolt 3 connectivity.
>
> Look I grew up with a 256K byte RAM Unix V6 system on an 11/34, 3 RK05s
> and an RK07 for storage.  We swapped.  Yeah, I never ran a window manager,
> but he had a number of 9600K terminals on DH11's and we were happy.  You
> could see it swapping like mad, but that system never crashed.  It just ran
> and ran and ran.
>
> IMO, this is what I think Jon is referring.   Those systems were stable
> because we tested them and found and fixed the issue.   These days, Apple
> no longer cares about Mac OS because iOS is where they now put their
> effort, although I'm not super impressed there either, but I also don't
> push it like I do Mac OS.  Sad really.   If I could get the day-2-day
> applications that I need to work on FreeBSD, I suspect I would be there in
> a heartbeat.
>
> Clem
> ᐧ
>
> On Sat, Jan 30, 2021 at 3:07 PM Tyler Adams <coppero1237@gmail.com> wrote:
>
>> Really? Except for one particularly incompetent team, I cannot recall
>> working with nor reviewing code that sacrificed clarity for performance.
>>
>>  Tyler
>>
>>
>> On Sat, Jan 30, 2021 at 9:51 PM Jon Steinhart <jon@fourwinds.com> wrote:
>>
>>> Tyler Adams writes:
>>> >
>>> > For sure, I've seen at least two interesting changes:
>>> > - market forces have pushed fast iteration and fast prototyping into
>>> the
>>> > mainstream in the form of Silicon valley "fail fast" culture and the
>>> > "agile" culture. This, over the disastrous "waterfall" style, has led
>>> to a
>>> > momentous improvement in overall productivity improvements.
>>> > - As coders get pulled away from the machine and performance is less
>>> and
>>> > less in coders hands, engineers aren't sucked into (premature)
>>> optimization
>>> > as much.
>>>
>>> It's interesting in more than one way.
>>>
>>> The "fail fast" culture seems to result in a lot more failure than I find
>>> acceptable.
>>>
>>> As performance is less in coders hands, performance is getting worse.  I
>>> haven't seen less premature optimization, I've just seen more premature
>>> optimization that didn't optimize anything.
>>>
>>> My take is that the above changes have resulted in less reliable products
>>> with poor performance being delivered more quickly.  I'm just kind of
>>> weird
>>> in that I'd prefer better products delivered more slowly.  Especially
>>> since
>>> much of what counts as a product these days is just churn to keep people
>>> buying, not to provide things that are actually useful.
>>>
>>

[-- Attachment #2: Type: text/html, Size: 6295 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 21:28               ` Clem Cole
  2021-01-30 21:42                 ` Dave Horsfall
  2021-01-30 21:45                 ` Tyler Adams
@ 2021-01-30 22:28                 ` Larry McVoy
  2021-01-30 23:11                   ` [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) Greg 'groggy' Lehey
  2021-01-30 23:09                 ` [TUHS] Favorite unix design principles? John Cowan
  3 siblings, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-01-30 22:28 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
> If I could get the day-2-day
> applications that I need to work on FreeBSD, I suspect I would be there in
> a heartbeat.

I dunno about that.  I tried out FreeBSD a couple of years ago when
Netflix was flirting with me.  The installer hasn't seen any loving in
30 years it would seem.  The disk setup tool sucks just as bad as it
did back in 1988.

I remember when Linux was this bad in the .90ish releases.  A long time
ago.  Now their install is painless, it's every bit as good as Windows
and maybe better.  And it got that way fast, I remember doing an install
on some machine around 1998 or 1999, I didn't have a mouse plugged in,
no worries, you could just move around with the keyboard.   X11 came up
as part of the install, the entire install was graphical and seamless.

FreeBSD is stuck in the 1990's in terms of user interface.  They've done
some good stuff in the kernel but it's not an end user system, I'd argue
that the install is much worse than any of the Sun installers and I
started with BSD.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 21:45                 ` Tyler Adams
@ 2021-01-30 22:31                   ` Larry McVoy
  0 siblings, 0 replies; 70+ messages in thread
From: Larry McVoy @ 2021-01-30 22:31 UTC (permalink / raw)
  To: Tyler Adams; +Cc: The Eunuchs Hysterical Society

On Sat, Jan 30, 2021 at 11:45:38PM +0200, Tyler Adams wrote:
> Fair points about macOS, I hated using my mac in 2018 (thankfully I've
> lived in linux bliss since), but I wouldn't say macOS is representative of
> the software industry as a whole. Especially as you said iOS is Apple's
> real cash cow and apple's been focusing on better *mac hardware *and from
> what I hear that M1 delivers*.*

M1 is a stunning pile of systems work.  Warms my heart.  Seems to be
faster, uses less power, and cheaper than Intel's best.  Kudos to Apple
for that.

> The modern software industry is mostly over the browser (or an app), and
> honestly almost every site I use these days is pretty fast and stable.
> Unless it has too many ads.

Ad blocker to the rescue, I use ublock origin and it works great.

--lm

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 21:28               ` Clem Cole
                                   ` (2 preceding siblings ...)
  2021-01-30 22:28                 ` Larry McVoy
@ 2021-01-30 23:09                 ` John Cowan
  2021-01-30 23:22                   ` Jon Steinhart
  3 siblings, 1 reply; 70+ messages in thread
From: John Cowan @ 2021-01-30 23:09 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1994 bytes --]

On Sat, Jan 30, 2021 at 4:29 PM Clem Cole <clemc@ccc.com> wrote:


> These days, Apple no longer cares about Mac OS because iOS is where they
> now put their effort, although I'm not super impressed there either, but I
> also don't push it like I do Mac OS.  Sad really.   If I could get the
> day-2-day applications that I need to work on FreeBSD, I suspect I would be
> there in a heartbeat.
>

Decades ago I worked out the lifecycle of tech companies:

1) Engineering-driven: the goal is to make and sell high-quality products.
DEC and HP were like this for a long time.

2) Sales-driven: the goal is to sell products, high-quality or not.  Too
many examples to specify.

3) Finance-driven: the goal is to make money, whether you sell products or
not.  The classic case here is Carnegie Steel.  When Carnegie told his
direct reports they were going into the railroad business and they
protested that the company knew nothing about it, he said "Carnegie Steel
isn't about making steel, it's about making money, and anyone who forgets
that is fired."

4) Survival-driven: the goal is to keep the company going whether you make
money or not.  Auto companies just after bailouts are in this step.  In
particular, Chrysler was bailed out in 1979 and again in 2009: see Tom
Paxton's song "I'm Changing My Name To Chrysler" (covered by Arlo and Pete
on _Precious Friends_).

On Sat, Jan 30, 2021 at 4:43 PM Dave Horsfall <dave@horsfall.org> wrote:

You had *three* RK05s, an RK07, *and* a DH11?  We had just two, and a
> crappy DJ-11 on our /40s.  Pretty much stable; any bugs were fixed on the
> spot because we had the source code after all.
>

I cut my teeth on a PDP-8 with a single TU58 DECtape and a TD8/E controller
(no interrupts, no data break cycles; I/O was more or less analogous to
terminal I/O).  Uphill both ways in the snow.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
So they play that [tune] on their fascist banjos, eh?
        --Great-Souled Sam

[-- Attachment #2: Type: text/html, Size: 4342 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 22:28                 ` Larry McVoy
@ 2021-01-30 23:11                   ` Greg 'groggy' Lehey
  2021-01-30 23:17                     ` Larry McVoy
                                       ` (2 more replies)
  0 siblings, 3 replies; 70+ messages in thread
From: Greg 'groggy' Lehey @ 2021-01-30 23:11 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2443 bytes --]

On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>> If I could get the day-2-day
>> applications that I need to work on FreeBSD, I suspect I would be there in
>> a heartbeat.
>
> I dunno about that.  I tried out FreeBSD a couple of years ago when
> Netflix was flirting with me.  The installer hasn't seen any loving in
> 30 years it would seem.  The disk setup tool sucks just as bad as it
> did back in 1988.

You could be right there, for some value of 1988 (FreeBSD came into
being in 1992).  The tools work without being good.  But how often do
you use them?  I've been using FreeBSD since the beginning, and I
can't recall when I last used the disk partitioning tool, though I'm
sure that when I did I overrode a lot of (all?) the suggestions.

> I remember when Linux was this bad in the .90ish releases.  A long
> time ago.  Now their install is painless, it's every bit as good as
> Windows and maybe better.

FWIW, I find Microsoft "Windows" installation terminally confusing
(that's what you were talking about, right?).  And I've run into
serious problems with various Linux installations too.  That doesn't
make the FreeBSD tools better, but maybe it relativizes it.

> And it got that way fast, I remember doing an install on some
> machine around 1998 or 1999, I didn't have a mouse plugged in, no
> worries, you could just move around with the keyboard.  X11 came up
> as part of the install, the entire install was graphical and
> seamless.

The FreeBSD installer *does* install X if you select it.

> FreeBSD is stuck in the 1990's in terms of user interface.

You're still talking about the installer, aren't you?  The normal user
interface is via the shell, which hasn't changed, and for a good
reason.

> They've done some good stuff in the kernel but it's not an end user
> system,

There I have to agree with you.  A little TLC would go a long way.
But I hope that you're not advocating the "change your GUI with your
underwear" attitude that Microsoft, Apple and many Linux distros
have.  One of the reasons I don't use Linux is because every time I
try, the interface has changed.

Greg
--
Sent from my desktop computer.
Finger grog@lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:11                   ` [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) Greg 'groggy' Lehey
@ 2021-01-30 23:17                     ` Larry McVoy
  2021-01-30 23:22                       ` Warner Losh
  2021-01-31  0:39                     ` Steve Nickolas
  2021-01-31  1:47                     ` Will Senn
  2 siblings, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-01-30 23:17 UTC (permalink / raw)
  To: Greg 'groggy' Lehey; +Cc: The Eunuchs Hysterical Society

On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg 'groggy' Lehey wrote:
> > I remember when Linux was this bad in the .90ish releases.  A long
> > time ago.  Now their install is painless, it's every bit as good as
> > Windows and maybe better.
> 
> FWIW, I find Microsoft "Windows" installation terminally confusing
> (that's what you were talking about, right?).  And I've run into
> serious problems with various Linux installations too.  That doesn't
> make the FreeBSD tools better, but maybe it relativizes it.

Um, my mother could install any Linux system today and 10-20 years ago.
There is not the slightest chance that she could install FreeBSD.

> The FreeBSD installer *does* install X if you select it.

Linux installers start in X.  No "select it" required.

> > FreeBSD is stuck in the 1990's in terms of user interface.
> 
> You're still talking about the installer, aren't you?  

Yup.  If FreeBSD wants anyone to use it, fix that installer.  99.99%
of people would give up after seeing that, you'd never get them to 
userland.

> > They've done some good stuff in the kernel but it's not an end user
> > system,
> 
> There I have to agree with you.  A little TLC would go a long way.
> But I hope that you're not advocating the "change your GUI with your
> underwear" attitude that Microsoft, Apple and many Linux distros
> have.  One of the reasons I don't use Linux is because every time I
> try, the interface has changed.

Try xubuntu, that's what I use.  Pretty light weight UI but all the
parts are there and it doesn't change much.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:17                     ` Larry McVoy
@ 2021-01-30 23:22                       ` Warner Losh
  2021-01-30 23:31                         ` [TUHS] [SPAM] " Larry McVoy
  2021-02-09  2:15                         ` [TUHS] " Will Senn
  0 siblings, 2 replies; 70+ messages in thread
From: Warner Losh @ 2021-01-30 23:22 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2143 bytes --]

On Sat, Jan 30, 2021 at 4:19 PM Larry McVoy <lm@mcvoy.com> wrote:

> On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg 'groggy' Lehey wrote:
> > > I remember when Linux was this bad in the .90ish releases.  A long
> > > time ago.  Now their install is painless, it's every bit as good as
> > > Windows and maybe better.
> >
> > FWIW, I find Microsoft "Windows" installation terminally confusing
> > (that's what you were talking about, right?).  And I've run into
> > serious problems with various Linux installations too.  That doesn't
> > make the FreeBSD tools better, but maybe it relativizes it.
>
> Um, my mother could install any Linux system today and 10-20 years ago.
> There is not the slightest chance that she could install FreeBSD.
>

I find that hard to believe. The defaults just work on the vast majority of
systems, even if the interface is text-based and not a fancy GUI...


> > The FreeBSD installer *does* install X if you select it.
>
> Linux installers start in X.  No "select it" required.
>

Yea. Once upon a  time, this was super dangerous. These days it's kinda
required.


> > > FreeBSD is stuck in the 1990's in terms of user interface.
> >
> > You're still talking about the installer, aren't you?
>
> Yup.  If FreeBSD wants anyone to use it, fix that installer.  99.99%
> of people would give up after seeing that, you'd never get them to
> userland.
>

No argument there...  Part of the problem is that, up until relatively
lately, the whole X experience sucked really badly on FreeBSD. Now that it
doesn't suck, it's time for a re-evaluation...


> > > They've done some good stuff in the kernel but it's not an end user
> > > system,
> >
> > There I have to agree with you.  A little TLC would go a long way.
> > But I hope that you're not advocating the "change your GUI with your
> > underwear" attitude that Microsoft, Apple and many Linux distros
> > have.  One of the reasons I don't use Linux is because every time I
> > try, the interface has changed.
>
> Try xubuntu, that's what I use.  Pretty light weight UI but all the
> parts are there and it doesn't change much.
>

But yet it's not stuck?

Warner

[-- Attachment #2: Type: text/html, Size: 3299 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] Favorite unix design principles?
  2021-01-30 23:09                 ` [TUHS] Favorite unix design principles? John Cowan
@ 2021-01-30 23:22                   ` Jon Steinhart
  0 siblings, 0 replies; 70+ messages in thread
From: Jon Steinhart @ 2021-01-30 23:22 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

John Cowan writes:
>
> Decades ago I worked out the lifecycle of tech companies:
>
> 1) Engineering-driven: the goal is to make and sell high-quality products.
> DEC and HP were like this for a long time.
>
> 2) Sales-driven: the goal is to sell products, high-quality or not.  Too
> many examples to specify.
>
> 3) Finance-driven: the goal is to make money, whether you sell products or
> not.  The classic case here is Carnegie Steel.  When Carnegie told his
> direct reports they were going into the railroad business and they
> protested that the company knew nothing about it, he said "Carnegie Steel
> isn't about making steel, it's about making money, and anyone who forgets
> that is fired."
>
> 4) Survival-driven: the goal is to keep the company going whether you make
> money or not.  Auto companies just after bailouts are in this step.  In
> particular, Chrysler was bailed out in 1979 and again in 2009: see Tom
> Paxton's song "I'm Changing My Name To Chrysler" (covered by Arlo and Pete
> on _Precious Friends_).

Don't want to get too off topic here, but I consider myself fortunate
to have spent the prime of my career working on #1.  I always believed
that if one made high quality-products that met a customer's need that
the rest would take care of itself.  Yeah, I'm naive.  I see 2-4 as
indicative of Silicon Valley today where I don't think that I could work.
It's no longer about making great things, it's about making money, if if
absolutely necessary, making something to support that goal.  If you took
the trouble to read the Microsoft antitrust trial depositions, you'd see
that this was exactly Bill Gate's MO.  He would try to tank competition
through any means necessary except making a better product; that only
happened when everything else failed.

I've probably ranted about this before, but to me this is emblematic of
how America has lost its way.  My big evil villians are Jack Welch and
Bill Gates.  When I was growing up and kids were asked "What do you want
to be when you grow up?", the answer was always astronaut first, then
doctor, lawyer, engineer, and so on.  Post Microsoft, the answer was "I
want to be as rich as Bill Gates."  People just want the outcome without
having to do any of the work, and just want the results without the joy
of actual accomplishment.  I find it sad.

Jon

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:22                       ` Warner Losh
@ 2021-01-30 23:31                         ` Larry McVoy
  2021-01-30 23:37                           ` Jon Steinhart
  2021-02-09  2:15                         ` [TUHS] " Will Senn
  1 sibling, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-01-30 23:31 UTC (permalink / raw)
  To: Warner Losh; +Cc: The Eunuchs Hysterical Society

On Sat, Jan 30, 2021 at 04:22:49PM -0700, Warner Losh wrote:
> On Sat, Jan 30, 2021 at 4:19 PM Larry McVoy <lm@mcvoy.com> wrote:
> 
> > On Sun, Jan 31, 2021 at 10:11:19AM +1100, Greg 'groggy' Lehey wrote:
> > > > I remember when Linux was this bad in the .90ish releases.  A long
> > > > time ago.  Now their install is painless, it's every bit as good as
> > > > Windows and maybe better.
> > >
> > > FWIW, I find Microsoft "Windows" installation terminally confusing
> > > (that's what you were talking about, right?).  And I've run into
> > > serious problems with various Linux installations too.  That doesn't
> > > make the FreeBSD tools better, but maybe it relativizes it.
> >
> > Um, my mother could install any Linux system today and 10-20 years ago.
> > There is not the slightest chance that she could install FreeBSD.
> 
> I find that hard to believe. The defaults just work on the vast majority of
> systems, even if the interface is text-based and not a fancy GUI...

I speak from experience of trying to install FreeBSD on a netflix server
a couple of years back.  It wasn't pleasant and it seems pretty identical
to the installer I used decades ago.

> > > > FreeBSD is stuck in the 1990's in terms of user interface.
> > >
> > > You're still talking about the installer, aren't you?
> >
> > Yup.  If FreeBSD wants anyone to use it, fix that installer.  99.99%
> > of people would give up after seeing that, you'd never get them to
> > userland.
> 
> No argument there...  Part of the problem is that, up until relatively
> lately, the whole X experience sucked really badly on FreeBSD. Now that it
> doesn't suck, it's time for a re-evaluation...

It's 20 years past the time for that re-evaluation.  Seriously, Linux
distros have been installing in X for at least 20 years, I think maybe
more.  If they can do it....

> > > > They've done some good stuff in the kernel but it's not an end user
> > > > system,
> > >
> > > There I have to agree with you.  A little TLC would go a long way.
> > > But I hope that you're not advocating the "change your GUI with your
> > > underwear" attitude that Microsoft, Apple and many Linux distros
> > > have.  One of the reasons I don't use Linux is because every time I
> > > try, the interface has changed.
> >
> > Try xubuntu, that's what I use.  Pretty light weight UI but all the
> > parts are there and it doesn't change much.
> 
> But yet it's not stuck?

Nope, it's not remotely stuck.  I get it, you like FreeBSD.  That's fine
but be honest about where it is.  How many of the developers with a commit
bit actually run FreeBSD on their desktop and laptops?  As their daily
platform?

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:31                         ` [TUHS] [SPAM] " Larry McVoy
@ 2021-01-30 23:37                           ` Jon Steinhart
  2021-01-30 23:54                             ` Larry McVoy
  2021-01-31  0:00                             ` [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?) Bakul Shah
  0 siblings, 2 replies; 70+ messages in thread
From: Jon Steinhart @ 2021-01-30 23:37 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Larry McVoy writes:
> It's 20 years past the time for that re-evaluation.  Seriously, Linux
> distros have been installing in X for at least 20 years, I think maybe
> more.  If they can do it....

That doesn't mean that it's great.  It's only easy to use once one learns
the quirks, like it appearing to hang with no feedback when you hit buttons,
and the way one has to click "done" when you're not actually done.  The
number of options is so small that I think that it would be better to just
have the whole configuration on a single screen so that newbies wouldn't
have to wonder where things were done and whether or not they'd be able to
"go back" without trying it.

> Nope, it's not remotely stuck.  I get it, you like FreeBSD.  That's fine
> but be honest about where it is.  How many of the developers with a commit
> bit actually run FreeBSD on their desktop and laptops?  As their daily
> platform?

Huh, I thought that most of the linux developers these days ran on Macs.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:37                           ` Jon Steinhart
@ 2021-01-30 23:54                             ` Larry McVoy
  2021-01-31 12:23                               ` [TUHS] [SPAM] Re: FreeBSD behind the times? Dermot Tynan
  2021-01-31  0:00                             ` [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?) Bakul Shah
  1 sibling, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-01-30 23:54 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

On Sat, Jan 30, 2021 at 03:37:29PM -0800, Jon Steinhart wrote:
> Larry McVoy writes:
> > It's 20 years past the time for that re-evaluation.  Seriously, Linux
> > distros have been installing in X for at least 20 years, I think maybe
> > more.  If they can do it....
> 
> That doesn't mean that it's great.  

It's on the same level as Windows and has been for a long time.  In fact,
it has been better than Windows for at least 10 years and probably longer.
It takes me way longer to install Windows and then go get this driver, 
and then go get that one, whoops, that's the wrong one, they wanted 
32 bit emulation mode blah blah blah, go get that one. 

Linux, other than closed source drivers and even that is pretty pleasant
these days, just works.

> Huh, I thought that most of the linux developers these days ran on Macs.

I can imagine that, Macs have nice screens.  But they run Linux on their
Macs.

Look, this is turning into a Linux vs BSD pissing contest.  You do
realize that I started in the best BSD ever, SunOS, right?  I was a BSD
guy long before I was a Linux guy, UW-Madison was all BSD on the vaxen
and I loved it.  But I didn't have to install it.  So it wasn't until
later I got to see the hell that is the FreeBSD installer.

You like FreeBSD, cool, it's fine.  Nobody will ever know because nobody
is going to put up with a 1990's era installer.  Nobody.  It's 2020, shit
is just supposed to work, it's supposed do so in a graphical environment
let works with or without a mouse.  Anyone who doesn't see that isn't
going to get any market share.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:37                           ` Jon Steinhart
  2021-01-30 23:54                             ` Larry McVoy
@ 2021-01-31  0:00                             ` Bakul Shah
  1 sibling, 0 replies; 70+ messages in thread
From: Bakul Shah @ 2021-01-31  0:00 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Jan 30, 2021, at 3:37 PM, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> Larry McVoy writes:
>> It's 20 years past the time for that re-evaluation.  Seriously, Linux
>> distros have been installing in X for at least 20 years, I think maybe
>> more.  If they can do it....
> 
> That doesn't mean that it's great.  It's only easy to use once one learns
> the quirks, like it appearing to hang with no feedback when you hit buttons,
> and the way one has to click "done" when you're not actually done.  The
> number of options is so small that I think that it would be better to just
> have the whole configuration on a single screen so that newbies wouldn't
> have to wonder where things were done and whether or not they'd be able to
> "go back" without trying it.

I vastly prefer FreeBSD over Linux but Larry has a point. Presenting *any*
configuration choice to a newbie can be confusing as they often lack the
knowledge to make an "informed choice". I do think your idea of a single
screen config is great, but with defaults chosen! The installer should
proceed after a time if the user doesn't change anything. Perhaps this can
be a useful GSOC project?

But we should bring this back to design principles! Ease of use should
certainly be one of them.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:11                   ` [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) Greg 'groggy' Lehey
  2021-01-30 23:17                     ` Larry McVoy
@ 2021-01-31  0:39                     ` Steve Nickolas
  2021-01-31  1:47                     ` Will Senn
  2 siblings, 0 replies; 70+ messages in thread
From: Steve Nickolas @ 2021-01-31  0:39 UTC (permalink / raw)
  To: Greg 'groggy' Lehey; +Cc: The Eunuchs Hysterical Society

On Sun, 31 Jan 2021, Greg 'groggy' Lehey wrote:

> But I hope that you're not advocating the "change your GUI with your
> underwear" attitude that Microsoft, Apple and many Linux distros
> have.  One of the reasons I don't use Linux is because every time I
> try, the interface has changed.

I'm still using the same GUI I've used for the last decade, although its 
name changed.

Personally, I think a lot of Linux people who haven't come from a Unix 
background don't understand Linux, because they don't understand Unix. And 
because they don't understand Linux, they're trying to mutilate it into 
being something other than what it was meant to be.

I thought of creating a system built on top of Linux that represented the 
system as I visualized it.  And I've come a fair way, but I don't yet have 
the important parts (the ability to self-host, including a working kernel, 
C compiler and libc) up and running, but there's pretty much your basic 
stuff in /bin, /usr/bin etc., and there's a curses library, and there's ed 
and vi (more or less)...so once those are available there'll be enough to 
get the rest of it functional.  Of course, as I visualized it will seem 
remarkably antiquated to most people!

-uso.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:11                   ` [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) Greg 'groggy' Lehey
  2021-01-30 23:17                     ` Larry McVoy
  2021-01-31  0:39                     ` Steve Nickolas
@ 2021-01-31  1:47                     ` Will Senn
  2021-01-31  2:25                       ` Larry McVoy
  2 siblings, 1 reply; 70+ messages in thread
From: Will Senn @ 2021-01-31  1:47 UTC (permalink / raw)
  To: Greg 'groggy' Lehey; +Cc: The Eunuchs Hysterical Society

Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has NEVER let me down and I’ve done my share of stupid user error. Now that Linux has ZFS, it doesn’t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I think :). But, I’m always up for talking up FBSD. I use it in my classes, too and the system is much more coherent for my systems programming classes than linux.

Will

Sent from my iPhone

> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com> wrote:
> 
>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>> If I could get the day-2-day
>>> applications that I need to work on FreeBSD, I suspect I would be there in
>>> a heartbeat.
>> 
>> I dunno about that.  I tried out FreeBSD a couple of years ago when
>> Netflix was flirting with me.  The installer hasn't seen any loving in
>> 30 years it would seem.  The disk setup tool sucks just as bad as it
>> did back in 1988.
> 
> You could be right there, for some value of 1988 (FreeBSD came into
> being in 1992).  The tools work without being good.  But how often do
> you use them?  I've been using FreeBSD since the beginning, and I
> can't recall when I last used the disk partitioning tool, though I'm
> sure that when I did I overrode a lot of (all?) the suggestions.
> 
>> I remember when Linux was this bad in the .90ish releases.  A long
>> time ago.  Now their install is painless, it's every bit as good as
>> Windows and maybe better.
> 
> FWIW, I find Microsoft "Windows" installation terminally confusing
> (that's what you were talking about, right?).  And I've run into
> serious problems with various Linux installations too.  That doesn't
> make the FreeBSD tools better, but maybe it relativizes it.
> 
>> And it got that way fast, I remember doing an install on some
>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>> worries, you could just move around with the keyboard.  X11 came up
>> as part of the install, the entire install was graphical and
>> seamless.
> 
> The FreeBSD installer *does* install X if you select it.
> 
>> FreeBSD is stuck in the 1990's in terms of user interface.
> 
> You're still talking about the installer, aren't you?  The normal user
> interface is via the shell, which hasn't changed, and for a good
> reason.
> 
>> They've done some good stuff in the kernel but it's not an end user
>> system,
> 
> There I have to agree with you.  A little TLC would go a long way.
> But I hope that you're not advocating the "change your GUI with your
> underwear" attitude that Microsoft, Apple and many Linux distros
> have.  One of the reasons I don't use Linux is because every time I
> try, the interface has changed.
> 
> Greg
> --
> Sent from my desktop computer.
> Finger grog@lemis.com for PGP public key.
> See complete headers for address and phone numbers.
> This message is digitally signed.  If your Microsoft mail program
> reports problems, please read http://lemis.com/broken-MUA

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  1:47                     ` Will Senn
@ 2021-01-31  2:25                       ` Larry McVoy
  2021-01-31  2:52                         ` Will Senn
                                           ` (3 more replies)
  0 siblings, 4 replies; 70+ messages in thread
From: Larry McVoy @ 2021-01-31  2:25 UTC (permalink / raw)
  To: Will Senn; +Cc: The Eunuchs Hysterical Society

If you like ZFS you don't understand operating systems design.  I do.
Jeff Bonwick was a stats student at Stanford when he took my OS class,
I convinced him to come to Sun.  Bill Moore worked for me.  That's the two
main ZFS guys and I thought I had taught them well but they let me down.

ZFS doesn't use the page cache, they said it was too hard because ZFS
is compressed.  A typical file system just has block numbers, a compressed
one needs another int per block, it's the int that says these many bytes
are a block uncompressed.  It's not that hard, it is 2 ints instead of 1.

In case I'm not being clear, the page cache is what everyone else uses
but ZFS has its own cache.  So if you want to mmap() a ZFS file, ZFS
has to bcopy() the data into the page cache and then spend a shit ton
of code to make sure that the page cache data is in sync with the ZFS
cache data.

SunOS came from BSD but SunOS added mmap.  Which had the same problem,
the BSD buffer cache was exactly the same as the ZFS cache, Sun spent 
years of effort to get rid of the buffer cache, everything is in the 
page cache.  So ZFS was a HUGE step backwards in systems design.  Might
be the best file system ever (it is not) but it was not a good player
in the OS world.

Those guys said that it was too hard to make a compressed file fit in
the page cache.  BitKeeper has that code and proves that it can be done.
Be happy to walk anyone who cares through that code, I didn't write that,
Wayne Scott did, but it's some of the best written code I've ever seen.
Up there with Mojo's work on the SunOS VM system.  (I'll bet that noone
takes me up on this offer, people love to argue but most don't want to
learn.  Prove me wrong, please).

So good on you that you like ZFS and FreeBSD.  I don't and I don't for
really good reasons.

Let's try it this way.  Get back to me when you can show me 40 people 
who have installed FreeBSD on their own, with no help.  In the same 
time, I can show you 40,000 people who have installed Linux on their
own, with no help.  Probably 400,000.

Technology is great, ease of use is what gets you users.  ZFS is
great but doesn't play nice with the OS.

That's my oh brother.

On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up for talking up FBSD. I use it in my classes, too and the system is much more coherent for my systems programming classes than linux.
> 
> Will
> 
> Sent from my iPhone
> 
> > On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com> wrote:
> > 
> >> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
> >>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
> >>> If I could get the day-2-day
> >>> applications that I need to work on FreeBSD, I suspect I would be there in
> >>> a heartbeat.
> >> 
> >> I dunno about that.  I tried out FreeBSD a couple of years ago when
> >> Netflix was flirting with me.  The installer hasn't seen any loving in
> >> 30 years it would seem.  The disk setup tool sucks just as bad as it
> >> did back in 1988.
> > 
> > You could be right there, for some value of 1988 (FreeBSD came into
> > being in 1992).  The tools work without being good.  But how often do
> > you use them?  I've been using FreeBSD since the beginning, and I
> > can't recall when I last used the disk partitioning tool, though I'm
> > sure that when I did I overrode a lot of (all?) the suggestions.
> > 
> >> I remember when Linux was this bad in the .90ish releases.  A long
> >> time ago.  Now their install is painless, it's every bit as good as
> >> Windows and maybe better.
> > 
> > FWIW, I find Microsoft "Windows" installation terminally confusing
> > (that's what you were talking about, right?).  And I've run into
> > serious problems with various Linux installations too.  That doesn't
> > make the FreeBSD tools better, but maybe it relativizes it.
> > 
> >> And it got that way fast, I remember doing an install on some
> >> machine around 1998 or 1999, I didn't have a mouse plugged in, no
> >> worries, you could just move around with the keyboard.  X11 came up
> >> as part of the install, the entire install was graphical and
> >> seamless.
> > 
> > The FreeBSD installer *does* install X if you select it.
> > 
> >> FreeBSD is stuck in the 1990's in terms of user interface.
> > 
> > You're still talking about the installer, aren't you?  The normal user
> > interface is via the shell, which hasn't changed, and for a good
> > reason.
> > 
> >> They've done some good stuff in the kernel but it's not an end user
> >> system,
> > 
> > There I have to agree with you.  A little TLC would go a long way.
> > But I hope that you're not advocating the "change your GUI with your
> > underwear" attitude that Microsoft, Apple and many Linux distros
> > have.  One of the reasons I don't use Linux is because every time I
> > try, the interface has changed.
> > 
> > Greg
> > --
> > Sent from my desktop computer.
> > Finger grog@lemis.com for PGP public key.
> > See complete headers for address and phone numbers.
> > This message is digitally signed.  If your Microsoft mail program
> > reports problems, please read http://lemis.com/broken-MUA

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  2:25                       ` Larry McVoy
@ 2021-01-31  2:52                         ` Will Senn
  2021-01-31  3:00                           ` Larry McVoy
  2021-02-04  5:43                         ` Dave Horsfall
                                           ` (2 subsequent siblings)
  3 siblings, 1 reply; 70+ messages in thread
From: Will Senn @ 2021-01-31  2:52 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society

Ha. Zfs may not be the be all and end all, but like I said, it’s never failed me. Whereas extX and btrfs, and, and, and have many times. Please don’t denigrate my knowledge, as so far as I know, we’ve never met, and nothing I said warrants such. The installer reminds me of Redhat’s old anaconda installer, I’ll grant you it’s dated. However, I typically install a new linux distro every week and there are many, many installers that are far more confusing - Open Suse and Fedora are two that come to mind, Debian as well. I would hazard to guess your favorite Linux is based on a distro that lacks a decent installer (Ubuntu and Mint are Debian based). 

Will

Sent from my iPhone

> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> If you like ZFS you don't understand operating systems design.  I do.
> Jeff Bonwick was a stats student at Stanford when he took my OS class,
> I convinced him to come to Sun.  Bill Moore worked for me.  That's the two
> main ZFS guys and I thought I had taught them well but they let me down.
> 
> ZFS doesn't use the page cache, they said it was too hard because ZFS
> is compressed.  A typical file system just has block numbers, a compressed
> one needs another int per block, it's the int that says these many bytes
> are a block uncompressed.  It's not that hard, it is 2 ints instead of 1.
> 
> In case I'm not being clear, the page cache is what everyone else uses
> but ZFS has its own cache.  So if you want to mmap() a ZFS file, ZFS
> has to bcopy() the data into the page cache and then spend a shit ton
> of code to make sure that the page cache data is in sync with the ZFS
> cache data.
> 
> SunOS came from BSD but SunOS added mmap.  Which had the same problem,
> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent 
> years of effort to get rid of the buffer cache, everything is in the 
> page cache.  So ZFS was a HUGE step backwards in systems design.  Might
> be the best file system ever (it is not) but it was not a good player
> in the OS world.
> 
> Those guys said that it was too hard to make a compressed file fit in
> the page cache.  BitKeeper has that code and proves that it can be done.
> Be happy to walk anyone who cares through that code, I didn't write that,
> Wayne Scott did, but it's some of the best written code I've ever seen.
> Up there with Mojo's work on the SunOS VM system.  (I'll bet that noone
> takes me up on this offer, people love to argue but most don't want to
> learn.  Prove me wrong, please).
> 
> So good on you that you like ZFS and FreeBSD.  I don't and I don't for
> really good reasons.
> 
> Let's try it this way.  Get back to me when you can show me 40 people 
> who have installed FreeBSD on their own, with no help.  In the same 
> time, I can show you 40,000 people who have installed Linux on their
> own, with no help.  Probably 400,000.
> 
> Technology is great, ease of use is what gets you users.  ZFS is
> great but doesn't play nice with the OS.
> 
> That's my oh brother.
> 
>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
>> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up for talking up FBSD. I use it in my classes, too and the system is much more coherent for my systems programming classes than linux.
>> 
>> Will
>> 
>> Sent from my iPhone
>> 
>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com> wrote:
>>>> 
>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>>>> If I could get the day-2-day
>>>>> applications that I need to work on FreeBSD, I suspect I would be there in
>>>>> a heartbeat.
>>>> 
>>>> I dunno about that.  I tried out FreeBSD a couple of years ago when
>>>> Netflix was flirting with me.  The installer hasn't seen any loving in
>>>> 30 years it would seem.  The disk setup tool sucks just as bad as it
>>>> did back in 1988.
>>> 
>>> You could be right there, for some value of 1988 (FreeBSD came into
>>> being in 1992).  The tools work without being good.  But how often do
>>> you use them?  I've been using FreeBSD since the beginning, and I
>>> can't recall when I last used the disk partitioning tool, though I'm
>>> sure that when I did I overrode a lot of (all?) the suggestions.
>>> 
>>>> I remember when Linux was this bad in the .90ish releases.  A long
>>>> time ago.  Now their install is painless, it's every bit as good as
>>>> Windows and maybe better.
>>> 
>>> FWIW, I find Microsoft "Windows" installation terminally confusing
>>> (that's what you were talking about, right?).  And I've run into
>>> serious problems with various Linux installations too.  That doesn't
>>> make the FreeBSD tools better, but maybe it relativizes it.
>>> 
>>>> And it got that way fast, I remember doing an install on some
>>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>>>> worries, you could just move around with the keyboard.  X11 came up
>>>> as part of the install, the entire install was graphical and
>>>> seamless.
>>> 
>>> The FreeBSD installer *does* install X if you select it.
>>> 
>>>> FreeBSD is stuck in the 1990's in terms of user interface.
>>> 
>>> You're still talking about the installer, aren't you?  The normal user
>>> interface is via the shell, which hasn't changed, and for a good
>>> reason.
>>> 
>>>> They've done some good stuff in the kernel but it's not an end user
>>>> system,
>>> 
>>> There I have to agree with you.  A little TLC would go a long way.
>>> But I hope that you're not advocating the "change your GUI with your
>>> underwear" attitude that Microsoft, Apple and many Linux distros
>>> have.  One of the reasons I don't use Linux is because every time I
>>> try, the interface has changed.
>>> 
>>> Greg
>>> --
>>> Sent from my desktop computer.
>>> Finger grog@lemis.com for PGP public key.
>>> See complete headers for address and phone numbers.
>>> This message is digitally signed.  If your Microsoft mail program
>>> reports problems, please read http://lemis.com/broken-MUA
> 
> -- 
> ---
> Larry McVoy                     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  2:52                         ` Will Senn
@ 2021-01-31  3:00                           ` Larry McVoy
  2021-01-31  3:06                             ` Will Senn
  0 siblings, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-01-31  3:00 UTC (permalink / raw)
  To: Will Senn; +Cc: The Eunuchs Hysterical Society

I think this has gone on along enough.  Don't want it to be personal, not
my intent.

Like I said it seems like a Linux vs FreeBSD thing.  Don't want that.
You can search the archives about Ted talking about how ext was not
all that (I'm a fan).

On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will Senn wrote:
> Ha. Zfs may not be the be all and end all, but like I said, it???s never failed me. Whereas extX and btrfs, and, and, and have many times. Please don???t denigrate my knowledge, as so far as I know, we???ve never met, and nothing I said warrants such. The installer reminds me of Redhat???s old anaconda installer, I???ll grant you it???s dated. However, I typically install a new linux distro every week and there are many, many installers that are far more confusing - Open Suse and Fedora are two that come to mind, Debian as well. I would hazard to guess your favorite Linux is based on a distro that lacks a decent installer (Ubuntu and Mint are Debian based). 
> 
> Will
> 
> Sent from my iPhone
> 
> > On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > 
> > If you like ZFS you don't understand operating systems design.  I do.
> > Jeff Bonwick was a stats student at Stanford when he took my OS class,
> > I convinced him to come to Sun.  Bill Moore worked for me.  That's the two
> > main ZFS guys and I thought I had taught them well but they let me down.
> > 
> > ZFS doesn't use the page cache, they said it was too hard because ZFS
> > is compressed.  A typical file system just has block numbers, a compressed
> > one needs another int per block, it's the int that says these many bytes
> > are a block uncompressed.  It's not that hard, it is 2 ints instead of 1.
> > 
> > In case I'm not being clear, the page cache is what everyone else uses
> > but ZFS has its own cache.  So if you want to mmap() a ZFS file, ZFS
> > has to bcopy() the data into the page cache and then spend a shit ton
> > of code to make sure that the page cache data is in sync with the ZFS
> > cache data.
> > 
> > SunOS came from BSD but SunOS added mmap.  Which had the same problem,
> > the BSD buffer cache was exactly the same as the ZFS cache, Sun spent 
> > years of effort to get rid of the buffer cache, everything is in the 
> > page cache.  So ZFS was a HUGE step backwards in systems design.  Might
> > be the best file system ever (it is not) but it was not a good player
> > in the OS world.
> > 
> > Those guys said that it was too hard to make a compressed file fit in
> > the page cache.  BitKeeper has that code and proves that it can be done.
> > Be happy to walk anyone who cares through that code, I didn't write that,
> > Wayne Scott did, but it's some of the best written code I've ever seen.
> > Up there with Mojo's work on the SunOS VM system.  (I'll bet that noone
> > takes me up on this offer, people love to argue but most don't want to
> > learn.  Prove me wrong, please).
> > 
> > So good on you that you like ZFS and FreeBSD.  I don't and I don't for
> > really good reasons.
> > 
> > Let's try it this way.  Get back to me when you can show me 40 people 
> > who have installed FreeBSD on their own, with no help.  In the same 
> > time, I can show you 40,000 people who have installed Linux on their
> > own, with no help.  Probably 400,000.
> > 
> > Technology is great, ease of use is what gets you users.  ZFS is
> > great but doesn't play nice with the OS.
> > 
> > That's my oh brother.
> > 
> >> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
> >> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up for talking up FBSD. I use it in my classes, too and the system is much more coherent for my systems programming classes than linux.
> >> 
> >> Will
> >> 
> >> Sent from my iPhone
> >> 
> >>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com> wrote:
> >>>> 
> >>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
> >>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
> >>>>> If I could get the day-2-day
> >>>>> applications that I need to work on FreeBSD, I suspect I would be there in
> >>>>> a heartbeat.
> >>>> 
> >>>> I dunno about that.  I tried out FreeBSD a couple of years ago when
> >>>> Netflix was flirting with me.  The installer hasn't seen any loving in
> >>>> 30 years it would seem.  The disk setup tool sucks just as bad as it
> >>>> did back in 1988.
> >>> 
> >>> You could be right there, for some value of 1988 (FreeBSD came into
> >>> being in 1992).  The tools work without being good.  But how often do
> >>> you use them?  I've been using FreeBSD since the beginning, and I
> >>> can't recall when I last used the disk partitioning tool, though I'm
> >>> sure that when I did I overrode a lot of (all?) the suggestions.
> >>> 
> >>>> I remember when Linux was this bad in the .90ish releases.  A long
> >>>> time ago.  Now their install is painless, it's every bit as good as
> >>>> Windows and maybe better.
> >>> 
> >>> FWIW, I find Microsoft "Windows" installation terminally confusing
> >>> (that's what you were talking about, right?).  And I've run into
> >>> serious problems with various Linux installations too.  That doesn't
> >>> make the FreeBSD tools better, but maybe it relativizes it.
> >>> 
> >>>> And it got that way fast, I remember doing an install on some
> >>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
> >>>> worries, you could just move around with the keyboard.  X11 came up
> >>>> as part of the install, the entire install was graphical and
> >>>> seamless.
> >>> 
> >>> The FreeBSD installer *does* install X if you select it.
> >>> 
> >>>> FreeBSD is stuck in the 1990's in terms of user interface.
> >>> 
> >>> You're still talking about the installer, aren't you?  The normal user
> >>> interface is via the shell, which hasn't changed, and for a good
> >>> reason.
> >>> 
> >>>> They've done some good stuff in the kernel but it's not an end user
> >>>> system,
> >>> 
> >>> There I have to agree with you.  A little TLC would go a long way.
> >>> But I hope that you're not advocating the "change your GUI with your
> >>> underwear" attitude that Microsoft, Apple and many Linux distros
> >>> have.  One of the reasons I don't use Linux is because every time I
> >>> try, the interface has changed.
> >>> 
> >>> Greg
> >>> --
> >>> Sent from my desktop computer.
> >>> Finger grog@lemis.com for PGP public key.
> >>> See complete headers for address and phone numbers.
> >>> This message is digitally signed.  If your Microsoft mail program
> >>> reports problems, please read http://lemis.com/broken-MUA
> > 
> > -- 
> > ---
> > Larry McVoy                     lm at mcvoy.com             http://www.mcvoy.com/lm 

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  3:00                           ` Larry McVoy
@ 2021-01-31  3:06                             ` Will Senn
  2021-01-31  3:32                               ` John Cowan
  0 siblings, 1 reply; 70+ messages in thread
From: Will Senn @ 2021-01-31  3:06 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society

No worries. Maybe it’s time for a vi emacs discussion... totally kidding :)

Sent from my iPhone

> On Jan 30, 2021, at 9:00 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> I think this has gone on along enough.  Don't want it to be personal, not
> my intent.
> 
> Like I said it seems like a Linux vs FreeBSD thing.  Don't want that.
> You can search the archives about Ted talking about how ext was not
> all that (I'm a fan).
> 
>> On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will Senn wrote:
>> Ha. Zfs may not be the be all and end all, but like I said, it???s never failed me. Whereas extX and btrfs, and, and, and have many times. Please don???t denigrate my knowledge, as so far as I know, we???ve never met, and nothing I said warrants such. The installer reminds me of Redhat???s old anaconda installer, I???ll grant you it???s dated. However, I typically install a new linux distro every week and there are many, many installers that are far more confusing - Open Suse and Fedora are two that come to mind, Debian as well. I would hazard to guess your favorite Linux is based on a distro that lacks a decent installer (Ubuntu and Mint are Debian based). 
>> 
>> Will
>> 
>> Sent from my iPhone
>> 
>>> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm@mcvoy.com> wrote:
>>> 
>>> If you like ZFS you don't understand operating systems design.  I do.
>>> Jeff Bonwick was a stats student at Stanford when he took my OS class,
>>> I convinced him to come to Sun.  Bill Moore worked for me.  That's the two
>>> main ZFS guys and I thought I had taught them well but they let me down.
>>> 
>>> ZFS doesn't use the page cache, they said it was too hard because ZFS
>>> is compressed.  A typical file system just has block numbers, a compressed
>>> one needs another int per block, it's the int that says these many bytes
>>> are a block uncompressed.  It's not that hard, it is 2 ints instead of 1.
>>> 
>>> In case I'm not being clear, the page cache is what everyone else uses
>>> but ZFS has its own cache.  So if you want to mmap() a ZFS file, ZFS
>>> has to bcopy() the data into the page cache and then spend a shit ton
>>> of code to make sure that the page cache data is in sync with the ZFS
>>> cache data.
>>> 
>>> SunOS came from BSD but SunOS added mmap.  Which had the same problem,
>>> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent 
>>> years of effort to get rid of the buffer cache, everything is in the 
>>> page cache.  So ZFS was a HUGE step backwards in systems design.  Might
>>> be the best file system ever (it is not) but it was not a good player
>>> in the OS world.
>>> 
>>> Those guys said that it was too hard to make a compressed file fit in
>>> the page cache.  BitKeeper has that code and proves that it can be done.
>>> Be happy to walk anyone who cares through that code, I didn't write that,
>>> Wayne Scott did, but it's some of the best written code I've ever seen.
>>> Up there with Mojo's work on the SunOS VM system.  (I'll bet that noone
>>> takes me up on this offer, people love to argue but most don't want to
>>> learn.  Prove me wrong, please).
>>> 
>>> So good on you that you like ZFS and FreeBSD.  I don't and I don't for
>>> really good reasons.
>>> 
>>> Let's try it this way.  Get back to me when you can show me 40 people 
>>> who have installed FreeBSD on their own, with no help.  In the same 
>>> time, I can show you 40,000 people who have installed Linux on their
>>> own, with no help.  Probably 400,000.
>>> 
>>> Technology is great, ease of use is what gets you users.  ZFS is
>>> great but doesn't play nice with the OS.
>>> 
>>> That's my oh brother.
>>> 
>>>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
>>>> Oh brother. I use FreeBSD all the time. I prefer it for its stability and ZFS which has NEVER let me down and I???ve done my share of stupid user error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages, but uptime on my fbsd instance is 10x any of my Linux instances. We are soooo off topic, I think :). But, I???m always up for talking up FBSD. I use it in my classes, too and the system is much more coherent for my systems programming classes than linux.
>>>> 
>>>> Will
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com> wrote:
>>>>>>> 
>>>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
>>>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
>>>>>>> If I could get the day-2-day
>>>>>>> applications that I need to work on FreeBSD, I suspect I would be there in
>>>>>>> a heartbeat.
>>>>>> 
>>>>>> I dunno about that.  I tried out FreeBSD a couple of years ago when
>>>>>> Netflix was flirting with me.  The installer hasn't seen any loving in
>>>>>> 30 years it would seem.  The disk setup tool sucks just as bad as it
>>>>>> did back in 1988.
>>>>> 
>>>>> You could be right there, for some value of 1988 (FreeBSD came into
>>>>> being in 1992).  The tools work without being good.  But how often do
>>>>> you use them?  I've been using FreeBSD since the beginning, and I
>>>>> can't recall when I last used the disk partitioning tool, though I'm
>>>>> sure that when I did I overrode a lot of (all?) the suggestions.
>>>>> 
>>>>>> I remember when Linux was this bad in the .90ish releases.  A long
>>>>>> time ago.  Now their install is painless, it's every bit as good as
>>>>>> Windows and maybe better.
>>>>> 
>>>>> FWIW, I find Microsoft "Windows" installation terminally confusing
>>>>> (that's what you were talking about, right?).  And I've run into
>>>>> serious problems with various Linux installations too.  That doesn't
>>>>> make the FreeBSD tools better, but maybe it relativizes it.
>>>>> 
>>>>>> And it got that way fast, I remember doing an install on some
>>>>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
>>>>>> worries, you could just move around with the keyboard.  X11 came up
>>>>>> as part of the install, the entire install was graphical and
>>>>>> seamless.
>>>>> 
>>>>> The FreeBSD installer *does* install X if you select it.
>>>>> 
>>>>>> FreeBSD is stuck in the 1990's in terms of user interface.
>>>>> 
>>>>> You're still talking about the installer, aren't you?  The normal user
>>>>> interface is via the shell, which hasn't changed, and for a good
>>>>> reason.
>>>>> 
>>>>>> They've done some good stuff in the kernel but it's not an end user
>>>>>> system,
>>>>> 
>>>>> There I have to agree with you.  A little TLC would go a long way.
>>>>> But I hope that you're not advocating the "change your GUI with your
>>>>> underwear" attitude that Microsoft, Apple and many Linux distros
>>>>> have.  One of the reasons I don't use Linux is because every time I
>>>>> try, the interface has changed.
>>>>> 
>>>>> Greg
>>>>> --
>>>>> Sent from my desktop computer.
>>>>> Finger grog@lemis.com for PGP public key.
>>>>> See complete headers for address and phone numbers.
>>>>> This message is digitally signed.  If your Microsoft mail program
>>>>> reports problems, please read http://lemis.com/broken-MUA
>>> 
>>> -- 
>>> ---
>>> Larry McVoy                     lm at mcvoy.com             http://www.mcvoy.com/lm 
> 
> -- 
> ---
> Larry McVoy                     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  3:06                             ` Will Senn
@ 2021-01-31  3:32                               ` John Cowan
  0 siblings, 0 replies; 70+ messages in thread
From: John Cowan @ 2021-01-31  3:32 UTC (permalink / raw)
  To: Will Senn; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 7985 bytes --]

Since, I am an ex(1) troglodyte, I'll be happy to umpire.  (I know ed(1) is
the standard editor, but I'm willing to trade a little standardosity for
more convenience.)

On Sat, Jan 30, 2021 at 10:06 PM Will Senn <will.senn@gmail.com> wrote:

> No worries. Maybe it’s time for a vi emacs discussion... totally kidding :)
>
> Sent from my iPhone
>
> > On Jan 30, 2021, at 9:00 PM, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > I think this has gone on along enough.  Don't want it to be personal, not
> > my intent.
> >
> > Like I said it seems like a Linux vs FreeBSD thing.  Don't want that.
> > You can search the archives about Ted talking about how ext was not
> > all that (I'm a fan).
> >
> >> On Sat, Jan 30, 2021 at 08:52:09PM -0600, Will Senn wrote:
> >> Ha. Zfs may not be the be all and end all, but like I said, it???s
> never failed me. Whereas extX and btrfs, and, and, and have many times.
> Please don???t denigrate my knowledge, as so far as I know, we???ve never
> met, and nothing I said warrants such. The installer reminds me of
> Redhat???s old anaconda installer, I???ll grant you it???s dated. However,
> I typically install a new linux distro every week and there are many, many
> installers that are far more confusing - Open Suse and Fedora are two that
> come to mind, Debian as well. I would hazard to guess your favorite Linux
> is based on a distro that lacks a decent installer (Ubuntu and Mint are
> Debian based).
> >>
> >> Will
> >>
> >> Sent from my iPhone
> >>
> >>> On Jan 30, 2021, at 8:25 PM, Larry McVoy <lm@mcvoy.com> wrote:
> >>>
> >>> If you like ZFS you don't understand operating systems design.  I do.
> >>> Jeff Bonwick was a stats student at Stanford when he took my OS class,
> >>> I convinced him to come to Sun.  Bill Moore worked for me.  That's the
> two
> >>> main ZFS guys and I thought I had taught them well but they let me
> down.
> >>>
> >>> ZFS doesn't use the page cache, they said it was too hard because ZFS
> >>> is compressed.  A typical file system just has block numbers, a
> compressed
> >>> one needs another int per block, it's the int that says these many
> bytes
> >>> are a block uncompressed.  It's not that hard, it is 2 ints instead of
> 1.
> >>>
> >>> In case I'm not being clear, the page cache is what everyone else uses
> >>> but ZFS has its own cache.  So if you want to mmap() a ZFS file, ZFS
> >>> has to bcopy() the data into the page cache and then spend a shit ton
> >>> of code to make sure that the page cache data is in sync with the ZFS
> >>> cache data.
> >>>
> >>> SunOS came from BSD but SunOS added mmap.  Which had the same problem,
> >>> the BSD buffer cache was exactly the same as the ZFS cache, Sun spent
> >>> years of effort to get rid of the buffer cache, everything is in the
> >>> page cache.  So ZFS was a HUGE step backwards in systems design.  Might
> >>> be the best file system ever (it is not) but it was not a good player
> >>> in the OS world.
> >>>
> >>> Those guys said that it was too hard to make a compressed file fit in
> >>> the page cache.  BitKeeper has that code and proves that it can be
> done.
> >>> Be happy to walk anyone who cares through that code, I didn't write
> that,
> >>> Wayne Scott did, but it's some of the best written code I've ever seen.
> >>> Up there with Mojo's work on the SunOS VM system.  (I'll bet that noone
> >>> takes me up on this offer, people love to argue but most don't want to
> >>> learn.  Prove me wrong, please).
> >>>
> >>> So good on you that you like ZFS and FreeBSD.  I don't and I don't for
> >>> really good reasons.
> >>>
> >>> Let's try it this way.  Get back to me when you can show me 40 people
> >>> who have installed FreeBSD on their own, with no help.  In the same
> >>> time, I can show you 40,000 people who have installed Linux on their
> >>> own, with no help.  Probably 400,000.
> >>>
> >>> Technology is great, ease of use is what gets you users.  ZFS is
> >>> great but doesn't play nice with the OS.
> >>>
> >>> That's my oh brother.
> >>>
> >>>> On Sat, Jan 30, 2021 at 07:47:41PM -0600, Will Senn wrote:
> >>>> Oh brother. I use FreeBSD all the time. I prefer it for its stability
> and ZFS which has NEVER let me down and I???ve done my share of stupid user
> error. Now that Linux has ZFS, it doesn???t seem as stuck in the dark ages,
> but uptime on my fbsd instance is 10x any of my Linux instances. We are
> soooo off topic, I think :). But, I???m always up for talking up FBSD. I
> use it in my classes, too and the system is much more coherent for my
> systems programming classes than linux.
> >>>>
> >>>> Will
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>>>> On Jan 30, 2021, at 5:11 PM, Greg 'groggy' Lehey <grog@lemis.com>
> wrote:
> >>>>>>>
> >>>>>>> On Saturday, 30 January 2021 at 14:28:54 -0800, Larry McVoy wrote:
> >>>>>>> On Sat, Jan 30, 2021 at 04:28:26PM -0500, Clem Cole wrote:
> >>>>>>> If I could get the day-2-day
> >>>>>>> applications that I need to work on FreeBSD, I suspect I would be
> there in
> >>>>>>> a heartbeat.
> >>>>>>
> >>>>>> I dunno about that.  I tried out FreeBSD a couple of years ago when
> >>>>>> Netflix was flirting with me.  The installer hasn't seen any loving
> in
> >>>>>> 30 years it would seem.  The disk setup tool sucks just as bad as it
> >>>>>> did back in 1988.
> >>>>>
> >>>>> You could be right there, for some value of 1988 (FreeBSD came into
> >>>>> being in 1992).  The tools work without being good.  But how often do
> >>>>> you use them?  I've been using FreeBSD since the beginning, and I
> >>>>> can't recall when I last used the disk partitioning tool, though I'm
> >>>>> sure that when I did I overrode a lot of (all?) the suggestions.
> >>>>>
> >>>>>> I remember when Linux was this bad in the .90ish releases.  A long
> >>>>>> time ago.  Now their install is painless, it's every bit as good as
> >>>>>> Windows and maybe better.
> >>>>>
> >>>>> FWIW, I find Microsoft "Windows" installation terminally confusing
> >>>>> (that's what you were talking about, right?).  And I've run into
> >>>>> serious problems with various Linux installations too.  That doesn't
> >>>>> make the FreeBSD tools better, but maybe it relativizes it.
> >>>>>
> >>>>>> And it got that way fast, I remember doing an install on some
> >>>>>> machine around 1998 or 1999, I didn't have a mouse plugged in, no
> >>>>>> worries, you could just move around with the keyboard.  X11 came up
> >>>>>> as part of the install, the entire install was graphical and
> >>>>>> seamless.
> >>>>>
> >>>>> The FreeBSD installer *does* install X if you select it.
> >>>>>
> >>>>>> FreeBSD is stuck in the 1990's in terms of user interface.
> >>>>>
> >>>>> You're still talking about the installer, aren't you?  The normal
> user
> >>>>> interface is via the shell, which hasn't changed, and for a good
> >>>>> reason.
> >>>>>
> >>>>>> They've done some good stuff in the kernel but it's not an end user
> >>>>>> system,
> >>>>>
> >>>>> There I have to agree with you.  A little TLC would go a long way.
> >>>>> But I hope that you're not advocating the "change your GUI with your
> >>>>> underwear" attitude that Microsoft, Apple and many Linux distros
> >>>>> have.  One of the reasons I don't use Linux is because every time I
> >>>>> try, the interface has changed.
> >>>>>
> >>>>> Greg
> >>>>> --
> >>>>> Sent from my desktop computer.
> >>>>> Finger grog@lemis.com for PGP public key.
> >>>>> See complete headers for address and phone numbers.
> >>>>> This message is digitally signed.  If your Microsoft mail program
> >>>>> reports problems, please read http://lemis.com/broken-MUA
> >>>
> >>> --
> >>> ---
> >>> Larry McVoy                     lm at mcvoy.com
> http://www.mcvoy.com/lm
> >
> > --
> > ---
> > Larry McVoy                     lm at mcvoy.com
> http://www.mcvoy.com/lm
>

[-- Attachment #2: Type: text/html, Size: 11005 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] [SPAM] Re: FreeBSD behind the times?
  2021-01-30 23:54                             ` Larry McVoy
@ 2021-01-31 12:23                               ` Dermot Tynan
  0 siblings, 0 replies; 70+ messages in thread
From: Dermot Tynan @ 2021-01-31 12:23 UTC (permalink / raw)
  To: tuhs

On 30/01/2021 23:54, Larry McVoy wrote:
> You like FreeBSD, cool, it's fine. Nobody will ever know because nobody
> is going to put up with a 1990's era installer.  Nobody.  It's 2020, shit
> is just supposed to work, it's supposed do so in a graphical environment
> let works with or without a mouse.  Anyone who doesn't see that isn't
> going to get any market share.

It's "horses for courses". I run a mix of FreeBSD and Linux servers. As 
they're headless, the last thing I want is X on there. Ideally, I'd 
strip out the automount crap as well, to say nothing of the horror that 
is systemd. But yeah, those people with desktops want a simple GUI 
install and they want their USB stick to appear as a "folder". I run 
Ubuntu on my desktop for vaguely those reasons (and Dropbox!).

I think the biggest threat to FreeBSD is docker/k8s and things like 
libvirt. Unfortunately, most open source (and closed source) software 
runs de facto on Linux (if it runs on Unix-esque systems), and the ports 
to FreeBSD are slow in coming. Last time I looked, there was no native 
Dropbox client.

As a server though, if you don't need to use it as a hypervisor or a 
container system, it gets my vote. But again it's down to preference. 
Linux has the breeze now, and is streaking ahead of the *BSDs. Sad, but 
true.
                         - Der

-- 
Dermot Tynan
dtynan@kalopa.com


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  2:25                       ` Larry McVoy
  2021-01-31  2:52                         ` Will Senn
@ 2021-02-04  5:43                         ` Dave Horsfall
  2021-02-04  6:10                           ` Angus Robinson
  2021-02-04 15:45                           ` Will Senn
  2021-02-04  7:46                         ` Chris Torek
  2021-02-11 21:01                         ` Angel M Alganza
  3 siblings, 2 replies; 70+ messages in thread
From: Dave Horsfall @ 2021-02-04  5:43 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, 30 Jan 2021, Larry McVoy wrote:

[ Usual insightful...  insights ]

> If you like ZFS you don't understand operating systems design.  I do.

Indeed...

> Jeff Bonwick was a stats student at Stanford when he took my OS class, I 
> convinced him to come to Sun.  Bill Moore worked for me.  That's the two 
> main ZFS guys and I thought I had taught them well but they let me down.

{ ... ]

There's no way that I'd use ZFS; lose a block in an ordinary file, well, 
you now have a hole (but not in the file-system sense); lose a block in a 
compressed system, well...

Or perhaps I'm becoming conservative in my old age; I remember when I once 
rewrote utilities that when writing a zero block merely did a seek instead 
(or something like that; you had to remember to actually write out the 
last block).  I wouldn't try it these days, as Unix file systems were 
simple back then.

[ ... ]

> Let's try it this way.  Get back to me when you can show me 40 people 
> who have installed FreeBSD on their own, with no help.  In the same 
> time, I can show you 40,000 people who have installed Linux on their 
> own, with no help.  Probably 400,000.

Well, I did (but without ZFS) on several boxes, with zero help.  Having
had SunOS experience (4.4 was the best) helped :-)

I can't stand Penguin/OS; it looks too much like Windoze for my liking
(and does its best to be almost-Unix-but-not-quite).

-- Dave, a grey-beard

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04  5:43                         ` Dave Horsfall
@ 2021-02-04  6:10                           ` Angus Robinson
  2021-02-04  7:46                             ` Andy Kosela
  2021-02-04 22:25                             ` Dave Horsfall
  2021-02-04 15:45                           ` Will Senn
  1 sibling, 2 replies; 70+ messages in thread
From: Angus Robinson @ 2021-02-04  6:10 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]

Not entirely sure as it's been a while since iI have used it but last time
I remember TrueOS which used FreeBSD was easy to use for a newbie. There
are other FreeBSD distro's like GhostBSD,etc that have an installer like
Linux.

My 2c is that FreeBSD is not trying to get people in that are newbies, it's
for the server environment and it works extremely well.

Kind Regards,
Angus Robinson


On Thu, Feb 4, 2021 at 7:45 AM Dave Horsfall <dave@horsfall.org> wrote:

> On Sat, 30 Jan 2021, Larry McVoy wrote:
>
> [ Usual insightful...  insights ]
>
> > If you like ZFS you don't understand operating systems design.  I do.
>
> Indeed...
>
> > Jeff Bonwick was a stats student at Stanford when he took my OS class, I
> > convinced him to come to Sun.  Bill Moore worked for me.  That's the two
> > main ZFS guys and I thought I had taught them well but they let me down.
>
> { ... ]
>
> There's no way that I'd use ZFS; lose a block in an ordinary file, well,
> you now have a hole (but not in the file-system sense); lose a block in a
> compressed system, well...
>
> Or perhaps I'm becoming conservative in my old age; I remember when I once
> rewrote utilities that when writing a zero block merely did a seek instead
> (or something like that; you had to remember to actually write out the
> last block).  I wouldn't try it these days, as Unix file systems were
> simple back then.
>
> [ ... ]
>
> > Let's try it this way.  Get back to me when you can show me 40 people
> > who have installed FreeBSD on their own, with no help.  In the same
> > time, I can show you 40,000 people who have installed Linux on their
> > own, with no help.  Probably 400,000.
>
> Well, I did (but without ZFS) on several boxes, with zero help.  Having
> had SunOS experience (4.4 was the best) helped :-)
>
> I can't stand Penguin/OS; it looks too much like Windoze for my liking
> (and does its best to be almost-Unix-but-not-quite).
>
> -- Dave, a grey-beard
>

[-- Attachment #2: Type: text/html, Size: 2615 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  2:25                       ` Larry McVoy
  2021-01-31  2:52                         ` Will Senn
  2021-02-04  5:43                         ` Dave Horsfall
@ 2021-02-04  7:46                         ` Chris Torek
  2021-02-04 15:47                           ` Will Senn
  2021-02-11 21:01                         ` Angel M Alganza
  3 siblings, 1 reply; 70+ messages in thread
From: Chris Torek @ 2021-02-04  7:46 UTC (permalink / raw)
  To: lm, will.senn; +Cc: tuhs

For what it's worth, you don't *have* to use compression on ZFS.
But everything still goes through the ARC, which is ... messy.
And eats memory for breakfast and then more memory for snacks and
lunch and more snacks and so on.  Fortunately memory is cheap, if
you have a modern box.  Unfortunately, I still don't, yet.

ZFS has a ton of stuff in it.  That, also, is messy, and not a
great thing in terms of kernel size and security and alacrity and
so on.  But it has some really cool ideas in it.  I'm perfectly
happy to use it, or will be once I build a new box (still haven't
made the jump to an AMD system with ECC).

Chris

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04  6:10                           ` Angus Robinson
@ 2021-02-04  7:46                             ` Andy Kosela
  2021-02-04 22:25                             ` Dave Horsfall
  1 sibling, 0 replies; 70+ messages in thread
From: Andy Kosela @ 2021-02-04  7:46 UTC (permalink / raw)
  To: Angus Robinson; +Cc: The Eunuchs Hysterical Society

On 2/4/21, Angus Robinson <angus@fairhaven.za.net> wrote:
>  it's for the server environment and it works extremely well.
>

Well...20 years ago it was a catchy phrase  when FreeBSD was still in
stiff competition with Linux.  Now 20 years later we all see who won
the Unix server wars.  Literally every corporate entity is using some
form of commercial Red Hat Enterprise Linux now.  It is running
everywhere.

There are some individual companies that still utilizes FreeBSD like
Netflix, but the overall majority of server systems are based on
Linux.  I am not saying it is better; I am just pointing out the
facts.  I generally despise all modern Unix: be it Linux with systemd
or FreeBSD as being too bloated and complex.  My personal opinion is
that Linux peaked around late 90s-early/mid 2000s.  It was still
pretty logical and simple, but excelled in performance.  It went
downhill since then...

--Andy

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04  5:43                         ` Dave Horsfall
  2021-02-04  6:10                           ` Angus Robinson
@ 2021-02-04 15:45                           ` Will Senn
  2021-02-04 16:03                             ` Henry Bent
                                               ` (2 more replies)
  1 sibling, 3 replies; 70+ messages in thread
From: Will Senn @ 2021-02-04 15:45 UTC (permalink / raw)
  To: tuhs

On 2/3/21 11:43 PM, Dave Horsfall wrote:
> On Sat, 30 Jan 2021, Larry McVoy wrote:
>
> [ Usual insightful...  insights ]
>
>> If you like ZFS you don't understand operating systems design.  I do.
> ...
>
> There's no way that I'd use ZFS; lose a block in an ordinary file, 
> well, you now have a hole (but not in the file-system sense); lose a 
> block in a compressed system, well...
>
ZFS needn't be compressed, and I don't generally do compression or 
encryption unless required by law, so I can't speak from personal 
experience on those use cases (others, far more experienced can). I do 
know that it's truly a pain to recover from issues with either.

In response to the negative vibes around ZFS. I've never lost a file (or 
a piece of a file) in 10+ years of using ZFS. I get the feeling we may 
not be talking about the same ZFS. My experience is with the ZFS FreeBSD 
comes with, not the version that Oracle owns. Perhaps the info is a 
little out of date for the naysayers. In my experience, using ZFS is 
fairly transparent and simple to use - no partitioning to deal with, no 
need to worry about generating filesystems, none of that - add your 
disks to a pool, choose your RAID levels and it gets mounted, no fuss. 
I've lost plenty of disks along the way, but ZFS just keeps on chugging 
along nicely until I replace them and then rebuilds the arrays, again, 
no fuss other than replacing the hardware. In terms of massive system 
updates and such, I just snapshot the environment (a near instantaneous 
operation) before making significant changes to my system, that might 
break things and when they do break (and they do, more often than I'd 
like), I just rollback. man bectl. Painless (and I mean painless, 
hundreds of times, or mor). I'm sure it all sounds scifi, but it's my 
experience along with plenty of other folks, and this ZFS sucks thread 
seems to be FUD to me - ala Microsoft vs Linux, or at best informed 
hypothetical speculations - reminds me of an if statement conversation I 
had online in the early 1990's where one group of folks claimed that 
braces worked a certain way, based on the then current standard, and 
another group of folks (I'd be on this side of things), tested the 
theory with a host of compilers, observed the functions effects, shook 
their heads and wondered why it didn't match up with the theory, and 
said it worked another. Who was right? I'm still not entirely sure, from 
a philosophical perspective, but I have since coded my if statements 
according to my environment, not the standards.

As I mentioned in the prior thread, I've lost my share of files and file 
systems (many, many times since 1993 when I started with linux - 0.9 
kernel, slackware, then redhat, then debian, now mint) with ext3/4, and 
btrfs, though, and the only recovery was backup (a time intensive 
process). I really don't see the logic behind the negative arguments. 
Don't like it, fine, say it and live it. Claim it sucks? Then, back it 
up with a real-world, current experience and I'll cede the point - I'll 
keep using ZFS though :).

I want to be clear, I don't dislike Linux. I don't think FreeBSD is 
superior. I like both. I use both... daily. With enough prep and 
planning, my linux environment is similarly recoverable, but with 
freebsd, the prep and planning requires a lot less time and effort. 
Personally, I heart linux Mint - it's based on Debian and Ubuntu - is a 
straightforward install, works well, has zfs (not yet on boot), has 
timeshift (lovely piece of software), and can be quite pretty.

Vive la difference.

Will





^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04  7:46                         ` Chris Torek
@ 2021-02-04 15:47                           ` Will Senn
  0 siblings, 0 replies; 70+ messages in thread
From: Will Senn @ 2021-02-04 15:47 UTC (permalink / raw)
  To: Chris Torek, lm; +Cc: tuhs

On 2/4/21 1:46 AM, Chris Torek wrote:
> For what it's worth, you don't *have* to use compression on ZFS.
> But everything still goes through the ARC, which is ... messy.
> And eats memory for breakfast and then more memory for snacks and
> lunch and more snacks and so on.  Fortunately memory is cheap, if
> you have a modern box.  Unfortunately, I still don't, yet.
>
> ZFS has a ton of stuff in it.  That, also, is messy, and not a
> great thing in terms of kernel size and security and alacrity and
> so on.  But it has some really cool ideas in it.  I'm perfectly
> happy to use it, or will be once I build a new box (still haven't
> made the jump to an AMD system with ECC).
>
> Chris
OMG! It's a memory hog, for sure, and messy :)

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 15:45                           ` Will Senn
@ 2021-02-04 16:03                             ` Henry Bent
  2021-02-04 16:32                             ` Dan Cross
  2021-02-05 20:50                             ` Dave Horsfall
  2 siblings, 0 replies; 70+ messages in thread
From: Henry Bent @ 2021-02-04 16:03 UTC (permalink / raw)
  To: Will Senn; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 4431 bytes --]

I don't really know enough about filesystem internals to comment on the
design one way or another.  I can say that Oberlin College Computer Science
switched to using ZFS with snapshots in 2008 or so (on Solaris 10) and it
greatly simplified restoring from backups.  Home directories had weekly
snapshots taken, rolling over a month or two, and it was a godsend when a
student (or a professor...) accidentally deleted something important.  No
more trawling through backup tapes by the admin to restore the file you
wanted, it could easily be taken from an on-disk snapshot.  Obviously it
required a certain greater amount of disk space, but I contend that it was
worth it.

-Henry

On Thu, 4 Feb 2021 at 10:47, Will Senn <will.senn@gmail.com> wrote:

> On 2/3/21 11:43 PM, Dave Horsfall wrote:
> > On Sat, 30 Jan 2021, Larry McVoy wrote:
> >
> > [ Usual insightful...  insights ]
> >
> >> If you like ZFS you don't understand operating systems design.  I do.
> > ...
> >
> > There's no way that I'd use ZFS; lose a block in an ordinary file,
> > well, you now have a hole (but not in the file-system sense); lose a
> > block in a compressed system, well...
> >
> ZFS needn't be compressed, and I don't generally do compression or
> encryption unless required by law, so I can't speak from personal
> experience on those use cases (others, far more experienced can). I do
> know that it's truly a pain to recover from issues with either.
>
> In response to the negative vibes around ZFS. I've never lost a file (or
> a piece of a file) in 10+ years of using ZFS. I get the feeling we may
> not be talking about the same ZFS. My experience is with the ZFS FreeBSD
> comes with, not the version that Oracle owns. Perhaps the info is a
> little out of date for the naysayers. In my experience, using ZFS is
> fairly transparent and simple to use - no partitioning to deal with, no
> need to worry about generating filesystems, none of that - add your
> disks to a pool, choose your RAID levels and it gets mounted, no fuss.
> I've lost plenty of disks along the way, but ZFS just keeps on chugging
> along nicely until I replace them and then rebuilds the arrays, again,
> no fuss other than replacing the hardware. In terms of massive system
> updates and such, I just snapshot the environment (a near instantaneous
> operation) before making significant changes to my system, that might
> break things and when they do break (and they do, more often than I'd
> like), I just rollback. man bectl. Painless (and I mean painless,
> hundreds of times, or mor). I'm sure it all sounds scifi, but it's my
> experience along with plenty of other folks, and this ZFS sucks thread
> seems to be FUD to me - ala Microsoft vs Linux, or at best informed
> hypothetical speculations - reminds me of an if statement conversation I
> had online in the early 1990's where one group of folks claimed that
> braces worked a certain way, based on the then current standard, and
> another group of folks (I'd be on this side of things), tested the
> theory with a host of compilers, observed the functions effects, shook
> their heads and wondered why it didn't match up with the theory, and
> said it worked another. Who was right? I'm still not entirely sure, from
> a philosophical perspective, but I have since coded my if statements
> according to my environment, not the standards.
>
> As I mentioned in the prior thread, I've lost my share of files and file
> systems (many, many times since 1993 when I started with linux - 0.9
> kernel, slackware, then redhat, then debian, now mint) with ext3/4, and
> btrfs, though, and the only recovery was backup (a time intensive
> process). I really don't see the logic behind the negative arguments.
> Don't like it, fine, say it and live it. Claim it sucks? Then, back it
> up with a real-world, current experience and I'll cede the point - I'll
> keep using ZFS though :).
>
> I want to be clear, I don't dislike Linux. I don't think FreeBSD is
> superior. I like both. I use both... daily. With enough prep and
> planning, my linux environment is similarly recoverable, but with
> freebsd, the prep and planning requires a lot less time and effort.
> Personally, I heart linux Mint - it's based on Debian and Ubuntu - is a
> straightforward install, works well, has zfs (not yet on boot), has
> timeshift (lovely piece of software), and can be quite pretty.
>
> Vive la difference.
>
> Will
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 5149 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 15:45                           ` Will Senn
  2021-02-04 16:03                             ` Henry Bent
@ 2021-02-04 16:32                             ` Dan Cross
  2021-02-04 16:49                               ` Will Senn
                                                 ` (2 more replies)
  2021-02-05 20:50                             ` Dave Horsfall
  2 siblings, 3 replies; 70+ messages in thread
From: Dan Cross @ 2021-02-04 16:32 UTC (permalink / raw)
  To: Will Senn; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 2349 bytes --]

On Thu, Feb 4, 2021 at 10:47 AM Will Senn <will.senn@gmail.com> wrote:

> [snip]
>
> In response to the negative vibes around ZFS. [snip]
>

I think the discordance is around the semantics ZFS's implementation
implies. Larry's point about mmap() vs a buffer cache is entirely valid; it
took lots of people heroic amounts of work worthy of Greek sagas to bridge
the difference between the original buffer and VM page caches, but ZFS
says, "meh. too much work; not worth it." The practical implication of that
is that memory mapped IO (via `mmap`) is no longer coherent with file IO
(via `open`/`close`/`read`/`write`) without lots of work that both degrades
performance and add complexity.

The question that a lot of folks who use ZFS regularly ask is, "does that
matter?" And perhaps it doesn't: if I've got a file server sitting there
serving NFS, do I care what it's kernel is doing? As long as it's
saturating the network and disks, and it's reliable...not really.
(Incidentally, that was kind of the philosophy behind the original plan9
file server kernel...as I heard the story, the rate of change of the plan9
kernel proper was too high, so Ken split off the file server portion into
its own, special-purpose kernel, and it stayed like that for ~20 years).
Similarly, if I'm on the local machine and the required coherence code is
there and largely works, then again, perhaps as a consumer of the
filesystem, I just don't care. After all, one can still get work done, and
ZFS has a bunch of other features that make it very attractive, right? In
particular, it's very good at NOT losing my data, kernel purity be damned.

On the other hand, if we're discussing OS design and implementation,
(re)splitting the VM and buffer caches is a poor decision. One might well
ask, "why?" and the answer may be, "because it adds significant complexity
to the kernel." This to me seems like the crux of the disagreement.
Satisfied users of ZFS might legitimately ask, "who cares?" and one might
respond, "kernel maintainers." If the kernel is mostly transparent as far
as a particular use case goes, though, then I can see why one would bulk at
the suggestion that this matters. If one is concerned with the design and
implementation of kernels, I could see why one would care very much.

Like many things, it's a matter of perspective.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2861 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 16:32                             ` Dan Cross
@ 2021-02-04 16:49                               ` Will Senn
  2021-02-04 17:46                               ` Larry McVoy
  2021-02-04 18:41                               ` Bakul Shah
  2 siblings, 0 replies; 70+ messages in thread
From: Will Senn @ 2021-02-04 16:49 UTC (permalink / raw)
  To: Dan Cross; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 3160 bytes --]

On 2/4/21 10:32 AM, Dan Cross wrote:
> On Thu, Feb 4, 2021 at 10:47 AM Will Senn <will.senn@gmail.com 
> <mailto:will.senn@gmail.com>> wrote:
>
>     [snip]
>
>     In response to the negative vibes around ZFS. [snip]
>
>
> I think the discordance is around the semantics ZFS's implementation 
> implies. Larry's point about mmap() vs a buffer cache is entirely 
> valid; it took lots of people heroic amounts of work worthy of Greek 
> sagas to bridge the difference between the original buffer and VM page 
> caches, but ZFS says, "meh. too much work; not worth it." The 
> practical implication of that is that memory mapped IO (via `mmap`) is 
> no longer coherent with file IO (via `open`/`close`/`read`/`write`) 
> without lots of work that both degrades performance and add complexity.
>
> The question that a lot of folks who use ZFS regularly ask is, "does 
> that matter?" And perhaps it doesn't: if I've got a file server 
> sitting there serving NFS, do I care what it's kernel is doing? As 
> long as it's saturating the network and disks, and it's reliable...not 
> really. (Incidentally, that was kind of the philosophy behind the 
> original plan9 file server kernel...as I heard the story, the rate of 
> change of the plan9 kernel proper was too high, so Ken split off the 
> file server portion into its own, special-purpose kernel, and it 
> stayed like that for ~20 years). Similarly, if I'm on the local 
> machine and the required coherence code is there and largely works, 
> then again, perhaps as a consumer of the filesystem, I just don't 
> care. After all, one can still get work done, and ZFS has a bunch of 
> other features that make it very attractive, right? In particular, 
> it's very good at NOT losing my data, kernel purity be damned.
>
> On the other hand, if we're discussing OS design and implementation, 
> (re)splitting the VM and buffer caches is a poor decision. One might 
> well ask, "why?" and the answer may be, "because it adds significant 
> complexity to the kernel." This to me seems like the crux of the 
> disagreement. Satisfied users of ZFS might legitimately ask, "who 
> cares?" and one might respond, "kernel maintainers." If the kernel is 
> mostly transparent as far as a particular use case goes, though, then 
> I can see why one would bulk at the suggestion that this matters. If 
> one is concerned with the design and implementation of kernels, I 
> could see why one would care very much.
>
> Like many things, it's a matter of perspective.
>
>         - Dan C.
>
Thanks for the comments, Dan. I see your point. I was thinking as a 
user/admin. In this light, I'll admit that I'm not an expert on the 
internals and say that I can only imagine the breadth of design 
tradeoffs that were contemplated and the many decisions that were made 
when coming up with ZFS. I'm glad somebody thought through them, and 
worked through them, though. So I could consume their work, 
Frankensteinian though it may be. Now, if somebody would only get it 
working properly in Linux (boot environments included), or even get 
BTRFS to be more realiable, I'd be a happy camper, at least for a while :).


[-- Attachment #2: Type: text/html, Size: 4608 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 16:32                             ` Dan Cross
  2021-02-04 16:49                               ` Will Senn
@ 2021-02-04 17:46                               ` Larry McVoy
  2021-02-04 18:41                               ` Bakul Shah
  2 siblings, 0 replies; 70+ messages in thread
From: Larry McVoy @ 2021-02-04 17:46 UTC (permalink / raw)
  To: Dan Cross; +Cc: TUHS main list

+1 to everything Dan said, including the "who cares if it works?".

I was definitely coming at it from the kernel perspective, I joined Sun
just after SunOS 4.0 came out; that was the release that made mmap/page
cache be a coherent thing, the buffer cache was almost completely gone,
I think it was still used for inodes and directories but no user data
was in there.

The amount of work that went into that was huge but the resulting kernel
was actually smaller (I think) and much easier to understand and maintain
(I know).  And the kernel team was quite proud of that work, with good
reason.

Which is why it blows my mind that the same organization that saw the
value of have one way to do things, did a 180 and said the page cache
isn't that important.  When I talked to the ZFS guys about it, their
plan was that ZFS was going to be the only local file system so it sort
of didn't matter.  But that's nonsense, you are going to have tmpfs,
nfs, vfat, ntfs, whatever apple uses, etc.  So you are going to have a
page cache for all of those, which means the page cache is never going
away, which means that mmap() wants pages (remember, you can mmap stuff
that isn't in memory, you need the hardware to fault for you so you can
go get it).  The ZFS guys said making that work is too hard, we did it
in BitKeeper, it works fine.  

So the combination of going back to something anyone with OS smarts 
knows is a bad idea plus the fact that I know from personal experience
that they could have done in the page cache and still have compression
and XOR, yeah, I lost a lot of respect for those guys.  

ZFS is very useful, it could have been that and played nice with the 
page cache.

On Thu, Feb 04, 2021 at 11:32:31AM -0500, Dan Cross wrote:
> On Thu, Feb 4, 2021 at 10:47 AM Will Senn <will.senn@gmail.com> wrote:
> 
> > [snip]
> >
> > In response to the negative vibes around ZFS. [snip]
> >
> 
> I think the discordance is around the semantics ZFS's implementation
> implies. Larry's point about mmap() vs a buffer cache is entirely valid; it
> took lots of people heroic amounts of work worthy of Greek sagas to bridge
> the difference between the original buffer and VM page caches, but ZFS
> says, "meh. too much work; not worth it." The practical implication of that
> is that memory mapped IO (via `mmap`) is no longer coherent with file IO
> (via `open`/`close`/`read`/`write`) without lots of work that both degrades
> performance and add complexity.
> 
> The question that a lot of folks who use ZFS regularly ask is, "does that
> matter?" And perhaps it doesn't: if I've got a file server sitting there
> serving NFS, do I care what it's kernel is doing? As long as it's
> saturating the network and disks, and it's reliable...not really.
> (Incidentally, that was kind of the philosophy behind the original plan9
> file server kernel...as I heard the story, the rate of change of the plan9
> kernel proper was too high, so Ken split off the file server portion into
> its own, special-purpose kernel, and it stayed like that for ~20 years).
> Similarly, if I'm on the local machine and the required coherence code is
> there and largely works, then again, perhaps as a consumer of the
> filesystem, I just don't care. After all, one can still get work done, and
> ZFS has a bunch of other features that make it very attractive, right? In
> particular, it's very good at NOT losing my data, kernel purity be damned.
> 
> On the other hand, if we're discussing OS design and implementation,
> (re)splitting the VM and buffer caches is a poor decision. One might well
> ask, "why?" and the answer may be, "because it adds significant complexity
> to the kernel." This to me seems like the crux of the disagreement.
> Satisfied users of ZFS might legitimately ask, "who cares?" and one might
> respond, "kernel maintainers." If the kernel is mostly transparent as far
> as a particular use case goes, though, then I can see why one would bulk at
> the suggestion that this matters. If one is concerned with the design and
> implementation of kernels, I could see why one would care very much.
> 
> Like many things, it's a matter of perspective.
> 
>         - Dan C.

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 16:32                             ` Dan Cross
  2021-02-04 16:49                               ` Will Senn
  2021-02-04 17:46                               ` Larry McVoy
@ 2021-02-04 18:41                               ` Bakul Shah
  2021-02-04 22:28                                 ` George Michaelson
  2 siblings, 1 reply; 70+ messages in thread
From: Bakul Shah @ 2021-02-04 18:41 UTC (permalink / raw)
  To: Dan Cross; +Cc: TUHS main list

On Feb 4, 2021, at 8:34 AM, Dan Cross <crossd@gmail.com> wrote:
> 
> On the other hand, if we're discussing OS design and implementation, (re)splitting the VM and buffer caches is a poor decision. One might well ask, "why?" and the answer may be, "because it adds significant complexity to the kernel." This to me seems like the crux of the disagreement. Satisfied users of ZFS might legitimately ask, "who cares?" and one might respond, "kernel maintainers." If the kernel is mostly transparent as far as a particular use case goes, though, then I can see why one would bulk at the suggestion that this matters. If one is concerned with the design and implementation of kernels, I could see why one would care very much.

Largely agree; though the complexity battle has long been lost. On multiple fronts. Many of us are happy to use such complex systems for their ease of use or their feature set but wouldn’t want to maintain these systems!

I have used ZFS since 2005 and largely happy with it. Replaced all the disks twice. Moved the same set of disks to a new machine. etc. Features: cheap and fast snapshots, send/receive, clone, adding disks, checksummed blocks, redundancy etc. The dedup impl. is suboptimal so I don't use it. No idea if they considered using a bloom filter and a cache to reduce memory use. If a new FS came along with a similar set of features and a simpler, better integrated implementation, I'd switch.



^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04  6:10                           ` Angus Robinson
  2021-02-04  7:46                             ` Andy Kosela
@ 2021-02-04 22:25                             ` Dave Horsfall
  1 sibling, 0 replies; 70+ messages in thread
From: Dave Horsfall @ 2021-02-04 22:25 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On Thu, 4 Feb 2021, Angus Robinson wrote:

> Not entirely sure as it's been a while since iI have used it but last 
> time I remember TrueOS which used FreeBSD was easy to use for a newbie. 
> There are other FreeBSD distro's like GhostBSD,etc that have an 
> installer like Linux. My 2c is that FreeBSD is not trying to get 
> people in that are newbies, it's for the server environment and it works 
> extremely well. 

Exactly; FreeBSD is not for newbies: I'll leave Penguin/OS to that market.

Mind you, OpenBSD was a bit of a bugger; all services are off by default. 
It's perfect for a firewall :-)

-- Dave

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 18:41                               ` Bakul Shah
@ 2021-02-04 22:28                                 ` George Michaelson
  2021-02-04 22:41                                   ` Bakul Shah
  2021-02-05  0:33                                   ` Larry McVoy
  0 siblings, 2 replies; 70+ messages in thread
From: George Michaelson @ 2021-02-04 22:28 UTC (permalink / raw)
  To: TUHS main list

snapshots of FS state existed for years before ZFS/BTRFS did it,
because FS journaling existed. The underlying model of COW forking of
the inodes and a journal were all there.

What ZFS did, and what Docker does, and snap does, and flatpack does,
is package things in a way which make modalities for use for  modern
sysadmin "just work"

I don't recall seeing a backup model built on UFS+journalling with an
integrated command tooling to do what zfs snapshot; zfs send | zfs
receive does as an incremental copy mechanism. I'm not trying to
disrespect the prior art, if there was a good packaging, I probably
missed it. ZFS made this "yea, I get it now" for a lot of people.

I think ZFS is fine. I have adopted it wholesale in the work context
for the last 15+ years and at home for about a year. The FUD around
minimum memory for ARC is a misunderstanding of an Oracle document
"back then" and you can run ZFS on rPi class systems fine, if you can
accept some lumpy paging and VM behaviour, and if you don't enable
compression which burns the ARC. (you need the extent of memory to
deal with things a lot more) -And there are modern OpenZFS docs which
patiently explain this stuff.

Zsys is going to get better. This means rollback from systems update
under Ubuntu snaps and things, will get better. It will be a lot less
risky to do significant upgrade on systems. -Think about Android and
the two-root model of updating the passive root in the background, so
the new OS is live across a reboot without having the spinning
beachball of upgrade on your phone, or analogues of the "keep this
setting or go back" model WIndows does, and Windows has had OS
snapshots for a long time now, and offers "undo that major update"
models to users: people expect this.

I also think ZFS was a godsend for getting over the smart RAID card
mistakes of the past. I HAVE lost data with these puppies. I've moved
multi TB fs between FreeBSD and Linux and back again (with care) under
ZFS, and you can't do that with Apples FS, or EXT3 with anything like
the same confidence.

Really? I like UUID. UUID are a godsend, for making things have
unique,  but asynchronously generated identity, so when you move them
and mix them, you can stop worrying about device/bus order and simply
re-create them as they were defined semantically. ZFS does this too:
zfs import is leveraging what UUID does for you.

Basically Larry, I think you are kindof wrong. These alumni of yours
did what all kids should do: they ran ahead. Did they scrape their
knees doing it? Sure. But if they don't try things their teachers say
are bad, how do they advance the art? If we'd listened to Eddy
Dijkstra, we'd never have got BGP: He said it couldn't scale, even
though it was based on his own work.

-G

On Fri, Feb 5, 2021 at 4:41 AM Bakul Shah <bakul@iitbombay.org> wrote:
>
> On Feb 4, 2021, at 8:34 AM, Dan Cross <crossd@gmail.com> wrote:
> >
> > On the other hand, if we're discussing OS design and implementation, (re)splitting the VM and buffer caches is a poor decision. One might well ask, "why?" and the answer may be, "because it adds significant complexity to the kernel." This to me seems like the crux of the disagreement. Satisfied users of ZFS might legitimately ask, "who cares?" and one might respond, "kernel maintainers." If the kernel is mostly transparent as far as a particular use case goes, though, then I can see why one would bulk at the suggestion that this matters. If one is concerned with the design and implementation of kernels, I could see why one would care very much.
>
> Largely agree; though the complexity battle has long been lost. On multiple fronts. Many of us are happy to use such complex systems for their ease of use or their feature set but wouldn’t want to maintain these systems!
>
> I have used ZFS since 2005 and largely happy with it. Replaced all the disks twice. Moved the same set of disks to a new machine. etc. Features: cheap and fast snapshots, send/receive, clone, adding disks, checksummed blocks, redundancy etc. The dedup impl. is suboptimal so I don't use it. No idea if they considered using a bloom filter and a cache to reduce memory use. If a new FS came along with a similar set of features and a simpler, better integrated implementation, I'd switch.
>
>

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 22:28                                 ` George Michaelson
@ 2021-02-04 22:41                                   ` Bakul Shah
  2021-02-05  0:33                                   ` Larry McVoy
  1 sibling, 0 replies; 70+ messages in thread
From: Bakul Shah @ 2021-02-04 22:41 UTC (permalink / raw)
  To: TUHS main list

On Feb 4, 2021, at 2:28 PM, George Michaelson <ggm@algebras.org> wrote:
> 
> Basically Larry, I think you are kindof wrong. These alumni of yours
> did what all kids should do: they ran ahead. Did they scrape their
> knees doing it? Sure. But if they don't try things their teachers say
> are bad, how do they advance the art? If we'd listened to Eddy
> Dijkstra, we'd never have got BGP: He said it couldn't scale, even
> though it was based on his own work.

I think Larry is talking about ZFS *implementation* design choices;
which we, as ZFS users, mostly don't care about!

It was OSPF, not BGP, that used Dijkstra's SPF algorithm. The last
I knew BGP was all about "policy" -- why random bad actors can sling
half of the internet traffic through China or Vanuatu or whatever.


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 22:28                                 ` George Michaelson
  2021-02-04 22:41                                   ` Bakul Shah
@ 2021-02-05  0:33                                   ` Larry McVoy
  2021-02-05  5:17                                     ` Bakul Shah
  1 sibling, 1 reply; 70+ messages in thread
From: Larry McVoy @ 2021-02-05  0:33 UTC (permalink / raw)
  To: George Michaelson; +Cc: TUHS main list

On Fri, Feb 05, 2021 at 08:28:12AM +1000, George Michaelson wrote:
> What ZFS did, and what Docker does, and snap does, and flatpack does,
> is package things in a way which make modalities for use for  modern
> sysadmin "just work"

No argument there.

> Basically Larry, I think you are kindof wrong. These alumni of yours
> did what all kids should do: they ran ahead. Did they scrape their
> knees doing it? Sure. But if they don't try things their teachers say
> are bad, how do they advance the art? 

Before I show you I'm not wrong, if you are saying (and I think you
are) that you like ZFS and find it useful, I have no disagreement with
that.  I'm in no way arguing that ZFS isn't useful.  If you are just 
an end user and you don't run into any of the coherency problems, 
it's great.

I'm arguing from the point of view of how a kernel is supposed to work.
What ZFS did is a gross violation of how the kernel is supposed to work
and both Bonwick and Moore have admitted that, they just thought it was
too hard to do it right.  There is a body of code in BitKeeper that does
the exact part that they thought was too hard, a layer that takes a page
fault and fills in the page from a compressed and xor-ed data source.
Works great, one guy did it in a few months or so.  It's not that hard.

So why is what ZFS did so wrong?

Ignoring the page cache and make their own cache has big problems.
You can mmap() ZFS files and doing so means that when a page is referenced
it is copied from the ZFS cache to the page cache.  That creates a
coherency problem, I can write via the mapping and I can write via
write(2) and now you have two copies of the data that don't match,
that's pretty much OS no-no #1.

You can get around it, I know, because I've written the coherency code
for SGI's IRIX when I did the bulk data server that went around the page
cache and made NFS go at 60MB/sec for a single stream (and many times
that for multiple streams).  So I'm not talking out of my ass, I know
what coherency means when you have the same data in two different places,
I know it is possible to make it work, I've done that, and I don't think
it is a good idea (it was OK in SGI's case, it was for O_DIRECT which
exists to completely bypass the page cache; so a special case that wasn't
so bad and wasn't general).

It's messy.  You could remove the data from the ZFS cache when you put
it in the page cache but ZFS is compresses so it's not going line up on
page boundaries like you'd want it to.  That means you're removing more
than your page which sort of sucks.

You could never map the pages writable, take a fault every time someone
wants to write the page and then do the write back to the ZFS cache.
That doesn't really work because you take the fault before the write
completes, not after.  You can make it work, the write fault has to
get an exclusive lock on the data in the ZFS cache, then return, then
the page gets modified, now someone has to wake up and copy that data
from the page cache to ZFS.  It's messy and it performs really poorly,
nobody would do it this way.  

You could lock the data in the ZFS cache, making it read only.  That
doesn't work because you can write via mmap() and read via ZFS and you
get old data.

All of these sorts of problems, which are solvable, I've solved them,
Sun solved them, are why you don't really want what ZFS did.  It's non
ending case of wack a mole as the code evolves and someone slips in
something that makes the page cache and the ZFS cache incoherent again.
There isn't a pleasant way to make this stuff work, that's exactly
why Sun made everything live in the page cache, there was only one
copy of any chunk of data.

Which makes it baffling to me that Sun would allow ZFS into the kernel
but I guess the benefits were perceived to outweigh the ongoing work to
make the caches coherent.  Personally, I think just doing it right is
way easier.

--lm

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05  0:33                                   ` Larry McVoy
@ 2021-02-05  5:17                                     ` Bakul Shah
  2021-02-05 14:18                                       ` Larry McVoy
  0 siblings, 1 reply; 70+ messages in thread
From: Bakul Shah @ 2021-02-05  5:17 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list

On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> Ignoring the page cache and make their own cache has big problems.
> You can mmap() ZFS files and doing so means that when a page is referenced
> it is copied from the ZFS cache to the page cache.  That creates a
> coherency problem, I can write via the mapping and I can write via
> write(2) and now you have two copies of the data that don't match,
> that's pretty much OS no-no #1.

Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
in trouble in any case. Similarly read(2)ing. And you can keep track of
mapped pages and read/write from them if necessary even if you have
a  separate cache for any compressed pages. I haven’t read zfs code
but this doesn’t seem like a tricky problem.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05  5:17                                     ` Bakul Shah
@ 2021-02-05 14:18                                       ` Larry McVoy
  2021-02-05 18:16                                         ` Warner Losh
                                                           ` (3 more replies)
  0 siblings, 4 replies; 70+ messages in thread
From: Larry McVoy @ 2021-02-05 14:18 UTC (permalink / raw)
  To: Bakul Shah; +Cc: TUHS main list

On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > 
> > Ignoring the page cache and make their own cache has big problems.
> > You can mmap() ZFS files and doing so means that when a page is referenced
> > it is copied from the ZFS cache to the page cache.  That creates a
> > coherency problem, I can write via the mapping and I can write via
> > write(2) and now you have two copies of the data that don't match,
> > that's pretty much OS no-no #1.
> 
> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
> in trouble in any case. Similarly read(2)ing. 

The entire point of the SunOS 4.0 VM system was that the page you
saw via mmap(2) is the exact same page you saw via read(2).  It's
the page cache, it has page sized chunks of memory that cache 
file,offset pairs.

There is one, and only one, copy of the truth.  Doesn't matter how
you get at it, there is only one "it".

ZFS broke that contract and that was a step backwards in terms of
OS design.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 14:18                                       ` Larry McVoy
@ 2021-02-05 18:16                                         ` Warner Losh
  2021-02-05 18:21                                         ` ron minnich
                                                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 70+ messages in thread
From: Warner Losh @ 2021-02-05 18:16 UTC (permalink / raw)
  To: Larry McVoy; +Cc: Bakul Shah, TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1540 bytes --]

On Fri, Feb 5, 2021, 7:19 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> > On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > >
> > > Ignoring the page cache and make their own cache has big problems.
> > > You can mmap() ZFS files and doing so means that when a page is
> referenced
> > > it is copied from the ZFS cache to the page cache.  That creates a
> > > coherency problem, I can write via the mapping and I can write via
> > > write(2) and now you have two copies of the data that don't match,
> > > that's pretty much OS no-no #1.
> >
> > Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
> > in trouble in any case. Similarly read(2)ing.
>
> The entire point of the SunOS 4.0 VM system was that the page you
> saw via mmap(2) is the exact same page you saw via read(2).  It's
> the page cache, it has page sized chunks of memory that cache
> file,offset pairs.
>
> There is one, and only one, copy of the truth.  Doesn't matter how
> you get at it, there is only one "it".
>
> ZFS broke that contract and that was a step backwards in terms of
> OS design.
>

The double copy is the primary reason we don't use it to store videos we
serve. It's a performance bottleneck as well.

And fixing it is... rather involved... possible, but a lot of work to teach
the ARC about the buffer cache or the buffer cache about the ARC...

But for everything else I do, I accept the imperfect design because of all
the other features it unlocks.

Warner

>

[-- Attachment #2: Type: text/html, Size: 2396 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 14:18                                       ` Larry McVoy
  2021-02-05 18:16                                         ` Warner Losh
@ 2021-02-05 18:21                                         ` ron minnich
  2021-02-06  0:03                                         ` Bakul Shah
  2021-02-06  1:18                                         ` John Gilmore
  3 siblings, 0 replies; 70+ messages in thread
From: ron minnich @ 2021-02-05 18:21 UTC (permalink / raw)
  To: Larry McVoy; +Cc: Bakul Shah, TUHS main list

I think hearing *why* they did that would be interesting.

On Fri, Feb 5, 2021 at 6:19 AM Larry McVoy <lm@mcvoy.com> wrote:
>
> On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> > On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
> > >
> > > Ignoring the page cache and make their own cache has big problems.
> > > You can mmap() ZFS files and doing so means that when a page is referenced
> > > it is copied from the ZFS cache to the page cache.  That creates a
> > > coherency problem, I can write via the mapping and I can write via
> > > write(2) and now you have two copies of the data that don't match,
> > > that's pretty much OS no-no #1.
> >
> > Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
> > in trouble in any case. Similarly read(2)ing.
>
> The entire point of the SunOS 4.0 VM system was that the page you
> saw via mmap(2) is the exact same page you saw via read(2).  It's
> the page cache, it has page sized chunks of memory that cache
> file,offset pairs.
>
> There is one, and only one, copy of the truth.  Doesn't matter how
> you get at it, there is only one "it".
>
> ZFS broke that contract and that was a step backwards in terms of
> OS design.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-04 15:45                           ` Will Senn
  2021-02-04 16:03                             ` Henry Bent
  2021-02-04 16:32                             ` Dan Cross
@ 2021-02-05 20:50                             ` Dave Horsfall
  2021-02-06  0:21                               ` Brad Spencer
                                                 ` (2 more replies)
  2 siblings, 3 replies; 70+ messages in thread
From: Dave Horsfall @ 2021-02-05 20:50 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Thu, 4 Feb 2021, Will Senn wrote:

> ZFS needn't be compressed, and I don't generally do compression or 
> encryption unless required by law, so I can't speak from personal 
> experience on those use cases (others, far more experienced can). I do 
> know that it's truly a pain to recover from issues with either.

[...]

Thanks; I'd heard that ZFS was a compressed file system, so I stopped 
right there (I had lots of experience in recovering from corrupted RK05s, 
and didn't need any more trouble).

Ah, the RK05 - evil incarnate.  I mean, a disk drive exposed to the air? 
Out There Somewhere [tm] is a picture of a human hair compared with the 
head clearance; yikes!

Once a month, our DEC ginger-beer[*] would PM our 40s, and I was most 
perturbed to learn that the sole job of the internal NiCd battery was to 
drag those heads back in the event of a power failure; what, you were 
writing at the time?  Too bad...

[*]
His surname was "Roth", so naturally his nickname was "Portnoy".

-- Dave

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 14:18                                       ` Larry McVoy
  2021-02-05 18:16                                         ` Warner Losh
  2021-02-05 18:21                                         ` ron minnich
@ 2021-02-06  0:03                                         ` Bakul Shah
  2021-02-06  2:06                                           ` Dan Cross
  2021-02-06  1:18                                         ` John Gilmore
  3 siblings, 1 reply; 70+ messages in thread
From: Bakul Shah @ 2021-02-06  0:03 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list

On Feb 5, 2021, at 6:18 AM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
>> On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
>>> 
>>> Ignoring the page cache and make their own cache has big problems.
>>> You can mmap() ZFS files and doing so means that when a page is referenced
>>> it is copied from the ZFS cache to the page cache.  That creates a
>>> coherency problem, I can write via the mapping and I can write via
>>> write(2) and now you have two copies of the data that don't match,
>>> that's pretty much OS no-no #1.
>> 
>> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
>> in trouble in any case. Similarly read(2)ing. 
> 
> The entire point of the SunOS 4.0 VM system was that the page you
> saw via mmap(2) is the exact same page you saw via read(2).  It's
> the page cache, it has page sized chunks of memory that cache 
> file,offset pairs.
> 
> There is one, and only one, copy of the truth.  Doesn't matter how
> you get at it, there is only one "it".
> 
> ZFS broke that contract and that was a step backwards in terms of
> OS design.

Let me repeat a part of my response you cut out:

    And you can keep track of mapped pages and read/write from them if
    necessary even if you have a  separate cache for any compressed pages.

In essence you pass the ownership of a page's data from a compressed
page cache to the mapped page. Just like in processor cache coherence
algorithms there is one source of truth: the current owner of a cached
unit (line or page or whatever). In other words, the you see via mmap(2)
will be the exact same page you will see via read(2). Not having actually
tried this I may have missed corner cases + any practical considerations
complicating things but *conceptually* this doesn't seem hard.

Warner mentions not using ZFS for its double copying. May be omething
like the above can a step in the direction of integrating the caches?

As Ron says, I too would like to hear what the authors of ZFS have to say....


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 20:50                             ` Dave Horsfall
@ 2021-02-06  0:21                               ` Brad Spencer
  2021-02-06  2:22                               ` Rico Pajarola
  2021-02-06  4:55                               ` John Cowan
  2 siblings, 0 replies; 70+ messages in thread
From: Brad Spencer @ 2021-02-06  0:21 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: tuhs

Dave Horsfall <dave@horsfall.org> writes:

> On Thu, 4 Feb 2021, Will Senn wrote:
>
>> ZFS needn't be compressed, and I don't generally do compression or 
>> encryption unless required by law, so I can't speak from personal 
>> experience on those use cases (others, far more experienced can). I do 
>> know that it's truly a pain to recover from issues with either.
>
> [...]
>
> Thanks; I'd heard that ZFS was a compressed file system, so I stopped 
> right there (I had lots of experience in recovering from corrupted RK05s, 
> and didn't need any more trouble).

[snip]

I have some real world experience with ZFS and bad blocks.  As mentioned
by others, compression is not required, but it would not matter too
much.  ZFS is somewhat odd in that you can mount and use a damaged
filesystem without too much trouble and recover anything possible.  A
real example (and to keep it a TUHS topic), we had an older Solaris 10
Sparc at the $DAYJOB about two years ago that developed bad blocks on
the drive (ZFS detected this for us).  Not a surprise given the age of
the system, being 6 to 8 years old at the time, we replaced the drive
without issue and started a re-silver (ZFS's version of fsck and raid
rebuild) and during that re-silver another drive developed block errors.
This was a RAIDZ1 set up, so having two drives go out at the same time
would cause something to be lost.  If it were another RAID variant, it
would probably have been fatal.  What ZFS did was told us exactly which
file was effected by the bad block and the system kept going, no
unmounts, reboots or down time.  We replaced the second drive and the
only problem we had was a single lost file.  Compression would not have
changed this situation much, except perhaps making the problem a bit
bigger, we may have lost two files or something.  I have another example
with a Sun NFS appliance that runs under the covers Solaris 11 that
developed multiple drive fails.  In the end, we estimated that around
40% or so of the multi terabyte filesystem was damaged.  It was possible
to still use the appliance and we were able to get a lot off of it as it
continued to function as best as it could you just could not access the
files that had bad blocks in them.  ZFS has honestly been amazing in the
face of the problems I have seen.



-- 
Brad Spencer - brad@anduin.eldar.org - KC8VKS - http://anduin.eldar.org

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 14:18                                       ` Larry McVoy
                                                           ` (2 preceding siblings ...)
  2021-02-06  0:03                                         ` Bakul Shah
@ 2021-02-06  1:18                                         ` John Gilmore
  2021-02-06  1:43                                           ` joe mcguckin
  2021-02-06  1:55                                           ` Bakul Shah
  3 siblings, 2 replies; 70+ messages in thread
From: John Gilmore @ 2021-02-06  1:18 UTC (permalink / raw)
  To: Bakul Shah, Larry McVoy, TUHS main list

On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
> in trouble in any case. Similarly read(2)ing. 

Uh, no.  You misunderstand completely.

The purpose of the kernel is to provide a reliable interface to system
facilities, that lets processes NOT DEPEND on what other processes are
doing.

The decision about whether Tool X uses mmap() versus read() to access a
file, or mmap() versus write() to change one, is a decision that DOES
NOT DEPEND on what Tool Y is doing.  Tools X and Y may have been written
by different groups in different decades.  Tool X may have been written
to use stdio, which used read().  Three years later, stdio got rewritten
to use mmap() for speed, but that's invisible to the author of Tool X.
And maybe an end user in 2025 decides to use both Tool X and Tool Y on
the same file.  So only much later will any malign interactions between
read/write and mmap actually be noticed by end users.  And the fix is
not to create new dependencies between Tool X, stdio, and Tool Y.  It is
to fix the kernel so they do not depend on each other!

Here is a real-life example from my own experience.

There is a long-standing bug in the Linux kernel, in which the inotify()
system call simply didn't work on nested file systems.  This caused a
long-standing bug in Ubuntu, which I reported in 2012 here:

  https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847

The symptom was that after booting from a LiveCD image, "apt-get
install" for system services (in my case an NFS client package) wouldn't
work.  Turned out the system startup scripts used inotify() to notice
and start newly installed system services.  The root cause was that
inotify failed because the root file system was an "overlayfs" that
overlaid a RAMdisk on top of the read-only LiveCD file system.  The
people who implemented "overlayfs" didn't think inotify() was important,
or they thought it would be too much work to make it actually meet its
specs, so they just made it ignore changes to the files in the overlaid
file system.  So the startup daemon's inotify() would never report the
creation of new files about the new services, because those files were
in the overlaying RAM disk, and so it would not start them and the user
would notice the error.

The underlying overlayfs bug was reported in 2011 here:

  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147

As far as I know it has never been fixed.  (The bug report was
closed in 2019 for one of the usual bogus reasons.)

The problem came because real tools (like systemd, or the tail command)
actually started using inotify, assuming that as a well documented
kernel interface, it would actually meet its specs.  And because a
completely unrelated other real tool (like the LiveCD installer)
actually started using overlayfs, assuming that as a well documented
kernel interface, it too would actually meet its specs.  And then one
day somebody tried to use both those tools together and they failed.

That's why telling people "Don't use mmap() on the same file that you
use read() on" is an invalid attitude for a Real Kernel Maintainer.
Props to Larry McVoy for caring about this.  Boos to the Linux
maintainers of overlayfs who didn't give a shit.

	John
	

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  1:18                                         ` John Gilmore
@ 2021-02-06  1:43                                           ` joe mcguckin
  2021-02-06  1:55                                           ` Bakul Shah
  1 sibling, 0 replies; 70+ messages in thread
From: joe mcguckin @ 2021-02-06  1:43 UTC (permalink / raw)
  To: John Gilmore; +Cc: Bakul Shah, TUHS main list

[-- Attachment #1: Type: text/plain, Size: 4476 bytes --]

When I was working for a big chip testing company, a STDIO vs MMAP problem came up.

The tester was at it’s heart a SPARC VME system running Solaris.  The tester read in ‘patterns’ 
from disk, it it literally took hours to read in all the test patterns. At the scale of the large chip vendors,
every minute you can’t test because the tester is booting, etc means dollars are lost.

We wrote a bunch of macros that replaced the STDIO file system I/O calls with the equivalent 
mmap calls. . It turns out STDIO does a lot of prefetching and has some assumptions that you’re 
going to read a file linearly from beginning to end, whereas we wanted to jump around a lot in the pattern files.

Pattern loading went from 4 hours to 30 minutes. Our customer was ecstatic.

Joe McGuckin
ViaNet Communications

joe@via.net
650-207-0372 cell
650-213-1302 office
650-969-2124 fax



> On Feb 5, 2021, at 5:18 PM, John Gilmore <gnu@toad.com> wrote:
> 
> On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
>> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
>> in trouble in any case. Similarly read(2)ing. 
> 
> Uh, no.  You misunderstand completely.
> 
> The purpose of the kernel is to provide a reliable interface to system
> facilities, that lets processes NOT DEPEND on what other processes are
> doing.
> 
> The decision about whether Tool X uses mmap() versus read() to access a
> file, or mmap() versus write() to change one, is a decision that DOES
> NOT DEPEND on what Tool Y is doing.  Tools X and Y may have been written
> by different groups in different decades.  Tool X may have been written
> to use stdio, which used read().  Three years later, stdio got rewritten
> to use mmap() for speed, but that's invisible to the author of Tool X.
> And maybe an end user in 2025 decides to use both Tool X and Tool Y on
> the same file.  So only much later will any malign interactions between
> read/write and mmap actually be noticed by end users.  And the fix is
> not to create new dependencies between Tool X, stdio, and Tool Y.  It is
> to fix the kernel so they do not depend on each other!
> 
> Here is a real-life example from my own experience.
> 
> There is a long-standing bug in the Linux kernel, in which the inotify()
> system call simply didn't work on nested file systems.  This caused a
> long-standing bug in Ubuntu, which I reported in 2012 here:
> 
>  https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847
> 
> The symptom was that after booting from a LiveCD image, "apt-get
> install" for system services (in my case an NFS client package) wouldn't
> work.  Turned out the system startup scripts used inotify() to notice
> and start newly installed system services.  The root cause was that
> inotify failed because the root file system was an "overlayfs" that
> overlaid a RAMdisk on top of the read-only LiveCD file system.  The
> people who implemented "overlayfs" didn't think inotify() was important,
> or they thought it would be too much work to make it actually meet its
> specs, so they just made it ignore changes to the files in the overlaid
> file system.  So the startup daemon's inotify() would never report the
> creation of new files about the new services, because those files were
> in the overlaying RAM disk, and so it would not start them and the user
> would notice the error.
> 
> The underlying overlayfs bug was reported in 2011 here:
> 
>  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147
> 
> As far as I know it has never been fixed.  (The bug report was
> closed in 2019 for one of the usual bogus reasons.)
> 
> The problem came because real tools (like systemd, or the tail command)
> actually started using inotify, assuming that as a well documented
> kernel interface, it would actually meet its specs.  And because a
> completely unrelated other real tool (like the LiveCD installer)
> actually started using overlayfs, assuming that as a well documented
> kernel interface, it too would actually meet its specs.  And then one
> day somebody tried to use both those tools together and they failed.
> 
> That's why telling people "Don't use mmap() on the same file that you
> use read() on" is an invalid attitude for a Real Kernel Maintainer.
> Props to Larry McVoy for caring about this.  Boos to the Linux
> maintainers of overlayfs who didn't give a shit.
> 
> 	John
> 	


[-- Attachment #2: Type: text/html, Size: 7707 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  1:18                                         ` John Gilmore
  2021-02-06  1:43                                           ` joe mcguckin
@ 2021-02-06  1:55                                           ` Bakul Shah
  1 sibling, 0 replies; 70+ messages in thread
From: Bakul Shah @ 2021-02-06  1:55 UTC (permalink / raw)
  To: John Gilmore; +Cc: TUHS main list

Please see my followup message. My fault for mixing two separate things
(what a user should not do vs how the kernel can still provide coherence).

> On Feb 5, 2021, at 5:18 PM, John Gilmore <gnu@toad.com> wrote:
> 
> On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
>> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
>> in trouble in any case. Similarly read(2)ing. 
> 
> Uh, no.  You misunderstand completely.
> 
> The purpose of the kernel is to provide a reliable interface to system
> facilities, that lets processes NOT DEPEND on what other processes are
> doing.
> 
> The decision about whether Tool X uses mmap() versus read() to access a
> file, or mmap() versus write() to change one, is a decision that DOES
> NOT DEPEND on what Tool Y is doing.  Tools X and Y may have been written
> by different groups in different decades.  Tool X may have been written
> to use stdio, which used read().  Three years later, stdio got rewritten
> to use mmap() for speed, but that's invisible to the author of Tool X.
> And maybe an end user in 2025 decides to use both Tool X and Tool Y on
> the same file.  So only much later will any malign interactions between
> read/write and mmap actually be noticed by end users.  And the fix is
> not to create new dependencies between Tool X, stdio, and Tool Y.  It is
> to fix the kernel so they do not depend on each other!
> 
> Here is a real-life example from my own experience.
> 
> There is a long-standing bug in the Linux kernel, in which the inotify()
> system call simply didn't work on nested file systems.  This caused a
> long-standing bug in Ubuntu, which I reported in 2012 here:
> 
>  https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847
> 
> The symptom was that after booting from a LiveCD image, "apt-get
> install" for system services (in my case an NFS client package) wouldn't
> work.  Turned out the system startup scripts used inotify() to notice
> and start newly installed system services.  The root cause was that
> inotify failed because the root file system was an "overlayfs" that
> overlaid a RAMdisk on top of the read-only LiveCD file system.  The
> people who implemented "overlayfs" didn't think inotify() was important,
> or they thought it would be too much work to make it actually meet its
> specs, so they just made it ignore changes to the files in the overlaid
> file system.  So the startup daemon's inotify() would never report the
> creation of new files about the new services, because those files were
> in the overlaying RAM disk, and so it would not start them and the user
> would notice the error.
> 
> The underlying overlayfs bug was reported in 2011 here:
> 
>  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147
> 
> As far as I know it has never been fixed.  (The bug report was
> closed in 2019 for one of the usual bogus reasons.)
> 
> The problem came because real tools (like systemd, or the tail command)
> actually started using inotify, assuming that as a well documented
> kernel interface, it would actually meet its specs.  And because a
> completely unrelated other real tool (like the LiveCD installer)
> actually started using overlayfs, assuming that as a well documented
> kernel interface, it too would actually meet its specs.  And then one
> day somebody tried to use both those tools together and they failed.
> 
> That's why telling people "Don't use mmap() on the same file that you
> use read() on" is an invalid attitude for a Real Kernel Maintainer.
> Props to Larry McVoy for caring about this.  Boos to the Linux
> maintainers of overlayfs who didn't give a shit.
> 
> 	John
> 	


^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  0:03                                         ` Bakul Shah
@ 2021-02-06  2:06                                           ` Dan Cross
  2021-02-06  3:01                                             ` Bakul Shah
  0 siblings, 1 reply; 70+ messages in thread
From: Dan Cross @ 2021-02-06  2:06 UTC (permalink / raw)
  To: Bakul Shah; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 3633 bytes --]

On Fri, Feb 5, 2021 at 7:04 PM Bakul Shah <bakul@iitbombay.org> wrote:

> On Feb 5, 2021, at 6:18 AM, Larry McVoy <lm@mcvoy.com> wrote:
> >
> > On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote:
> >> On Feb 4, 2021, at 4:33 PM, Larry McVoy <lm@mcvoy.com> wrote:
> >>>
> >>> Ignoring the page cache and make their own cache has big problems.
> >>> You can mmap() ZFS files and doing so means that when a page is
> referenced
> >>> it is copied from the ZFS cache to the page cache.  That creates a
> >>> coherency problem, I can write via the mapping and I can write via
> >>> write(2) and now you have two copies of the data that don't match,
> >>> that's pretty much OS no-no #1.
> >>
> >> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you
> >> in trouble in any case. Similarly read(2)ing.
> >
> > The entire point of the SunOS 4.0 VM system was that the page you
> > saw via mmap(2) is the exact same page you saw via read(2).  It's
> > the page cache, it has page sized chunks of memory that cache
> > file,offset pairs.
> >
> > There is one, and only one, copy of the truth.  Doesn't matter how
> > you get at it, there is only one "it".
> >
> > ZFS broke that contract and that was a step backwards in terms of
> > OS design.
>
> Let me repeat a part of my response you cut out:
>
>     And you can keep track of mapped pages and read/write from them if
>     necessary even if you have a  separate cache for any compressed pages.
>
> In essence you pass the ownership of a page's data from a compressed
> page cache to the mapped page. Just like in processor cache coherence
> algorithms there is one source of truth: the current owner of a cached
> unit (line or page or whatever). In other words, the you see via mmap(2)
> will be the exact same page you will see via read(2). Not having actually
> tried this I may have missed corner cases + any practical considerations
> complicating things but *conceptually* this doesn't seem hard.


In essence, that's what the merged page/buffer cache is all about: file IO
(read/write) operations are satisfied from the same memory cache that backs
up VM objects. I agree that conceptually it's not that complex; but that's
not what ZFS does.

Of course the original Unix buffer cache didn't do that either, because no
one was mmap'ing files on the PDP-11, let alone the PDP-7. A RAM-based
buffer cache for blocks as the nexus around which the system serialized
access to the disc-resident filesystem sufficed. When virtual address
spaces got bigger (starting on the VAX) and folks wanted to start being
more clever with marrying virtual memory and IO, you had an impedance
mismatch with a fairly large extant kernel that had developed not taking
into consideration memory-mapped IO to/from files. Sun fixed this, at what
I take to be great expense (I followed keenly the same path of development
in *BSD and Linux at the time and saw how long that took, so I believe
this). But then the same Sun broke it again.

Warner mentions not using ZFS for its double copying. May be omething
> like the above can a step in the direction of integrating the caches?
>

But the cache was integrated! Until it wasn't again....

As Ron says, I too would like to hear what the authors of ZFS have to
> say....
>

Sounds like they thought it was too hard because compression means the
place on storage where an offset in a file lands is no longer a linear
function of the file's contents. Presumably the compressed contents are not
kept in RAM in any of the caches (aside from a temporary buffer to which
the compressed contents are read or something).

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 4759 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 20:50                             ` Dave Horsfall
  2021-02-06  0:21                               ` Brad Spencer
@ 2021-02-06  2:22                               ` Rico Pajarola
  2021-02-06  2:55                                 ` Larry McVoy
  2021-02-06  4:55                               ` John Cowan
  2 siblings, 1 reply; 70+ messages in thread
From: Rico Pajarola @ 2021-02-06  2:22 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 4534 bytes --]

On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall <dave@horsfall.org> wrote:

>
> [...]
>
> Thanks; I'd heard that ZFS was a compressed file system, so I stopped
> right there (I had lots of experience in recovering from corrupted RK05s,
> and didn't need any more trouble).
>
That's funny, for me this is the main reason to use ZFS... What really sets
ZFS apart from everything else is the lack of trouble and its resilience to
failures.  We used to have lots and lots of ZFS filesystems at work, and
I've been using ZFS exclusively at home ever since. I have run into a
non-importable ZFS file system (all drives are there, but it's corrupt and
won't import) only once, and was able to fix it with zdb.

ZFS compression is completely optional, and not even on by default. I've
only tried it once and found it cost too much performance on something
that's not very fast to begin with, but I don't think it affects data
recovery much (the way ZFS stripes data makes traditional data recovery
tools pretty useless anyway).

I personally don't care about purity of implementation, because everything
is a trade-off. The argument really reminds me of Tanenbaums criticism of
the Linux Monokernel (was Tanenbaum right? Maybe, but who cares, because
Linux took over the world, and Minix didn't, so from a practical point of
view, Linus was right). The other one it reminds me of is the criticism of
TCPs "blatant layering violation" (vs OSI). But IMHO the critics were just
jealous of the cool things they couldn't do because they needed to respect
the division of labor along those pesky layers.

I remember reading on one of the Sun engineers blogs (remember when Sun
allowed their engineers to keep blogs about Solaris development? Good
times!) about the heated discussions they had over the ARC and bypassing
the page cache. I don't remember the actual arguments for it, but it was
certainly not a decision that was made out of laziness.

Performance wise, ZFS is not the best, and if that's all you care about,
there are better options. It needs a lot of tuning to just reach
"acceptable" and it definitely does not play well with doing other stuff on
the same machine (it pretty much assumes that your storage appliance is
dedicated). It has particularly abysmal performance when you do lots of
small random writes and then try to read that back in order, but if you
care about not losing your data, it's 2nd to none.

In $JOB-1 (almost 15 years ago), we spent a few weeks stress testing ZFS.
The setup was 24x4TB SATA drives, divided into 2 12 drive raidz2 vdevs or
something like that. All tests were done while it was busy reading/writing
checksummed test files at full speed, 1GB/s or so (see? Performance was not
impressive. We definitely got a lot more out of that with UFS). What was
absolutely stunning was the fact that in all our tests it never served one
bit of corrupted data. It either had it, or it returned an error.

We tortured the storage in any way we could imagine. Wiggled the cables,
yanked out drives, used dd to overwrite random parts or entire drives,
smashed a drive with a hammer and put it back in, put in drives of the
wrong size, put in known bad drives, yanked out drives while it was
resilvering, put drives back into a different slot, overwrote stuff while
it was resilvering. Unplugged the entire storage, plugged the storage into
another machine and imported it, plugged the drives back into the first
machine in a different order. We even did things like "copy a drive onto a
spare with dd, remove 3 drives, and then substitute the spare drive for the
removed one" (this led to some data loss because making the copy was not
atomic, but most of the data was recoverable). And no matter what we did,
it just kept going unless the data was simply not there, and even then, it
kept serving the files (or parts of files) that were available, and
indicated exactly which files were affected by data loss. And when you put
the drives back (or restored the overwritten parts), it would continue as
if nothing had ever happened.

If you've ever wrestled a hardware RAID controller, or VxFS/JFS/HPFS, or
mdadm, you know that none of that can be taken for granted, and that doing
any of the stupid things mentioned above would most likely lead to complete
data loss and/or serving lots of random corrupted data and no way to tell
what had been corrupted.

I remember some performance issues with mmap, but I don't remember how we
fixed it. Probably just sucked it up. Using ZFS was not for maximum
performance.

[-- Attachment #2: Type: text/html, Size: 5261 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  2:22                               ` Rico Pajarola
@ 2021-02-06  2:55                                 ` Larry McVoy
  2021-02-06  3:07                                   ` Will Senn
  2021-02-27  8:54                                   ` Stuart Remphrey
  0 siblings, 2 replies; 70+ messages in thread
From: Larry McVoy @ 2021-02-06  2:55 UTC (permalink / raw)
  To: Rico Pajarola; +Cc: The Eunuchs Hysterical Society

On Fri, Feb 05, 2021 at 06:22:32PM -0800, Rico Pajarola wrote:
> On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall <dave@horsfall.org> wrote:
> > Thanks; I'd heard that ZFS was a compressed file system, so I stopped
> > right there (I had lots of experience in recovering from corrupted RK05s,
> > and didn't need any more trouble).
> >
> That's funny, for me this is the main reason to use ZFS... What really sets
> ZFS apart from everything else is the lack of trouble and its resilience to
> failures.  

I'm gonna call Bill tomorrow and get his take again, that's Bill Moore
one of the two main guys who did ZFS.

This whole thread is sort of silly.  There are the users of ZFS who love
it for what it does for them.  I have no argument with them.  Then there
are the much smaller, depressingly so, group of people who care about OS
design that think ZFS took a step backwards.

I think Dennis might have stepped in here, if he was still with us, and
had some words.

I think Dennis would have brought us back to lets talk about the kernel
and what is right.  ZFS is useful, no doubt, but it is not right from
a kernel guy's point of view.

I miss Dennis.

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  2:06                                           ` Dan Cross
@ 2021-02-06  3:01                                             ` Bakul Shah
  0 siblings, 0 replies; 70+ messages in thread
From: Bakul Shah @ 2021-02-06  3:01 UTC (permalink / raw)
  To: Dan Cross; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]



> On Feb 5, 2021, at 6:06 PM, Dan Cross <crossd@gmail.com> wrote:
> 
> On Fri, Feb 5, 2021 at 7:04 PM Bakul Shah <bakul@iitbombay.org <mailto:bakul@iitbombay.org>> wrote:
> 
>     And you can keep track of mapped pages and read/write from them if
>     necessary even if you have a  separate cache for any compressed pages.
> 
> In essence you pass the ownership of a page's data from a compressed
> page cache to the mapped page. Just like in processor cache coherence
> algorithms there is one source of truth: the current owner of a cached
> unit (line or page or whatever). In other words, the you see via mmap(2)
> will be the exact same page you will see via read(2). Not having actually
> tried this I may have missed corner cases + any practical considerations
> complicating things but *conceptually* this doesn't seem hard.
> 
> In essence, that's what the merged page/buffer cache is all about: file IO (read/write) operations are satisfied from the same memory cache that backs up VM objects. I agree that conceptually it's not that complex; but that's not what ZFS does.

Good that we agree on that at least! This is why it would be good to hear from Bonwick/Moore.


[-- Attachment #2: Type: text/html, Size: 2562 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  2:55                                 ` Larry McVoy
@ 2021-02-06  3:07                                   ` Will Senn
  2021-02-27  8:54                                   ` Stuart Remphrey
  1 sibling, 0 replies; 70+ messages in thread
From: Will Senn @ 2021-02-06  3:07 UTC (permalink / raw)
  To: Larry McVoy, Rico Pajarola; +Cc: The Eunuchs Hysterical Society

On 2/5/21 8:55 PM, Larry McVoy wrote:
> I'm gonna call Bill tomorrow and get his take again, that's Bill Moore
> one of the two main guys who did ZFS.
>
> This whole thread is sort of silly.  There are the users of ZFS who love
> it for what it does for them.  I have no argument with them.  Then there
> are the much smaller, depressingly so, group of people who care about OS
> design that think ZFS took a step backwards.
>
> I think Dennis might have stepped in here, if he was still with us, and
> had some words.
>
> I think Dennis would have brought us back to lets talk about the kernel
> and what is right.  ZFS is useful, no doubt, but it is not right from
> a kernel guy's point of view.
>
> I miss Dennis.

Larry,

Now, after the last 50 emails or so on this topic, I get it :). At 
least, I understand that technical decision were made in creating ZFS 
that were likely ill considered, the impact of those changes dubbed 
insignificant, or even possibly sound design principles ignored. It's 
debatable whether or not these decisions were deliberately contrary to 
good OS design, but I appreciate your and the other experts hanging in 
there and explaining your perspectives. I'm a systems guy, so a lot of 
the detailed OS discussions go over my head, but after enough of them, 
it kinda makes sense. At least, now, I'll pay closer attention to the 
ZFS developer list discussions :).

Will

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-05 20:50                             ` Dave Horsfall
  2021-02-06  0:21                               ` Brad Spencer
  2021-02-06  2:22                               ` Rico Pajarola
@ 2021-02-06  4:55                               ` John Cowan
  2 siblings, 0 replies; 70+ messages in thread
From: John Cowan @ 2021-02-06  4:55 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1340 bytes --]

On Fri, Feb 5, 2021 at 3:50 PM Dave Horsfall <dave@horsfall.org> wrote:


> Ah, the RK05 - evil incarnate.  I mean, a disk drive exposed to the air?
> Out There Somewhere [tm] is a picture of a human hair compared with the
> head clearance; yikes!
>

IIRC, the picture showed that even a smoke particle or a fingerprint is
bigger than the gap, never mind the hair.

Once I had to take a PDP-11 accounting application from $EMPLOYER's NJ
office to a customer in KC.  It fit on a single RK, so I took two identical
disks just in case.  This was in the late 70s.  They were already X-raying
carry-on luggage then, and I had a Bad Feeling about what might happen, so
I said no.

"No problem.  Does it open up?"

"Well, yes, but dust will ruin it."

"Well, what should we do?"

"Trust me?"

"Well -- okay.  Go ahead."

Of course when I got there both RKs were scrambled completely, so I took
the next plane home.  The next weekend, a colleague took the application on
the same flight.  On floppies.  Floppy floppies.  Lots and lots of them.
It installed fine.  Of course it took him all day and much of the following
day.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
You cannot enter here.  Go back to the abyss prepared for you!  Go back!
Fall into the nothingness that awaits you and your Master.  Go! --Gandalf

[-- Attachment #2: Type: text/html, Size: 3916 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-30 23:22                       ` Warner Losh
  2021-01-30 23:31                         ` [TUHS] [SPAM] " Larry McVoy
@ 2021-02-09  2:15                         ` Will Senn
  2021-02-09  2:16                           ` Will Senn
  1 sibling, 1 reply; 70+ messages in thread
From: Will Senn @ 2021-02-09  2:15 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

On 1/30/21 5:22 PM, Warner Losh wrote some pretty good stuff and I 
needed a hook back into the thread...

Today, I found out that on Jan 12, distrowatch transitioned to FreeBSD. 
I guess it's not so far behind the times to scare off the folks that 
track the bleeding edge :)


[-- Attachment #2.1: Type: text/html, Size: 557 bytes --]

[-- Attachment #2.2: alcoppkfalhecjmi.png --]
[-- Type: image/png, Size: 55860 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-09  2:15                         ` [TUHS] " Will Senn
@ 2021-02-09  2:16                           ` Will Senn
  2021-02-09  2:30                             ` Greg 'groggy' Lehey
  0 siblings, 1 reply; 70+ messages in thread
From: Will Senn @ 2021-02-09  2:16 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

On 2/8/21 8:15 PM, Will Senn wrote:
> On 1/30/21 5:22 PM, Warner Losh wrote some pretty good stuff and I 
> needed a hook back into the thread...
>
> Today, I found out that on Jan 12, distrowatch transitioned to 
> FreeBSD. I guess it's not so far behind the times to scare off the 
> folks that track the bleeding edge :)
>
OK. So it was Jan 12, of 2020 - sue me :)

[-- Attachment #2.1: Type: text/html, Size: 966 bytes --]

[-- Attachment #2.2: alcoppkfalhecjmi.png --]
[-- Type: image/png, Size: 55860 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-09  2:16                           ` Will Senn
@ 2021-02-09  2:30                             ` Greg 'groggy' Lehey
  0 siblings, 0 replies; 70+ messages in thread
From: Greg 'groggy' Lehey @ 2021-02-09  2:30 UTC (permalink / raw)
  To: Will Senn; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

On Monday,  8 February 2021 at 20:16:50 -0600, Will Senn wrote:
> On 2/8/21 8:15 PM, Will Senn wrote:
>> On 1/30/21 5:22 PM, Warner Losh wrote some pretty good stuff and I
>> needed a hook back into the thread...
>>
>> Today, I found out that on Jan 12, distrowatch transitioned to
>> FreeBSD. I guess it's not so far behind the times to scare off the
>> folks that track the bleeding edge :)
>>
> OK. So it was Jan 12, of 2020 - sue me :)

No, just proof that we're behind the times :-)

Greg
--
Sent from my desktop computer.
Finger grog@lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-01-31  2:25                       ` Larry McVoy
                                           ` (2 preceding siblings ...)
  2021-02-04  7:46                         ` Chris Torek
@ 2021-02-11 21:01                         ` Angel M Alganza
  3 siblings, 0 replies; 70+ messages in thread
From: Angel M Alganza @ 2021-02-11 21:01 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, Jan 30, 2021 at 06:25:00PM -0800, Larry McVoy wrote:

> BitKeeper has that code and proves that it can be done.

I'm still using BitKeeper and enjoing it a lot.
I like it much better than Git.  :-)

> So good on you that you like ZFS and FreeBSD.  I don't and I don't for
> really good reasons.

I like FreeBSD, and I like ZFS (as a user, without knowing the
internals), but I like Btrfs much better.  I wish it was supported by
the BSD's, mainly OpenBSD.

What are your views on BTRFS, Larry?  I'd like to know.  :-)

Regards.
Ángel

^ permalink raw reply	[flat|nested] 70+ messages in thread

* Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?)
  2021-02-06  2:55                                 ` Larry McVoy
  2021-02-06  3:07                                   ` Will Senn
@ 2021-02-27  8:54                                   ` Stuart Remphrey
  1 sibling, 0 replies; 70+ messages in thread
From: Stuart Remphrey @ 2021-02-27  8:54 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Eunuchs Hysterical Society, Rico Pajarola

[-- Attachment #1: Type: text/plain, Size: 2651 bytes --]

Hi Larry et al,

Just curious about this: was there any feedback from Jeff Bonwick and/or
Bill Moore re the ARC -vs- page cache?

Or would any of the design notes document the reasoning behind the decision?
Surely it must have come up and been justified or got an exception in the
Solaris architecture review (SARC "20 Q's", wasn't it called?) Since AFAICS
it affected Solaris O/S interface (former-)guarantees. Although those notes
are probably lost / inaccessible now...


There's also the monthly OpenZFS leadership meeting, Matt Ahrens et al are
in there: I wonder if they would have access to some of the original
reasoning; how it was justified / why it was permitted.


Dave, btw: check out the high-level structure of ZFS metadata -- every
block is checksummed, and the checksum kept in the parent block (i.e. *not*
kept together), applicable for both data and metadata blocks, and at least
two copies are kept of metadata (but you can request more depending on your
paranoia, see also "ditto" blocks). Compression is optional at the
filesystem level (not held at the pool aka volume level; a pool may contain
multiple filesystems), when compression is enabled if affects future
created files, same if unset or changed to another algorithm; the
filesystem handles a mix of files (blocks, even; I forget offhand) existing
with various or no compression.

Rgds, Stuart.


On Sat, 6 Feb 2021 at 10:56, Larry McVoy <lm@mcvoy.com> wrote:

> On Fri, Feb 05, 2021 at 06:22:32PM -0800, Rico Pajarola wrote:
> > On Fri, Feb 5, 2021 at 12:51 PM Dave Horsfall <dave@horsfall.org> wrote:
> > > Thanks; I'd heard that ZFS was a compressed file system, so I stopped
> > > right there (I had lots of experience in recovering from corrupted
> RK05s,
> > > and didn't need any more trouble).
> > >
> > That's funny, for me this is the main reason to use ZFS... What really
> sets
> > ZFS apart from everything else is the lack of trouble and its resilience
> to
> > failures.
>
> I'm gonna call Bill tomorrow and get his take again, that's Bill Moore
> one of the two main guys who did ZFS.
>
> This whole thread is sort of silly.  There are the users of ZFS who love
> it for what it does for them.  I have no argument with them.  Then there
> are the much smaller, depressingly so, group of people who care about OS
> design that think ZFS took a step backwards.
>
> I think Dennis might have stepped in here, if he was still with us, and
> had some words.
>
> I think Dennis would have brought us back to lets talk about the kernel
> and what is right.  ZFS is useful, no doubt, but it is not right from
> a kernel guy's point of view.
>
> I miss Dennis.
>

[-- Attachment #2: Type: text/html, Size: 3314 bytes --]

^ permalink raw reply	[flat|nested] 70+ messages in thread

end of thread, other threads:[~2021-02-27  8:55 UTC | newest]

Thread overview: 70+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-25 11:10 [TUHS] Favorite unix design principles? Tyler Adams
2021-01-25 12:32 ` Steve Nickolas
2021-01-26  2:06   ` M Douglas McIlroy
2021-01-26  2:53     ` Steve Nickolas
2021-01-26 10:22     ` Tyler Adams
2021-01-26 12:26       ` John P. Linderman
2021-01-26 15:23       ` Clem Cole
2021-01-26 16:00         ` Niklas Karlsson
2021-01-26 16:13           ` Adam Thornton
     [not found]       ` <CAKH6PiXKjksEpQOMMMQTbcsMvX2thz3WzqjoRWJAsXnZ4Eq_iQ@mail.gmail.com>
2021-01-30 19:01         ` Tyler Adams
2021-01-30 19:50           ` Jon Steinhart
2021-01-30 20:06             ` Tyler Adams
2021-01-30 21:28               ` Clem Cole
2021-01-30 21:42                 ` Dave Horsfall
2021-01-30 21:45                 ` Tyler Adams
2021-01-30 22:31                   ` Larry McVoy
2021-01-30 22:28                 ` Larry McVoy
2021-01-30 23:11                   ` [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) Greg 'groggy' Lehey
2021-01-30 23:17                     ` Larry McVoy
2021-01-30 23:22                       ` Warner Losh
2021-01-30 23:31                         ` [TUHS] [SPAM] " Larry McVoy
2021-01-30 23:37                           ` Jon Steinhart
2021-01-30 23:54                             ` Larry McVoy
2021-01-31 12:23                               ` [TUHS] [SPAM] Re: FreeBSD behind the times? Dermot Tynan
2021-01-31  0:00                             ` [TUHS] [SPAM] Re: FreeBSD behind the times? (was: Favorite unix design principles?) Bakul Shah
2021-02-09  2:15                         ` [TUHS] " Will Senn
2021-02-09  2:16                           ` Will Senn
2021-02-09  2:30                             ` Greg 'groggy' Lehey
2021-01-31  0:39                     ` Steve Nickolas
2021-01-31  1:47                     ` Will Senn
2021-01-31  2:25                       ` Larry McVoy
2021-01-31  2:52                         ` Will Senn
2021-01-31  3:00                           ` Larry McVoy
2021-01-31  3:06                             ` Will Senn
2021-01-31  3:32                               ` John Cowan
2021-02-04  5:43                         ` Dave Horsfall
2021-02-04  6:10                           ` Angus Robinson
2021-02-04  7:46                             ` Andy Kosela
2021-02-04 22:25                             ` Dave Horsfall
2021-02-04 15:45                           ` Will Senn
2021-02-04 16:03                             ` Henry Bent
2021-02-04 16:32                             ` Dan Cross
2021-02-04 16:49                               ` Will Senn
2021-02-04 17:46                               ` Larry McVoy
2021-02-04 18:41                               ` Bakul Shah
2021-02-04 22:28                                 ` George Michaelson
2021-02-04 22:41                                   ` Bakul Shah
2021-02-05  0:33                                   ` Larry McVoy
2021-02-05  5:17                                     ` Bakul Shah
2021-02-05 14:18                                       ` Larry McVoy
2021-02-05 18:16                                         ` Warner Losh
2021-02-05 18:21                                         ` ron minnich
2021-02-06  0:03                                         ` Bakul Shah
2021-02-06  2:06                                           ` Dan Cross
2021-02-06  3:01                                             ` Bakul Shah
2021-02-06  1:18                                         ` John Gilmore
2021-02-06  1:43                                           ` joe mcguckin
2021-02-06  1:55                                           ` Bakul Shah
2021-02-05 20:50                             ` Dave Horsfall
2021-02-06  0:21                               ` Brad Spencer
2021-02-06  2:22                               ` Rico Pajarola
2021-02-06  2:55                                 ` Larry McVoy
2021-02-06  3:07                                   ` Will Senn
2021-02-27  8:54                                   ` Stuart Remphrey
2021-02-06  4:55                               ` John Cowan
2021-02-04  7:46                         ` Chris Torek
2021-02-04 15:47                           ` Will Senn
2021-02-11 21:01                         ` Angel M Alganza
2021-01-30 23:09                 ` [TUHS] Favorite unix design principles? John Cowan
2021-01-30 23:22                   ` Jon Steinhart

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).