The Unix Heritage Society mailing list
 help / color / Atom feed
* Re: [TUHS] Tech Sq elevator (Was: screen editors)
@ 2020-01-12 13:43 Doug McIlroy
  2020-01-12 16:56 ` Warner Losh
  0 siblings, 1 reply; 21+ messages in thread
From: Doug McIlroy @ 2020-01-12 13:43 UTC (permalink / raw)
  To: tuhs

Mention of elevators at Tech Square reminds me of visiting there
to see the Lisp machine. I was struck by cultural differences.

At the time we were using Jerqs, where multiple windows ran
like multiple time-sharing sessions. To me that behavior was a
no-brainer. Surprisingly, Lisp-machine windows didn't work that
way; only the user-selected active window got processor time.

The biggest difference was emacs, which no one used at Bell
Labs. Emacs, of course was native to the Lisp machine and
provided a powerful and smoothly extensible environment. For
example, its reflective ability made it easy to display a
list of its commands. "Call elevator" stood out amng mundane
programmering actions like cut, paste and run.

After scrolling through the command list, I wondered how long
it was and asked to have it counted. Easy, I thought, just
pass it to a wc-like program. But "just pass it" and "wc-like"
were not givens as they are in Unix culture.  It took several
minutes for the gurus to do it--without leaving emacs, if I
remember right.

Doug

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 13:43 [TUHS] Tech Sq elevator (Was: screen editors) Doug McIlroy
@ 2020-01-12 16:56 ` Warner Losh
  2020-01-12 17:21   ` markus schnalke
  2020-01-12 20:25   ` Kevin Bowling
  0 siblings, 2 replies; 21+ messages in thread
From: Warner Losh @ 2020-01-12 16:56 UTC (permalink / raw)
  To: Doug McIlroy; +Cc: The Eunuchs Hysterical Society


[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]

On Sun, Jan 12, 2020 at 6:44 AM Doug McIlroy <doug@cs.dartmouth.edu> wrote:

> Mention of elevators at Tech Square reminds me of visiting there
> to see the Lisp machine. I was struck by cultural differences.
>
> At the time we were using Jerqs, where multiple windows ran
> like multiple time-sharing sessions. To me that behavior was a
> no-brainer. Surprisingly, Lisp-machine windows didn't work that
> way; only the user-selected active window got processor time.
>
> The biggest difference was emacs, which no one used at Bell
> Labs. Emacs, of course was native to the Lisp machine and
> provided a powerful and smoothly extensible environment. For
> example, its reflective ability made it easy to display a
> list of its commands. "Call elevator" stood out amng mundane
> programmering actions like cut, paste and run.
>
> After scrolling through the command list, I wondered how long
> it was and asked to have it counted. Easy, I thought, just
> pass it to a wc-like program. But "just pass it" and "wc-like"
> were not givens as they are in Unix culture.  It took several
> minutes for the gurus to do it--without leaving emacs, if I
> remember right.
>

It should have been something like  (list-length (command-list-fn)) but
I'll bet that ? was bound to a complicated function that just displayed the
results and didn't properly abstract out the UI (printing) from the data
collection (getting a list), which is what made it so hard. I've had so
many gnu emacs experiences like this over the years, but to the community's
credit, there's fewer and fewer as time goes by.

ObUnix: This shows the power of having the right abstractions and being
disciplined to code to those ideal abstractions any time there might be
reuse...

Warner

[-- Attachment #2: Type: text/html, Size: 2245 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 16:56 ` Warner Losh
@ 2020-01-12 17:21   ` markus schnalke
  2020-01-12 20:25   ` Kevin Bowling
  1 sibling, 0 replies; 21+ messages in thread
From: markus schnalke @ 2020-01-12 17:21 UTC (permalink / raw)
  To: tuhs

Hoi.

[2020-01-12 09:56] Warner Losh <imp@bsdimp.com>
>
> It should have been something like  (list-length (command-list-fn)) but I'll
> bet that ? was bound to a complicated function that just displayed the results
> and didn't properly abstract out the UI (printing) from the data collection
> (getting a list), which is what made it so hard. I've had so many gnu emacs
> experiences like this over the years, but to the community's credit, there's
> fewer and fewer as time goes by.
> 
> ObUnix: This shows the power of having the right abstractions and being
> disciplined to code to those ideal abstractions any time there might be
> reuse...

True words.

These concepts (what you call abstractions) is no optional geek
pleasure but the most important information for everyone using the
system or language. It's important to teach them, make them explicit,
deal with them, substract points in exams for correct solutions that
have bad style, keep the libraries and books on a high level, ...

This is one of the lesser regarded aspects of what Unix did really
well.


meillo

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 16:56 ` Warner Losh
  2020-01-12 17:21   ` markus schnalke
@ 2020-01-12 20:25   ` Kevin Bowling
  2020-01-12 20:32     ` Larry McVoy
  2020-01-12 20:34     ` Jon Steinhart
  1 sibling, 2 replies; 21+ messages in thread
From: Kevin Bowling @ 2020-01-12 20:25 UTC (permalink / raw)
  To: Warner Losh; +Cc: The Eunuchs Hysterical Society, Doug McIlroy

This is kind of illustrative of the '60s acid trip that perpetuates in
programming "Everything's a string maaaaan".  The output is seen as
truth because the representation is for some reason too hard to get at
or too hard to cascade through the system.

There's a total comedy of work going on in the unix way of a wc
pipeline versus calling a length function on a list.  Nonetheless, the
unix pipeline was and is often magnitude easier for a single user to
get at.  This kind of thing is amusing and endearing to me about our
profession in modern day.

Regards,
Kevin

On Sun, Jan 12, 2020 at 9:57 AM Warner Losh <imp@bsdimp.com> wrote:
>
>
>
> On Sun, Jan 12, 2020 at 6:44 AM Doug McIlroy <doug@cs.dartmouth.edu> wrote:
>>
>> Mention of elevators at Tech Square reminds me of visiting there
>> to see the Lisp machine. I was struck by cultural differences.
>>
>> At the time we were using Jerqs, where multiple windows ran
>> like multiple time-sharing sessions. To me that behavior was a
>> no-brainer. Surprisingly, Lisp-machine windows didn't work that
>> way; only the user-selected active window got processor time.
>>
>> The biggest difference was emacs, which no one used at Bell
>> Labs. Emacs, of course was native to the Lisp machine and
>> provided a powerful and smoothly extensible environment. For
>> example, its reflective ability made it easy to display a
>> list of its commands. "Call elevator" stood out amng mundane
>> programmering actions like cut, paste and run.
>>
>> After scrolling through the command list, I wondered how long
>> it was and asked to have it counted. Easy, I thought, just
>> pass it to a wc-like program. But "just pass it" and "wc-like"
>> were not givens as they are in Unix culture.  It took several
>> minutes for the gurus to do it--without leaving emacs, if I
>> remember right.
>
>
> It should have been something like  (list-length (command-list-fn)) but I'll bet that ? was bound to a complicated function that just displayed the results and didn't properly abstract out the UI (printing) from the data collection (getting a list), which is what made it so hard. I've had so many gnu emacs experiences like this over the years, but to the community's credit, there's fewer and fewer as time goes by.
>
> ObUnix: This shows the power of having the right abstractions and being disciplined to code to those ideal abstractions any time there might be reuse...
>
> Warner

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:25   ` Kevin Bowling
@ 2020-01-12 20:32     ` Larry McVoy
  2020-01-12 20:34     ` Jon Steinhart
  1 sibling, 0 replies; 21+ messages in thread
From: Larry McVoy @ 2020-01-12 20:32 UTC (permalink / raw)
  To: Kevin Bowling; +Cc: The Eunuchs Hysterical Society, Doug McIlroy

I'll counter with Linux's /proc where everything is a string.  It's super
pleasant and I say that coming from SysV /proc where it was all a mess.

If performance is the metric, everything is a string is not so much.
If performance is not the high order bit, a /proc that you can read
with cat or grep is super pleasant.

On Sun, Jan 12, 2020 at 01:25:13PM -0700, Kevin Bowling wrote:
> This is kind of illustrative of the '60s acid trip that perpetuates in
> programming "Everything's a string maaaaan".  The output is seen as
> truth because the representation is for some reason too hard to get at
> or too hard to cascade through the system.
> 
> There's a total comedy of work going on in the unix way of a wc
> pipeline versus calling a length function on a list.  Nonetheless, the
> unix pipeline was and is often magnitude easier for a single user to
> get at.  This kind of thing is amusing and endearing to me about our
> profession in modern day.
> 
> Regards,
> Kevin
> 
> On Sun, Jan 12, 2020 at 9:57 AM Warner Losh <imp@bsdimp.com> wrote:
> >
> >
> >
> > On Sun, Jan 12, 2020 at 6:44 AM Doug McIlroy <doug@cs.dartmouth.edu> wrote:
> >>
> >> Mention of elevators at Tech Square reminds me of visiting there
> >> to see the Lisp machine. I was struck by cultural differences.
> >>
> >> At the time we were using Jerqs, where multiple windows ran
> >> like multiple time-sharing sessions. To me that behavior was a
> >> no-brainer. Surprisingly, Lisp-machine windows didn't work that
> >> way; only the user-selected active window got processor time.
> >>
> >> The biggest difference was emacs, which no one used at Bell
> >> Labs. Emacs, of course was native to the Lisp machine and
> >> provided a powerful and smoothly extensible environment. For
> >> example, its reflective ability made it easy to display a
> >> list of its commands. "Call elevator" stood out amng mundane
> >> programmering actions like cut, paste and run.
> >>
> >> After scrolling through the command list, I wondered how long
> >> it was and asked to have it counted. Easy, I thought, just
> >> pass it to a wc-like program. But "just pass it" and "wc-like"
> >> were not givens as they are in Unix culture.  It took several
> >> minutes for the gurus to do it--without leaving emacs, if I
> >> remember right.
> >
> >
> > It should have been something like  (list-length (command-list-fn)) but I'll bet that ? was bound to a complicated function that just displayed the results and didn't properly abstract out the UI (printing) from the data collection (getting a list), which is what made it so hard. I've had so many gnu emacs experiences like this over the years, but to the community's credit, there's fewer and fewer as time goes by.
> >
> > ObUnix: This shows the power of having the right abstractions and being disciplined to code to those ideal abstractions any time there might be reuse...
> >
> > Warner

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:25   ` Kevin Bowling
  2020-01-12 20:32     ` Larry McVoy
@ 2020-01-12 20:34     ` Jon Steinhart
  2020-01-12 20:40       ` Kevin Bowling
  1 sibling, 1 reply; 21+ messages in thread
From: Jon Steinhart @ 2020-01-12 20:34 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Kevin Bowling writes:
> This is kind of illustrative of the '60s acid trip that perpetuates in
> programming "Everything's a string maaaaan".  The output is seen as
> truth because the representation is for some reason too hard to get at
> or too hard to cascade through the system.
>
> There's a total comedy of work going on in the unix way of a wc
> pipeline versus calling a length function on a list.  Nonetheless, the
> unix pipeline was and is often magnitude easier for a single user to
> get at.  This kind of thing is amusing and endearing to me about our
> profession in modern day.
>
> Regards,
> Kevin

Can you please elaborate?  I read your post, and while I can see that it
contains English words I can't make any sense out of what you said.

Thanks,
	Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:34     ` Jon Steinhart
@ 2020-01-12 20:40       ` Kevin Bowling
  2020-01-12 20:44         ` Jon Steinhart
  0 siblings, 1 reply; 21+ messages in thread
From: Kevin Bowling @ 2020-01-12 20:40 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

I honestly can't tell if this is genius level snark :) in case you're
sincere we generally go to great lengths to build up data types and
structures (in C lingo) when programming only to tear those useful
attributes off often at inopportune times.  Basically type
systems/type safety have been too expensive or too difficult to use
through history.

Think of sitting at an SQL prompt as a counterpoint.  You can pretty
easily get at the underlying representation and relationships of the
data and the output is just a side effect.  Not saying SQL is the
ultimate answer, just that most people have a bit of experience with
it and UNIX so can mentally compare the two for themselves and see the
pros and cons to preserving the underlying representations.

Regards,
Kevin

On Sun, Jan 12, 2020 at 1:34 PM Jon Steinhart <jon@fourwinds.com> wrote:
>
> Kevin Bowling writes:
> > This is kind of illustrative of the '60s acid trip that perpetuates in
> > programming "Everything's a string maaaaan".  The output is seen as
> > truth because the representation is for some reason too hard to get at
> > or too hard to cascade through the system.
> >
> > There's a total comedy of work going on in the unix way of a wc
> > pipeline versus calling a length function on a list.  Nonetheless, the
> > unix pipeline was and is often magnitude easier for a single user to
> > get at.  This kind of thing is amusing and endearing to me about our
> > profession in modern day.
> >
> > Regards,
> > Kevin
>
> Can you please elaborate?  I read your post, and while I can see that it
> contains English words I can't make any sense out of what you said.
>
> Thanks,
>         Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:40       ` Kevin Bowling
@ 2020-01-12 20:44         ` Jon Steinhart
  2020-01-12 21:03           ` Kevin Bowling
  2020-01-12 21:41           ` [TUHS] Tech Sq elevator (Was: screen editors) Bakul Shah
  0 siblings, 2 replies; 21+ messages in thread
From: Jon Steinhart @ 2020-01-12 20:44 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Kevin Bowling writes:
> I honestly can't tell if this is genius level snark :) in case you're
> sincere we generally go to great lengths to build up data types and
> structures (in C lingo) when programming only to tear those useful
> attributes off often at inopportune times.  Basically type
> systems/type safety have been too expensive or too difficult to use
> through history.
>
> Think of sitting at an SQL prompt as a counterpoint.  You can pretty
> easily get at the underlying representation and relationships of the
> data and the output is just a side effect.  Not saying SQL is the
> ultimate answer, just that most people have a bit of experience with
> it and UNIX so can mentally compare the two for themselves and see the
> pros and cons to preserving the underlying representations.
>
> Regards,
> Kevin
>
> On Sun, Jan 12, 2020 at 1:34 PM Jon Steinhart <jon@fourwinds.com> wrote:
> >
> > Kevin Bowling writes:
> > > This is kind of illustrative of the '60s acid trip that perpetuates in
> > > programming "Everything's a string maaaaan".  The output is seen as
> > > truth because the representation is for some reason too hard to get at
> > > or too hard to cascade through the system.
> > >
> > > There's a total comedy of work going on in the unix way of a wc
> > > pipeline versus calling a length function on a list.  Nonetheless, the
> > > unix pipeline was and is often magnitude easier for a single user to
> > > get at.  This kind of thing is amusing and endearing to me about our
> > > profession in modern day.
> > >
> > > Regards,
> > > Kevin
> >
> > Can you please elaborate?  I read your post, and while I can see that it
> > contains English words I can't make any sense out of what you said.
> >
> > Thanks,
> >         Jon

I wasn't being snarky.  You said

  "The output is seen as truth because the representation is for some
  reason too hard to get at or too hard to cascade through the system."

I honestly have no idea what that means.

Likewise, 

  "There's a total comedy of work going on in the unix way of
  a wc pipeline versus calling a length function on a list."

I just don't know what you mean.

Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:44         ` Jon Steinhart
@ 2020-01-12 21:03           ` Kevin Bowling
  2020-01-12 21:37             ` [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ] Jon Steinhart
  2020-01-12 21:41           ` [TUHS] Tech Sq elevator (Was: screen editors) Bakul Shah
  1 sibling, 1 reply; 21+ messages in thread
From: Kevin Bowling @ 2020-01-12 21:03 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

On Sun, Jan 12, 2020 at 1:45 PM Jon Steinhart <jon@fourwinds.com> wrote:
>
> Kevin Bowling writes:
> > I honestly can't tell if this is genius level snark :) in case you're
> > sincere we generally go to great lengths to build up data types and
> > structures (in C lingo) when programming only to tear those useful
> > attributes off often at inopportune times.  Basically type
> > systems/type safety have been too expensive or too difficult to use
> > through history.
> >
> > Think of sitting at an SQL prompt as a counterpoint.  You can pretty
> > easily get at the underlying representation and relationships of the
> > data and the output is just a side effect.  Not saying SQL is the
> > ultimate answer, just that most people have a bit of experience with
> > it and UNIX so can mentally compare the two for themselves and see the
> > pros and cons to preserving the underlying representations.
> >
> > Regards,
> > Kevin
> >
> > On Sun, Jan 12, 2020 at 1:34 PM Jon Steinhart <jon@fourwinds.com> wrote:
> > >
> > > Kevin Bowling writes:
> > > > This is kind of illustrative of the '60s acid trip that perpetuates in
> > > > programming "Everything's a string maaaaan".  The output is seen as
> > > > truth because the representation is for some reason too hard to get at
> > > > or too hard to cascade through the system.
> > > >
> > > > There's a total comedy of work going on in the unix way of a wc
> > > > pipeline versus calling a length function on a list.  Nonetheless, the
> > > > unix pipeline was and is often magnitude easier for a single user to
> > > > get at.  This kind of thing is amusing and endearing to me about our
> > > > profession in modern day.
> > > >
> > > > Regards,
> > > > Kevin
> > >
> > > Can you please elaborate?  I read your post, and while I can see that it
> > > contains English words I can't make any sense out of what you said.
> > >
> > > Thanks,
> > >         Jon
>
> I wasn't being snarky.  You said
>
>   "The output is seen as truth because the representation is for some
>   reason too hard to get at or too hard to cascade through the system."
>
> I honestly have no idea what that means.

If the SQL prompt example did not clarify you are welcome to go one on
one if this is something you think is curious to you, I think I've
explained the point I was making adequately for a general audience.

>
> Likewise,
>
>   "There's a total comedy of work going on in the unix way of
>   a wc pipeline versus calling a length function on a list."
>
> I just don't know what you mean.
>

Reason through what happens in a shell pipeline, the more detail the
better.  A quick nudge is fork/exec, what happens in the kernel, what
happens in page tables, what happens at the buffered output, tty layer
etc.  Very few people actually understand all these steps even at a
general level in modern systems.

If you had a grocery list on a piece of paper would you
a) count the lines or read it directly off the last line number on the
paper if it is numbered

b) copy each character letter by letter to a new piece of equipment
(say, a word processor), until you encounter a special character that
happens to be represented as a space on the screen, increment a
counter, repeat until you reach another special character, output the
result and then destroy and throw away both the list and the word
processor equipment.

This kind of thing doesn't really matter in the small or at all for
performance because computers are fast.  But most programming bugs in
the large eventually boil down to some kind of misunderstanding where
the representation was lost and recast in a way that does not make
sense.

Regards,
Kevin

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-12 21:03           ` Kevin Bowling
@ 2020-01-12 21:37             ` Jon Steinhart
       [not found]               ` <CAEoi9W4fXLaTRM1mv4wnVbifCFBEw_iKL9cds8ds-FBRTwM-=g@mail.gmail.com>
  0 siblings, 1 reply; 21+ messages in thread
From: Jon Steinhart @ 2020-01-12 21:37 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Kevin Bowling writes:
> On Sun, Jan 12, 2020 at 1:45 PM Jon Steinhart <jon@fourwinds.com> wrote:
> >
> > Kevin Bowling writes:
> > > I honestly can't tell if this is genius level snark :) in case you're
> > > sincere we generally go to great lengths to build up data types and
> > > structures (in C lingo) when programming only to tear those useful
> > > attributes off often at inopportune times.  Basically type
> > > systems/type safety have been too expensive or too difficult to use
> > > through history.
> > >
> > > Think of sitting at an SQL prompt as a counterpoint.  You can pretty
> > > easily get at the underlying representation and relationships of the
> > > data and the output is just a side effect.  Not saying SQL is the
> > > ultimate answer, just that most people have a bit of experience with
> > > it and UNIX so can mentally compare the two for themselves and see the
> > > pros and cons to preserving the underlying representations.
> > >
> > > Regards,
> > > Kevin
> > >
> > > On Sun, Jan 12, 2020 at 1:34 PM Jon Steinhart <jon@fourwinds.com> wrote:
> > > >
> > > > Kevin Bowling writes:
> > > > > This is kind of illustrative of the '60s acid trip that perpetuates in
> > > > > programming "Everything's a string maaaaan".  The output is seen as
> > > > > truth because the representation is for some reason too hard to get at
> > > > > or too hard to cascade through the system.
> > > > >
> > > > > There's a total comedy of work going on in the unix way of a wc
> > > > > pipeline versus calling a length function on a list.  Nonetheless, the
> > > > > unix pipeline was and is often magnitude easier for a single user to
> > > > > get at.  This kind of thing is amusing and endearing to me about our
> > > > > profession in modern day.
> > > > >
> > > > > Regards,
> > > > > Kevin
> > > >
> > > > Can you please elaborate?  I read your post, and while I can see that it
> > > > contains English words I can't make any sense out of what you said.
> > > >
> > > > Thanks,
> > > >         Jon
> >
> > I wasn't being snarky.  You said
> >
> >   "The output is seen as truth because the representation is for some
> >   reason too hard to get at or too hard to cascade through the system."
> >
> > I honestly have no idea what that means.
>
> If the SQL prompt example did not clarify you are welcome to go one on
> one if this is something you think is curious to you, I think I've
> explained the point I was making adequately for a general audience.
>
> >
> > Likewise,
> >
> >   "There's a total comedy of work going on in the unix way of
> >   a wc pipeline versus calling a length function on a list."
> >
> > I just don't know what you mean.
> >
>
> Reason through what happens in a shell pipeline, the more detail the
> better.  A quick nudge is fork/exec, what happens in the kernel, what
> happens in page tables, what happens at the buffered output, tty layer
> etc.  Very few people actually understand all these steps even at a
> general level in modern systems.
>
> If you had a grocery list on a piece of paper would you
> a) count the lines or read it directly off the last line number on the
> paper if it is numbered
>
> b) copy each character letter by letter to a new piece of equipment
> (say, a word processor), until you encounter a special character that
> happens to be represented as a space on the screen, increment a
> counter, repeat until you reach another special character, output the
> result and then destroy and throw away both the list and the word
> processor equipment.
>
> This kind of thing doesn't really matter in the small or at all for
> performance because computers are fast.  But most programming bugs in
> the large eventually boil down to some kind of misunderstanding where
> the representation was lost and recast in a way that does not make
> sense.
>
> Regards,
> Kevin

OK, I have trouble correlating this with your original post but I think
that I understand it well enough to comment.

I agree that it is a problem that very few people understand what's going on
inside anything today from a toaster to a computer.  On the computer end of
things this concerns me a lot and improving the quality of education in this
area is one of my main late-in-life missions.  I'm under the illusion that
I've helped some based on comments that I've received from people who have
tracked me down and let me know how much the information in my book helped them.

On to your example...

If I had a grocery list on a piece of paper I would count the lines because I
don't number my grocery lists.  I'm going to guess that few people do.  So, I
would count the lines in my head and remember the result.  This is pretty much
equivalent to what happens when something is piped into wc.

I don't see much difference between a and b in your example.  That's because
when I count up the number of lines in the list, I am making a temporary copy
of the list in my head and then forgetting what was on the list (which may
account for the late night trip to the grocery store a couple of days ago).

So I think that the point that you're trying to make, correct me if I'm wrong,
is that if lists just knew how long they were you could just ask and that it
would be more efficient.

While that may be true, it sort of assume that this is something so common that
the extra overhead for line counting should be part of every list.  And it doesn't
address the issue that while maybe you want a line count I may want a character
count or a count of all lines that begin with the letter A.  Limiting this example
to just line numbers ignores the fact that different people might want different
information that can't all be predicted in advance and built into every program.

It also seems to me that the root problem here is that the data in the original
example was in an emacs-specific format instead of the default UNIX text file
format.

The beauty of UNIX is that with a common file format one can create tools that
process data in different ways that then operate on all data.  Yes, it's not as
efficient as creating a custom tool for a particular purpose, but is much better
for casual use.  One can always create a special purpose tool if a particular
use becomes so prevalent that the extra efficiency is worthwhile.  If you're not
familiar with it, find a copy of the Communications of the ACM issue where Knuth
presented a clever search algorithm (if I remember correctly) and McIlroy did a
critique.  One of the things that Doug pointed out what that while Don's code was
more efficient, by creating a new pile of special-purpose code he introduced bugs.

Many people have claimed, incorrectly in my opinion, that this model fails in the
modern era because it only works on text data.  They change the subject when I
point out that ImageMagick works on binary data.  And, there are now stream
processing utilities for JSON data and such that show that the UNIX model still
works IF you understand it and know how to use it.

I don't agree with your closing comment about "most programming bugs".  Do you
have any data to support this or is it just an opinion?  My opinion is that most
programming bugs today result from total incompetence as one can prety much get
a computer science degree today without every learning that programs run on
computers or what a computer is.  That's something I'm trying to change, but it's
probably a lost cause.  A long topic, and not necessarily appropriate for this list.

Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 20:44         ` Jon Steinhart
  2020-01-12 21:03           ` Kevin Bowling
@ 2020-01-12 21:41           ` Bakul Shah
  2020-01-12 21:47             ` Jon Steinhart
  1 sibling, 1 reply; 21+ messages in thread
From: Bakul Shah @ 2020-01-12 21:41 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sun, 12 Jan 2020 12:44:30 -0800 Jon Steinhart <jon@fourwinds.com> wrote:
>
> I wasn't being snarky.  You said
>
>   "The output is seen as truth because the representation is for some
>   reason too hard to get at or too hard to cascade through the system."
>
> I honestly have no idea what that means.
>
> Likewise, 
>
>   "There's a total comedy of work going on in the unix way of
>   a wc pipeline versus calling a length function on a list."
>
> I just don't know what you mean.

May be this is not what Kevin meant but the way I interpreted
his message: smushing everything down to strings for output as
in unix pipelines loses any underlying structure which then
may have to be rediscovered by the next program in the
pipeline by parsing. This is slower, indirect and errorprone.
If one can communicate a structured representation of data between
components that would be better.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors)
  2020-01-12 21:41           ` [TUHS] Tech Sq elevator (Was: screen editors) Bakul Shah
@ 2020-01-12 21:47             ` Jon Steinhart
  0 siblings, 0 replies; 21+ messages in thread
From: Jon Steinhart @ 2020-01-12 21:47 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Bakul Shah writes:
> On Sun, 12 Jan 2020 12:44:30 -0800 Jon Steinhart <jon@fourwinds.com> wrote:
> >
> > I wasn't being snarky.  You said
> >
> >   "The output is seen as truth because the representation is for some
> >   reason too hard to get at or too hard to cascade through the system."
> >
> > I honestly have no idea what that means.
> >
> > Likewise, 
> >
> >   "There's a total comedy of work going on in the unix way of
> >   a wc pipeline versus calling a length function on a list."
> >
> > I just don't know what you mean.
>
> May be this is not what Kevin meant but the way I interpreted
> his message: smushing everything down to strings for output as
> in unix pipelines loses any underlying structure which then
> may have to be rediscovered by the next program in the
> pipeline by parsing. This is slower, indirect and errorprone.
> If one can communicate a structured representation of data between
> components that would be better.

Ah.  So lines IS the structure that a lot of people want.  As I just
said in a post, the JSON tools are a good example of communicating
structured data although I would have done them differently.

Jon

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
       [not found]                 ` <CAEoi9W6LedGGjWPO=ZgZzVdGLqs8drhqcWkvA_DfKTOtMDgegQ@mail.gmail.com>
@ 2020-01-13 23:46                   ` Dan Cross
  2020-01-14 23:17                     ` Kevin Bowling
  2020-01-18 15:45                     ` Michael Parson
  0 siblings, 2 replies; 21+ messages in thread
From: Dan Cross @ 2020-01-13 23:46 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society


[-- Attachment #1: Type: text/plain, Size: 5654 bytes --]

[Resending as this got squashed a few days ago. Jon, sorry for the
duplicate. Again.]

On Sun, Jan 12, 2020 at 4:38 PM Jon Steinhart <jon@fourwinds.com> wrote:

> [snip]
> So I think that the point that you're trying to make, correct me if I'm
> wrong,
> is that if lists just knew how long they were you could just ask and that
> it
> would be more efficient.
>

What I understood was that, by translating into a lowest-common-denominator
format like text, one loses much of the semantic information implicit in a
richer representation. In particular, much of the internal knowledge (like
type information...) is lost during translation and presentation. Put
another way, with text as usually used by the standard suite of Unix tools,
type information is implicit, rather than explicit. I took this to be less
an issue of efficiency and more of expressiveness.

It is, perhaps, important to remember that Unix works so well because of
heavy use of convention: to take Doug's example, the total number of
commands might be easy to find with `wc` because one assumes each command
is presented on a separate line, with no gaudy header or footer information
or extraneous explanatory text.

This sort of convention, where each logical "record" is a line by itself,
is pervasive on Unix systems, but is not guaranteed. In some sense, those
representations are fragile: a change in output might break something else
downstream in the pipeline, whereas a representation that captures more
semantic meaning is more robust in the face of change but, as in Doug's
example, often harder to use. The Lisp Machine had all sorts of cool
information in the image and a good Lisp hacker familiar with the machine's
structures could write programs to extract and present that information.
But doing so wasn't trivial in the way that '| wc -l' in response to a
casual query is.

While that may be true, it sort of assume that this is something so common
> that
> the extra overhead for line counting should be part of every list.  And it
> doesn't
> address the issue that while maybe you want a line count I may want a
> character
> count or a count of all lines that begin with the letter A.  Limiting this
> example
> to just line numbers ignores the fact that different people might want
> different
> information that can't all be predicted in advance and built into every
> program.
>

This I think illustrates an important point: Unix conventions worked well
enough in practice that many interesting tasks were not just tractable, but
easy and in some cases trivial. Combining programs was easy via pipelines.
Harder stuff involving more elaborate data formats was possible, but, well,
harder and required more involved programming. By contrast, the Lisp
machine could do the hard stuff, but the simple stuff also required
non-trivial programming.

The SQL database point was similarly interesting: having written programs
to talk to relational databases, yes, one can do powerful things: but the
amount of programming required is significant at a minimum and often
substantial.


> It also seems to me that the root problem here is that the data in the
> original
> example was in an emacs-specific format instead of the default UNIX text
> file
> format.
>
> The beauty of UNIX is that with a common file format one can create tools
> that
> process data in different ways that then operate on all data.  Yes, it's
> not as
> efficient as creating a custom tool for a particular purpose, but is much
> better
> for casual use.  One can always create a special purpose tool if a
> particular
> use becomes so prevalent that the extra efficiency is worthwhile.  If
> you're not
> familiar with it, find a copy of the Communications of the ACM issue where
> Knuth
> presented a clever search algorithm (if I remember correctly) and McIlroy
> did a
> critique.  One of the things that Doug pointed out what that while Don's
> code was
> more efficient, by creating a new pile of special-purpose code he
> introduced bugs.
>

The flip side is that one often loses information in the conversion to
text: yes, there are structured data formats with text serializations that
can preserve the lost information, but consuming and processing those with
the standard Unix tools can be messy. Seemingly trivial changes in text,
like reversing the order of two fields, can break programs that consume
that data. Data must be suitable for pipelining (e.g., perhaps free-form
text must be free of newlines or something). These are all limitations.
Where I think the argument went awry is in not recognizing that very often
those problems, while real, are at least tractable.

Many people have claimed, incorrectly in my opinion, that this model fails
> in the
> modern era because it only works on text data.  They change the subject
> when I
> point out that ImageMagick works on binary data.  And, there are now stream
> processing utilities for JSON data and such that show that the UNIX model
> still
> works IF you understand it and know how to use it.
>

Certainly. I think you hit the nail on the head with the proviso that one
must _understand_ the Unix model and how to use it. If one does so, it's
very powerful indeed, and it really is applicable more often than not. But
it is not a panacea (not that anyone suggested it is). As an example, how
do I apply an unmodified `grep` to arbitrary JSON data (which may span more
than one line)? Perhaps there is a way (I can imagine a 'record2line'
program that consumes a single JSON object and emits it as a syntactically
valid one-liner...) but I can also imagine all sorts of ways that might go
wrong.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 6962 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-13 23:46                   ` Dan Cross
@ 2020-01-14 23:17                     ` Kevin Bowling
  2020-01-18 15:45                     ` Michael Parson
  1 sibling, 0 replies; 21+ messages in thread
From: Kevin Bowling @ 2020-01-14 23:17 UTC (permalink / raw)
  To: Dan Cross; +Cc: The Eunuchs Hysterical Society

Thanks Dan, this message is exactly what I was trying to express.  To
piggyback on some of your idea, one limitation on getting at the
representation is the simplicity of the shell.  If you look back at
The Mother of All Demos, or environments like the LispM, Project
Oberon, Alto, even the BLIT it seems to me like there may be ways to
harmonize the underlying representation with ({use?, implementation?,
mental model?} What is it that makes the unix shell and pipelines so
desirable and unchanged?

One thing I notice is that as my millenial peers learn, adopt, and use
unix as software or doctrine, it is a fixture.  It seems like older
generations have some of this as well but we can make computers do
whatever we want;  there are no rules only conventions.

I'm not trying to convince anyone of anything, there is mounting
science that we basically never change our minds on anything.  The
conversation was useful for solidifying my own views and maybe making
people who have done this for a long time to express their views on
the basics for shared consideration.

Regards,
Kevin

On Mon, Jan 13, 2020 at 4:48 PM Dan Cross <crossd@gmail.com> wrote:
>
> [Resending as this got squashed a few days ago. Jon, sorry for the duplicate. Again.]
>
> On Sun, Jan 12, 2020 at 4:38 PM Jon Steinhart <jon@fourwinds.com> wrote:
>>
>> [snip]
>> So I think that the point that you're trying to make, correct me if I'm wrong,
>> is that if lists just knew how long they were you could just ask and that it
>> would be more efficient.
>
>
> What I understood was that, by translating into a lowest-common-denominator format like text, one loses much of the semantic information implicit in a richer representation. In particular, much of the internal knowledge (like type information...) is lost during translation and presentation. Put another way, with text as usually used by the standard suite of Unix tools, type information is implicit, rather than explicit. I took this to be less an issue of efficiency and more of expressiveness.
>
> It is, perhaps, important to remember that Unix works so well because of heavy use of convention: to take Doug's example, the total number of commands might be easy to find with `wc` because one assumes each command is presented on a separate line, with no gaudy header or footer information or extraneous explanatory text.
>
> This sort of convention, where each logical "record" is a line by itself, is pervasive on Unix systems, but is not guaranteed. In some sense, those representations are fragile: a change in output might break something else downstream in the pipeline, whereas a representation that captures more semantic meaning is more robust in the face of change but, as in Doug's example, often harder to use. The Lisp Machine had all sorts of cool information in the image and a good Lisp hacker familiar with the machine's structures could write programs to extract and present that information. But doing so wasn't trivial in the way that '| wc -l' in response to a casual query is.
>
>> While that may be true, it sort of assume that this is something so common that
>> the extra overhead for line counting should be part of every list.  And it doesn't
>> address the issue that while maybe you want a line count I may want a character
>> count or a count of all lines that begin with the letter A.  Limiting this example
>> to just line numbers ignores the fact that different people might want different
>> information that can't all be predicted in advance and built into every program.
>
>
> This I think illustrates an important point: Unix conventions worked well enough in practice that many interesting tasks were not just tractable, but easy and in some cases trivial. Combining programs was easy via pipelines. Harder stuff involving more elaborate data formats was possible, but, well, harder and required more involved programming. By contrast, the Lisp machine could do the hard stuff, but the simple stuff also required non-trivial programming.
>
> The SQL database point was similarly interesting: having written programs to talk to relational databases, yes, one can do powerful things: but the amount of programming required is significant at a minimum and often substantial.
>
>>
>> It also seems to me that the root problem here is that the data in the original
>> example was in an emacs-specific format instead of the default UNIX text file
>> format.
>>
>> The beauty of UNIX is that with a common file format one can create tools that
>> process data in different ways that then operate on all data.  Yes, it's not as
>> efficient as creating a custom tool for a particular purpose, but is much better
>> for casual use.  One can always create a special purpose tool if a particular
>> use becomes so prevalent that the extra efficiency is worthwhile.  If you're not
>> familiar with it, find a copy of the Communications of the ACM issue where Knuth
>> presented a clever search algorithm (if I remember correctly) and McIlroy did a
>> critique.  One of the things that Doug pointed out what that while Don's code was
>> more efficient, by creating a new pile of special-purpose code he introduced bugs.
>
>
> The flip side is that one often loses information in the conversion to text: yes, there are structured data formats with text serializations that can preserve the lost information, but consuming and processing those with the standard Unix tools can be messy. Seemingly trivial changes in text, like reversing the order of two fields, can break programs that consume that data. Data must be suitable for pipelining (e.g., perhaps free-form text must be free of newlines or something). These are all limitations. Where I think the argument went awry is in not recognizing that very often those problems, while real, are at least tractable.
>
>> Many people have claimed, incorrectly in my opinion, that this model fails in the
>> modern era because it only works on text data.  They change the subject when I
>> point out that ImageMagick works on binary data.  And, there are now stream
>> processing utilities for JSON data and such that show that the UNIX model still
>> works IF you understand it and know how to use it.
>
>
> Certainly. I think you hit the nail on the head with the proviso that one must _understand_ the Unix model and how to use it. If one does so, it's very powerful indeed, and it really is applicable more often than not. But it is not a panacea (not that anyone suggested it is). As an example, how do I apply an unmodified `grep` to arbitrary JSON data (which may span more than one line)? Perhaps there is a way (I can imagine a 'record2line' program that consumes a single JSON object and emits it as a syntactically valid one-liner...) but I can also imagine all sorts of ways that might go wrong.
>
>         - Dan C.
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-13 23:46                   ` Dan Cross
  2020-01-14 23:17                     ` Kevin Bowling
@ 2020-01-18 15:45                     ` Michael Parson
  2020-01-18 18:45                       ` Jon Steinhart
  2020-01-21 21:57                       ` Derek Fawcus
  1 sibling, 2 replies; 21+ messages in thread
From: Michael Parson @ 2020-01-18 15:45 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Mon, 13 Jan 2020, Dan Cross wrote:
> 
> [Resending as this got squashed a few days ago. Jon, sorry for the
> duplicate. Again.]
>
> On Sun, Jan 12, 2020 at 4:38 PM Jon Steinhart <jon@fourwinds.com> wrote:

<snip>

>> Many people have claimed, incorrectly in my opinion, that this model
>> fails in the modern era because it only works on text data.  They
>> change the subject when I point out that ImageMagick works on binary
>> data.  And, there are now stream processing utilities for JSON data
>> and such that show that the UNIX model still works IF you understand
>> it and know how to use it.
>
>
> Certainly. I think you hit the nail on the head with the proviso
> that one must _understand_ the Unix model and how to use it. If one
> does so, it's very powerful indeed, and it really is applicable more
> often than not. But it is not a panacea (not that anyone suggested it
> is). As an example, how do I apply an unmodified `grep` to arbitrary
> JSON data (which may span more than one line)? Perhaps there is a way
> (I can imagine a 'record2line' program that consumes a single JSON
> object and emits it as a syntactically valid one-liner...) but I can
> also imagine all sorts of ways that might go wrong.

And here, understanding the model is important, namely, grep is the
wrong tool for searching/parsing JSON output. Dealing with JSON from the
shell, you should use jq.  I've been dragged kicking and screaming into
dealing with JSON, and about all I can say about it is, I like it about
this >< much more than XML. :)

-- 
Michael Parson
Pflugerville, TX
KF5LGQ

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-18 15:45                     ` Michael Parson
@ 2020-01-18 18:45                       ` Jon Steinhart
  2020-01-18 18:59                         ` Michael Parson
  2020-01-21 21:57                       ` Derek Fawcus
  1 sibling, 1 reply; 21+ messages in thread
From: Jon Steinhart @ 2020-01-18 18:45 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Michael Parson writes:
> On Mon, 13 Jan 2020, Dan Cross wrote:
> > 
> > [Resending as this got squashed a few days ago. Jon, sorry for the
> > duplicate. Again.]
> >
> > On Sun, Jan 12, 2020 at 4:38 PM Jon Steinhart <jon@fourwinds.com> wrote:
>
> <snip>
>
> >> Many people have claimed, incorrectly in my opinion, that this model
> >> fails in the modern era because it only works on text data.  They
> >> change the subject when I point out that ImageMagick works on binary
> >> data.  And, there are now stream processing utilities for JSON data
> >> and such that show that the UNIX model still works IF you understand
> >> it and know how to use it.
> >
> >
> > Certainly. I think you hit the nail on the head with the proviso
> > that one must _understand_ the Unix model and how to use it. If one
> > does so, it's very powerful indeed, and it really is applicable more
> > often than not. But it is not a panacea (not that anyone suggested it
> > is). As an example, how do I apply an unmodified `grep` to arbitrary
> > JSON data (which may span more than one line)? Perhaps there is a way
> > (I can imagine a 'record2line' program that consumes a single JSON
> > object and emits it as a syntactically valid one-liner...) but I can
> > also imagine all sorts of ways that might go wrong.
>
> And here, understanding the model is important, namely, grep is the
> wrong tool for searching/parsing JSON output. Dealing with JSON from the
> shell, you should use jq.  I've been dragged kicking and screaming into
> dealing with JSON, and about all I can say about it is, I like it about
> this >< much more than XML. :)
>
> -- 
> Michael Parson
> Pflugerville, TX
> KF5LGQ

Slight disagreement here.  I would say that grep is *a* a tool for JSON and
that jq is *a better* one.  The UNIX model part of this is that jq is another
tool in a toolbox and plays well with others.  What disturbs me about a lot
of software (and my tractor) it when it does the equivalent of throwing a
random metric bolt into something that's built from SAE hardware.

Jon


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-18 18:45                       ` Jon Steinhart
@ 2020-01-18 18:59                         ` Michael Parson
  2020-01-18 20:31                           ` Adam Thornton
  0 siblings, 1 reply; 21+ messages in thread
From: Michael Parson @ 2020-01-18 18:59 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, 18 Jan 2020, Jon Steinhart wrote:
> Michael Parson writes:
>> On Mon, 13 Jan 2020, Dan Cross wrote:
>>>
>>> [Resending as this got squashed a few days ago. Jon, sorry for the
>>> duplicate. Again.]
>>>
>>> On Sun, Jan 12, 2020 at 4:38 PM Jon Steinhart <jon@fourwinds.com> wrote:
>>
>> <snip>
>>
>>>> Many people have claimed, incorrectly in my opinion, that this model
>>>> fails in the modern era because it only works on text data.  They
>>>> change the subject when I point out that ImageMagick works on binary
>>>> data.  And, there are now stream processing utilities for JSON data
>>>> and such that show that the UNIX model still works IF you understand
>>>> it and know how to use it.
>>>
>>>
>>> Certainly. I think you hit the nail on the head with the proviso
>>> that one must _understand_ the Unix model and how to use it. If one
>>> does so, it's very powerful indeed, and it really is applicable more
>>> often than not. But it is not a panacea (not that anyone suggested it
>>> is). As an example, how do I apply an unmodified `grep` to arbitrary
>>> JSON data (which may span more than one line)? Perhaps there is a way
>>> (I can imagine a 'record2line' program that consumes a single JSON
>>> object and emits it as a syntactically valid one-liner...) but I can
>>> also imagine all sorts of ways that might go wrong.
>>
>> And here, understanding the model is important, namely, grep is the
>> wrong tool for searching/parsing JSON output. Dealing with JSON from the
>> shell, you should use jq.  I've been dragged kicking and screaming into
>> dealing with JSON, and about all I can say about it is, I like it about
>> this >< much more than XML. :)
>>
>> --
>> Michael Parson
>> Pflugerville, TX
>> KF5LGQ
>
> Slight disagreement here.  I would say that grep is *a* a tool for JSON and
> that jq is *a better* one.

OK, yeah, I'll use grep to find what I'm looking for in the json output,
then fight with^W^Wuse jq to extract the info I'm looking for in the json
output.

> The UNIX model part of this is that jq is another tool in a toolbox
> and plays well with others.  What disturbs me about a lot of software
> (and my tractor) it when it does the equivalent of throwing a random
> metric bolt into something that's built from SAE hardware.
>
> Jon
>

-- 
Michael Parson
Pflugerville, TX
KF5LGQ

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-18 18:59                         ` Michael Parson
@ 2020-01-18 20:31                           ` Adam Thornton
  0 siblings, 0 replies; 21+ messages in thread
From: Adam Thornton @ 2020-01-18 20:31 UTC (permalink / raw)
  To: The Eunuchs Historic Society

On Jan 18, 2020, at 11:59 AM, Michael Parson <mparson@bl.org> wrote:
> OK, yeah, I'll use grep to find what I'm looking for in the json output,
> then fight with^W^Wuse jq to extract the info I'm looking for in the json
> output.

Glad to know it’s not just me that finds the jq query language…difficult to comprehend.

Adam

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-18 15:45                     ` Michael Parson
  2020-01-18 18:45                       ` Jon Steinhart
@ 2020-01-21 21:57                       ` Derek Fawcus
  2020-01-22  7:21                         ` arnold
  1 sibling, 1 reply; 21+ messages in thread
From: Derek Fawcus @ 2020-01-21 21:57 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, Jan 18, 2020 at 09:45:22AM -0600, Michael Parson wrote:
> 
> And here, understanding the model is important, namely, grep is the
> wrong tool for searching/parsing JSON output. Dealing with JSON from the
> shell, you should use jq.  I've been dragged kicking and screaming into
> dealing with JSON, and about all I can say about it is, I like it about
> this >< much more than XML. :)

If push comes to shove, one can always use xmlawk (xml extension for gawk)
to beat the latter in to submission.

DF

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-21 21:57                       ` Derek Fawcus
@ 2020-01-22  7:21                         ` arnold
  2020-01-22  7:29                           ` Tyler Adams
  0 siblings, 1 reply; 21+ messages in thread
From: arnold @ 2020-01-22  7:21 UTC (permalink / raw)
  To: tuhs, dfawcus+lists-tuhs

> On Sat, Jan 18, 2020 at 09:45:22AM -0600, Michael Parson wrote:
> > 
> > And here, understanding the model is important, namely, grep is the
> > wrong tool for searching/parsing JSON output. Dealing with JSON from the
> > shell, you should use jq.  I've been dragged kicking and screaming into
> > dealing with JSON, and about all I can say about it is, I like it about
> > this >< much more than XML. :)

Derek Fawcus <dfawcus+lists-tuhs@employees.org> wrote:
> If push comes to shove, one can always use xmlawk (xml extension for gawk)
> to beat the latter in to submission.

There is also a simple JSON extension for gawk in the gawkextlib project
that can suck in a single JSON record and turn it into a gawk
multidimensional array, and vice versa.  (Gawk has true multidimensional
arrays and isarray/typeof functions so that you can easily traverse
such an array.)

(<horn ownership="mine">Toot!</horn>)

Arnold

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ]
  2020-01-22  7:21                         ` arnold
@ 2020-01-22  7:29                           ` Tyler Adams
  0 siblings, 0 replies; 21+ messages in thread
From: Tyler Adams @ 2020-01-22  7:29 UTC (permalink / raw)
  To: arnold; +Cc: tuhs


[-- Attachment #1: Type: text/plain, Size: 1254 bytes --]

Also for anything where jq gets too complicated, you can use python to whip
up an arbitrary json filter

Theres only 5 lines of boiler plate

import json
import sys
d = json.load(sys.stdin)
# your code here
print(json.dumps(d))

Tyler

On Wed, Jan 22, 2020, 09:22 <arnold@skeeve.com> wrote:

> > On Sat, Jan 18, 2020 at 09:45:22AM -0600, Michael Parson wrote:
> > >
> > > And here, understanding the model is important, namely, grep is the
> > > wrong tool for searching/parsing JSON output. Dealing with JSON from
> the
> > > shell, you should use jq.  I've been dragged kicking and screaming into
> > > dealing with JSON, and about all I can say about it is, I like it about
> > > this >< much more than XML. :)
>
> Derek Fawcus <dfawcus+lists-tuhs@employees.org> wrote:
> > If push comes to shove, one can always use xmlawk (xml extension for
> gawk)
> > to beat the latter in to submission.
>
> There is also a simple JSON extension for gawk in the gawkextlib project
> that can suck in a single JSON record and turn it into a gawk
> multidimensional array, and vice versa.  (Gawk has true multidimensional
> arrays and isarray/typeof functions so that you can easily traverse
> such an array.)
>
> (<horn ownership="mine">Toot!</horn>)
>
> Arnold
>

[-- Attachment #2: Type: text/html, Size: 1987 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, back to index

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-12 13:43 [TUHS] Tech Sq elevator (Was: screen editors) Doug McIlroy
2020-01-12 16:56 ` Warner Losh
2020-01-12 17:21   ` markus schnalke
2020-01-12 20:25   ` Kevin Bowling
2020-01-12 20:32     ` Larry McVoy
2020-01-12 20:34     ` Jon Steinhart
2020-01-12 20:40       ` Kevin Bowling
2020-01-12 20:44         ` Jon Steinhart
2020-01-12 21:03           ` Kevin Bowling
2020-01-12 21:37             ` [TUHS] Tech Sq elevator (Was: screen editors) [ really I think efficiency now ] Jon Steinhart
     [not found]               ` <CAEoi9W4fXLaTRM1mv4wnVbifCFBEw_iKL9cds8ds-FBRTwM-=g@mail.gmail.com>
     [not found]                 ` <CAEoi9W6LedGGjWPO=ZgZzVdGLqs8drhqcWkvA_DfKTOtMDgegQ@mail.gmail.com>
2020-01-13 23:46                   ` Dan Cross
2020-01-14 23:17                     ` Kevin Bowling
2020-01-18 15:45                     ` Michael Parson
2020-01-18 18:45                       ` Jon Steinhart
2020-01-18 18:59                         ` Michael Parson
2020-01-18 20:31                           ` Adam Thornton
2020-01-21 21:57                       ` Derek Fawcus
2020-01-22  7:21                         ` arnold
2020-01-22  7:29                           ` Tyler Adams
2020-01-12 21:41           ` [TUHS] Tech Sq elevator (Was: screen editors) Bakul Shah
2020-01-12 21:47             ` Jon Steinhart

The Unix Heritage Society mailing list

Archives are clonable: git clone --mirror http://inbox.vuxu.org/tuhs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.tuhs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git