From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from minnie.tuhs.org (minnie.tuhs.org [45.79.103.53]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 8dfe2c0d for ; Wed, 4 Mar 2020 23:25:56 +0000 (UTC) Received: by minnie.tuhs.org (Postfix, from userid 112) id 99F709D72C; Thu, 5 Mar 2020 09:25:55 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 7368E9D71C; Thu, 5 Mar 2020 09:25:21 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=pass (2048-bit key; unprotected) header.d=jon-es.20150623.gappssmtp.com header.i=@jon-es.20150623.gappssmtp.com header.b="oumjbCtB"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id CD3699D68F; Thu, 5 Mar 2020 09:25:18 +1000 (AEST) Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by minnie.tuhs.org (Postfix) with ESMTPS id 679C49D68F for ; Thu, 5 Mar 2020 09:25:17 +1000 (AEST) Received: by mail-lj1-f174.google.com with SMTP id e3so3947273lja.10 for ; Wed, 04 Mar 2020 15:25:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jon-es.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3d0o7Da8MWjXBFxHYx2ET25FI7IZUMmUo31UnA+0OU8=; b=oumjbCtB0rHqfJHSAL0+/6cOQz8s6xWLjt7RY6V+PieoJneGHe61APxAxjjY1iUtTd hANlvF6F1gBe7j2KJZxGOQK1GDpsozDP2JPGuj0KpXRzu01Mk66buG56lF6KN2ZSwTUw XbwhnC+pRillq+6wx+1Zmc89a8QuGSOtTCGkj+RrRpIdpRBBxUv8QUZMTjjgQ/DBy691 /K2sBy3b0El/sOejtu+G76Anj9gYSNS81Q7DK5auHlhrQAwsV/qAkyHHBpd27LOk+BZg DD5afOf+L7e0s6Tws618ha7I/Yoi9zk/0NCZ/anP1ESAScmkc8Wly0+rwO3KBvwtwypq Ei/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3d0o7Da8MWjXBFxHYx2ET25FI7IZUMmUo31UnA+0OU8=; b=E7JqhNOXao7zYmvlGqPVfQK0CtuG5XkG5KNyC1zaZfTCgUAq2B5LlPOgmJCTEJw7EP 9xpzAuFHxCHj8kKqFmEHHH3B68jsDBHU26M/eTkP/7kHAtxNWGK6FgBZrs0JEpa6//2z 3cs4w5MZcPNhS/yD6XlvhepEZ4ZB88YGX9NWaF1SGBFsPgD3a4lXlUitAmzpfc2sfSKx UQK0/H9wpiZe0y5e1tH+dxmB11kz3M5PHrH1kQKs5f/11zwPw7/qbwCmAlDveSDsQKKu aAxGFmAhoosYIJuhqDeEe/I+6oBvKLiUPdXmYR8bDVhlFXJN+0yKNHWqZp35sW+GKJZS 6VbA== X-Gm-Message-State: ANhLgQ2HABXNpGGPEV22TfgvUuRdqadFoxafd/LKhtGCfU6yZ6JuHLkv HJuUQxAdlnT4ULMwCDXL+ATSGzP4sDNRqAB0Gpg= X-Google-Smtp-Source: ADFU+vtxb/diDwPb2GFDfgQnG85iuafRAa0t8++/n+MVIe4VkuPecCzgOb/7CNzbVbbREkYJldRd48dasFsBVqO7DHA= X-Received: by 2002:a05:651c:204f:: with SMTP id t15mr3185596ljo.240.1583364315687; Wed, 04 Mar 2020 15:25:15 -0800 (PST) MIME-Version: 1.0 References: <202003031815.023IFSlD493028@darkstar.fourwinds.com> <43d31402-fd1a-4324-a8fb-ef72406d41f3@www.fastmail.com> In-Reply-To: <43d31402-fd1a-4324-a8fb-ef72406d41f3@www.fastmail.com> From: Terry Jones Date: Thu, 5 Mar 2020 00:25:04 +0100 Message-ID: To: Random832 Content-Type: multipart/alternative; boundary="000000000000962fe205a00fbb1a" Subject: Re: [TUHS] Command line options and complexity X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Grant Taylor via TUHS Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --000000000000962fe205a00fbb1a Content-Type: text/plain; charset="UTF-8" On Wed, Mar 4, 2020 at 11:04 PM Random832 wrote: > Hardly *any* commands you'd use in a pipeline really operate on > unstructured bytes. Compression, I suppose. But other than that, you have > just as much need to know what commands operate on what structure in Unix > as in Powershell - the only difference is that the serialization is > explicitly part of the interface... and due to the typical inability to > escape delimiters, leaky. > Another difference is that probably most people on this list are extremely familiar with the various quirks and I/O nuances of the tools many have been using every day for decades. Just as the native speakers of a natural language can't so easily see/appreciate its complexity (e.g., pronunciation in English!), I suspect many of us have internalized these idiosyncrasies. I teach occasional shell/Python courses to absolute beginners (no computing experience at all) and came to appreciate how weird the shell is (in the sense of having baked-in historical accidents that cannot / will not / should not be "corrected"). Some of my appreciation of that was due to discussions on this list (e.g., regarding comment syntax, and the : command) - so thanks! I know what follows won't be to everyone's taste, but I like Python and I love shell pipelines, so I tried to write a shell that gave you both and which allowed fairly free mixing of invoking UNIX tools and running Python. You can send anything down its pipelines - lines of text, atoms, numbers, Python objects, whatever (in the Python _ variable). Of course the receiving end of the pipeline needs to know (or figure out) what it's getting. One advantage is that you have a carefully designed programming language (no offence intended!) underlying the shell, so you can e.g., write shell functions in Python (and put them in a start-up file if you want) and just pipe regular UNIX output into them and pipe their output into whatever's next (more Python, another UNIX command, etc). Probably almost no one would actually want to regularly do the following on the command line, but you could: >>> from os import stat >>> def fd(): return [name for (name, time) in sorted((f, stat(f).st_mtime) for f in _)] >>> ls | fd() | tail -n 3 Here I've stuck a simple (DSU - see [1]) Python function in between two UNIX commands and use it to get the most recently modified files. You probably wouldn't want to do this either, but you could: >>> seq 0 9 | list(map(lambda x: 2 ** int(x), _)) | tee /tmp/powers-of-two | sum(map(int, _))1023>>> cat /tmp/powers-of-two1248163264128256512 Of course it also lets you do things you *would* want to do :-) More at https://github.com/terrycojones/daudin Python has fairly nice tools for reading and evaluating Python code, which meant that getting a first version of this implemented took only one evening of playing around. It's pretty simple (and still has plenty of rough edges). Apologies if this seems like self-promotion, but I very much enjoy thinking about things in this thread and about how we work with information. I'm also constantly blown away by how elegant UNIX is and how the core ideas have endured. Pipelines are really wonderful, as "natural" alternative to function composition as a mathematician or programmer would do it (see point #1 at https://github.com/terrycojones/daudin#background--thanks), and I wanted to build a shell that preserved that, while giving you Python. The overview of their history on pages 67-70 of bwk's recent book [2] is very interesting. Terry [1] https://en.wikipedia.org/wiki/Schwartzian_transform [2] https://www.amazon.com/UNIX-History-Memoir-Brian-Kernighan/dp/1695978552 --000000000000962fe205a00fbb1a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Wed, Mar 4, 2020 at 11:04 PM Random832= <random832@fastmail.com&g= t; wrote:
Hardly *any* commands you'd use in a pipeline real= ly operate on unstructured bytes. Compression, I suppose. But other than th= at, you have just as much need to know what commands operate on what struct= ure in Unix as in Powershell - the only difference is that the serializatio= n is explicitly part of the interface... and due to the typical inability t= o escape delimiters, leaky.

Another dif= ference is that probably most people on this list are extremely familiar wi= th the various quirks and I/O nuances of the tools many have been using eve= ry day for decades. Just as the native speakers of a natural language can&#= 39;t so easily see/appreciate its complexity (e.g., pronunciation in Englis= h!), I suspect many of us have internalized these idiosyncrasies. I teach o= ccasional shell/Python courses to absolute beginners (no computing experien= ce at all) and came to appreciate how weird the shell is (in the sense of h= aving baked-in historical accidents that cannot / will not / should not be = "corrected"). Some of my appreciation of that was due to discussi= ons on this list (e.g., regarding comment syntax, and the : command) - so t= hanks!

I know what follows won't be to everyon= e's taste, but I like Python and I love shell pipelines, so I tried to = write a shell that gave you both and which allowed fairly free mixing of in= voking UNIX tools and running Python. You can send anything down its pipeli= nes - lines of text, atoms, numbers, Python objects, whatever (in the Pytho= n _ variable).=C2=A0 Of course= the receiving end of the pipeline needs to know (or figure out) what it= 9;s getting. One advantage is that you have a carefully designed programmin= g language (no offence intended!) underlying the shell, so you can e.g., wr= ite shell functions in Python (and put them in a start-up file if you want)= and just pipe regular UNIX output into them and pipe their output into wha= tever's next (more Python, another UNIX command, etc). Probably almost = no one would actually want to regularly do the following on the command lin= e, but you could:

>>> from os import stat
>&g= t;> def fd(): return [name for (name, time) in sorted((f, stat(f).st_mti= me) for f in _)] =C2=A0=C2=A0
>>> ls | fd() | tail -n = 3

Here I've stuck a simple (DSU - see [= 1]) Python function in between two UNIX commands and use it to get the most= recently modified files.

You probably wouldn'= t want to do this either, but you could:
>>> seq 0 9 | list(map(lambda x: 2 ** int(x), _)) | te=
e /tmp/=
powers-of-two | sum<=
/span>(map(i=
nt, _))
1023
>>> cat /tmp=
/powers-of-two
1
2
4
8
16
32
64
128
256
512
Of course it also le= ts you do things you would want to do :-)

M= ore at https://github.co= m/terrycojones/daudin=C2=A0=C2=A0 Python has fairly nice tools for read= ing and evaluating Python code, which meant that getting a first version of= this implemented took only one evening of playing around. It's pretty = simple (and still has plenty of rough edges).=C2=A0 Apologies if this seems= like self-promotion, but I very much enjoy thinking about things in this t= hread and about how we work with information. I'm also constantly blown= away by how elegant UNIX is and how the core ideas have endured. Pipelines= are really wonderful, as "natural" alternative to function compo= sition as a mathematician or programmer would do it (see point #1 at https://git= hub.com/terrycojones/daudin#background--thanks), and I wanted to build = a shell that preserved that, while giving you Python. The overview of their= history on pages 67-70 of bwk's recent book [2] is very interesting.

Terry

=
--000000000000962fe205a00fbb1a--