From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 8010 invoked from network); 3 Jan 2023 23:04:02 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 3 Jan 2023 23:04:02 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id A7B544252B; Wed, 4 Jan 2023 09:04:00 +1000 (AEST) Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) by minnie.tuhs.org (Postfix) with ESMTPS id 83F1242528 for ; Wed, 4 Jan 2023 09:03:56 +1000 (AEST) Received: by mail-ej1-f47.google.com with SMTP id m18so77905889eji.5 for ; Tue, 03 Jan 2023 15:03:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=hBYJnm9HxVMfIrXJKJimogLsZQUw5hJ6/pB7VLLI23w=; b=bBFPfGM59tqYw452dQUYqdgNgqMENCuCzczI5veNBKmURdUrSgAOYlbpE3jO8aXT47 oorAky23yD3BIjN7nHrixt2+TAqcxJcQ2+nZl+ztIIYUwdxUqxhpdIFfueHts5sQdKH8 ABG+IEGVHjmotIXfZerExi/mX73O6q0BAHxdLHOVJkzJZtr3FZ2O37J1xVYVBKqeLwkH XuekCFKMQu73CeQ5zCYcPsl5CC+pL3fXCpieC+dtvedgQ6gkMfZPp5LI1YgTVwWUisrf 5njv0wEz5mEhwUULazq3X6noJO8vs/wZqWJAJOuB5eJEWdlFaPMn+ygP6shHhWDUsMdo obxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hBYJnm9HxVMfIrXJKJimogLsZQUw5hJ6/pB7VLLI23w=; b=mX2vbpEJ2+u28urDyNkNf97y2gSqR3L6f/zL0JJgV9Ic+/bIoWbm043KF27fMknbbv tDLUAXJTMN5T296r+oeGRENMxUg99LyvzhP2sUfOcagTNxKpwNtBBAET+i/6zCpmnm+t hqW0bLRXRtbCdmc46H7j/YaQ8r/EvR0KlfWwV/HY6fx0Ze5VXQTYdna20do9AsVXBAWr UVsqioRt2TKk1VezfKRXRqSNyW7RModkaRX2EAc+yp0ZomESpbmc/phrMRjxQM0UpUtS /m5iGBqiKF/Sx6ttbUt+VH0bPfns2JSM51Ko1xWs1N3ZFExxA5KUkRA1G/vY0wsJ6a3+ 2kaQ== X-Gm-Message-State: AFqh2kouHe1jkZNgmxpxBD84Wn5MrF0Ps/8xV1hu5g0JWR6uDBIvJ6Ti Fp7aUY2iO3bOI2dYiVa4dnMa+HOMlit2Mm2yzUOAWUNxBOw= X-Google-Smtp-Source: AMrXdXvyWWFev90mpshvmah0EfvinJtaJ7n0mozcvUEY9uLt7jNFTfURk1fm2dMRoatYMxoYIOeKTJ6q2a9WL8x6Pdk= X-Received: by 2002:a17:906:da07:b0:78d:9022:f146 with SMTP id fi7-20020a170906da0700b0078d9022f146mr3866078ejb.656.1672786974540; Tue, 03 Jan 2023 15:02:54 -0800 (PST) MIME-Version: 1.0 References: <52FB6638-AEFF-4A4F-8C2E-32089D577BA0@planet.nl> <464819f0-d2f6-2a60-6481-a194f4428b4d@case.edu> <20221230200246.GW5825@mcvoy.com> <8ca17d52-a25a-dbbf-e1f0-d743b8884cfa@in-ulm.de> In-Reply-To: From: Adam Thornton Date: Tue, 3 Jan 2023 16:02:42 -0700 Message-ID: To: Computer Old Farts Followers Content-Type: multipart/alternative; boundary="000000000000670ca405f164114f" Message-ID-Hash: G564YW4GQ3GQXE4NVPYEWHHNYB6ENDAH X-Message-ID-Hash: G564YW4GQ3GQXE4NVPYEWHHNYB6ENDAH X-MailFrom: athornton@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [COFF] Re: [TUHS] Re: Command Line Editing in the Terminal Driver (Was: A few comments on porting the Bourne shell) List-Id: Computer Old Farts Forum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --000000000000670ca405f164114f Content-Type: text/plain; charset="UTF-8" (moving to COFF) On Tue, Jan 3, 2023 at 9:55 AM Marshall Conover wrote: > Along these lines but not quite, Jupyter Notebooks have stood out to me as > another approach to this concept, with behavior I'd like to see implemented > in a shell. > > Well, you know, if you're OK with bash: https://github.com/takluyver/bash_kernel Or zsh: https://pypi.org/project/zsh-jupyter-kernel/ One of the big things I do is work on the Notebook Aspect of the Rubin Science Platform. Each JupyterLab notebook session is tied to a "kernel" which is a specific language-and-extension environment. At the Rubin Observatory, we support a Python 3.10 environment with our processing pipeline included. Although JupyterLab itself is capable of doing other languages (notably Julia and R, which are the other two from which the name "Jupyter" came), many others have been adapted to the notebook environment (including at least the two shells above). And while researchers are welcome to write their own code and work with raw images, we work under the presumption that almost everyone is going to use the software tools the Rubin Observatory provides to work with the data it collects, because writing your own data processing pipeline from scratch is a pretty monumental project. Most astronomers are perfectly happy with what we provide, which is Python plus our processing pipelines, which are all Python from the scientist-facing perspective (much of the pipeline code is implemented in C++ for speed, but it then gets Python bindings, so unless you're actually working very deep down in the image processing code, it's Python as far as you're concerned). However, a certain class of astronomers still loves their FORTRAN. This class, unfortunately, tends to be the older ones, which means the ones who have societal stature, tenure, can relatively easily get big grants, and wield a lot of power within their institutions. I know that it is *possible* to run gfortran as a Jupyter kernel. I've seen it done. I was impressed. Fortunately, no one with the power to make it stick has demanded we provide a kernel like that. The initial provision of the service wouldn't be the problem. It's that the support would be a nightmare. No one on my team is great at FORTRAN; I'm probably the closest to fluent, and I'm not very, and I really don't enjoy working in FORTRAN, and because FORTRAN really doesn't lend itself easily to the kind of Python REPL exploration that notebooks are all about, and because someone who loves FORTRAN and hates Python probably has a very different concept of what good software engineering practices look like than I do, trying to support someone working in a notebook environment in a FORTRAN kernel would very likely be exquisitely painful. And fortunately, since there are not FORTRAN bindings to the C++ classes providing the core algorithms, much less FORTRAN bindings to the Python implementations (because all the things that don't particularly need to be fast are just written in Python in the first place), a gfortran kernel wouldn't be nearly as useful as our Python-plus-our-tools. Now, sure, if we had paying customers who were willing to throw enough money at us to make it worth the pain and both bring a FORTRAN implementation to feature-parity with the reference environment and make a gfortran kernel available, then we would do it. But I get paid a salary that is not directly tied to my support burden, and I get to spend a lot more of my time working on fun things and providing features for astronomers who are not mired in 1978 if I can avoid spending my time supporting huge time sinks that aren't in widespread use. We are scoped to provide Python. We are not scoped to provide FORTRAN. We are not making money off of sales: we're employed to provide stable infrastructure services so the scientists using our platform and observatory can get their research done. And thus far we've been successful in saying "hey, we've got finite resources, we're not swimming in spare cycles, no, we can't support [ x for x in things-someone-wants-but-are-not-in-the-documented-scope ]". (To be fair, this has more often been things like additional visualization libraries than whole new languages, but the principle is the same.) We have a process for proposing items for inclusion, and it's not at all rare that we add them, but it's generally a considered decision about how generally useful the proposed item will be for the project as a whole and how much time it's likely to consume to support. So this gave me a little satori about why I think POSIX.2 is a perfectly reasonable spec to support and why I'm not wild about making all my shell scripts instead compatible with the subset of v7 sh that works (almost) everywhere. It's not all that much more work up front, but odds are that a customer that wants to run new software, but who can't guarantee a POSIX /bin/sh, will be a much more costly customer to support than one who can, just as someone who wants a notebook environment but insists on FORTRAN in it is very likely going to be much harder to support than someone who's happy with the Python environment we already supply. Adam --000000000000670ca405f164114f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
(moving to COFF)

On Tue, Jan= 3, 2023 at 9:55 AM Marshall Conover <marzhall.o@gmail.com> wrote:
Along these lines but not q= uite, Jupyter Notebooks have stood out=20 to me as another approach to this concept, with behavior I'd like to se= e implemented in a shell.


We= ll, you know, if you're OK with bash: https://github.com/takluyver/bash_kernel
=

<= div>One of the big things I do is work on the Notebook Aspect of the Rubin = Science Platform.=C2=A0 Each JupyterLab notebook session is tied to a "= ;kernel" which is a specific language-and-extension environment.=C2=A0= At the Rubin Observatory, we support a Python 3.10 environment with our pr= ocessing pipeline included.=C2=A0 Although JupyterLab itself is capable of = doing other languages (notably Julia and R, which are the other two from wh= ich the name "Jupyter" came), many others have been adapted to th= e notebook environment (including at least the two shells above).=C2=A0 And= while researchers are welcome to write their own code and work with raw im= ages, we work under the presumption that almost everyone is going to use th= e software tools the Rubin Observatory provides to work with the data it co= llects, because writing your own data processing pipeline from scratch is a= pretty monumental project.

Most astronomers a= re perfectly happy with what we provide, which is Python plus our processin= g pipelines, which are all Python from the scientist-facing perspective (mu= ch of the pipeline code is implemented in C++ for speed, but it then gets P= ython bindings, so unless you're actually working very deep down in the= image processing code, it's Python as far as you're concerned).=C2= =A0 However, a certain class of astronomers still loves their FORTRAN.=C2= =A0 This class, unfortunately, tends to be the older ones, which means the = ones who have societal stature, tenure, can relatively easily get big grant= s, and wield a lot of power within their institutions.

=
I know that it is *possible* to run gfortran as a Jupyter kernel.=C2= =A0 I've seen it done.=C2=A0 I was impressed.

<= div>Fortunately, no one with the power to make it stick has demanded we pro= vide a kernel like that.=C2=A0 The initial provision of the service wouldn&= #39;t be the problem.=C2=A0 It's that the support would be a nightmare.= =C2=A0 No one on my team is great at FORTRAN; I'm probably the closest = to fluent, and I'm not very, and I really don't enjoy working in FO= RTRAN, and because FORTRAN really doesn't lend itself easily to the kin= d of Python REPL exploration that notebooks are all about, and because some= one who loves FORTRAN and hates Python probably has a very different concep= t of what good software engineering practices look like than I do, trying t= o support someone working in a notebook environment in a FORTRAN kernel wou= ld very likely be exquisitely painful.=C2=A0 And fortunately, since there a= re not FORTRAN bindings to the C++ classes providing the core algorithms, m= uch less FORTRAN bindings to the Python implementations (because all the th= ings that don't particularly need to be fast are just written in Python= in the first place), a gfortran kernel wouldn't be nearly as useful as= our Python-plus-our-tools.

Now, sure, if we h= ad paying customers who were willing to throw enough money at us to make it= worth the pain and both bring a FORTRAN implementation to feature-parity w= ith the reference environment and make a gfortran kernel available, then we= would do it.=C2=A0 But I get paid a salary that is not directly tied to my= support burden, and I get to spend a lot more of my time working on fun th= ings and providing features for astronomers who are not mired in 1978 if I = can avoid spending my time supporting huge time sinks that aren't in wi= despread use.=C2=A0 We are scoped to provide Python.=C2=A0 We are not scope= d to provide FORTRAN.=C2=A0 We are not making money off of sales: we're= employed to provide stable infrastructure services so the scientists using= our platform and observatory can get their research done.=C2=A0 And thus f= ar we've been successful in saying "hey, we've got finite reso= urces, we're not swimming in spare cycles, no, we can't support [ x= for x in things-someone-wants-but-are-not-in-the-documented-scope ]".= =C2=A0 (To be fair, this has more often been things like additional visuali= zation libraries than whole new languages, but the principle is the same.)= =C2=A0 We have a process for proposing items for inclusion, and it's no= t at all rare that we add them, but it's generally a considered decisio= n about how generally useful the proposed item will be for the project as a= whole and how much time it's likely to consume to support.

So this gave me a little satori about why I think PO= SIX.2 is a perfectly reasonable spec to support and why I'm not wild ab= out making all my shell scripts instead compatible with the subset of v7 sh= that works (almost) everywhere.=C2=A0 It's not all that much more work= up front, but odds are that a customer that wants to run new software, but= who can't guarantee a POSIX /bin/sh, will be a much more costly custom= er to support than one who can, just as someone who wants a notebook enviro= nment but insists on FORTRAN in it is very likely going to be much harder t= o support than someone who's happy with the Python environment we alrea= dy supply.

Adam
--000000000000670ca405f164114f--