The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Proliferation of options is great simplification of pipes, really?
@ 2021-02-22  2:34 Will Senn
  2021-02-22  3:32 ` G. Branden Robinson
                   ` (2 more replies)
  0 siblings, 3 replies; 24+ messages in thread
From: Will Senn @ 2021-02-22  2:34 UTC (permalink / raw)
  To: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

All,

So, we've been talking low-level design for a while. I thought I would 
ask a fundamental question. In days of old, we built small 
single-purpose utilities and used pipes to pipeline the data and 
transformations. Even back in the day, it seemed that there was tension 
to add yet another option to every utility. Today, as I was marveling at 
groff's abilities with regard to printing my man pages directly to my 
printer in 2021, I read the groff(1) page:

example here: https://linux.die.net/man/1/groff

What struck me (the wrong way) was the second paragraph of the description:

The groff program allows to control the whole groff system by command 
line options. This is a great simplification in comparison to the 
classical case (which uses pipes only).

Here is the current plethora of options:
groff [-abcegilpstzCEGNRSUVXZ] [-d cs] [-f fam] [-F dir] [-I dir] [-L 
arg] [-m name] [-M dir] [-n num] [-o list] [-P arg] [-r cn] [-T dev] [-w 
name] [-W name] [file ...]

Now, I appreciate groff, don't get me wrong, but my sensibilities were 
offended by the idea that a kazillion options was in any way simpler 
than pipelining single-purpose utilities. What say you? Is this the 
perfected logical extension of the unix pioneers' work, or have we gone 
horribly off the trail.

Regards,

Will

[-- Attachment #2: Type: text/html, Size: 1822 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  2:34 [TUHS] Proliferation of options is great simplification of pipes, really? Will Senn
@ 2021-02-22  3:32 ` G. Branden Robinson
  2021-02-22  4:32   ` Dan Stromberg
                     ` (2 more replies)
  2021-02-22  7:20 ` Rich Morin
  2021-02-22 18:27 ` Jon Steinhart
  2 siblings, 3 replies; 24+ messages in thread
From: G. Branden Robinson @ 2021-02-22  3:32 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 4945 bytes --]

At 2021-02-21T20:34:55-0600, Will Senn wrote:
> All,
> 
> So, we've been talking low-level design for a while. I thought I would
> ask a fundamental question. In days of old, we built small
> single-purpose utilities and used pipes to pipeline the data and
> transformations. Even back in the day, it seemed that there was
> tension to add yet another option to every utility. Today, as I was
> marveling at groff's abilities with regard to printing my man pages
> directly to my printer in 2021, I read the groff(1) page:
> 
> example here: https://linux.die.net/man/1/groff

A more up to date copy is available at the Linux man-pages site.

https://man7.org/linux/man-pages/man1/groff.1.html

> What struck me (the wrong way) was the second paragraph of the
> description:
> 
> The groff program allows to control the whole groff system by command
> line options. This is a great simplification in comparison to the
> classical case (which uses pipes only).

What strikes _me_ about the above is the awful Denglish in it.  I fixed
this back in 2017 and the correction shipped as part of groff 1.22.4 in
December 2018.

> Here is the current plethora of options:
> groff [-abcegilpstzCEGNRSUVXZ] [-d cs] [-f fam] [-F dir] [-I dir] [-L arg]
> [-m name] [-M dir] [-n num] [-o list] [-P arg] [-r cn] [-T dev] [-w name]
> [-W name] [file ...]
> 
> Now, I appreciate groff, don't get me wrong, but my sensibilities were
> offended by the idea that a kazillion options was in any way simpler
> than pipelining single-purpose utilities. What say you? Is this the
> perfected logical extension of the unix pioneers' work, or have we
> gone horribly off the trail.

I'd say it's neither, and reflects (1) the limitations of the Unix
filter model, or at least the linear topology of Unix pipelines[1]; and
(2) an arbitrary set of rules determined by convention and common
practice with respect to sequencing.

Consider the first the question of which *roff preprocessor languages
should be embeddable in another preprocessor's language.  Should you be
able to embed equations in tables?  What about tables inside equations
(not too insane an idea--consider matrix literals)?  Nothing in the Unix
filter model implies a choice between these decisions, but an ordering
decision must be made.

V7 Unix tbl(1)'s man page[3] took a moderately strong position on
preprocessor ordering based on more practical concerns (I suppose
loading on shared systems).

	When it is used with
	.I eqn
	or
	.I neqn
	the
	.I tbl
	command should be first, to minimize the volume
	of data passed through
	pipes.

Another factor is ergonomics.  As the number of preprocessors expands,
the number of potential orderings of a document processing pipeline also
grows--combinatorially.  Here's the chunk of the groff front-end
program that determines the ordering of the pipeline it constructs for
the user.

	// grap, chem, and ideal must come before pic;
	// tbl must come before eqn
	const int PRECONV_INDEX = 0;
	const int SOELIM_INDEX = PRECONV_INDEX + 1;
	const int REFER_INDEX = SOELIM_INDEX + 1;
	const int GRAP_INDEX = REFER_INDEX + 1;
	const int CHEM_INDEX = GRAP_INDEX + 1;
	const int IDEAL_INDEX = CHEM_INDEX + 1;
	const int PIC_INDEX = IDEAL_INDEX + 1;
	const int TBL_INDEX = PIC_INDEX + 1;
	const int GRN_INDEX = TBL_INDEX + 1;
	const int EQN_INDEX = GRN_INDEX + 1;
	const int TROFF_INDEX = EQN_INDEX + 1;
	const int POST_INDEX = TROFF_INDEX + 1;
	const int SPOOL_INDEX = POST_INDEX + 1;

Sure, you could have a piece of paper with the above ordering taped to
the wall near your terminal, but why?  Isn't it better to have a tool to
keep track of these arbitrary complexities instead?

groff, as a front-end and pipeline manager, is much smaller than the
actual formatter.  According to sloccount, it's 1,195 lines to troff's
23,023 (measurements taken on groff Git HEAD, where I spend much of my
time).

If you need to alter the pipeline or truncate it, to debug an input
document or resequence the processing order, you can, and groff supplies
the -V flag to help you do so.

A traditionalist need never type the groff command if it offends one's
sensibilities--it would be a welcome change from people grousing about
copyleft.  All the pieces of the pipeline are still there and can be
directly invoked.

For an alternative approach to *roff document interpretation and
rendering, albeit in a limited domain, see the mandoc project[4].  It
interprets the man(7) and mdoc(7) macro languages, a subset of *roff,
and tbl(1)'s mini-language with, as I understand it, a single parser.

Regards,
Branden

[1] Tom Duff noted this a long time ago in his paper presenting the rc
shell[2]; see §9.

[2] https://archive.org/details/rc-shell/page/n2/mode/1up
[3] https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/man/man1/tbl.1
[4] https://mandoc.bsd.lv/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  3:32 ` G. Branden Robinson
@ 2021-02-22  4:32   ` Dan Stromberg
  2021-02-22  4:34   ` Will Senn
  2021-02-22 15:49   ` John P. Linderman
  2 siblings, 0 replies; 24+ messages in thread
From: Dan Stromberg @ 2021-02-22  4:32 UTC (permalink / raw)
  To: G. Branden Robinson; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On Sun, Feb 21, 2021 at 7:33 PM G. Branden Robinson <g.branden.robinson@gmai

>
> > The groff program allows to control the whole groff system by command
> > line options. This is a great simplification in comparison to the
> > classical case (which uses pipes only).
>
> What strikes _me_ about the above is the awful Denglish in it.  I fixed
> this back in 2017 and the correction shipped as part of groff 1.22.4 in
> December 2018.
>
I like the easy composability of pipes, but I don't mind some options.

I don't like the huge, all-purpose applications called web browsers nearly
as much.  They strike me as Very un-unixy.  But much can be justified by
not having to get users to download a client application, and not having to
get sysadmins to punch a hole through their firewalls.

I'd say it's neither, and reflects (1) the limitations of the Unix
> filter model, or at least the linear topology of Unix pipelines[1]
>
I don't think they have to be linear:
http://joeyh.name/code/moreutils/ (see the unfortunately-named "pee"
utility)
and:
https://stromberg.dnsalias.org/~strombrg/mtee.html

Full disclosure: I wrote mtee.

[-- Attachment #2: Type: text/html, Size: 1831 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  3:32 ` G. Branden Robinson
  2021-02-22  4:32   ` Dan Stromberg
@ 2021-02-22  4:34   ` Will Senn
  2021-02-22  5:45     ` G. Branden Robinson
  2021-02-22 15:49   ` John P. Linderman
  2 siblings, 1 reply; 24+ messages in thread
From: Will Senn @ 2021-02-22  4:34 UTC (permalink / raw)
  To: tuhs

On 2/21/21 9:32 PM, G. Branden Robinson wrote:
> At 2021-02-21T20:34:55-0600, Will Senn wrote:
>> All,
>>
>> So, we've been talking low-level design for a while. I thought I would
>> ask a fundamental question. In days of old, we built small
>> single-purpose utilities and used pipes to pipeline the data and
>> transformations. Even back in the day, it seemed that there was
>> tension to add yet another option to every utility. Today, as I was
>> marveling at groff's abilities with regard to printing my man pages
>> directly to my printer in 2021, I read the groff(1) page:
>>
>> example here: https://linux.die.net/man/1/groff
> A more up to date copy is available at the Linux man-pages site.
>
> https://man7.org/linux/man-pages/man1/groff.1.html

I just picked the first hit in google :) shoulda known better. However, 
it's the same text that's in my mac's install (Mojave).

>
>> What struck me (the wrong way) was the second paragraph of the
>> description:
>>
>> The groff program allows to control the whole groff system by command
>> line options. This is a great simplification in comparison to the
>> classical case (which uses pipes only).
> What strikes _me_ about the above is the awful Denglish in it.  I fixed
> this back in 2017 and the correction shipped as part of groff 1.22.4 in
> December 2018.
Mac Mojave: Groff Version 1.19.2              3 July 
2005                         GROFF(1)

>> Here is the current plethora of options:
>> groff [-abcegilpstzCEGNRSUVXZ] [-d cs] [-f fam] [-F dir] [-I dir] [-L arg]
>> [-m name] [-M dir] [-n num] [-o list] [-P arg] [-r cn] [-T dev] [-w name]
>> [-W name] [file ...]
>>
>> Now, I appreciate groff, don't get me wrong, but my sensibilities were
>> offended by the idea that a kazillion options was in any way simpler
>> than pipelining single-purpose utilities. What say you? Is this the
>> perfected logical extension of the unix pioneers' work, or have we
>> gone horribly off the trail.
> I'd say it's neither, and reflects (1) the limitations of the Unix
> filter model, or at least the linear topology of Unix pipelines[1]; and
> (2) an arbitrary set of rules determined by convention and common
> practice with respect to sequencing.
snip...

Very informative post, Branden. I appreciate the details. I gotta read 
more code :).

Will

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  4:34   ` Will Senn
@ 2021-02-22  5:45     ` G. Branden Robinson
  0 siblings, 0 replies; 24+ messages in thread
From: G. Branden Robinson @ 2021-02-22  5:45 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 1684 bytes --]

At 2021-02-21T22:34:10-0600, Will Senn wrote:
> On 2/21/21 9:32 PM, G. Branden Robinson wrote:
> > What strikes _me_ about the above is the awful Denglish in it.  I
> > fixed this back in 2017 and the correction shipped as part of groff
> > 1.22.4 in December 2018.
> Mac Mojave: Groff Version 1.19.2              3 July
> 2005                         GROFF(1)

Yikes.  Yeah, every once in a while a macOS user reports a known defect
to the groff list, one we've fixed years ago.  Apple's insistence on
shipping a 15-year old version is pretty frustrating.

I'm given to understand that "brew" can be used straightforwardly to
obtain much more recent groff builds, and I know for sure that we have
macOS users contributing reports when something in the toolchain goes
wrong and we need to accommodate it.  Here's a recent example[1].

I've been soliciting help from Windows users to keep our build in good
shape over there, to no effect lately.  This may have something to do
with Microsoft's latest Unix compatibility effort being a bundled Ubuntu
distribution--I don't know the details.  It may be that going forward
there will simply be no audience for "native" Windows support in groff.

> Very informative post, Branden. I appreciate the details. I gotta read
> more code :).

Thank you!  TUHS has been a tremendously useful resource in helping me
to document where things came from, as well as to figure out when
some element of surprising behavior is just a bug versus a historical
compatibility feature.

V9 sources sure would be nice to have, as would DWB versions other than
3.3... <hopeful face>

[1] https://savannah.gnu.org/bugs/?60035

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  2:34 [TUHS] Proliferation of options is great simplification of pipes, really? Will Senn
  2021-02-22  3:32 ` G. Branden Robinson
@ 2021-02-22  7:20 ` Rich Morin
  2021-02-22 18:27 ` Jon Steinhart
  2 siblings, 0 replies; 24+ messages in thread
From: Rich Morin @ 2021-02-22  7:20 UTC (permalink / raw)
  To: TUHS main list

I've been happily using pipes since I found out about pipes, back in the early 80's (Thanks, Doug!).  However, until recently I didn't write applications in a programming language which supported them "internally".  Recently, however, I've been using Elixir, which does:

Pipe Operator
https://elixirschool.com/en/lessons/basics/pipe-operator/

Note that, although the basic pipe implementation simply does composition of functions with error handling, the Stream variant offers lazy evaluation:

Enumerables and Streams
https://elixir-lang.org/getting-started/enumerables-and-streams.html

Yes, I know that F# (and probably other languages) had pipes first, but I still give points to José Valim for stealing wisely and well.  Various folks then built onto the basic pipe mechanism, e.g.:

- https://github.com/batate/elixir-pipes - extension library for using pattern matching with pipes, etc.

- https://hexdocs.pm/broadway/Broadway.html - concurrent, multi-stage tool for building data ingestion and data processing pipelines with back pressure, etc.

fun stuff...

-r

 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  3:32 ` G. Branden Robinson
  2021-02-22  4:32   ` Dan Stromberg
  2021-02-22  4:34   ` Will Senn
@ 2021-02-22 15:49   ` John P. Linderman
  2021-02-22 15:57     ` William Cheswick
                       ` (3 more replies)
  2 siblings, 4 replies; 24+ messages in thread
From: John P. Linderman @ 2021-02-22 15:49 UTC (permalink / raw)
  To: G. Branden Robinson; +Cc: The Unix Heritage Society

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

I can imagine a simple perl (or python or whatever) script that would run
through groff input, determine which preprocessors are *actually* needed,
and set up a pipeline to run through (only) the needed preprocessors in the
proper order. I wouldn't have to tell groff what preprocessors I think are
needed, and groff wouldn't have to change (although my script would) when
another preprocessor comes into existence. Modern processors are fast
enough, and groff input small enough, that the "extra" pass wouldn't be
burdensome. And it would take the burden off me to remember exactly which
preprocessors are essential. -- jpl

[-- Attachment #2: Type: text/html, Size: 733 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 15:49   ` John P. Linderman
@ 2021-02-22 15:57     ` William Cheswick
  2021-02-22 16:03       ` John P. Linderman
  2021-02-22 16:02     ` Warner Losh
                       ` (2 subsequent siblings)
  3 siblings, 1 reply; 24+ messages in thread
From: William Cheswick @ 2021-02-22 15:57 UTC (permalink / raw)
  To: John P Linderman (jpl); +Cc: The Unix Heritage Society

This proposal reminds me of Paul Glick’s lp command.  It took whatever file you gave it, and processed
it as necessary for whatever printer you chose.  It was very useful, simple AI.

> On Feb 22, 2021, at 10:49 AM, John P. Linderman <jpl.jpl@gmail.com> wrote:
> 
> I can imagine a simple perl (or python or whatever) script that would run through groff input, determine which preprocessors are actually needed, and set up a pipeline to run through (only) the needed preprocessors in the proper order. I wouldn't have to tell groff what preprocessors I think are needed, and groff wouldn't have to change (although my script would) when another preprocessor comes into existence. Modern processors are fast enough, and groff input small enough, that the "extra" pass wouldn't be burdensome. And it would take the burden off me to remember exactly which preprocessors are essential. -- jpl


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 15:49   ` John P. Linderman
  2021-02-22 15:57     ` William Cheswick
@ 2021-02-22 16:02     ` Warner Losh
  2021-02-22 16:12     ` Robert Clausecker
  2021-02-22 21:14     ` G. Branden Robinson
  3 siblings, 0 replies; 24+ messages in thread
From: Warner Losh @ 2021-02-22 16:02 UTC (permalink / raw)
  To: John P. Linderman; +Cc: The Unix Heritage Society

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

On Mon, Feb 22, 2021 at 8:50 AM John P. Linderman <jpl.jpl@gmail.com> wrote:

> I can imagine a simple perl (or python or whatever) script that would run
> through groff input, determine which preprocessors are *actually* needed,
> and set up a pipeline to run through (only) the needed preprocessors in the
> proper order. I wouldn't have to tell groff what preprocessors I think are
> needed, and groff wouldn't have to change (although my script would) when
> another preprocessor comes into existence. Modern processors are fast
> enough, and groff input small enough, that the "extra" pass wouldn't be
> burdensome. And it would take the burden off me to remember exactly which
> preprocessors are essential. -- jpl
>

Yea, that's the main benefit of extra flags to commands: you can optimize
the number of filters that data passes through, or you can do things with
'hidden state' that's hard to do in another phase of the output. ls is a
good example. ls -lt is relatively easy to do the sorting of times and the
formatting of times inside ls, but harder to do as a filter since times are
hard to sort...

Warner

[-- Attachment #2: Type: text/html, Size: 1576 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 15:57     ` William Cheswick
@ 2021-02-22 16:03       ` John P. Linderman
  2021-02-22 21:16         ` G. Branden Robinson
  0 siblings, 1 reply; 24+ messages in thread
From: John P. Linderman @ 2021-02-22 16:03 UTC (permalink / raw)
  To: William Cheswick; +Cc: The Unix Heritage Society

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

On a brute-forcier note, when I was doing a lot more troff, I wrote a
command that ran input through *ALL* the preprocessors I might need. Even
on 70's processors, it was fast enough, and msde my life a tiny bit
better.  -- jpl

On Mon, Feb 22, 2021 at 10:57 AM William Cheswick <ches@cheswick.com> wrote:

> This proposal reminds me of Paul Glick’s lp command.  It took whatever
> file you gave it, and processed
> it as necessary for whatever printer you chose.  It was very useful,
> simple AI.
>
> > On Feb 22, 2021, at 10:49 AM, John P. Linderman <jpl.jpl@gmail.com>
> wrote:
> >
> > I can imagine a simple perl (or python or whatever) script that would
> run through groff input, determine which preprocessors are actually needed,
> and set up a pipeline to run through (only) the needed preprocessors in the
> proper order. I wouldn't have to tell groff what preprocessors I think are
> needed, and groff wouldn't have to change (although my script would) when
> another preprocessor comes into existence. Modern processors are fast
> enough, and groff input small enough, that the "extra" pass wouldn't be
> burdensome. And it would take the burden off me to remember exactly which
> preprocessors are essential. -- jpl
>
>

[-- Attachment #2: Type: text/html, Size: 1672 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 15:49   ` John P. Linderman
  2021-02-22 15:57     ` William Cheswick
  2021-02-22 16:02     ` Warner Losh
@ 2021-02-22 16:12     ` Robert Clausecker
  2021-02-22 17:15       ` John Cowan
  2021-02-23  0:24       ` Steffen Nurpmeso
  2021-02-22 21:14     ` G. Branden Robinson
  3 siblings, 2 replies; 24+ messages in thread
From: Robert Clausecker @ 2021-02-22 16:12 UTC (permalink / raw)
  To: tuhs

> I can imagine a simple perl (or python or whatever) script that would run
> through groff input, determine which preprocessors are *actually* needed,
> and set up a pipeline to run through (only) the needed preprocessors in the
> proper order. I wouldn't have to tell groff what preprocessors I think are
> needed, and groff wouldn't have to change (although my script would) when
> another preprocessor comes into existence. Modern processors are fast
> enough, and groff input small enough, that the "extra" pass wouldn't be
> burdensome. And it would take the burden off me to remember exactly which
> preprocessors are essential. -- jpl

I'm not sure if it would be that simple.  With preprocessors like
soelim, your script would have to be able to open arbitrary external
files to find out what preprocessors are needed.  And perhaps other
preprocessors too could trigger dependencies on additional
preprocessors depending on how they are used.

Yours,
Robert Clausecker

-- 
()  ascii ribbon campaign - for an 8-bit clean world 
/\  - against html email  - against proprietary attachments

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 16:12     ` Robert Clausecker
@ 2021-02-22 17:15       ` John Cowan
  2021-02-23  0:24       ` Steffen Nurpmeso
  1 sibling, 0 replies; 24+ messages in thread
From: John Cowan @ 2021-02-22 17:15 UTC (permalink / raw)
  To: Robert Clausecker; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

On Mon, Feb 22, 2021 at 11:17 AM Robert Clausecker <fuz@fuz.su> wrote:


> I'm not sure if it would be that simple.  With preprocessors like
> soelim, your script would have to be able to open arbitrary external
> files to find out what preprocessors are needed.  And perhaps other
> preprocessors too could trigger dependencies on additional
> preprocessors depending on how they are used.
>

True enough, but you'd be no better off in principle if you were using an
explicit pipeline, especially if you are sourcing files that you didn't
write: your knowledge of the content may be vague.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
Heckler: "Go on, Al, tell 'em all you know.  It won't take long."
Al Smith: "I'll tell 'em all we *both* know.  It won't take any longer."

[-- Attachment #2: Type: text/html, Size: 1983 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22  2:34 [TUHS] Proliferation of options is great simplification of pipes, really? Will Senn
  2021-02-22  3:32 ` G. Branden Robinson
  2021-02-22  7:20 ` Rich Morin
@ 2021-02-22 18:27 ` Jon Steinhart
  2021-02-22 19:30   ` Richard Salz
  2 siblings, 1 reply; 24+ messages in thread
From: Jon Steinhart @ 2021-02-22 18:27 UTC (permalink / raw)
  To: TUHS main list

Will Senn writes:
> This is a multi-part message in MIME format.
>
> All,
>
> So, we've been talking low-level design for a while. I thought I would 
> ask a fundamental question. In days of old, we built small 
> single-purpose utilities and used pipes to pipeline the data and 
> transformations. Even back in the day, it seemed that there was tension 
> to add yet another option to every utility. Today, as I was marveling at 
> groff's abilities with regard to printing my man pages directly to my 
> printer in 2021, I read the groff(1) page:
>
> example here: https://linux.die.net/man/1/groff
>
> What struck me (the wrong way) was the second paragraph of the description:
>
> The groff program allows to control the whole groff system by command 
> line options. This is a great simplification in comparison to the 
> classical case (which uses pipes only).
>
> Here is the current plethora of options:
> groff [-abcegilpstzCEGNRSUVXZ] [-d cs] [-f fam] [-F dir] [-I dir] [-L 
> arg] [-m name] [-M dir] [-n num] [-o list] [-P arg] [-r cn] [-T dev] [-w 
> name] [-W name] [file ...]
>
> Now, I appreciate groff, don't get me wrong, but my sensibilities were 
> offended by the idea that a kazillion options was in any way simpler 
> than pipelining single-purpose utilities. What say you? Is this the 
> perfected logical extension of the unix pioneers' work, or have we gone 
> horribly off the trail.
>
> Regards,
>
> Will

I'm 99% happy with groff and its many options.  Why?  Because the various
programs (troff, pic, tbl, eqn, ...) are still available and can be composed
into pipelines of my own choosing.  The 1% unhappiness is because I think
that groff should be a shell script which it doesn't appear to be.  In my
opinion, if groff was a bad thing then one would have to question things
like scripts and aliases in general.  Groff is a composer, and composability
is a core UNIXism to me.  It would be way wrong if it replaced all of the
programs that it invoked, but it doesn't.

As an interesting example of the composability of the troff system, I did
the diagrams for my book using pic because pic is awesome.  But, despite
what it says in the No Starch Press author guidelines, they really only
accept material in word format.  I could have rendered each image as a
bitmap, but that just seemed so 80s.  Turns out that while it doesn't do
a great job, word will accept vector graphics in SVG format.  So I ran
each image through pic, through groff, through ps2pdf (embedding fonts),
through pdf2svg, and finally through inkscape to crop the image.  A tad
cumbersome, but it works, and wouldn't be easy to do on any other system.

I also did my original draft in troff and wrote a script to convert it
into openoffice XML format so that it could be word-ified.  Only part
that I couldn't figure out was how to include the figures; I could
generate the XML but it didn't work and there were no useful diagnostics
so I had to import them by hand.

Since Rob is on the list and (in)famous for the "cat -v" argument, I would
agree with him that that is not the "right" way.  Being consistent with
my position on groff, I would go for a separate show-nonprinting utility
and then, if widely used, a script that composed that and cat.

Jon

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 18:27 ` Jon Steinhart
@ 2021-02-22 19:30   ` Richard Salz
  0 siblings, 0 replies; 24+ messages in thread
From: Richard Salz @ 2021-02-22 19:30 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 64 bytes --]

Is anyone upset with CC and its options and internal pipelines?

[-- Attachment #2: Type: text/html, Size: 111 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 15:49   ` John P. Linderman
                       ` (2 preceding siblings ...)
  2021-02-22 16:12     ` Robert Clausecker
@ 2021-02-22 21:14     ` G. Branden Robinson
  3 siblings, 0 replies; 24+ messages in thread
From: G. Branden Robinson @ 2021-02-22 21:14 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Hi John,
At 2021-02-22T10:49:52-0500, John P. Linderman wrote:
> I can imagine a simple perl (or python or whatever) script that would
> run through groff input, determine which preprocessors are *actually*
> needed, and set up a pipeline to run through (only) the needed
> preprocessors in the proper order.

This is _almost_ what the groff grog(1) command does.  It's been present
as far back as our history goes, to groff 1.02 in June 1991.

* It's a Perl script.
* It uses pattern-matching heuristics to infer which arguments groff(1)
  will need to format the document (not just for preprocessors, but
  macro packages as well).
* Depending on its own options, it writes the constructed command to
  stderr, executes it, or both.

The only thing it doesn't handle is ordering, because groff(1) already
takes care of that.

> I wouldn't have to tell groff what preprocessors I think are needed,
> and groff wouldn't have to change (although my script would) when
> another preprocessor comes into existence. Modern processors are fast
> enough, and groff input small enough, that the "extra" pass wouldn't
> be burdensome. And it would take the burden off me to remember exactly
> which preprocessors are essential. -- jpl

We don't get a lot of bug reports about grog.  Maybe it's not given
enough prominence in groff's own documentation.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 16:03       ` John P. Linderman
@ 2021-02-22 21:16         ` G. Branden Robinson
  0 siblings, 0 replies; 24+ messages in thread
From: G. Branden Robinson @ 2021-02-22 21:16 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

At 2021-02-22T11:03:43-0500, John P. Linderman wrote:
> On a brute-forcier note, when I was doing a lot more troff, I wrote a
> command that ran input through *ALL* the preprocessors I might need.
> Even on 70's processors, it was fast enough, and msde my life a tiny
> bit better.  -- jpl

No crime in that.  An alias or shell function to call "groff -Rpet"
would take care of all the V7 Unix preprocessors.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-22 16:12     ` Robert Clausecker
  2021-02-22 17:15       ` John Cowan
@ 2021-02-23  0:24       ` Steffen Nurpmeso
  1 sibling, 0 replies; 24+ messages in thread
From: Steffen Nurpmeso @ 2021-02-23  0:24 UTC (permalink / raw)
  To: Robert Clausecker; +Cc: tuhs

Robert Clausecker wrote in
 <YDPX3sDuIQzjCrXL@fuz.su>:
 |> I can imagine a simple perl (or python or whatever) script that would run
 |> through groff input, determine which preprocessors are *actually* needed,
 |> and set up a pipeline to run through (only) the needed preprocessors \
 |> in the
 |> proper order. I wouldn't have to tell groff what preprocessors I \
 |> think are
 |> needed, and groff wouldn't have to change (although my script would) when
 |> another preprocessor comes into existence. Modern processors are fast
 |> enough, and groff input small enough, that the "extra" pass wouldn't be
 |> burdensome. And it would take the burden off me to remember exactly which
 |> preprocessors are essential. -- jpl
 |
 |I'm not sure if it would be that simple.  With preprocessors like
 |soelim, your script would have to be able to open arbitrary external
 |files to find out what preprocessors are needed.  And perhaps other
 |preprocessors too could trigger dependencies on additional
 |preprocessors depending on how they are used.

Newer incarnations of man(1) support a shebang-alike control line
<^'\" >followed by concat of [egprtv]+ and include $MANROFFSEQ
content into this list, then do

   case "${preproc_arg}" in
                  e)      pipeline="$pipeline | $EQN" ;;
                  g)      GRAP  ;; # Ignore for compatibility.
                  p)      pipeline="$pipeline | $PIC" ;;
                  r)      pipeline="$pipeline | $REFER" ;;
                  t)      pipeline="$pipeline | $TBL" ;;
                  v)      pipeline="$pipeline | $VGRIND" ;;
                  *)      usage ;;
                  esac

(I copied all this from 2014 text, do not ask me no questions.)
It would make very much sense to extend this syntax for roff
usage, so that document creators can define how manual consumers
generate the result.  This should/could include specification and
thus automatic adjustment of the used character set.
The problem with pipes is that they are academic.  You can write
wrapper scripts or shell functions or for simple cases even
aliases to give the desire a name, but it does not fit pretty well
the all-graphical shader-improved wiping experience people now
have.  You also want good manuals and a shell with a good history
feature and a nice line editor and possibly tabulator completion,
just in case you have forgotten something or made an error or are
too lazy to type that much.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
@ 2021-02-24 19:38 Norman Wilson
  0 siblings, 0 replies; 24+ messages in thread
From: Norman Wilson @ 2021-02-24 19:38 UTC (permalink / raw)
  To: tuhs

To fill out the historical record, the earliest doctype I know of
was a shell (not rc) script.  From my basement heater that happens
to run 10/e:

b$ man doctype | uniq

     DOCTYPE(1)                                             DOCTYPE(1)

     NAME
          doctype - guess command line for formatting a document

     SYNOPSIS
          doctype [ option ... ] [ file ]

     DESCRIPTION
          Doctype guesses and prints on the standard output the com-
          mand line for printing a document that uses troff(1),
          related preprocessors like eqn(1), and the ms(6) and mm
          macro packages.

          Option -n invokes nroff instead of troff. Other options are
          passed to troff.

     EXAMPLES
          eval `doctype chapter.?` | apsend
               Typeset files named chapter.0, chapter.1, ...

     SEE ALSO
          troff(1), eqn(1), tbl(1), refer(1), prefer(1), pic(1),
          ideal(1), grap(1), ped(9.1), mcs(6), ms(6), man(6)

     BUGS
          It's pretty dumb about guessing the proper macro package.

     Page 1                    Tenth Edition       (printed 2/24/2021)

doctype(1) is in the 8/e manual, so it existed in early 1985;
I bet it's actually older than that.  The manual page is on
the V8 tape, but, oddly, not the program; neither is it in
the V10 pseudo-tape I cobbled together for Warren long ago.
I'm not sure why not.

The version in rc is, of course, a B-movie remake of the
original.

Norman Wilson
Toronto ON

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-23 15:04 Steve Simon
@ 2021-02-24  2:42 ` M Douglas McIlroy
  0 siblings, 0 replies; 24+ messages in thread
From: M Douglas McIlroy @ 2021-02-24  2:42 UTC (permalink / raw)
  To: Steve Simon; +Cc: tuhs

doctyoe it was.

Thanks,
Doug

On Tue, Feb 23, 2021 at 10:12 AM Steve Simon <steve@quintile.net> wrote:
>
> its written in rc(1) and uses plan9 regex which
> sometimes differ from unix ones a little but
> there is doctype:
>
> http://9p.io/magic/man2html/1/doctype
> http://9p.io/sources/plan9/rc/bin/doctype
>
> -Steve

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
@ 2021-02-23 15:04 Steve Simon
  2021-02-24  2:42 ` M Douglas McIlroy
  0 siblings, 1 reply; 24+ messages in thread
From: Steve Simon @ 2021-02-23 15:04 UTC (permalink / raw)
  To: tuhs

its written in rc(1) and uses plan9 regex which
sometimes differ from unix ones a little but
there is doctype:

http://9p.io/magic/man2html/1/doctype
http://9p.io/sources/plan9/rc/bin/doctype

-Steve

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-23  2:47 M Douglas McIlroy
  2021-02-23 10:42 ` Jaap Akkerhuis
@ 2021-02-23 13:49 ` Ralph Corderoy
  1 sibling, 0 replies; 24+ messages in thread
From: Ralph Corderoy @ 2021-02-23 13:49 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: tuhs

Hi Doug,

> > I can imagine a simple perl (or python or whatever) script that
> > would run through groff input [and] determine which preprocessors
> > are actually needed ...
>
> Brian imagined such and implemented it way back when. Though I used
> it, I've forgotten its name.

Was it ‘doctype’?  That's what it's called in Kernighan & Pike's ‘Unix
Programming Environment’, pp. 306-8.  Groff had something similar called
grog(1) which had flaws when I rewrote it in sh and mainly awk back
in 2002.  (My version never made it in because an FSF copyright
assignment was required and their answers to some of my questions meant
I wouldn't sign.)  Someone else since rewrote Groff's in Perl.

> defensive groff -pet that I often use

I've taken to putting the information needed as a comment at the start
of the main source file whence it's picked up by a generic run-off
script.  Similar to man(1) looking for a «'\"» comment with code
letters: ‘p’ for pic, ‘v’ for vgrind, etc.

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-23 10:42 ` Jaap Akkerhuis
@ 2021-02-23 13:23   ` Brantley Coile
  0 siblings, 0 replies; 24+ messages in thread
From: Brantley Coile @ 2021-02-23 13:23 UTC (permalink / raw)
  To: Jaap Akkerhuis; +Cc: tuhs

#!/bin/rc
# doctype:  synthesize proper command line for troff

troff=troff
eqn=eqn
prefer=prefer
opt=''
dev=''
while(~ $1 -*){
	switch($1){
	case -n;
		troff=nroff
		eqn=neqn
		prefer='prefer -n'
	case -T
		dev=$1
	case -*
		opt=$opt' $1'
	}
	shift
}
ifs='
'{
	files=`{echo $*}
}
grep -h '\$LIST|\|reference|Jp|^\.(EQ|TS|\[|PS|IS|GS|G1|GD|PP|BM|LP|BP|PI|cstart|begin|TH...|TI)|^\.P$' $* |
sort -u |
awk '
BEGIN	{ files = "'$"files'" }
/\$LIST/ { e++ }
/^\.PP/	{ ms++ }
/^\.LP/	{ ms++ }
/^\.EQ/	{ eqn++ }
/^\.TS/	{ tbl++ }
/^\.PS/	{ pic++ }
/^\.IS/	{ ideal++ }
/^\.GS/ { tped++ }
/^\.G1/ { grap++; pic++ }
/^\.GD/ { dag++; pic++ }
/^\.\[/	{ refer++ }
/\|reference/ { prefer++ }
/^\.cstart/ { chem++; pic++ }
/^\.begin +dformat/ { dformat++; pic++ }
/^\.TH.../ { man++ }
/^\.BM/ { lbits++ }
/^\.P$/	{ mm++ }
/^\.BP/	{ pictures++ }
/^\.PI/	{ pictures++ }
/^\.TI/	{ mcs++ }
/^\.ft *Jp|\\f\(Jp/ { nihongo++ }
END {
	x = ""
	if (refer) {
		if (e)	x = "refer/refer -e " files " | "
		else	x =  "refer/refer " files "| "
		files = ""
	}
	else if (prefer) { x = "cat " files "| '$prefer'| "; files = "" }
	if (tped)  { x = x "tped " files " | "; files = "" }
	if (dag)  { x = x "dag " files " | "; files = "" }
	if (ideal) { x = x "ideal -q " files " | "; files = "" }
	if (grap)  { x = x "grap " files " | "; files = "" }
	if (chem)  { x = x "chem " files " | "; files = "" }
	if (dformat)  { x = x "dformat " files " | "; files = "" }
	if (pic)   { x = x "pic " files " | "; files = "" }
	if (tbl)   { x = x "tbl " files " | "; files = "" }
	if (eqn)   { x = x "'$eqn' '$dev' " files " | "; files = "" }
	x = x "'$troff' "
	if (man) x = x "-man"
	else if (ms) x = x "-ms"
	else if (mm) x = x "-mm"
	if (mcs) x = x " -mcs"
	if (lbits) x = x " -mbits"
	if (pictures) x = x " -mpictures"
	if (nihongo) x = x " -mnihongo"
	x = x " '$opt' '$dev' " files
	print x
}'

> On Feb 23, 2021, at 5:42 AM, Jaap Akkerhuis <jaapna@xs4all.nl> wrote:
> 
> 
> 
>> On Feb 23, 2021, at 3:47, M Douglas McIlroy <m.douglas.mcilroy@dartmouth.edu> wrote:
>> 
>>> I can imagine a simple perl (or python or whatever) script that would run
>>> through groff input [and] determine which preprocessors are actually
>>> needed ...
>> 
>> Brian imagined such and implemented it way back when. Though I used
>> it, I've forgotten its name. One probably could have fooled it by
>> tricks like calling pic only in a .so file and perhaps renaming .so.
>> But I never heard of it failing in real life. It does impose an extra
>> pass over the input, but may well save a pass compared to the
>> defensive groff -pet that I often use or to the rerun necessary when I
>> forget to mention some or all of the filters.
> 
> 
> If I remember correctly, it was an awk script printing out the
> suggested pipeline to use.  One could then cut and paste that line.
> 
> 	jaap


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] Proliferation of options is great simplification of pipes, really?
  2021-02-23  2:47 M Douglas McIlroy
@ 2021-02-23 10:42 ` Jaap Akkerhuis
  2021-02-23 13:23   ` Brantley Coile
  2021-02-23 13:49 ` Ralph Corderoy
  1 sibling, 1 reply; 24+ messages in thread
From: Jaap Akkerhuis @ 2021-02-23 10:42 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]



> On Feb 23, 2021, at 3:47, M Douglas McIlroy <m.douglas.mcilroy@dartmouth.edu> wrote:
> 
>> I can imagine a simple perl (or python or whatever) script that would run
>> through groff input [and] determine which preprocessors are actually
>> needed ...
> 
> Brian imagined such and implemented it way back when. Though I used
> it, I've forgotten its name. One probably could have fooled it by
> tricks like calling pic only in a .so file and perhaps renaming .so.
> But I never heard of it failing in real life. It does impose an extra
> pass over the input, but may well save a pass compared to the
> defensive groff -pet that I often use or to the rerun necessary when I
> forget to mention some or all of the filters.


If I remember correctly, it was an awk script printing out the
suggested pipeline to use.  One could then cut and paste that line.

	jaap

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 267 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [TUHS] Proliferation of options is great simplification of pipes, really?
@ 2021-02-23  2:47 M Douglas McIlroy
  2021-02-23 10:42 ` Jaap Akkerhuis
  2021-02-23 13:49 ` Ralph Corderoy
  0 siblings, 2 replies; 24+ messages in thread
From: M Douglas McIlroy @ 2021-02-23  2:47 UTC (permalink / raw)
  To: tuhs

> I can imagine a simple perl (or python or whatever) script that would run
> through groff input [and] determine which preprocessors are actually
> needed ...

Brian imagined such and implemented it way back when. Though I used
it, I've forgotten its name. One probably could have fooled it by
tricks like calling pic only in a .so file and perhaps renaming .so.
But I never heard of it failing in real life. It does impose an extra
pass over the input, but may well save a pass compared to the
defensive groff -pet that I often use or to the rerun necessary when I
forget to mention some or all of the filters.

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2021-02-24 19:38 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-22  2:34 [TUHS] Proliferation of options is great simplification of pipes, really? Will Senn
2021-02-22  3:32 ` G. Branden Robinson
2021-02-22  4:32   ` Dan Stromberg
2021-02-22  4:34   ` Will Senn
2021-02-22  5:45     ` G. Branden Robinson
2021-02-22 15:49   ` John P. Linderman
2021-02-22 15:57     ` William Cheswick
2021-02-22 16:03       ` John P. Linderman
2021-02-22 21:16         ` G. Branden Robinson
2021-02-22 16:02     ` Warner Losh
2021-02-22 16:12     ` Robert Clausecker
2021-02-22 17:15       ` John Cowan
2021-02-23  0:24       ` Steffen Nurpmeso
2021-02-22 21:14     ` G. Branden Robinson
2021-02-22  7:20 ` Rich Morin
2021-02-22 18:27 ` Jon Steinhart
2021-02-22 19:30   ` Richard Salz
2021-02-23  2:47 M Douglas McIlroy
2021-02-23 10:42 ` Jaap Akkerhuis
2021-02-23 13:23   ` Brantley Coile
2021-02-23 13:49 ` Ralph Corderoy
2021-02-23 15:04 Steve Simon
2021-02-24  2:42 ` M Douglas McIlroy
2021-02-24 19:38 Norman Wilson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).