Re: [TUHS] core

Re: [TUHS] core

[Nelson H. F. Beebe]

The discussion of the last few days about the Colossus is getting
somewhat off-topic for Unix heritage and history, but perhaps this
list of titles (truncated to 80 characters) and locations may be of
interest to some TUHS list readers.

------------------------------------------------------------------------

    Papers and book chapters about the Colossus at Bletchley Park

Journal- and subject specific Bibliography files are found at

	http://www.math.utah.edu/pub/tex/bib/NAME.bib

and author-specific ones under

	http://www.math.utah.edu/pub/bibnet/authors/N/

where N is the first letter of the bibliography filename.

The first line in each group contains the bibliography filename, and
the citation label of the cited publication.  DOIs are supplied
whenever they are available.  Paragraphs are sorted by filename.

adabooks.bib             	 Randell:1976:C
The COLOSSUS

annhistcomput.bib        	 Good:1979:EWC
Early Work on Computers at Bletchley
https://doi.org/10.1109/MAHC.1979.10011

annhistcomput.bib        	 deLeeuw:2007:HIS
Tunny and Colossus: breaking the Lorenz Schl{\"u}sselzusatz traffic
https://www.sciencedirect.com/science/article/pii/B978044451608450016X

babbage-charles.bib      	 Randell:1976:C
The COLOSSUS

cacm2010.bib             	 Anderson:2013:MNF
Max Newman: forgotten man of early British computing
https://doi.org/10.1145/2447976.2447986

computer1990.bib         	 Anonymous:1996:UWI
Update: WW II Colossus computer is restored to operation

cryptography.bib         	 Good:1979:EWC
Early Work on Computers at Bletchley

cryptography1990.bib     	 Anonymous:1997:CBP
The Colossus of Bletchley Park, IEEE Rev., vol. 41, no. 2, pp. 55--59 [Book Revi
https://doi.org/10.1109/MAHC.1997.560758

cryptography2000.bib     	 Rojas:2000:FCH
The first computers: history and architectures

cryptography2010.bib     	 Copeland:2010:CBG
Colossus: Breaking the German `Tunny' Code at Bletchley Park. An Illustrated His

cryptologia.bib          	 Michie:2002:CBW
Colossus and the Breaking of the Wartime ``Fish'' Codes

debroglie-louis.bib      	 Homberger:1970:CMN
The Cambridge mind: ninety years of the booktitle Cambridge Review, 1879--1969

dijkstra-edsger-w.bib    	 Randell:1980:C
The COLOSSUS

dyson-freeman-j.bib      	 Brockman:2015:WTA
What to think about machines that think: today's leading thinkers on the age of

fparith.bib              	 Randell:1977:CGC
Colossus: Godfather of the Computer

ieeeannhistcomput.bib    	 Anonymous:1997:CBP
The Colossus of Bletchley Park, IEEE Rev., vol. 41, no. 2, pp. 55--59 [Book Revi
https://doi.org/10.1109/MAHC.1997.560758

lncs1997b.bib            	 Carter:1997:BLC
The Breaking of the Lorenz Cipher: An Introduction to the Theory behind the Oper

lncs2000.bib             	 Sale:2000:CGL
Colossus and the German Lorenz Cipher --- Code Breaking in WW II

master.bib               	 Randell:1977:CGC
Colossus: Godfather of the Computer

mathcw.bib               	 Randell:1982:ODC
The Origins of Digital Computers: Selected Papers
http://dx.doi.org/10.1007/978-3-642-61812-3

rutherford-ernest.bib    	 Homberger:1970:CMN
The Cambridge mind: ninety years of the booktitle Cambridge Review, 1879--1969

rutherfordj.bib          	 Copeland:2010:CBG
Colossus: Breaking the German `Tunny' Code at Bletchley Park. An Illustrated His

sigcse1990.bib           	 Plimmer:1998:MIW
Machines invented for WW II code breaking
https://doi.org/10.1145/306286.306309

turing-alan-mathison.bib 	 Randell:1976:C
The COLOSSUS

von-neumann-john.bib     	 Randell:1982:ODC
The Origins of Digital Computers: Selected Papers

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

Re: [TUHS] core

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

On 06/21/2018 04:44 PM, Nelson H. F. Beebe wrote:
> The discussion of the last few days about the Colossus is getting somewhat 
> off-topic for Unix heritage and history

This is the the first time that this problem has come up.

Would it be worth while to have another (sub)mailing list that such 
topics can be moved to?

Given that the topics linger makes me think that people are interested 
in having the conversations.  So perhaps another venue is worthwhile.

Sort of like cctalk & cctech.

I make a motion for tuhs-off-topic.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] core

[Toby Thain]

On 2018-06-21 7:07 PM, Grant Taylor via TUHS wrote:
> On 06/21/2018 04:44 PM, Nelson H. F. Beebe wrote:
>> The discussion of the last few days about the Colossus is getting
>> somewhat off-topic for Unix heritage and history
> 
> This is the the first time that this problem has come up.
> 
> Would it be worth while to have another (sub)mailing list that such
> topics can be moved to?
> 
> Given that the topics linger makes me think that people are interested
> in having the conversations.  So perhaps another venue is worthwhile.
> 
> Sort of like cctalk & cctech.

...seems like you just named an appropriate venue for that conversation...

--T

> 
> I make a motion for tuhs-off-topic.
> 
> 
> 

Re: [TUHS] off-topic list

[Warren Toomey]

On Thu, Jun 21, 2018 at 05:07:48PM -0600, Grant Taylor via TUHS wrote:
>This is the the first time that this problem has come up.

No :-)

>Would it be worth while to have another (sub)mailing list that such 
>topics can be moved to? I make a motion for tuhs-off-topic.

I'm happy to make a mailing list here for it. But perhaps a name that
reflects its content. Computing history is maybe too generic. Ideas?

	Warren

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/21/2018 05:47 PM, Warren Toomey wrote:
>> This is the the first time that this problem has come up.

This is *not* the first time that this problem has come up.

Apparently I can't articulate myself properly.

> No :-)

;-)

> I'm happy to make a mailing list here for it. But perhaps a name that 
> reflects its content. Computing history is maybe too generic. Ideas?

"tuhs-open-discussion"  It's a play on Open Systems and still related to 
things tangentially related to Unix.

"tuhs-cantina"

¯\_(ツ)_/¯



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[Robert Brockway]

On Fri, 22 Jun 2018, Warren Toomey wrote:

> I'm happy to make a mailing list here for it. But perhaps a name that
> reflects its content. Computing history is maybe too generic. Ideas?

Other groups have often elected to go with -chat.

[TUHS-chat]?

Rob

Re: [TUHS] off-topic list

[Dave Horsfall]

On Fri, 22 Jun 2018, Warren Toomey wrote:

> I'm happy to make a mailing list here for it. But perhaps a name that
> reflects its content. Computing history is maybe too generic. Ideas?

COFF - Computer Old Fart Followers.

-- Dave

Re: [TUHS] off-topic list

[Arthur Krewat]

I was trying to come up with an acronym involving "old" - you got it ;)


On 6/22/2018 12:18 AM, Dave Horsfall wrote:
> COFF - Computer Old Fart Followers. 

Re: [TUHS] off-topic list

[Larry McVoy]

For the record, I'm fine with old stuff getting discussed on TUHS.
Even not Unix stuff.  We wandered into Linux/ext2 with Ted, that was fun.
We've wandered into the VAX history (UWisc got an 8600 and called it
"speedy" - what a mistake) and I learned stuff I didn't know, so that
was fun (and sounded just like history at every company I've worked for,
sadly :)

I think this list has self selected for adults who are reasonable.  So
we mostly are fine.

Warren, I'd ask your heavy hitters, like Ken, if it's ok if the list
wanders a bit.  If he and his ilk are fine, I'd just leave it all on
one list.  It's a fun list.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

Re: [TUHS] off-topic list

[Tim Bradshaw]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

On 22 Jun 2018, at 15:28, Larry McVoy <lm@mcvoy.com> wrote:
> 
> For the record, I'm fine with old stuff getting discussed on TUHS.
> Even not Unix stuff.  We wandered into Linux/ext2 with Ted, that was fun.
> We've wandered into the VAX history (UWisc got an 8600 and called it
> "speedy" - what a mistake) and I learned stuff I didn't know, so that
> was fun (and sounded just like history at every company I've worked for,
> sadly :)

As a perpetrator of some of this off-topicness I think (on reflection, I originally also wondered about a different list) that a good approach would be to allow anyone, if something is just clearly off-topic, to say 'please take this to a more appropriate forum', but if no-one does so it should be fine.  Obviously this requires people to actually do that, but I hope no-one sits and fumes without saying anything.  Equally obviously this is just my opinion.

(My main problem with off-topic things is now that the OSX mail client no longer seems to be able to reliably thread things which I find an astonishing regression, so I have to delete several threads.  I suspect it gets precisely no care and feeding from Apple, at least not from anyone who understands email.)

--tim


[-- Attachment #2: Type: text/html, Size: 5474 bytes --]

Re: [TUHS] off-topic list

[Larry McVoy]

On Fri, Jun 22, 2018 at 03:46:06PM +0100, Tim Bradshaw wrote:
> (My main problem with off-topic things is now that the OSX mail client no longer seems to be able to reliably thread things which I find an astonishing regression, so I have to delete several threads.  I suspect it gets precisely no care and feeding from Apple, at least not from anyone who understands email.)

Try mutt, it's what I use and it threads topics just fine.

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Larry McVoy wrote in <20180622145402.GT21272@mcvoy.com>:
 |On Fri, Jun 22, 2018 at 03:46:06PM +0100, Tim Bradshaw wrote:
 |> (My main problem with off-topic things is now that the OSX mail client \
 |> no longer seems to be able to reliably thread things which I find \
 |> an astonishing regression, so I have to delete several threads.  \
 |> I suspect it gets precisely no care and feeding from Apple, at least \
 |> not from anyone who understands email.)
 |
 |Try mutt, it's what I use and it threads topics just fine.
 --End of <20180622145402.GT21272@mcvoy.com>

I understood that as a request that people seem to have forgotten
that In-Reply-To: should be removed when doing the "Yz (Was: Xy)",
so that a new thread is created, actually.  Or take me: i seem to
never have learned that at first!

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On 06/22/2018 09:17 AM, Steffen Nurpmeso wrote:
> I understood that as a request that people seem to have forgotten that 
> In-Reply-To: should be removed when doing the "Yz (Was: Xy)", so that 
> a new thread is created, actually.  Or take me: i seem to never have 
> learned that at first!

I agree that removing In-Reply-To and References is a good thing to do 
when breaking a thread.  But not all MUAs make that possible, much less 
easy.  Which means that you're left with starting a new message to the 
same recipients.

I've also seen how (sub)threads tend to drift from their original intent 
and then someone modifies the subject some time later.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Grant Taylor via TUHS wrote in <b6ef82de-739a-ed8e-0e91-3abfa2fb5f07@spa\
mtrap.tnetconsulting.net>:
 |On 06/22/2018 09:17 AM, Steffen Nurpmeso wrote:
 |> I understood that as a request that people seem to have forgotten that 
 |> In-Reply-To: should be removed when doing the "Yz (Was: Xy)", so that 
 |> a new thread is created, actually.  Or take me: i seem to never have 
 |> learned that at first!
 |
 |I agree that removing In-Reply-To and References is a good thing to do 
 |when breaking a thread.  But not all MUAs make that possible, much less 
 |easy.  Which means that you're left with starting a new message to the 
 |same recipients.

True, but possible since some time, for the thing i maintain, too,
unfortunately.  It will be easy starting with the next release,
however.

 |I've also seen how (sub)threads tend to drift from their original intent 
 |and then someone modifies the subject some time later.

Yes.  Yes.  And then, whilst not breaking the thread stuff as
such, there is the "current funny thing to do", which also impacts
thread visualization sometimes.  For example replacing spaces with
tabulators so that the "is the same thread subject" compression
cannot work, so one first thinks the subject has really changed.
At the moment top posting seems to be a fascinating thing to do.
And you seem to be using DMARC, which irritates the list-reply
mechanism of at least my MUA.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]

On 06/22/2018 01:25 PM, Steffen Nurpmeso wrote:
> True, but possible since some time, for the thing i maintain, too, 
> unfortunately.  It will be easy starting with the next release, however.

I just spent a few minutes looking at how to edit headers in reply 
messages in Thunderbird and I didn't quickly find it.  (I do find an 
Add-On that allows editing messages in reader, but not the composer.)

> Yes.  Yes.  And then, whilst not breaking the thread stuff as such, 
> there is the "current funny thing to do", which also impacts thread 
> visualization sometimes.  For example replacing spaces with tabulators 
> so that the "is the same thread subject" compression cannot work, so 
> one first thinks the subject has really changed.

IMHO that's the wrong way to thread.  I believe threading should be done 
by the In-Reply-To: and References: headers.

I consider Subject: based threading to be a hack.  But it's a hack that 
many people use.  I think Thunderbird even uses it by default.  (I've 
long since disabled it.)

> At the moment top posting seems to be a fascinating thing to do.

I blame ignorance and the prevalence of tools that encourage such behavior.

> And you seem to be using DMARC, which irritates the list-reply mechanism 
> of at least my MUA.

Yes I do use DMARC as well as DKIM and SPF (w/ -all).  I don't see how 
me using that causes problems with "list-reply".

My working understanding is that "list-reply" should reply to the list's 
posting address in the List-Post: header.

List-Post: <mailto:tuhs@minnie.tuhs.org>

What am I missing or not understanding?



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Hello.

Grant Taylor via TUHS wrote in <89e5ae21-ccc0-5c84-837b-120a1a7d9e26@spa\
mtrap.tnetconsulting.net>:
 |On 06/22/2018 01:25 PM, Steffen Nurpmeso wrote:
 |> True, but possible since some time, for the thing i maintain, too, 
 |> unfortunately.  It will be easy starting with the next release, however.
 |
 |I just spent a few minutes looking at how to edit headers in reply 
 |messages in Thunderbird and I didn't quickly find it.  (I do find an 
 |Add-On that allows editing messages in reader, but not the composer.)

Oh, I do not know: i have never used a graphical MUA, only pine,
then mutt, and now, while anytime before now and then anyway, BSD
Mail.  Only once i implemented RFC 2231 support i started up Apple
Mail (of an up-to-date Snow Leopard i had back then) to see how
they do it, and that was a brainwave because they i think
misinterpreted RFC 2231 to only allow MIME paramaters to be split
in ten sections.  (But then i recall that i have retested that
with a Tiger Mail once i had a chance to, and there the bug was
corrected.)

Graphical user interfaces are a difficult abstraction, or tedious
to use.  I have to use a graphical browser, and it is always
terrible to enable Cookies for a site.  For my thing i hope we
will at some future day be so resilient that users can let go the
nmh mailer without loosing any freedom.

I mean, user interfaces are really a pain, and i think this will
not go away until we come to that brain implant which i have no
doubt will arrive some day, and then things may happen with
a think.  Things like emacs or Acme i can understand, and the
latter is even Unix like in the way it works.

Interesting that most old a.k.a. established Unix people give up
that Unix freedom of everything-is-a-file, that was there for
email access via nupas -- the way i have seen it in Plan9 (i never
ran a research Unix), at least -- in favour of a restrictive
graphical user interface!

 |> Yes.  Yes.  And then, whilst not breaking the thread stuff as such, 
 |> there is the "current funny thing to do", which also impacts thread 
 |> visualization sometimes.  For example replacing spaces with tabulators 
 |> so that the "is the same thread subject" compression cannot work, so 
 |> one first thinks the subject has really changed.
 |
 |IMHO that's the wrong way to thread.  I believe threading should be done 
 |by the In-Reply-To: and References: headers.
 |
 |I consider Subject: based threading to be a hack.  But it's a hack that 
 |many people use.  I think Thunderbird even uses it by default.  (I've 
 |long since disabled it.)

No, we use the same threading algorithm that Zawinski described
([1], "the threading algorithm that was used in Netscape Mail and
News 2.0 and 3.0").  I meant, in a threaded display, successive
follow-up messages which belong to the same thread will not
reiterate the Subject:, because it is the same one as before, and
that is irritating.

  [1] http://www.jwz.org/doc/threading.html

 |> At the moment top posting seems to be a fascinating thing to do.
 |
 |I blame ignorance and the prevalence of tools that encourage such behavior.

I vote for herdes instinct, but that is a non-scientific view.  We
here where i live had the "fold rear view mirrors even if not done
automatically by the car" until some young men started crashing
non-folded ones with baseball bats (say: who else would crash
rearview mirrors, i have not seen one doing this, but a lot of
damaged mirrors by then), and since a couple of years we have
"cut trees and hedges and replace with grass", which is really,
really terrible and i am not living at the same place than before.
Many birds, bats etc. think the same, so i am not alone; i hope
this ends soon.  I can have a walk to reach nightingales, though,
and still.  I would hope for "turning off the lights to be able to
see a true night sky", but .. i am dreaming.

 |> And you seem to be using DMARC, which irritates the list-reply mechanism 
 |> of at least my MUA.
 |
 |Yes I do use DMARC as well as DKIM and SPF (w/ -all).  I don't see how 
 |me using that causes problems with "list-reply".
 |
 |My working understanding is that "list-reply" should reply to the list's 
 |posting address in the List-Post: header.
 |
 |List-Post: <mailto:tuhs@minnie.tuhs.org>
 |
 |What am I missing or not understanding?

That is not how it works for the MUAs i know.  It is an
interesting idea.  And in fact it is used during the "mailing-list
address-massage" if possible.  But one must or should
differentiate in between a subscribed list and a non-subscribed
list, for example.  This does not work without additional
configuration (e.g., we have `mlist' and `mlsubscribe' commands to
make known mailing-lists to the machine), though List-Post: we use
for automatic configuration (as via `mlist').

And all in all this is a complicated topic (there are
Mail-Followup-To: and Reply-To:, for example), and before you say
"But what i want is a list reply!", yes, of course you are right.
But.  For my thing i hope i have found a sensible way through
this, and initially also one that does not deter users of console
MUA number one (mutt).

   --End of <89e5ae21-ccc0-5c84-837b-120a1a7d9e26@spamtrap.tnetconsulting.net>

Cheerio.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Toby Thain]

On 2018-06-23 10:49 AM, Steffen Nurpmeso wrote:
> Hello.
> 
> Grant Taylor via TUHS wrote in <89e5ae21-ccc0-5c84-837b-120a1a7d9e26@spa\
> mtrap.tnetconsulting.net>:
>  |On 06/22/2018 01:25 PM, Steffen Nurpmeso wrote:
>  |> True, but possible since some time, for the thing i maintain, too, 
>  |> unfortunately.  It will be easy starting with the next release, however.
>  |
>  |I just spent a few minutes looking at how to edit headers in reply 
>  |messages in Thunderbird and I didn't quickly find it.  (I do find an 
>  |Add-On that allows editing messages in reader, but not the composer.)
> 
> Oh, I do not know: i have never used a graphical MUA, only pine,
> then mutt, and now, while anytime before now and then anyway, BSD
> Mail.  Only once i implemented RFC 2231 support ...


Like most retrocomputing lists, only one off-topic list is ever really
needed: "Email RFCs and etiquette"

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/23/2018 08:49 AM, Steffen Nurpmeso wrote:
> Hello.

Hi,

> Oh, I do not know: i have never used a graphical MUA, only pine, then 
> mutt, and now, while anytime before now and then anyway, BSD Mail.

I agree that text mode MUAs from before the turn of the century do have 
a LOT more functionality than most GUI MUAs that came after that point 
in time.

Thankfully we are free to use what ever MUA we want to.  :-)

> they i think misinterpreted RFC 2231 to only allow MIME paramaters to be 
> split in ten sections.

I've frequently found things that MUAs (and other applications) don't do 
properly.  That's when it becomes a learning game to see what subset of 
the defined standard was implemented incorrectly and deciding if I want 
to work around it or not.

> Graphical user interfaces are a difficult abstraction, or tedious to use. 
> I have to use a graphical browser, and it is always terrible to enable 
> Cookies for a site.

Cookies are their own problem as of late.  All the "We use 
cookies...<bla>...<bla>...<bla>...." warnings that we now seem to have 
to accept get really annoying.  I want a cookie, or more likely a 
header, that says "I accept (first party) cookies." as a signal to not 
pester me.

> For my thing i hope we will at some future day be so resilient that 
> users can let go the nmh mailer without loosing any freedom.

Why do you have to let go of one tool?  Why can't you use a suite of 
tools that collectively do what you want?

> I mean, user interfaces are really a pain, and i think this will not 
> go away until we come to that brain implant which i have no doubt will 
> arrive some day, and then things may happen with a think.  Things like 
> emacs or Acme i can understand, and the latter is even Unix like in the 
> way it works.

You can keep the brain implant.  I have a desire to not have one.

> Interesting that most old a.k.a. established Unix people give up that 
> Unix freedom of everything-is-a-file, that was there for email access via 
> nupas -- the way i have seen it in Plan9 (i never ran a research Unix), 
> at least -- in favour of a restrictive graphical user interface!

Why do you have to give up one tool to start using a different tool?

I personally use Thunderbird as my primary MUA but weekly use mutt 
against the same mailbox w/ data going back 10+ years.  I extensively 
use Procmail to file messages into the proper folders.  I recently wrote 
a script that checks (copies of) messages that are being written to 
folders to move a message with that Message-ID from the Inbox to the 
Trash.  (The point being to remove copies that I got via To: or CC: when 
I get a copy from the mailing list.)

It seems to be like I'm doing things that are far beyond what 
Thunderbird can do by leveraging other tools to do things for me.  I 
also have a handful devices checking the same mailbox.

> No, we use the same threading algorithm that Zawinski described ([1], 
> "the threading algorithm that was used in Netscape Mail and News 2.0 
> and 3.0").  I meant, in a threaded display, successive follow-up messages 
> which belong to the same thread will not reiterate the Subject:, because 
> it is the same one as before, and that is irritating.
> 
>    [1] http://www.jwz.org/doc/threading.html

I *LOVE* threaded views.  I've been using threaded view for longer than 
I can remember.  I can't fathom not using threaded view.

I believe I mistook your statement to mean that you wanted to thread 
based on Subject: header, not the In-Reply-To: or References: header.

>>> And you seem to be using DMARC, which irritates the list-reply mechanism
>>> of at least my MUA.
>> 
>> Yes I do use DMARC as well as DKIM and SPF (w/ -all).  I don't see how
>> me using that causes problems with "list-reply".
>> 
>> My working understanding is that "list-reply" should reply to the list's
>> posting address in the List-Post: header.
>> 
>> List-Post: <mailto:tuhs@minnie.tuhs.org>
>> 
>> What am I missing or not understanding?
> 
> That is not how it works for the MUAs i know.  It is an interesting idea. 
> And in fact it is used during the "mailing-list address-massage" 
> if possible.  But one must or should differentiate in between a 
> subscribed list and a non-subscribed list, for example.  This does 
> not work without additional configuration (e.g., we have `mlist' and 
> `mlsubscribe' commands to make known mailing-lists to the machine), 
> though List-Post: we use for automatic configuration (as via `mlist').
> 
> And all in all this is a complicated topic (there are Mail-Followup-To: 
> and Reply-To:, for example), and before you say "But what i want is a 
> list reply!", yes, of course you are right.  But.  For my thing i hope 
> i have found a sensible way through this, and initially also one that 
> does not deter users of console MUA number one (mutt).

How does my use of DMARC irritate the list-reply mechanism of your MUA?

DMARC is completely transparent to message contents.  Sure, DKIM adds 
headers with a signature.  But I don't see anything about DKIM's use 
that has any impact on how any MUA handles a message.

Or are you referring to the fact that some mailing lists modify the 
From: header to be DMARC compliant?

Please elaborate on what you mean by "DMARC irritate the list-reply 
mechanism of your MUA".



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[Tom Ivar Helbekkmo via TUHS]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

Grant Taylor via TUHS <tuhs@minnie.tuhs.org> writes:

> How does my use of DMARC irritate the list-reply mechanism of your MUA?
>
> DMARC is completely transparent to message contents.  Sure, DKIM adds
> headers with a signature.  But I don't see anything about DKIM's use
> that has any impact on how any MUA handles a message.
>
> Or are you referring to the fact that some mailing lists modify the
> From: header to be DMARC compliant?
>
> Please elaborate on what you mean by "DMARC irritate the list-reply
> mechanism of your MUA".

Thanks, Grant!  It's always a pleasure when someone decides to follow
these things through, and I'm glad to see you doing it now (even though
I certainly wouldn't have the stamina to do it myself.

-tih
-- 
Most people who graduate with CS degrees don't understand the significance
of Lisp.  Lisp is the most important idea in computer science.  --Alan Kay

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [TUHS] off-topic list

[Michael Parson]

On Sat, 23 Jun 2018, Grant Taylor via TUHS wrote:
<snip>

> I recently wrote a script that checks (copies of) messages that are
> being written to folders to move a message with that Message-ID from
> the Inbox to the Trash.  (The point being to remove copies that I got
> via To: or CC: when I get a copy from the mailing list.)

The first rule in my .procmailrc does this with formail:

:0 Wh: msgid.lock
| /usr/local/bin/formail -D 8192 .msgid.cache

Doesn't move duplicates to the trash, it just prevents multiple copies
of the same message-id from being delivered at all.  The 8192 specfies
how large of a cache (stored in ~/.msgid.cache) of message-ids to keep
track of.

-- 
Michael Parson
Pflugerville, TX
KF5LGQ

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/23/2018 03:21 PM, Michael Parson wrote:
> The first rule in my .procmailrc does this with formail:

That works well for many.  But it does not work at all for me.

There are many additional factors that I have to consider:

  - I use different email addresses for different things.
  - Replies come to one email address.
  - Emails from the mailing list come to a different address.
  - Replies are direct and arrive sooner.
  - Emails from the mailing list are indirect and arrive later.
This means that the emails I get from the mailing list are to different 
addresses than message that were CCed directly to me.
  - I want the copy from the mailing list.
  - I want to not arbitrarily toss the direct reply in case the message 
never arrives from the mailing list.

> Doesn't move duplicates to the trash, it just prevents multiple copies 
> of the same message-id from being delivered at all.  The 8192 specfies 
> how large of a cache (stored in ~/.msgid.cache) of message-ids to keep 
> track of.

In light of the above facts, I can't simply reject subsequent copies of 
Message-IDs as that does not accomplish the aforementioned desires.

I also want to move the existing message to the Trash folder instead of 
blindly removing it so that I have the option of going to Trash and 
recovering it via my MUA if I feel the need or desire to do so.

What I do is quite likely extremely atypical.  But it does what I want. 
I have my own esoteric reasons (some are better than others) for doing 
what I do.  I'm happy to share if anyone is interested.



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[Larry McVoy]

On Sat, Jun 23, 2018 at 05:31:06PM -0600, Grant Taylor via TUHS wrote:
> On 06/23/2018 03:21 PM, Michael Parson wrote:
> >The first rule in my .procmailrc does this with formail:
> 
> That works well for many.  But it does not work at all for me.
> 
> There are many additional factors that I have to consider:
> 
>  - I use different email addresses for different things.

I think you misunderstood what formail is doing.  It's looking
at the Message-ID header.  The one for the email you sent
looks like:

Message-ID: <ea179b08-d58c-e116-ae07-cee882f306f8@spamtrap.tnetconsulting.net>

All formail is doing is keeping a recent cache of those ids and only letting
one get through.

So suppose someone reply-alls to you and the list (very common).  Without
the formail trick you'll see that message twice.

Re: [TUHS] off-topic list

[Larry McVoy]

Never mind, I didn't read far enough into your email, my bad.

On Sat, Jun 23, 2018 at 04:36:06PM -0700, Larry McVoy wrote:
> On Sat, Jun 23, 2018 at 05:31:06PM -0600, Grant Taylor via TUHS wrote:
> > On 06/23/2018 03:21 PM, Michael Parson wrote:
> > >The first rule in my .procmailrc does this with formail:
> > 
> > That works well for many.  But it does not work at all for me.
> > 
> > There are many additional factors that I have to consider:
> > 
> >  - I use different email addresses for different things.
> 
> I think you misunderstood what formail is doing.  It's looking
> at the Message-ID header.  The one for the email you sent
> looks like:
> 
> Message-ID: <ea179b08-d58c-e116-ae07-cee882f306f8@spamtrap.tnetconsulting.net>
> 
> All formail is doing is keeping a recent cache of those ids and only letting
> one get through.
> 
> So suppose someone reply-alls to you and the list (very common).  Without
> the formail trick you'll see that message twice.

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/23/2018 05:37 PM, Larry McVoy wrote:
> Never mind, I didn't read far enough into your email, my bad.

;-)

I'm always happy to have people (politely) challenge me.  I find it 
keeps me on my toes and helps me make sure that I didn't make a mistake.

So thank you for challenging me.  :-)



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[Dave Horsfall]

On Sat, 23 Jun 2018, Michael Parson wrote:

> The first rule in my .procmailrc does this with formail:

Anyone with any concept of security will not be running Procmail; it's not 
even supported by its author any more, due to its opaque syntax and likely 
vulnerabilities (it believes user-supplied headers and runs shell commands 
based upon them).

-- Dave VK2KFU

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/24/2018 08:53 PM, Dave Horsfall wrote:
> Anyone with any concept of security will not be running Procmail;

I'm going to have to throw a flag and cry foul on that play.

1)  "Anyone (with)" is a rather large group.
2)  "any concept of security" is a rather large (sub)group.
3)  "will not" is rather absolute.

I do believe that I have a better concept of security than many (but not 
all) of my colleagues.

  - I've got leading (if not bleeding) edge email security.
  - I've got full disk encryption on multiple server and workstations.
  - I use encrypted email when ever I can.
  - I play with 802.1ae MACsec (encrypted Ethernet frames).
  - I use salted hashes in proof of concepts.
  - I advocate for proper use of sudo...
  - ...and go out of my way to educate others on how to use sudo properly.

I could go on, but you probably don't care.  In short, I believe I fall 
squarely in categories #1 and #2.

Seeing as how I run procmail I invalidate #3.

So, I ask that you retract or amend your statement.  Or at least admit 
it's (partial) inaccuracies.

> it's not even supported by its author any more,

Many of the software packages that TUHS subscribers run on physical and 
/ or virtual systems are not supported by their authors any more.  Some 
of them are directly connected to the Internet too.

How many copies if (Open)VMS are running on (virtual) VAX (emulators)? 
Don't like (Open)VMS, then how about ancient versions of BSD or AT&T SYS 
V?  How many people are running wide array ancient BBSs on as many 
platforms?

How many people in corporate offices are running software that went End 
of Support 18 months ago?

Lack of support does not make something useless.

> due to its opaque syntax

I'm not aware of Procmail ever having claimed to have simple syntax.  I 
also believe that Procmail is not alone in this.

m4 is known for being obtuse, as is Sendmail, both of which are still 
used too.  SQL is notorious for being finicky.  I think there's a lot of 
C and C++ code that can fall in the same category.  (LISP … enough said)

> and likely vulnerabilities

Everything has vulnerabilities.  It's about how risky the (known) 
vulnerabilities are, and how likely they are to be exploited.  It's a 
balancing act.  Every administrator (or company directing said 
administrator) performs a risk assessment and makes a decision.

> (it believes user-supplied headers

Does the latest and greatest SMTP server from Google believe the 
information that the user supplies to it?  What about the Nginx web 
server that seems to be in vogue, does it believe the verb, URL, HTTP 
version and Host: header that users supply?

Does Mailman that hosts the TUHS mailing list believe the information 
that minnie provides that was originally user supplied?

Does your web browser believe and act on the information that the web 
server you are connecting to provided?

Applications are designed to trust the information that is provided to 
them.  Sure, run some sanity checks on it.  But ultimately it's the job 
of software to act on the user supplied information.

> and runs shell commands based upon them).

I've seen exceedingly few procmail recipes that actually run shell 
commands.  Almost all of the procmail recipes that I've seen and use do 
a very simple match for fixed text and file the message away in a 
folder.  These are functions built into procmail and NOT shell commands.

The very few procmail recipes that I've seen that do run shell commands 
are passing the message into STDIN of another utility that is itself 
designed to accept user supplied data, ideally in a safe way.

So, I believe your statement, "Anyone with any concept of security will 
not be running Procmail", is false, literally from word one.

If you want to poke fun at something, take a look at SpamAssassin and 
it's Perl.  Both of which are still actively supported.  Or how about 
all the NPM stuff that people are using in web page that are being 
pulled from places that God only knows.



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[arnold]

Dave Horsfall <dave@horsfall.org> wrote:

> On Sat, 23 Jun 2018, Michael Parson wrote:
>
> > The first rule in my .procmailrc does this with formail:
>
> Anyone with any concept of security will not be running Procmail; it's not 
> even supported by its author any more, due to its opaque syntax and likely 
> vulnerabilities (it believes user-supplied headers and runs shell commands 
> based upon them).
>
> -- Dave VK2KFU

So what is the alternative?  I've been using it for years with
a pretty static setup to route incoming mail to different places.
I need *something* to do what it does.

Thanks,

Arnold

Re: [TUHS] off-topic list

[Bakul Shah]

On Mon, 25 Jun 2018 00:15:42 -0600 arnold@skeeve.com wrote:
> 
> > On Sat, 23 Jun 2018, Michael Parson wrote:
> >
> > > The first rule in my .procmailrc does this with formail:
> >
> > Anyone with any concept of security will not be running Procmail; it's not 
> > even supported by its author any more, due to its opaque syntax and likely 
> > vulnerabilities (it believes user-supplied headers and runs shell commands 
> > based upon them).
> >
> > -- Dave VK2KFU
> 
> So what is the alternative?  I've been using it for years with
> a pretty static setup to route incoming mail to different places.
> I need *something* to do what it does.

My crude method has worked better than anything else for me.
[in used for over two decades]

As I read only a subset of messages from mailing lists, if I
directly filed such messages into their own folders, I would
either have to waste more time scanning much larger mail
folders &/or miss paying attention to some messages even
once[1].

Fortunately, in MH one can use named sequences (that map to
set of picked messages). In essence, I use sequences as "work
space" and other folders as storage space.

For example

  $ <run spam filtering script>
  $ pick -seq me -to bakul -or -cc bakul -or -bcc bakul 
  $ pick -seq tuhs -to tuhs@tuhs -or -cc tuhs@tuhs
  ...

When I have some idle time, I type

  $ inc # to incorporate new messages into inbox
  $ pickall # my script for creating sequences

Next I scan these sequences in a priority order to see if
anything seems interesting and then process these messages.
Once done, I file them into their own folders and move on to
the next sequence. The whole process takes a few minutes at
most[2] and at the end the inbox is "zeroed"! By zeroing it
each time, I ensure that the next time I will be processing
only new messages, and typically spend less than a second per
message summary line.

[1] This happens to me on Apple Mail.
[2] Unless I decide to reply!

Re: [TUHS] off-topic list

[Michael Parson]

On 2018-06-25 01:15, arnold@skeeve.com wrote:
> Dave Horsfall <dave@horsfall.org> wrote:
> 
>> On Sat, 23 Jun 2018, Michael Parson wrote:
>> 
>> > The first rule in my .procmailrc does this with formail:
>> 
>> Anyone with any concept of security will not be running Procmail; it's 
>> not
>> even supported by its author any more, due to its opaque syntax and 
>> likely
>> vulnerabilities (it believes user-supplied headers and runs shell 
>> commands
>> based upon them).
>> 
>> -- Dave VK2KFU
> 
> So what is the alternative?  I've been using it for years with
> a pretty static setup to route incoming mail to different places.
> I need *something* to do what it does.

Sieve[0] is what I've seen suggested a bit.  I've skimmed over the docs 
some, but haven't invested the time in figuring out how much effort 
would be involved in replacing procmail with it, at first glance, it 
does not seem to be a drop-in replacement.

-- 
Michael Parson
Pflugerville, TX
KF5LGQ

[0] http://sieve.info/

Re: [TUHS] off-topic list

[arnold]

Michael Parson <mparson@bl.org> wrote:

> On 2018-06-25 01:15, arnold@skeeve.com wrote:
> > So what is the alternative?  I've been using it [procmail] for years with
> > a pretty static setup to route incoming mail to different places.
> > I need *something* to do what it does.
>
> Sieve[0] is what I've seen suggested a bit.  I've skimmed over the docs 
> some, but haven't invested the time in figuring out how much effort 
> would be involved in replacing procmail with it, at first glance, it 
> does not seem to be a drop-in replacement.
>
> -- 
> Michael Parson
> Pflugerville, TX
> KF5LGQ
>
> [0] http://sieve.info/

Thanks for the answer.  This looks medium complicated.

It's starting to feel like I could do this myself in my favorite
programming language (gawk) simply by writing a library to parse 
the headers and then writing awk code to check things and send emails
to the right place.

If I didn't already have more side projects than I can curently handle ...

Thanks,

Arnold

Re: [TUHS] off-topic list

[arnold]

> > [0] http://sieve.info/

This looks like it might be useful:

https://github.com/tonioo/sievelib

It's a client side library and would need a wrapper to make
it standalone.

Thanks,

Arnold

Re: [TUHS] off-topic list

[Adam Sampson]

arnold@skeeve.com writes:

> So what is the alternative?  I've been using it for years with
> a pretty static setup to route incoming mail to different places.
> I need *something* to do what it does.

I use maildrop from the Courier project, which does pretty much the same
job as procmail, with a C-ish filtering language:

http://www.courier-mta.org/maildrop/

I switched over from procmail to maildrop in 2004 and it's worked nicely
for me. There are also a couple of different filtering languages that
Exim understands, if that's your MTA of choice.

-- 
Adam Sampson <ats@offog.org>                         <http://offog.org/>

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 1750 bytes --]

On 06/25/2018 12:15 AM, arnold@skeeve.com wrote:
> So what is the alternative?  I've been using it for years with a 
> pretty static setup to route incoming mail to different places.  I need 
> *something* to do what it does.

As others have pointed out, Sieve and Maildrop are a couple of options.

I've looked at both of them a few times, somewhat in earnest when I last 
built my mail server,  but didn't feel that they would be drop in 
replacements for my existing LDA needs.

Perhaps my limitations are my own making.  I use Maildir and want an LDA 
that is compatible with that.  At (currently) 453 procmail recipes, I'd 
like something that is closer to a drop in replacement.  I will rewrite 
things if I must, but I'd rather not if it's not required.

I think one of the reasons the LDA space is languishing is that many 
mail servers seem to migrating to mail stores that are more centralized 
/ virtualized under one unix account, and they come with their own 
purpose built LDA.

The other option that Arnold mentioned, writing your own LDA, seems 
somewhat unpleasant to me.  Can it be done, absolutely.  I'm afraid that 
anything I would produce would likely share similar security problems 
that Procmail may very well have.  Which leaves me wondering, am I 
better off using a tool with a questionable security posture and others 
looking at it and trying to abuse it -or- my own tool with a completely 
unknown security posture that nobody else is looking at.  Thus I choose 
the lesser of the evils and continue to use Procmail.

Thanks to Michael and Adam for mentioning Sieve and Maildrop 
(respectively) while I was too lazy to comment on my phone in bed.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Derek Fawcus]

On Mon, Jun 25, 2018 at 12:15:42AM -0600, arnold@skeeve.com wrote:
> 
> So what is the alternative?

Well, I've generally found that > 90% of my mail filtering needs
can be handled by the use of extension addresses (see my From: line).

I started doing that with qmail back in the 90s, and these days
tend to use a similar mechanism which is present in postfix.
I've even found that some sendmail installs (as a previous employer used)
support as similar mechanism.

It is only occasionally that I have to resort to procmail (say for
parsing stuff out of Atlassian tools messages), and can then make
its use simpler by having recipices which match a bit, then feed
the message back in to a extension address.  At which point delivery
may occur, or some other procmail rules may be run.

DF

[TUHS] email filtering (was Re: off-topic list)

[Perry E. Metzger]

On Mon, Jun 25, 2018 at 12:15:42AM -0600, arnold@skeeve.com wrote:
>
> So what is the alternative?

I subscribe to a truly ridiculous number of email lists, and I run my
own mail server, so I thought I'd mention my Rube Goldberg style setup
given that it's being discussed. For those that don't like how long
this is, the main keys are "IMAP + sieve for filing all mailing lists
into their own IMAP folder". The rest is pretty boring.

Boring part:

0) My MTA is Postfix, which allows even a relatively small site to get
pretty damn good anti-spam capabilities. It's also really well written
(it's a flock of small tools each of which does only one thing well)
and quite secure, which is important. I have most of the interesting
bells and whistles (like opportunistic TLS) turned on.

1) Email for me goes through procmail to run it through a logging
system, then spam assassin to tag whatever spam got past Postfix, and
finally it's passed to Dovecot's "sieve" system for final filing in
the IMAP server.

2) Sieve breaks up my incoming email into many, many
mailboxes. There's a box for every mailing list I'm on, so I can read
them more or less like newsgroups. I don't put any email for a list
into my main inbox, which gets only mail addressed to me personally so
I can reply to such messages very fast.

Sieve is a nice, standardized language for mailbox filtering, and the
Dovecot implementation works quite well. It's flexible enough that I
can deal with the fact that many mailing lists are hard to pick out
from the mail stream thanks to their operators not following standards.

Spam that made it past Postfix into Spam Assassin might or might not
really be spam, so I have Spam Assassin tag it rather than deleting
it, and Sieve is instructed to file that into its own box just in
case. A cron job expires spam after a couple of weeks so that mailbox
doesn't get too large.

3) As I hinted, my email is stored in an IMAP server, specifically
Dovecot. This allows me to read my mail with a dozen different tools,
including my phone, a couple different MUAs on the desktop, etc. I
dislike (see below) that this limits my choice in tools to process the
mail, but I really need the multi-MUA setup, and I can't afford to
move the mail onto any of my diverse end systems because I need to be
able to read it on all of them.

4) I used MH for many years, and I really wish MH/NMH worked with IMAP
properly, because like others here, I really loved its scriptability,
but the need to use multiple MUAs trumps it at this point. It's also
necessary to communicate with muggles/mundanes/ordinary folk too often
and most of them use HTML email and so I need to be able to read it
even if I don't generate it, which means some use of GUI emailers.

5) As for the future, I'm hoping at some point to have something that
works sort of like MH did but which can speak native IMAP, and I'm
hoping to start using Emacs as an MUA most of the time if I can find a
client that will handle HTML email + IMAP a bit better than most of
them do. (Stop looking at me that way for saying Emacs — I've been
using it as my editor since 1983 and my fingers just don't know any
better any more.)

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Grant Taylor via TUHS wrote in <ce6f617c-cf8e-63c6-8186-27e09c78020c@spa\
mtrap.tnetconsulting.net>:
 |On 06/23/2018 08:49 AM, Steffen Nurpmeso wrote:
 |> Oh, I do not know: i have never used a graphical MUA, only pine, then 
 |> mutt, and now, while anytime before now and then anyway, BSD Mail.
 |
 |I agree that text mode MUAs from before the turn of the century do have 
 |a LOT more functionality than most GUI MUAs that came after that point 
 |in time.
 |
 |Thankfully we are free to use what ever MUA we want to.  :-)

Absolutely true.  And hoping that, different to web browsers, no
let me call it pseudo feature race is started that results in less
diversity instead of anything else.

 |> they i think misinterpreted RFC 2231 to only allow MIME paramaters to be 
 |> split in ten sections.
 |
 |I've frequently found things that MUAs (and other applications) don't do 
 |properly.  That's when it becomes a learning game to see what subset of 
 |the defined standard was implemented incorrectly and deciding if I want 
 |to work around it or not.

If there is the freedom for a decision.  That is how it goes, yes.
For example one may have renamed "The Ethiopian Jewish Exodus --
Narratives of the Migration Journey to Israel 1977 - 1985" to
"Falasha" in order to use the book as an email attachment.  Which
would be pretty disappointing.  But it is also nice to see that
there are standards which were not fully thought through and
required backward incompatible hacks to fill the gaps.  Others
like DNS are pretty perfect and scale fantastic.  Thus.

 |> Graphical user interfaces are a difficult abstraction, or tedious \
 |> to use. 
 |> I have to use a graphical browser, and it is always terrible to enable 
 |> Cookies for a site.
 |
 |Cookies are their own problem as of late.  All the "We use 
 |cookies...<bla>...<bla>...<bla>...." warnings that we now seem to have 
 |to accept get really annoying.  I want a cookie, or more likely a 
 |header, that says "I accept (first party) cookies." as a signal to not 
 |pester me.

Ah, it has become a real pain.  It is ever so astounding how much
HTML5 specifics, scripting and third party analytics can be
involved for a page that would have looked better twenty years
ago, or say whenever CSS 2.0 came around.  Today almost each and
every commercial site i visit is in practice a denial of service
attack.  For me and my little box at least, and without gaining
any benefit from that!

The thing is, i do not want be lied to, but what those boxes you
refer to say are often plain lies.  It has nothing to do with my
security, or performance boosts, or whatever ... maximally so in
a juristically unobjectionable way.  Very disappointing.

 |> For my thing i hope we will at some future day be so resilient that 
 |> users can let go the nmh mailer without loosing any freedom.
 |
 |Why do you have to let go of one tool?  Why can't you use a suite of 
 |tools that collectively do what you want?

It is just me and my desire to drive this old thing forward so
that it finally will be usable for something real.  Or what i deem
for that.  I actually have no idea of nmh, but i for one think the
sentence of the old BSD Mail manual stating it is an "intelligent
mail processing system, which has a command syntax reminiscent of
ed(1) with lines replaced by messages" has always been a bit
excessive.  And the fork i maintain additionally said it "is also
usable as a mail batch language, both for sending and receiving
mail", adding onto that.

 |> I mean, user interfaces are really a pain, and i think this will not 
 |> go away until we come to that brain implant which i have no doubt will 
 |> arrive some day, and then things may happen with a think.  Things like 
 |> emacs or Acme i can understand, and the latter is even Unix like in the 
 |> way it works.
 |
 |You can keep the brain implant.  I have a desire to not have one.

Me too, me too.  Oh, me too.  Unforgotten those american black and
white films with all those sneaky alien attacks, and i was
a teenager when i saw them.  Yeah, but in respect to that i am
afraid the real rabble behind the real implants will be even
worse, and drive General Motors or something. ^.^  I have no
idea..  No, but.. i really do not think future humans will get
around that, just imagine automatic 911 / 112, biologic anomaly
detection, and then you do not need to learn latin vocabulary but
can buy the entire set for 99.95$, and have time for learning
something real important.  For example.
It is all right, just be careful and ensure that not all police
men go to the tourist agency to book a holiday in Texas at the
very same time.  I think this is manageable.

 |> Interesting that most old a.k.a. established Unix people give up that 
 |> Unix freedom of everything-is-a-file, that was there for email access \
 |> via 
 |> nupas -- the way i have seen it in Plan9 (i never ran a research Unix), 
 |> at least -- in favour of a restrictive graphical user interface!
 |
 |Why do you have to give up one tool to start using a different tool?
 |
 |I personally use Thunderbird as my primary MUA but weekly use mutt 
 |against the same mailbox w/ data going back 10+ years.  I extensively 
 |use Procmail to file messages into the proper folders.  I recently wrote 
 |a script that checks (copies of) messages that are being written to 
 |folders to move a message with that Message-ID from the Inbox to the 
 |Trash.  (The point being to remove copies that I got via To: or CC: when 
 |I get a copy from the mailing list.)
 |
 |It seems to be like I'm doing things that are far beyond what 
 |Thunderbird can do by leveraging other tools to do things for me.  I 
 |also have a handful devices checking the same mailbox.

You say it.  In the research Unix nupas mail (as i know it from
Plan9) all those things could have been done with a shell script
and standard tools like grep(1) and such.  Even Thunderbird would
simply be a maybe even little nice graphical application for
display purposes.  The actual understanding of storage format and
email standards would lie solely within the provider of the file
system.  Now you use several programs which all ship with all the
knowledge.  I for one just want my thing to be easy and reliable
controllable via a shell script.  You could replace procmail
(which is i think perl and needs quite some perl modules) with
a monolithic possibly statically linked C program.  Then.  With
full error checking etc.  This is a long road ahead, for my thing.

 |> No, we use the same threading algorithm that Zawinski described ([1], 
 |> "the threading algorithm that was used in Netscape Mail and News 2.0 
 |> and 3.0").  I meant, in a threaded display, successive follow-up \
 |> messages 
 |> which belong to the same thread will not reiterate the Subject:, because 
 |> it is the same one as before, and that is irritating.
 |> 
 |>    [1] http://www.jwz.org/doc/threading.html
 |
 |I *LOVE* threaded views.  I've been using threaded view for longer than 
 |I can remember.  I can't fathom not using threaded view.
 |
 |I believe I mistook your statement to mean that you wanted to thread 
 |based on Subject: header, not the In-Reply-To: or References: header.

It seems you did not have a chance not do.  My fault, sorry.

 |>>> And you seem to be using DMARC, which irritates the list-reply \
 |>>> mechanism
 |>>> of at least my MUA.
 |>> 
 |>> Yes I do use DMARC as well as DKIM and SPF (w/ -all).  I don't see how
 |>> me using that causes problems with "list-reply".
 |>> 
 |>> My working understanding is that "list-reply" should reply to the list's
 |>> posting address in the List-Post: header.
 |>> 
 |>> List-Post: <mailto:tuhs@minnie.tuhs.org>
 |>> 
 |>> What am I missing or not understanding?
 |> 
 |> That is not how it works for the MUAs i know.  It is an interesting \
 |> idea. 
 |> And in fact it is used during the "mailing-list address-massage" 
 |> if possible.  But one must or should differentiate in between a 
 |> subscribed list and a non-subscribed list, for example.  This does 
 |> not work without additional configuration (e.g., we have `mlist' and 
 |> `mlsubscribe' commands to make known mailing-lists to the machine), 
 |> though List-Post: we use for automatic configuration (as via `mlist').
 |> 
 |> And all in all this is a complicated topic (there are Mail-Followup-To: 
 |> and Reply-To:, for example), and before you say "But what i want is a 
 |> list reply!", yes, of course you are right.  But.  For my thing i hope 
 |> i have found a sensible way through this, and initially also one that 
 |> does not deter users of console MUA number one (mutt).
 |
 |How does my use of DMARC irritate the list-reply mechanism of your MUA?

So ok, it does not, actually.  It chooses your "Grant Taylor via
TUHS" which ships with the TUHS address, so one may even see this
as an improvement to DMARC-less list replies, which would go to
TUHS, with or without the "The Unix Heritage Society".
I am maybe irritated by the 'dkim=fail reason="signature
verification failed"' your messages produce.  It would not be good
to filter out failing DKIMs, at least on TUHS.

 |DMARC is completely transparent to message contents.  Sure, DKIM adds 
 |headers with a signature.  But I don't see anything about DKIM's use 
 |that has any impact on how any MUA handles a message.
 |
 |Or are you referring to the fact that some mailing lists modify the 
 |From: header to be DMARC compliant?

Ach, it has been said without much thought.  Maybe yes.  There is
a Reply-To: of yours.

 |Please elaborate on what you mean by "DMARC irritate the list-reply 
 |mechanism of your MUA".

My evil subconsciousness maybe does not like the MARC and DKIM
standards.  That could be it.  I do not know anything better,
maybe the next OpenPGP and S/MIME standards will include
a checksum of some headers in signed-only data, will specify that
some headers have to be repeated in the MIME part and extend the
data to-be-verified to those, whatever.  I have not read the
OpenPGP draft nor anyting S/MIMEish after RFC 5751.

A nice sunday i wish.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

On 06/23/2018 04:38 PM, Steffen Nurpmeso wrote:
> Absolutely true.  And hoping that, different to web browsers, no let me 
> call it pseudo feature race is started that results in less diversity 
> instead of anything else.

I'm not sure I follow.  I feel like we do have the choice of MUAs or web 
browsers.  Sadly some choices are lacking compared to other choices. 
IMHO the maturity of some choices and lack there of in other choices 
does not mean that we don't have choices.

> If there is the freedom for a decision.  That is how it goes, yes.  For 
> example one may have renamed "The Ethiopian Jewish Exodus -- Narratives 
> of the Migration Journey to Israel 1977 - 1985" to "Falasha" in order to 
> use the book as an email attachment.  Which would be pretty disappointing.

I don't follow what you're getting at.

> But it is also nice to see that there are standards which were not fully 
> thought through and required backward incompatible hacks to fill the gaps.

Why is that a "nice" thing?

> Others like DNS are pretty perfect and scale fantastic.  Thus.

Yet I frequently see DNS problems for various reasons.  Not the least of 
which is that many clients do not gracefully fall back to the secondary 
DNS server when the primary is unavailable.

> Ah, it has become a real pain.  It is ever so astounding how much HTML5 
> specifics, scripting and third party analytics can be involved for a page 
> that would have looked better twenty years ago, or say whenever CSS 2.0 
> came around.

I'm going to disagree with you there.  IMHO the standard is completely 
separate with what people do with it.

> Today almost each and every commercial site i visit is in practice 
> a denial of service attack.  For me and my little box at least, and 
> without gaining any benefit from that!

I believe those webmasters have made MANY TERRIBLE choices and have 
ended up with the bloat that they now have.  -  I do not blame the 
technology.  I blame what the webmasters have done with the technology.

Too many people do not know what they are doing and load "yet another 
module" to do something that they can already do with what's already 
loaded on their page.  But they don't know that.  They just glue things 
togehter until they work.  Even if that means that they are loading the 
same thing multiple times because multiple of their 3rd party components 
loads a common module themselves.  This is particularly pernicious with 
JavaScript.

> The thing is, i do not want be lied to, but what those boxes you refer 
> to say are often plain lies.  It has nothing to do with my security, 
> or performance boosts, or whatever ... maximally so in a juristically 
> unobjectionable way.  Very disappointing.

I don't understand what you're saying.

Who is lying to you?  How and where are they lying to you?

What boxes are you saying I'm referring to?

> It is just me and my desire to drive this old thing forward so that it 
> finally will be usable for something real.  Or what i deem for that.

I don't see any reason that you can't continue to use and improve what 
ever tool you want to.

> I actually have no idea of nmh, but i for one think the sentence of 
> the old BSD Mail manual stating it is an "intelligent mail processing 
> system, which has a command syntax reminiscent of ed(1) with lines 
> replaced by messages" has always been a bit excessive.  And the fork i 
> maintain additionally said it "is also usable as a mail batch language, 
> both for sending and receiving mail", adding onto that.

What little I know about the MH type mail stores and associated 
utilities are indeed quite powerful.  I think they operate under the 
premise that each message is it's own file and that you work in 
something akin to a shell if not your actual OS shell.  I think the MH 
commands are quite literally unix command that can be called from the 
unix shell.  I think this is in the spirit of simply enhancing the shell 
to seem as if it has email abilities via the MH commands.  Use any 
traditional unix text processing utilities you want to manipulate email.

MH has always been attractive to me, but I've never used it myself.

> You say it.  In the research Unix nupas mail (as i know it from Plan9) 
> all those things could have been done with a shell script and standard 
> tools like grep(1) and such.

I do say that and I do use grep, sed, awk, formail, procmail, cp, mv, 
and any number of traditional unix file / text manipulation utilities on 
my email weekly.  I do this both with the Maildir (which is quite 
similar to MH) on my mail server and to the Thumderbird message store 
that is itself a variant of Maildir with a file per message in a 
standard directory structure.

> Even Thunderbird would simply be a maybe even little nice graphical 
> application for display purposes.

The way that I use Thunderbird, that's exactly what it is.  A friendly 
and convenient GUI front end to access my email.

> The actual understanding of storage format and email standards would 
> lie solely within the provider of the file system.

The emails are stored in discrete files that themselves are effectively 
mbox files with one message therein.

> Now you use several programs which all ship with all the knowledge.

I suppose if you count greping for a line in a text file as knowledge of 
the format, okay.

egrep "^Subject: " message.txt

There's nothing special about that.  It's a text file with a line that 
looks like this:

Subject: Re: [TUHS] off-topic list

> I for one just want my thing to be easy and reliable controllable via 
> a shell script.

That's a laudable goal.  I think MH is very conducive to doing that.

> You could replace procmail (which is i think perl and needs quite some 
> perl modules) with a monolithic possibly statically linked C program.

I'm about 95% certain that procmail is it's own monolithic C program. 
I've never heard of any reference to Perl in association with procmail. 
Are you perhaps thinking of a different local delivery agent?

> Then.  With full error checking etc.  This is a long road ahead, for 
> my thing.

Good luck to you.

> So ok, it does not, actually.  It chooses your "Grant Taylor via TUHS" 
> which ships with the TUHS address, so one may even see this as an 
> improvement to DMARC-less list replies, which would go to TUHS, with or 
> without the "The Unix Heritage Society".

Please understand, that's not how I send the emails.  I send them with 
my name and my email address.  The TUHS mailing list modifies them.

Aside:  I support the modification that it is making.

> I am maybe irritated by the 'dkim=fail reason="signature verification 
> failed"' your messages produce.  It would not be good to filter out 
> failing DKIMs, at least on TUHS.

Okay.  That is /an/ issue.  But I believe it's not /my/ issue to solve.

My server DKIM signs messages that it sends out.  From everything that 
I've seen and tested (and I actively look for problems) the DKIM 
signatures are valid and perfectly fine.

That being said, the TUHS mailing list modifies message in the following 
ways:

1)  Modifies the From: when the sending domain uses DMARC.
2)  Modifies the Subject to prepend "[TUHS] ".
3)  Modifies the body to append a footer.

All three of these actions modify the data that receiving DKIM filters 
calculate hashes based on.  Since the data changed, obviously the hash 
will be different.

I do not fault THUS for this.

But I do wish that TUHS stripped DKIM and associated headers of messages 
going into the mailing list.  By doing that, there would be no data to 
compare to that wouldn't match.

I think it would be even better if TUHS would DKIM sign messages as they 
leave the mailing list's mail server.

> Ach, it has been said without much thought.  Maybe yes.  There is a 
> Reply-To: of yours.

I do see the Reply-To: header that you're referring to.  I don't know 
where it's coming from.  I am not sending it.  I just confirmed that 
it's not in my MUA's config, nor is it in my sent Items.

> My evil subconsciousness maybe does not like the MARC and DKIM standards. 
> That could be it.  I do not know anything better, maybe the next 
> OpenPGP and S/MIME standards will include a checksum of some headers 
> in signed-only data, will specify that some headers have to be repeated 
> in the MIME part and extend the data to-be-verified to those, whatever. 
> I have not read the OpenPGP draft nor anyting S/MIMEish after RFC 5751.

I don't know if PGP or S/MIME will ever mandate anything about headers 
which are structurally outside of their domain.

I would like to see an option in MUAs that support encrypted email for 
something like the following:

    Subject:  (Subject in encrypted body.)

Where the encrypted body included a header like the following:

    Encrypted-Subject: Re: [TUHS] off-topic list

I think that MUAs could then display the subject that was decrypted out 
of the encrypted body.

> A nice sunday i wish.

You too.



-- 
Grant. . . .
unix || die

Re: [TUHS] off-topic list

[Michael Kjörling]

On 23 Jun 2018 18:18 -0600, from tuhs@minnie.tuhs.org (Grant Taylor via TUHS):
>> Now you use several programs which all ship with all the knowledge.
> 
> I suppose if you count greping for a line in a text file as
> knowledge of the format, okay.
> 
> egrep "^Subject: " message.txt
> 
> There's nothing special about that.  It's a text file with a line
> that looks like this:
> 
> Subject: Re: [TUHS] off-topic list

The problem, of course (and I hope this is keeping this Unixy enough),
with that approach is that it won't handle headers split across
multiple lines (I'm looking at you, Received:, but you aren't alone),
and that it'll match lines in the body of the message as well (such as
the "Subject: " line in the body of your message), unless the body
happens to be e.g. Base64 encoded which instead complicates searching
for non-header material.

For sure neither is insurmountable even with standard tools, but it
does require a bit more complexity than a simple egrep to properly
parse even a single message, let alone a combination of multiple ones
(as seen in mbox mailboxes, for example). At that point having
specific tools, such as formail, that understand the specific file
format does start to make sense...

There isn't really much conceptual difference between writing, say,
    formail -X Subject: < message.txt
and
    egrep "^Subject: " message.txt
but the way the former handles certain edge cases is definitely better
than that of the latter.

Make everything as simple as possible, but not simpler. (That goes for
web pages, too, by the way.)


> But I do wish that TUHS stripped DKIM and associated headers of
> messages going into the mailing list.  By doing that, there would be
> no data to compare to that wouldn't match.
> 
> I think it would be even better if TUHS would DKIM sign messages as
> they leave the mailing list's mail server.

I believe the correct way would indeed be to validate, strip and
possibly re-sign. That way, everyone would (should) be making correct
claims about a message's origin.

FWIW, SPF presents a similar problem with message forwarding without
address rewriting... so it's definitely not just DKIM.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
  “The most dangerous thought that you can have as a creative person
              is to think you know what you’re doing.” (Bret Victor)

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Michael Kjörling wrote in <20180624100438.GY10129@h-174-65.A328.priv.ba\
hnhof.se>:
 |On 23 Jun 2018 18:18 -0600, from tuhs@minnie.tuhs.org (Grant Taylor \
 |via TUHS):
 |>> Now you use several programs which all ship with all the knowledge.
 |> 
 |> I suppose if you count greping for a line in a text file as
 |> knowledge of the format, okay.
 |> 
 |> egrep "^Subject: " message.txt
 |> 
 |> There's nothing special about that.  It's a text file with a line
 |> that looks like this:
 |> 
 |> Subject: Re: [TUHS] off-topic list
 |
 |The problem, of course (and I hope this is keeping this Unixy enough),
 |with that approach is that it won't handle headers split across
 |multiple lines (I'm looking at you, Received:, but you aren't alone),
 |and that it'll match lines in the body of the message as well (such as
 |the "Subject: " line in the body of your message), unless the body
 |happens to be e.g. Base64 encoded which instead complicates searching
 |for non-header material.
 |
 |For sure neither is insurmountable even with standard tools, but it
 |does require a bit more complexity than a simple egrep to properly
 |parse even a single message, let alone a combination of multiple ones
 |(as seen in mbox mailboxes, for example). At that point having
 |specific tools, such as formail, that understand the specific file
 |format does start to make sense...
 |
 |There isn't really much conceptual difference between writing, say,
 |    formail -X Subject: < message.txt
 |and
 |    egrep "^Subject: " message.txt
 |but the way the former handles certain edge cases is definitely better
 |than that of the latter.
 |
 |Make everything as simple as possible, but not simpler. (That goes for
 |web pages, too, by the way.)

 |> But I do wish that TUHS stripped DKIM and associated headers of
 |> messages going into the mailing list.  By doing that, there would be
 |> no data to compare to that wouldn't match.
 |> 
 |> I think it would be even better if TUHS would DKIM sign messages as
 |> they leave the mailing list's mail server.
 |
 |I believe the correct way would indeed be to validate, strip and
 |possibly re-sign. That way, everyone would (should) be making correct
 |claims about a message's origin.

DKIM reuses the *SSL key infrastructure, which is good.  (Many eyes
see the code in question.)  It places records in DNS, which is
also good, now that we have DNS over TCP/TLS and (likely) DTLS.
In practice however things may differ and to me DNS security is
all in all not given as long as we get to the transport layer
security.

I personally do not like DKIM still, i have opendkim around and
thought about it, but i do not use it, i would rather wish that
public TLS certificates could also be used in the same way as
public S/MIME certificates or OpenPGP public keys work, then only
by going to a TLS endpoint securely once, there could be
end-to-end security.  I am not a cryptographer, however.  (I also
have not read the TLS v1.3 standard which is about to become
reality.)  The thing however is that for DKIM a lonesome user
cannot do anything -- you either need to have your own SMTP
server, or you need to trust your provider.  But our own user
interface is completely detached.  (I mean, at least if no MTA is
used one could do the DKIM stuff, too.)

 |FWIW, SPF presents a similar problem with message forwarding without
 |address rewriting... so it's definitely not just DKIM.
 --End of <20180624100438.GY10129@h-174-65.A328.priv.bahnhof.se>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 2820 bytes --]

On 06/25/2018 10:10 AM, Steffen Nurpmeso wrote:
> DKIM reuses the *SSL key infrastructure, which is good.

Are you saying that DKIM relies on the traditional PKI via CA 
infrastructure?  Or are you saying that it uses similar technology that 
is completely independent of the PKI / CA infrastructure?

> (Many eyes see the code in question.)  It places records in DNS, which 
> is also good, now that we have DNS over TCP/TLS and (likely) DTLS. 
> In practice however things may differ and to me DNS security is all in 
> all not given as long as we get to the transport layer security.

I believe that a secure DNS /transport/ is not sufficient.  Simply 
security the communications channel does not mean that the entity on the 
other end is not lying.  Particularly when not talking to the 
authoritative server, likely by relying on caching recursive resolvers.

> I personally do not like DKIM still, i have opendkim around and 
> thought about it, but i do not use it, i would rather wish that public 
> TLS certificates could also be used in the same way as public S/MIME 
> certificates or OpenPGP public keys work, then only by going to a TLS 
> endpoint securely once, there could be end-to-end security.

All S/MIME interactions that I've seen do use certificates from a well 
know CA via the PKI.

(My understanding of) what you're describing is encryption of data in 
flight.  That does nothing to encrypt / protect data at rest.

S/MIME /does/ provide encryption / authentication of data in flight 
/and/ data at rest.

S/MIME and PGP can also be used for things that never cross the wire.

> I am not a cryptographer, however.  (I also have not read the TLS v1.3 
> standard which is about to become reality.)  The thing however is that 
> for DKIM a lonesome user cannot do anything -- you either need to have 
> your own SMTP server, or you need to trust your provider.

I don't think that's completely accurate.  DKIM is a method of signing 
(via cryptographic hash) headers as you see (send) them.  I see no 
reason why a client can't add DKIM headers / signature to messages it 
sends to the MSA.

Granted, I've never seen this done.  But I don't see anything preventing 
it from being the case.

> But our own user interface is completely detached.  (I mean, at least 
> if no MTA is used one could do the DKIM stuff, too.)

I know that it is possible to do things on the receiving side.  I've got 
the DKIM Verifier add-on installed in Thunderbird, which gives me client 
side UI indication if the message that's being displayed still passes 
DKIM validation or not.  The plugin actually calculates the DKIM hash 
and compares it locally.  It's not just relying on a header added by 
receiving servers.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

[TUHS] mail (Re: off-topic list

[Bakul Shah]

On Jun 23, 2018, at 5:18 PM, Grant Taylor via TUHS <tuhs@minnie.tuhs.org> wrote:
> 
>> I actually have no idea of nmh, but i for one think the sentence of the old BSD Mail manual stating it is an "intelligent mail processing system, which has a command syntax reminiscent of ed(1) with lines replaced by messages" has always been a bit excessive.  And the fork i maintain additionally said it "is also usable as a mail batch language, both for sending and receiving mail", adding onto that.
> 
> What little I know about the MH type mail stores and associated utilities are indeed quite powerful.  I think they operate under the premise that each message is it's own file and that you work in something akin to a shell if not your actual OS shell.  I think the MH commands are quite literally unix command that can be called from the unix shell.  I think this is in the spirit of simply enhancing the shell to seem as if it has email abilities via the MH commands.  Use any traditional unix text processing utilities you want to manipulate email.
> 
> MH has always been attractive to me, but I've never used it myself.

One of the reasons I continue using MH (now nmh) as my primary
email system is because it works so well with Unixy tools. I
can e.g. do

    pick +TUHS -after '31 Dec 2017' -and -from bakul | xargs scan

To see summary lines of messages I posted to this group this
year. Using MH tools with non-MH tools is a bit clunky though
useful; I just have to cd to the right Mail directory first.

A mail message is really a structured object and if we 
represent it as a directory, all the standard unix tools can
be used. This is what Presotto's upasfs on plan9 does. With it
you can do things like

    cd /mail/fs/mbox
    grep -l foo */from | sed 's/from/to/' | xargs grep -l bar

But you soon realize standard unix tools are quite a bit
clunky in dealing with structured objects - often you want to
deal with a set of sub components simultaneously (e.g. the 
pick command above). This problem is actually pretty common in
the unix world and you have to have context/target specific 
tools. At the same time a unix sensibility can guide the
architecture / design. Something like MH toolset would be
easier to implement if you assume an underlying mbox
filesystem.  Conversely, the same sort of tools can perhaps be
useful elsewhere.

\vague-idea{
Seems to me that we have taken the unix (& plan9) model for
granted and not really explored it at this level much. That
is, can we map structured objects into trees/graphs in an
efficient way such that standard tools can be used on them?
Can we extend standard tools to explore substructures in a
more generic way?
}

Re: [TUHS] mail (Re: off-topic list

[Lyndon Nerenberg]

> On Jun 24, 2018, at 5:43 PM, Bakul Shah <bakul@bitblocks.com> wrote:
> 
> \vague-idea{
> Seems to me that we have taken the unix (& plan9) model for
> granted and not really explored it at this level much. That
> is, can we map structured objects into trees/graphs in an
> efficient way such that standard tools can be used on them?
> Can we extend standard tools to explore substructures in a
> more generic way?
> }

One of the main reasons I bailed out of nmh development was the disinterest in trying new and offside ways of addressing the mail store model.  I have been an MH user since the mid-80s, and I still think it is more functional than any other MUA in use today (including Alpine).  Header voyeurs will note I'm sending this from Mail.app.  As was mentioned earlier in this conversation, there is no one MUA, just as there is no one text editing tool.  I used a half dozen MUAs on a daily basis, depending on my needs.

But email, as a structured subset of text, deserves its own set of dedicated tools.  formail and procmail are immediate examples.  As MIME intruded on the traditional ASCII-only UNIX mbox format, grep and friends became less helpful.  HTML just made it worse.

Plan9's upas/fs abstraction of the underlying mailbox helps a lot.  By undoing all the MIME encoding, and translating the character sets to UTF8, grep and friends work again.  That filesystem abstraction has so much more potential, too.  E.g. pushing a pgpfs layer in there to transparently decode PGP content.

I really wish there was a way to bind Plan9-style file servers into the UNIX filename space.  Matt Blaze's CFS is the closest example of this I have seen.  Yet even it needs superuser privs.  (Yes, there's FUSE, but it just re-invents the NFS interface with no added benefit.)

Meanwhile, I have been hacking on a version of nmh that uses a native UTF8 message store.  I.e. it undoes all the MIME encoding, and undoes non-UTF8 charsets.  So grep and friends work, once again.

--lyndon

Re: [TUHS] mail (Re: off-topic list

[George Michaelson]

your email Lyndon, was uncannily like what I wanted to say. I think.

a) (n)MH is great
b) I can't live in one mail UI right now for a number of reasons.
Thats unfortunate
3) integration of MH into pop/imap was abortive and requires effort.
If thats improved, I'd love to know
4) we stopped advancing the art (tm) for handling data via pipes and
grep-like workflows

the plan9 plumber is amazing. I remain in awe of the work done to get
there, but its observable it isn't well applied anywhere else in the
ecology, that I can see. Which is odd, and .a shame given Pike works
in Google now. I guess people move into different roles (I know I
have)

If you put your version of NMH up anywhere I'd love to see it.

the one thing I do know, is that I have far more email accounts than I
should do (three, with two subvariants) and far more UI than I want
(four)

Re: [TUHS] mail (Re: off-topic list

[Larry McVoy]

On Mon, Jun 25, 2018 at 12:44:15PM +1000, George Michaelson wrote:
> your email Lyndon, was uncannily like what I wanted to say. I think.
> 
> a) (n)MH is great
> b) I can't live in one mail UI right now for a number of reasons.
> Thats unfortunate
> 3) integration of MH into pop/imap was abortive and requires effort.
> If thats improved, I'd love to know

I use rackspace for mcvoy.com but plain old mutt for reading it here
and sending.  I use fetchmail to get the mail locally.  Works for me
because mcvoy.com used to be a mail server and is still set up from
those times to send mail.  Kind of hacky but rackspace does the spam
filtering and I got sick of babysitting that.

> 4) we stopped advancing the art (tm) for handling data via pipes and
> grep-like workflows

So the source management system I started, BitKeeper, has sort of an
answer for the processing question.  It's a stretch but if you look 
at it enough a mail message is a little like a commit, they both have
fields.

Below is an example of the little "language" we built for processing
deltas in a revision history graph.  Some notes on how it works:

	:SOMETHING: means take struct delta->SOMETHING and replace the
	:SOMETHING: with that value.

	Control statements begin with $, so $if (expr).
	From awk we get $begin and $end (this whole language is very awk
	like with what awk would consider a line we hand the language
	a struct delta.)

	We invented a $each(:MULTI_LINE_FILE:) {
		:MULTI_LINE_FILE:
	}
	that is an interator, the variable in the body evaluates to 
	the next line of the multi line thing.  Weird but it works.

	$json(:FIELD:) json encodes the field.

	"text" is just printed.

	We gave you 10 registers/variables in $0 .. $9, they default to
	false.

This little script is running through each commit and printing out the
information in json.  Examples after the script.

It's important to understand that the $begin/end are run before/after
the deltas and then the main part of the script is run once per delta,
just like awk.

# dspec-v2
# The dspec used by 'bk changes -json'

$begin {
	"[\n"
}

$if (:CHANGESET: && !:COMPONENT_V:) {
	$if($0 -eq 1) {
		"\},\n"
	}
	"\{\n"
	"  \"key\": \":MD5KEY:\",\n"
	"  \"user\": \":USER:\",\n"
	"  \"host\": \":HOST:\",\n"
	"  \"date\": \":Dy:-:Dm:-:Dd:T:T::TZ:\",\n"
	"  \"serial\": :DS:,\n"
	"  \"comments\": \"" $each(:C:){$json{(:C:)}\\n} "\",\n"
        $if (:TAGS:) {
             "  \"tags\": [ "
             $each(:TAGS:){:JOIN:"\""(:TAGS:)"\""}
             " ],\n"
        }
        "  \"parents\": [ "
            $if(:PARENT:){"\"" :MD5KEY|PARENT: "\""}
            $if(:PARENT: && :MPARENT:){," "}
            $if(:MPARENT:){"\"" :MD5KEY|MPARENT: "\""}
            " ]\n"
	${0=1}		 		# we need to close off the changeset
}

$end {
	$if($0 -eq 1) {
		"\}\n"
	}
	"]\n"
}

So here is human readable output:

$ bk changes -1
ChangeSet@1.2926, 2018-03-12 08:00:33-04:00, rob@bugs.(none)
  L: Fix bug where "defaultx:" would be scanned as a T_DEFAULT
  followed by a T_COLON. The "x" and anything else after
  "default" and before the colon would be ignored.
  
  So if you ever had an option name that began with "default",
  it wouldn't be scanned correctly.
  
  This bug was reported by user GNX on the BitKeeper user's forum
  (Little language area).

And here is the same thing run through that scrip above.

$ bk changes -1 --json
[
{
  "key": "5aa66be1MaS_1t5lQkNCflPexCwd2w",
  "user": "rob",
  "host": "bugs.(none)",
  "date": "2018-03-12T08:00:33-04:00",
  "serial": 11178,
  "comments": "L: Fix bug where \"defaultx:\" would be scanned as a T_DEFAULT\nf
ollowed by a T_COLON. The \"x\" and anything else after\n\"default\" and before 
the colon would be ignored.\n\nSo if you ever had an option name that began with
 \"default\",\nit wouldn't be scanned correctly.\n\nThis bug was reported by use
r GNX on the BitKeeper user's forum\n(Little language area).\n",
  "parents": [ "5a2d8748bf8TYIOquTa3CZInTjC7KQ" ]
}
]


If you have read this far, maybe some ideas from this stuff could be used
to get a sort of processing system for structured data.  You'd need a plugin
for each structure but the harness itself could be reused.

Re: [TUHS] mail (Re: off-topic list

[Bakul Shah]

On Jun 24, 2018, at 6:15 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:
> 
> But email, as a structured subset of text, deserves its own set of dedicated tools.  formail and procmail are immediate examples.  As MIME intruded on the traditional ASCII-only UNIX mbox format, grep and friends became less helpful.  HTML just made it worse.

Storing messages in mbox format files is like keeping
tarfiles around. You wouldn't (usually) run grep on a
tarfile - you'd unpack it and then select files based
on extensions etc before running grep. In the same way
running grep on what should be a transport only mail
format doesn't make much sense.

> Meanwhile, I have been hacking on a version of nmh that uses a native UTF8 message store.  I.e. it undoes all the MIME encoding, and undoes non-UTF8 charsets.  So grep and friends work, once again.

Note that Mail.app already does this (more or less).

I have a paper design for (n)mh<->imap connection &
associated command changes. I didn't want to touch nmh
code so I have been doing some prototyping in Go but
any project progress is in fits and starts....

Re: [TUHS] mail (Re: off-topic list

[Steffen Nurpmeso]

Lyndon Nerenberg wrote in <42849F79-D132-4059-8A94-FFF8B141C49E@orthanc.ca>:
 |> On Jun 24, 2018, at 5:43 PM, Bakul Shah <bakul@bitblocks.com> wrote:
 |> 
 |> \vague-idea{
 |> Seems to me that we have taken the unix (& plan9) model for
 |> granted and not really explored it at this level much. That
 |> is, can we map structured objects into trees/graphs in an
 |> efficient way such that standard tools can be used on them?
 |> Can we extend standard tools to explore substructures in a
 |> more generic way?
 |>}
 |
 |One of the main reasons I bailed out of nmh development was the disinter\
 |est in trying new and offside ways of addressing the mail store model. \
 | I have been an MH user since the mid-80s, and I still think it is \
 |more functional than any other MUA in use today (including Alpine). \
 | Header voyeurs will note I'm sending this from Mail.app.  As was mentio\
 |ned earlier in this conversation, there is no one MUA, just as there \
 |is no one text editing tool.  I used a half dozen MUAs on a daily basis, \
 |depending on my needs.
 |
 |But email, as a structured subset of text, deserves its own set of \
 |dedicated tools.  formail and procmail are immediate examples.  As \
 |MIME intruded on the traditional ASCII-only UNIX mbox format, grep \
 |and friends became less helpful.  HTML just made it worse.
 |
 |Plan9's upas/fs abstraction of the underlying mailbox helps a lot. \
 | By undoing all the MIME encoding, and translating the character sets \
 |to UTF8, grep and friends work again.  That filesystem abstraction \
 |has so much more potential, too.  E.g. pushing a pgpfs layer in there \
 |to transparently decode PGP content.
 |
 |I really wish there was a way to bind Plan9-style file servers into \
 |the UNIX filename space.  Matt Blaze's CFS is the closest example of \
 |this I have seen.  Yet even it needs superuser privs.  (Yes, there's \
 |FUSE, but it just re-invents the NFS interface with no added benefit.)
 |
 |Meanwhile, I have been hacking on a version of nmh that uses a native \
 |UTF8 message store.  I.e. it undoes all the MIME encoding, and undoes \
 |non-UTF8 charsets.  So grep and friends work, once again.

The SMTPUTF8 standard can this, and nmh now too.  There was
a conference presentation of i think a Microsoft employee that was
linked on one of the lists i track.  He also said this, "finally
being able to just look at the mail as it is (again)".  He showed
one in an editor too.  But the presentation could be seen as the
whale of a lie, as propaganda, there was no traceroute, no
signatures, no attachments.  Cool and so, but not very realistic.

I do not want to look at the plain source of a mail message in
almost all cases.  It is otherwise mostly for development.
Therefore i am having trouble to understand all this, i mean,
i need a PDFviewer to look into a PDF, [..etc...], just the same
is true for email.  Maybe because i like MBOX a lot, it is just
a database, a single file, that i can easily take and move.
I have never understood why people get excited about Maildir, you
have trees full of files with names which hurt the eyes, and they
miss a From_ line (and are thus not valid MBOX files, i did not
miss that in the other mail).
Despite that, having a possibility to look easily i miss much.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] mail (Re: off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 1950 bytes --]

On 06/25/2018 10:26 AM, Steffen Nurpmeso wrote:
> I have never understood why people get excited about Maildir, you have 
> trees full of files with names which hurt the eyes,

First, a single Maildir is a parent directory and three sub-directories. 
  Many Maildir based message stores are collections of multiple 
individual Maildirs.

Second, the names may look ugly, but they are named independently of the 
contents of the message.

Third, I prefer the file per message as opposed to monolithic mbox for 
performance reasons.  Thus I message corruption impacts a single message 
and not the entire mail folder (mbox).

Aside:  I already have a good fast database that most people call a file 
system and it does a good job tracking metadata for me.

Fourth, I found maildir to be faster on older / slower servers because 
it doesn't require copying (backing up) the monolithic mbox file prior 
to performing an operation on it.  It splits reads & writes into much 
smaller chunks that are more friendly (and faster) to the I/O sub-system.

Could many of the same things be said about MH?  Very likely.  What I 
couldn't say about MH at the time I went looking and comparing (typical) 
unix mail stores was the readily available POP3 and IMAP interface. 
Seeing as how POP3 and IMAP were a hard requirement for me, MH was a 
non-starter.

> they miss a From_ line (and are thus not valid MBOX files, i did not 
> miss that in the other mail).

True.  Though I've not found that to be a limitation.  I'm fairly 
confident that the Return-Path: header that is added / replaced by my 
MTA does functionally the same thing.

I'm sure similar differences can be had between many different solutions 
in Unix's history.  SYS V style init.d scripts vs BSD style /etc/rc\d 
style init scripts vs systemd or Solaris's SMD (I think that's it's name).

To each his / her own.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 2742 bytes --]

On Sat, Jun 23, 2018 at 8:18 PM, Grant Taylor via TUHS <tuhs@minnie.tuhs.org
> wrote:
>
>
> What little I know about the MH type mail stores and associated utilities
> are indeed quite powerful.

​Yep, their power and flaw all rolled together actually.     Until I had
Pachyderm on Tru64/Alpha with AltaVista under the covers (which was gmail's
predecessor), I ran a flavor of MH from the time Bruce (Borden - MH's
author) first released it on the 6th edition on the Rand USENIX tape. ​
I'm going to guess for about 25 years.   Although for the last 8-10 years,
I ran a post processor user interface called 'HM' (also from Rand) that was
curses based that split the screen into two.




>   I think they operate under the premise that each message is it's own
> file

​Correct - which is great, other than on small systems it chews up inodes
and disk space which for v6 and v7 could be a problem.   ​But it means
everything was always ASCII and easy to grok and any tool from an editor to
macro processor could be inserted.   It also meant that unlike AT&T "mail",
the division between the MUA and the MTA was first declared by the Rand and
understood in Unix and used in the original UofI ArpaNet code (before
Kurt's delivermail  [sendmail's predecessor] which was part of UCB Mail, or
the MIT mailer ArpaNet hacks that would come later).

BTW: I may have the the original Rand MH release somewhere.  We ran it at
Tektronix on V6 on the 11/60 and then V7 on the TekLabs 11/70, as I brought
it with me.  We hacked the MTA portion to talk smtpd under Bruce's UNET
code to our VMS/SMTPD at some point.



> and that you work in something akin to a shell if not your actual OS shell.

​Exactly.   ​Your shell or emacs if you so desired - whatever your native
system interface was.  HM took the idea a little further to make things
more screen oriented and later versions of MH picked some of the HM stuff
I'm told; but I had started to use Pachyderm - which was search based.



>   I think the MH commands are quite literally unix command that can be
> called from the unix shell.  I think this is in the spirit of simply
> enhancing the shell to seem as if it has email abilities via the MH
> commands.  Use any traditional unix text processing utilities you want to
> manipulate email.

​Absolutely.  I do find myself, pulling things out of gmail, sometimes so I
can do Unix tricks to inbound mail that gmail will not let me do.   And
when I want to do anything really automating on the send side, I have MH
installed and it calls the local MTA.   But I admit, the indexing that
search gives you is incredibly powerful for day to day use and I could not
go back. to MH.

[-- Attachment #2: Type: text/html, Size: 4475 bytes --]

Re: [TUHS] off-topic list [ really mh ]

[Jon Steinhart]

I'm a big fan of mh (nmh now) and a sometimes maintainer.
What I love about it is that it's a separate set of commands as
opposed to an integrated blob.  That means that I can script.

I also love the one-file-per-message thing, although it's somewhat
less useful with mime; can't just grep anymore.

One of the features that added to nmh to allow it to invoke external
programs when doing its own message processing.  I use this to maintain
a parallel ElasticSearch database of my mail; just a simple word to
message folder/file thing.  I actually go through the trouble of
decoding many mime times when building this database.  This gives me
the ability to very quickly locate messages based on their contents.

Re: [TUHS] off-topic list [ really mh ]

[Ralph Corderoy]

Hi Jon,

> I'm a big fan of mh (nmh now) and a sometimes maintainer.

Ditto × 2.  :-)  https://www.nongnu.org/nmh/

> What I love about it is that it's a separate set of commands as
> opposed to an integrated blob.  That means that I can script.

Yes, this is key.  I don't procmail(1) my emails into folders for later
reading by theme, instead I have a shell script that runs through the
inbox looking for emails to process.

pick(1) finds things of interest, e.g.
    pick --list-id '<tuhs\.minnie\.tuhs\.org>'
I then display those emails in a variety of ways:
    one-line summary of each;
    scrape and summarise signal from third-party noise with sed(1),
        etc., having decoded the MIME; 
    read each in full in less(1) having configured `K', `S', ... to
        update nmh's sequences called `keep', `spam', ..., and move onto
        the next email.
And finally have read(1) prompt me for an action with a common default,
delete, refile, etc.
Then the script does it all again for the pick from the inbox.

The result is I'm lead through all the routine processing, mainly
hitting Enter with the odd bit of `DDDKSD'.  A bit similiar to using the
rn(1) Usenet reader's `do the next thing' `Space' key.

In interactive use, old-MH hands don't type `pick -subject ...', but
instead have a personal ~/bin/* that suits their needs.  For me, `s'
shows an email, `sc' gives a scan(1) listing, one per line, of the end
of the current folder;  just enough based on `tput lines'.  I search
with ~/bin/-sub, and so on.

Much more flexible than a silo like mail(1) by benefiting from the
`programming tools' model.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Grant Taylor via TUHS wrote in <09ee8833-c8c0-8911-751c-906b737209b7@spa\
mtrap.tnetconsulting.net>:
 |On 06/23/2018 04:38 PM, Steffen Nurpmeso wrote:
 |> Absolutely true.  And hoping that, different to web browsers, no let me 
 |> call it pseudo feature race is started that results in less diversity 
 |> instead of anything else.
 |
 |I'm not sure I follow.  I feel like we do have the choice of MUAs or web 
 |browsers.  Sadly some choices are lacking compared to other choices. 
 |IMHO the maturity of some choices and lack there of in other choices 
 |does not mean that we don't have choices.

Interesting.  I do not see a real choice for me.  Netsurf may be
one but cannot do Javascript once i looked last, so this will not
work out for many, and increasing.  Just an hour ago i had a fight
with Chromium on my "secure box" i use for the stuff which knows
about bank accounts and passwords in general, and (beside the fact
it crashes when trying to prepare PDF printouts) it is just
unusable slow.  I fail to understand (well actually i fail to
understand a lot of things but) why you need a GPU process for
a 2-D HTML page.  And then this:

  libGL error: unable to load driver: swrast_dri.so
  libGL error: failed to load driver: swrast
  [5238:5251:0625/143535.184020:ERROR:bus.cc(394)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
  [5238:5279:0625/143536.401172:ERROR:bus.cc(394)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
  libGL error: unable to load driver: swrast_dri.so
  libGL error: failed to load driver: swrast
  [5293:5293:0625/143541.299106:ERROR:gl_implementation.cc(292)] Failed to load /usr/lib/chromium/swiftshader/libGLESv2.so: Error loading shared library /usr/lib/chromium/swiftshader/libGLESv2.so: No such file or directory
  [5293:5293:0625/143541.404733:ERROR:viz_main_impl.cc(196)] Exiting GPU process due to errors during initialization
[..repeating..]
  [5238:5269:0625/143544.747819:ERROR:browser_gpu_channel_host_factory.cc(121)] Failed to launch GPU process.
  [5238:5238:0625/143544.749814:ERROR:gpu_process_transport_factory.cc(1009)] Lost UI shared context.

I mean, it is not there, ok?
I cannot really imagine how hard it is to write a modern web
browser, with the highly integrative DOM tree, CSS and Javascript
and such, however, and the devil is in the details anyway.
I realize that Chromium does not seem to offer options for Cookies
and such - i have a different elder browser for that.
All of that is off-topic anyway, but i do not know why you say
options.

 |> If there is the freedom for a decision.  That is how it goes, yes.  For 
 ..
 |> But it is also nice to see that there are standards which were not fully 
 |> thought through and required backward incompatible hacks to fill \
 |> the gaps.
 |
 |Why is that a "nice" thing?

Good question.  Then again, young men (and women) need to have
a chance to do anything at all.  Practically speaking.  For
example we see almost thousands of new RFCs per year.  That is
more than in the first about two decades all in all.  And all is
getting better.

 |> Others like DNS are pretty perfect and scale fantastic.  Thus.
 |
 |Yet I frequently see DNS problems for various reasons.  Not the least of 
 |which is that many clients do not gracefully fall back to the secondary 
 |DNS server when the primary is unavailable.

Really?  Not that i know of.  Resolvers should be capable to
provide quality of service, if multiple name servers are known,
i would say.  This is even RFC 1034 as i see, SLIST and SBELT,
whereas the latter i filled in from "nameserver" as of
/etc/resolv.conf, it should have multiple, then.  (Or point to
localhost and have a true local resolver or something like
dnsmasq.)

I do see DNS failures via Wifi but that is not the fault of DNS,
but of the provider i use.

P.S.: actually the only three things i have ever hated about DNS,
and i came to that in 2004 with EDNS etc. all around yet, is that
backward compatibly has been chosen for domain names, and
therefore we gained IDNA, which is a terribly complicated
brainfuck thing that actually caused incompatibilities, but these
where then waved through and ok.  That is absurd.

(If UTF-8 is too lengthy, i would have used UTF-7 for this, the
average octet/character ratio is still enough for most things on
the internet.  An EDNS bit could have been used for extended
domain name/label lengths, and if that would have been done 25
years ago we were fine out in practice.  Until then registrars and
administrators had to decide whether they want to use extended
names or not.  And labels of 25 characters are not common even
today, nowhere i ever looked.)

And the third is DNSSEC, which i read the standard of and said
"no".  Just last year or the year before that we finally got DNS
via TCP/TLS and DNS via DTLS, that is, normal transport security!
Twenty years too late, but i really had good days when i saw those
standards flying by!  Now all we need are zone administrators
which publish certificates via DNS and DTLS and TCP/TLS consumers
which can integrate those in their own local pool (for at least
their runtime).

 |> Ah, it has become a real pain.  It is ever so astounding how much HTML5 
 |> specifics, scripting and third party analytics can be involved for \
 |> a page 
 |> that would have looked better twenty years ago, or say whenever CSS 2.0 
 |> came around.
 |
 |I'm going to disagree with you there.  IMHO the standard is completely 
 |separate with what people do with it.

They have used <object> back in the 90s, an equivalent should have
made it into the standard back then.  I mean this direction was
clear, right?, maybe then the right people would have more
influence still.  Not that it would matter, all those many people
from the industry will always sail sair own regatta and use what
is hip.

 |> Today almost each and every commercial site i visit is in practice 
 |> a denial of service attack.  For me and my little box at least, and 
 |> without gaining any benefit from that!
 |
 |I believe those webmasters have made MANY TERRIBLE choices and have 
 |ended up with the bloat that they now have.  -  I do not blame the 
 |technology.  I blame what the webmasters have done with the technology.
 |
 |Too many people do not know what they are doing and load "yet another 
 |module" to do something that they can already do with what's already 
 |loaded on their page.  But they don't know that.  They just glue things 
 |togehter until they work.  Even if that means that they are loading the 
 |same thing multiple times because multiple of their 3rd party components 
 |loads a common module themselves.  This is particularly pernicious with 
 |JavaScript.

Absolutely.  And i am watching car industry as they present
themselves in Germany pretty closely, and they are all a nuisance,
in how they seem to track each other and step onto their
footsteps.  We had those all-script-based slide effects, and they
all need high definition images and need to load them all at once,
non-progressively.  It becomes absurd if you need to download 45
megabytes of data, and >43 of those are an image of a car in
nature with a very clean model wearing an axe.  This is "cool"
only if you have a modern environment and the data locally
available.  Im my humble opinion.

  ...
 |> I actually have no idea of nmh, but i for one think the sentence of 
 |> the old BSD Mail manual stating it is an "intelligent mail processing 
 |> system, which has a command syntax reminiscent of ed(1) with lines 
 |> replaced by messages" has always been a bit excessive.  And the fork i 
 |> maintain additionally said it "is also usable as a mail batch language, 
 |> both for sending and receiving mail", adding onto that.
 |
 |What little I know about the MH type mail stores and associated 
 |utilities are indeed quite powerful.  I think they operate under the 
 |premise that each message is it's own file and that you work in 
 |something akin to a shell if not your actual OS shell.  I think the MH 
 |commands are quite literally unix command that can be called from the 
 |unix shell.  I think this is in the spirit of simply enhancing the shell 
 |to seem as if it has email abilities via the MH commands.  Use any 
 |traditional unix text processing utilities you want to manipulate email.
 |
 |MH has always been attractive to me, but I've never used it myself.

I actually hate the concept very, very much ^_^, for me it has
similarities with brainfuck.  I could not use it.

 |> You say it.  In the research Unix nupas mail (as i know it from Plan9) 
 |> all those things could have been done with a shell script and standard 
 |> tools like grep(1) and such.
 |
 |I do say that and I do use grep, sed, awk, formail, procmail, cp, mv, 
 |and any number of traditional unix file / text manipulation utilities on 
 |my email weekly.  I do this both with the Maildir (which is quite 
 |similar to MH) on my mail server and to the Thumderbird message store 
 |that is itself a variant of Maildir with a file per message in a 
 |standard directory structure.
 |
 |> Even Thunderbird would simply be a maybe even little nice graphical 
 |> application for display purposes.
 |
 |The way that I use Thunderbird, that's exactly what it is.  A friendly 
 |and convenient GUI front end to access my email.
 |
 |> The actual understanding of storage format and email standards would 
 |> lie solely within the provider of the file system.
 |
 |The emails are stored in discrete files that themselves are effectively 
 |mbox files with one message therein.
 |
 |> Now you use several programs which all ship with all the knowledge.
 |
 |I suppose if you count greping for a line in a text file as knowledge of 
 |the format, okay.
 |
 |egrep "^Subject: " message.txt
 |
 |There's nothing special about that.  It's a text file with a line that 
 |looks like this:
 |
 |Subject: Re: [TUHS] off-topic list

Except that this will work only for all-english, as otherwise
character sets come into play, text may be in a different
character set, mail standards may impose a content-transfer
encoding, and then what you are looking at is actually a database,
not the data as such.

This is what i find so impressive about that Plan9 approach, where
the individual subparts of the message are available as files in
the filesystem, subjects etc. as such, decoded and available as
normal files.  I think this really is .. impressive.

 |> I for one just want my thing to be easy and reliable controllable via 
 |> a shell script.
 |
 |That's a laudable goal.  I think MH is very conducive to doing that.

Maybe.

 |> You could replace procmail (which is i think perl and needs quite some 
 |> perl modules) with a monolithic possibly statically linked C program.
 |
 |I'm about 95% certain that procmail is it's own monolithic C program. 
 |I've never heard of any reference to Perl in association with procmail. 
 |Are you perhaps thinking of a different local delivery agent?

Oh really?  Then likely so, yes.  I have never used this.

 |> Then.  With full error checking etc.  This is a long road ahead, for 
 |> my thing.
 |
 |Good luck to you.

Thanks, eh, thanks.  Time will bring.. or not.

 |> So ok, it does not, actually.  It chooses your "Grant Taylor via TUHS" 
 |> which ships with the TUHS address, so one may even see this as an 
 |> improvement to DMARC-less list replies, which would go to TUHS, with or 
 |> without the "The Unix Heritage Society".
 |
 |Please understand, that's not how I send the emails.  I send them with 
 |my name and my email address.  The TUHS mailing list modifies them.
 |
 |Aside:  I support the modification that it is making.

It is of course not the email that leaves you no more.  It is not
just headers are added to bring the traceroute path.  I do have
a bad feeling with these, but technically i do not seem to have an
opinion.

 |> I am maybe irritated by the 'dkim=fail reason="signature verification 
 |> failed"' your messages produce.  It would not be good to filter out 
 |> failing DKIMs, at least on TUHS.
 |
 |Okay.  That is /an/ issue.  But I believe it's not /my/ issue to solve.
 |
 |My server DKIM signs messages that it sends out.  From everything that 
 |I've seen and tested (and I actively look for problems) the DKIM 
 |signatures are valid and perfectly fine.
 |
 |That being said, the TUHS mailing list modifies message in the following 
 |ways:
 |
 |1)  Modifies the From: when the sending domain uses DMARC.
 |2)  Modifies the Subject to prepend "[TUHS] ".
 |3)  Modifies the body to append a footer.
 |
 |All three of these actions modify the data that receiving DKIM filters 
 |calculate hashes based on.  Since the data changed, obviously the hash 
 |will be different.
 |
 |I do not fault THUS for this.
 |
 |But I do wish that TUHS stripped DKIM and associated headers of messages 
 |going into the mailing list.  By doing that, there would be no data to 
 |compare to that wouldn't match.
 |
 |I think it would be even better if TUHS would DKIM sign messages as they 
 |leave the mailing list's mail server.

Well, this adds the burden onto TUHS.  Just like i have said.. but
you know, more and more SMTP servers connect directly via STARTSSL
or TCP/TLS right away.  TUHS postfix server does not seem to do so
on the sending side -- you know, i am not an administor, no
earlier but on the 15th of March this year i realized that my
Postfix did not reiterate all the smtpd_* variables as smtp_*
ones, resulting in my outgoing client connections to have an
entirely different configuration than what i provided for what
i thought is "the server", then i did, among others

  smtpd_tls_security_level = may
 +smtp_tls_security_level = $smtpd_tls_security_level

But if TUHS did, why should it create a DKIM signature?
Ongoing is the effort to ensure SMTP uses TLS all along the route,
i seem to recall i have seen RFCs pass by which accomplish that.
Or only drafts??  Hmmm.

  ...
 |I don't know if PGP or S/MIME will ever mandate anything about headers 
 |which are structurally outside of their domain.
 |
 |I would like to see an option in MUAs that support encrypted email for 
 |something like the following:
 |
 |    Subject:  (Subject in encrypted body.)
 |
 |Where the encrypted body included a header like the following:
 |
 |    Encrypted-Subject: Re: [TUHS] off-topic list
 |
 |I think that MUAs could then display the subject that was decrypted out 
 |of the encrypted body.

Well S/MIME does indeed specify this mode of encapsulating the
entire message including the headers, and enforce MUAs to
completely ignore the outer envelope in this case.  (With a RFC
discussing problems of this approach.)  The BSD Mail clone
i maintain does not support this yet, along with other important
aspects of S/MIME, like the possibility to "self-encrypt" (so that
the message can be read again, a.k.a. that the encrypted version
lands on disk in a record, not the plaintext one).  I hope it will
be part of the OpenPGP, actually privacy rewrite this summer.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 8873 bytes --]

On 06/25/2018 09:51 AM, Steffen Nurpmeso wrote:
> I cannot really imagine how hard it is to write a modern web browser, 
> with the highly integrative DOM tree, CSS and Javascript and such, 
> however, and the devil is in the details anyway.

Apparently it's difficult.  Or people aren't sufficiently motivated.

I know that I personally can't do it.

> Good question.  Then again, young men (and women) need to have a chance 
> to do anything at all.  Practically speaking.  For example we see almost 
> thousands of new RFCs per year.  That is more than in the first about 
> two decades all in all.  And all is getting better.

Yep.  I used to have aspirations of reading RFCs.  But work and life 
happened, then the rate of RFC publishing exploded, and I gave up.

> Really?  Not that i know of.  Resolvers should be capable to provide 
> quality of service, if multiple name servers are known, i would say.

I typically see this from the authoritative server side.  I've 
experienced this (at least) once myself and have multiple colleagues 
that have also experienced this (at least) once themselves too.

If the primary name server is offline for some reason (maintenance, 
outage, hardware, what have you) clients start reporting a problem that 
manifests itself as a complete failure.  They seem to not fall back to 
the secondary name server.

I'm not talking about "slowdown" complaints.  I'm talking about 
"complete failure" and "inability to browse the web".

I have no idea why this is.  All the testing that I've done indicate 
that clients fall back to secondary name servers.  Yet multiple 
colleagues and myself have experienced this across multiple platforms.

It's because of these failure symptoms that I'm currently working with a 
friend & colleague to implement VRRP between his name servers so that 
either of them can serve traffic for both DNS service IPs / Virtual IPs 
(VIPs) in the event that the other server is unable to do so.

We all agree that a 5 ~ 90 second (depending on config) outage with 
automatic service continuation after VIP migration is a LOT better than 
end users experiencing what are effectively service outages of the 
primary DNS server.

Even in the outages, the number of queries to the secondary DNS 
server(s) don't increase like you would expect as clients migrate from 
the offline primary to the online secondary.

In short, this is a big unknown that I, and multiple colleagues, have 
seen and can't explain.  So, we have each independently decided to 
implement solutions to keep the primary DNS IP address online using what 
ever method we prefer.

> This is even RFC 1034 as i see, SLIST and SBELT, whereas the latter 
> i filled in from "nameserver" as of /etc/resolv.conf, it should have 
> multiple, then.  (Or point to localhost and have a true local resolver 
> or something like dnsmasq.)

I completely agree that per all documentation, things SHOULD work with 
just the secondary.  Yet my experience from recursive DNS server 
operator stand point, things don't work.

> I do see DNS failures via Wifi but that is not the fault of DNS, but of 
> the provider i use.

Sure.  That's the nature of a wireless network.  It's also unrelated to 
the symptoms I describe above.

> P.S.: actually the only three things i have ever hated about DNS, and 
> i came to that in 2004 with EDNS etc. all around yet, is that backward 
> compatibly has been chosen for domain names, and therefore we gained 
> IDNA, which is a terribly complicated brainfuck thing that actually 
> caused incompatibilities, but these where then waved through and ok. 
> That is absurd.

I try to avoid dealing with IDNs.  Fortunately I'm fairly lucky in doing so.

> And the third is DNSSEC, which i read the standard of and said "no". 
> Just last year or the year before that we finally got DNS via TCP/TLS 
> and DNS via DTLS, that is, normal transport security!

My understanding is that DNSSEC provides verifiable authenticity of the 
information transported by DNSSEC.  It's also my understanding that 
DTLS, DOH, DNScrypt, etc provide DNS /transport/ encryption 
(authentication & privacy).

As I understand it, there is nothing about DTLS, DOH, DNScrypt, etc that 
prevent servers on the far end of said transports from modifying queries 
/ responses that pass through them, or serving up spoofed content.

To me, DNSSEC serves a completely different need than DTLS, DOH, 
DNScrypt, etc.

Please correct me and enlighten me if I'm wrong or have oversimplified 
things.

> Twenty years too late, but i really had good days when i saw those 
> standards flying by!  Now all we need are zone administrators which 
> publish certificates via DNS and DTLS and TCP/TLS consumers which can 
> integrate those in their own local pool (for at least their runtime).

DANE has been waiting on that for a while.

I think DANE does require DNSSEC.  I think DKIM does too.

> I actually hate the concept very, very much ^_^, for me it has 
> similarities with brainfuck.  I could not use it.

Okay.  To each his / her own.

> Except that this will work only for all-english, as otherwise character 
> sets come into play, text may be in a different character set, mail 
> standards may impose a content-transfer encoding, and then what you are 
> looking at is actually a database, not the data as such.

Hum.  I've not considered non-English as I so rarely see it in raw email 
format.

> This is what i find so impressive about that Plan9 approach, where 
> the individual subparts of the message are available as files in the 
> filesystem, subjects etc. as such, decoded and available as normal files. 
> I think this really is .. impressive.

I've never had the pleasure of messing with Plan9.  It's on my 
proverbial To-Do-Someday list.  It does sound very interesting.

> Thanks, eh, thanks.  Time will bring.. or not.

;-)

> It is of course not the email that leaves you no more.  It is not just 
> headers are added to bring the traceroute path.  I do have a bad feeling 
> with these, but technically i do not seem to have an opinion.

I too have some unease with things while not having any technical 
objection to them.

> Well, this adds the burden onto TUHS.  Just like i have said.. but 
> you know, more and more SMTP servers connect directly via STARTSSL or 
> TCP/TLS right away.  TUHS postfix server does not seem to do so on the 
> sending side

The nature of email is (and has been) changing.

We're no longer trying to use 486s to manage mainstream email.  My 
opinion is that we have the CPU cycles to support current standards.

> -- you know, i am not an administor, no earlier but on the 15th of March 
> this year i realized that my Postfix did not reiterate all the smtpd_* 
> variables as smtp_* ones, resulting in my outgoing client connections 
> to have an entirely different configuration than what i provided for 
> what i thought is "the server", then i did, among others
> 
> smtpd_tls_security_level = may +smtp_tls_security_level = 
> $smtpd_tls_security_level

Oy vey.

> But if TUHS did, why should it create a DKIM signature?  Ongoing is the 
> effort to ensure SMTP uses TLS all along the route, i seem to recall i 
> have seen RFCs pass by which accomplish that.  Or only drafts??  Hmmm.

SMTP over TLS (STARTTLS) is just a transport.  I can send anything I 
want across said transport.

DKIM is about enabling downstream authentication in email.  Much like 
DNSSEC does for DNS.

There is nothing that prevents sending false information down a secure 
communications channel.

> Well S/MIME does indeed specify this mode of encapsulating the entire 
> message including the headers, and enforce MUAs to completely ignore the 
> outer envelope in this case.  (With a RFC discussing problems of this 
> approach.)

Hum.  That's contrary to my understanding.  Do you happen to have RFC 
and section numbers handy?  I've wanted to, needed to, go read more on 
S/MIME.  It now sounds like I'm missing something.

I wonder if the same can be said about PGP.

> The BSD Mail clone i maintain does not support this yet, along with other 
> important aspects of S/MIME, like the possibility to "self-encrypt" (so 
> that the message can be read again, a.k.a. that the encrypted version 
> lands on disk in a record, not the plaintext one).  I hope it will be 
> part of the OpenPGP, actually privacy rewrite this summer.

I thought that many MUAs handled that problem by adding the sender as an 
additional recipient in the S/MIME structure.  That way the sender could 
extract the ephemeral key using their private key and decrypt the 
encrypted message that they sent.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Steffen Nurpmeso]

Hello.  And sorry for the late reply, "lovely weather we're
having"..

Grant Taylor via TUHS wrote in <c4debab5-181d-c071-850a-a8865d6e98d0@spa\
mtrap.tnetconsulting.net>:
 |On 06/25/2018 09:51 AM, Steffen Nurpmeso wrote:
 ...
 |> I cannot really imagine how hard it is to write a modern web browser, 
 |> with the highly integrative DOM tree, CSS and Javascript and such, 
 |> however, and the devil is in the details anyway.
 |
 |Apparently it's difficult.  Or people aren't sufficiently motivated.
 |
 |I know that I personally can't do it.
 |
 |> Good question.  Then again, young men (and women) need to have a chance 
 |> to do anything at all.  Practically speaking.  For example we see almost 
 |> thousands of new RFCs per year.  That is more than in the first about 
 |> two decades all in all.  And all is getting better.
 |
 |Yep.  I used to have aspirations of reading RFCs.  But work and life 
 |happened, then the rate of RFC publishing exploded, and I gave up.

I think it is something comparable.

 |> Really?  Not that i know of.  Resolvers should be capable to provide 
 |> quality of service, if multiple name servers are known, i would say.
 |
 |I typically see this from the authoritative server side.  I've 
 |experienced this (at least) once myself and have multiple colleagues 
 |that have also experienced this (at least) once themselves too.
 |
 |If the primary name server is offline for some reason (maintenance, 
 |outage, hardware, what have you) clients start reporting a problem that 
 |manifests itself as a complete failure.  They seem to not fall back to 
 |the secondary name server.
 |
 |I'm not talking about "slowdown" complaints.  I'm talking about 
 |"complete failure" and "inability to browse the web".
 |
 |I have no idea why this is.  All the testing that I've done indicate 
 |that clients fall back to secondary name servers.  Yet multiple 
 |colleagues and myself have experienced this across multiple platforms.
 |
 |It's because of these failure symptoms that I'm currently working with a 
 |friend & colleague to implement VRRP between his name servers so that 
 |either of them can serve traffic for both DNS service IPs / Virtual IPs 
 |(VIPs) in the event that the other server is unable to do so.
 |
 |We all agree that a 5 ~ 90 second (depending on config) outage with 
 |automatic service continuation after VIP migration is a LOT better than 
 |end users experiencing what are effectively service outages of the 
 |primary DNS server.
 |
 |Even in the outages, the number of queries to the secondary DNS 
 |server(s) don't increase like you would expect as clients migrate from 
 |the offline primary to the online secondary.
 |
 |In short, this is a big unknown that I, and multiple colleagues, have 
 |seen and can't explain.  So, we have each independently decided to 
 |implement solutions to keep the primary DNS IP address online using what 
 |ever method we prefer.

That is beyond me, i have never dealt with the server side.  I had
a pretty normal QOS client: a sequentially used searchlist,
configurable DNS::conf_retries, creating SERVFAIL cache entries if
all bets were off.  Placing failing servers last in the SBELT, but
keeping them; i see a dangling TODO note for an additional place for
failing nameservers, to have them rest for a while.

 |> This is even RFC 1034 as i see, SLIST and SBELT, whereas the latter 
 |> i filled in from "nameserver" as of /etc/resolv.conf, it should have 
 |> multiple, then.  (Or point to localhost and have a true local resolver 
 |> or something like dnsmasq.)
 |
 |I completely agree that per all documentation, things SHOULD work with 
 |just the secondary.  Yet my experience from recursive DNS server 
 |operator stand point, things don't work.

That is bad.

 |> I do see DNS failures via Wifi but that is not the fault of DNS, but of 
 |> the provider i use.
 |
 |Sure.  That's the nature of a wireless network.  It's also unrelated to 
 |the symptoms I describe above.

Well, in fact i think there is something like a scheduler error
around there, too, because sometimes i am placed nowhere and get
no bits at all, most often at minute boundaries, and for minutes.
But well...

 |> P.S.: actually the only three things i have ever hated about DNS, and 
 |> i came to that in 2004 with EDNS etc. all around yet, is that backward 
 |> compatibly has been chosen for domain names, and therefore we gained 
 |> IDNA, which is a terribly complicated brainfuck thing that actually 
 |> caused incompatibilities, but these where then waved through and ok. 
 |> That is absurd.
 |
 |I try to avoid dealing with IDNs.  Fortunately I'm fairly lucky in \
 |doing so.

I censor this myself.  I for one would vote for UTF-7 with
a different ACE (ASCII compatible encoding) prefix, UTF-7 is
somewhat cheap to implement and used for, e.g., the pretty
omnipresent IMAP.  I mean, i (but who am i) have absolutely
nothing against super smart algorithms somewhere, for example in
a software that domain registrars have to run in order to grant
domain names.  But the domain name as such should just be it,
otherwise it is .. not.  Yes that is pretty Hollywood but if
i mistype an ASCII hostname i also get a false result.

Anyway: fact is that for example German IDNA 2008 rules are
incompatible with the earlier ones: really, really bizarre.

 |> And the third is DNSSEC, which i read the standard of and said "no". 
 |> Just last year or the year before that we finally got DNS via TCP/TLS 
 |> and DNS via DTLS, that is, normal transport security!
 |
 |My understanding is that DNSSEC provides verifiable authenticity of the 
 |information transported by DNSSEC.  It's also my understanding that 
 |DTLS, DOH, DNScrypt, etc provide DNS /transport/ encryption 
 |(authentication & privacy).
 |
 |As I understand it, there is nothing about DTLS, DOH, DNScrypt, etc that 
 |prevent servers on the far end of said transports from modifying queries 
 |/ responses that pass through them, or serving up spoofed content.
 |
 |To me, DNSSEC serves a completely different need than DTLS, DOH, 
 |DNScrypt, etc.
 |
 |Please correct me and enlighten me if I'm wrong or have oversimplified 
 |things.

No, a part is that you can place signatures which can be used to
verify the content that is actually delivered.  This is right.
And yes, this is different to transport layer security.  And DNS
is different in that data is cached anywhere in the distributed
topology, so having the data ship with verifieable signatures may
sound sound.  But when i look around this is not what is in use,
twenty years thereafter.  I mean you can lookup netbsd.org and see
how it works, and if you have a resolver who can make use that
would be ok (mine can not), but as long as those signatures are
not mandatory they can be left out somewhere on the way, can they.
My VM hoster offers two nameservers and explicitly does not
support DNSSEC for example (or did so once we made the contract,
about 30 months ago).

Then again i wish (i mean, it is not that bad in respect to this,
politics or philosophy are something different) i could reach out
securely to the mentioned servers.  In the end you need to put
trust in someone, i mean, most of you have smartphones, and some
processors run entire operating systems aside the running
operating system, and the one software producer who offers its
entire software open on a public hoster is condemned whereas
others who do not publish any source code are taken along to the
toilet and into the bedroom, that is anything but not in
particular in order.

So i am sitting behind that wall and have to take what i get.
And then i am absolutely convinced that humans make a lot of
faults, and anything that does not autoconfigure correctly is
prone to misconfiguration and errors, whatever may be the cause,
from a hoped-for girl friend to a dying suffering wife, you know.

I think i personally could agree with these signatures if the
transport would be secure, and if i could make a short single
connection to the root server of a zone and get the certificate
along with the plain TLS handshake.  What i mean is, the DNS would
automatically provide signatures based on the very same key that
is used for TLS, and the time to live for all delivered records is
en par with the lifetime of that certificate at maximum.
No configuration at all, automatically secure, a lot code less to
maintain.  And maybe even knowledge whether signatures are to be
expected for a zone, so that anything else can be treated as
spoofed.

All this is my personal blabla though.  I think that thinking is
not bad, however.  Anyway anybody who is not driving all the
server her- or himself is putting trust since decades and all the
time, and if i can have a secure channel to those people i (put)
trust (in) then this is a real improvement.  If those people do
the same then i have absolutely zero problems with only encrypted
transport as opposed to open transport and signed data.

 |> Twenty years too late, but i really had good days when i saw those 
 |> standards flying by!  Now all we need are zone administrators which 
 |> publish certificates via DNS and DTLS and TCP/TLS consumers which can 
 |> integrate those in their own local pool (for at least their runtime).
 |
 |DANE has been waiting on that for a while.
 |
 |I think DANE does require DNSSEC.  I think DKIM does too.

I have the RFCs around... but puuh.  DKIM says

   The DNS is proposed as the initial mechanism for the public keys.
   Thus, DKIM currently depends on DNS administration and the security
   of the DNS system.  DKIM is designed to be extensible to other key
   fetching services as they become available.

 |> I actually hate the concept very, very much ^_^, for me it has 
 |> similarities with brainfuck.  I could not use it.
 |
 |Okay.  To each his / her own.

Of course.

 |> Except that this will work only for all-english, as otherwise character 
 |> sets come into play, text may be in a different character set, mail 
 |> standards may impose a content-transfer encoding, and then what you are 
 |> looking at is actually a database, not the data as such.
 |
 |Hum.  I've not considered non-English as I so rarely see it in raw email 
 |format.
 |
 |> This is what i find so impressive about that Plan9 approach, where 
 |> the individual subparts of the message are available as files in the 
 |> filesystem, subjects etc. as such, decoded and available as normal \
 |> files. 
 |> I think this really is .. impressive.
 |
 |I've never had the pleasure of messing with Plan9.  It's on my 
 |proverbial To-Do-Someday list.  It does sound very interesting.

To me it is more about some of the concepts in there, i actually
cannot use it with the defaults.  I cannot the editors Sam nor
Acme, and i do not actually like using the mouse, which is
a problem.

 |> Thanks, eh, thanks.  Time will bring.. or not.
 |
 |;-)

Oh, i am hoping for "will", but it takes a lot of time.  It would
have been easier to start from scratch, and, well, in C++.  I have
never been a real application developer, i am more for libraries
or say, interfaces which enable you to do something.  Staying
a bit in the back, but providing support as necessary.  Well.
I hope we get there.

 |> It is of course not the email that leaves you no more.  It is not just 
 |> headers are added to bring the traceroute path.  I do have a bad feeling 
 |> with these, but technically i do not seem to have an opinion.
 |
 |I too have some unease with things while not having any technical 
 |objection to them.
 |
 |> Well, this adds the burden onto TUHS.  Just like i have said.. but 
 |> you know, more and more SMTP servers connect directly via STARTSSL or 
 |> TCP/TLS right away.  TUHS postfix server does not seem to do so on the 
 |> sending side
 |
 |The nature of email is (and has been) changing.
 |
 |We're no longer trying to use 486s to manage mainstream email.  My 
 |opinion is that we have the CPU cycles to support current standards.
 |
 |> -- you know, i am not an administor, no earlier but on the 15th of March 
 |> this year i realized that my Postfix did not reiterate all the smtpd_* 
 |> variables as smtp_* ones, resulting in my outgoing client connections 
 |> to have an entirely different configuration than what i provided for 
 |> what i thought is "the server", then i did, among others
 |> 
 |> smtpd_tls_security_level = may +smtp_tls_security_level = 
 |> $smtpd_tls_security_level
 |
 |Oy vey.

Hm.

 |> But if TUHS did, why should it create a DKIM signature?  Ongoing is the 
 |> effort to ensure SMTP uses TLS all along the route, i seem to recall i 
 |> have seen RFCs pass by which accomplish that.  Or only drafts??  Hmmm.
 |
 |SMTP over TLS (STARTTLS) is just a transport.  I can send anything I 
 |want across said transport.

That is true.

 |DKIM is about enabling downstream authentication in email.  Much like 
 |DNSSEC does for DNS.
 |
 |There is nothing that prevents sending false information down a secure 
 |communications channel.

I mean, that argument is an all destroying hammer.  But it is
true.  Of course.

 |> Well S/MIME does indeed specify this mode of encapsulating the entire 
 |> message including the headers, and enforce MUAs to completely ignore the 
 |> outer envelope in this case.  (With a RFC discussing problems of this 
 |> approach.)
 |
 |Hum.  That's contrary to my understanding.  Do you happen to have RFC 
 |and section numbers handy?  I've wanted to, needed to, go read more on 
 |S/MIME.  It now sounds like I'm missing something.

In draft "Considerations for protecting Email header with S/MIME"
by Melnikov there is

    [RFC5751] describes how to protect the Email message
    header [RFC5322], by wrapping the message inside a message/rfc822
    container [RFC2045]:

    In order to protect outer, non-content-related message header
    fields (for instance, the "Subject", "To", "From", and "Cc"
    fields), the sending client MAY wrap a full MIME message in a
    message/rfc822 wrapper in order to apply S/MIME security services
    to these header fields.  It is up to the receiving client to
    decide how to present this "inner" header along with the
    unprotected "outer" header.

    When an S/MIME message is received, if the top-level protected
    MIME entity has a Content-Type of message/rfc822, it can be
    assumed that the intent was to provide header protection.  This
    entity SHOULD be presented as the top-level message, taking into
    account header merging issues as previously discussed.

I am a bit behind, the discussion after the Efail report this
spring has revealed some new development to me that i did not know
about, and i have not yet found the time to look at those.

 |I wonder if the same can be said about PGP.

Well yes, i think the same is true for such, i have already
received encrypted mails which ship a MIME multipart message, the
first being 'Content-Type: text/rfc822-headers;
protected-headers="v1"', the latter providing the text.

 |> The BSD Mail clone i maintain does not support this yet, along with \
 |> other 
 |> important aspects of S/MIME, like the possibility to "self-encrypt" (so 
 |> that the message can be read again, a.k.a. that the encrypted version 
 |> lands on disk in a record, not the plaintext one).  I hope it will be 
 |> part of the OpenPGP, actually privacy rewrite this summer.
 |
 |I thought that many MUAs handled that problem by adding the sender as an 
 |additional recipient in the S/MIME structure.  That way the sender could 
 |extract the ephemeral key using their private key and decrypt the 
 |encrypted message that they sent.

Yes, that is how this is done.  The former maintainer implemented
this rather as something like GnuPG's --hidden-recipient option,
more or less: each receiver gets her or his own S/MIME encrypted
copy.  The call chain itself never sees such a mail.

 --End of <c4debab5-181d-c071-850a-a8865d6e98d0@spamtrap.tnetconsulting.net>

Grant Taylor via TUHS wrote in <5da463dd-fb08-f601-68e3-197e720d5716@spa\
mtrap.tnetconsulting.net>:
 |On 06/25/2018 10:10 AM, Steffen Nurpmeso wrote:
 |> DKIM reuses the *SSL key infrastructure, which is good.
 |
 |Are you saying that DKIM relies on the traditional PKI via CA 
 |infrastructure?  Or are you saying that it uses similar technology that 
 |is completely independent of the PKI / CA infrastructure?

I mean it uses those *SSL tools and thus an infrastructure that is
standardized as such and very widely used, seen by many eyes.

 |> (Many eyes see the code in question.)  It places records in DNS, which 
 |> is also good, now that we have DNS over TCP/TLS and (likely) DTLS. 
 |> In practice however things may differ and to me DNS security is all in 
 |> all not given as long as we get to the transport layer security.
 |
 |I believe that a secure DNS /transport/ is not sufficient.  Simply 
 |security the communications channel does not mean that the entity on the 
 |other end is not lying.  Particularly when not talking to the 
 |authoritative server, likely by relying on caching recursive resolvers.

That record distribution as above, yes, but still: caching those
TSIGs or what their name was is not mandatory i think.

 |> I personally do not like DKIM still, i have opendkim around and 
 |> thought about it, but i do not use it, i would rather wish that public 
 |> TLS certificates could also be used in the same way as public S/MIME 
 |> certificates or OpenPGP public keys work, then only by going to a TLS 
 |> endpoint securely once, there could be end-to-end security.
 |
 |All S/MIME interactions that I've seen do use certificates from a well 
 |know CA via the PKI.
 |
 |(My understanding of) what you're describing is encryption of data in 
 |flight.  That does nothing to encrypt / protect data at rest.
 |
 |S/MIME /does/ provide encryption / authentication of data in flight 
 |/and/ data at rest.
 |
 |S/MIME and PGP can also be used for things that never cross the wire.

As above, i just meant that if the DNS server is TLS protected,
that key could be used to automatically sign the entire zone data,
so that the entire signing and verification is automatic and can
be deduced from the key used for and by TLS.  Using the same
library, a single configuration line etc.  Records could then be
cached anywhere just the same as now.  Just an idea.

You can use self-signed S/MIME, you can use an OpenPGP key without
any web of trust.  Different to the latter, where anyone can upload
her or his key to the pool of OpenPGP (in practice rather GnuPG only
i would say) servers (sks-keyservers.net, which used a self-signed
TLS certificate last time i looked btw., fingerprint
79:1B:27:A3:8E:66:7F:80:27:81:4D:4E:68:E7:C4:78:A4:5D:5A:17),
there is no such possibility for the former.  So whereas everybody
can look for the fingerprint i claim via hkps:// and can be pretty
sure that this really is me, no such thing exists for S/MIME.
(I for one was very disappointed when the new German passport had
been developed around Y2K, and no PGP and S/MIME was around.  The
Netherlands i think are much better in that.  But this is
a different story.)

But i think things happen here, that HTTP .well-known/ mechanism
seems to come into play for more and more things, programs learn
to use TOFU (trust on first use), and manage a dynamic local pool
of trusted certificates.  (I do not know exactly, but i have seen
fly by messages which made me think the mutt MUA put some work
into that, for example.)  So this is decentralized, then.

 |> I am not a cryptographer, however.  (I also have not read the TLS v1.3 
 |> standard which is about to become reality.)  The thing however is that 
 |> for DKIM a lonesome user cannot do anything -- you either need to have 
 |> your own SMTP server, or you need to trust your provider.
 |
 |I don't think that's completely accurate.  DKIM is a method of signing 
 |(via cryptographic hash) headers as you see (send) them.  I see no 
 |reason why a client can't add DKIM headers / signature to messages it 
 |sends to the MSA.
 |
 |Granted, I've never seen this done.  But I don't see anything preventing 
 |it from being the case.
 |
 |> But our own user interface is completely detached.  (I mean, at least 
 |> if no MTA is used one could do the DKIM stuff, too.)
 |
 |I know that it is possible to do things on the receiving side.  I've got 
 |the DKIM Verifier add-on installed in Thunderbird, which gives me client 
 |side UI indication if the message that's being displayed still passes 
 |DKIM validation or not.  The plugin actually calculates the DKIM hash 
 |and compares it locally.  It's not just relying on a header added by 
 |receiving servers.

I meant that the MUA could calculate the DKIM stuff itself, but
this works only if the MTA does not add or change headers.  That
is what i referred to, but DKIM verification could be done by
a MUA, if it could.  Mine cannot.

 --End of <5da463dd-fb08-f601-68e3-197e720d5716@spamtrap.tnetconsulting.net>

Grant Taylor via TUHS wrote in <8c0da561-f786-8039-d2fc-907f2ddd09e3@spa\
mtrap.tnetconsulting.net>:
 |On 06/25/2018 10:26 AM, Steffen Nurpmeso wrote:
 |> I have never understood why people get excited about Maildir, you have 
 |> trees full of files with names which hurt the eyes,
 |
 |First, a single Maildir is a parent directory and three sub-directories. 
 |  Many Maildir based message stores are collections of multiple 
 |individual Maildirs.

This is Maildir+ i think was the name, yes.

 |Second, the names may look ugly, but they are named independently of the 
 |contents of the message.
 |
 |Third, I prefer the file per message as opposed to monolithic mbox for 
 |performance reasons.  Thus I message corruption impacts a single message 
 |and not the entire mail folder (mbox).

Corruption should not happen, then.  This is true for each
database or repository it think.

 |Aside:  I already have a good fast database that most people call a file 
 |system and it does a good job tracking metadata for me.

This can be true, yes.  I know of files systems which get very
slow when there are a lot of files in a directory, or which even
run into limits in the worst case.  (I have indeed once seen the
latter on a FreeBSD system with which i though were good file
system defaults.)

 |Fourth, I found maildir to be faster on older / slower servers because 
 |it doesn't require copying (backing up) the monolithic mbox file prior 
 |to performing an operation on it.  It splits reads & writes into much 
 |smaller chunks that are more friendly (and faster) to the I/O sub-system.

I think this depends.  Mostly anything incoming is an appending
write, in my use case then everything is moved away in one go,
too.  Then it definetely depends on the disks you have.  Before
i was using a notebook which had a 3600rpm hard disk, then you
want it compact.  And then it also depends on the cleverness of
the software.  Unfortunately my MUA cannot, but you could have an
index like git has or like the already mentioned Zawinski
describes as it was used for Netscape.  I think the i think
Enterprise Mail server dovecot also uses MBOX plus index by
default, but i could be mistaken.  I mean, if you control the file
you do not need to perform an action immediately, for example --
and then, when synchronization time happens, you end up with
a potentially large write on a single file, instead of having to
fiddle around with a lot of files.  But your experience may vary.

 |Could many of the same things be said about MH?  Very likely.  What I 
 |couldn't say about MH at the time I went looking and comparing (typical) 
 |unix mail stores was the readily available POP3 and IMAP interface. 
 |Seeing as how POP3 and IMAP were a hard requirement for me, MH was a 
 |non-starter.
 |
 |> they miss a From_ line (and are thus not valid MBOX files, i did not 
 |> miss that in the other mail).
 |
 |True.  Though I've not found that to be a limitation.  I'm fairly 
 |confident that the Return-Path: header that is added / replaced by my 
 |MTA does functionally the same thing.

With From_ line i meant that legendary line that i think was
introduced in Unix V5 mail and which prepends each mail message in
a MBOX file, and which causes that ">From" quoting at the
beginning of lines of non MIME-willing (or so configured) MUAs.

 |I'm sure similar differences can be had between many different solutions 
 |in Unix's history.  SYS V style init.d scripts vs BSD style /etc/rc\d 
 |style init scripts vs systemd or Solaris's SMD (I think that's it's name).

Oh!  I am happy to have the former two on my daily machines again,
and i have been reenabled to tell you what is actually going on
there (in user space), even without following some external
software development.  Just in case of interest.

 |To each his / her own.

It seems you will never be able to 1984 that from the top,
especially not over time.

 --End of <8c0da561-f786-8039-d2fc-907f2ddd09e3@spamtrap.tnetconsulting.net>

Ciao.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [TUHS] off-topic list

[Tim Bradshaw]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On 22 Jun 2018, at 15:54, Larry McVoy <lm@mcvoy.com> wrote:
> 
> Try mutt, it's what I use and it threads topics just fine.

The trouble is I have > 10 years of mail sitting in the OSX mail system and although I could probably export it all (it at least used to be relatively straightforward to do this) the sheer terror of doing that is, well, terrifying because there's a lot of stuff in there that matters.

Using the system-provided mail tool was a stupid decision, and one I managed to avoid with the browser &c, but it's too late now.

(Now this is an off-topic discussion from a discussion about off-topicness.)

[-- Attachment #2: Type: text/html, Size: 1481 bytes --]

Re: [TUHS] off-topic list

[Steve Johnson]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

I, for one, am happy to see some off-topic stuff.   The people on
this list, for the most part, represent a way of looking at the world
that is,
sadly, rather uncommon these days.   I enjoy looking at more
contemporary events through those eyes, and musing on the changes that
have taken place...

Steve

----- Original Message -----
From:
 "Tim Bradshaw" <tfb@tfeb.org>

To:
"Larry McVoy" <lm@mcvoy.com>
Cc:
"The Eunuchs Hysterical Society" <tuhs@tuhs.org>
Sent:
Fri, 22 Jun 2018 17:07:57 +0100
Subject:
Re: [TUHS] off-topic list

 On 22 Jun 2018, at 15:54, Larry McVoy <lm@mcvoy.com [1]> wrote:

Try mutt, it's what I use and it threads topics just fine.

The trouble is I have > 10 years of mail sitting in the OSX mail
system and although I could probably export it all (it at least used
to be relatively straightforward to do this) the sheer terror of doing
that is, well, terrifying because there's a lot of stuff in there that
matters.

Using the system-provided mail tool was a stupid decision, and one I
managed to avoid with the browser &c, but it's too late now.

(Now this is an off-topic discussion from a discussion about
off-topicness.)
 

Links:
------
[1] mailto:lm@mcvoy.com


[-- Attachment #2: Type: text/html, Size: 2215 bytes --]

Re: [TUHS] off-topic list

[Bakul Shah]

On Jun 22, 2018, at 9:07 AM, Tim Bradshaw <tfb@tfeb.org> wrote:
> 
> On 22 Jun 2018, at 15:54, Larry McVoy <lm@mcvoy.com> wrote:
>> 
>> Try mutt, it's what I use and it threads topics just fine.
> 
> The trouble is I have > 10 years of mail sitting in the OSX mail system and although I could probably export it all (it at least used to be relatively straightforward to do this) the sheer terror of doing that is, well, terrifying because there's a lot of stuff in there that matters.

At least on my Mac messages seem to be stored one per file.
Attachments are stored separately. Doesn't seem hard to figure out.
You can periodically rsync to a different place and experiment.

> Using the system-provided mail tool was a stupid decision, and one I managed to avoid with the browser &c, but it's too late now.

There is not much available that is decent.I too use Apple Mail
but also have a separate MH store that goes way back. MH is great
for bulk operations but not for viewing MIME infested mail or
simple things like attaching documents. What I really want is a
combined MUA.

> (Now this is an off-topic discussion from a discussion about off-topicness.)

meta-meta-meta!

Re: [TUHS] off-topic list

[Ralph Corderoy]

Larry wrote:
> For the record, I'm fine with old stuff getting discussed on TUHS.
> Even not Unix stuff.

+1

> I think this list has self selected for adults who are reasonable.
> So we mostly are fine.

And when it's trying to recreate the Usenet glory days of `Is this the
longest thread ever?', Warren steps in to admonish.

Cheers, Ralph.

Re: [TUHS] off-topic list

[SPC]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

​

El vie., 22 jun. 2018 a las 16:59, Ralph Corderoy (<ralph@inputplus.co.uk>)
escribió:

> Larry wrote:
> > For the record, I'm fine with old stuff getting discussed on TUHS.
> > Even not Unix stuff.
>

​Me too. But...

Just in case, try something like 'old-iron', 'tuhs-old-iron-chat',
'tuhs-alt-chat'​...

Anyway, as more-or-less community manager on modern social networks, I
think it won't be easy to separate the topics of both lists.
​​
Gracias | Regards - Saludos | Greetings | Freundliche Grüße | Salutations
​
-- 
*Sergio Pedraja*
--
http://www.linkedin.com/in/sergiopedraja
-----
No crea todo lo que ve, ni crea que está viéndolo todo
-----
"El estado de una Copia de Seguridad es desconocido
hasta que intentas restaurarla" (- nixCraft)
-----

[-- Attachment #2: Type: text/html, Size: 4882 bytes --]

Re: [TUHS] off-topic list

[Larry McVoy]

On Fri, Jun 22, 2018 at 03:52:14PM +0100, Ralph Corderoy wrote:
> And when it's trying to recreate the Usenet glory days of `Is this the
> longest thread ever?', Warren steps in to admonish.

I've been trying (not always succeeding) to not help things get to that
point.  Warren is awesome but we all should have a goal of not "needing"
him to step in.  

--lm

Re: [TUHS] off-topic list

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On Fri, Jun 22, 2018 at 10:28 AM, Larry McVoy <lm@mcvoy.com> wrote:

> I think this list has self selected for adults

​...right ... problem is if some one never grew up how would they know ....

Seriously, I think you pretty much nailed it.   It is about being
respectful of everyone on the list, particularly those where were there and
lived the history, it allows us all to learn from some fun memories and
broaden our understanding beyond what we thought we knew as complete
'fact'.   I've enjoyed filling in tid bits I've collected along my strange
journey, as well as having others clue me in on some details I did not
know.     As Larry said, it really is an interesting community and think
this list has helped to set the historical record straight in a couple of
places that I am aware.

ᐧ

[-- Attachment #2: Type: text/html, Size: 1808 bytes --]

Re: [TUHS] off-topic list

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]

On 06/22/2018 08:28 AM, Larry McVoy wrote:
> For the record, I'm fine with old stuff getting discussed on TUHS. 
> Even not Unix stuff.

I agree.

It seems like a number of people are okay with non-TUHS specific topis 
being discussed on TUHS.  I'm sure there are others that want non-TUHS 
specific topics to stay off TUHS.  I feel like each group is entitled to 
their opinions.

So, perhaps we could leverage technology to satisfy both groups.

I believe that Warren hosts TUHS using Mailman.  I'm fairly certain that 
Mailman supports topics / key words.  Thus we could possible configure 
Mailman to support such topics and allow subscribers to select which 
topics they care about and the ever so important if they want to receive 
messages that don't match any defined topics.

> We wandered into Linux/ext2 with Ted, that was fun.  We've wandered 
> into the VAX history (UWisc got an 8600 and called it "speedy" - what a 
> mistake) and I learned stuff I didn't know, so that was fun (and sounded 
> just like history at every company I've worked for, sadly :)

I routinely find things being discussed that I didn't know I was 
interested in, but learned that I was while reading.  I almost always 
learn something from every (major) thread.

> I think this list has self selected for adults who are reasonable. 
> So we mostly are fine.

:-)



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3982 bytes --]

Re: [TUHS] off-topic list

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

On Fri, Jun 22, 2018 at 10:29 AM Larry McVoy <lm@mcvoy.com> wrote:

> For the record, I'm fine with old stuff getting discussed on TUHS.
> Even not Unix stuff.


With the caveat that I must acknowledge that I've been guilty of wandering
off topic more often than I should, it occurs to me that when discussing
the history of something, very often the histories of other things
necessarily feed into that history and become intertwined and inseparable.
There was a lot of "stuff" happening in the computer industry at the time
Unix was created, in it's early years, and on in its (continuing)
evolution, and that "stuff" surely impacted Unix in one way or another. If
we really want to understand where Unix came from and why it is, we must
open ourselves to understanding those influences as well.

That said, of course, there's a balance. Having a place one could point to
and respectfully say, "Hey, this has gone on for 50+ messages; could you
move it over to off-tuhs?" might be useful for folks who want to deep-dive
on something.

We wandered into Linux/ext2 with Ted, that was fun.
>

Indeed. I'll go further and confess that I use TUHS as a learning resource
that influences by work professionally. There's a lot of good information
that comes across this list that gets filed away in my brain and manifests
itself in surprising ways by informing my work. I selfishly want that to
continue.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2011 bytes --]

Re: [TUHS] off-topic list

[Cág]

Dave Horsfall wrote:
 
> COFF - Computer Old Fart Followers.

Is it "Old farts who follow computers" or "Followers of computer old
farts"? Or "old farts who follow other old farts"? :)

-- 
caóc