* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
@ 2020-06-03 0:43 Bakul Shah
2020-06-03 0:49 ` Clem Cole
0 siblings, 1 reply; 8+ messages in thread
From: Bakul Shah @ 2020-06-03 0:43 UTC (permalink / raw)
To: markus schnalke; +Cc: TUHS main list
> Would you be so kind to explain a bit about the hm version of MH.
Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my
Fortune box until I retired it. It provided a 2D interface. Later I
tried xmh but didn't like it.
The early history of MH is covered in some detail in Willis Ware's "RAND
and the Information Evolution" book from page 128 onward. hm & Guyton
get a paragraph on page 136.
https://www.rand.org/content/dam/rand/pubs/corporate_pubs/2008/RAND_CP537.pdf
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-06-03 0:43 [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) Bakul Shah @ 2020-06-03 0:49 ` Clem Cole 0 siblings, 0 replies; 8+ messages in thread From: Clem Cole @ 2020-06-03 0:49 UTC (permalink / raw) To: Bakul Shah; +Cc: TUHS main list [-- Attachment #1: Type: text/plain, Size: 425 bytes --] On Tue, Jun 2, 2020 at 8:43 PM Bakul Shah <bakul@iitbombay.org> wrote: > > Would you be so kind to explain a bit about the hm version of MH. > > Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my > Fortune box until I retired it. It provided a 2D interface. > Yep, that would be the program .. Jim did a great job with it. It was built using a bunch of MH programs behind it, and was a curses front end. [-- Attachment #2: Type: text/html, Size: 1006 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* [TUHS] fmt(1): history, POSIX, -t, -c @ 2020-05-28 12:34 markus schnalke 2020-05-28 13:46 ` Clem Cole 0 siblings, 1 reply; 8+ messages in thread From: markus schnalke @ 2020-05-28 12:34 UTC (permalink / raw) To: tuhs Hoi, personally I use fmt(1) a lot for email formatting and such. Typically I only use the `-w' parameter. Now someone asked me about `-t' and `-c' of *GNU* fmt(1). I wasn't able to find good documentation on them. The manpage only tells that they have to do with different indentation for the first or first two lines. But what are the use cases? How would source text for these parameters look like? A look into the description and rationale sections of POSIX, which often provides helpful information, was not possible because fmt(1) is not part of POSIX (only fold(1) is). Why's that? Is it because fmt(1) differs so much between Unix implementations? On BSD `-c' centers text and `-t' sets tab widths. Plan 9 has none of these options. But still, `-w' could have been standardized. Or was the line filling algorithm different as well? How does fold(1) fit into the picture? Maybe you can answer some of these questions or give hints on where I could find answers myself. meillo ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] fmt(1): history, POSIX, -t, -c 2020-05-28 12:34 [TUHS] fmt(1): history, POSIX, -t, -c markus schnalke @ 2020-05-28 13:46 ` Clem Cole 2020-05-31 13:01 ` [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) markus schnalke 0 siblings, 1 reply; 8+ messages in thread From: Clem Cole @ 2020-05-28 13:46 UTC (permalink / raw) To: markus schnalke; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 1876 bytes --] On Thu, May 28, 2020 at 8:41 AM markus schnalke <meillo@marmaro.de> wrote: > A look into the description and rationale sections of POSIX, which > often provides helpful information, was not possible because fmt(1) > is not part of POSIX (only fold(1) is). Why's that? > It was not in SVID and nobody from the BSD side of the war at the time felt it was worth arguing about to add it to the standard. Basically, during the writing of both POSIX.1 and .2, there was huge pressure from AT&T to just take the SVID and try to make that the standard. In fact, IIRC, Jim Issack got AT&T to release the copyright on it and we used some of the original AT&T troff source. But many of us pushed back saying even if there was a marketing campaign: "AT&T UNIX®, Consider it Standard" it was not hardly so. And many BSD additions (improvements) were taken into the standard. For instance, sockets was the prefered to networking interface, although to save face AT&T managed to get the TLI allowed in as an alternative to sockets in the first version of the network specification. (Funny, I don't know of a FIPS-151 registered UNIX implementation that used TLI). Remember, the primary driver for the POSIX work was for the ISV's - to make it easier for them to create software that they could sell. Early on, Heinz in particular, wanted an ABI, not an API (many of us, myself in that camp) shouted him down. Since those days, I've sometimes wondered if we had earlier on figured out how to do that; maybe the UNIX Wars would have worked out differently (but thats a different discussion). Back to fmt(1), like you, I have used it for years, particularly in email. I usually forked it from vi to paginate my message was what I did for years until I finally switched from mh (actually the hm version) to the Gmail interface as my MUI client. Clem [-- Attachment #2: Type: text/html, Size: 3138 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-28 13:46 ` Clem Cole @ 2020-05-31 13:01 ` markus schnalke 2020-05-31 14:53 ` Clem Cole 0 siblings, 1 reply; 8+ messages in thread From: markus schnalke @ 2020-05-31 13:01 UTC (permalink / raw) To: tuhs Hoi. [2020-05-28 09:46] Clem Cole <clemc@ccc.com> > > [...] until > I finally switched from mh (actually the hm version) to the Gmail interface as > my MUI client. Would you be so kind to explain a bit about the hm version of MH. Ten years ago I wanted to improve nmh, because I found it bad that it took me months to configure it in so many ways to get it usable for modern emailing. Even at that point I hadn't found some of its cool features, which all were deactivated by default. I argued but couldn't convince the nmh community. Later I used my master's thesis as the opportunity to create an experimental version of nmh, to convince by demonstration. Have a look at my master's thesis, if you like: http://marmaro.de/docs/master/ Actually it became a fork, now named mmh. The project's still active. Especially Philipp Takacs has done a lot, among that replacing m_getfld(), a highly optimized mail reading function. See the pre-mmh version of it for an entertaining read: http://git.marmaro.de/?p=mmh;a=blob;f=docs/m_getfld.c.humor;h=46449095d This is our replacement: http://git.marmaro.de/?p=mmh;a=blob;f=sbr/m_getfld2.c;h=b9a618d16 I'm much interested in any MH background. Shockingly I cannot recall having read about hm before ... meillo ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-31 13:01 ` [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) markus schnalke @ 2020-05-31 14:53 ` Clem Cole 2020-05-31 16:25 ` Richard Salz 2020-05-31 17:09 ` Ralph Corderoy 0 siblings, 2 replies; 8+ messages in thread From: Clem Cole @ 2020-05-31 14:53 UTC (permalink / raw) To: markus schnalke; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 7241 bytes --] On Sun, May 31, 2020 at 9:02 AM markus schnalke <meillo@marmaro.de> wrote: > Would you be so kind to explain a bit about the hm version of MH. > I can try ... At the time V6 (and later V7) had a program called /bin/mail. The important thing about this program is that it was both a Mail Transport (and delivery) Agent as well as a Mail User Interface. The original version only delivered mail locally. When networking, like UUCP, was added to UNIX, it was easy to build a scheme that could send a transfer a message using UUCP and then fork /bin/mail on the proper system to deliver it. IIRC it was in Seventh Edition that /bin/mail was extended to recognize email addresses in the prefix form of: host!user, strip the first host! part and pass it to uucp. I never used it on anything like Spider, so I have not idea what it did internally and in fact, the prefix address form could have predated UUCP. We would need someone like Doug, Steve or Ken to tell us how email worked host to host, pre-UUCP. I believe it was Bruce Borden and team developed the original Rand Message Handler or MH. I do not know who was the primary author, we need to ask Bruce or one of the old Rand folks. MH ran on 6th edition when it was first released to rest of the USENIX community (at least I never saw it on V5, but it's possible it went back further). I just remember talking to Bruce about it at any early USENIX. MH used a new Mailbox (on-disk) scheme and formatted all messages in RFC733 form with addresses being flat an in the form: user@host. The new format, used a line a control-As followed by a nl, before and after the message. The headers of course where RFC733 (type: value with a trailing nl) and were separated from the body of the message by a single nl. MH assumed a traditional UNIX command line for the user interface and had a number of programs that ran behind the scene for delivery. This is important because the original ARPAnet NCP used FTP to do mail transfer. At least one version could call the early ARPAnet subsystem to perform host to host communications. The key point is that MH separated the MTA and MUI. A number of us ran MH at different places. I don't remember if it was on a USENIX tape or I had sent Bruce a tape @ Rand when I got a copy in the late 1970's (77-79 timeframe, I've forgotten). I'm pretty sure Goble, as did the Purdue crew ran it, as did Holmgrem, Greg Chesson at al, @ UofI and I would not surprised if it was the mailer at Harvard, given the Harvard/Rand connection in those days. By the time of Seventh Edition of UNIX, the Mail Transport Agent (MTA) that was part of the MH subsystem, could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses and /bin/mail replacement knew to work like the AT&T mailer and pass it off to UUCP. So, at that point, life was good for those of us in strickly ARPA and/or UUCP land. BTW: At some point, the BBN TCP code was releases and a separate SMTPD was included in the release. I don't remember if it was the Rand folks or someone else, but at some point, the MH delivery agent was updated to call it or be called by it. Similarly, by the late 1970s, when Bruce, Greg Shaw, and Bob Metcalfe formed 3Com; Bruce and Greg brought MH with them and added an SMTPD that they wrote for their commercial product, UNET. This was the smtpd, I ran on the Teklabs machine before I went off to UCB. Meanwhile, as Mary Ann and I noted in earlier messages, Kurt Shoen's wrote a different MTA called Mail(1ucb). Unlike MH, Kurt continues to support the original UNIX mailbox format (later named 'mbox' format). The header lines were in a specific UNIX style prose starting with the ACSII 'From' and there are no special characters to demark the messages in the format, so recovery can be fraught with error, there are the famous From-line munging issues etc. (lots of details in other places start with https://en.wikipedia.org/wiki/Mbox). At some point in time, UCB built the 'Berk-net' (whose original code was written by ABC/Google's Eric Schmidt). BTW: Eric would have seen the Spider Network in his summers at BTL. The key thing with Berknet was cheap. It ran over 3 wire RS-232C between hosts at 9600 baud. Like UUCP was used to transfer files and email. Like UUCP it used a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP was flat. Where Mary Ann and I differ in our memories is who wrote the original version of UCB's delivermail (8ucb) program. We both agree that it is possible it was Eric Schmidt, as the switch to using delivermail(8ucb) was were Berknet was spliced into the email namespace. I had thought Kurt wrote it, Mary Ann thought it was Eric Allman. We agree Eric Allman was hacking on it for the Ing70. For this response, it doesn't really matter other than to try to get the history right, because it does not matter for the Rand Mail subsystem. Around, now I arrive at UCB. I was not the only person that had used MH, but I had the advantage of having 3 Vaxen 780 in the CAD lab. The key change we made at that point was to stop using the Rand MTA and make it use delivermail. There was hackery to allow the shared mbox to say, but in each user directory it stored the messages separately, MH style so the MH 'ui' suite of tools 'just worked.' [There were a number of arguments at the time. I remember having one with Sam. He hated MH because 'of all the small files and it chewed up inodes]. Anyway around this time, the curses library was created by Ken Arnold (originally to support Rogue) by pulling the screen code out of vi and using Mary Ann's termcap stuff. A couple of us in the USENIX community started playing with screen-based front ends to MH, including the folks at Rand. Numerous messages were exchanged, and a collaboration started (Rand team should get 99% percent of the work, I knew how to make curses go). The comment in the main code was, 'Well it sure isn't MH' -- so it was called HM. I later brought it Masscomp, and it was the start of their mailer. The key with HM, is that it uses the MH backend for all the real work. It would fork scan, but save the output in a text file that could be manipulated with the arrow keys. The user interface is not unlike Gmail. I only stopped running HM when I finally switched to Gmail a few years ago. In fact, when I was a DEC, the precursor to Gmail, was called Pachyderm which used the Altavista search. I glued MH to Pachyderm for a short period. I even ported them both to FreeBSD to run at home in the mid-1990s. Then I left DEC, the authors of Pachyderm headed for Google, *etc*... I got a chance to be an early Gmail/Google Apps tester for ccc.com and I have not switched back since. FWIW: It's been long enough, that the sources are no longer easy to find. It's all very incestuous. We all have seen other ideas and good ideas seem to have a way of reappearing in different places. Clem Note the part of the story I left out was when Rand MH spun out to one of the other Universities, UCI I think. I was never much involved with that team, so I can not tell you much about it and their code base. [-- Attachment #2: Type: text/html, Size: 12458 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-31 14:53 ` Clem Cole @ 2020-05-31 16:25 ` Richard Salz 2020-05-31 16:50 ` Larry McVoy 2020-05-31 17:09 ` Ralph Corderoy 1 sibling, 1 reply; 8+ messages in thread From: Richard Salz @ 2020-05-31 16:25 UTC (permalink / raw) To: Clem Cole; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 1538 bytes --] "I'll try" Wow, that was quite some "try" Kudo's! > This is important because the original ARPAnet NCP used FTP to do mail transfer. If you look at RFC 765 (dated June 1980) , there are nearly a dozen mail-related FTP commands. By RFC 959, it's successor five years later, all of them were gone. Another eary MTA was the MMDF, the Multichannel Memorandum Distribution Facility, from U Delware. BBN ran it. One key point is that somehow when used with MH you could get real-time address verification before sending, "user rs@bbn.com doesn't exist." BBN ran MMDF for a long time because a key exec liked/needed that feature, long after 4.2 and sendmail. (My group was one of the first to run sendmail, which I liked because of the "R$" lines in its CF files. (Not really :) My Usenet/email gateway code had to support MMDF but it was only ever used on bbn.com) > could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses Originally ihnp4!mirror!rs@seismo.arpa, then ihnp4!mirror!rs@seismo.css.gov and then rs%mirror.uucp@seismo.css.gov and then rs@mirror.tmc.com Thanks to Mary Ann and the "UUCP Mapping Project" for making much of that possible and Peter Honeyman for Pathlias, and MX records for the last part. The pre-MX styles received much scorn from Research, and if you search for "that hideous name" you can find a paper on it. There's also a pathlias paper, which Honey later said "was too good for Usenix" but he had nowhere else to submit it. [-- Attachment #2: Type: text/html, Size: 2834 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-31 16:25 ` Richard Salz @ 2020-05-31 16:50 ` Larry McVoy 0 siblings, 0 replies; 8+ messages in thread From: Larry McVoy @ 2020-05-31 16:50 UTC (permalink / raw) To: Richard Salz; +Cc: The Eunuchs Hysterical Society On Sun, May 31, 2020 at 12:25:47PM -0400, Richard Salz wrote: > Originally ihnp4!mirror!rs@seismo.arpa, then ihnp4!mirror!rs@seismo.css.gov > and then rs%mirror.uucp@seismo.css.gov and then rs@mirror.tmc.com Thanks > to Mary Ann and the "UUCP Mapping Project" for making much of that possible > and Peter Honeyman for Pathlias My first real research paper was with Udi Manber and it was all about compressing the pathalias maps. I had them on a 40MB disk on a Masscomp that had 20 users. The format was seismo.css.gov geowhiz!geophiz!uwisc!ihnp4!mirror in other words destination path!to!get!there I went to Udi and explained that all of our stuff started with geowhiz!geophiz!uwisc so that string was replicated over and over and it took too much space. Once Udi understood the problem he went "You have time best case, space worst case, you can reverse them like so" seismo -> mirror mirror -> ihnp4 ihnp4 -> uwisc uswisc -> geophiz geophiz -> geowhiz and you'll have time worst case and space best case. I was like "oh, yeah, you are right, cool, thanks" and got up to leave. Not so fast, says Udi, that's not "interesting". What would be interesting is if you could approximate time best case and space best case. With that, I started down the only dynamic programming problem I have ever solved. Udi's idea was to break the graph at what he called pivot points, where the pivot points as much as possible got rid of the replicated strings. He also wanted to limit lookups to just 2. So I had to read in the maps and build a graph from them on my creaky microvax with 4MB of ram. Doing so lead me to the first place I taught Udi about systems. I was fgets() each line and mallocing a copy of it. Then I sorted with qsort() and it took overnight to get the sorted array. Udi asked what sort? Qsort. Oh, that's garbage, use a radix sort, that will be better. WTF is a radix sort, look that up, implemented it, it was worse. So I start poking around and somehow realized we were thrashing VM, we didn't fit, and I have no idea how I figured out that malloc was fragmenting memory but I did. Probably figured out that most of my program was backed by swap, not files, and that's usually malloc or brk. So I wrote my own malloc that grabbed memory in 300KB chunks (why that size, I dunno, seemed right). Then I allocated memory out of those chunks. Got rid of the fragmentation and we fit in 4MB. qsort worked fine, took about 20 minutes (we didn't really fit but we were way closer). Udi was watching as I was doing all this, I think I was working in his office, it was his microvax. The fragmentation thing taught him that when Unix says it has virtual memory and you can just use it, it only works well when you fit. So the programming problem turned out that you calculated the space it work take for every possible pivot point and then at the end, you took the set that gave you the minimum space. We wrote, heh, who am I kidding, Udi wrote, a paper about it. I was still green on writing papers, they were hard for me. I told Udi that and he stared at me blankly and said "Writing papers is easy if you know the content. You just make a good outline and fill it in." He's right, that's how you do it, and it is easy if you know and can make the outline. I went on to do a lot of work with/for him. I wrote a user level thread library that made me write swtch() in assembler for the VAX (I believe I did a 68K version too but the source I have is VAX only). I "cheated" and did 99% of the work in C and popped down the rabbit hole to assembler, did the nasty, and popped back out as a different thread. 112 lines: http://www.mcvoy.com/lm/T/src/Tasm.S Funny Udi story, I was young and wild and drove too fast and got tickets. So I got a radar detector that beeped when it saw the radar gun. Udi upgraded the microvax to some newer release of X10 or X11 and the new xclock beeped on the hour. Every time it beeped, I jumped because it sounded just like the radar detector. Udi asked about it, I explained, he laughed and said "Larry, you're hacking too fast" each time I jumped :) Good times. I almost did a PhD under him but I didn't care for the CS department at Tucson where he went and I liked industry too much. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-31 14:53 ` Clem Cole 2020-05-31 16:25 ` Richard Salz @ 2020-05-31 17:09 ` Ralph Corderoy 2020-05-31 17:25 ` Jon Steinhart 1 sibling, 1 reply; 8+ messages in thread From: Ralph Corderoy @ 2020-05-31 17:09 UTC (permalink / raw) To: Clem Cole; +Cc: The Eunuchs Hysterical Society Hi Clem, > I believe it was Bruce Borden and team developed the original Rand > Message Handler or MH. I do not know who was the primary author, we > need to ask Bruce or one of the old Rand folks. https://en.wikipedia.org/wiki/MH_Message_Handling_System#History says MH was proposed by R. Stockton Gaines and Norman Shapiro and developed by Bruce S. Borden and others at RAND Corporation. and cites https://www.rand.org/content/dam/rand/pubs/notes/2009/N3017.pdf Norman Shapiro, or Norm, is still using nmh well into his eighties, nmh is a descendant of MH, and pops up on the nmh-workers mailing list to either fill in bits of history or ask for installation advice as the Linux distro shifts under his feet. He did visit Bell Labs, though in '54. :-) Details at https://en.wikipedia.org/wiki/Norman_Shapiro > The new format, used a line a control-As followed by a nl, before and > after the message. MH's mts.conf(5) had the ability to set mmdelim{1,2} to something other than \ca\ca\ca\ca\n; no idea why. And that magic marker would be escaped if it occurred in the middle of a message, to \cb\ca\ca\ca\n. > Note the part of the story I left out was when Rand MH spun out to one > of the other Universities, UCI I think. Wikipedia again: Subsequently development was taken over by Marshall T. Rose and John L. Romine working at University of California, Irvine. nmh... was forked [from that] by Richard Coleman while working at the Georgia Institute of Technology. It incorporates the "LBL changes" made to MH in the late 80s by Van Jacobson, Mike Karels and Craig Leres. MH also had support for UCI's bulletin boards called ‘BBoards’, including on ZOTnet. They were optionally available over POP3 and NNTP, but only one or the other. :-) There was an MH shell of some kind called msh, and a visual mh, vmh. ZOTnet was a LAN for mail by Marshall Rose at UCI with gateways for CSnet and ARPAnet. https://escholarship.org/content/qt4bm4k2vh/qt4bm4k2vh.pdf The ZOTnet: a local area mailing network, 1983 (I still use nmh and I'm using it for this reply.) -- Cheers, Ralph. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) 2020-05-31 17:09 ` Ralph Corderoy @ 2020-05-31 17:25 ` Jon Steinhart 0 siblings, 0 replies; 8+ messages in thread From: Jon Steinhart @ 2020-05-31 17:25 UTC (permalink / raw) To: The Eunuchs Hysterical Society Ralph Corderoy writes: > ... > > (I still use nmh and I'm using it for this reply.) +1 on that. I also still use nmh. To me, the set of shell commands exemplifies the UNIX philosophy as I can easily "show | wc -l" or whatever that is cumbersome with monolithic packages. My minor contributions were adding the attachment support and the hooks. The hooks allow other programs to be triggered behind the scenes when nmh commands are run; I use it to keep an elasticsearch database of my messages. I have two additional command script to support thise: gpick and gscan which are the equivalent of pick and scan except that they search the elasticsearch database and pretty much return results instantaneously. Jon ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-06-03 0:50 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-06-03 0:43 [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) Bakul Shah 2020-06-03 0:49 ` Clem Cole -- strict thread matches above, loose matches on Subject: below -- 2020-05-28 12:34 [TUHS] fmt(1): history, POSIX, -t, -c markus schnalke 2020-05-28 13:46 ` Clem Cole 2020-05-31 13:01 ` [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) markus schnalke 2020-05-31 14:53 ` Clem Cole 2020-05-31 16:25 ` Richard Salz 2020-05-31 16:50 ` Larry McVoy 2020-05-31 17:09 ` Ralph Corderoy 2020-05-31 17:25 ` Jon Steinhart
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).