From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mimir.eigenstate.org ([206.124.132.107]) by ewsd; Sun Sep 20 14:05:12 EDT 2020 Received: from abbatoir.fios-router.home (pool-74-101-2-6.nycmny.fios.verizon.net [74.101.2.6]) by mimir.eigenstate.org (OpenSMTPD) with ESMTPSA id 1c0b03e3 (TLSv1.2:ECDHE-RSA-AES256-SHA:256:NO) for <9front@9front.org>; Sun, 20 Sep 2020 11:05:03 -0700 (PDT) Message-ID: To: 9front@9front.org Subject: /mail/lib: cleanup proposal Date: Sun, 20 Sep 2020 11:05:01 -0700 From: ori@eigenstate.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit List-ID: <9front.9front.org> List-Help: X-Glyph: ➈ X-Bullshit: extensible compliant hypervisor shader polling-aware manager While it's fresh in my mind, I'm looking at /mail/lib. There are a lot of example files there, and it's a bit confusing what needs to be configured, when the scripts are invoked, what's intended to be configuration, and what's probably best to leave alone. What I'd like is to ship 9ront up with a minimal config that works out of the box for sending email through a remote SMTP server. Ideally, all we'd need would be setting a from address, and adding the keys in factotum. In an ideal world, I'd just have to do: echo 'ori@eigenstate.org' >> $home/lib/mail/sender I'd also like to take a minimal config for receiving email on @EXAMPLE.COM and put that into a separate directory, like: /mail/lib/example.direct For that, I need to understand what exactly belongs in a config, and what it does. So, to that end, here's my (likely wrong) understanding of what the files in /lib/mail do, and whether we actually need them. It seems like the shit in /mail/lib falls into five categories: - Config (and scripts that act as config) - Example config (which isn't labelled as such) - Bodies for /bin/upas/foo stubs - State for upas programs - Admin utilities not invoked by the mail system So, my current understanding of the files: blocked: optional; a list of ip addresses that are probably spammy. Horribly out of date, should probably not be in our default config. classify.re: useful for mail servers, but not human config, can we move to a place that makes it clear? /mail/state, or something? gone.fishing: optional: away message handling. would like to document, but not put into the example config. gone.msg: see above. ignore: needed, but why? a list of headers for upas/fs to filter out. I don't think any of our mail clients show the headers raw, seems like we can drop this file...? isspam.rc: spam.rc: unspam.rc: msgcat.rc: mandatory and not config. Exec'ed by /bin/upas/isspam; doesn't seem to be intended for edits by user. would like to the contents to /bin/upas/isspam, and maybe add an 'if(test -f /mail/lib/isspam.rc) exec...' for allowing customization. At the very least, would like to move out of /mail/lib to a place that makes it clear it's probably not for users to touch in most cases. /mail/rc? /mail/lib/rc? justqmail: optional? admin script? used by lazyqmail kickqueue: optional? admin util; do we need to have this in /lib/mail? mailnews: obsolete: seems to be a util for sending newsletters about plan 9 to a list of subscribers. namefiles: mandatory for smtpd: a list of alias lists; needed for smtpd config, not for local mailing. names.local: mandatory for smtpd: the alias list included by namefiles. patterns: optional. hardcoded spam regexes -- horribly out of date, probably not useful any more for fighting spam. If you want to get the same effect, you can probably do better with a pipeto; I'd like to delete support entirely. pipeto.bayes: optional: an example of how to send email to our bayesian spam filter. upas/bayes needs documentation. pipeto.lib: not config: a set of utility scripts that the .rc files include. Move to /mail/rc? prof.mbox: prof.spam: optional? not entirely sure what these are for. Seems to be some sort of state tracking for spam filtering? Anyone? If it's state tracking, see notes on classify.re qmail: optional? another admin tool? can't find anything that calls this directly in upas or /mail/lib. remotemail: mandatory: used for sending email. rewrite: mandatory: used for rerouting email. rewrite.direct: rewrite.gateway: example: copy to rewrite and edit setup.bayes: optional, stupid: sets up files that the bayesian filter uses. The bayesian filter should just do that itself if the files don't exist. Should fix bayes and delete this. smtpd.conf: mandatory (for smtp servers): exactly what it says on the tin, the smtp config. validateaddress: mandatory for smtpd: checks if the destination of a received email is valid for this system. maybe should turn it into an optional script, and instead teach smtpd to check if a user or alias exists directly. validateattachment: optional? mostly seems silly to have on by default. white.starter: example? an example of what whitelists look like, probably not needed. With that, it seems like we could clean things up so that /mail/lib defaults to containing: /mail/lib/rewrite.example /mail/lib/remotemail /mail/lib/ignore (todo, remove?) We can rewrite remotemail to default to routing based on factotum and the from address, rather than needing a bunch of config. Ideally, it'd be something like: # allow per-user overrides if(test -f $home/lib/mail/remotemail) exec $home/lib/mail/remotemail if(test -f $home/lib//mail/sender) from=`$nl{cat $home/lib/mail/sender # scrape server, username from factotum, route based # on that. user=`$nl{awk '/...server=.../{print user}' /mnt/factotum/ctl} user=`$nl{awk '/...server=.../{print server}' /mnt/factotum/ctl} I'm not sure how to get our domain into the rewrite rules, so until we figure that out it'll have to remain an example. For smtpd, it seems like we can have: /mail/lib/directmail.example/smtpd.conf /mail/lib/directmail.example/validateaddress /mail/lib/directmail.example/rewrite /mail/lib/directmail.example/namefiles /mail/lib/directmail.example/alias.local The rest can probably be put into other examples. /mail/lib/spam.example/... /mail/lib/fishing.example/... ...etc... The script stubs could either be inlined into /bin/upas/foo, or put into: /mail/rc/*.rc and the admin scripts into /mail/util/$script I think this would make it easier to figure out what's going on with upas configuration, and get a working system. Thoughts? Complaints? "Don't touch the artwork!"?