9front - general discussion about 9front
 help / color / mirror / Atom feed
* [9front] maildir history
@ 2021-02-06 11:10 Steve Simon
  2021-02-06 11:35 ` Kurt H Maier
  0 siblings, 1 reply; 13+ messages in thread
From: Steve Simon @ 2021-02-06 11:10 UTC (permalink / raw)
  To: 9front

Hi,

Anyone know the relationship between Erik Quanstro's maildir changes
from 9atom and the code on 9front.

Is the 9front an independant creation or based (evolved) from erik's code?

Also, honest question, how does the maildir changes help? I understand that
if you keep a large mailbox (hundreds of items) it will vastly reduce the load
on the server, but what are the other wins?

Thanks

-Steve

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 11:10 [9front] maildir history Steve Simon
@ 2021-02-06 11:35 ` Kurt H Maier
  2021-02-06 14:15   ` hiro
                     ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Kurt H Maier @ 2021-02-06 11:35 UTC (permalink / raw)
  To: 9front

On Sat, Feb 06, 2021 at 11:10:08AM +0000, Steve Simon wrote:
> Hi,
> 
> Anyone know the relationship between Erik Quanstro's maildir changes
> from 9atom and the code on 9front.

maildir and mdir are not the same -- erik's writing covers the main
differences. 
https://web.archive.org/web/20150611121250/http://www.quanstro.net/plan9/nupas.pdf

> Is the 9front an independant creation or based (evolved) from erik's code?

it's not unchanged from 9atom at this point but we merged his code
(nupas) in, entirely supplanting the labs upas system.

> Also, honest question, how does the maildir changes help? I understand that
> if you keep a large mailbox (hundreds of items) it will vastly reduce the load
> on the server, but what are the other wins?

any opportunity to reduce the number of mbox in the world should be
taken, as it's the worst file format.  but the biggest advantage is that
mdir doesn't spam the worm like mbox does.

khm

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 11:35 ` Kurt H Maier
@ 2021-02-06 14:15   ` hiro
  2021-02-06 14:38     ` hiro
  2021-02-06 15:06     ` Stanley Lieber
  2021-02-06 22:25   ` Steffen Nurpmeso
  2021-02-09 10:37   ` Steve Simon
  2 siblings, 2 replies; 13+ messages in thread
From: hiro @ 2021-02-06 14:15 UTC (permalink / raw)
  To: 9front

9front upas is all based on erik's nupas code.

afterwards it got modified a lot, so much that it might be a rewrite
would have been faster.

steve: not sure i get your specific question: are you asking if mdir
is better than mbox?
if yes, then i would agree that without more tooling around it,
without indexes around it, our mdir format is still not a very
accessible mail archive, but if you make your own indexes and some
tooling i think it's much easier based on mdir than on mbox where you
would mostly need to do byte-wise seek offsets as opposed to
file-granular read operations when you want to access a mail.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 14:15   ` hiro
@ 2021-02-06 14:38     ` hiro
  2021-02-06 15:06     ` Stanley Lieber
  1 sibling, 0 replies; 13+ messages in thread
From: hiro @ 2021-02-06 14:38 UTC (permalink / raw)
  To: 9front

can also check easily in the history, now that we have all the
overhead of real version control it's easy enough ;)

cpu% cd /sys/src/cmd/upas/
cpu% 	bind -ac /dist/plan9front /
cpu% hg log .
Not trusting file /.hg/hgrc from untrusted user glenda, group sys
Not trusting file /.hg/hgrc from untrusted user glenda, group sys
changeset:   8077:7c1a3a6b3252
user:        Alex Musolino <alex@musolino.id.au>
date:        Wed Nov 25 19:54:52 2020 +1030
summary:     upas: fix mk nuke

changeset:   8075:08ae111bfb23
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Nov 22 11:36:23 2020 -0800
summary:     upas/*: fix mkfile issues (thanks amavect)

changeset:   8069:ca80ec6a6ea3
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sat Nov 21 16:13:20 2020 -0800
summary:     upas: make 'all' the default target

changeset:   8053:a7cc619b0017
user:        Ori Bernstein <ori@eigenstate.org>
date:        Tue Nov 17 18:37:38 2020 -0800
summary:     upas/*: cleanup mkfiles (thanks amavect)

changeset:   8043:cbc842a5093b
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Nov 08 14:21:14 2020 -0800
summary:     imap4d: imap4date should ignore time and timezone (RFC3501, 6.4.4)

changeset:   8041:eef7bc730924
user:        Ori Bernstein <ori@eigenstate.org>
date:        Fri Nov 06 18:15:15 2020 -0800
summary:     upas/marshal: add -S saveto to save outgoing mail, fix -F

changeset:   8035:50ad211fb12f
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Nov 02 13:12:34 2020 -0800
summary:     upas/common: delete library on 'mk clean'

changeset:   8016:3bbbf995d422
user:        khm
date:        Thu Oct 22 13:46:00 2020 -0700
summary:     upas/fs/mbox.c:  fix cosmetic typo

changeset:   8005:8e68b66a3b8d
parent:      7998:e05e4b6c6546
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sat Oct 17 18:59:36 2020 -0700
summary:     upas: fix appendfolder timestamps (thanks umbraticus)

changeset:   7997:44eae9a7e98b
user:        cinap_lenrek@felloff.net
date:        Sun Oct 11 14:59:49 2020 +0200
summary:     upas/fs: fix truncation of plumb date

changeset:   7985:35c4d11d3692
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sat Sep 26 11:20:42 2020 -0700
summary:     upas: convert to tmdate, change timezone format

changeset:   7972:730a2ee35853
user:        Ori Bernstein <ori@eigenstate.org>
date:        Thu Sep 17 16:11:10 2020 -0700
summary:     upas/fs: fix handing of multi-line header fields (thanks theinicke)

changeset:   7952:bd2247115785
user:        cinap_lenrek@felloff.net
date:        Tue Sep 01 23:08:58 2020 +0200
summary:     upas: appendfolder() don't be picky about the timezone of ctime()

changeset:   7951:b2f4c4f31a19
user:        cinap_lenrek@felloff.net
date:        Tue Sep 01 23:01:45 2020 +0200
summary:     upas: appendfolder(): skip the address from unix header before date

changeset:   7950:acfe59170e4e
user:        cinap_lenrek@felloff.net
date:        Tue Sep 01 21:39:45 2020 +0200
summary:     upas/fs: extract proper date from unix header

changeset:   7949:969b9a225c4a
user:        Alex Musolino <alex@musolino.id.au>
date:        Tue Sep 01 22:25:06 2020 +0930
summary:     upas/fs: remove unused function date822tounix

changeset:   7920:0560b1c36eba
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Aug 09 19:46:38 2020 -0700
summary:     upas/fs: port date parsing to libc apis

changeset:   7896:75e23c8335ab
user:        kvik <kvik@a-b.xyz>
date:        Sat Aug 01 15:27:28 2020 +0200
summary:     pre-lib9p servers: fix incorrect Tversion handling

changeset:   7887:bff7007ed367
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Jul 26 19:12:48 2020 -0700
summary:     upasfs: make imap debug logging less noisy

changeset:   7873:7bd3f3b9dc76
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 22:15:02 2020 +0200
summary:     imap4d: fix missing return in %δ format of Dfmt()

changeset:   7872:134d12a3e3f8
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 21:18:26 2020 +0200
summary:     imap4d: get rid of unixdate/unixfrom handling

changeset:   7871:476362d1a4b3
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 21:16:41 2020 +0200
summary:     imap4d: fix mkfile to have the correct default target

changeset:   7870:69d58526946b
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 21:15:55 2020 +0200
summary:     upas/fs: wait until the index becomes unlocked

changeset:   7869:93802f4f66ef
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 19:51:59 2020 +0200
summary:     upas/fs: fix wrong nparts field index (changed in previous commit)

changeset:   7868:3726ca81f61c
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 18:17:20 2020 +0200
summary:     nedmail: don't try to shoot down subcommand on interrupt

changeset:   7867:bd1c93238019
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 17:44:32 2020 +0200
summary:     upas/fs: put date822 into the index, fix from and replyto handling

changeset:   7866:625f20e679ca
user:        cinap_lenrek@felloff.net
date:        Sun Jul 05 17:05:49 2020 +0200
summary:     upas/fs: fix memory leak in ref822()

changeset:   7865:7c44146d4f0f
user:        Ori Bernstein <ori@eigenstate.org>
date:        Fri Jul 03 11:29:45 2020 -0700
summary:     imap4d: respect errors from read(), remove debug prints

changeset:   7859:d6eda6ce40f2
parent:      7857:d7ed7cf2ea4c
user:        cinap_lenrek@felloff.net
date:        Mon Jun 29 00:23:37 2020 +0200
summary:     upas/fs: move unixheader handling to parseheaders()

changeset:   7854:3b340d09ab2f
user:        cinap_lenrek@felloff.net
date:        Wed Jun 24 19:21:47 2020 +0200
summary:     upas/fs: remove now unused Mtrunc mimeflags constant

changeset:   7853:bc90a5dd69f3
user:        cinap_lenrek@felloff.net
date:        Wed Jun 24 19:18:37 2020 +0200
summary:     upas/fs: use memchr() instead of strchr() in hdrlen()

changeset:   7849:8e47ca4955f0
user:        Alex Musolino <alex@musolino.id.au>
date:        Mon Jun 22 16:34:31 2020 +0930
summary:     upas/marshal: make attachment failure fatal

changeset:   7785:d45cfe9072cd
user:        cinap_lenrek@felloff.net
date:        Sat May 30 13:03:05 2020 +0200
summary:     imap4d: reject invalid month in date2tm()

changeset:   7740:0ee17db5d86b
user:        cinap_lenrek@felloff.net
date:        Sat May 02 17:32:01 2020 +0200
summary:     make bind(2) error handling consistent

changeset:   7640:5bd143853139
parent:      7634:f8fd9ee6ec75
user:        Alex Musolino <alex@musolino.id.au>
date:        Wed Apr 01 22:49:19 2020 +1030
summary:     upas/fs: fix sending of "delete" plumb messages

changeset:   7556:98aedbd462ee
user:        Ori Bernstein <ori@eigenstate.org>
date:        Wed Feb 05 14:11:15 2020 -0800
summary:     upas/fs plumb modify messages for self-changed flags

changeset:   7503:f1523de908ce
user:        Ori Bernstein <ori@eigenstate.org>
date:        Tue Dec 10 23:13:25 2019 -0800
summary:     remove unused code.

changeset:   7502:85e122fecb8d
user:        Ori Bernstein <ori@eigenstate.org>
date:        Tue Dec 10 23:01:06 2019 -0800
summary:     only ensurecache() on doplumb().

changeset:   7500:2cc069392228
parent:      7498:23da1e1ca63a
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Dec 09 12:46:27 2019 -0800
summary:     upas/fs plumb flag changes.

changeset:   7492:3a34c309d6ec
parent:      7489:04f685ebb1b7
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Dec 08 11:54:59 2019 -0800
summary:     fix filetype detecton by suffix so that multiple dots
dont confuse it. (thanks kvik)

changeset:   7486:77938288732a
user:        Ori Bernstein <ori@eigenstate.org>
date:        Thu Dec 05 00:16:15 2019 -0800
summary:     delete obsolete comments (replies are flagged elsewhere)

changeset:   7474:aca41046f6ee
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Dec 02 14:50:53 2019 -0800
summary:     Remove reply print.

changeset:   7471:6a728bb0f4e0
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Dec 02 13:56:15 2019 -0800
summary:     remove debug print

changeset:   7470:0a8a0a41a808
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Dec 02 13:53:57 2019 -0800
summary:     simplify flag parsing.

changeset:   7464:6d87e952f4e5
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Dec 01 17:24:02 2019 -0800
summary:     fix typo: we don't have cache insurance.

changeset:   7463:c6d4c49b1653
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Dec 01 17:14:13 2019 -0800
summary:     show and update flags in acme mail

changeset:   7462:99a26b67689a
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sun Dec 01 17:12:19 2019 -0800
summary:     upas/fs imap fixes and improvements

changeset:   7457:5ff983aedf0b
user:        cinap_lenrek@felloff.net
date:        Sun Nov 24 03:46:53 2019 +0100
summary:     upas/fs: remove useless loop in rf822()

changeset:   7456:db1f385e3b27
user:        Ori Bernstein <ori@eigenstate.org>
date:        Sat Nov 23 08:23:21 2019 -0800
summary:     fix ref822 again: remove uniqarray(), fix case with many
entries in 'n'.

changeset:   7455:8b2040ba4785
user:        Alex Musolino <alex@musolino.id.au>
date:        Fri Nov 22 17:29:35 2019 +1030
summary:     upas/marshal: fix printinreplyto function

changeset:   7454:a27566709691
user:        Ori Bernstein <ori@eigenstate.org>
date:        Thu Nov 21 17:17:54 2019 -0800
summary:     fix ref882 reference parsing.

changeset:   7453:610084672873
user:        Ori Bernstein <ori@eigenstate.org>
date:        Thu Nov 21 10:48:13 2019 -0800
summary:     show urls in html messages.

changeset:   7422:714b929020e9
user:        Alex Musolino <alex@musolino.id.au>
date:        Thu Oct 31 09:41:03 2019 +1030
summary:     upas/fs: fix handling of numeric timezone offsets in strtotm

changeset:   7421:fe8aefb6ab34
user:        Ori Bernstein <ori@eigenstate.org>
date:        Mon Oct 28 14:12:44 2019 -0700
summary:     Check if 'm' is null when updating messages.

changeset:   7413:9c36e8f222fb
user:        Ori Bernstein <ori@eigenstate.org>
date:        Thu Oct 10 11:52:22 2019 -0700
summary:     Fetch IMAP flags from server. This makes us sync
read/answered/... flags with unix.

changeset:   7405:82225c0fc1a2
user:        cinap_lenrek@felloff.net
date:        Thu Oct 03 15:49:53 2019 +0200
summary:     upas/fs: speedup mtree and henter()

changeset:   7396:8eff72134197
user:        cinap_lenrek@felloff.net
date:        Sat Sep 21 23:36:44 2019 +0200
summary:     cmd/ip/*: chown the network connection after authentication

changeset:   7288:765db6013c41
user:        cinap_lenrek@felloff.net
date:        Thu Jun 20 12:13:51 2019 +0200
summary:     upas/smtp: handle temporary authentication failures

changeset:   7251:c67d805b0614
user:        Alex Musolino <alex@musolino.id.au>
date:        Mon May 20 15:00:14 2019 +0930
summary:     upas/fs: remove read timeout via alarm(2) in pop3resp

changeset:   7098:a55cb6124fca
user:        cinap_lenrek@felloff.net
date:        Sat Mar 23 05:47:17 2019 +0100
summary:     upas/fs: release mailbox syncing lock after encountering error

changeset:   7044:5f04b19f5ad5
user:        cinap_lenrek@felloff.net
date:        Mon Feb 25 03:39:31 2019 +0100
summary:     upas/fs: fix %.*s format usage

changeset:   7043:ee85e5437edf
user:        cinap_lenrek@felloff.net
date:        Mon Feb 25 03:34:25 2019 +0100
summary:     upas/ned: fix %.*s format in parsesearch(), improve mkfile

changeset:   7042:4564c92bb864
user:        cinap_lenrek@felloff.net
date:        Mon Feb 25 03:31:50 2019 +0100
summary:     upas/scanmail: fix %.*s format in xprint(), improve mkfile

changeset:   7041:a80696e30622
user:        cinap_lenrek@felloff.net
date:        Mon Feb 25 03:29:19 2019 +0100
summary:     upas/smtp: fix %.*s format for challenge in smtpcram()

changeset:   7025:1eb3c24dbd05
user:        cinap_lenrek@felloff.net
date:        Mon Feb 11 23:40:34 2019 +0100
summary:     upas/smtpd: implement ipv6 support for ip blacklist,
replace v4parsecidr() with parseipandmask()

changeset:   6997:0bffbf9207f3
user:        Alex Musolino <alex@musolino.id.au>
date:        Tue Jan 22 15:11:53 2019 +1030
summary:     upas/fs: remove unused function pop3log

changeset:   6985:8c50688cb280
user:        cinap_lenrek@felloff.net
date:        Wed Jan 02 10:26:38 2019 +0100
summary:     upas/fs: don't put messages on the lru that cannot be uncached

changeset:   6984:90b5dee2d4a4
user:        cinap_lenrek@felloff.net
date:        Wed Jan 02 10:22:27 2019 +0100
summary:     upas/fs: include mkupas in mkfile last to get default target

changeset:   6983:b861e3d8e3d2
user:        Alex Musolino <alex@musolino.id.au>
date:        Mon Dec 31 00:00:09 2018 +1030
summary:     upas/fs: fix infinite loop in putcache (again)

changeset:   6962:24e68244d610
parent:      6961:00ae79a6ba50
parent:      6960:adbb3b4685d4
user:        Alex Musolino <alex@musolino.id.au>
date:        Thu Dec 13 12:00:32 2018 +1030
summary:     merge

changeset:   6961:00ae79a6ba50
parent:      6959:d4064946b257
user:        Alex Musolino <alex@musolino.id.au>
date:        Thu Dec 13 11:59:34 2018 +1030
summary:     upas/fs: remove now unnecessary "force" argument to cachefree

changeset:   6960:adbb3b4685d4
user:        cinap_lenrek@felloff.net
date:        Wed Dec 12 19:33:08 2018 +0100
summary:     upas/fs: handle errors from fetch for cachebody()/cacheheaders()

changeset:   6958:b32199e0f90a
parent:      6951:582967287638
user:        Alex Musolino <alex@musolino.id.au>
date:        Wed Dec 12 12:48:53 2018 +1030
summary:     upas/fs: fix infinite loop in putcache

changeset:   6921:6bdbb3f0939d
user:        cinap_lenrek@felloff.net
date:        Wed Nov 28 21:12:11 2018 +0100
summary:     upas: get rid of broken validatesender example, remove
redundant upas/misc and setup target

changeset:   6917:0b5e650cc3de
user:        cinap_lenrek@felloff.net
date:        Mon Nov 26 21:36:51 2018 +0100
summary:     upas/fs: avoid switch on vlong breaking build (thanks k0ga)

changeset:   6916:f6914965edf5
user:        cinap_lenrek@felloff.net
date:        Mon Nov 26 21:29:33 2018 +0100
summary:     upas/fs: another mistake ...

changeset:   6915:4be7385a9bf4
user:        cinap_lenrek@felloff.net
date:        Mon Nov 26 20:15:30 2018 +0100
summary:     upas/fs: fix broken body.ext in walk (thanks sl)

changeset:   6907:f0aabf0313a7
user:        cinap_lenrek@felloff.net
date:        Thu Nov 22 08:55:13 2018 +0100
summary:     upas/fs: keep messages *marked* for deletion in the index

changeset:   6906:bbc1c90e6f8f
user:        cinap_lenrek@felloff.net
date:        Thu Nov 22 08:49:41 2018 +0100
summary:     upas/fs: allow msgincref() with mb->refs == 0

changeset:   6905:a0cf603ccd3f
user:        cinap_lenrek@felloff.net
date:        Wed Nov 21 21:48:55 2018 +0100
summary:     upas/fs: checkfid() might be called with uninitialized fid, ignore

changeset:   6904:ef814b99942a
user:        cinap_lenrek@felloff.net
date:        Wed Nov 21 17:32:36 2018 +0100
summary:     upas/fs: fix walk

changeset:   6903:22cdf0573f9b
user:        cinap_lenrek@felloff.net
date:        Wed Nov 21 16:19:51 2018 +0100
summary:     upas/fs: don't enter mailbox into hashtree when it is being freed

changeset:   6900:fd0c1315d6ef
user:        cinap_lenrek@felloff.net
date:        Wed Nov 21 00:37:35 2018 +0100
summary:     upas/fs: rework the refcounting, use 64bit qid path, add checks

changeset:   6884:be96aba40574
user:        cinap_lenrek@felloff.net
date:        Mon Nov 12 20:43:25 2018 +0100
summary:     upa/imap4d: ignore messages without digest info

changeset:   6835:0b8c924d2d71
user:        BurnZeZ
date:        Sat Oct 20 18:44:09 2018 +0000
summary:     fix misleading/wrong fd checks

changeset:   6826:4160a02cb25d
user:        mischief <mischief@offblast.org>
date:        Thu Oct 18 08:46:44 2018 -0700
summary:     upas/fs: delete extra binaries

changeset:   6619:df98fe390855
user:        Alex Musolino <alex@musolino.id.au>
date:        Sun Jul 08 12:18:06 2018 +0930
summary:     upas/smtp: use current login id in absence of -u option
(as per man page)

changeset:   6618:e33e7f04e316
user:        Alex Musolino <alex@musolino.id.au>
date:        Sun Jul 08 12:05:42 2018 +0930
summary:     upas/smtp: allow remote port to be set to something other
than smtp (e.g. ssmtp)

changeset:   6577:60aec31a4d49
user:        Alex Musolino <alex@musolino.id.au>
date:        Tue Jun 12 20:04:16 2018 +0930
summary:     upas/nedmail: marshal(1) options -n and -8 are mutually exclusive

changeset:   6353:0b97536e2bcd
user:        cinap_lenrek@felloff.net
date:        Mon Feb 05 19:16:29 2018 +0100
summary:     upas/fs: fix imap atom quoting (thanks Piotr Kubaj)

changeset:   6340:70461aaaa9fe
user:        cinap_lenrek@felloff.net
date:        Sat Jan 27 19:02:31 2018 +0100
summary:     upas/fs: try to deal with nil mail body (can happen when
fetch fails)

changeset:   6339:897d87b868aa
user:        cinap_lenrek@felloff.net
date:        Thu Jan 25 02:14:51 2018 +0100
summary:     upas/fs: deal with imap returning more uid's than
allocated from previus "messages" command

changeset:   6267:ce5613e709c7
user:        cinap_lenrek@felloff.net
date:        Tue Dec 19 20:57:24 2017 +0100
summary:     upas/fs: more bugs

changeset:   6266:b2b72544dd69
user:        cinap_lenrek@felloff.net
date:        Tue Dec 19 20:44:15 2017 +0100
summary:     upas/fs: fix precedence bugs, compare digest pointer to nil

changeset:   6192:d1d26e691894
user:        cinap_lenrek@felloff.net
date:        Mon Oct 23 06:08:18 2017 +0200
summary:     upas/smtpd: don't call syslog() from the note handler,
this can deadlock

changeset:   6170:537b93c10ad5
user:        cinap_lenrek@felloff.net
date:        Sat Sep 23 17:33:05 2017 +0200
summary:     upas/fs: fix putcache(), sub-part messages should never
go into the lru

changeset:   6167:ead71ae69a9a
user:        cinap_lenrek@felloff.net
date:        Wed Sep 13 23:24:10 2017 +0200
summary:     upas/fs: replace fixed cache table with lru linked list

changeset:   6066:eddb055f0c15
user:        cinap_lenrek@felloff.net
date:        Thu Jun 22 20:46:04 2017 +0200
summary:     upas/fs: handle plumbing for new messages for concurrent
index updates

changeset:   6049:7d9c3867e36d
user:        cinap_lenrek@felloff.net
date:        Sun Jun 18 18:52:54 2017 +0200
summary:     upas/fs: work in progress...

changeset:   6006:bb15fa50e71a
user:        cinap_lenrek@felloff.net
date:        Wed Jun 07 07:38:10 2017 +0200
summary:     upas: fix infinite loop in common/mkfile (thanks raiz)

changeset:   5967:36315eb9647c
user:        cinap_lenrek@felloff.net
date:        Tue May 02 04:07:21 2017 +0200
summary:     upas/fs: handle partial fetches (imap server returning
less messages than available)

changeset:   5938:186958754877
user:        cinap_lenrek@felloff.net
date:        Thu Apr 27 20:06:00 2017 +0200
summary:     upas/fs: fix warning about unused Err: label

changeset:   5928:c5d492742770
user:        cinap_lenrek@felloff.net
date:        Sun Apr 23 19:00:08 2017 +0200
summary:     libsec: sha256 support for thumbprint files, use it in ssh as well

changeset:   5927:c912f0408c34
user:        cinap_lenrek@felloff.net
date:        Sun Apr 23 18:42:26 2017 +0200
summary:     upas/fs: don't abort on malformed unix header

changeset:   5916:d0fb38041ac1
parent:      5914:5e89ce097040
user:        spew
date:        Sat Apr 22 13:59:37 2017 -0500
summary:     libavl: lookup can return the closest match

changeset:   5881:651cff8fa868
user:        cinap_lenrek@felloff.net
date:        Sun Apr 09 04:36:47 2017 +0200
summary:     upas/fs: remove imap lastread debounding

changeset:   5880:16a9b7551d52
user:        cinap_lenrek@felloff.net
date:        Sun Apr 09 04:32:38 2017 +0200
summary:     upas/fs: fix sync condition

changeset:   5879:08e3ed454443
user:        cinap_lenrek@felloff.net
date:        Sat Apr 08 23:01:19 2017 +0200
summary:     upas/fs: make use of Maxmsg consistent

changeset:   5878:ccca6b8de9d4
user:        cinap_lenrek@felloff.net
date:        Sat Apr 08 23:00:48 2017 +0200
summary:     upas/fs: simplify imap4read()

changeset:   5877:9a3556ee638d
user:        cinap_lenrek@felloff.net
date:        Sat Apr 08 22:59:15 2017 +0200
summary:     upas/fs: fix dir comparsion and skip directories in mdir,
avoid stat

changeset:   5876:84e830aa7a74
user:        cinap_lenrek@felloff.net
date:        Sat Apr 08 22:52:06 2017 +0200
summary:     upas/send: use openfolder() to deliver mail to avoid code
duplication

changeset:   5875:fb490ac6e677
user:        cinap_lenrek@felloff.net
date:        Sat Apr 08 22:50:17 2017 +0200
summary:     upas/common: deliver mail to mdir as .tmp file and rename
after it has been fully written

changeset:   5828:66e9a9215bed
user:        cinap_lenrek@felloff.net
date:        Thu Mar 23 01:31:35 2017 +0100
summary:     upas/smtp: fix cram-md5 auth, simplify doauth(), check
varargs for dBprint()

changeset:   5827:041a4ca9640b
user:        cinap_lenrek@felloff.net
date:        Thu Mar 23 00:36:57 2017 +0100
summary:     upas/smtp: generate 128-bit random message id (was 32
bit), use dBprint(), cleanup unused variables

changeset:   5824:732beb9d498c
user:        spew
date:        Wed Mar 22 17:10:35 2017 -0500
summary:     upas/smtp: Revert smtp dial string behavior to match that
of old upas (thanks sam-d)

changeset:   5822:9b7fd9b3a9d3
user:        cinap_lenrek@felloff.net
date:        Tue Mar 21 20:31:15 2017 +0100
summary:     remove accidentally commited binary
/sys/src/cmd/upas/fs/chkidx file (thanks mischief)

changeset:   5808:223404c8343e
user:        cinap_lenrek@felloff.net
date:        Thu Mar 16 22:06:31 2017 +0100
summary:     upas/fs: tls sni support for pop3/imap

changeset:   5807:46235b1e855c
user:        cinap_lenrek@felloff.net
date:        Thu Mar 16 00:05:08 2017 +0100
summary:     upas/fs: fix memory leaks in tls code, handle tls in a
common wraptls() function

changeset:   5806:778cbffa5b69
user:        cinap_lenrek@felloff.net
date:        Wed Mar 15 19:30:33 2017 +0100
summary:     upas/fs: getting rid of the fine-grain locking...

changeset:   5805:a6a7ca49c2e7
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 23:46:35 2017 +0100
summary:     upas/fs: more cleanup...

changeset:   5804:a6ee28d812be
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 23:31:10 2017 +0100
summary:     upas/fs: remove more debugging crap

changeset:   5803:737ef7fca344
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 23:21:38 2017 +0100
summary:     upas/fs: remove unused segbrk memory allocator

changeset:   5802:f82cdded4a18
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 23:19:17 2017 +0100
summary:     upas/fs: remove idx semaphore, cannot happen as
symcmbox() always called with mailbox locked

changeset:   5801:690af8ac4c64
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 23:06:39 2017 +0100
summary:     upas/fs: fix more locking bugs, remove debugging clutter,
remove planb mbox code

changeset:   5800:dc15d7af36fa
user:        cinap_lenrek@felloff.net
date:        Tue Mar 14 19:32:20 2017 +0100
summary:     upas/fs: make sure mbox is locked during msgdecref() in rclunk()

changeset:   5793:7d6755e932a5
user:        cinap_lenrek@felloff.net
date:        Sun Mar 12 21:29:28 2017 +0100
summary:     upas/fs: lock mbox in syncallmboxes() to avoid assert crash

changeset:   5792:c397b7e89847
user:        cinap_lenrek@felloff.net
date:        Sun Mar 12 21:23:21 2017 +0100
summary:     upas/fs: remove useless syncsem, make sure mbox is locked
when fetching / syncing

changeset:   5791:535fa0436664
user:        cinap_lenrek@felloff.net
date:        Sun Mar 12 18:50:27 2017 +0100
summary:     imap4d: simplify auth with encodefmt (backport from 9front)

changeset:   5790:7d9d0f3f113c
user:        aiju
date:        Sun Mar 12 17:19:20 2017 +0000
summary:     nupas: delete okcidr(); always trust spf records

changeset:   5787:d238391ca3fd
parent:      5786:44d718112eaa
parent:      5785:005608e70320
user:        cinap_lenrek@felloff.net
date:        Sun Mar 12 17:20:13 2017 +0100
summary:     merge

changeset:   5785:005608e70320
parent:      5783:42edc103c688
user:        cinap_lenrek@felloff.net
date:        Sun Mar 12 17:15:03 2017 +0100
summary:     merging erik quanstros nupas

...

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 14:15   ` hiro
  2021-02-06 14:38     ` hiro
@ 2021-02-06 15:06     ` Stanley Lieber
  2021-02-06 15:18       ` hiro
  1 sibling, 1 reply; 13+ messages in thread
From: Stanley Lieber @ 2021-02-06 15:06 UTC (permalink / raw)
  To: 9front

On February 6, 2021 9:15:26 AM EST, hiro <23hiro@gmail.com> wrote:
>9front upas is all based on erik's nupas code.
>
>afterwards it got modified a lot, so much that it might be a rewrite
>would have been faster.
>
>steve: not sure i get your specific question: are you asking if mdir
>is better than mbox?
>if yes, then i would agree that without more tooling around it,
>without indexes around it, our mdir format is still not a very
>accessible mail archive, but if you make your own indexes and some
>tooling i think it's much easier based on mdir than on mbox where you
>would mostly need to do byte-wise seek offsets as opposed to
>file-granular read operations when you want to access a mail.
>

read the mail chapter in UNIX-HATERS, and read about why maildir was created. all of these problems are as old as e-mail.

sl

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 15:06     ` Stanley Lieber
@ 2021-02-06 15:18       ` hiro
  2021-02-06 21:36         ` Kurt H Maier
  0 siblings, 1 reply; 13+ messages in thread
From: hiro @ 2021-02-06 15:18 UTC (permalink / raw)
  To: 9front

which part of the mail chapter exactly?
i grepped for maildir also, couldnt find it. neither could i find any
mention of mbox.

On 2/6/21, Stanley Lieber <sl@stanleylieber.com> wrote:
> On February 6, 2021 9:15:26 AM EST, hiro <23hiro@gmail.com> wrote:
>>9front upas is all based on erik's nupas code.
>>
>>afterwards it got modified a lot, so much that it might be a rewrite
>>would have been faster.
>>
>>steve: not sure i get your specific question: are you asking if mdir
>>is better than mbox?
>>if yes, then i would agree that without more tooling around it,
>>without indexes around it, our mdir format is still not a very
>>accessible mail archive, but if you make your own indexes and some
>>tooling i think it's much easier based on mdir than on mbox where you
>>would mostly need to do byte-wise seek offsets as opposed to
>>file-granular read operations when you want to access a mail.
>>
>
> read the mail chapter in UNIX-HATERS, and read about why maildir was
> created. all of these problems are as old as e-mail.
>
> sl
>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 15:18       ` hiro
@ 2021-02-06 21:36         ` Kurt H Maier
  0 siblings, 0 replies; 13+ messages in thread
From: Kurt H Maier @ 2021-02-06 21:36 UTC (permalink / raw)
  To: 9front

On Sat, Feb 06, 2021 at 04:18:22PM +0100, hiro wrote:
> which part of the mail chapter exactly?

Page 79 is probably the most relevant, as it talks about sendmail
mangling messages where more than one line matches /^From:/

This problem still exists in mbox, because according to the
specification you identify the start of a new message by detecting
/\n^From:/

 From there you have a problem.

maildir isn't in Unix-Haters; more info here[1] and in the maildir(5)
man page on your nearest SunOS box.

1 - https://cr.yp.to/proto/maildir.html

khm

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 11:35 ` Kurt H Maier
  2021-02-06 14:15   ` hiro
@ 2021-02-06 22:25   ` Steffen Nurpmeso
  2021-02-06 22:46     ` Kurt H Maier
  2021-02-09 10:37   ` Steve Simon
  2 siblings, 1 reply; 13+ messages in thread
From: Steffen Nurpmeso @ 2021-02-06 22:25 UTC (permalink / raw)
  To: 9front

Kurt H Maier wrote in
 <YB5/Dyl/YlLwQpIg@wopr>:
 |On Sat, Feb 06, 2021 at 11:10:08AM +0000, Steve Simon wrote:
 |> Hi,
 |> 
 |> Anyone know the relationship between Erik Quanstro's maildir changes
 |> from 9atom and the code on 9front.
 |
 |maildir and mdir are not the same -- erik's writing covers the main
 |differences. 
 |https://web.archive.org/web/20150611121250/http://www.quanstro.net/plan9\
 |/nupas.pdf
 |
 |> Is the 9front an independant creation or based (evolved) from erik's \
 |> code?
 |
 |it's not unchanged from 9atom at this point but we merged his code
 |(nupas) in, entirely supplanting the labs upas system.
 |
 |> Also, honest question, how does the maildir changes help? I understand \
 |> that
 |> if you keep a large mailbox (hundreds of items) it will vastly reduce \
 |> the load
 |> on the server, but what are the other wins?
 |
 |any opportunity to reduce the number of mbox in the world should be
 |taken, as it's the worst file format.  but the biggest advantage is that

With MIME encoded mails it is not, just a flat text-based database
of variable length entries.  If all your software agrees in
storing properly MIME encoded data, all is fine.
If not you have the ^From_ trouble.

 |mdir doesn't spam the worm like mbox does.

Which means re-encoding as necessary to avoid ^From_ quoting,
which not many do support (my s-nail for example does not :).
All the ASCII centrism .. was trouble.
It was the thorn .. in his mutter side.
She tried in vain .. but it never caused her nothing but shame.

 |khm
 --End of <YB5/Dyl/YlLwQpIg@wopr>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 22:25   ` Steffen Nurpmeso
@ 2021-02-06 22:46     ` Kurt H Maier
  2021-02-07  0:02       ` Steffen Nurpmeso
  0 siblings, 1 reply; 13+ messages in thread
From: Kurt H Maier @ 2021-02-06 22:46 UTC (permalink / raw)
  To: 9front

On Sat, Feb 06, 2021 at 11:25:12PM +0100, Steffen Nurpmeso wrote:
> Which means re-encoding as necessary to avoid ^From_ quoting,
> which not many do support (my s-nail for example does not :).

No, on Plan 9 it still sucks, because you're writing a whole new version
of the file every single time you add or delete a message from an mbox
store.  This has consequences for big churny mailstores, and they fill
up disks fast.  mbox is better with MIME encoding but it still
irreparably sucks as a storage format even when you ignore parsing
problems.  I hate it and I want it to die.

khm

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 22:46     ` Kurt H Maier
@ 2021-02-07  0:02       ` Steffen Nurpmeso
  2021-02-07  0:20         ` Kurt H Maier
  2021-02-07  0:21         ` hiro
  0 siblings, 2 replies; 13+ messages in thread
From: Steffen Nurpmeso @ 2021-02-07  0:02 UTC (permalink / raw)
  To: 9front

Kurt H Maier wrote in
 <YB8cQCDKe/3SDEka@wopr>:
 |On Sat, Feb 06, 2021 at 11:25:12PM +0100, Steffen Nurpmeso wrote:
 |> Which means re-encoding as necessary to avoid ^From_ quoting,
 |> which not many do support (my s-nail for example does not :).
 |
 |No, on Plan 9 it still sucks, because you're writing a whole new version
 |of the file every single time you add or delete a message from an mbox
 |store.  This has consequences for big churny mailstores, and they fill

That can be optimized.  For POSIX mailx with "append" set, and if
i recall correctly, my s-nail correctly extends this via
appending.  Removal, ok.

 |up disks fast.  mbox is better with MIME encoding but it still
 |irreparably sucks as a storage format even when you ignore parsing
 |problems.  I hate it and I want it to die.

I seem to recall even dovecot was going away from it, though if
you control a file and have an external info aka index file or
structure, just like git has for its database, no modifications
actually occur but setting a flag or two, with occasional cleanup
by writing a new version top down.  Well.  I once read on
a FreeBSD list about an administrator saying "my boss as a mailbox
with millions of files, it is no fun.  We use archeopterix to
index it" (at least in spirit, been a few years).  Like so maildir
is a problem, too.  Need a db with index to make that acceptable,
wasn't it Zawinski who talked in the 90s, for Netscape....

  https://www.jwz.org/doc/mailsum.html

There you go, 1998:

  Right now I'm looking at a folder in 3.0. It has 15,466 messages
  in it. Selecting this folder takes less than a second (it's hard
  to eyeball it, but I'd say it takes about 1/2 to 3/4 second from
  when I click to when I see the message summary on the
  screen). The BSD mbox file is 57.2MB (1.2 million lines) and the
  summary file is 1.3MB (2% of the size of the folder.)

  This is on a P266 with a local IDE disk (Linux.) 

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-07  0:02       ` Steffen Nurpmeso
@ 2021-02-07  0:20         ` Kurt H Maier
  2021-02-07  0:21         ` hiro
  1 sibling, 0 replies; 13+ messages in thread
From: Kurt H Maier @ 2021-02-07  0:20 UTC (permalink / raw)
  To: 9front

Yes, mdir has an index file.  It's described in the paper I linked about
nupas.

Again:  mdir and maildir are not the same

khm

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-07  0:02       ` Steffen Nurpmeso
  2021-02-07  0:20         ` Kurt H Maier
@ 2021-02-07  0:21         ` hiro
  1 sibling, 0 replies; 13+ messages in thread
From: hiro @ 2021-02-07  0:21 UTC (permalink / raw)
  To: 9front

or we just store mail in the cloud, then we will not be allowed to
make an index, we will not be allowed to seek and we need one GET
request and 9 captchas for accessing one single mail.

what is so off-putting about ordering data in files?

On 2/7/21, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
> Kurt H Maier wrote in
>  <YB8cQCDKe/3SDEka@wopr>:
>  |On Sat, Feb 06, 2021 at 11:25:12PM +0100, Steffen Nurpmeso wrote:
>  |> Which means re-encoding as necessary to avoid ^From_ quoting,
>  |> which not many do support (my s-nail for example does not :).
>  |
>  |No, on Plan 9 it still sucks, because you're writing a whole new version
>  |of the file every single time you add or delete a message from an mbox
>  |store.  This has consequences for big churny mailstores, and they fill
>
> That can be optimized.  For POSIX mailx with "append" set, and if
> i recall correctly, my s-nail correctly extends this via
> appending.  Removal, ok.
>
>  |up disks fast.  mbox is better with MIME encoding but it still
>  |irreparably sucks as a storage format even when you ignore parsing
>  |problems.  I hate it and I want it to die.
>
> I seem to recall even dovecot was going away from it, though if
> you control a file and have an external info aka index file or
> structure, just like git has for its database, no modifications
> actually occur but setting a flag or two, with occasional cleanup
> by writing a new version top down.  Well.  I once read on
> a FreeBSD list about an administrator saying "my boss as a mailbox
> with millions of files, it is no fun.  We use archeopterix to
> index it" (at least in spirit, been a few years).  Like so maildir
> is a problem, too.  Need a db with index to make that acceptable,
> wasn't it Zawinski who talked in the 90s, for Netscape....
>
>   https://www.jwz.org/doc/mailsum.html
>
> There you go, 1998:
>
>   Right now I'm looking at a folder in 3.0. It has 15,466 messages
>   in it. Selecting this folder takes less than a second (it's hard
>   to eyeball it, but I'd say it takes about 1/2 to 3/4 second from
>   when I click to when I see the message summary on the
>   screen). The BSD mbox file is 57.2MB (1.2 million lines) and the
>   summary file is 1.3MB (2% of the size of the folder.)
>
>   This is on a P266 with a local IDE disk (Linux.)
>
> --steffen
> |
> |Der Kragenbaer,                The moon bear,
> |der holt sich munter           he cheerfully and one by one
> |einen nach dem anderen runter  wa.ks himself off
> |(By Robert Gernhardt)
>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9front] maildir history
  2021-02-06 11:35 ` Kurt H Maier
  2021-02-06 14:15   ` hiro
  2021-02-06 22:25   ` Steffen Nurpmeso
@ 2021-02-09 10:37   ` Steve Simon
  2 siblings, 0 replies; 13+ messages in thread
From: Steve Simon @ 2021-02-09 10:37 UTC (permalink / raw)
  To: 9front

> it's not unchanged from 9atom at this point but we merged his code
> (nupas) in, entirely supplanting the labs upas system.

Thanks, that is what I suspected but I wanted to know for sure.

I am running the labs kernel (and have done since 04) so I still have mboxes.
It is not a problem as I tend to delete old mail - my mbox has 10 or 15 messages in it.

maybe this is why I don't feel the need for maildirs.

good info though, thanks.

-Steve

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2021-02-09 10:43 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-06 11:10 [9front] maildir history Steve Simon
2021-02-06 11:35 ` Kurt H Maier
2021-02-06 14:15   ` hiro
2021-02-06 14:38     ` hiro
2021-02-06 15:06     ` Stanley Lieber
2021-02-06 15:18       ` hiro
2021-02-06 21:36         ` Kurt H Maier
2021-02-06 22:25   ` Steffen Nurpmeso
2021-02-06 22:46     ` Kurt H Maier
2021-02-07  0:02       ` Steffen Nurpmeso
2021-02-07  0:20         ` Kurt H Maier
2021-02-07  0:21         ` hiro
2021-02-09 10:37   ` Steve Simon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).