nnmaildir

nnmaildir

[Thomas Schwinge]

Hello!

Convinced by the power of Gnus I want to make it my default mail and
news client.

Regarding news: Converting from slrn to Gnus was no problem at all.

However regarding mail the situation is more complicated:

Since all of my mail folders are in maildir format my backend of choice
would be nnmaildir.

#v+
thomas@server:~ > ls -l Mail/ Mail-nic-nac-project/
Mail-nic-nac-project/:
total 28
drwx------    6 thomas   users         224 Feb  9 20:59 Drafts
drwxr-xr-x    6 thomas   users         224 Feb  9 20:57 INBOX
drwx------    6 thomas   users         224 Feb  9 20:58 Sent
drwxr-xr-x    6 thomas   users         152 Jan 26 18:21 lists.bincimap\.org.binc
drwxr-xr-x    6 thomas   users         152 Jan 26 18:22 lists.bincimap\.org.binc-dev
drwxr-xr-x    6 thomas   users         152 Feb  6 18:23 lists.bincimap\.org.binc-news
[...]
drwxr-xr-x    6 thomas   users         152 Jan 27 16:18 lists.gnus\.org.ding
[...]

Mail/:
total 2404
lrwxrwxrwx    1 thomas   users          30 Feb  9 20:58 Drafts -> ../Mail-nic-nac-project/Drafts
drwx------    6 thomas   users         224 Feb  9 20:55 Elektronik
drwx------    6 thomas   users         224 Feb  9 19:40 INBOX
drwx------    6 thomas   users         224 Feb  9 20:55 INBOX.TODO
lrwxrwxrwx    1 thomas   users          29 Feb  9 20:57 INBOX.nic-nac-project -> ../Mail-nic-nac-project/INBOX
[...]
lrwxrwxrwx    1 thomas   users          28 Feb  9 20:58 Sent -> ../Mail-nic-nac-project/Sent
[...]
#v-

The folders in Mail-nic-nac-project/ are synchronized between several
hosts using maildirsync <URL:http://hacks.dlux.hu/maildirsync/>.

Now you may wonder about the naming scheme, so let me explain this: This
is the IMAPdir layout <URL:http://www.bincimap.org/bincimap-imapdir.html>
used by Binc IMAP <URL:http://www.bincimap.org/> to get the folders
hierarchically displayed in the IMAP client:
#v+
Mail +- INBOX
     |    \- TODO
     +- Drafts
     +- Sent
     +- [...]
     +- lists -+- bincimap.org -+- binc
               |                |- binc-dev
               |                \- binc-news
               +- gnus.org --- ding
               +- [...]
#v-

Parallel to using IMAP I've been using mutt - operating natively on the
maildirs.

I added
#v+
(setq gnus-secondary-select-methods
  '((nnmaildir ""                (directory "~/Mail/"))
    (nnmaildir "nic-nac-project" (directory "~/Mail-nic-nac-project/"))))
#v-
to .gnus.el and the folders showed up in Gnus - although I was not able
to enter the ones whose are containing a '\': The were refused with:
#v+
Retrieving newsgroup: nnmaildir+nic-nac-project:lists.gnus.org.ding...
gnus-select-newsgroup: Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding
#v-
The backslash got weeded out somewhere - amazing that this can even
happen in Lisp and not only in shell programming. :-)

The issue remaining to be resolved for me is that I'd like to have the
status flags synchronized between Gnus and the maildirs because it is
very inconvenient to see messages marked 'read' in Gnus flash up again
as 'unread' ones in mutt.
I have to care about this since I'm not able to fully switch over to
Gnus - it won't be installed on all systems I'm working on - and I don't
accept that I won't be able to use two different mail clients on the
same mail sources.

Looking at the documentation and throwing a quick glance at the list's
archives I heard about this synchronisation not being implemented into
nnmaildir at present but that nnmaildirs maintainer, Paul Jarc, would
eventually be willing to implement it.

With nnmaildir the marks are stored under
[maildir]/.nnmaildir/marks/[type of mark]/[message indentifier].

The problem however is that Gnus uses more marks than the current
maildir specification <URL:http://cr.yp.to/proto/maildir.html> provides:
#v+
[file names in maildir/cur are of the form uniq:info]

info is morally equivalent to the Status field used by mbox readers.
It'd be useful to have MUAs agree on the meaning of info, so I'm keeping a list of info semantics.
Here it is.

info starting with "1,": Experimental semantics.

info starting with "2,": Each character after the comma is an independent flag.

    * Flag "P" (passed): the user has resent/forwarded/bounced this message to someone else.
    * Flag "R" (replied): the user has replied to this message.
    * Flag "S" (seen): the user has viewed this message, though perhaps he didn't read all the way through it.
    * Flag "T" (trashed): the user has moved this message to the trash; the trash will be emptied by a later user action.
    * Flag "D" (draft): the user considers this message a draft; toggled at user discretion.
    * Flag "F" (flagged): user-defined flag; toggled at user discretion. 

New flags may be defined later. Flags must be stored in ASCII order: e.g., "2,FRS".
#v-

I was not yet able to figure out which of the possible marks from
<URL:http://www.gnus.org/manual/gnus_64.html> are actually used and
stored on disk by nnmaildir - and btw.: How is this problem handled e.g.
with Gnus' imap backend? - but the ones also being part of maildir's
specification could easily be used and for the others (dormant, ...)
we could either try to convince DJB to add them to the specification
or the 'experimental semantics' described above could be used.

What do you think?

Regards,
 Thomas

Re: nnmaildir

[Paul Jarc]

Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> although I was not able to enter the ones whose are containing a
> '\': The were refused with:
> #v+
> Retrieving newsgroup: nnmaildir+nic-nac-project:lists.gnus.org.ding...
> gnus-select-newsgroup: Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding
> #v-

I'm not sure what's happening here.  It works for me, with a CVS
checkout from 2004-01-25.  What version of Gnus are you using?

You could work around it by making a new directory for nnmaildir to
look at, which would contain symlinks to the actual maildirs.  The
symlink names could omit "\".

> The issue remaining to be resolved for me is that I'd like to have the
> status flags synchronized between Gnus and the maildirs

You might find this useful for now:
http://article.gmane.org/gmane.emacs.gnus.general/55100

> I was not yet able to figure out which of the possible marks from
> <URL:http://www.gnus.org/manual/gnus_64.html> are actually used and
> stored on disk by nnmaildir

The ones I know of are 'dormant, 'expire, 'forward, 'read, 'reply,
'save, and 'tick.  Note that there is no one-to-one correspondence
between the "marks" that appear in the user interface and the "marks"
that are stored persistently (either by Gnus or by a specific
backend).  For example, "r", "R", and "O" are all represented by the
'read mark in the persistent storage.

> for the others (dormant, ...)  we could either try to convince DJB
> to add them to the specification or the 'experimental semantics'
> described above could be used.

The experimental semantics wouldn't be useful; that would mean that
the message's info would look like ":1,..." which other MUAs wouldn't
understand.


paul

Re: nnmaildir

[Thomas Schwinge]

On Wed, Feb 11, 2004 at 12:42:23PM -0500, Paul Jarc wrote:
> Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> > although I was not able to enter the ones whose are containing a
> > '\': The were refused with:
> > #v+
> > Retrieving newsgroup: nnmaildir+nic-nac-project:lists.gnus.org.ding...
> > gnus-select-newsgroup: Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding
> > #v-
> 
> I'm not sure what's happening here.  It works for me, with a CVS
> checkout from 2004-01-25.  What version of Gnus are you using?

I've been using the CVS checkout with release tag v5-10-6.
nnmaildir.el is dated '2003-04-15'.
I'll try the CVS HEAD version tomorrorw.
Is it sufficient to just copy nnmaildir.el?

> You could work around it by making a new directory for nnmaildir to
> look at, which would contain symlinks to the actual maildirs.  The
> symlink names could omit "\".

Yes, I saw that possibility but I didn't want to go that way because I
didn't want to maintain another hierarchy of folders.
But of course I'll do it that way until I know enough of Emacs' Lisp to
implement the needed functionality by myself. :-)

> > The issue remaining to be resolved for me is that I'd like to have the
> > status flags synchronized between Gnus and the maildirs
> 
> You might find this useful for now:
> http://article.gmane.org/gmane.emacs.gnus.general/55100

So, do you have a working version of such a script? I didn't try this
one because of Bijan Soleymani writing in his second email:
#v+
Oops, I just noticed a couple of small errors in the script. Oh
well... Just consider it as a proof of concept thing. Yes such a
simple script can be written :)
#v-

If you're not going to implement the mark synchronisation functionality
into nnmaildir, that will be the way to go.

> > I was not yet able to figure out which of the possible marks from
> > <URL:http://www.gnus.org/manual/gnus_64.html> are actually used and
> > stored on disk by nnmaildir
> 
> The ones I know of are 'dormant, 'expire, 'forward, 'read, 'reply,
> 'save, and 'tick.

Gnus      | Maildir
----------+---------
dormant   |
expire    | T
forward   | P
read      | S
reply     | R
save      |
tick      | F

'dormant' and 'save' are missing. I'm not quite happy with 'expire'
but it should be possible.

What's the purpose of 'save'? I don't quite get this out of
<URL:http://www.gnus.org/manual/gnus_67.html>.

I'd say that 'dormant' is similar to 'tick', so what about that:
If a message which is new to Gnus has 'F' set, it is displayed as
'ticked'. When the user changes the mark to 'dormant' this is recorded
using the current scheme but the 'F' is preserved.
Clients that don't know how to differentiate between these two will see
maildir's 'F' and Gnus is able to tell if a message is marked as
'ticked' or 'dormant'.

> > for the others (dormant, ...)  we could either try to convince DJB
> > to add them to the specification or the 'experimental semantics'
> > described above could be used.
> 
> The experimental semantics wouldn't be useful; that would mean that
> the message's info would look like ":1,..." which other MUAs wouldn't
> understand.

I had not tried that before - I thought that MUAs would leave the flags
alone the didn't understand.
Now I've got disabused (Does one use this word that way? - I'm always
willing to improve my English.)
Mutt does leave them alone - as long as you don't change any flags.
Doing so, unknown ones will get weeded out.
That's no good behaviour as DJB declares that "New flags may be defined
later." in <URL:http://cr.yp.to/proto/maildir.html>.

Regards,
 Thomas

Re: nnmaildir

[Paul Jarc]

Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> I'll try the CVS HEAD version tomorrorw.
> Is it sufficient to just copy nnmaildir.el?

Probably, but to be sure you can use one whole, consistent checkout.
The backslash removal might not be happening within nnmaildir anyway.

>> You might find this useful for now:
>> http://article.gmane.org/gmane.emacs.gnus.general/55100
>
> So, do you have a working version of such a script?

No.

> If you're not going to implement the mark synchronisation functionality
> into nnmaildir, that will be the way to go.

I'm just not sure when I'll get to it.

> What's the purpose of 'save'?

In a *Summary* buffer, there are some gnus-summary-save-* commands
(type [O C-h] to see a list); if you run one of these commands to save
an article to a file or whatever, then the 'save mark will be set for
that article.

> If a message which is new to Gnus has 'F' set, it is displayed as
> 'ticked'. When the user changes the mark to 'dormant' this is recorded
> using the current scheme but the 'F' is preserved.

I would make F correspond exactly to 'tick, without involving 'dormant
at all.  Otherwise it's too complicated, and the stored information
may be ambiguous.

>> The experimental semantics wouldn't be useful; that would mean that
>> the message's info would look like ":1,..." which other MUAs wouldn't
>> understand.
>
> I had not tried that before - I thought that MUAs would leave the flags
> alone the didn't understand.

Yes, but they also wouldn't be able to interpret them.  ":1,F" does
not necessarily mean the same thing as ":2,F".  ":2,F" should mean the
same thing to all MUAs, but different MUAs may experiment with
different schemes, and might both use ":1,F" to mean different
things.  An MUA shouldn't change something it doesn't understand, but
with ":1," info, different MUAs may think they understand each other
when they really don't.

> Now I've got disabused (Does one use this word that way? - I'm always
> willing to improve my English.)

I would say "Now I have been disabused of that idea".


paul

Re: nnmaildir

[Thomas Schwinge]

On Wed, Feb 11, 2004 at 06:22:47PM -0500, Paul Jarc wrote:
> Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> > I'll try the CVS HEAD version tomorrorw.
> > Is it sufficient to just copy nnmaildir.el?
> 
> Probably, but to be sure you can use one whole, consistent checkout.
> The backslash removal might not be happening within nnmaildir anyway.

Now I'm running that latest CVS checkout of Gnus.
Starting it gives some (harmless I presume) warnings:
#v+
[...]
Loading easymenu...done
Loading gnus-start...done
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nntp.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnoo.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nndraft.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmail.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/mail-source.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmh.el' newer than byte-compiled file
Reading /home/thomas/.newsrc.eld...
Loading gnus-group...done
Reading active file from news.schwinge.homeip.net via nntp...
Reading active file via nnmaildir...
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.el' newer than byte-compiled file
Opening nnmaildir server...done
Reading active file from nic-nac-project via nnmaildir...
Opening nnmaildir server on nic-nac-project...done
Loading gnus-cache...done
Loading timezone...done
Opening nnmaildir server...done
Opening nnmaildir server on nic-nac-project...done
No new newsgroups
Checking new news...
Loading gnus-sum...done
Checking new news...done
Loading gnus-topic...done
Retrieving newsgroup: nnmaildir+nic-nac-project:lists.gnus.org.ding...
Loading gnus-ml...
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/mm-view.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/mm-uu.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/mml2015.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/pgg.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/pgg-def.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/pgg-parse.el' newer than byte-compiled file
Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/password.el' newer than byte-compiled file
Loading gnus-ml...done
Opening nnmaildir server on nic-nac-project...done
gnus-select-newsgroup: Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding
#v-
#v+
thomas@speedy:~ > ls -ld Mail-nic-nac-project/lists.gnus\\.org.ding/
drwxr-xr-x    6 thomas   users         152 Jan 27 16:18 Mail-nic-nac-project/lists.gnus\.org.ding/
thomas@speedy:~ > tail -3 .gnus.el
(setq gnus-secondary-select-methods
  '((nnmaildir ""                (directory "~/Mail/"))
    (nnmaildir "nic-nac-project" (directory "~/Mail-nic-nac-project/"))))
#v-

I checked it again with a new folder
#v+
thomas@speedy:~ > maildirmake Mail/test\\folder
#v-
which got to
#v+
       0: nnmaildir:testfolder
#v-

Shall I post the strace of Gnus (emacs) when entering such a maildir?

> > If you're not going to implement the mark synchronisation functionality
> > into nnmaildir, that will be the way to go.
> 
> I'm just not sure when I'll get to it.

OK.
I'm aware that you have more important things to do that that. :-)
I wrote to Bijan Soleymani and asked him if he has a working version of
the synchronisation script - otherwise I'll write it myself.

> > What's the purpose of 'save'?
> 
> In a *Summary* buffer, there are some gnus-summary-save-* commands
> (type [O C-h] to see a list); if you run one of these commands to save
> an article to a file or whatever, then the 'save mark will be set for
> that article.

I got that; although I don't quite get what I would do with the
information that an article I'll be reading again later has been saved -
since saving is just copying the article to another folger - if I got
that right.

> > If a message which is new to Gnus has 'F' set, it is displayed as
> > 'ticked'. When the user changes the mark to 'dormant' this is recorded
> > using the current scheme but the 'F' is preserved.
> 
> I would make F correspond exactly to 'tick, without involving 'dormant
> at all.  Otherwise it's too complicated, and the stored information
> may be ambiguous.

That would of course be the cleaner solution.
But how should 'dormant and 'save then be reflected?
In what way would my solution be ambiguous?

> >> The experimental semantics wouldn't be useful; that would mean that
> >> the message's info would look like ":1,..." which other MUAs wouldn't
> >> understand.
> >
> > I had not tried that before - I thought that MUAs would leave the flags
> > alone the didn't understand.
> 
> Yes, but they also wouldn't be able to interpret them.

So it should leave them alone.

> ":1,F" does
> not necessarily mean the same thing as ":2,F".  ":2,F" should mean the
> same thing to all MUAs, but different MUAs may experiment with
> different schemes, and might both use ":1,F" to mean different
> things.  An MUA shouldn't change something it doesn't understand, but
> with ":1," info, different MUAs may think they understand each other
> when they really don't.

OK. So using experimental semantics is not the way to go.
But I was quite surprised when I saw mutt just weed out the flags it
didn't understand.
I think I'll write to the maintainer about that (and I'll check other
maildir-using programs I' using).

Regards,
 Thomas

Re: nnmaildir

[Paul Jarc]

Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> Now I'm running that latest CVS checkout of Gnus.
> Starting it gives some (harmless I presume) warnings:
> #v+
> [...]
> Loading easymenu...done
> Loading gnus-start...done
> Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nntp.el' newer than byte-compiled file

Hm... I wonder if you're really running the version you think you're
running.  Make sure your .emacs adds the CVS Gnus to your load-path
before you load any Gnus components.  This may require starting emacs
with --no-site-file, since site-start.el is loaded before ~/.emacs,
and site-start.el might load some Gnus components.  (You can have your
.emacs load site-start.el after your load-path changes.)  If Gnus is
properly installed and properly loaded, I don't think these warnings
should appear.

> Shall I post the strace of Gnus (emacs) when entering such a maildir?

No, but a backtrace would be helpful.  Before triggering the error, do
M-x set-variable RET debug-on-error RET t RET.

> I don't quite get what I would do with the information that an
> article I'll be reading again later has been saved - since saving is
> just copying the article to another folger

Someone thought it was useful once, so now Gnus has it.  Such is the
way of Gnus. :)

> But how should 'dormant and 'save then be reflected?

I don't think there is any way to make them visible to other MUAs.
They will have to stay as they are now.

> In what way would my solution be ambiguous?

Suppose a message has the 'tick mark, and the user changes it to
'dormant.  Gnus tells nnmaildir to remove 'tick and add 'dormant.  You
say that nnmaildir should modify its own marks, but leave "F" in
place.  So now, according to the standard maildir flags, the message
is "F"lagged, but according to nnmaildir's separate marks, the message
is not 'tick-ed.

But there is another way we can arrive at this same situation: suppose
a message initially has neither 'tick or 'dormant, and the user adds
'dormant.  Later, another MUA adds "F".  So now, again, the message
has "F" and 'dormant but not 'tick.

So how do we interpret this state of stored information the next time
Gnus starts?  In the first case, we should presumably say that the
message still does not have 'tick.  In the second case, we would like
to say that the message now has 'tick - the "F" mark was added by
another MUA, and we want to pick up that change in Gnus too.  But
since the stored information is the same in these two cases, we can't
distinguish them, and so we can't get them both right.

>>> I had not tried that before - I thought that MUAs would leave the flags
>>> alone the didn't understand.
>>
>> Yes, but they also wouldn't be able to interpret them.
>
> So it should leave them alone.

I agree.

> But I was quite surprised when I saw mutt just weed out the flags it
> didn't understand.

Yes, that is surprising, and seems buggy.


paul

Re: nnmaildir

[Thomas Schwinge]

On Thu, Feb 12, 2004 at 11:35:58AM -0500, Paul Jarc wrote:
> Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> > Now I'm running that latest CVS checkout of Gnus.
> > Starting it gives some (harmless I presume) warnings:
> > #v+
> > [...]
> > Loading easymenu...done
> > Loading gnus-start...done
> > Source file `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nntp.el' newer than byte-compiled file
> 
> Hm... I wonder if you're really running the version you think you're
> running.

I'm quite sure:
#v+
thomas@speedy:~ > strace -f -e open emacs -f gnus 2>&1 | grep '/package.*gnus' | cut -d \" -f 2
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/info
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/etc
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/etc/gnus
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/etc/smilies
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/gnus.elc
[...]
/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.elc
[...]
thomas@speedy:~ > readlink -f /package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus
/fs/packages/mount/package/misc/spf/gnus-2004-02-12-13:01:23-1
thomas@speedy:~ > readlink -f /package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.elc
/fs/packages/mount/package/misc/spf/gnus-2004-02-12-13:01:23-1/share/emacs/site-lisp/nnmaildir.elc
thomas@speedy:~ > stat /package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.el*
  File: `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.el'
  Size: 63235           Blocks: 136        IO Block: 4096   regular file
Device: fe05h/65029d    Inode: 135956      Links: 1    
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2004-02-12 14:18:22.000000000 +0100
Modify: 2004-02-12 14:17:19.000000000 +0100
Change: 2004-02-12 14:18:33.000000000 +0100

  File: `/package/misc/spf/emacs-21.3-1/share/emacs/site-lisp/gnus/share/emacs/site-lisp/nnmaildir.elc'
  Size: 101500          Blocks: 208        IO Block: 4096   regular file
Device: fe05h/65029d    Inode: 135862      Links: 1    
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2004-02-12 22:56:42.000000000 +0100
Modify: 2004-02-12 14:17:18.000000000 +0100
Change: 2004-02-12 14:18:33.000000000 +0100
#v-

So in fact the .elc file is 1 second newer than the .el file.
Strange.
I've been installing it via sp-foreign <http://multivac.cwru.edu./spf/>.

Now I reinstalled it:
#v+
thomas@speedy:~ > stat /fs/packages/mount/package/misc/spf/gnus-2004-02-12-13:01:23-2/share/emacs/site-lisp/nnmaildir.el*
  File: /fs/packages/mount/package/misc/spf/gnus-2004-02-12-13:01:23-2/share/emacs/site-lisp/nnmaildir.el'
  Size: 63235           Blocks: 136        IO Block: 4096   regular file
Device: fe05h/65029d    Inode: 134768      Links: 1
Access: (0644/-rw-r--r--)  Uid: (  500/  thomas)   Gid: (  100/   users)
Access: 2004-02-12 23:08:15.000000000 +0100
Modify: 2004-02-12 23:08:15.000000000 +0100
Change: 2004-02-12 23:08:15.000000000 +0100

  File: /fs/packages/mount/package/misc/spf/gnus-2004-02-12-13:01:23-2/share/emacs/site-lisp/nnmaildir.elc'
  Size: 101468          Blocks: 208        IO Block: 4096   regular file
Device: fe05h/65029d    Inode: 135556      Links: 1
Access: (0644/-rw-r--r--)  Uid: (  500/  thomas)   Gid: (  100/   users)
Access: 2004-02-12 23:08:16.000000000 +0100
Modify: 2004-02-12 23:08:16.000000000 +0100
Change: 2004-02-12 23:08:16.000000000 +0100
#v-

Restarting Gnus, the warnings did not show up again.


> > Shall I post the strace of Gnus (emacs) when entering such a maildir?
> 
> No, but a backtrace would be helpful.  Before triggering the error, do
> M-x set-variable RET debug-on-error RET t RET.

The folder is displayed the wrong way in the group buffer:
#v+
       *: nnmaildir+nic-nac-project:lists.gnus.org.ding
#v-
whereas it should be
#v+
       *: nnmaildir+nic-nac-project:lists.gnus\.org.ding
#v-

So it's quite obvious that Gnus will print the following:
#v+
Debugger entered--Lisp error: (error "Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding")
  signal(error ("Couldn't activate group nnmaildir+nic-nac-project:lists.gnus.org.ding: No such group: lists.gnus.org.ding"))
  error("Couldn't activate group %s: %s" "nnmaildir+nic-nac-project:lists.gnus.org.ding" "No such group: lists.gnus.org.ding")
  gnus-select-newsgroup("nnmaildir+nic-nac-project:lists.gnus.org.ding" nil nil)
  gnus-summary-read-group-1("nnmaildir+nic-nac-project:lists.gnus.org.ding" nil t nil nil nil)
  gnus-summary-read-group("nnmaildir+nic-nac-project:lists.gnus.org.ding" nil t nil nil nil nil)
  gnus-group-read-group(nil t)
  gnus-group-select-group(nil)
  gnus-topic-select-group(nil)
  call-interactively(gnus-topic-select-group)
#v-

So the error is likely be somewhere in the code that recognises new
folders and adds them to the group buffer.


> > I don't quite get what I would do with the information that an
> > article I'll be reading again later has been saved - since saving is
> > just copying the article to another folger
> 
> Someone thought it was useful once, so now Gnus has it.  Such is the
> way of Gnus. :)

So it wouldn't be a great loss if 'save would not be reflected in the
maildir?

> > In what way would my solution be ambiguous?
> 
> Suppose a message has the 'tick mark, and the user changes it to
> 'dormant.  Gnus tells nnmaildir to remove 'tick and add 'dormant.  You
> say that nnmaildir should modify its own marks, but leave "F" in
> place.  So now, according to the standard maildir flags, the message
> is "F"lagged, but according to nnmaildir's separate marks, the message
> is not 'tick-ed.
> 
> But there is another way we can arrive at this same situation: suppose
> a message initially has neither 'tick or 'dormant, and the user adds
> 'dormant.
Additionally nnmaildir will set 'F'.
So, the message has 'F' in maildir and 'dormant in Gnus.
The message it flagged in mutt and dormant in Gnus.

Now I unflag it in mutt: 'F' is removed.
I start Gnus, nnmaildir sees 'dormant set but not 'F' and thus removes
dormant - recognized as being old / invalid.
The message is unflagged in mutt and neither ticked nor dormant in Gnus.

Later - using mutt - I decide that the message actually _is_ important:
I flag it, 'F' is set.
Starting Gnus, nnmaildir sees 'F' but neither 'tick nor 'dormant - they
were deleted in the older session -  so the message is recognized as
being newly flagged: 'tick is set.
The message is flagged in mutt and ticked in Gnus.

Now I decide the the message is not actually _that_ important, but I'd
be interested in follow-ups, so I set it to 'dormant.
'tick is removed, 'dormant is set, 'F' is preserved.
The message is flagged in mutt and dormant in Gnus.

Wouldn't that work?

Regards,
 Thomas

Re: nnmaildir

[Paul Jarc]

Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> So the error is likely be somewhere in the code that recognises new
> folders and adds them to the group buffer.

Ok... try opening the *Server* buffer (type [^] from the *Group*
buffer) and enter the nnmaildir server.  Then switch to the " *nntpd*"
buffer (note the leading space).  See how the group names show up
there - that will at least tell us whether the problem is inside
nnmaildir or outside.

> So it wouldn't be a great loss if 'save would not be reflected in the
> maildir?

Right.  Other MUAs may not have any such concept, and the standard
maildir info doesn't provide any way to represent that information
anyway.

>> But there is another way we can arrive at this same situation: suppose
>> a message initially has neither 'tick or 'dormant, and the user adds
>> 'dormant.
>
> Additionally nnmaildir will set 'F'.

Ok, that would remove the ambiguity.  But even so, I'd rather keep it
simple, and have "F" correspond exactly to 'tick.


paul

Re: nnmaildir

[Thomas Schwinge]

On Thu, Feb 12, 2004 at 05:56:45PM -0500, Paul Jarc wrote:
> Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> > So the error is likely be somewhere in the code that recognises new
> > folders and adds them to the group buffer.
> 
> Ok... try opening the *Server* buffer (type [^] from the *Group*
> buffer) and enter the nnmaildir server.  Then switch to the " *nntpd*"
> buffer (note the leading space).  See how the group names show up
> there - that will at least tell us whether the problem is inside
> nnmaildir or outside.

#v+
[...]
lists.skarnet\.org.supervision 46 1 y
lists.gnus\.org.ding 38 1 y
INBOX 26 1 y
[...]
#v-

This is correct whereas it's incorrent when browsing the nnmaildir
server:
#v+
[...]
      38: lists.gnus.org.ding
      72: lists.isdn4linux.de.isdn4linux
     131: lists.multivac.cwru.edu.prjware
[...]
#v-


> >> But there is another way we can arrive at this same situation: suppose
> >> a message initially has neither 'tick or 'dormant, and the user adds
> >> 'dormant.
> >
> > Additionally nnmaildir will set 'F'.
> 
> Ok, that would remove the ambiguity.  But even so, I'd rather keep it
> simple, and have "F" correspond exactly to 'tick.

I wouldn't, but of course that's less urgent than getting the whole
thing working at all.

Regards,
 Thomas

Re: nnmaildir

[Paul Jarc]

Thomas Schwinge <schwinge-lists-gnus.org-ding@nic-nac-project.de> wrote:
> On Thu, Feb 12, 2004 at 05:56:45PM -0500, Paul Jarc wrote:
>> Then switch to the " *nntpd*" buffer
...
> lists.skarnet\.org.supervision 46 1 y
> lists.gnus\.org.ding 38 1 y

Ok, so nnmaildir is reporting the names correctly, but then the
backslashes are lost when Gnus parses this buffer.  Or perhaps there's
some undocumented encoding that needs to be applied to the group
names.  Does anyone know?


paul

Re: nnmaildir

[Kai Grossjohann]

On a tangent: does someone have etymology on the "#v+" and "#v-"
markers?

Kai

#v+, #v- (was: nnmaildir)

[lawrence mitchell]

Kai Grossjohann wrote:

> On a tangent: does someone have etymology on the "#v+" and "#v-"
> markers?

It's a slrnism, they're meant to mark verbatim text. e.g.

#v+
Here is some text
(defun foo (bar)
  bar)
it should be considered as "verbatim"
#v-

i.e. anything in between them is (in slrn) fontified in a
different colour, and not wrapped.

I believe they're not supported in other mail/newsreaders
though.

-- 
lawrence mitchell <s0198183+ding@sms.ed.ac.uk>

Re: #v+, #v-

[Kai Grossjohann]

lawrence mitchell <s0198183+ding@sms.ed.ac.uk> writes:

> I believe they're not supported in other mail/newsreaders
> though.

Well, anyone up to doing it for Gnus?  Sounds like a cool feature.  We
*do* already have MIME multiparts, but one feature more, what can it
hurt...

Kai

Re: #v+, #v-

[Karl Pflästerer]

On 14 Feb 2004, Kai Grossjohann <- kai@emptydomain.de wrote:

> lawrence mitchell <s0198183+ding@sms.ed.ac.uk> writes:

>> I believe they're not supported in other mail/newsreaders
>> though.

> Well, anyone up to doing it for Gnus?  Sounds like a cool feature.  We
> *do* already have MIME multiparts, but one feature more, what can it
> hurt...


Well I did a first study to see what maybe had to be done but had quick
some questions which I couldn't answer myself without deeply studying
the sources.

(defun gnus-article-search-verbose ()
  (let (verbose pt)
    (save-match-data
      (save-excursion
	(goto-char (point-min))
	(while (re-search-forward "^#v\\+" nil t)
	  (setq pt (match-beginning 0))
	  (and (re-search-forward "^#v\\(.\\)" nil t)
	       (string-equal (match-string 1) "-")
	       (push (cons pt (match-end 0)) verbose))
	  (unless (eobp) (goto-char (point-at-bol))))))
    (nreverse verbose)))

(defun gnus-article-highlight-verbose () 
  (dolist (region (gnus-article-search-verbose))
    (gnus-overlay-put (gnus-make-overlay (car region) (cdr region))
		      'face 'gnus-header-subject-face)))

(defvar gnus-treat-highlight-verbose t)
(put 'gnus-treat-highlight-verbose 'highlight t)

(setq gnus-treatment-function-alist
	(nconc gnus-treatment-function-alist
	       '((gnus-treat-highlight-verbose gnus-article-highlight-verbose))))


The above code finds the regions of interest and highlights them (I
choose some arbitrary face just to have an effect).

But we wanted more: wrapping (== filling) should be inhibited IIRC.  Can
this be done with overlays?  How should the highlighting be done? Like
here with just one face or do we want different faces for different
tokens?



   KP

-- 
Männer der Wissenschaft! Man sagt ihr viele nach, 
aber die meisten mit Unrecht.  
                             Karl Kraus 'Aphorismen'

Re: nnmaildir

[Paul Jarc]

Harry Putnam <reader@newsguy.com> wrote:
> There is one small thing that I can think of that might be improved
> some how.  Those filenames are pretty ridiculous if you do anything
> with those files outside gnus.  Seems like a simple incremental
> numbering scheme once the messags are brought from `new' would work.

It works as long as you never have two processes moving mail from new/
to cur/ at the same time.  Otherwise, you still need an uncoordinated
way to ensure filenames don't collide.


paul

Re: nnmaildir

[Paul Jarc]

Josh Huber <huber@alum.wpi.edu> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> nnmaildir stores NOV information in one file per message.  Finding
>> an article's NOV information is as fast as finding the article
>> itself.
>
> Indeed, I've been meaning to try nnmaildir out.

Note that finding the NOV information is also just as *slow* as
finding the article. :)  It's up to you to either keep your groups
small or use a filesystem that can deal with large directories.

> Would it be very painful to migrate from nnml to maildir?

Setting up the server should be pretty straightforward.  Copying the
articles is as easy as B c.  The group parameters for expiry will be
different from what you're used to, but hopefully not too troublesome.


paul