Re: Experimental new Maildir backend

["Juan José García-Ripoll" <juanjose.garciaripoll@gmail.com>]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:
> Juan José García-Ripoll <juanjose.garciaripoll@gmail.com> writes:
>> Hi Eric, nice to see you also are interested in this format. I also
>> started tweaking nnmaildir. I got to a point where I fixed the
>> separator, introducing a configuration parameter that starts with ":"
>> but can be configured to other values. Unfortunately the result was
>> extremely slow.
>
> You mean making that character configurable actually introduced a
> slowdown? It seems odd that it would be that much of a factor.

No, sorry for the confusion: the change of the separator is trivial, and
does not affect overall performance, but the actual backend is simply
too slow for platforms where file creation, time stamp checking and
linking are not cheap operations.

>> I started using inheritance, but it did not work. The parent backend did
>> not get its variables properly assigned
>
> To me this is usually a sign that there are code paths that don't hit
> `nnoo-change-server'. I see you've got that in `nnmaild-open-server',
> which gets called in `nnmaild-possibly-change-directory', but I would
> take a look at the various code entry points and see if any of them
> sneak past that. Also, you're calling `nnmaild-server-opened', but that
> function doesn't seem to be defined?

Again, sorry for my confusing explanation. What follows is an
explanation of how I attempted it, not how it is done in my
repository. In the version from GitHub, everything works and no
inheritance is needed. To be fair no inheritance is possible because I
ended up doing things differently from nnml, with a different caching
protocol, and a totally different way of handling marks.

As for my first attempts, I started using inheritance as per the nndir
example. In that case, I wanted to reuse nnml's file scanning and NOV
caching, which is why I created manually the nnmaild-request-*
functions. These attempts would perform some tasks before calling the
the functions of the parent backend with the same name. The problem
there is that while all variables are properly defined in the child
(including those variables that in defvoo specify alternative names for
the child framework, i.e. nnmaild-directory translated to
nnml-directory), the variables were arbitrarily restored to default
values when the parent functions were alled. I am sorry, but I do not
keep that code around, although it would not be difficult to try again.

> This is too bad, and something else that I would (theoretically,
> eventually) like to work on. If a new backend looks highly similar to an
> existing backend, it should be able to share a lot of the code.

There was no real code reuse. nnmaildir's functions are impossible to
reuse; nnml's relies heavily on numbered files and uses a different way
of caching NOV's

> What do you mean exactly, "tell Gnus the attributes of a message"?

It took me quite a lot of debugging to develop the functions that modify
the info structure replacing the marks with those that are given by the
Maildir backend. It was not clear from the documentation where that is
supposed to happen. nnml has no implementation for marks, delegating
everything on the Gnus dribble files. nnimap has an undocumented
implementation that uses the fact that it has a request and a finish-
processing stages.

> If you have concrete suggestions as to how Gnus could do a better job
> of allowing inheritance, I'd love to hear it.

To be fair, I am quite lost regarding how inheritance is handled and how
the whole infrastructure works.

- I find it confusing the way defvoo works and the fact that everything
is based on variables instead of objects, methods and slots. 

- In particular, coming from an OO background (both in Common Lisp and
C++), I do not understand when and how those variables are rewritten. In
OO paradigms, one fixes the slots of the parent classes during the
construction phase; here it seems there is some magic rewrite happening
at hidden places.

- This made it confusing to me how I can write my own defvoo method and,
from within that method call equivalent methods from other
backends. That does not seem to be supported.

- On a more architectural note, it is also strange that methods are
expected to work with global state, instead of getting properly formed
records. This makes the handling of group, server and info objects very
confusing -- when do they have a value, vs. when can they be just nil,
still puzzles me.

- I also find that, for proper inheritance, some of the backends should
have less assumptions about the underlying system (e.g. nnml should not
assume numbering of files), allowing the rewrite of specific methods
(e.g. retrieval and writing of marks), which in the current framework
seems very difficult, as there are no real virtual functions.

Hope this helps clarify how confused I am about inheritance. This said,
I think I now understand how to write the backend _without_ inheritance.

Juanjo

-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es