Using different backends

Using different backends

[Andreas Jaeger]

I'm currently trying to use different backends for different
mailfolders.  Mails are split via procmail into one directory:

(setq mail-sources
      '(
	(directory :path "/mail-aj/Mail/procmail-work/"
		  :plugged t
		  :suffix ".spool")
	))

And I'd like to use - depending on the folder - either nnmaildir or
nnfolder for incoming mails.  I also have some nnml folders that I'm
currently converting to either nnmaildir or nnfolder:

(setq gnus-secondary-select-methods '((nnmaildir "gromit"
						 (directory "/mail-aj/Maildirs/")
						 (get-new-mail t)
						 )
				      (nnml "")
				      (nnfolder "archive")))


But what seems to happen is that now all files in my procmail dir are
now caught by nnmaildir and new (duplicated) folders exist now - one
old nnfolder/nnml and one new nnmaildir :-(.

So, how can I get this working?  I do not want nnmaildir create new
groups found in my procmail spool, it should pass them on to one of
the other methods.


I'm using Oort Gnus from CVS,
Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: Using different backends

[Kai Großjohann]

Andreas Jaeger <aj@suse.de> writes:

> I'm currently trying to use different backends for different
> mailfolders.  Mails are split via procmail into one directory:
>
> (setq mail-sources
>       '(
> 	(directory :path "/mail-aj/Mail/procmail-work/"
> 		  :plugged t
> 		  :suffix ".spool")
> 	))
>
> And I'd like to use - depending on the folder - either nnmaildir or
> nnfolder for incoming mails.

You could tell procmail to use two different directories.  And then
you can configure a different mail source for each server.  I forget
whether it works via server parameters, or what, but it's possible
somehow.

-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: Using different backends

[Andreas Jaeger]

kai.grossjohann@gmx.net (Kai Großjohann) writes:

> Andreas Jaeger <aj@suse.de> writes:
>
>> I'm currently trying to use different backends for different
>> mailfolders.  Mails are split via procmail into one directory:
>>
>> (setq mail-sources
>>       '(
>> 	(directory :path "/mail-aj/Mail/procmail-work/"
>> 		  :plugged t
>> 		  :suffix ".spool")
>> 	))
>>
>> And I'd like to use - depending on the folder - either nnmaildir or
>> nnfolder for incoming mails.
>
> You could tell procmail to use two different directories.  And then
> you can configure a different mail source for each server.  I forget
> whether it works via server parameters, or what, but it's possible
> somehow.


Can anybody help me here?  I looked through the manual and through the
sources but couldn't find anything.  The idea sounds fine,

Thanks,
Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: Using different backends

[Paul Jarc]

Andreas Jaeger <aj@suse.de> wrote:
> I'm currently trying to use different backends for different
> mailfolders.  Mails are split via procmail into one directory:
>
> (setq mail-sources
>       '(
> 	(directory :path "/mail-aj/Mail/procmail-work/"
> 		  :plugged t
> 		  :suffix ".spool")
> 	))
>
> And I'd like to use - depending on the folder - either nnmaildir or
> nnfolder for incoming mails.

I'd suggest reconfiguring procmail so that messages destined for
nnmaildir are delivered directly to your nnmaildir groups (maildirs)
by procmail.  (That's safe with nnmaildir, but not other backends.)
Whatever's left can then be delivered to procmail-work as it is now,
and then split by nnfolder.

> (setq gnus-secondary-select-methods '((nnmaildir "gromit"
> 						 (directory "/mail-aj/Maildirs/")
> 						 (get-new-mail t)
> 						 )
> 				      (nnml "")
> 				      (nnfolder "archive")))

The default for nnml and nnfolder is to do mail splitting, so this
means that all three will do mail splitting, which isn't what you
want.  It's generally not useful to Have more than one server do
splitting, because whichever one does it first will get all the mail.
Along with the above change, I think you should remove get-new-mail
for nnmaildir and explicitly add (nnml-get-new-mail nil) for nnml.


paul

Re: Using different backends

[Andreas Jaeger]

prj@po.cwru.edu (Paul Jarc) writes:

> Andreas Jaeger <aj@suse.de> wrote:
>> I'm currently trying to use different backends for different
>> mailfolders.  Mails are split via procmail into one directory:
>>
>> (setq mail-sources
>>       '(
>> 	(directory :path "/mail-aj/Mail/procmail-work/"
>> 		  :plugged t
>> 		  :suffix ".spool")
>> 	))
>>
>> And I'd like to use - depending on the folder - either nnmaildir or
>> nnfolder for incoming mails.
>
> I'd suggest reconfiguring procmail so that messages destined for
> nnmaildir are delivered directly to your nnmaildir groups (maildirs)
> by procmail.  (That's safe with nnmaildir, but not other backends.)

Ok, but then I need to use duplicate suppresion in procmail and not in
Gnus since nnmaildir does not support it.

> Whatever's left can then be delivered to procmail-work as it is now,
> and then split by nnfolder.
>
>> (setq gnus-secondary-select-methods '((nnmaildir "gromit"
>> 						 (directory "/mail-aj/Maildirs/")
>> 						 (get-new-mail t)
>> 						 )
>> 				      (nnml "")
>> 				      (nnfolder "archive")))
>
> The default for nnml and nnfolder is to do mail splitting, so this
> means that all three will do mail splitting, which isn't what you
> want.  It's generally not useful to Have more than one server do
> splitting, because whichever one does it first will get all the mail.

So, this is broken by design ;-).

> Along with the above change, I think you should remove get-new-mail
> for nnmaildir and explicitly add (nnml-get-new-mail nil) for nnml.

And then I could use nnmaildir and nnfolder in parallel.  But I fear
there's no possibility to use all three in parallel with getting new
mail?  What a pity...

Thanks,
Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: Using different backends

[Paul Jarc]

Andreas Jaeger <aj@suse.de> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> I'd suggest reconfiguring procmail so that messages destined for
>> nnmaildir are delivered directly to your nnmaildir groups (maildirs)
>> by procmail.  (That's safe with nnmaildir, but not other backends.)
>
> Ok, but then I need to use duplicate suppresion in procmail and not in
> Gnus since nnmaildir does not support it.

Currently, yes.

>> It's generally not useful to Have more than one server do
>> splitting, because whichever one does it first will get all the
>> mail.
>
> So, this is broken by design ;-).

Right.

>> Along with the above change, I think you should remove get-new-mail
>> for nnmaildir and explicitly add (nnml-get-new-mail nil) for nnml.
>
> And then I could use nnmaildir and nnfolder in parallel.  But I fear
> there's no possibility to use all three in parallel with getting new
> mail?  What a pity...

What do you mean?  You don't want to split any mail into nnml groups,
right?  So add (nnml-get-new-mail nil) to that select method.  Then,
if you also reconfigure procmail, you'll no longer want to split mail
into nnmaildir groups (not in Gnus, anyway, since it's already done by
procmail), so you remove (get-new-mail t) from that select method.
Then everything will work the way you want (modulo duplicate
suppression): nnmaildir gets the appropriate messages from procmail;
nnml gets no messages; nnfolder gets the appropriate messages from
procmail+Gnus splitting.


paul

Re: Using different backends

[Michael Teichgräber]

[-- Attachment #1: Type: text/plain, Size: 2279 bytes --]

Andreas Jaeger <aj@suse.de> writes:

> Can anybody help me here?  I looked through the manual and through the
> sources but couldn't find anything.  The idea sounds fine,

In my case procmail is splitting mail into mbox-folders within two
directories 'incoming-nnfolder' and 'incoming-nnml'. Gnus will copy
per-folder from these directories via 'mail-sources' into nnfolder and
nnml groups.

The only way I got this working is by changing the key binding of 'g'
for getting new mail to a function that runs gnus-group-get-new-news
twice in different environments:

------------------------------------------------------------>8----------
(setq gnus-select-method '(nntp "news.vr.in-berlin.de"))

(setq gnus-secondary-select-methods '((nnfolder "") (nnml "")))

(defun my-gnus-group-get-new-news (&optional foo) (interactive "i")
  (let (mail-sources)
    (let ((gnus-secondary-select-methods '((nnml "") (nnfolder ""))))
      (setq mail-sources
            '((directory :path "~/Mail/incoming-nnml"
                         :suffix ""
                         :plugged t)))
      (gnus-group-get-new-news))
    (setq mail-sources
          '((file :plugged t)
            (directory :path "~/Mail/incoming-nnfolder"
                       :suffix ""
                       :plugged t)))
    (gnus-group-get-new-news)))

(add-hook 'gnus-group-mode-hook
          '(lambda()
             (local-set-key ?g 'my-gnus-group-get-new-news)))
----------8<------------------------------------------------------------

This approach was suggested some time ago in a News posting.

It makes Gnus check twice for new News articles as well. Trying to
avoid this, I have set gnus-select-method first to (nnml "")
                   and gnus-secondary-select-method to nil,

so that new News articles would be checked for only once. But, for
unknown reasons the process of getting mail from these local folders
then gets quite slow.

Since Gnus does not support this way of using two backends for local
mail retrieval, there is probably something strange with this
approach. Is there a better one, that allows using procmail to split
mails, and makes Gnus getting it into nnml or nnfolder groups,
depending on folder name?

-- 
Michael

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Using different backends

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> Since Gnus does not support this way of using two backends for local
> mail retrieval, there is probably something strange with this
> approach. Is there a better one, that allows using procmail to split
> mails, and makes Gnus getting it into nnml or nnfolder groups,
> depending on folder name?

I still believe it is possible to specify the mail sources in the
server spec.  Maybe like this:

(setq gnus-secondary-select-methods
      '( (nnml "" (mail-sources ((file))))
         (nnfolder "" (mail-sources ((pop :server ...)))) ))

But I've never tested it, and I'm not sure about the exact syntax.  I
tried to skim the source code once to figure it out, but couldn't
find anything conclusive.

Possible alternatives to (mail-sources ((file))):

  - (mail-sources . ((file)))
  - (mail-source (file))
  - (mail-source . (file))

Anyone willing to try it out and report findings?  The manual needs
to be fixed.

-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: Using different backends

[Michael Teichgräber]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

> Anyone willing to try it out and report findings?  The manual needs
> to be fixed.

(setq gnus-secondary-select-methods 
      '((nnfolder "" 
          (mail-sources '((file :plugged t) ; get mail from system mailbox
                          (directory :path "~/Mail/incoming-nnfolder"
                                     :suffix ""
                                     :plugged t))))
        (nnml "" 
          (mail-sources '((directory :path "~/Mail/incoming-nnml"
                                     :suffix ""
                                     :plugged t))))))

Indeed, this works from what I can tell after some tests. 

But, why is it much slower than the approach in my previous mail? That
one needs around 1 second, this one now takes around 7 seconds (for 20
nnml and 30 nnfolder groups).

Does this different scope of 'mail-sources' perhaps introduce an
overhead that slows down processing of a group?

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Using different backends

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

>> Anyone willing to try it out and report findings?  The manual needs
>> to be fixed.
>
> (setq gnus-secondary-select-methods 
>       '((nnfolder "" 
>           (mail-sources '((file :plugged t) ; get mail from system mailbox
                          ^bogus?
>                           (directory :path "~/Mail/incoming-nnfolder"
>                                      :suffix ""
>                                      :plugged t))))
>         (nnml "" 
>           (mail-sources '((directory :path "~/Mail/incoming-nnml"
                          ^bogus?
>                                      :suffix ""
>                                      :plugged t))))))
>
> Indeed, this works from what I can tell after some tests. 

But the inner quotes look bogus.  Please try to remove them.

> But, why is it much slower than the approach in my previous mail? That
> one needs around 1 second, this one now takes around 7 seconds (for 20
> nnml and 30 nnfolder groups).
>
> Does this different scope of 'mail-sources' perhaps introduce an
> overhead that slows down processing of a group?

It could be that Gnus is trying a mail-source `quote' which is
failing.
-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: Using different backends

[Michael Teichgräber]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

>> Indeed, this works from what I can tell after some tests. 
>
> But the inner quotes look bogus.  Please try to remove them.

Yes, they were bogus (furthermore, Gnus didn't start anymore).

Now it is really fast (again), but Gnus does check for nnfolder only
once, at the beginning. Apart from that, global mail-sources value is
set to

   ((directory :path "~/Mail/incoming-nnml" :suffix "" :plugged t)) ,

i.e. the 'nnml'-setting, after the Group buffer appeared.

Since 'nnml' is the last entry in my list of secondary select
methods, 'mail-sources' has perhaps a static value after the first
getting of new mail and news (that seems to be when Gnus is starting).

It looks suspicious to me, that the mail-sources settings within
g-s-s-m affect the global value. If I set mail-sources to nil in the
init file, it is overwritten by the value above, so it has no
buffer-local binding. Hm.

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

mail-sources backend specific? (was: Using different backends)

[Michael Teichgräber]

[-- Attachment #1: Type: text/plain, Size: 3975 bytes --]

> It looks suspicious to me, that the mail-sources settings within
> g-s-s-m affect the global value. If I set mail-sources to nil in the
> init file, it is overwritten by the value above, so it has no
> buffer-local binding. Hm.

<some investigation about what's going on>

List elements after the server name (=defs) within an select-method
are evaluated (for nnfolder, nnml) in nnoo.el:nnoo-change-server by
these lines:

      (unless (cdr (assoc server (cddr bstate)))
	(while (setq def (pop defs))
	  (unless (assq (car def) bvariables)
	    (nconc bvariables
		   (list (cons (car def) (and (boundp (car def))
					      (symbol-value (car def)))))))
	  (if (equal server "*internal-non-initialized-backend*")
	      (set (car def) (symbol-value (cadr def)))
	    (set (car def) (cadr def)))))

'bvariables' contain a part of 'nnoo-definition-alist' that is related
to current backend. Its change will alter nnoo-definition-alist.

According to this code, if I specify

 (setq gnus-secondary-select-methods
       '((nnfolder ""
           (mail-sources ((file :plugged t) ; get mail from system mailbox
                          (directory :path "~/Mail/incoming-nnfolder"
                                     :suffix ""
                                     :plugged t))))
         (nnml "" 
           (mail-sources ((directory :path "~/Mail/incoming-nnml"
                                     :suffix ""
                                     :plugged t))))))

and mail-sources is nil in the beginning, the first call for backend
'nnfolder' will add 

    (list (cons mail-sources nil))  to bvariables.

When nnoo-change-server is then called for backend 'nnml',
mail-sources has the value that was set by the nnfolder-backend before,
therefore

    (list (cons mail-sources ((file :plugged t)(directory :path...))))

is added to the nnml-part of nnoo-definition-alist:

  ((nnfolder nil ((nnfolder-* . *)
                  (mail-sources)))
   (nnml nil     ((nnml-* . *)
                  (mail-sources (file :plugged t)
                                (directory :path "~/Mail/incoming-nnfolder"
                                           :suffix "" :plugged t))))
   (...))


This is not what has been expected.


<alternative approach>

If I understand it right, the intention of the code in
nnoo-change-server (listed above) is to provide a means to add
variable-value pairs to the list of server variables of a backend.

Would it break the design, if mail-sources was handled as server
variable too, so that -- depending on backend -- "nn*-mail-sources"
could be specified?

nndiary, for instance, is using nndiary-mail-sources instead of
mail-sources.

As an example, the following patch makes nnmail-get-new-mail use
              <backend>-mail-sources 
rather than   mail-sources:

------------------------------------------------------------>8----------
Index: nnmail.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnmail.el,v
retrieving revision 6.74
diff -u -p -r6.74 nnmail.el
--- nnmail.el	17 Apr 2003 19:01:45 -0000	6.74
+++ nnmail.el	3 May 2003 15:30:23 -0000
@@ -1646,7 +1646,8 @@ See the Info node `(gnus)Fancy Mail Spli
 (defun nnmail-get-new-mail (method exit-func temp
 				   &optional group spool-func)
   "Read new incoming mail."
-  (let* ((sources (or mail-sources
+  (let* ((sources (or (nnmail-get-value "%s-mail-sources" method)
+		      mail-sources
 		      (if (listp nnmail-spool-file) nnmail-spool-file
 			(list nnmail-spool-file))))
 	 fetching-sources
----------8<------------------------------------------------------------


It seems to work, if I change mail-sources to nnfolder-mail-sources
resp. nnml-mail-sources in my g-s-s-m definition.

This would also allow the use of a global mail-sources with, for
instance, a different one for backend 'nnfolder'.

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Using different backends

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> Now it is really fast (again), but Gnus does check for nnfolder only
> once, at the beginning.

What does this mean?  I keep trying to parse this, but fail :-(
Sorry.
-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: mail-sources backend specific?

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> This is not what has been expected.

What was expected?

I must be having a blonde day today.

> <alternative approach>
>
> Would it break the design, if mail-sources was handled as server
> variable too, so that -- depending on backend -- "nn*-mail-sources"
> could be specified?

That sounds like a plan.  For completeness, some defvoos might have
to be added, but it looks quite nice.

But OTOH, server variables may be useful for other things, so what do
you think about fixing the server variable handling?

-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: Using different backends

[Michael Teichgräber]

kai.grossjohann@gmx.net (Kai Großjohann) writes:

>> Now it is really fast (again), but Gnus does check for nnfolder only
>> once, at the beginning.
>
> What does this mean?  I keep trying to parse this, but fail :-(

When I used the lines containing the bogus quotes, gnus was slow when
getting mail with 'g'. Without the quotes, it is as fast as before.

Then, I expected Gnus to check for local mail waiting for both
nnfolder _and_ nnml groups, when pressing 'g'. Instead, it seemed to
get mail for nnml groups only. Mails available for the nnfolder
backend were recognized only _directly after Gnus' invocation_, when
the Group buffer first comes up, not when pressing 'g' later.

> Sorry.

No, I'm sorry.

Re: mail-sources backend specific?

[Michael Teichgräber]

kai.grossjohann@gmx.net (Kai Großjohann) writes:

>> This is not what has been expected.
>
> What was expected?

s/expected/desired/

It worked as it was told, but it did not result in the behaviour I
wanted: retrieving local mail for more than one backend.

And, the "mail-sources" entries, that had been inserted into
nnoo-definition-alist, looked strange. This was because the
"mail-sources" parameter of the select-method (obviously) didn't have
a nnfolder- or nnml- prefix (as for other server variables). So the
first call of nnoo-change-server (for backend nnml) could influence
the later call (for backend nnfolder) by means of the current
global "mail-sources" value.


>> <alternative approach>
>>
>> Would it break the design, if mail-sources was handled as server
>> variable too, so that -- depending on backend -- "nn*-mail-sources"
>> could be specified?
>
> That sounds like a plan.  For completeness, some defvoos might have
> to be added, but it looks quite nice.

E.g. (defvoo nn*-mail-sources nil) for all backends that retrieve
mail: nnmh, nnml, nnfolder, nnimap ...


> But OTOH, server variables may be useful for other things, so what do
> you think about fixing the server variable handling?

With 'fixing' do you mean making it work with variable-value pairs as
desired some mails before? -- Such as

    (setq gnus-secondary-select-methods 
	  '((nnfolder "" (A X)))
	    (nnml "" (A Y))) ,

so that A (not having nnfolder-, nnml- prefixes) may have different
values in contexts of these backends? Or did you refer to my "is not
what has been expected"?

Re: Using different backends

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> Then, I expected Gnus to check for local mail waiting for both
> nnfolder _and_ nnml groups, when pressing 'g'. Instead, it seemed to
> get mail for nnml groups only. Mails available for the nnfolder
> backend were recognized only _directly after Gnus' invocation_, when
> the Group buffer first comes up, not when pressing 'g' later.

Weird.

Do you think you can fix it?  If you can't, maybe the
nnchoke-mail-sources approach you suggested would be better.

Opinions?
-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: mail-sources backend specific?

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> kai.grossjohann@gmx.net (Kai Großjohann) writes:
>
>>> This is not what has been expected.
>>
>> What was expected?
>
> s/expected/desired/
>
> It worked as it was told, but it did not result in the behaviour I
> wanted: retrieving local mail for more than one backend.
>
> And, the "mail-sources" entries, that had been inserted into
> nnoo-definition-alist, looked strange. This was because the
> "mail-sources" parameter of the select-method (obviously) didn't have
> a nnfolder- or nnml- prefix (as for other server variables). So the
> first call of nnoo-change-server (for backend nnml) could influence
> the later call (for backend nnfolder) by means of the current
> global "mail-sources" value.

Hm.

But what happens if you have two nnml backends and in both you set a
variable (a parameter?) nnml-foo?  Then the first can influence the
second in the same way, no?

Is there a more serious bug here?

>>> <alternative approach>
>>>
>>> Would it break the design, if mail-sources was handled as server
>>> variable too, so that -- depending on backend -- "nn*-mail-sources"
>>> could be specified?
>>
>> That sounds like a plan.  For completeness, some defvoos might have
>> to be added, but it looks quite nice.
>
> E.g. (defvoo nn*-mail-sources nil) for all backends that retrieve
> mail: nnmh, nnml, nnfolder, nnimap ...

Yep.

>> But OTOH, server variables may be useful for other things, so what do
>> you think about fixing the server variable handling?
>
> With 'fixing' do you mean making it work with variable-value pairs as
> desired some mails before? -- Such as
>
>     (setq gnus-secondary-select-methods 
> 	  '((nnfolder "" (A X)))
> 	    (nnml "" (A Y))) ,
>
> so that A (not having nnfolder-, nnml- prefixes) may have different
> values in contexts of these backends? Or did you refer to my "is not
> what has been expected"?

What I meant is to make 

(setq gnus-secondary-select-methods
      '((nnfolder "" (mail-sources foo))
        (nnml "" (mail-sources bar))))

behave as you (and I) desire: fetch mail from foo for the nnfolder
backend and from bar for the nnml backend.

Maybe it's enough to make the variables local to the current buffer?
But probably it isn't -- Gnus jumps around the buffers a lot.

Maybe it works to let-bind the variables.

-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: mail-sources backend specific?

[Michael Teichgräber]

kai.grossjohann@gmx.net (Kai Großjohann) writes:

> But what happens if you have two nnml backends and in both you set a
> variable (a parameter?) nnml-foo?  Then the first can influence the
> second in the same way, no?

In such a case you would choose a different server name (instead of
"") for the second backend, wouldn't you? So there wouldn't be a
problem.

(If you use the Agent, you already are using multiple nnml backends.)


> Is there a more serious bug here?

No, it works very well. ;-)

From what I have learned so far, each backend has a current
server. There are global variables nn<backend>-* that represent the
settings of this current server. These variables coexist with nn*-*
variables of other backends without problems because of different
"nn*-" prefixes.

If the server changes for a certain backend foo from current server
bar to new server baz (e.g. to read mail from baz now),
nnoo-push-server stores all global variables nnfoo-* (that have an
entry in nnoo-definition-alist) into the sub-list within
nnoo-state-alist that corresponds to bar. Then the values of server
baz are extracted from nnoo-state-alist and stored into the global
variables nnfoo-*.

Since global variables of current backend-server pairs must
coexist (they do if being prefixed properly), ...

>>>> <alternative approach>
>>> But OTOH, server variables may be useful for other things, so what do
>>> you think about fixing the server variable handling?
>>
>> [...]
>
> What I meant is to make 

> (setq gnus-secondary-select-methods
>       '((nnfolder "" (mail-sources foo))
>         (nnml "" (mail-sources bar))))
>
> behave as you (and I) desire: fetch mail from foo for the nnfolder
> backend and from bar for the nnml backend.

... ^^^ this will be difficult to achieve, as there cannot be
different "mail-sources" variables in the same (global) namespace. :-(


> Maybe it's enough to make the variables local to the current buffer?
> But probably it isn't -- Gnus jumps around the buffers a lot.

Even if servers were changed only in the Group buffer, there still
would be the problem of variables of current backend-server pairs
having to coexist.


But, nnoo-change-server could be extended to automagically
provide proper prefixes for variable names that do not start with
"nn[^-]+-". A "mail-sources" would then be translated to
"nnfolder-mail-sources" for backend "nnfolder". This would allow the
init file to look like

------------------------------------------------------------>8----------
(setq gnus-secondary-select-methods
      '((nnfolder ""
                  (mail-sources ((file :plugged t)
                                 (directory :path "~/Mail/incoming-nnfolder"
                                            :suffix ""
                                            :plugged t)))
                  (directory "~/Mail/nnfolder")
                  (active-file "~/Mail/nnfolder/active")
                  (get-new-mail t))

        (nnml ""
              (mail-sources ((directory :path "~/Mail/incoming-nnml"
                                        :suffix ""
                                        :plugged t)))
              (directory "~/Mail/nnml")
              (active-file "~/Mail/nnml/active")
              (get-new-mail t)))))
----------8<------------------------------------------------------------

The same for gnus-message-archive-method. If have tested this with a
patch to nnoo.el, and it seems to work (it's practically the same as
specifying nnfolder- resp. nnml-).

Opinions?

Re: mail-sources backend specific?

[Kai Großjohann]

Michael Teichgräber <mt@wmipf.in-berlin.de> writes:

> kai.grossjohann@gmx.net (Kai Großjohann) writes:
>
>> But what happens if you have two nnml backends and in both you set a
>> variable (a parameter?) nnml-foo?  Then the first can influence the
>> second in the same way, no?
>
> In such a case you would choose a different server name (instead of
> "") for the second backend, wouldn't you? So there wouldn't be a
> problem.
>
> (If you use the Agent, you already are using multiple nnml backends.)

You were giving an example with nnfolder and nnml.  And you set that
nnoo sets the variables globally.  So if the nnfolder server sets
variable mail-sources, and nnml sets the same variable, then the two
settings will step on each other's toes, because the name is the same.

Now if you have two nnml servers and both servers set
nnml-mail-sources, then you have the same stepping-on-toes problem.

But this is just my understanding of what you said previously, and I
think that understanding is way off.

I'll read your original message again.

>> Is there a more serious bug here?
>
> No, it works very well. ;-)
>
> From what I have learned so far, each backend has a current
> server. There are global variables nn<backend>-* that represent the
> settings of this current server. These variables coexist with nn*-*
> variables of other backends without problems because of different
> "nn*-" prefixes.
>
> If the server changes for a certain backend foo from current server
> bar to new server baz (e.g. to read mail from baz now),
> nnoo-push-server stores all global variables nnfoo-* (that have an
> entry in nnoo-definition-alist) into the sub-list within
> nnoo-state-alist that corresponds to bar. Then the values of server
> baz are extracted from nnoo-state-alist and stored into the global
> variables nnfoo-*.
>
> Since global variables of current backend-server pairs must
> coexist (they do if being prefixed properly), ...

Hm.  So does this mean that nnoo treats nnchoke-foo variables
differently from frumple-foo variables, because nnchoke is the name
of the backend and frumple isn't?

>>>>> <alternative approach>
>>>> But OTOH, server variables may be useful for other things, so what do
>>>> you think about fixing the server variable handling?
>>>
>>> [...]
>>
>> What I meant is to make 
>
>> (setq gnus-secondary-select-methods
>>       '((nnfolder "" (mail-sources foo))
>>         (nnml "" (mail-sources bar))))
>>
>> behave as you (and I) desire: fetch mail from foo for the nnfolder
>> backend and from bar for the nnml backend.
>
> ... ^^^ this will be difficult to achieve, as there cannot be
> different "mail-sources" variables in the same (global) namespace. :-(

Okay.  Well.  Hm.  Err.

Suppose you have this situation:

(setq gnus-secondary-select-methods
      '((nnml "one" (nnml-mail-sources foo))
        (nnml "two" (nnml-mail-sources bar))))

Clearly there must be a mechanism that prevents the two settings for
nnml-mail-sources stomping on each other.

So can't this mechanism be used to also protect mail-sources, instead
of only nnml-mail-sources?

>> Maybe it's enough to make the variables local to the current buffer?
>> But probably it isn't -- Gnus jumps around the buffers a lot.
>
> Even if servers were changed only in the Group buffer, there still
> would be the problem of variables of current backend-server pairs
> having to coexist.
>
>
> But, nnoo-change-server could be extended to automagically
> provide proper prefixes for variable names that do not start with
> "nn[^-]+-". A "mail-sources" would then be translated to
> "nnfolder-mail-sources" for backend "nnfolder". This would allow the
> init file to look like
>
> ------------------------------------------------------------>8----------
> (setq gnus-secondary-select-methods
>       '((nnfolder ""
>                   (mail-sources ((file :plugged t)
>                                  (directory :path "~/Mail/incoming-nnfolder"
>                                             :suffix ""
>                                             :plugged t)))
>                   (directory "~/Mail/nnfolder")
>                   (active-file "~/Mail/nnfolder/active")
>                   (get-new-mail t))
>
>         (nnml ""
>               (mail-sources ((directory :path "~/Mail/incoming-nnml"
>                                         :suffix ""
>                                         :plugged t)))
>               (directory "~/Mail/nnml")
>               (active-file "~/Mail/nnml/active")
>               (get-new-mail t)))))
> ----------8<------------------------------------------------------------
>
> The same for gnus-message-archive-method. If have tested this with a
> patch to nnoo.el, and it seems to work (it's practically the same as
> specifying nnfolder- resp. nnml-).
>
> Opinions?

That's a cool trick.  But it still requires variables
nnchoke-mail-sources.  So your previous patch of frobbing
nnchoke-get-new-mail is still necessary, I believe.  Not sure if that
helps at all.

-- 
file-error; Data: (Opening input file no such file or directory ~/.signature)

Re: mail-sources backend specific?

[Paul Jarc]

kai.grossjohann@gmx.net (Kai Großjohann) wrote:
> Suppose you have this situation:
>
> (setq gnus-secondary-select-methods
>       '((nnml "one" (nnml-mail-sources foo))
>         (nnml "two" (nnml-mail-sources bar))))
>
> Clearly there must be a mechanism that prevents the two settings for
> nnml-mail-sources stomping on each other.

Yes: the nnml backend has the notion of a "current server", and only
one setting is in effect at a time.  nnoo is used to set
nnml-mail-sources, according to the server name ("one" or "two").

> So can't this mechanism be used to also protect mail-sources, instead
> of only nnml-mail-sources?

No, because the mechanism is tied to the backend's notion of its
current server.  If two backends are involved, there are two current
servers, so variables must have different names to avoid interference.

nnmaildir does not use nnoo, and its server parameters are not
prefixed with "nnmaildir-".  It does not use global variables to
represent its server parameters.  So there is no problem here:
(setq gnus-secondary-select-methods
      '((nnmaildir "one" (directory "~/one"))
        (nnmaildir "two" (directory "~/two"))))
This works because nnmaildir does not set "directory" as a global
variable; instead, it looks up the string keyed by the symbol
'directory in the select method.  (Actually, the information in the
select method is copied into a more efficient vector, but that's not
important.  The point is that private, anonymous data structures are
used instead of global variables.)  The same could be made to work
with nnoo-derived backends, but they would not be able to refer to
such parameters as if they were global variables.


paul

Re: mail-sources backend specific?

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> nnmaildir does not use nnoo, and its server parameters are not
> prefixed with "nnmaildir-".  It does not use global variables to
> represent its server parameters.  So there is no problem here:

Well, err, it sounds as if the mail-sources trick wouldn't work for
nnmaildir in that case.

Sounds like a rather complicated situation.  Maybe we could introduce
something like gnus-group-find-parameter (or was it
gnus-group-get-parameter?) for server `variables'?  Then all places
would say (mumble 'mail-sources) instead of mail-sources, and mumble
would know to look in the server first, and then in the global
variable.

But that would require large-scale changing of the source.

Hm.

Oh!  There is already a mechanism for local variables -- the
buffers.  So do people think it would work to set the variables local
to the right buffers?

Another possibility might be to wrap calls to interesting functions
in a macro which constructs a let form that binds all variables to
the server parameters, if server parameters do indeed exist.

Complicated.  I don't like it.  But I think it's just the current
design.  Is there a chance for making it more sane?

I think what Gnus needs is an OOPS.
-- 
OOA: object-oriented analysis
OOD: object-oriented design
OOPS: object-oriented mistakes

Re: mail-sources backend specific?

[Paul Jarc]

kai.grossjohann@gmx.net (Kai Großjohann) wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> nnmaildir does not use nnoo, and its server parameters are not
>> prefixed with "nnmaildir-".  It does not use global variables to
>> represent its server parameters.  So there is no problem here:
>
> Well, err, it sounds as if the mail-sources trick wouldn't work for
> nnmaildir in that case.

It's not such a big deal for that particular variable, since you can
safely deliver messages directly to nnmaildir groups.

> Oh!  There is already a mechanism for local variables -- the
> buffers.  So do people think it would work to set the variables local
> to the right buffers?

What buffer is nnchoke-request-scan called from?  That's where
mail-sources will be used, I think.

> Complicated.  I don't like it.  But I think it's just the current
> design.  Is there a chance for making it more sane?

I still think the situation could be greatly improved by moving the
work of nnmail-get-new-mail out of backends and into Gnus proper.  It
would be a lot of work, but it could be done with minimal changes to
existing configurations while enabling all the flexibility we'd like.


paul

Re: mail-sources backend specific?

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> I still think the situation could be greatly improved by moving the
> work of nnmail-get-new-mail out of backends and into Gnus proper.  It
> would be a lot of work, but it could be done with minimal changes to
> existing configurations while enabling all the flexibility we'd like.

That seems to be the clean design.  I think Lars didn't do that for
efficiency reasons.

How does the current implementation arrange for things to be faster,
and could we adapt it for the clean design?
-- 
This line is not blank.

Re: mail-sources backend specific?

[Paul Jarc]

kai.grossjohann@gmx.net (Kai Großjohann) wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> I still think the situation could be greatly improved by moving the
>> work of nnmail-get-new-mail out of backends and into Gnus proper.  It
>> would be a lot of work, but it could be done with minimal changes to
>> existing configurations while enabling all the flexibility we'd like.
>
> That seems to be the clean design.  I think Lars didn't do that for
> efficiency reasons.
>
> How does the current implementation arrange for things to be faster,
> and could we adapt it for the clean design?

The current implementation gets its efficiency by passing a list of
groups to the backend function.  The backend then adds the article to
all of those groups; this can use hard linking or any other speedups
appropriate for the backend in question.

But we can simply make a variant of nnchoke-request-accept-article
that takes a list of groups, and get the same speedup.  There's no
need for a separate function nnchoke-active-number to generate the
next article number first; that's already done in
nnchoke-request-accept-article, and could be done similarly in the
multi-group variant.  There's no need for nnchoke-request-scan to be
responsible for looking at mail-sources and nnmail-split-fancy; that
can be done by Gnus, just before or after calling
nnchoke-request-scan.


paul

Re: mail-sources backend specific?

[Lars Magne Ingebrigtsen]

prj@po.cwru.edu (Paul Jarc) writes:

> The current implementation gets its efficiency by passing a list of
> groups to the backend function.  The backend then adds the article to
> all of those groups; this can use hard linking or any other speedups
> appropriate for the backend in question.

Well, the main speed-up is that since the backend controls the
splitting, it knows what it can cache and what it can't.  For
instance, nnml splitting only writes the .overview files once, no
matter how many articles arrive to each group.

But, as you say, you may do it the other way around -- do all the
splitting, and then feed a list of (say) buffer/group names to a new
backend function that puts all those buffers in all those groups.
(And sort the thing by backend, so that backend X gets all its
articles, then backend Y, etc.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen