setup for imap server with many users

setup for imap server with many users

[Gour]

Hello!

I fetch mail from several imap accounts  with getmail and deliver to
dovecot running on my localhost using maildrop.

Now I'd like to configure Gnus to access this email which is stored in
~/dovecot/user1@domain1.com
~/dovecot/user2@domain2.com etc.

Some parts from my gnus-init.el:

(setq gnus-secondary-select-methods
        '((nnimap "localhost-IMAP" 
                  (nnimap-address "localhost")
                  (nnimap-stream network)
                  (nnimap-authinfo-file "~/.authinfo")
                  (nnir-search-engine imap)) 


but I've problem configuring gnus-parameters.

Tried with something like:

("^nnimap\\+localhost-IMAP:user1@domain1.com.*"

but cannot figure out the correct regex to select the above-mentioned
virtual dovecot users?

Moreover, authorization with ~/.authinfo does not work as well.

Here is the entry in it:

machine localhost login user1@domain1.com password somepasswd port 143

I use Emacs/Gnus from Debain Wheezy (23.4.1)

Any hint?


Sincerely,
Gour

-- 
The working senses are superior to dull matter; mind is higher 
than the senses; intelligence is still higher than the mind; 
and he [the soul] is even higher than the intelligence.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

Re: setup for imap server with many users

[Abramov Aleksey]

Gour <gour@atmarama.net> writes:

> Hello!
>
> I fetch mail from several imap accounts  with getmail and deliver to
> dovecot running on my localhost using maildrop.
>
> Now I'd like to configure Gnus to access this email which is stored in
> ~/dovecot/user1@domain1.com
> ~/dovecot/user2@domain2.com etc.
>
> Some parts from my gnus-init.el:
>
> (setq gnus-secondary-select-methods
>         '((nnimap "localhost-IMAP" 
>                   (nnimap-address "localhost")
>                   (nnimap-stream network)
>                   (nnimap-authinfo-file "~/.authinfo")
>                   (nnir-search-engine imap)) 
>
>
> but I've problem configuring gnus-parameters.
>
> Tried with something like:
>
> ("^nnimap\\+localhost-IMAP:user1@domain1.com.*"
>
> but cannot figure out the correct regex to select the above-mentioned
> virtual dovecot users?
>
> Moreover, authorization with ~/.authinfo does not work as well.
>
> Here is the entry in it:
>
> machine localhost login user1@domain1.com password somepasswd port 143
>
> I use Emacs/Gnus from Debain Wheezy (23.4.1)
>
> Any hint?
>
>
> Sincerely,
> Gour

I do not recommend this scheme for you. It will be too slow, that you
will run away from Gnus after few months. 

I think It would be better to set Maildir backend in you delivery agent,
and then use it by Gnus. But the best decision (as I see it) is to set
`mail-sources' variable, and use `nnml' as your back-end.

,----[ C-h v mail-sources RET ]
| mail-sources is a variable defined in `mail-source.el'.
| Its value is
| [deleted]
|
| Original value was ((file))
| 
| 
| Documentation:
| Where the mail backends will look for incoming mail.
| This variable is a list of mail source specifiers.
| See Info node `(gnus)Mail Source Specifiers'.
| 
| You can customize this variable.
| 
| This variable was introduced, or its default value was changed, in
| version 23.1 of Emacs.
| 
| [back]
`----

-- 
Sincerely,
Abramov Aleksey.

Re: setup for imap server with many users

[Gour]

On Wed, 16 Jan 2013 10:05:58 +0400
Abramov Aleksey <levenson@mmer.org> wrote:


> I do not recommend this scheme for you. It will be too slow, that you
> will run away from Gnus after few months. 

Really?

I was running Gnus in such setup in the past and it was OK except some
slowness when fetching nntp groups.

> I think It would be better to set Maildir backend in you delivery
> agent, and then use it by Gnus. 

This is also interesting 'cause I was always recommended to use imap
backend over maildir due to the latter being buggy.

> But the best decision (as I see it) is to set `mail-sources' variable,
> and use `nnml' as your back-end.

I've found the following in docs: "The nnml spool mail format isn't
compatible with any other known format. It should be used with some
caution." which does not look well?


Sincerely,
Gour


-- 
An intelligent person does not take part in the sources of misery, 
which are due to contact with the material senses. O son of Kuntī, 
such pleasures have a beginning and an end, and so the wise man does 
not delight in them.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

-- 
What is night for all beings is the time of awakening 
for the self-controlled; and the time of awakening for 
all beings is night for the introspective sage.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810



_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: setup for imap server with many users

[William Gardella]

Gour <gour-XQYBc/g6bIvk1uMJSBkQmQ@public.gmane.org> writes:

> Any hint?

AFAIK, the Emacs `auth-source' library used by Gnus does not understand
multiple users on the same domain, so it will be very tricky to use a
local IMAP configuration like the one you're describing.  Auth-source
expects a 1:1 correspondence between users and domains.

If, as I infer from your description of your local Dovecot setup, all of
these IMAP users are you, you'd probably be better off having all of
your remote IMAP accounts go to a single inbox on the Dovecot server.
You can use (info "(gnus) Posting Styles") to make sure that replies are
handled correctly for each remote IMAP user and mail-splitting or
server-side IMAP folders to make the deluge of mail easier to manage.
Multiple virtual users on Dovecot to handle one person's mail is not a
great solution.

Hope this helps,
WGG

Re: setup for imap server with many users

[Gour]

On Wed, 16 Jan 2013 17:52:15 +0000
William Gardella <gardellawg@gmail.com>
wrote:

> AFAIK, the Emacs `auth-source' library used by Gnus does not
> understand multiple users on the same domain, so it will be very
> tricky to use a local IMAP configuration like the one you're
> describing.  Auth-source expects a 1:1 correspondence between users
> and domains.

Hmm...I'd bet that I had such setup working normally in the past except
that instead of having:

user1@domain1.com
user2@domain2.com
...

I was having just: 

user1
user2
...

> If, as I infer from your description of your local Dovecot setup, all
> of these IMAP users are you, you'd probably be better off having all
> of your remote IMAP accounts go to a single inbox on the Dovecot
> server. 

There are actually three users with 6 accounts - 4 are mine + my wife &
mother.

> You can use (info "(gnus) Posting Styles") to make sure that
> replies are handled correctly for each remote IMAP user and
> mail-splitting or server-side IMAP folders to make the deluge of mail
> easier to manage. 

I was using posting styles in the past and have them setup now, but the
problem is selecting right IMAP folders.

Too bad, I haven't preserve the whole old setup, but will try to google
to find posts on which it was based...there were many posts about using
Gnus+local dovecot instead of using (buggy) maildir back-end.


Sincerely,
Gour


-- 
One who restrains his senses, keeping them under full control, 
and fixes his consciousness upon Me, is known as a man of 
steady intelligence.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

Re: setup for imap server with many users

[William Gardella]

Gour <gour-XQYBc/g6bIvk1uMJSBkQmQ@public.gmane.org> writes:

> On Wed, 16 Jan 2013 17:52:15 +0000
> William Gardella <gardellawg-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>
>> AFAIK, the Emacs `auth-source' library used by Gnus does not
>> understand multiple users on the same domain, so it will be very
>> tricky to use a local IMAP configuration like the one you're
>> describing.  Auth-source expects a 1:1 correspondence between users
>> and domains.
>
> Hmm...I'd bet that I had such setup working normally in the past except
> that instead of having:
>
> user1-WNbSnoOWcWBBDgjK7y7TUQ@public.gmane.org
> user2-dL8jOpOjefRBDgjK7y7TUQ@public.gmane.org
> ...
>
> I was having just: 
>
> user1
> user2
> ...

Wow.  Perhaps I'm mistaken, but I've never seen any such setup for
auth-source and have no clue how you would support it.

>> If, as I infer from your description of your local Dovecot setup, all
>> of these IMAP users are you, you'd probably be better off having all
>> of your remote IMAP accounts go to a single inbox on the Dovecot
>> server. 
>
> There are actually three users with 6 accounts - 4 are mine + my wife &
> mother.

All going to your Emacs and Gnus?  It might make more sense to give each
of them their own Emacs initfile and system user.

>> You can use (info "(gnus) Posting Styles") to make sure that
>> replies are handled correctly for each remote IMAP user and
>> mail-splitting or server-side IMAP folders to make the deluge of mail
>> easier to manage. 
>
> I was using posting styles in the past and have them setup now, but the
> problem is selecting right IMAP folders.
>
> Too bad, I haven't preserve the whole old setup, but will try to google
> to find posts on which it was based...there were many posts about using
> Gnus+local dovecot instead of using (buggy) maildir back-end.

FTR, the maildir backend isn't particularly buggy these days.  Also, the
local Dovecot hack for Gnus pertains mostly to an era when nnimap was
too slow for acceptable use, which is no longer really the case.
Particularly with the Agent and (setq gnus-asynchronous t), its
performance is just fine.  If I were you, I'd cut out the Dovecot step
entirely and just use Gnus+nnimap directly to the remote hosts.  It is
much less of a "Rube Goldberg machine" than the setup you currently
have.

Sincerely,
Gour

-- 
I use grml (http://grml.org/)

Re: setup for imap server with many users

[Gour]

On Wed, 16 Jan 2013 19:55:09 +0000
William Gardella <gardellawg@gmail.com>
wrote:

> Wow.  Perhaps I'm mistaken, but I've never seen any such setup for
> auth-source and have no clue how you would support it.

Heh, too bad I do not have time-machine to go in the past to
demonstrate you. :-)

> All going to your Emacs and Gnus?  It might make more sense to give
> each of them their own Emacs initfile and system user.

No. They use Claws-mail which is easier for them.

Moreover, for their minimal usage of computer, there is no sense to
create system account for them...and Claws-mail operates normally in
this setup where I now access mail for each 'user' from corresponding
dovecot's virtual user...so, don't understand fully why it should be
problem for Gnus?

I would like to (again) use Gnus to take advantage of orgmode's support
for it.

> FTR, the maildir backend isn't particularly buggy these days.  

I'm aware some time has passed in between, but posts like this

http://stackoverflow.com/questions/2587291/using-maildir-as-a-backend-for-gnus

has made me think that nnimap is (still) better solution.

> Also, the local Dovecot hack for Gnus pertains mostly to an era when
> nnimap was too slow for acceptable use, which is no longer really the
> case.

Really? 

Do you mean latest Gnus or the one coming with Emacs-24 'cause I'm
running Debian Wheezy and have 23.4.1?

> Particularly with the Agent and (setq gnus-asynchronous t), its
> performance is just fine.  

Thank you. I was not aware of the above setting.

> If I were you, I'd cut out the Dovecot step entirely and just use
> Gnus+nnimap directly to the remote hosts.  

OK, but this still involves Gnus talking to several IMAP servers and
fetching mail for several different users? How the setup should look in
such case?

> It is much less of a "Rube Goldberg machine" than the setup you
> currently have.

Yes, I agree that present (past) setup is a bit convoluted due to
performance issues.


Sincerely,
Gour

-- 
One who is not disturbed in mind even amidst the threefold 
miseries or elated when there is happiness, and who is free 
from attachment, fear and anger, is called a sage of steady mind.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

Re: setup for imap server with many users

[William Gardella]

Gour <gour-XQYBc/g6bIvk1uMJSBkQmQ@public.gmane.org> writes:

> On Wed, 16 Jan 2013 19:55:09 +0000 William Gardella
> <gardellawg-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>
>> Wow.  Perhaps I'm mistaken, but I've never seen any such setup for
>> auth-source and have no clue how you would support it.
>
> Heh, too bad I do not have time-machine to go in the past to
> demonstrate you. :-)
>
>> All going to your Emacs and Gnus?  It might make more sense to give
>> each of them their own Emacs initfile and system user.
>
> No. They use Claws-mail which is easier for them.

Then what does that have to do with Gnus?  I'm confused.

> Moreover, for their minimal usage of computer, there is no sense to
> create system account for them...and Claws-mail operates normally in
> this setup where I now access mail for each 'user' from corresponding
> dovecot's virtual user...so, don't understand fully why it should be
> problem for Gnus?

It's a problem for Gnus because neither auth-source.el nor Gnus supports
such configurations well, in terms of the semantics/design of the
software.  For auth-source it is expected that there is exactly one set
of authentication secrets for each machine+protocol combination.  For
Gnus it is expected that each "select method" (i.e. server definition)
designates a particular machine to connect to, or other backend; Gnus
defers to auth-source for how to log in to that machine, including which
username to use.  The fact that it works in Claws has nothing to do with
getting it working in Gnus :) Hopefully that clarifies the problem.

> I would like to (again) use Gnus to take advantage of orgmode's
> support for it.
>
>> FTR, the maildir backend isn't particularly buggy these days.
>
> I'm aware some time has passed in between, but posts like this
>
> http://stackoverflow.com/questions/2587291/using-maildir-as-a-backend-for-gnus
>
> has made me think that nnimap is (still) better solution.

a) I've rarely found stackoverflow a reliable source on Emacs stuff,
fwiw.  b) As indicated in the answers to that poster, your
maildir-syncing tool would be responsible for synchronizing flags to the
IMAP server.  Gnus does move stuff in maildir and set flags in a
standard way, though.

>> Also, the local Dovecot hack for Gnus pertains mostly to an era when
>> nnimap was too slow for acceptable use, which is no longer really the
>> case.
>
> Really?
>
> Do you mean latest Gnus or the one coming with Emacs-24 'cause I'm
> running Debian Wheezy and have 23.4.1?

I am using Emacs 24 on Debian Sid now but the performance with Emacs 23
connecting via nnimap to 2 remote IMAP servers was perfectly fine for
me.  Certainly not troublesome enough to justify the additional
complexity of having a local IMAP server in between.

As I mentioned, you can also use nnimap+gnus agent to create a
completely offline mailreader.  This setup can be further enhanced by
using (info "(gnus) Batching Agents") if you wish.

>> Particularly with the Agent and (setq gnus-asynchronous t), its
>> performance is just fine.
>
> Thank you. I was not aware of the above setting.
>
>> If I were you, I'd cut out the Dovecot step entirely and just use
>> Gnus+nnimap directly to the remote hosts.
>
> OK, but this still involves Gnus talking to several IMAP servers and
> fetching mail for several different users? How the setup should look
> in such case?

The setup would have each of the several IMAP servers listed in
`gnus-secondary-select-methods', with the authentication secrets for
each in ~/.netrc or ~/.authinfo or ~/.authinfo.gpg or whatever you use
as your auth-source file.

e.g.

(setq gnus-secondary-select-methods
      '((nnimap "imap.somewhere.edu"
        (nnimap-server-port 993)
        (nnimap-stream ssl))
       ((nnimap "imap.somewhereelse.com"
        ...))))

etc. etc. etc.

Best,

WGG

Re: setup for imap server with many users

[Gour]

On Wed, 16 Jan 2013 21:04:03 +0000
William Gardella <gardellawg@gmail.com>
wrote:

> Then what does that have to do with Gnus?  I'm confused.

Well, my mother & wife are going to use Claws, while I want to use
Gnus, but both clients access same Maildir 'pool' served by dovecot.

> It's a problem for Gnus because neither auth-source.el nor Gnus
> supports such configurations well, in terms of the semantics/design
> of the software.  

Hmm...

> For auth-source it is expected that there is
> exactly one set of authentication secrets for each machine+protocol
> combination.  

Now I really wonder what kind of setup I had in the past that simply
worked...

> For Gnus it is expected that each "select method" (i.e.  server
> definition) designates a particular machine to connect to, or other
> backend; Gnus defers to auth-source for how to log in to that machine,
> including which username to use.  

In my gnus-init.el I've:

(setq mail-sources nil)

> The fact that it works in Claws has nothing to do with getting it
> working in Gnus :) Hopefully that clarifies the problem.

Sure...I'm just confused a bit considering it's not very strange to have
several email accounts on different servers and accessing them *all*
within one mail client.

> b) As indicated in the answers to that poster, your maildir-syncing
> tool would be responsible for synchronizing flags to the IMAP server.
> Gnus does move stuff in maildir and set flags in a standard way,
> though.

In my case there is no maildir-syncing tool 'cause I want to keep all my
mail on my localhost machine to which I've physical access.

> I am using Emacs 24 on Debian Sid now but the performance with Emacs
> 23 connecting via nnimap to 2 remote IMAP servers was perfectly fine
> for me.  Certainly not troublesome enough to justify the additional
> complexity of having a local IMAP server in between.

OK.

> The setup would have each of the several IMAP servers listed in
> `gnus-secondary-select-methods', with the authentication secrets for
> each in ~/.netrc or ~/.authinfo or ~/.authinfo.gpg or whatever you use
> as your auth-source file.

I'm trying to put together such setup by following setup described in
posts like: http://www.cataclysmicmutation.com/2010/11/multiple-gmail-accounts-in-gnus/

> (setq gnus-secondary-select-methods
>       '((nnimap "imap.somewhere.edu"
>         (nnimap-server-port 993)
>         (nnimap-stream ssl))
>        ((nnimap "imap.somewhereelse.com"
>         ...))))
> 
> etc. etc. etc.

I've the following in my init file:

(setq gnus-secondary-select-methods
        '((nnimap "ggdasa" 
                  (nnimap-address "localhost")
                  (nnimap-list-pattern ("INBOX" "*"))
                  (nnimap-stream network)
                  (nnimap-authinfo-file "~/.authinfo")
                  (nnir-search-engine imap)) 
          (nntp "news.t-com.hr")))


and the following:

(setq gnus-parameters

;;  ggdasa@gmail.com
("^nnimap\+ggdasa:*"
 (posting-style
  (name "Gour")
  (address "ggdasa@gmail.com")
  (signature-file "~/.getmail/ggdasa.signature"))
  (gcc-self . "nnimap+ggdasa:ggdasa.Sent"))) ; ggdasa@gmail.com


but I always get (tried several variations of regexp):

byte-code: Error in ~/.emacs.d/gnus-init.el: ^nnimap+ggdasa:INBOX

and have no clue what is wrong and/or how to access IMAP folders of the
user 'ggdasa' having email in: ~/dovecot/ggdasa/Maildir/*

Any hint?

I've ~/.authinfo with the credentials to login to this account.


Sincerely,
Gour

-- 
As fire is covered by smoke, as a mirror is covered by dust, 
or as the embryo is covered by the womb, the living entity is 
similarly covered by different degrees of this lust.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810