Agent and IMAP

Agent and IMAP

[Divya Ranjan]

Hello Gnusers,

The documentation on Agent and IMAP configuration seems incomplete, is this because the implementation of it is incomplete in itself? I currently retrieve my mail on Gnus through nnimap, I have the method added to `gnus-agent-auto-agentize-methods`, but I don’t think its working.

What is it that needs to be done? Are there any bugs or previous discussions that I can look at?

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[Greg Troxel]

Divya Ranjan <divya@subvertising.org> writes:

> The documentation on Agent and IMAP configuration seems incomplete, is
> this because the implementation of it is incomplete in itself? I
> currently retrieve my mail on Gnus through nnimap, I have the method
> added to `gnus-agent-auto-agentize-methods`, but I dont
> think its working.

No clue about your query, but I wanted to point out that gmane use
causes spam filtering woes.  Your domain has a DMARC reject policy,
and the message as it arrived did not have a DKIM header from your
domain, nor was it from your domain's SPF record.

Spamassassin says

	*  1.8 DMARC_REJECT DMARC reject policy

and I can't say it's wrong.   I would say that it's not ok to use gmane
to messages whose From: domain has a DMARC policy (other than none).
Yes, that used to be normal...

Re: Agent and IMAP

[Divya Ranjan]

Greg Troxel <gdt@lexort.com> writes:

> No clue about your query, but I wanted to point out that gmane use
> causes spam filtering woes.  Your domain has a DMARC reject policy,
> and the message as it arrived did not have a DKIM header from your
> domain, nor was it from your domain's SPF record.
>
> Spamassassin says
>
> 	*  1.8 DMARC_REJECT DMARC reject policy
>
> and I can't say it's wrong.   I would say that it's not ok to use gmane
> to messages whose From: domain has a DMARC policy (other than none).
> Yes, that used to be normal...

Can you describe more as to what the issue here is? I’m not sure how DMARC affects me, or my engagement with gmane-based newsgroups. With regards to my email domain, subvertising.org is one of the aliases provided by my email service, Autistici/Inventati (autistici.org). What exactly are the consequences of using gmane with domains that have DMARC policy?

Apologies if it has or might cause any inconvenience, I am unaware of this.

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[Greg Troxel]

Divya Ranjan <divya@subvertising.org> writes:

> Greg Troxel <gdt@lexort.com> writes:
>
>> No clue about your query, but I wanted to point out that gmane use
>> causes spam filtering woes.  Your domain has a DMARC reject policy,
>> and the message as it arrived did not have a DKIM header from your
>> domain, nor was it from your domain's SPF record.
>>
>> Spamassassin says
>>
>> 	*  1.8 DMARC_REJECT DMARC reject policy
>>
>> and I can't say it's wrong.   I would say that it's not ok to use gmane
>> to messages whose From: domain has a DMARC policy (other than none).
>> Yes, that used to be normal...
>
> Can you describe more as to what the issue here is? I’m not sure how
> DMARC affects me, or my engagement with gmane-based newsgroups. With
> regards to my email domain, subvertising.org is one of the aliases
> provided by my email service, Autistici/Inventati
> (autistici.org). What exactly are the consequences of using gmane with
> domains that have DMARC policy?

DKIM is a mail standard, by which a domain publishes (in DNS) keys, and
then cryptographically signs outgoing messages, pointing to the keys via
DNS.  With a valid DKIM signature, one can have confidence that a
messsage was emitted by an authorized MTA for that domain.  (This is not
a signatture from the author, just the domain.)  Properly set up domains
(that do DKIM) sign all outgoing mail.

In 2024, domains that are not set up for DKIM are considered deficient
by any, and will have trouble sending to various large mail systems.

With DKIM, one can "welcomelist_from_dkim" and only pass mail that is
not only From: that user but also DKIM signed.  Hence forged mail from
that user won't pass and won't get welcomelisted.

DMARC is a mail standard, by which a domains publishes (in DNS) a policy
about how mail that claims to be from that domain is handled.  The
choices are basically

  don't publish a record
  none
  quarantine
  reject

The first two are more or less equivalent.  For the second two, there is
a concept of passing validation.  There are two ways to pass.  One is to
pass DKIM from that domain.  The other is to pass SPF, which means the
message was delivered from an IP address listed in the SPF record (also
in DNS).   If a message passes, it is supposed to be treated normally.
If a message does not pass, and the policy is quarantine, it is supposed
to be filed as spam.   If a message does not pass, and the policy is
reject, it is supposed to be rejected at the MTA level.

> Apologies if it has or might cause any inconvenience, I am unaware of this.

Don't feel bad -- many people are not aware of this, and it's
complicated stuff.

Your domain -- operated by your provider -- has a DMARC reject policy,
said my spam filter.  That's ok, and normal these days.   But, your
provider should have explained to you that thet *only* way you can send
mail from your domain is via their outgoing mail servers.   Not gmail,
not some other random ISP, and not gmane....

When you send mail via some other server, it will lack a DKIM signature.
And it will not arrive from autistici/inventati servers, as listed in
the SPF record.  Thus the proper response of a receiver is to reject it.

https://mxtoolbox.com/SuperTool.aspx?action=dmarc%3asubvertising.org&run=toolpage#
https://mxtoolbox.com/SuperTool.aspx?action=spf%3asubvertising.org&run=toolpage#

yahoo/aol/verizon is notable for having a dmarc reject policy.   I
increasing expect banks etc. to have that; it protects users from mail
with forged return addresses.

Some will say that everyone should welcomelist mail from lists they are
on.  Perhaps, but I don't think that makes it ok to send non-passing
mail from a DMARC domain.

The painful details:

https://datatracker.ietf.org/doc/html/rfc6376
https://datatracker.ietf.org/doc/rfc7489/

Re: Agent and IMAP

[Divya Ranjan]

Thank you for the details Greg. Now, what do you suggest I can do here? Use another email address for gmane? Or is there something I can ask the mail provider to do from their side?

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[Adam Sjøgren]

If you have access to set DNS records for your domain, I once wrote up
how I add DKIM signatures to my news articles:

 · https://koldfront.dk/adding_dkim_to_news_articles_1617


  Best regards,

    Adam

-- 
 "KOMPRESSOR sound is hard and angry, good words            Adam Sjøgren
  used for song and electronic."                       asjo@koldfront.dk

Re: Agent and IMAP

[Divya Ranjan]

Adam Sjøgren <asjo@koldfront.dk> writes:

> If you have access to set DNS records for your domain, I once wrote up
> how I add DKIM signatures to my news articles:
>
>  · https://koldfront.dk/adding_dkim_to_news_articles_1617
>

Thank you, Adam. I shall refer to it when setting up my self-hosted email.

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[Greg Troxel]

Divya Ranjan <divya@subvertising.org> writes:

> Thank you for the details Greg. Now, what do you suggest I can do
> here? Use another email address for gmane? Or is there something I can
> ask the mail provider to do from their side?

I suggest that you:

  cease using gmane from a domain with a DMARC reject or quarantine
  policy, because it creates mess for recipients

  read the 'how to DKIM sign your news articles' post just sent

  talk to your provider, but keep in mind that almost certainly, from
  their point of view, you using gmane is irregular

  consider whether you'd rather self-host email or stop using gmane
  

Re: Agent and IMAP

[Divya Ranjan]

Greg Troxel <gdt@lexort.com> writes:

> Divya Ranjan <divya@subvertising.org> writes:
>
>> Thank you for the details Greg. Now, what do you suggest I can do
>> here? Use another email address for gmane? Or is there something I can
>> ask the mail provider to do from their side?
>
> I suggest that you:
>
>   cease using gmane from a domain with a DMARC reject or quarantine
>   policy, because it creates mess for recipients
>
>   read the 'how to DKIM sign your news articles' post just sent
>
>   talk to your provider, but keep in mind that almost certainly, from
>   their point of view, you using gmane is irregular
>
>   consider whether you'd rather self-host email or stop using gmane   

Okay, thank you for the information Greg. I’d refrain my engagement here for now. I was planning to eventually host my own email in some time. I guess time has come for that.

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[James Thomas]

Divya Ranjan wrote:

> The documentation on Agent and IMAP configuration seems incomplete, is
> this because the implementation of it is incomplete in itself?

No, because I use it daily.

> I currently retrieve my mail on Gnus through nnimap, I have the method
> added to `gnus-agent-auto-agentize-methods`, but I don’t think its
> working.

Was it 'nnimap that you added to it?

> What is it that needs to be done? Are there any bugs or previous
> discussions that I can look at?

I did it as per (info "(gnus) Agent Basics"):

  Go to the server buffer (‘^’ in the group buffer) and press ‘J a’ on
  the server (or servers) that you wish to have covered by the Agent

--

Re: Agent and IMAP

[Divya Ranjan]

Hello James,

James Thomas <jimjoe@gmx.net> writes:

> No, because I use it daily.

Would you be willing to share your configuration?

> Was it 'nnimap that you added to it?

Yes

> I did it as per (info "(gnus) Agent Basics"):
I also have it set according to that, but then when I look at (info "(gnus) (Gnus Unplugged) Agent and IMAP"), it says the following:

> The Agent works with any Gnus back end, including nnimap.  However, since there are some conceptual differences between NNTP and IMAP, this section (should) provide you with some information to make Gnus Agent work smoother as a IMAP Disconnected Mode client

Let me know more about how you’ve set gnus with IMAP using 'nnimap.

Regards,
-- 
Divya Ranjan,
Philosophy, Mathematics, Libre Software.

Re: Agent and IMAP

[James Thomas]

Divya Ranjan wrote:

> James Thomas writes:
>
>> No, because I use it daily.
>
> Would you be willing to share your configuration?

I don't have any Agent-related configuration. Just do 'J a' on the
nnimap server in the *Server* buffer.

The next time you open the group and read an article, it'll be saved
(the docs of (info "(gnus) Agent Caveats") have recently been updated).
Then you can unplug ('J j') and then enter the group when offline to
read them.

>> Was it 'nnimap that you added to it?
>
> Yes

OK. Though it seems to need only that, I can't help more because I don't
use it that way.

>> I did it as per (info "(gnus) Agent Basics"):
> I also have it set according to that, but then when I look at (info
> "(gnus) (Gnus Unplugged) Agent and IMAP"), it says the following:
>
>> The Agent works with any Gnus back end, including nnimap. However,
>> since there are some conceptual differences between NNTP and IMAP,
>> this section (should) provide you with some information to make Gnus
>> Agent work smoother as a IMAP Disconnected Mode client

Well, "smooth*er*": not that it doesn't work, no? (It's only an
introduction to what follows).

> Let me know more about how you’ve set gnus with IMAP using 'nnimap.

FWIW the only remotely related part of my config is my gnus-daemon
function, which uses Agent features (comments added):

--8<---------------cut here---------------start------------->8---
(defun my/gnus-demon-scan-mail ()
  (gnus-agent-while-plugged
    ;; Only level 3 servers should be agentized, for this.
    (gnus-agent-synchronize-flags)
    ;; Inbox.
    (gnus-group-get-new-news 3 t)
    (with-temp-message ""
      ;; Spam.
      (gnus-group-get-new-news 4 t)
      (gnus-agent-fetch-session))
    (unless (equal (current-message) gnus-no-groups-message)
      ;; Custom alert function.
      (alert))
    (call-interactively #'gnus-group-list-groups)))

(gnus-demon-add-handler 'my/gnus-demon-scan-mail 90 11)
--8<---------------cut here---------------end--------------->8---

...for my workflow: to mark messages (for future downloading as well) if
I'm checking later when offline.

Oh, and I have a 'downloaded mark' in my Summary line:

--8<---------------cut here---------------start------------->8---
(setq gnus-summary-line-format
"%U%R%O %-16&user-date; %3i %*%B%«%-23,23n%»%T: %s\n")
--8<---------------cut here---------------end--------------->8---

Regards,
James

Re: Agent and IMAP

[James Thomas]

James Thomas wrote:

> (defun my/gnus-demon-scan-mail ()
>   (gnus-agent-while-plugged
>     ;; Only level 3 servers should be agentized, for this.
>     (gnus-agent-synchronize-flags)
>     ;; Inbox.
>     (gnus-group-get-new-news 3 t)
>     (with-temp-message ""
>       ;; Spam.
>       (gnus-group-get-new-news 4 t)
>       (gnus-agent-fetch-session))
>     (unless (equal (current-message) gnus-no-groups-message)
>       ;; Custom alert function.
>       (alert))
>     (call-interactively #'gnus-group-list-groups)))

Actually, this is the function I used to use:

--8<---------------cut here---------------start------------->8---

(defun my/gnus-demon-scan-mail ()
  (gnus-agent-while-plugged
    ;; Get marked.
    (gnus-agent-fetch-session)
    (gnus-agent-possibly-synchronize-flags)
    (gnus-group-get-new-news 4 t)
    (with-temp-message ""
      ;; Get new headers.
      (gnus-agent-fetch-session))
    (unless (equal (current-message)
		   gnus-no-groups-message)
      (alert))))

--8<---------------cut here---------------end--------------->8---

I'd removed some stuff that I (wrongly) thought unnecessary. Also, these
are in my .gnus.el:

(setq gnus-agent-synchronize-flags t)
(setq gnus-agent-go-online t)

And the default (info "(gnus) Agent Categories") was changed from
'short' to 'false'.

Just FYI.

Regards,
James