Gnus development mailing list
 help / color / mirror / Atom feed
* on downloading too much data when there's no news
@ 2020-08-06 14:13 Wayne Harris
  2020-08-06 19:47 ` Adam Sjøgren
  0 siblings, 1 reply; 9+ messages in thread
From: Wayne Harris @ 2020-08-06 14:13 UTC (permalink / raw)
  To: ding

Here's my secondary select method.

  (setq gnus-secondary-select-methods 
        '((nntp "news.fidonet.fi" 119)))

I subscribe only the group fidonews.  Every time I check for new news,
about 12 KiB of data is downloaded --- I watch it being downloaded.  But
there's no news at all.  These 12 KiB seem to be due to the group
fidonews.  If I add other groups, the amount of data increases.  So it
seems that Gnus is always downloading a considerable amount of data even
though there is nothing new to download.  Thunderbird doesn't seem to
download almost anything when there's nothing new.

Any ideas as to how news.fidonet.fi is fooling /my/ Gnus?  Thank you.

  GNU Emacs 24.3.1 (i386-mingw-nt6.2.9200) of 2013-03-17 on MARVIN
  Gnus v5.13



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-06 14:13 on downloading too much data when there's no news Wayne Harris
@ 2020-08-06 19:47 ` Adam Sjøgren
  2020-08-06 19:54   ` Adam Sjøgren
  2020-08-07  0:23   ` Wayne Harris
  0 siblings, 2 replies; 9+ messages in thread
From: Adam Sjøgren @ 2020-08-06 19:47 UTC (permalink / raw)
  To: ding

Wayne writes:

> Every time I check for new news, about 12 KiB of data is downloaded

Spying a little with ngrep:

If I don't have any subscriptions on news.fidonet.fi, Gnus issues a LIST
command, which returns all the newsgroups on the server, every time I
check for news. Seems excessive, but ok.

Next test is subscribing to fidonews. When I check news, Gnus then
issues a "LIST ACTIVE fidonews" command. Which also returns a list of
all newsgroups. Huh, I expected it to only return information for the
group fidonews.

Usually "LIST ACTIVE <newsgroup>" only returns the group asked for, I
think. For example, talking to another newsserver, running INN:

  asjo@tullinup:~$ telnet news.gmane.io nntp
  Trying 116.202.254.214...
  Connected to ciao.gmane.io.
  Escape character is '^]'.
  200 news.gmane.io InterNetNews NNRP server INN 2.6.3 ready (posting ok)
  LIST ACTIVE gmane.test
  215 Newsgroups in form "group high low status"
  gmane.test 0000008623 0000000051 m
  .
  QUIT
  205 Bye!
  Connection closed by foreign host.
  asjo@tullinup:~$ 

But news.fidonet.fi returns the full list:

  asjo@tullinup:~$ telnet news.fidonet.fi nntp
  Trying 2001:41d0:401:3100::1030...
  Connected to news.fidonet.fi.
  Escape character is '^]'.
  200 Welcome to SmapiNNTPd/Linux/IPv6 1.3 (posting may be allowed)
  AUTHINFO USER XXX
  381 Received login name, now send password
  AUTHINFO PASS YYY
  281 Authentication accepted
  LIST ACTIVE fidonews
  215 List of newsgroups follows
  fido7.su.pol 41628 1 y
  fido7.su.pol.history 283 1 y
    [...]
  zyxel 0 0 y
  netmail 0 0 y
  +getaccess+ 14 1 y
  +localchat+ 59 1 y
  +localtest+ 104 1 y
  .
  QUIT
  205 Goodbye
  Connection closed by foreign host.
  asjo@tullinup:~$ 

So it looks like it's specific to news.fidonet.fi.

Maybe Thunderbird uses a different method than "LIST ACTIVE <newsgroup>"
to check for news?

Yeah, Thunderbird doesn't use "LIST ACTIVE fidonews", instead it does a
"GROUP fidonews". Which in the case of news.fidonet.fi (SmapiNNTPd/
Linux/IPv6 1.3) makes a difference.

Maybe Gnus should detect that the server doesn't do LIST ACTIVE -
because HELP on it says:

  200 Welcome to SmapiNNTPd/Linux/IPv6 1.3 (posting may be allowed)
  HELP
  100 Help text follows
  Recognized commands:

  ARTICLE
  AUTHINFO
  BODY
  GROUP
  HEAD
  HELP
  IHAVE (not implemented, messages are always rejected)
  LAST
  LIST
  NEWGROUPS (not implemented, always returns an empty list)
  NEWNEWS (not implemented, always returns an empty list)
  NEXT
  QUIT
  SLAVE (has no effect)
  STAT
  XOVER (partially implemented, byte count and line count are always empty)

  SmapiNNTPd supports most of RFC-977 and also has support for AUTHINFO and
  limited XOVER support (RFC-2980)
  .

But now we are deeper into how Gnus works than what I know :-)


  Best regards,

    Adam

-- 
 "Jabba the Hutt meets Rudolf the Reindeer. I dunno,        Adam Sjøgren
  Mom."                                                asjo@koldfront.dk



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-06 19:47 ` Adam Sjøgren
@ 2020-08-06 19:54   ` Adam Sjøgren
  2020-08-07  0:39     ` Wayne Harris
  2020-08-07  0:23   ` Wayne Harris
  1 sibling, 1 reply; 9+ messages in thread
From: Adam Sjøgren @ 2020-08-06 19:54 UTC (permalink / raw)
  To: ding

Adam writes:

> But now we are deeper into how Gnus works than what I know :-)

It looks like you can configure Gnus to use GROUP instead of LIST
ACTIVE:

,----[ C-h v nntp-server-list-active-group RET ]
| nntp-server-list-active-group is a variable defined in ‘nntp.el’.
| Its value is nil
| 
| Documentation:
| If nil, then always use GROUP instead of LIST ACTIVE.
| This is usually slower, but on misconfigured servers that don’t
| update their active files often, this can help.
| 
| This is a Gnus server variable.  See Info node ‘(gnus)Select Methods’.
`----

So maybe try something similar to:

        (nntp "fidonet" 
              (nntp-server-list-active-group nil)
              (nntp-address "news.fidonet.fi"))

in your secondary select methods.

For me that helps, ngrep shows this when fetching news, subscribed to
fidonews:

  asjo@tullinup:~$ sudo ngrep -q -Wbyline host news.fidonet.fi
  interface: wlp82s0 (192.168.1.0/255.255.255.0)
  filter: ( host news.fidonet.fi ) and ((ip || ip6) || (vlan && (ip || ip6)))

  T 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 -> 2001:41d0:401:3100::1030:119 [AP] #1
  GROUP fidonews.


  T 2001:41d0:401:3100::1030:119 -> 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 [AP] #2
  211 3290 1 3290 fidonews Group selected.


  Best regards,

    Adam

-- 
 "Subdued flamboyance"                                      Adam Sjøgren
                                                       asjo@koldfront.dk



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-06 19:47 ` Adam Sjøgren
  2020-08-06 19:54   ` Adam Sjøgren
@ 2020-08-07  0:23   ` Wayne Harris
  1 sibling, 0 replies; 9+ messages in thread
From: Wayne Harris @ 2020-08-07  0:23 UTC (permalink / raw)
  To: ding

Adam Sjøgren <asjo@koldfront.dk> writes:

> Wayne writes:
>
>> Every time I check for new news, about 12 KiB of data is downloaded
>
> Spying a little with ngrep:

[...]

That was a great analysis!  Very grateful.  Thank you.



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-06 19:54   ` Adam Sjøgren
@ 2020-08-07  0:39     ` Wayne Harris
  2020-08-07  1:07       ` Amin Bandali
                         ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Wayne Harris @ 2020-08-07  0:39 UTC (permalink / raw)
  To: ding

Adam Sjøgren <asjo@koldfront.dk> writes:

> Adam writes:
>
>> But now we are deeper into how Gnus works than what I know :-)
>
> It looks like you can configure Gnus to use GROUP instead of LIST
> ACTIVE:
>
> ,----[ C-h v nntp-server-list-active-group RET ]
> | nntp-server-list-active-group is a variable defined in ‘nntp.el’.
> | Its value is nil
> | 
> | Documentation:
> | If nil, then always use GROUP instead of LIST ACTIVE.
> | This is usually slower, but on misconfigured servers that don’t
> | update their active files often, this can help.
> | 
> | This is a Gnus server variable.  See Info node ‘(gnus)Select Methods’.
> `----

How can I build such a beautiful ASCII-sign like that? (Jealous!)

> So maybe try something similar to:
>
>         (nntp "fidonet" 
>               (nntp-server-list-active-group nil)
>               (nntp-address "news.fidonet.fi"))
>
> in your secondary select methods.

That improved my news reading so much.  It's so much faster now.  Thank
you so much for this great show of problem-solving skills.

Now, I'm looking at 

  https://tools.ietf.org/html/rfc3977#section-7.6.3

and it seems news.fidonet.net is not violating it.  The server does not
seem to advertise the READER capability.  I'm not sure how to see all
capabilities.  I suppose news.fidonet.fi just advertises what it says
when we connect and that's it.

--8<---------------cut here---------------start------------->8---
7.6.3.  LIST ACTIVE

   This keyword MUST be supported by servers advertising the READER
   capability.

   LIST ACTIVE returns a list of valid newsgroups and associated
   information.  If no wildmat is specified, the server MUST include
   every group that the client is permitted to select with the GROUP
   command (Section 6.1.1).
--8<---------------cut here---------------end--------------->8---

> For me that helps, ngrep shows this when fetching news, subscribed to
> fidonews:

It helped amazingly!

>
>
>   asjo@tullinup:~$ sudo ngrep -q -Wbyline host news.fidonet.fi
>   interface: wlp82s0 (192.168.1.0/255.255.255.0)
>   filter: ( host news.fidonet.fi ) and ((ip || ip6) || (vlan && (ip || ip6)))
>
>   T 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 -> 2001:41d0:401:3100::1030:119 [AP] #1
>   GROUP fidonews.
>
>
>   T 2001:41d0:401:3100::1030:119 -> 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 [AP] #2
>   211 3290 1 3290 fidonews Group selected.

Why are you using ngrep and not just telnet here?



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-07  0:39     ` Wayne Harris
@ 2020-08-07  1:07       ` Amin Bandali
  2020-08-07  2:45         ` Wayne Harris
  2020-08-07 17:40       ` Russ Allbery
  2020-08-07 17:44       ` Adam Sjøgren
  2 siblings, 1 reply; 9+ messages in thread
From: Amin Bandali @ 2020-08-07  1:07 UTC (permalink / raw)
  To: ding

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

Wayne Harris writes:

> Adam Sjøgren writes:
>
>> Adam writes:
>>
>>> But now we are deeper into how Gnus works than what I know :-)
>>
>> It looks like you can configure Gnus to use GROUP instead of LIST
>> ACTIVE:
>>
>> ,----[ C-h v nntp-server-list-active-group RET ]
>> | nntp-server-list-active-group is a variable defined in ‘nntp.el’.
>> | Its value is nil
>> | 
>> | Documentation:
>> | If nil, then always use GROUP instead of LIST ACTIVE.
>> | This is usually slower, but on misconfigured servers that don’t
>> | update their active files often, this can help.
>> | 
>> | This is a Gnus server variable.  See Info node ‘(gnus)Select Methods’.
>> `----
>
> How can I build such a beautiful ASCII-sign like that? (Jealous!)
>
[...]

That would be boxquote <http://www.davep.org/emacs/#Fboxquote.el>.
It's a super neat little package. :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-07  1:07       ` Amin Bandali
@ 2020-08-07  2:45         ` Wayne Harris
  0 siblings, 0 replies; 9+ messages in thread
From: Wayne Harris @ 2020-08-07  2:45 UTC (permalink / raw)
  To: ding

Amin Bandali <bandali@gnu.org> writes:

> Wayne Harris writes:
>
>> Adam Sjøgren writes:
>>
>>> Adam writes:
>>>
>>>> But now we are deeper into how Gnus works than what I know :-)
>>>
>>> It looks like you can configure Gnus to use GROUP instead of LIST
>>> ACTIVE:
>>>
>>> ,----[ C-h v nntp-server-list-active-group RET ]
>>> | nntp-server-list-active-group is a variable defined in ‘nntp.el’.
>>> | Its value is nil
>>> | 
>>> | Documentation:
>>> | If nil, then always use GROUP instead of LIST ACTIVE.
>>> | This is usually slower, but on misconfigured servers that don’t
>>> | update their active files often, this can help.
>>> | 
>>> | This is a Gnus server variable.  See Info node ‘(gnus)Select Methods’.
>>> `----
>>
>> How can I build such a beautiful ASCII-sign like that? (Jealous!)
>>
> [...]
>
> That would be boxquote <http://www.davep.org/emacs/#Fboxquote.el>.
> It's a super neat little package. :-)

,----[ Loved it! ]
| I will have to agree.  But maybe I'll abuse it so much and nobody will
| want to read my posts anymore. :-)
`----



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-07  0:39     ` Wayne Harris
  2020-08-07  1:07       ` Amin Bandali
@ 2020-08-07 17:40       ` Russ Allbery
  2020-08-07 17:44       ` Adam Sjøgren
  2 siblings, 0 replies; 9+ messages in thread
From: Russ Allbery @ 2020-08-07 17:40 UTC (permalink / raw)
  To: ding

Wayne Harris <wharris1@protonmail.com> writes:

> Now, I'm looking at 

>   https://tools.ietf.org/html/rfc3977#section-7.6.3

> and it seems news.fidonet.net is not violating it.  The server does not
> seem to advertise the READER capability.  I'm not sure how to see all
> capabilities.  I suppose news.fidonet.fi just advertises what it says
> when we connect and that's it.

news.fidonet.fi doesn't support the CAPABILITIES command, so it's not in
compliance with RFC 3977.  It may predate the RFC, or it may not intend to
comply.  It's some NNTP implementation with which I was not previously
familiar:

200 Welcome to SmapiNNTPd/Linux/IPv6 1.3 (posting may be allowed)

It appears to be specific to file formats used by BBS systems and was
originally written in 2003, which does indeed predate RFC 3977.

    https://github.com/ftnapps/smapinntpd

That LIST ACTIVE behavior is unusual.  I've not seen it before.  It
doesn't comply with RFC 2980 either, which documents existing practice
prior to RFC 3977.  It looks like it's ignoring the ACTIVE keyword and
just returning an RFC 977 response to the LIST command, which is
consistent with the HELP output showing that the author appears to be
focused on RFC 977.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: on downloading too much data when there's no news
  2020-08-07  0:39     ` Wayne Harris
  2020-08-07  1:07       ` Amin Bandali
  2020-08-07 17:40       ` Russ Allbery
@ 2020-08-07 17:44       ` Adam Sjøgren
  2 siblings, 0 replies; 9+ messages in thread
From: Adam Sjøgren @ 2020-08-07 17:44 UTC (permalink / raw)
  To: ding

Wayne writes:

>>   asjo@tullinup:~$ sudo ngrep -q -Wbyline host news.fidonet.fi
>>   interface: wlp82s0 (192.168.1.0/255.255.255.0)
>>   filter: ( host news.fidonet.fi ) and ((ip || ip6) || (vlan && (ip || ip6)))
>>
>>   T 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 -> 2001:41d0:401:3100::1030:119 [AP] #1
>>   GROUP fidonews.
>>
>>
>>   T 2001:41d0:401:3100::1030:119 -> 2a06:4001:d06f:0:909a:cc99:d2b8:3fa6:45732 [AP] #2
>>   211 3290 1 3290 fidonews Group selected.
>
> Why are you using ngrep and not just telnet here?

I wanted to show what Gnus did with the configuration changed.


  Best regards,

    Adam

-- 
 "Some angel whore                                          Adam Sjøgren
  Who can learn a guitar lick                          asjo@koldfront.dk
  Hey, that's what I call muuuusic"



^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2020-08-07 17:45 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-06 14:13 on downloading too much data when there's no news Wayne Harris
2020-08-06 19:47 ` Adam Sjøgren
2020-08-06 19:54   ` Adam Sjøgren
2020-08-07  0:39     ` Wayne Harris
2020-08-07  1:07       ` Amin Bandali
2020-08-07  2:45         ` Wayne Harris
2020-08-07 17:40       ` Russ Allbery
2020-08-07 17:44       ` Adam Sjøgren
2020-08-07  0:23   ` Wayne Harris

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).