Re: Gnus horribly slow after upgrade to Fedora17

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> "Otto J. Makela" <om@iki.fi> writes:
>> For reasons that are beyond me, the Gnus that came with Fedora17
>> (version Gnus/5.13 Emacs/24.1) is terribly slow compared to the earlier
>> one I used with Fedora16 (Gnus/5.13 Emacs/23.3). The "g" command
>> (gnus-group-get-new-news) "nntp read" takes around 20× more time.
>
> Is there a lot of data in the " *server foo... *" buffers after you hit
> `g'?  (Note leading space in the buffer name.)

26881 lines in the buffer " *server news.saunalahti.fi nntp *nntpd**"
which matches what I have in my .newsrc -- when I go into a group it
seems to drop to one line.

This slow-down initially started way back when I upgraded from Fedora
16 to Fedora 17. Initially when gnus tried to open the nntp connection,
it just gave the error message:

nntp (news.saunalahti.fi) open error: 'Error when connecting: 200 Elisa
USENET News Server (Typhoon v2.2.2.503)^M'.  Continue? (y or n)

I got it working by setting the following per a hint I got from the net:

(setq gnus-select-method
 '(nntp "news.saunalahti.fi"
        (nntp-open-connection-function network-only)
))

But ever since the upgrade I've been plagued by slowness.
What can I do to debug where the time is spent?

-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Lars Magne Ingebrigtsen]

om@iki.fi (Otto J. Makela) writes:

> 26881 lines in the buffer " *server news.saunalahti.fi nntp *nntpd**"
> which matches what I have in my .newsrc -- when I go into a group it
> seems to drop to one line.

You read 26 thousand groups?

> But ever since the upgrade I've been plagued by slowness.
> What can I do to debug where the time is spent?

If it's requesting GROUP data from 26K groups, that's probably the
explanation for the slowness...

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> om@iki.fi (Otto J. Makela) writes:
>> 26881 lines in the buffer " *server news.saunalahti.fi nntp *nntpd**"
>> which matches what I have in my .newsrc -- when I go into a group it
>> seems to drop to one line.
>
> You read 26 thousand groups?

No, I only subscribe to about 75 groups, but the news
server I am using has that many groups available.

>> But ever since the upgrade I've been plagued by slowness.
>> What can I do to debug where the time is spent?
>
> If it's requesting GROUP data from 26K groups, that's probably the
> explanation for the slowness...

Very possible. Has the gnus behaviour changed in this respect, earlier
it seemed to only do that for the groups one actually subscribes to?

This message is posted from different machine (and different gnus
version, emacs-23.1-25.el6), with this combination there is no such
slowness, although there are 51516 groups on the server.

-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Lars Magne Ingebrigtsen]

om@iki.fi (Otto J. Makela) writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>
>> om@iki.fi (Otto J. Makela) writes:
>>> 26881 lines in the buffer " *server news.saunalahti.fi nntp *nntpd**"
>>> which matches what I have in my .newsrc -- when I go into a group it
>>> seems to drop to one line.
>>
>> You read 26 thousand groups?
>
> No, I only subscribe to about 75 groups, but the news
> server I am using has that many groups available.

[...]

> Very possible. Has the gnus behaviour changed in this respect, earlier
> it seemed to only do that for the groups one actually subscribes to?

The way Gnus retrieves group info for NNTP groups has changed, but it's
supposed to be faster, not slower.  >"?

What's the value of the `gnus-read-active-file' and
gnus-check-new-newsgroups' variables?

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> om@iki.fi (Otto J. Makela) writes:
>> Very possible. Has the gnus behaviour changed in this respect, earlier
>> it seemed to only do that for the groups one actually subscribes to?
>
> The way Gnus retrieves group info for NNTP groups has changed, but
> it's supposed to be faster, not slower. >"?
>
> What's the value of the `gnus-read-active-file' and
> gnus-check-new-newsgroups' variables?

gnus-read-active-file's value is some
gnus-check-new-newsgroups's value is ask-server

Can I request a bit more granularity in gnus version numbering, since
I have two machines with the "same" gnus version 5.13 but one works as
expected (RHEL6, Emacs/23.1) and the other one (Fedora 19, Emacs/24.3)
is horribly slow?

Of course one alternative is that the "fast" machine is actually doing
just the same work, it just happens to be on a much faster network.

Also, do you have ideas on correcting the nntp connection error
I detailed in my earlier message?

In any case, thank you for maintaining gnus!

-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Anssi Saari]

om@iki.fi (Otto J. Makela) writes:

> Also, do you have ideas on correcting the nntp connection error
> I detailed in my earlier message?

I reported that issue following the discussion last year and Lars fixed
it in December or thereabouts. He said then the fix will be included in
the Gnus that goes with Emacs 24.4. I guess it was too late to get it
into Emacs 24.3 at that point.

So, git Gnus or wait... Or bzr Emacs?

Re: Gnus horribly slow after upgrade to Fedora17

[Lars Magne Ingebrigtsen]

om@iki.fi (Otto J. Makela) writes:

> gnus-read-active-file's value is some
> gnus-check-new-newsgroups's value is ask-server

That should be OK, then.  Hm...

Could you set `nntp-record-commands', hit `g' and look in the *nntp-log*
buffer and see what it contains?

> Can I request a bit more granularity in gnus version numbering, since
> I have two machines with the "same" gnus version 5.13 but one works as
> expected (RHEL6, Emacs/23.1) and the other one (Fedora 19, Emacs/24.3)
> is horribly slow?

Well, the Emacs version provides some granularity.  >"?

> Of course one alternative is that the "fast" machine is actually doing
> just the same work, it just happens to be on a much faster network.

It's possible.

> Also, do you have ideas on correcting the nntp connection error
> I detailed in my earlier message?

I think I missed that message?

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> om@iki.fi (Otto J. Makela) writes:
>> gnus-read-active-file's value is some
>> gnus-check-new-newsgroups's value is ask-server
>
> That should be OK, then.  Hm...
>
> Could you set `nntp-record-commands', hit `g' and look in the
> *nntp-log* buffer and see what it contains?

After this, it contained 26881 lines, where gnus does LIST ACTIVE for
each and every group in the .newsrc:

20130803T103514.522 news.saunalahti.fi LIST ACTIVE zzz.3
20130803T103514.532 news.saunalahti.fi LIST ACTIVE zz.unity.netlink
20130803T103514.542 news.saunalahti.fi LIST ACTIVE zz.unity.chat
20130803T103514.552 news.saunalahti.fi LIST ACTIVE zybrkat.mlemke
20130803T103514.561 news.saunalahti.fi LIST ACTIVE zippo.spamhippo.top100
20130803T103514.571 news.saunalahti.fi LIST ACTIVE zippo.editorial
20130803T103514.581 news.saunalahti.fi LIST ACTIVE zetnet.support
20130803T103514.591 news.saunalahti.fi LIST ACTIVE zetnet.local
20130803T103514.600 news.saunalahti.fi LIST ACTIVE zeta.test.ausadmin
20130803T103514.611 news.saunalahti.fi LIST ACTIVE zer.z-netz.wissenschaft.psychologie
...
20130803T103938.977 news.saunalahti.fi LIST ACTIVE 0.verizon.windowsxp
20130803T103938.985 news.saunalahti.fi LIST ACTIVE 0.verizon.suggestion.box
20130803T103938.995 news.saunalahti.fi LIST ACTIVE 0.verizon.newsgroup.requests
20130803T103939.004 news.saunalahti.fi LIST ACTIVE 0.verizon.linux
20130803T103939.015 news.saunalahti.fi LIST ACTIVE 0.verizon.flame
20130803T103939.024 news.saunalahti.fi LIST ACTIVE 0.verizon.discussion-general
20130803T103939.034 news.saunalahti.fi LIST ACTIVE 0.verizon.announce
20130803T103939.043 news.saunalahti.fi LIST ACTIVE 0.verizon.adsl
20130803T103939.055 news.saunalahti.fi LIST ACTIVE 0.test
20130803T103939.063 news.saunalahti.fi LIST ACTIVE -

So no wonder it is slow, now I need to figure out why this happens.
This is what my .gnus has:

----
(setq gnus-select-method
 '(nntp "news.saunalahti.fi"
        (nntp-open-connection-function network-only)))
(setq gnus-default-posting-charset 'iso-8859-1)
(setq message-default-charset 'iso-8859-1)
(setq gnus-check-new-newsgroups 'ask-server)
(setq gnus-save-killed-list nil)
(setq gnus-user-cross-reference t)
(setq gnus-thread-indent-level 2)
(setq gnus-ignored-headers "^Xref:\\|^X-Trace:\\|^Mime-Version:\\|^X-Newsreader:")
(setq gnus-boring-article-headers '(empty followup-to reply-to newsgroups))
(setq gnus-article-date-local t)
(setq gnus-treat-date-local 'head)
(add-hook 'message-sent-hook 'gnus-score-followup-thread '10)
(setq gnus-summary-line-format "%U%R%z%I%[%3L:%-17,17f%d%] %s\n")
 (setq gnus-article-mode-line-format  "%1,40S")
(setq gnus-summary-expunge-below -1500)
(setq gnus-score-expiry-days 10)
(setq gnus-use-nocem t)
(setq gnus-nocem-check-article-limit 50)
(setq gnus-refer-article-method
      '(current
      (nnweb "refer" (nnweb-type google))))

(setq   gnus-thread-sort-functions
                '(gnus-thread-sort-by-number gnus-thread-sort-by-total-score))
;        gnus-thread-hide-subjects nil
;        gnus-thread-hide-subtree t
;        gnus-thread-indent-level 4)
;; (setq gnus-summary-same-subject "->")
(setq gnus-summary-gather-subject-limit 'fuzzy)
(setq gnus-simplify-subject-fuzzy-regexp "\\([Vv][Ss]\\|[Ss][Vv]\\):[ \t]*")
----

[...]
>> Also, do you have ideas on correcting the nntp connection error
>> I detailed in my earlier message?
>
> I think I missed that message?

It was the connection problem which turns out to already be fixed:

Anssi Saari <as@sci.fi> wrote:

> I reported that issue following the discussion last year and Lars
> fixed it in December or thereabouts. He said then the fix will be
> included in the Gnus that goes with Emacs 24.4. I guess it was too
> late to get it into Emacs 24.3 at that point.

-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Lars Magne Ingebrigtsen]

om@iki.fi (Otto J. Makela) writes:

> After this, it contained 26881 lines, where gnus does LIST ACTIVE for
> each and every group in the .newsrc:
>
> 20130803T103514.522 news.saunalahti.fi LIST ACTIVE zzz.3
> 20130803T103514.532 news.saunalahti.fi LIST ACTIVE zz.unity.netlink
> 20130803T103514.542 news.saunalahti.fi LIST ACTIVE zz.unity.chat

Ok, so you do have all these groups in your .newsrc file?  Kill them
off, and that will make Gnus not query the server about them.

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> Ok, so you do have all these groups in your .newsrc file?  Kill them
> off, and that will make Gnus not query the server about them.

Thank you, this made it work as expected. Funny, earlier non-subscribed
newsgroups in the .newsrc didn't seem to make a difference.
-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

om@iki.fi (Otto J. Makela) wrote:

> Thank you, this made it work as expected. Funny, earlier non-subscribed
> newsgroups in the .newsrc didn't seem to make a difference.

Also, tried to remove all unsubscribed groups from .newsrc on this
"different" 5.13 gnus running on emacs 23.1, and it then proceeded to
subscribe me to all those groups while adding the lines back :-/

-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Re: Gnus horribly slow after upgrade to Fedora17

[Lars Magne Ingebrigtsen]

"Otto J. Makela" <om@iki.fi> writes:

> For reasons that are beyond me, the Gnus that came with Fedora17
> (version Gnus/5.13 Emacs/24.1) is terribly slow compared to the earlier
> one I used with Fedora16 (Gnus/5.13 Emacs/23.3). The "g" command
> (gnus-group-get-new-news) "nntp read" takes around 20× more time.

Is there a lot of data in the " *server foo... *" buffers after you hit
`g'?  (Note leading space in the buffer name.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php

Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

For reasons that are beyond me, the Gnus that came with Fedora17
(version Gnus/5.13 Emacs/24.1) is terribly slow compared to the earlier
one I used with Fedora16 (Gnus/5.13 Emacs/23.3). The "g" command
(gnus-group-get-new-news) "nntp read" takes around 20× more time.

I do have to set the following in .emacs to prevent it trying to use
ssl over nntp (which is failing for some reason), but it's beyond me
if this is causing the slowdown?
	(setq gnus-select-method '(nntp "news.kolumbus.fi"
		(nntp-open-connection-function network-only)))

Any suggestions what could be wrong? What kind of debug log can I get?

-- 
    /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
   /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
  /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

For reasons that are beyond me, the Gnus that came with Fedora17 (I'm
writing this with it) is terribly slow compared to the earlier one I
used with Fedora16. The "g" command (gnus-group-get-new-news) "nntp
read" takes around 20× more time.

I do have to set the following in .emacs to prevent it trying to use
ssl over nntp (which is failing for some reason), but it's beyond me
if this is causing the slowdown?
	(setq gnus-select-method '(nntp "news.kolumbus.fi"
		(nntp-open-connection-function network-only)))

Any suggestions what could be wrong?
-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */

Gnus horribly slow after upgrade to Fedora17

[Otto J. Makela]

For reasons that are beyond me, the Gnus that came with Fedora17 (I'm
writing this with it) is terribly slow compared to the earlier one I
used with Fedora16. The "g" command (gnus-group-get-new-news) "nntp
read" takes around 20× more time.

I did have to set the following in .emacs to prevent it trying to use
ssl over nntp (which was failing for some reason), but it's beyond me if
this is causing the slowdown?
	(setq gnus-select-method '(nntp "news.kolumbus.fi"
		(nntp-open-connection-function network-only)))

Any suggestions what could be wrong?
-- 
   /* * * Otto J. Makela <om@iki.fi> * * * * * * * * * */
  /* Phone: +358 40 765 5772, ICBM: N 60 10' E 24 55' */
 /* Mail: Mechelininkatu 26 B 27,  FI-00100 Helsinki */
/* * * Computers Rule 01001111 01001011 * * * * * * */