different speed of opening a Gmane NG in Gnus

different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

When I open this group, i.e.
gmane.emacs.gnus.general, it is virtually
instantaneous that I get the summary buffer.

But with gmane.emacs.help it takes
several seconds!

I'm not stressed. But out of curiosity, why
is this?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Ted Zlatanov]

On Mon, 31 Jul 2017 11:22:00 +0200 Emanuel Berg <moasen@zoho.com> wrote: 

EB> When I open this group, i.e.
EB> gmane.emacs.gnus.general, it is virtually
EB> instantaneous that I get the summary buffer.

EB> But with gmane.emacs.help it takes
EB> several seconds!

EB> I'm not stressed. But out of curiosity, why
EB> is this?

It's either server-side or client-side.

To test the server, you could time the NNTP interactions when you do
`C-u 100 RET' to look at just the last 100 articles. If the timings are
similar, it's on the client side and you can profile it.

Ted

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Ted Zlatanov wrote:

> It's either server-side or client-side.

What is the server and client in this case?

If I'm the client, it is the same computer and
Internet connection.

If Gmane is the server, isn't it the
same Gmane?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Ted Zlatanov]

On Thu, 03 Aug 2017 23:31:36 +0200 Emanuel Berg <moasen@zoho.com> wrote: 

EB> Ted Zlatanov wrote:
>> It's either server-side or client-side.

EB> If I'm the client, it is the same computer and
EB> Internet connection.

Gnus (the client) can have different behavior for different groups
because of many things (e.g. scoring).

EB> If Gmane is the server, isn't it the same Gmane?

While I don't speak for the current Gmane operators, I know that when
Lars was running it he posted about the fun he had with storage
migration and optimization.

So it's entirely possible two groups could live on different storage
devices for administrative reasons, resulting in different performance
on the server.

Another issue may be filesystem or news server performance for small vs.
large groups.

Whatever the server-side reason, once you determine it's on the server,
there's not much you can do since Lars is not running it and the new
operators are not communicating with us. So let's hope it's on the
client side because then you could dig further.

HTH
Ted

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Ted Zlatanov wrote:

> So let's hope it's on the client side because
> then you could dig further.

It is not on the client side. I use the same
scoring, virtually, and it is not an extensive
operation. Besides, traffic is so low anyway
non of that should cause such a delay.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Ted Zlatanov]

On Sat, 05 Aug 2017 04:42:23 +0200 Emanuel Berg <moasen@zoho.com> wrote: 

EB> Ted Zlatanov wrote:
>> So let's hope it's on the client side because
>> then you could dig further.

EB> It is not on the client side. I use the same
EB> scoring, virtually, and it is not an extensive
EB> operation. Besides, traffic is so low anyway
EB> non of that should cause such a delay.

Did you test as I suggested?

Ted

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Ted Zlatanov wrote:

> Did you test as I suggested?

gmane.emacs.help is slower than
gmane.emacs.gnus.general which is all
but instantaneous.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Eric Abrahamsen]

Emanuel Berg <moasen@zoho.com> writes:

> Ted Zlatanov wrote:
>
>> Did you test as I suggested?
>
> gmane.emacs.help is slower than
> gmane.emacs.gnus.general which is all
> but instantaneous.

My Sherlock-omatic response is that it has to do with the relative sizes
of the group (emacs.help is probably an order of magnitude larger than
emacs.gnus.general), which would indicate some sort of caching issue:
for some reason Gnus is rebuilding something it shouldn't be rebuilding
when you open emacs.help.

How's that for useless advice?

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Eric Abrahamsen wrote:

> My Sherlock-omatic response is that it has to
> do with the relative sizes of the group
> (emacs.help is probably an order of magnitude
> larger than emacs.gnus.general), which would
> indicate some sort of caching issue: for some
> reason Gnus is rebuilding something it
> shouldn't be rebuilding when you open
> emacs.help.
>
> How's that for useless advice?

If so, rebuilding is the word since I use this
every day typically there are but a few
messages, and sometimes none.

gmane.emacs.help has ~110 000 posts available,
gmane.emacs.gnus.general ~80 000.

By the way even faster than
gmane.emacs.gnus.general are the
nntp.aioe.org Usenet groups, e.g.
rec.bicycles.tech, which are all lightning
fast, however that I understand since it is
another server with different technology.

Do you guys access gmane.emacs.help and
gmane.emacs.gnus.general with the same speed?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Ted Zlatanov wrote:

> To test the server, you could time the NNTP
> interactions when you do `C-u 100 RET' to look
> at just the last 100 articles. If the timings
> are similar, it's on the client side and you
> can profile it.

100 articles are not enough to detect a big
difference, altho a small difference is
detectable. Better do 1000.

Also when one has done 1000 once the second
time doing it will be much faster.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Maybe it *is* scoring!

I just realized, just because I have the same,
very small scoring files for both groups, it is
conceivable that there are no hits for
gmane.emacs.gnus.general, so there is nothing
to do, while on gmane.emacs.help on the other
hand, there are hits!

Can I access the groups with scoring disabled or
do I have to temporarily rename the files?

Also, how do you clear everything else to make
for a fair competition? Just restart Gnus?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Eric Abrahamsen]

Emanuel Berg <moasen@zoho.com> writes:

> Eric Abrahamsen wrote:
>
>> My Sherlock-omatic response is that it has to
>> do with the relative sizes of the group
>> (emacs.help is probably an order of magnitude
>> larger than emacs.gnus.general), which would
>> indicate some sort of caching issue: for some
>> reason Gnus is rebuilding something it
>> shouldn't be rebuilding when you open
>> emacs.help.
>>
>> How's that for useless advice?
>
> If so, rebuilding is the word since I use this
> every day typically there are but a few
> messages, and sometimes none.
>
> gmane.emacs.help has ~110 000 posts available,
> gmane.emacs.gnus.general ~80 000.

Huh, that's much more similar than I would have guessed.

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Eric Abrahamsen wrote:

> Huh, that's much more similar than I would
> have guessed.

Indeed.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Eric Abrahamsen wrote:

> Huh, that's much more similar than I would
> have guessed.

It is not scoring and not the size of the
group. For a single new message in
gmane.emacs.help, it takes several seconds, and
for some 30 new messages in
gmane.linux.debian.user (a group of
500 000+ messages), it takes but a second.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

I just clocked how long it took to access
gmane.emacs.help, which contained 6 new
messages. 20 seconds!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Adam Sjøgren]

Emanuel writes:

> I just clocked how long it took to access
> gmane.emacs.help, which contained 6 new
> messages. 20 seconds!

Maybe time to read some of these pages:

 · https://www.gnu.org/software/emacs/manual/html_node/elisp/Profiling.html
 · https://www.emacswiki.org/emacs/EmacsLispProfiler

?


  Good luck!

    Adam

-- 
 "Unlike Parmenides the IMAP designers has decided that       Adam Sjøgren
  things that doesn't exist actually does exist."        asjo@koldfront.dk

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Adam Sjøgren wrote:

> Maybe time to read some of these pages [...]

With `elp-instrument-function' for
`gnus-group-select-group', the results are,

    for 100 old posts from gmane.emacs.gnus.user,
    no .SCORE file:

        gnus-group-select-group  1    0.562393515   0.562393515

    ditto gmane.emacs.help:

        gnus-group-select-group  1    2.786039744   2.786039744

Too bad it didn't go on to say what parts
(i.e., individual function invocations) within
`gnus-group-select-group' are/is the "villain
in the play"...

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: different speed of opening a Gmane NG in Gnus

[Emanuel Berg]

Why is gmane.emacs.help so slow and
gmane.emacs.gnus.general much faster to access?

Here is one theory:

$ ls -lhS ~/**/.overview | head -n 5

-rw-r--r-- 1 incal incal  36M Nov 26 07:44 News/agent/nntp/news.gmane.org/gmane/emacs/help/.overview
-rw-r--r-- 1 incal incal 3.3M Nov 25 18:11 News/agent/nntp/news.gmane.org/gmane/emacs/gnus/general/.overview
-rw-r--r-- 1 incal incal 2.5M Nov  8 08:59 News/agent/nntp/news.gmane.org/gmane/emacs/devel/.overview
-rw------- 1 incal incal 855K Nov 25 23:50 Mail/mail/sent/.overview
-rw-r--r-- 1 incal incal 807K May 15  2016 News/agent/nntp/news.gmane.org/gmane/comp/video/blender/user/.overview

-- 
underground experts united
http://user.it.uu.se/~embe8573