* Re: Making IMAP fasterer
2016-07-09 8:27 Making IMAP fasterer Jaspn Tibbitts
@ 2016-07-09 9:03 ` Steinar Bang
2016-07-09 9:10 ` Steinar Bang
` (3 subsequent siblings)
4 siblings, 0 replies; 12+ messages in thread
From: Steinar Bang @ 2016-07-09 9:03 UTC (permalink / raw)
To: ding
>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> I think that the issue is that the path to my IMAP server has a bit of
> latency, which isn't surprising considering I'm on a Norwegian island
> not far from the arctic circle.
Which one?
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: Making IMAP fasterer
2016-07-09 8:27 Making IMAP fasterer Jaspn Tibbitts
2016-07-09 9:03 ` Steinar Bang
@ 2016-07-09 9:10 ` Steinar Bang
2016-07-09 10:58 ` Jaspn Tibbitts
2016-07-09 9:32 ` Eric Abrahamsen
` (2 subsequent siblings)
4 siblings, 1 reply; 12+ messages in thread
From: Steinar Bang @ 2016-07-09 9:10 UTC (permalink / raw)
To: ding
>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> I think that the issue is that the path to my IMAP server has a bit of
> latency, which isn't surprising considering I'm on a Norwegian island
> not far from the arctic circle. I have always had gnus-asynchronous
> set,
FWIW my value og gnus-asynchronous is nil.
> and I remember in the past it used to flash messages about the
> articles it was prefetching. I haven't seen that in a while, and it
> doesn't look like gnus is doing any article prefetching. So selecting
> an article takes a couple of seconds while it does a bunch of round
> trips.
I do no preselect, so this happens for me as well on a slow line.
However I have set gnus-agent to cache all articles read, so the next
time is much faster.
I never use agent prefetch because I've found its behaviour
annoying when I first started using agent, and have never had any
incentive to revisit it (there is a size cutoff that at the time left
out many of the articles I wanted to read, and I've never looked into
adjusting that to something useful).
> In addition, gnus-group-get-new-news always tells me "nnimap read NNNk
> from imap.math.uh.edu (initial sync of 4 groups; please wait)" while it
> fetches about 400k in total. This takes about ten seconds every time i
> hit 'g', which I'm trying to condition myself not to do.
Hm... that this happens the first time isn't any suprprise, but that it
happens every time is. Do you use the agent?
> So, does anyone have any tips for making it run a bit smoother?
I dunno, use the agent maybe? Seems to work for me, and have done so
since the late ninties.
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: Making IMAP fasterer
2016-07-09 9:10 ` Steinar Bang
@ 2016-07-09 10:58 ` Jaspn Tibbitts
0 siblings, 0 replies; 12+ messages in thread
From: Jaspn Tibbitts @ 2016-07-09 10:58 UTC (permalink / raw)
To: ding
Steinar Bang <sb@dod.no> writes:
>>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> I do no preselect, so this happens for me as well on a slow line.
Yeah, I was trying to have it do some readahead. And I know it used to
but at some point (probably not recently) it stopped.
> However I have set gnus-agent to cache all articles read, so the next
> time is much faster.
Except that I'm not really interested in looking at old mail; I'd just
like reading new mail to be a bit faster.
> Hm... that this happens the first time isn't any suprprise, but that it
> happens every time is. Do you use the agent?
I don't think so. It's always possible I turned it on at some point in
the last, oh, 25+ years that I've used gnus, but nothing matches /agent/
in that .gnus I attached.
- J<
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 8:27 Making IMAP fasterer Jaspn Tibbitts
2016-07-09 9:03 ` Steinar Bang
2016-07-09 9:10 ` Steinar Bang
@ 2016-07-09 9:32 ` Eric Abrahamsen
2016-07-09 10:52 ` Jaspn Tibbitts
2016-07-21 6:28 ` Steinar Bang
2017-06-15 6:57 ` Dave Abrahams
4 siblings, 1 reply; 12+ messages in thread
From: Eric Abrahamsen @ 2016-07-09 9:32 UTC (permalink / raw)
To: ding
Jaspn Tibbitts <tibbs@math.uh.edu> writes:
> I'm on vacation and instead of my usual method of ssh-ing into my work
> machine and running emacs in a screen session, I thought I'd try setting
> up gnus locally. And it works, but it's pretty slow. I'm running
> Fedora 24, with emacs 25.0.94 the version of gnus which comes with it
> (5.13).
>
> I think that the issue is that the path to my IMAP server has a bit of
> latency, which isn't surprising considering I'm on a Norwegian island
> not far from the arctic circle. I have always had gnus-asynchronous
> set, and I remember in the past it used to flash messages about the
> articles it was prefetching. I haven't seen that in a while, and it
> doesn't look like gnus is doing any article prefetching. So selecting
> an article takes a couple of seconds while it does a bunch of round
> trips.
>
> In addition, gnus-group-get-new-news always tells me "nnimap read NNNk
> from imap.math.uh.edu (initial sync of 4 groups; please wait)" while it
> fetches about 400k in total. This takes about ten seconds every time i
> hit 'g', which I'm trying to condition myself not to do.
That's a bit odd, it sounds like the groups are syncing, but the data
isn't getting saved properly or something, so it resyncs every time.
What IMAP server are you connecting to? How many groups are there? Is it
always four groups that do the initial sync? Can you tell if they're the
same four groups every time?
One thing you might try, if you're comfortable with edebugging, is going
into the lisp/gnus/nnimap.el file, finding the function
`nnimap-retrieve-group-data-early', and stepping through it. Initial
sync only happens in the `if' clause towards the bottom of the function
(probably line 1446 in your version). You could step through the
function and see which of the active|uidvalidity|unexist vars are nil at
that stage.
Who knows, maybe that will be enough to tip someone here off to the
nature of the problem.
Yours,
Eric
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 9:32 ` Eric Abrahamsen
@ 2016-07-09 10:52 ` Jaspn Tibbitts
2016-07-09 12:14 ` Eric Abrahamsen
2016-07-09 13:07 ` Steinar Bang
0 siblings, 2 replies; 12+ messages in thread
From: Jaspn Tibbitts @ 2016-07-09 10:52 UTC (permalink / raw)
To: Eric Abrahamsen; +Cc: ding
Eric Abrahamsen <eric@ericabrahamsen.net> writes:
> Jaspn Tibbitts <tibbs@math.uh.edu> writes:
> That's a bit odd, it sounds like the groups are syncing, but the data
> isn't getting saved properly or something, so it resyncs every time.
My .newsrc.eld dates from, oh, gnus 0.6 if not older. Probably from
when gnus first started writing out newsrc.eld.
> What IMAP server are you connecting to? How many groups are there?
It's Cyrus. Sadly an older version, 2.3.16 with a couple of security
patches.
> Is it always four groups that do the initial sync? Can you tell if
> they're the same four groups every time?
It's always four. I'm not sure how I'd tell if it's the same four.
> One thing you might try, if you're comfortable with edebugging
Yeah, I can do that. Might take me a bit to swap all of that emacs
knowledge back in but I'll see what I can come up with.
Would it be worth building a development version of emacs so I can get
the current gnus code?
Also, is it possible to just purge .newsrc.eld of everything besides my
subscriptions and scoring and group parameters and such, and have it
just resync everything? I honestly have no idea how much old cruft is
in there.
- J<
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 10:52 ` Jaspn Tibbitts
@ 2016-07-09 12:14 ` Eric Abrahamsen
2016-07-09 13:07 ` Steinar Bang
1 sibling, 0 replies; 12+ messages in thread
From: Eric Abrahamsen @ 2016-07-09 12:14 UTC (permalink / raw)
To: ding
Jaspn Tibbitts <tibbs@math.uh.edu> writes:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Jaspn Tibbitts <tibbs@math.uh.edu> writes:
>
>> That's a bit odd, it sounds like the groups are syncing, but the data
>> isn't getting saved properly or something, so it resyncs every time.
>
> My .newsrc.eld dates from, oh, gnus 0.6 if not older. Probably from
> when gnus first started writing out newsrc.eld.
>
>> What IMAP server are you connecting to? How many groups are there?
>
> It's Cyrus. Sadly an older version, 2.3.16 with a couple of security
> patches.
Here's where I hope someone with more intimate knowledge of IMAP servers
might jump in. I do know that Cyrus, particularly older versions of
Cyrus, are quirky, and it's not impossible that that's interacting badly
with Gnus in some way.
>> Is it always four groups that do the initial sync? Can you tell if
>> they're the same four groups every time?
>
> It's always four. I'm not sure how I'd tell if it's the same four.
I don't either. But my guess is that it is the same four, and the
problem is in the data storage in your newsrc.eld.
>> One thing you might try, if you're comfortable with edebugging
>
> Yeah, I can do that. Might take me a bit to swap all of that emacs
> knowledge back in but I'll see what I can come up with.
>
> Would it be worth building a development version of emacs so I can get
> the current gnus code?
No, if you're already using Emacs 25, the nnimap code won't have changed
significantly.
> Also, is it possible to just purge .newsrc.eld of everything besides my
> subscriptions and scoring and group parameters and such, and have it
> just resync everything? I honestly have no idea how much old cruft is
> in there.
The .newsrc.eld file is a delicate creature, but I think this will end
up being the solution. Deleting marks for non-imap servers might cause
you difficulty, but if it's only imap servers then gnus should behave
correctly: initial syncs for all the groups, then normal behavior from
there on out. Theoretically. Don't take my word for it: wait until
someone else chimes in, then back up your file, cut out the imap marks,
and try re-syncing. I'd also shut off the asynchronous stuff first --
try to make the resync as "clean" as possible.
If I were you, I'd also do the edebug test first. Who knows, you might
see something illuminating.
Eric
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 10:52 ` Jaspn Tibbitts
2016-07-09 12:14 ` Eric Abrahamsen
@ 2016-07-09 13:07 ` Steinar Bang
1 sibling, 0 replies; 12+ messages in thread
From: Steinar Bang @ 2016-07-09 13:07 UTC (permalink / raw)
To: ding
>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>> What IMAP server are you connecting to? How many groups are there?
> It's Cyrus. Sadly an older version, 2.3.16 with a couple of security
> patches.
FWIW I went from UoW imapd to cyrus in 1999 or 2000 or thereabouts,
because of quirks with the UoW imapd, then had a lot of issues for the
years I used cyrus (e.g. lost all of my marks twice during upgrades).
However, I switched to dovecot in 2008 and have had trouble free IMAP
operation since.
nntp://news.gmane.org/gmane.mail.imap.dovecot/32706
http://permalink.gmane.org/gmane.mail.imap.dovecot/32706
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 8:27 Making IMAP fasterer Jaspn Tibbitts
` (2 preceding siblings ...)
2016-07-09 9:32 ` Eric Abrahamsen
@ 2016-07-21 6:28 ` Steinar Bang
2016-07-21 21:02 ` Jason Tibbitts
2017-06-15 6:57 ` Dave Abrahams
4 siblings, 1 reply; 12+ messages in thread
From: Steinar Bang @ 2016-07-21 6:28 UTC (permalink / raw)
To: ding
>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> In addition, gnus-group-get-new-news always tells me "nnimap read NNNk
> from imap.math.uh.edu (initial sync of 4 groups; please wait)" while
> it fetches about 400k in total. This takes about ten seconds every
> time i hit 'g', which I'm trying to condition myself not to do.
Ok, now I've seen this as well. I'm on vacation, and have been using a
(supposedly) 4G ICE.net router, that is a replacement for my old 450MHz
ICE.net router, that no longer works, because they have switched off the
450MHz ntwork.
So far the new router has been a disappointment, and I've ended up using
campsites' free wi-fi or my cellphone instead... but anyway: I have now
tested slooow connections and it can be a pain to watch nnimap fetch the
same data time after time.
As you say, it says "initial sync", even though it fetches the same data
multiple times, so a guess is that it wasn't *intended* to be fetched
multiple times...?
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: Making IMAP fasterer
2016-07-21 6:28 ` Steinar Bang
@ 2016-07-21 21:02 ` Jason Tibbitts
2016-07-21 21:06 ` Steinar Bang
0 siblings, 1 reply; 12+ messages in thread
From: Jason Tibbitts @ 2016-07-21 21:02 UTC (permalink / raw)
To: ding
Steinar Bang <sb@dod.no> writes:
>>>>>> Jaspn Tibbitts <tibbs@math.uh.edu>:
> So far the new router has been a disappointment, and I've ended up using
> campsites' free wi-fi or my cellphone instead... but anyway: I have now
> tested slooow connections and it can be a pain to watch nnimap fetch the
> same data time after time.
It's good to know that I'm not having some weird completely unique
problem.
However, I haven't really found the time to dig into it, because I had
better network latency in Trondheim and now that I'm in Bergen things
are fast enough that it isn't really an issue. Plus I've trained myself
to not hit 'g' so often.
> As you say, it says "initial sync", even though it fetches the same data
> multiple times, so a guess is that it wasn't *intended* to be fetched
> multiple times...?
Are you always seeing it fetch four groups? Or do you see a different
number?
- J<
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Making IMAP fasterer
2016-07-09 8:27 Making IMAP fasterer Jaspn Tibbitts
` (3 preceding siblings ...)
2016-07-21 6:28 ` Steinar Bang
@ 2017-06-15 6:57 ` Dave Abrahams
4 siblings, 0 replies; 12+ messages in thread
From: Dave Abrahams @ 2017-06-15 6:57 UTC (permalink / raw)
To: ding
on Sat Jul 09 2016, Jaspn Tibbitts <tibbs-AT-math.uh.edu> wrote:
> I'm on vacation and instead of my usual method of ssh-ing into my work
> machine and running emacs in a screen session, I thought I'd try setting
> up gnus locally. And it works, but it's pretty slow. I'm running
> Fedora 24, with emacs 25.0.94 the version of gnus which comes with it
> (5.13).
>
> I think that the issue is that the path to my IMAP server has a bit of
> latency, which isn't surprising considering I'm on a Norwegian island
> not far from the arctic circle. I have always had gnus-asynchronous
> set, and I remember in the past it used to flash messages about the
> articles it was prefetching. I haven't seen that in a while, and it
> doesn't look like gnus is doing any article prefetching. So selecting
> an article takes a couple of seconds while it does a bunch of round
> trips.
>
> In addition, gnus-group-get-new-news always tells me "nnimap read NNNk
> from imap.math.uh.edu (initial sync of 4 groups; please wait)" while it
> fetches about 400k in total. This takes about ten seconds every time i
> hit 'g', which I'm trying to condition myself not to do.
>
> So, does anyone have any tips for making it run a bit smoother?
I use http://onmsg.github.io
--
-Dave
^ permalink raw reply [flat|nested] 12+ messages in thread