What happened to snappy low-bandwidth nnimap?

What happened to snappy low-bandwidth nnimap?

[David Engster]

I'm currently on vacation with no 3G and terrible reception to
boot. We're talking roughly 2kb/s, so it's totally, like, 1992 or so.

Gnus' nnimap used to be pretty snappy with low bandwidths, but now I get
"Initial sync of 2 groups (please wait)" and it downloads about 700kb of
data. While that in itself is pretty bad, I could probably live with it
if it actually did this once, but it does it almost *every* time; so
much for "initial".

So my most pressing question is: how do I turn that off? I guess it's
some new feature for fixing that long standing "wrong unread count"
thingy, but please, I need my snappy Gnus back.

-David

Re: What happened to snappy low-bandwidth nnimap?

[Richard Riley]

David Engster <deng@randomsample.de> writes:

> I'm currently on vacation with no 3G and terrible reception to
> boot. We're talking roughly 2kb/s, so it's totally, like, 1992 or so.
>
> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I get
> "Initial sync of 2 groups (please wait)" and it downloads about 700kb of
> data. While that in itself is pretty bad, I could probably live with it
> if it actually did this once, but it does it almost *every* time; so
> much for "initial".
>
> So my most pressing question is: how do I turn that off? I guess it's
> some new feature for fixing that long standing "wrong unread count"
> thingy, but please, I need my snappy Gnus back.
>
> -David
>

Having spent a lot of time getting to understand agent, queues, plugging
and levels and level specific downloads last summer on holiday I too
have now noted that imap connections are really really slow on low
bandwidth connections and it appears to be fetching a lot of data
despite nothing being new.

Re: What happened to snappy low-bandwidth nnimap?

[Steinar Bang]

>>>>> Richard Riley <rileyrg@gmail.com>:
> David Engster <deng@randomsample.de> writes:

[snip!]
>> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I
>> get "Initial sync of 2 groups (please wait)" and it downloads about
>> 700kb of data. While that in itself is pretty bad, I could probably
>> live with it if it actually did this once, but it does it almost
>> *every* time; so much for "initial".

[snip!]
> Having spent a lot of time getting to understand agent, queues, plugging
> and levels and level specific downloads last summer on holiday I too
> have now noted that imap connections are really really slow on low
> bandwidth connections and it appears to be fetching a lot of data
> despite nothing being new.

I guess it would be neat if one of you could use git bisect to find the
commit that triggered the behaviour...?

I haven't noticed it myself, but that probably means I don't have slow
enough lines even when on mobile networks.  So I can't do the bisect
myself.

What I do when bisecting, is:
 1. git bisect start HEAD good-commit (for "good commit" pick one that
    you're sure is back when things were OK)
 2. compile, with eg.
     (cd ~/git/gnus; make clean; make; cd lisp; make tags)
 3. start a new emacs, start gnus, and test
 4. If the nnimap operations were slow, do:
     git bisect bad
    If the nnimap operations were fast, do:
     git bisect good
    If git reports it has found the offending commit, move to step 5,
    otherwise move to step 2. and repeat from there
 5. Report the offending commit, then clean up the working directory:
     git bisect reset
     git checkout master

Re: What happened to snappy low-bandwidth nnimap?

[David Engster]

Steinar Bang writes:
>>>>>> Richard Riley <rileyrg@gmail.com>:
>> David Engster <deng@randomsample.de> writes:
>
> [snip!]
>>> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I
>>> get "Initial sync of 2 groups (please wait)" and it downloads about
>>> 700kb of data. While that in itself is pretty bad, I could probably
>>> live with it if it actually did this once, but it does it almost
>>> *every* time; so much for "initial".
>
> [snip!]
>> Having spent a lot of time getting to understand agent, queues, plugging
>> and levels and level specific downloads last summer on holiday I too
>> have now noted that imap connections are really really slow on low
>> bandwidth connections and it appears to be fetching a lot of data
>> despite nothing being new.
>
> I guess it would be neat if one of you could use git bisect to find the
> commit that triggered the behaviour...?

It's not that easy since actual messaging of these "initial sync"
thingies was added later. After looking into the logs it seems it has to
do with QRESYNC handling; I'm using Dovecot 1.2 and it should support
QRESYNC, hence I don't understand why I'm seeing so many "initial
syncs". It could be because my connection drops frequently (as I've
said, reception is terrible).

-David

Re: What happened to snappy low-bandwidth nnimap?

[Michael Welsh Duggan]

David Engster <deng@randomsample.de> writes:

> Steinar Bang writes:
>>>>>>> Richard Riley <rileyrg@gmail.com>:
>>> David Engster <deng@randomsample.de> writes:
>>
>>>> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I
>>>> get "Initial sync of 2 groups (please wait)" and it downloads about
>>>> 700kb of data. While that in itself is pretty bad, I could probably
>>>> live with it if it actually did this once, but it does it almost
>>>> *every* time; so much for "initial".
>>
>>> Having spent a lot of time getting to understand agent, queues, plugging
>>> and levels and level specific downloads last summer on holiday I too
>>> have now noted that imap connections are really really slow on low
>>> bandwidth connections and it appears to be fetching a lot of data
>>> despite nothing being new.
>>
>> I guess it would be neat if one of you could use git bisect to find the
>> commit that triggered the behaviour...?
>
> It's not that easy since actual messaging of these "initial sync"
> thingies was added later. After looking into the logs it seems it has to
> do with QRESYNC handling; I'm using Dovecot 1.2 and it should support
> QRESYNC, hence I don't understand why I'm seeing so many "initial
> syncs". It could be because my connection drops frequently (as I've
> said, reception is terrible).

In <kslijqdw90.fsf@netfonds.no>
(<URL:http://article.gmane.org/gmane.emacs.gnus.general/81943>), Peder
Klingenberg gives a diagnosis of this problem.

-- 
Michael Welsh Duggan
(mwd@cert.org)

Re: What happened to snappy low-bandwidth nnimap?

[David Engster]

Michael Welsh Duggan writes:
> David Engster <deng@randomsample.de> writes:
>
>> Steinar Bang writes:
>>>>>>>> Richard Riley <rileyrg@gmail.com>:
>>>> David Engster <deng@randomsample.de> writes:
>>>
>>>>> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I
>>>>> get "Initial sync of 2 groups (please wait)" and it downloads about
>>>>> 700kb of data. While that in itself is pretty bad, I could probably
>>>>> live with it if it actually did this once, but it does it almost
>>>>> *every* time; so much for "initial".
>>>
>>>> Having spent a lot of time getting to understand agent, queues, plugging
>>>> and levels and level specific downloads last summer on holiday I too
>>>> have now noted that imap connections are really really slow on low
>>>> bandwidth connections and it appears to be fetching a lot of data
>>>> despite nothing being new.
>>>
>>> I guess it would be neat if one of you could use git bisect to find the
>>> commit that triggered the behaviour...?
>>
>> It's not that easy since actual messaging of these "initial sync"
>> thingies was added later. After looking into the logs it seems it has to
>> do with QRESYNC handling; I'm using Dovecot 1.2 and it should support
>> QRESYNC, hence I don't understand why I'm seeing so many "initial
>> syncs". It could be because my connection drops frequently (as I've
>> said, reception is terrible).
>
> In <kslijqdw90.fsf@netfonds.no>
> (<URL:http://article.gmane.org/gmane.emacs.gnus.general/81943>), Peder
> Klingenberg gives a diagnosis of this problem.

Thank you; this helps. Like Peder describes, I've now also changed
`nnimap-retrieve-group-data-early' to always use SELECT instead of
EXAMINE. So far I haven't seen another "initial" sync.

-David

Re: What happened to snappy low-bandwidth nnimap?

[David Engster]

David Engster writes:
> Thank you; this helps. Like Peder describes, I've now also changed
> `nnimap-retrieve-group-data-early' to always use SELECT instead of
> EXAMINE. So far I haven't seen another "initial" sync.

Spoke too soon. Still happening.

-David

Re: What happened to snappy low-bandwidth nnimap?

[Lars Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> I guess it would be neat if one of you could use git bisect to find the
> commit that triggered the behaviour...?

It was introduced with the "nonexist" handling, I think.

It's a major problem, but the thing is that I've so far been unable to
reproduce it myself, so fixing it has been kinda...  er...  difficult.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Emacs

Re: What happened to snappy low-bandwidth nnimap?

[Dave Abrahams]

on Tue Aug 07 2012, David Engster <deng-AT-randomsample.de> wrote:

> I'm currently on vacation with no 3G and terrible reception to
> boot. We're talking roughly 2kb/s, so it's totally, like, 1992 or so.
>
> Gnus' nnimap used to be pretty snappy with low bandwidths, but now I get
> "Initial sync of 2 groups (please wait)" and it downloads about 700kb of
> data. While that in itself is pretty bad, I could probably live with it
> if it actually did this once, but it does it almost *every* time; so
> much for "initial".

Ditto.  My connections aren't as slow, but I'm having the same
"initial"-sync-over-and-over problem.

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost