Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Lars Magne Ingebrigtsen]

> +      (when (and (fboundp 'set-network-process-option)
> +                 (eq (process-type process) 'network))
> +        ;; Use TCP-keepalive so that connections that pass through a NAT router
> +        ;; don't hang when left idle.
> +        (set-network-process-option process :keepalive t))

I don't necessarily disagree with this patch, but I'm wondering what the
use case is.

Keepalive is usually useful (on the client side) if you have a protocol
where you're just sitting sleeping on a socket.  If the server dies,
you'll never get a TCP RST, so the keepalive option will ensure you get
that RST after a while.

However, the NNTP protocol isn't like that.  You send a command and you
get a response.  If you send a command to a server, and it doesn't
recognise you because you've changed IP address or whatever, it'll send
you a RST.

So the NNTP protocol doesn't seem like something that would need a
TCP keepalive option.  But I might be misthinking the use case.

(And, besides, if nntp.el needs keepalive, doesn't all network
connections need it?)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Stefan Monnier]

> Keepalive is usually useful (on the client side) if you have a protocol
> where you're just sitting sleeping on a socket.  If the server dies,
> you'll never get a TCP RST, so the keepalive option will ensure you get
> that RST after a while.

The use case is the one mentioned in the comment: you sit behind a NAT
router.  Gnus connects, sends some commands, gets some responses, then
sits still for some hours because the user goes on to do something else
or reads newsgroup from another server (the machine may be put to sleep
in the mean time, but not necessarily).  When the user comes back to
this newsserver, the IP will often be the same as before, but the NAT
router will have completely forgotten about this open TCP connection, so
Emacs never receives an RST or anything like that and Gnus just sits
there waiting for the answer that'll never come.

> (And, besides, if nntp.el needs keepalive, doesn't all network
> connections need it?)

All the ones that may stay open without activity, yes.
Oddly enough nowadays these are rarely used by the "new" protocols like
http, video-streaming, etc...


        Stefan

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Stefan Monnier]

>> (And, besides, if nntp.el needs keepalive, doesn't all network
>> connections need it?)

> All the ones that may stay open without activity, yes.
> Oddly enough nowadays these are rarely used by the "new" protocols like
> http, video-streaming, etc...

BTW, what is the use case for the nnimap-keepalive thingy?


        Stefan "who's had to look into keepalive because the NAT router
                in his temporary apartment drops TCP connections after
                only 5min of inactivity"

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[David Engster]

Lars Magne Ingebrigtsen writes:
>> +      (when (and (fboundp 'set-network-process-option)
>> +                 (eq (process-type process) 'network))
>> +        ;; Use TCP-keepalive so that connections that pass through a NAT router
>> +        ;; don't hang when left idle.
>> +        (set-network-process-option process :keepalive t))
>
> I don't necessarily disagree with this patch, but I'm wondering what the
> use case is.

In any case, the buildbot is now complaining that this (probably) won't
work on Emacs22 and XEmacs. :-)

(I've yet to figure out how to add the actual committer to the
blamelist, hence Yamaoka-san didn't receive a mail about this).

-David

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]


Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> +      (when (and (fboundp 'set-network-process-option)
>> +                 (eq (process-type process) 'network))
>> + ;; Use TCP-keepalive so that connections that pass through a NAT
>> router
>> +        ;; don't hang when left idle.
>> +        (set-network-process-option process :keepalive t))
>
> I don't necessarily disagree with this patch, but I'm wondering what the
> use case is.
>
> Keepalive is usually useful (on the client side) if you have a protocol
> where you're just sitting sleeping on a socket.  If the server dies,
> you'll never get a TCP RST, so the keepalive option will ensure you get
> that RST after a while.
>
> However, the NNTP protocol isn't like that.  You send a command and you
> get a response.  If you send a command to a server, and it doesn't
> recognise you because you've changed IP address or whatever, it'll send
> you a RST.

I've seen a misguided firewall  that breaks TCP connections after a
while, and seen it cause trouble with gnus/nntp.  Basically:

  gnus opens connection to server (across firewall)

  time passes
  
  gnus sends data on the tcp connection

  firewall drops the data

  nothing else happens until the tcp connection times out


In this particular case it's a firewall only, not NAT, but it's
basically the same issue.


> So the NNTP protocol doesn't seem like something that would need a
> TCP keepalive option.  But I might be misthinking the use case.

I think you're assuming that the sender will discover the brokenness
promptly.  In the absence of broken middleboxes, that might be true.
But then we wouldn't need keepalives at all.

> (And, besides, if nntp.el needs keepalive, doesn't all network
> connections need it?)

That is a fair point.

Another approach would be to have gnus have a timer so that it tears
down news connections after some period of inactivity, like perhaps
600s.  It seems that perhaps a gnus left idle for hours shouldn't have
any open connections.

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Antoine Levitt]

10/05/11 14:06, Greg Troxel
>> (And, besides, if nntp.el needs keepalive, doesn't all network
>> connections need it?)
>
> That is a fair point.
>
> Another approach would be to have gnus have a timer so that it tears
> down news connections after some period of inactivity, like perhaps
> 600s.  It seems that perhaps a gnus left idle for hours shouldn't have
> any open connections.

I have no idea if that's relevant to the discussion at hand, but in my
use case there is: I suspend my computer on the evening, and switch it
back on in the morning, creating large periods of idle for gnus. Since
connections are to localhost, either by NNTP with leafnode or by IMAP
with dovecot, I don't think any are lost.

I've seen nnimap bug out on this use case (mainly not getting new mail),
although I think nntp is fine.

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Richard Riley]

Antoine Levitt <antoine.levitt@gmail.com> writes:

> 10/05/11 14:06, Greg Troxel
>>> (And, besides, if nntp.el needs keepalive, doesn't all network
>>> connections need it?)
>>
>> That is a fair point.
>>
>> Another approach would be to have gnus have a timer so that it tears
>> down news connections after some period of inactivity, like perhaps
>> 600s.  It seems that perhaps a gnus left idle for hours shouldn't have
>> any open connections.
>
> I have no idea if that's relevant to the discussion at hand, but in my
> use case there is: I suspend my computer on the evening, and switch it
> back on in the morning, creating large periods of idle for gnus. Since
> connections are to localhost, either by NNTP with leafnode or by IMAP
> with dovecot, I don't think any are lost.
>
> I've seen nnimap bug out on this use case (mainly not getting new mail),
> although I think nntp is fine.
>

just FYI, I frequently have to kill gnus and then wait for a period of
time before a nntp session can be reconnected to eternalseptember nntp
servers. Invariably it hangs on post but I haven't noticed a reliable
"case" with which to duplicate the issue. It happens frequently however.

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Stefan Monnier]

>> I don't necessarily disagree with this patch, but I'm wondering what the
>> use case is.
> In any case, the buildbot is now complaining that this (probably) won't
> work on Emacs22 and XEmacs. :-)

I must say I have no idea what makes it think so.  The `fboundp' check
should avoid those problems.


        Stefan

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[David Engster]

Stefan Monnier writes:
>>> I don't necessarily disagree with this patch, but I'm wondering what the
>>> use case is.
>> In any case, the buildbot is now complaining that this (probably) won't
>> work on Emacs22 and XEmacs. :-)
>
> I must say I have no idea what makes it think so.  The `fboundp' check
> should avoid those problems.

You're checking for set-network-process-option, but not for
process-type. At least that will fix the warning for Emacs22.

Regarding XEmacs, I have no idea...

-David

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[David Engster]

David Engster writes:
> Stefan Monnier writes:
>>>> I don't necessarily disagree with this patch, but I'm wondering what the
>>>> use case is.
>>> In any case, the buildbot is now complaining that this (probably) won't
>>> work on Emacs22 and XEmacs. :-)
>>
>> I must say I have no idea what makes it think so.  The `fboundp' check
>> should avoid those problems.
>
> You're checking for set-network-process-option, but not for
> process-type. At least that will fix the warning for Emacs22.
>
> Regarding XEmacs, I have no idea...

Since those warnings are obviously spurious, if have now bound those
functions in lpath.el.

-David

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[David Engster]

David Engster writes:
> (I've yet to figure out how to add the actual committer to the
> blamelist, hence Yamaoka-san didn't receive a mail about this).

I've now changed the post-merge hook to use the committer as the
'author' which is send to the buildbot. I think this makes more sense
for our setup.

-David

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> BTW, what is the use case for the nnimap-keepalive thingy?

Most IMAP servers drop the connection quite fast, and virtually all IMAP
clients use NOOP to have them not do this.  I haven't quite understood
what the point is.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: [gnus git] branch master updated: n0-17-27-g43f8466 =1= nntp.el (nntp-open-connection): Set TCP keepalive option.

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The use case is the one mentioned in the comment: you sit behind a NAT
> router.  Gnus connects, sends some commands, gets some responses, then
> sits still for some hours because the user goes on to do something else
> or reads newsgroup from another server (the machine may be put to sleep
> in the mean time, but not necessarily).  When the user comes back to
> this newsserver, the IP will often be the same as before, but the NAT
> router will have completely forgotten about this open TCP connection, so
> Emacs never receives an RST or anything like that and Gnus just sits
> there waiting for the answer that'll never come.

Right; makes sense.

But should this perhaps be a global Emacs thing?  Are there any
instances where switching TCP keepalive on is a bad thing?

I have a suspicion that just making Emacs 24 switch on TCP keepalive on
all connections (by default) would be a win.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/