[9fans] icmp unreachable messages sometimes dropped

[9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

well, documentation is better than nothing.

i didn't have time to fully debug this situation today, but i
noticed that on some hosts, icmp unreachable messages do not
terminate the connection, though snoopy did see the icmp packet.

for example, on host a telnet 40.10.10.10 hangs, but on host b,
it returned connection unreachable or some such.  the gateway
returning the unreachable message was the same in both cases.

this smells like a race, as tcp accesses the connection outside the lock.

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Lyndon Nerenberg]

> i didn't have time to fully debug this situation today, but i
> noticed that on some hosts, icmp unreachable messages do not
> terminate the connection, though snoopy did see the icmp packet.

Nor should they.  A temporary routing flap is no reason to arbitrarily
tear down a viable TCP connection.  Instead, tell the application and let
it decide what to do.

And don't get me started about SSH and TCP keepalives, or the lack
thereof when I shut the lid on my laptop! (No, not the Plan9 ssh.)

--lyndon

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

On Wed Jun  1 21:18:26 EDT 2011, lyndon@orthanc.ca wrote:
> > i didn't have time to fully debug this situation today, but i
> > noticed that on some hosts, icmp unreachable messages do not
> > terminate the connection, though snoopy did see the icmp packet.
>
> Nor should they.  A temporary routing flap is no reason to arbitrarily
> tear down a viable TCP connection.  Instead, tell the application and let
> it decide what to do.

this was on initial connection.  plan 9 ignores icmp unless it's waiting for
the initial response.

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Lyndon Nerenberg]

> this was on initial connection.  plan 9 ignores icmp unless it's waiting for
> the initial response.

I'll shut up now and watch the hockey game ;-)

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

On Wed Jun  1 21:29:15 EDT 2011, lyndon@orthanc.ca wrote:
> > this was on initial connection.  plan 9 ignores icmp unless it's waiting for
> > the initial response.
>
> I'll shut up now and watch the hockey game ;-)

it's a good point, though.  icmp unreachables shouldn't make it easy
to dos somebody by guessing a few small numbers.

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Bakul Shah]

On Wed, 01 Jun 2011 21:23:48 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> On Wed Jun  1 21:18:26 EDT 2011, lyndon@orthanc.ca wrote:
> > > i didn't have time to fully debug this situation today, but i
> > > noticed that on some hosts, icmp unreachable messages do not
> > > terminate the connection, though snoopy did see the icmp packet.
> >
> > Nor should they.  A temporary routing flap is no reason to arbitrarily
> > tear down a viable TCP connection.  Instead, tell the application and let
> > it decide what to do.
>
> this was on initial connection.  plan 9 ignores icmp unless it's waiting for
> the initial response.

SHOULD abort a connection for destination unreachable codes
2-4, but not for codes 0, 1 & 5. See rfc1122 2.4.3.9.

Re: [9fans] icmp unreachable messages sometimes dropped

[Lyndon Nerenberg]

> See rfc1122 2.4.3.9.

ENOTFOUND

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

> > See rfc1122 2.4.3.9.
>
> ENOTFOUND

i'm not sure what your point is.  and there's no such section in rfc 1122.

http://tools.ietf.org/html/rfc1122#section-2.4

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Bakul Shah]

On Wed, 01 Jun 2011 22:05:49 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > > See rfc1122 2.4.3.9.
> >
> > ENOTFOUND
>
> i'm not sure what your point is.  and there's no such section in rfc 1122.
>
> http://tools.ietf.org/html/rfc1122#section-2.4

Sorry. 4.2.3.9

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

On Wed Jun  1 22:09:27 EDT 2011, bakul@bitblocks.com wrote:
> On Wed, 01 Jun 2011 22:05:49 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > > > See rfc1122 2.4.3.9.
> > >
> > > ENOTFOUND
> >
> > i'm not sure what your point is.  and there's no such section in rfc 1122.
> >
> > http://tools.ietf.org/html/rfc1122#section-2.4
>
> Sorry. 4.2.3.9

okay, could you explain some more.  are you agreeing that
plan 9 tcp has a problem?

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Bakul Shah]

On Wed, 01 Jun 2011 22:13:35 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> On Wed Jun  1 22:09:27 EDT 2011, bakul@bitblocks.com wrote:
> > On Wed, 01 Jun 2011 22:05:49 EDT erik quanstrom <quanstro@quanstro.net>  wr
> ote:
> > > > > See rfc1122 2.4.3.9.
> > > >
> > > > ENOTFOUND
> > >
> > > i'm not sure what your point is.  and there's no such section in rfc 1122
> .
> > >
> > > http://tools.ietf.org/html/rfc1122#section-2.4
> >
> > Sorry. 4.2.3.9
>
> okay, could you explain some more.  are you agreeing that
> plan 9 tcp has a problem?

I thought reading the section would be enough. Sorry. Too much
multiplexing.
1) you have to look at the codes to know whether what you
   observed is a bug.
2) ignoring ICMP *except* during initial tcp negotiation seems
   suboptimal -- the destination can disappear completely at
   any time.  May be this results in an extra delay before a
   connection is declared dead.  But I no longer remember
   many details of tcp to know if this matters.
3) my main point was to check the rfcs to resolve such issues

[is agrep ported to plan9?]
$ rfc() {
        agrep -t -i -d'$$' $* ~/rfc/rfc-index.txt
}
$ rfc 1122
0793 Transmission Control Protocol. J. Postel. September 1981.
     (Format: TXT=172710 bytes) (Updated by RFC1122, RFC3168) (Also
     STD0007) (Status: STANDARD)

1122 Requirements for Internet Hosts - Communication Layers. R.
     Braden, Ed.. October 1989. (Format: TXT=295992 bytes) (Updates
     RFC0793) (Updated by RFC1349, RFC4379) (Also STD0003) (Status:
     STANDARD)

1349 Type of Service in the Internet Protocol Suite. P. Almquist. July
     1992. (Format: TXT=68949 bytes) (Obsoleted by RFC2474) (Updates
     RFC1248, RFC1247, RFC1195, RFC1123, RFC1122, RFC1060, RFC0791)
     (Status: PROPOSED STANDARD)

1508 Generic Security Service Application Program Interface. J. Linn.
     September 1993. (Format: TXT=111228 bytes) (Obsoleted by RFC2078)
     (Status: PROPOSED STANDARD)

3781 Next Generation Structure of Management Information (SMIng)
     Mappings to the Simple Network Management Protocol (SNMP). F.
     Strauss, J. Schoenwaelder. May 2004. (Format: TXT=112267 bytes)
     (Status: EXPERIMENTAL)

4379 Detecting Multi-Protocol Label Switched (MPLS) Data Plane
     Failures. K. Kompella, G. Swallow. February 2006. (Format: TXT=116872
     bytes) (Updates RFC1122) (Updated by RFC5462) (Status: PROPOSED
     STANDARD)

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

> [is agrep ported to plan9?]
> $ rfc() {
>         agrep -t -i -d'$$' $* ~/rfc/rfc-index.txt
> }

minooka; cat /bin/rfcgrep
#!/bin/rc

echo ',x/^rfc.*\n(^ .*)+\n/ g☺'$"*'☺ p' | sam -d /lib/rfc/index >[2=] |sed 's:^rfc:/lib/rfc/rfc:g'


- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

> I thought reading the section would be enough. Sorry. Too much
> multiplexing.
> 1) you have to look at the codes to know whether what you
>    observed is a bug.
> 2) ignoring ICMP *except* during initial tcp negotiation seems
>    suboptimal -- the destination can disappear completely at
>    any time.  May be this results in an extra delay before a
>    connection is declared dead.  But I no longer remember
>    many details of tcp to know if this matters.
> 3) my main point was to check the rfcs to resolve such issues

the code has nothing to do with my argument.  let's take the
current behavior of terminating connection attempts on icmp
unreachable messages as correct.  nobody's argued differently.
so we can take the rfc out of the argument as well.

so unless you want your kernel to randomly respond to inputs,
an appropriate icmp unreachable matching your connection
attempt has got to always kill the connection.  surely you wouldn't
have it otherwise?

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Bakul Shah]

On Wed, 01 Jun 2011 22:57:13 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
>
> the code has nothing to do with my argument.  let's take the
> current behavior of terminating connection attempts on icmp
> unreachable messages as correct.  nobody's argued differently.
> so we can take the rfc out of the argument as well.
>
> so unless you want your kernel to randomly respond to inputs,
> an appropriate icmp unreachable matching your connection
> attempt has got to always kill the connection.  surely you wouldn't
> have it otherwise?

imho you are thinking too hard.  plan9 behavior doesn't have
to be "intuitively" correct; it has to match what the rfc
says.  your first message said "icmp unreachable messages do
not terminate the connection". this is not necessarily a bug
-- depends on type and code. should kill if type 3
(desintation unreachable) & code 2 (protocol unreachable), 3
(port unreachable), 4 (DF set and fragmentation needed). not
so for other codes.  ignoring icmp except during initial
connection seems like a bug to me.  bottom line: rtf rfc! or
if you want to check working code, look at *bsd's.  i have
more faith in the bsd tcp code than the plan9 tcp code.  i am
repeating myself so i will stop.

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

> imho you are thinking too hard.  plan9 behavior doesn't have
> to be "intuitively" correct; it has to match what the rfc
> says.  your first message said "icmp unreachable messages do
> not terminate the connection". this is not necessarily a bug

no.  that's not what i said.  i said, icmp unreachable messages
*sometimes* terminate the connection.  and there's no excuse
for that unreliablity.  the packet was not dropped by the nic.

this discussion would be shorter if you'd read what i wrote.

- erik

Re: [9fans] icmp unreachable messages sometimes dropped

[Bakul Shah]

On Thu, 02 Jun 2011 00:17:25 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > imho you are thinking too hard.  plan9 behavior doesn't have
> > to be "intuitively" correct; it has to match what the rfc
> > says.  your first message said "icmp unreachable messages do
> > not terminate the connection". this is not necessarily a bug
>
> no.  that's not what i said.  i said, icmp unreachable messages
> *sometimes* terminate the connection.  and there's no excuse
> for that unreliablity.  the packet was not dropped by the nic.

no such thing as "icmp unreachable". presumably you mean icmp
with type == destination unreachable? but even that is
ambiguous. did you actually check it is the *same* type &
code? if so it is indeed a bug. may be that is what you meant
but your message was ambiguous.

Re: [9fans] icmp unreachable messages sometimes dropped

[erik quanstrom]

> > no.  that's not what i said.  i said, icmp unreachable messages
> > *sometimes* terminate the connection.  and there's no excuse
> > for that unreliablity.  the packet was not dropped by the nic.
>
> no such thing as "icmp unreachable". presumably you mean icmp
> with type == destination unreachable? but even that is
> ambiguous. did you actually check it is the *same* type &
> code? if so it is indeed a bug. may be that is what you meant
> but your message was ambiguous.

that is what i ment.  i'm not sure what else i can say.
i thought that http://9fans.net/archive/2011/06/2 was pretty
clear that the only difference was the host itself.  the icmp
messages were the same modulo host and connection specific
bits.

so i'm going to give up now.  it's clear i can't get my point
across.

- erik