[9fans] telnet vs. godaddy whois

[9fans] telnet vs. godaddy whois

[erik quanstrom]

does anyone know why telnet has trouble with this?

	; echo godaddy.com|telnet -nr /net.alt/tcp!whois.godaddy.com!43
	connected to /net.alt/tcp!whois.godaddy.com!43 on /net.alt/tcp/12
	;

from a similarly-connected linux machine, linux telnet returns a
lengthy answer.

- erik

Re: [9fans] telnet vs. godaddy whois

[Russ Cox]

> does anyone know why telnet has trouble with this?
>
>     ; echo godaddy.com|telnet -nr /net.alt/tcp!whois.godaddy.com!43
>     connected to /net.alt/tcp!whois.godaddy.com!43 on /net.alt/tcp/12
>     ;
>
> from a similarly-connected linux machine, linux telnet returns a
> lengthy answer.

It's not telnet's fault.  It's a TCP bug.

Here's a trace on Linux.  Notice that godaddy's SYN|ACK packet (34822ms)
advertises a zero-length receive window, so Linux has to wait
until it gets an ACK to its ACK to open the window (34899ms)
before it sends (34900ms).

    # /usr/local/plan9/bin/snoopy -f 'tcp(sd=43)' eth0
    after optimize: ether(ip(tcp(sd = 43)))
    034744 ms
        ether(s=000feafc0dbe d=00095bdb3254 pr=0800 ln=74)
        ip(s=192.168.0.99 d=68.178.211.43 id=9ca5 frag=4000 ttl= 64 pr=6 ln=60)
        tcp(s=42805 d=43 seq=1897121382 ack=0 fl=S win=5840 ck=d993 opt4=(mss 1460) opt2=(4 ) opt10=(8 45155AC300000000) opt=NOOP opt3=(wscale 7))
    034822 ms
        ether(s=00095bdb3254 d=000feafc0dbe pr=0800 ln=60)
        ip(s=68.178.211.43 d=192.168.0.99 id=9ca5 frag=0000 ttl= 31 pr=6 ln=40)
        tcp(s=43 d=42805 seq=3642134677 ack=1897121383 fl=AS win=0 ck=8e61)
    034822 ms
        ether(s=000feafc0dbe d=00095bdb3254 pr=0800 ln=54)
        ip(s=192.168.0.99 d=68.178.211.43 id=9ca6 frag=4000 ttl= 64 pr=6 ln=40)
        tcp(s=42805 d=43 seq=1897121383 ack=3642134678 fl=A win=5840 ck=7792)
    034899 ms
        ether(s=00095bdb3254 d=000feafc0dbe pr=0800 ln=60)
        ip(s=68.178.211.43 d=192.168.0.99 id=34a4 frag=0000 ttl=111 pr=6 ln=40)
        tcp(s=43 d=42805 seq=3642134678 ack=1897121383 fl=A win=16384 ck=4e62)
    034900 ms
        ether(s=000feafc0dbe d=00095bdb3254 pr=0800 ln=66)
        ip(s=192.168.0.99 d=68.178.211.43 id=9ca7 frag=4000 ttl= 64 pr=6 ln=52)
        tcp(s=42805 d=43 seq=1897121383 ack=3642134678 fl=AP win=5840 ck=d90f)
        dump(godaddy.com\n)
    035195 ms
        ether(s=00095bdb3254 d=000feafc0dbe pr=0800 ln=60)
        ip(s=68.178.211.43 d=192.168.0.99 id=34d7 frag=0000 ttl=111 pr=6 ln=40)
        tcp(s=43 d=42805 seq=3642134678 ack=1897121395 fl=A win=65523 ck=8e62)
    035265 ms
        ether(s=00095bdb3254 d=000feafc0dbe pr=0800 ln=1434)
        ip(s=68.178.211.43 d=192.168.0.99 id=3504 frag=0000 ttl=111 pr=6 ln=1420)
        tcp(s=43 d=42805 seq=3642134678 ack=1897121395 fl=A win=65523 ck=a8b6)
        dump(The data contained in GoDaddy.co)

Plan 9 ignores the zero length window and sends a single byte (2456ms),
causing godaddy to hang up (2493ms).

    cpu% snoopy -N 1500 -f 'tcp(sd=43)' /net/ether1
    after optimize: ether(ip(tcp(sd = 43)))
    002343 ms
        ether(s=0004238ecb1a d=0007b3f12c00 pr=0800 ln=62)
        ip(s=18.26.4.98 d=68.178.211.43 id=9330 frag=0000 ttl=255 pr=6 ln=48)
        tcp(s=32619 d=43 seq=1578393267 ack=0 fl=S win=65535 ck=1767 opt4=(mss 1460) opt3=(wscale 3) opt=NOOP)
    002418 ms
        ether(s=0007b3f12c00 d=0004238ecb1a pr=0800 ln=64)
        ip(s=68.178.211.43 d=18.26.4.98 id=9330 frag=0000 ttl=223 pr=6 ln=40)
        tcp(s=43 d=32619 seq=2734158449 ack=1578393268 fl=AS win=0 ck=afb0)
    002437 ms
        ether(s=0004238ecb1a d=0007b3f12c00 pr=0800 ln=60)
        ip(s=18.26.4.98 d=68.178.211.43 id=9339 frag=0000 ttl=255 pr=6 ln=40)
        tcp(s=32619 d=43 seq=1578393268 ack=2734158450 fl=AP win=65535 ck=afa9)
    002456 ms
        ether(s=0004238ecb1a d=0007b3f12c00 pr=0800 ln=60)
        ip(s=18.26.4.98 d=68.178.211.43 id=933a frag=0000 ttl=255 pr=6 ln=41)
        tcp(s=32619 d=43 seq=1578393268 ack=2734158450 fl=A win=65535 ck=48b0)
        dump(g)
    002493 ms
        ether(s=0007b3f12c00 d=0004238ecb1a pr=0800 ln=64)
        ip(s=68.178.211.43 d=18.26.4.98 id=9339 frag=0000 ttl=223 pr=6 ln=40)
        tcp(s=43 d=32619 seq=2734158450 ack=1578393268 fl=AR win=65535 ck=afad)

The source is in /sys/src/9/ip/tcp.c.  Have fun.

Russ

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> Plan 9 ignores the zero length window and sends a single byte (2456ms),
> causing godaddy to hang up (2493ms).

it is probing the zero window (rfc793, section 3.7)

Re: [9fans] telnet vs. godaddy whois

[Michaelian Ennis]

On Wed, Apr 16, 2008 at 9:46 AM, Charles Forsyth <forsyth@terzarima.net> wrote:
> > Plan 9 ignores the zero length window and sends a single byte (2456ms),
>  > causing godaddy to hang up (2493ms).
>
>  it is probing the zero window (rfc793, section 3.7)

Not that I think that fingerprinting is definitive but nmap thinks
whois at godaddy is running:

Captor embedded, QNX 4.X

Ian

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

> > Plan 9 ignores the zero length window and sends a single byte (2456ms),
> > causing godaddy to hang up (2493ms).
>
> it is probing the zero window (rfc793, section 3.7)

for those, like me, who have not committed 793 to memory:

	"When the receiving TCP has a zero window and a segment arrives it must
	still send n acknowledgment show its next expected sequence number
	and current window (zero).

p. 41 near the bottom.  looks like godaddy's tcp has the bug.

- erik

Re: [9fans] telnet vs. godaddy whois

[ron minnich]

This is really an interesting discussion -- anybody think it could go
on the wiki? I enjoyed it anyway :-)

A good example of how correct behaviour (in this case Plan 9) can get
you spanked.

ron

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Wed, 16 Apr 2008 12:04:23 PDT "ron minnich" <rminnich@gmail.com>  wrote:
> This is really an interesting discussion -- anybody think it could go
> on the wiki? I enjoyed it anyway :-)
>
> A good example of how correct behaviour (in this case Plan 9) can get
> you spanked.

Er... "correct" seems a bit strong. Why is Plan9 sending one
byte of data when it knows the receiver's window is closed?

Here is part of the plan9 trace Russ posted:
    002418 ms
        ether(s=0007b3f12c00 d=0004238ecb1a pr=0800 ln=64)
        ip(s=68.178.211.43 d=18.26.4.98 id=9330 frag=0000 ttl=223 pr=6 ln=40)
        tcp(s=43 d=32619 seq=2734158449 ack=1578393268 fl=AS win=0 ck=afb0)
    002437 ms
        ether(s=0004238ecb1a d=0007b3f12c00 pr=0800 ln=60)
        ip(s=18.26.4.98 d=68.178.211.43 id=9339 frag=0000 ttl=255 pr=6 ln=40)
        tcp(s=32619 d=43 seq=1578393268 ack=2734158450 fl=AP win=65535 ck=afa9)
    002456 ms
        ether(s=0004238ecb1a d=0007b3f12c00 pr=0800 ln=60)
        ip(s=18.26.4.98 d=68.178.211.43 id=933a frag=0000 ttl=255 pr=6 ln=41)
        tcp(s=32619 d=43 seq=1578393268 ack=2734158450 fl=A win=65535 ck=48b0)
        dump(g)

See RFC793, page 4, last para:
    TCP provides a means for the receiver to govern the amount of data
    sent by the sender.  This is achieved by returning a "window" with
    every ACK indicating a range of acceptable sequence numbers beyond
    the last segment successfully received.  The window indicates an
    allowed number of octets that the sender may transmit before
    receiving further permission.

Re: [9fans] telnet vs. godaddy whois

[Tim Wiess]

> On Wed, 16 Apr 2008 12:04:23 PDT "ron minnich" <rminnich@gmail.com>  wrote:
>> This is really an interesting discussion -- anybody think it could go
>> on the wiki? I enjoyed it anyway :-)
>>
>> A good example of how correct behaviour (in this case Plan 9) can get
>> you spanked.
>
> Er... "correct" seems a bit strong. Why is Plan9 sending one
> byte of data when it knows the receiver's window is closed?
>

Lookup TCP persist timer and window probes.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> Er... "correct" seems a bit strong. Why is Plan9 sending one
> byte of data when it knows the receiver's window is closed?

read the section of the rfc i mentioned earlier.  it probably ought to probe
only after a retransmission timeout period, but that's not a predetermined value,
and can be short. very short. and plan 9 is positively curt, so it's just
typical that others should then complain.

Re: [9fans] telnet vs. godaddy whois

[Taj Khattra]

>  read the section of the rfc i mentioned earlier.  it probably ought to probe
>  only after a retransmission timeout period

i believe bsd-based tcp stacks also send 1-byte zero-window probes
but use a persist timer that starts at approx. 5 seconds (*)

i wonder if godaddy's tcp does not reset in that case - anyone care to verify?

(*) section 4 of
http://www.usenix.org/publications/library/proceedings/bos94/full_papers/lin.a

Re: [9fans] telnet vs. godaddy whois

[John Barham]

> >  read the section of the rfc i mentioned earlier.  it probably ought to probe
>  >  only after a retransmission timeout period
>
>  i believe bsd-based tcp stacks also send 1-byte zero-window probes
>  but use a persist timer that starts at approx. 5 seconds (*)

That's the behaviour that's suggested in Stevens' TCP/IP Illustrated,
Volume 1.  The relevant section is post (legally?) here:
http://www.uic.rsu.ru/doc/inet/tcp_stevens/tcp_pers.htm

Re: [9fans] telnet vs. godaddy whois

[C H Forsyth]

>i wonder if godaddy's tcp does not reset in that case - anyone care to verify?

i cannot think of a sensible reason it should do that: it seems more work for the receiver,
to run an extra timer to time the probes. even when the probes are based on the retransmission
timer they could easily occur on the order of tens or hundreds of microseconds on some networks.
you don't want something as big as 5 seconds because the window-opening packet might
have been zapped, and who wants 5 second delays because of that?
it's not for nothing that rfc793 and rfc1122 say what they do.

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Wed, 16 Apr 2008 21:49:26 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
> > Er... "correct" seems a bit strong. Why is Plan9 sending one
> > byte of data when it knows the receiver's window is closed?
>
> read the section of the rfc i mentioned earlier.  it probably ought to probe
> only after a retransmission timeout period, but that's not a predetermined va
> lue,
> and can be short. very short. and plan 9 is positively curt, so it's just
> typical that others should then complain.

RFC1122 page 92 says

 The transmitting host SHOULD send the first zero-window
 probe when a zero window has existed for the retransmission
 timeout period (see Section 4.2.2.15), and SHOULD increase
 exponentially the interval between successive probes.

Implementations other than Plan 9's certainly seem to wait
much longer than 19ms before the first probe.  RFC1122
suggests initializing RTO to 3 seconds and then adjusting it.

Anyway, what I really wanted to say was that somehow
"correctness" is not what comes to mind first when thinking
about TCP!  The important question is whether your
implementation interoperates with the ones your users care
about (or must deal with even if buggy).

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

>  The transmitting host SHOULD send the first zero-window
>  probe when a zero window has existed for the retransmission
>  timeout period (see Section 4.2.2.15), and SHOULD increase
>  exponentially the interval between successive probes.

that first part requires an action in a given case, to ensure it is done (the
context is given by some preceding text not quoted here). it doesn't specify
all cases in which it might be done.

having said that, i now suspect that sending one byte into a zero-window is not the problem.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> having said that, i now suspect that sending one byte into a zero-window is not the problem.

because the one-byte probe can only be done if there is data to send, and i already knew that
a plain connection (dial only) to that port also failed:

014045 ms
	ether(s=000e2e32f2a6 d=00a0cc3cd49b pr=0800 ln=62)
	ip(s=144.32.112.70 d=68.178.211.43 id=d002 frag=0000 ttl=255 pr=6 ln=48)
	tcp(s=59115 d=43 seq=3422370319 ack=0 fl=S win=65535 ck=7fb6 opt4=(mss 1460) opt3=(wscale 0) opt=NOOP)
014201 ms
	ether(s=00a0cc3cd49b d=000e2e32f2a6 pr=0800 ln=60)
	ip(s=68.178.211.43 d=144.32.112.70 id=d002 frag=0000 ttl=223 pr=6 ln=40)
	tcp(s=43 d=59115 seq=2009405788 ack=3422370320 fl=AS win=0 ck=1948)
014205 ms
	ether(s=000e2e32f2a6 d=00a0cc3cd49b pr=0800 ln=60)
	ip(s=144.32.112.70 d=68.178.211.43 id=d005 frag=0000 ttl=255 pr=6 ln=40)
	tcp(s=59115 d=43 seq=3422370320 ack=2009405789 fl=AP win=65535 ck=1941)
014342 ms
	ether(s=00a0cc3cd49b d=000e2e32f2a6 pr=0800 ln=60)
	ip(s=68.178.211.43 d=144.32.112.70 id=d005 frag=0000 ttl=223 pr=6 ln=40)
	tcp(s=43 d=59115 seq=2009405789 ack=3422370320 fl=AR win=65535 ck=1945)

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Thu, 17 Apr 2008 09:18:31 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
> > having said that, i now suspect that sending one byte into a zero-window is
>  not the problem.
>
> because the one-byte probe can only be done if there is data to send, and i
> already knew that a plain connection (dial only) to that port also failed:

Not setting the PSH bit on a pure ACK (== no data is being
sent) seems to fix this (see ip/tcp.c around line 2530).  May
be it tickles a bug on the receiver (0 byte read?).

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

> On Thu, 17 Apr 2008 09:18:31 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
>> > having said that, i now suspect that sending one byte into a zero-window is
>>  not the problem.
>>
>> because the one-byte probe can only be done if there is data to send, and i
>> already knew that a plain connection (dial only) to that port also failed:
>
> Not setting the PSH bit on a pure ACK (== no data is being
> sent) seems to fix this (see ip/tcp.c around line 2530).  May
> be it tickles a bug on the receiver (0 byte read?).

this does work for me.  is there some subtile reason *to* set the psh bit
on a pure ack?  in certain circumstances?

good call. from rfc793, p 42:

      [...]  If the the user signals a push function then the
      data must be sent even if it is a small segment.

minooka; 9diff ip/tcp.c
/n/sources/plan9//sys/src/9/ip/tcp.c:2529,2535 - ip/tcp.c:2529,2535
  			}
  		}

- 		if(sent+dsize == sndcnt)
+ 		if(sent+dsize == sndcnt && dsize)
  			seg.flags |= PSH;

  		/* keep track of balance of resent data */

- erik

Re: [9fans] telnet vs. godaddy whois

[Tim Wiess]

>> On Thu, 17 Apr 2008 09:18:31 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
>>> > having said that, i now suspect that sending one byte into a zero-window is
>>>  not the problem.
>>>
>>> because the one-byte probe can only be done if there is data to send, and i
>>> already knew that a plain connection (dial only) to that port also failed:
>>
>> Not setting the PSH bit on a pure ACK (== no data is being
>> sent) seems to fix this (see ip/tcp.c around line 2530).  May
>> be it tickles a bug on the receiver (0 byte read?).
>
> this does work for me.  is there some subtile reason *to* set the psh bit
> on a pure ack?  in certain circumstances?
>
> good call. from rfc793, p 42:
>
>       [...]  If the the user signals a push function then the
>       data must be sent even if it is a small segment.
>
> minooka; 9diff ip/tcp.c
> /n/sources/plan9//sys/src/9/ip/tcp.c:2529,2535 - ip/tcp.c:2529,2535
>   			}
>   		}
>
> - 		if(sent+dsize == sndcnt)
> + 		if(sent+dsize == sndcnt && dsize)
>   			seg.flags |= PSH;
>
>   		/* keep track of balance of resent data */
>
> - erik

    I noticed this some time ago when I was doing some work in the
    stack and thought it was very questionable.  But I never got a
    chance to go back and do further research.  Nevertheless I think
    it's the wrong behavior.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

>     I noticed this some time ago when I was doing some work in the
>     stack and thought it was very questionable.  But I never got a
>     chance to go back and do further research.  Nevertheless I think
>     it's the wrong behavior.

what's the definition of `wrong' here?

Re: [9fans] telnet vs. godaddy whois

[Tim Wiess]

>>     I noticed this some time ago when I was doing some work in the
>>     stack and thought it was very questionable.  But I never got a
>>     chance to go back and do further research.  Nevertheless I think
>>     it's the wrong behavior.
>
> what's the definition of `wrong' here?

    Meaning that the patch Eric proposed is probably the better way to
    deal with ACKs.  It wasn't meant to be taken too literally though,
    hence the "I think".

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Thu, 17 Apr 2008 15:29:42 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > On Thu, 17 Apr 2008 09:18:31 BST Charles Forsyth <forsyth@terzarima.net>  w
> rote:
> >> > having said that, i now suspect that sending one byte into a zero-window
>  is
> >>  not the problem.
> >>
> >> because the one-byte probe can only be done if there is data to send, and
> i
> >> already knew that a plain connection (dial only) to that port also failed:
> >
> > Not setting the PSH bit on a pure ACK (== no data is being
> > sent) seems to fix this (see ip/tcp.c around line 2530).  May
> > be it tickles a bug on the receiver (0 byte read?).
>
> this does work for me.  is there some subtile reason *to* set the psh bit
> on a pure ack?  in certain circumstances?

While I wouldn't call it "wrong" it seems silly to set the
PSH bit when no data is being sent.  See rfc1122 section
4.2.2.2. Among other things it says:

    At the receiver, the PSH bit forces buffered data to be
    delivered to the application (even if less than a full
    buffer has been received).

Because of this what is likely happening is that on receiving
the PSH bit read() completes and returns to the caller app
with a count = 0 which the app must think indicates EOF!

May be this behavior in plan9 is due to a misplaced right
brace?  Move the closing brace on line 2530 to below the PSH
bit logic and all is fine.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> Because of this what is likely happening is that on receiving
> the PSH bit read() completes and returns to the caller app
> with a count = 0 which the app must think indicates EOF!

that behaviour (by the remote) is correct?

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

>> Because of this what is likely happening is that on receiving
>> the PSH bit read() completes and returns to the caller app
>> with a count = 0 which the app must think indicates EOF!
>
> that behaviour (by the remote) is correct?

if we've sent an illegal packet, can we say any behaviour on
the remote end is incorrect?

- erik

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> what's the definition of `wrong' here?
>     Meaning that the patch Eric proposed is probably the better way to
>     deal with ACKs.  It wasn't meant to be taken too literally though,
>     hence the "I think".

what's the definition of `better' here?

well, i won't persist in pedantry. i was just curious about the rationale for the
adjectives. i'd say it isn't really to do with ACKs: the protocol definition rightly
has ACK and PSH interpreted by different sides at the destination: input for ACK and output for PSH.
in fact, the Plan 9 behaviour is rational for a sluggish or zero window: the receiving side might
delay delivering data to the application until a PSH, but won't open the window if that queue is full.
(thus rfc1122 mutters about deadlock in the absence of PSH, in 4.2.2.2.)

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

FreeBSD 5.0-CURRENT (which is old, but one i've got) didn't work with echo  | telnet godaddy either, although
i haven't traced that yet.   it's the same on my powerpc mac, though.  there's another tcp subtlety that
could cause that though, assuming it's not just telnet getting in the way.

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Thu, 17 Apr 2008 22:49:02 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
> > Because of this what is likely happening is that on receiving
> > the PSH bit read() completes and returns to the caller app
> > with a count = 0 which the app must think indicates EOF!
>
> that behaviour (by the remote) is correct?

First, I am just speculating -- I have no idea what they are
running -- but if this is what happens, it is about as
sensible as setting PSH on a pure ACK.

In a later email you say:
> in fact, the Plan 9 behaviour is rational for a sluggish or zero window:
> the receiving side might delay delivering data to the application until
> a PSH, but won't open the window if that queue is full.
> (thus rfc1122 mutters about deadlock in the absence of PSH, in 4.2.2.2.)

I think you are referring to

    An application program is logically required to set the
    PUSH flag in a SEND call whenever it needs to force
    delivery of the data to avoid a communication deadlock.

Note the phrase "to force delivery of the data".  There is no
data to be sent in the case we are discussing.  If you were
to set PSH when you send 1 byte (or more) to probe a closed
window that'd be fine but not with a pure ACK.  So I read it
differently.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> if we've sent an illegal packet, can we say any behaviour on
> the remote end is incorrect?

that packet is by no means `illegal'.

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

>> if we've sent an illegal packet, can we say any behaviour on
>> the remote end is incorrect?
>
> that packet is by no means `illegal'.

rfc 742 p. 42 says

      [...] If the the user signals a push function then the
      data must be sent even if it is a small segment.

by "illegal" i mean goes contrary to an rfc "must."  perhaps
i'm missing something.

- erik

Re: [9fans] telnet vs. godaddy whois

[Tim Wiess]

>> what's the definition of `wrong' here?
>>     Meaning that the patch Eric proposed is probably the better way to
>>     deal with ACKs.  It wasn't meant to be taken too literally though,
>>     hence the "I think".
>
> what's the definition of `better' here?
>
> well, i won't persist in pedantry. i was just curious about the rationale for the
> adjectives. i'd say it isn't really to do with ACKs: the protocol definition rightly
> has ACK and PSH interpreted by different sides at the destination: input for ACK and output for PSH.
> in fact, the Plan 9 behaviour is rational for a sluggish or zero window: the receiving side might
> delay delivering data to the application until a PSH, but won't open the window if that queue is full.
> (thus rfc1122 mutters about deadlock in the absence of PSH, in 4.2.2.2.)

    My rationale was the section of the rfc that Eric quoted with his
    patch, which seemed to address my earlier suspicions.  But I just
    went back over that section and realize that I misinterperted that
    quote a little.  So I don't believe that P9's behavior is "wrong"
    in the sense that's it's violating any assumptions in the rfc.  It
    is (I think) the first time I've seen PSH being set on a plain ACK,
    but I do understand your argument.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> rfc 742 p. 42 says
>
>       [...] If the the user signals a push function then the
>       data must be sent even if it is a small segment.
>
> by "illegal" i mean goes contrary to an rfc "must."  perhaps
> i'm missing something.

i don't see how what was sent is contrary to that requirement.

>sensible as setting PSH on a pure ACK.

i don't understand this reference to a  `pure' ACK. it's an ACK! in TCP/IP every
packet after SYN must have an ACK (or that really is -- explicitly -- illegal).
the ACK and PSH have nothing to do with each other.
in fact, the receiver isn't handling the PSH oddly because it's associated
with an ACK, but because it misinterpreted the standard, or the standard isn't clear.

Re: [9fans] telnet vs. godaddy whois

[Tim Wiess]

>> rfc 742 p. 42 says
>>
>>       [...] If the the user signals a push function then the
>>       data must be sent even if it is a small segment.
>>
>> by "illegal" i mean goes contrary to an rfc "must."  perhaps
>> i'm missing something.
>
> i don't see how what was sent is contrary to that requirement.
>
>>sensible as setting PSH on a pure ACK.
>
> i don't understand this reference to a  `pure' ACK. it's an ACK! in TCP/IP every
> packet after SYN must have an ACK (or that really is -- explicitly -- illegal).
> the ACK and PSH have nothing to do with each other.
> in fact, the receiver isn't handling the PSH oddly because it's associated
> with an ACK, but because it misinterpreted the standard, or the standard isn't clear.

    By pure I assume he meant an ACK with no data, which is what I
    also meant by "plain ACK".  But I agree with Charles here.  After
    going back over the related sections of the RFC I don't see how
    this behavior violates anything in the standard.  It's just not
    very common, and obviously not interpreted very well by this
    particular endpoint.  Has anybody ever experienced this problem
    before with any of there P9 systems?  I haven't.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

to be fair, this is one reason a few programming languages have non-trivial validation suites,
much of which check probable or historical misunderstandings,
and those suites are usually too small.  it takes a fair amount of back-and-forth through
the natural language text to build a supposedly complete specification.
the TCP/IP specification is tricky, partly because it suggests a programming interface as well,
which isn't quite the one that most people use today.  it's not just us: RFC1144 notes
	`PUSH' is a curious anachronism considered indispensable by certain members of the Internet
	community.  Since PUSH can (and does) change in any datagram, an
	information preserving compression scheme must pass it explicitly.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> Has anybody ever experienced this problem
>     before with any of there P9 systems?  I haven't.

not this particular problem, but years ago i had problems with plan 9 or perhaps it was inferno
(originally) not implementing the window test correctly
(leading to a RST storm with an incorrect AIX implementation),
and difficulty talking to implementations that completely screwed up the handling of the wrap-around
sequence number space, leading to needless disconnects depending on initial sequence number.
one was a tcp/ip implementation that's still popular in the embedded space.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

anyway, perhaps the more important question, at least for erik, is: will
his change cause trouble elsewhere?  unfortunately, we don't know, but we'll
see how he gets along!

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

On Thu Apr 17 19:07:09 EDT 2008, forsyth@terzarima.net wrote:
> anyway, perhaps the more important question, at least for erik, is: will
> his change cause trouble elsewhere?  unfortunately, we don't know, but we'll
> see how he gets along!
>

not setting the PSH bit when there's no data does fix the problem and
has run for several days on some well-used machines without causing
any issues with telnet, http, smtp, srv, cpu, import -E ssl &c.  i would be surprised
if the change caused any problem between plan 9 machines as the PSH bit
may be set but is never tested.

bwc points out that godaddy's behavior is very likely a violation of the rfc.
while the PSH bit may be mistaken, compliant implementations are supposed
to be liberal with what they accept.  i can only assume that they are trying to
defend against some sort of dos attack.  perhaps someone has a better suggestion?

it was suggested that the } was prehaps misplaced.  i think this is not correct
as the preceeding if modifieds dsize so i believe the ifs need to be seperate.

/n/sources/plan9//sys/src/9/ip/tcp.c:2529,2535 - tcp.c:2529,2535
  			}
  		}

- 		if(sent+dsize == sndcnt)
+ 		if(sent+dsize == sndcnt && dsize)
  			seg.flags |= PSH;

  		/* keep track of balance of resent data */

- erik

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> i can only assume that they are trying to
> defend against some sort of dos attack.  perhaps someone has a better suggestion?

it depends what they actually are running on that machine.
i've seen several broken tcp/ip implementations in embedded systems.
fairly often they mess up handling of the sequence number space, including
one that's (apparently) commonly used in consumer devices.

a stream of back-to-back acks would cause systems enough work with or without the PSH
so it's hard to see what DOS could be involved. i think it's more likely whoever wrote
it misread the spec, or simply made a mistake when coding it.

>is there something in particular that you suspect to misbehave?

to answer an earlier question, i think not setting PSH on a forced ACK will make no difference
provided the previous segment is (always) guaranteed to have PSH set, and no intervening
node removes that PSH (for instance by having failed to read the TCP/IP compression spec carefully
enough).  otherwise you might need the PSH to nudge a remote receiver that implements PSH
as part of its buffering scheme.

there isn't much to be done about the second case (since it involves other systems) but you'd need to check that
plan 9's implementation will always set PSH on the last-sent segment in the case(s) where one of
those forced ACKs would occur.

PSH isn't interpreted by (most) unix-like systems because their buffering scheme doesn't need it
(they typically queue data as it's received). if someone were to implement rfc793's suggestion
then they would need it, or the data will sit unread, which can mess up higher-level
protocols.  it's an old contentious topic: the tcp/ip compression rfc1144 grumbles

	`PUSH' is a curious anachronism considered indispensable by
	certain members of the Internet community.  Since PUSH can
	(and does) change in any datagram, an information preserving
	compression scheme must pass it explicitly.

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Mon, 21 Apr 2008 10:56:42 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
...
> bwc points out that godaddy's behavior is very likely a violation of the rfc.

I am not convinced any rfc covers this situation - it may be
that their tcp layer does the right thing and the bug is at
the application level. But in any case setting PSH on a
packet with no data serves no real purpose.  *BSD, Linux and
Windows don't set PSH on such packets either.

> it was suggested that the } was prehaps misplaced.  i think this is not
> correct as the preceeding if modifieds dsize so i believe the ifs need to be
> seperate.

I meant this:
                /* Pull out data to send */
                bp = nil;
                if(dsize != 0) {
                        bp = qcopy(s->wq, dsize, sent);
                        if(BLEN(bp) != dsize) {
                                seg.flags |= FIN;
                                dsize--;
                        }
                        if(sent+dsize == sndcnt)
                                seg.flags |= PSH;
                }

Seems clearer to me.  And equivalent!  I have been running
with this change since last Thursday. I don't stress my plan9
machine all that much but replica pulls, ftp, web browsing,
nfs etc. have worked fine.

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

> 	`PUSH' is a curious anachronism considered indispensable by
> 	certain members of the Internet community.  Since PUSH can
> 	(and does) change in any datagram, an information preserving
> 	compression scheme must pass it explicitly.

psh might be harder to understand than preserving message boundaries, but, hey,
it's less useful and easier to get wrong.

- erik

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> But in any case setting PSH on a
> packet with no data serves no real purpose.

i think that's incorrect: it ensures a push of any data that is already buffered but un-pushed
(ie, the immediately preceding segment had no PSH, and the receiver's implementation buffers
accordingly).

part of the problem with the continued specification of protocols, even 30 years on,
as `formal' english text is that it can appear to be ambiguous when it's just subtly precise.
that's why i had to say `i think' as opposed to QED

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> But in any case setting PSH on a
> packet with no data serves no real purpose.

i think that's incorrect: it ensures a push of any data that is already buffered but un-pushed
(ie, the immediately preceding segment had no PSH, and the receiver's implementation buffers
accordingly).

part of the problem with the continued specification of protocols, even 30 years on,
as `formal' english text is that it can appear to be ambiguous when it's just subtly precise.
that's why i had to say `i think' as opposed to QED

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> psh might be harder to understand than preserving message boundaries, but, hey,
> it's less useful and easier to get wrong.

absolutely!

worrying, isn't it.

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Mon, 21 Apr 2008 21:19:33 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
> > But in any case setting PSH on a
> > packet with no data serves no real purpose.
>
> i think that's incorrect: it ensures a push of any data that is already buffe
> red but un-pushed
> (ie, the immediately preceding segment had no PSH, and the receiver's impleme
> ntation buffers
> accordingly).

Other major implementations don't set PSH on a pkt with no
data so clearly they don't agree with you.  Here's why:

According to rfc1122 page 83,

    The PSH bit is not a record marker and is independent of
    segment boundaries.  The transmitter SHOULD collapse
    successive PSH bits when it packetizes data, to send the
    largest possible segment.

     A TCP MAY implement PUSH flags on SEND calls.  If PUSH
     flags are not implemented, then the sending TCP: (1)
     must not buffer data indefinitely, and (2) MUST set the
     PSH bit in the last buffered segment (i.e., when there
     is no more queued data to be sent).

The implication is that the "preceding segment" to a pkt with
no data *will have* PSH set.

> part of the problem with the continued specification of protocols, even 30
> years on, as `formal' english text is that it can appear to be ambiguous when
> it's just subtly precise. that's why i had to say `i think' as opposed to QED

Indeed.  One point to note is that RFCs (at least used to)
document existing implementations but some times not
everything is documented clearly and precisely.  Unlike
programming language implementations where things left
undefined by the specification can be implemented however you
wish, here you have to be compatible with existing
implementations as far as possible (in order to maximize
interoperability).

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

>      must not buffer data indefinitely, and (2) MUST set the
>      PSH bit in the last buffered segment (i.e., when there
>      is no more queued data to be sent).
>
> The implication is that the "preceding segment" to a pkt with
> no data *will have* PSH set.

so does the implementation do that?
can you prove it in all cases?
what will break if we just change it without knowing?
after all, it has been 15 years to come across a botched receiver's implementation
of PSH (ie, godaddy's) which is the only reason to change it.
that's what i was pointing out. i could do the work myself, i suppose, but i haven't got the incentive.

>here you have to be compatible with existing
>implementations as far as possible (in order to maximize
>interoperability).

i suspect arguments like that caused the current situation with HTML, CSS and Javascript.
computing is needlessly regressing.

Re: [9fans] telnet vs. godaddy whois

[Wes Kussmaul]

Charles Forsyth wrote:
> computing is needlessly regressing.
>

And it will continue to regress until one knowledgeable and independent
human being serves as final arbiter of standards.

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

> Charles Forsyth wrote:
> > computing is needlessly regressing.
> >
>
> And it will continue to regress until one knowledgeable and independent
> human being serves as final arbiter of standards.
>

good idea.  why don't you ask ken?

- erik

Re: [9fans] telnet vs. godaddy whois

[erik quanstrom]

> I meant this:
>                 /* Pull out data to send */
>                 bp = nil;
>                 if(dsize != 0) {
>                         bp = qcopy(s->wq, dsize, sent);
>                         if(BLEN(bp) != dsize) {
>                                 seg.flags |= FIN;
>                                 dsize--;
>                         }
>                         if(sent+dsize == sndcnt)
>                                 seg.flags |= PSH;
>                 }
>
> Seems clearer to me.  And equivalent!  I have been running
> with this change since last Thursday. I don't stress my plan9
> machine all that much but replica pulls, ftp, web browsing,
> nfs etc. have worked fine.

i think they are not equivalent with these values

	BLEN(bp) != dsize
	dsize == 1
	sent+0 == sndcnt

- erik

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

> And it will continue to regress until one knowledgeable and independent
> human being serves as final arbiter of standards.

i think some of it eventually will be formalised, much as we do with programming
languages (even Javascript, which i mentioned, at least has a plausible grammar),
but it seems we still haven't got a suitable tool to do it (or at least, not one that enough
people use without fuss).   even then a formalised version of something can still have
(more formal) bugs, that fail to express an intention correctly.

anyway, just to be helpful: Bakul is right that as in Erik's case, for networked applications particularly,
you end up having to be pragmatic when talking to other implementations.  for example,
the (old) Mac POP3 client demanded a space at a certain point, even when there was
no argument (required by the POP3 RFC).  most other server implementations included something
chatty there, and the POP3 client implementation had followed the servers, not the RFC.

Re: [9fans] telnet vs. godaddy whois

[Wes Kussmaul]

[-- Attachment #1: Type: text/html, Size: 870 bytes --]

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Mon, 21 Apr 2008 22:24:35 BST Charles Forsyth <forsyth@terzarima.net>  wrote:
> >      must not buffer data indefinitely, and (2) MUST set the
> >      PSH bit in the last buffered segment (i.e., when there
> >      is no more queued data to be sent).
> >
> > The implication is that the "preceding segment" to a pkt with
> > no data *will have* PSH set.
>
> so does the implementation do that?

Do you mean plan9 after the change?  The traces I looked at
seem to do that. Others certainly seem to do that.

> can you prove it in all cases?

Not in the formal sense. Not enough free time or incentive.

> what will break if we just change it without knowing?
> after all, it has been 15 years to come across a botched receiver's implement
> ation
> of PSH (ie, godaddy's) which is the only reason to change it.
> that's what i was pointing out. i could do the work myself, i suppose, but i
> haven't got the incentive.

I understand your concern about possibly breaking things with
this change. It should certainly be tested more thoroughly
but since the change brings Plan9 behavior more in line with
what *BSD/Linux/Windows do I am not as apprehensive as you
are.

> >here you have to be compatible with existing
> >implementations as far as possible (in order to maximize
> >interoperability).
>
> i suspect arguments like that caused the current situation with HTML, CSS and
>  Javascript.
> computing is needlessly regressing.

May be. Somehow this makes me think of E W Dijkstra :-)

Re: [9fans] telnet vs. godaddy whois

[Bakul Shah]

On Mon, 21 Apr 2008 17:49:35 EDT erik quanstrom <quanstro@quanstro.net>  wrote:
> > I meant this:
> >                 /* Pull out data to send */
> >                 bp = nil;
> >                 if(dsize != 0) {
> >                         bp = qcopy(s->wq, dsize, sent);
> >                         if(BLEN(bp) != dsize) {
> >                                 seg.flags |= FIN;
> >                                 dsize--;
> >                         }
> >                         if(sent+dsize == sndcnt)
> >                                 seg.flags |= PSH;
> >                 }
> >
> > Seems clearer to me.  And equivalent!  I have been running
> > with this change since last Thursday. I don't stress my plan9
> > machine all that much but replica pulls, ftp, web browsing,
> > nfs etc. have worked fine.
>
> i think they are not equivalent with these values
>
> 	BLEN(bp) != dsize
> 	dsize == 1
> 	sent+0 == sndcnt

I don't think all three conditions can be true. dsize is set
to ssize, ssize is initialized to sndcnt - sent and after
that it can only decrease. So if dsize is 1, sndcnt - send
must be at least 1.

Re: [9fans] telnet vs. godaddy whois

[Charles Forsyth]

having looked again at ip/tcp.c i think the code wasn't really intending
to resolve one of the stalled receiver cases i had in mind, although it happens to do so,
so changing it probably doesn't mess up some original intent.

mind you, one lesson i take from all this is that in retrospect one could expect
just about anything from a server run by a company like godaddy that
completely misses the point about black leather jackets.  (they look
cool only if you don't scribble adverts all over them.)