Re: [9fans] datakit

Re: [9fans] datakit

[dmr]

Geoff's brief recapitulation of Datakit is pretty
much on the mark.  I'd emphasize these aspects:

It definitely was a logically circuit-switched network.
This meant (e.g) there were natural ways for requesting bandwith
allocation (instead of inferring flows), though I don't
think this was really taken advantage of.

The design was meant to cater to host-to-host communication
and this was the way we really used it in our own group
and to some extent throughout Bell Labs.  However, most
of the sales were as serial asynchronous concentrators
over modems.  Direct host interfaces were always relatively
expensive; at the start, probably not too much more than Ethernet
interfaces, but the rapid growth of Ethernet and consequent
cost reductions wiped out the more proprietary scheme.

(There was indeed better local security because DK was not
a broadcast, shared-medium network like EN, but
that's not where most people's problems are today).

In the aftermath, perhaps the most valuable effect of
dealing with Datakit was to enourage the generalized
and flexible approach to networking begun in 8th edition Unix
that is carried forward into Plan 9.

	Dennis

Re: [9fans] datakit

[George Michaelson]

early ethernets were disgusting. I used one which was a standalone S100-bus
box the size of a microwave oven, with DR11-W cabling to a Vax, The only
network we had on it was cu/tip.

it would not have been hard for Datakit to be remarkably better than this.

as a single-vendor solution, from the days when the equipment seller was also
the network provider, I think Datakit would have been nice to use. But in a
multiple-provider world, the same pressures we have now on vendor neutrality,
inter-provider addressing, end-to-end-ness would be driving it into a plethora
of (non compatible) variants, and problems.


-George

[9fans] datakit

[Steve Simon]

Purely out of noseyness, is there any datakit left
or has it all been scrapped? If so when did it finally
bite the dust?

I always thought it a very elegant system - though I only read
about the hardware level.

-Steve

Re: [9fans] datakit

[geoff]

The Math and CS Research Datakit nodes were taken out of service while
I was at the Labs.  I was told that Datakit technology is used in the
field, though I don't recall details.  It was expensive and the 1127
folks seemed glad to be rid of it, though it certainly did seem to me
to have some very desirable properties for a network.

Instead we now have IP running over dumb networks that guarantee us
very little and for which the solution to every problem seems to be
another protocol/RFC.  At least it's a full-employment act for
programmers (or would be in a working economy).

Re: [9fans] datakit

[George Michaelson]

On Thu, 19 Aug 2004 18:35:18 -0700 geoff@collyer.net wrote:

>The Math and CS Research Datakit nodes were taken out of service while
>I was at the Labs.  I was told that Datakit technology is used in the
>field, though I don't recall details.  It was expensive and the 1127
>folks seemed glad to be rid of it, though it certainly did seem to me
>to have some very desirable properties for a network.

which ones? (no really, I'm interested whats your initial bullet list on this,
because I suspect people don't converge as much as they think they do on them)

>
>Instead we now have IP running over dumb networks that guarantee us
>very little and for which the solution to every problem seems to be
>another protocol/RFC.  At least it's a full-employment act for
>programmers (or would be in a working economy).

Is this code for 'I do not support the current embodyment of the
end-to-end principle' (which is a very over-used concept, but I still
think has some merit)

because dumb networks seem to 'work better' for many measures, beyond the ones
about maintaining the cabal of alchemists who run them.

Datakit never made it offshore that I know. US telco technology which works
often does make it off shore (the Lucent 802.11 cards swept the pool here) -So
its hard for me to say if it had 'merit' as a platform.

Faced with the non-Internet technologies on offer at the time, I am heartily
glad we're not running LAT, or DECnet or a host of other 'smart' nets.

Most of the network technologies I've used wind up lying: they mask the data timing
constraints which they depend on to work (LAT) or they over-state their ability
to provide global addressing (DECNET) or they over-engineer the wrong bits (OSI)

To argue against myself (for once) its notable that a lot of people are now putting
back into the Internet the kinds of things (kludges?) it left out, to try and
get session-layer, presentation layer behaviours.

-George

Re: [9fans] datakit

[geoff]

My experience with datakit was limited, but from what I understand
(and can remember) the advantages were:

• it provided some protection against snooping and meddling with data
in transit by running connections directly from the network controller
(kept in a locked closet) to the hosts and by not being a broadcast
medium, and by securing the connections between network controllers.

• partly as a consequence, when a call arrived and the datakit
controller said that it was from a particular destination, you could
believe it.

• when you placed a call, you handed a dial string like
`nj/astro/helix!uucp' to the datakit controller and it returned with
either an error message or a live connection.  You could also tell
that controller if you expected to need a lot of bandwidth (e.g.  for
file transfer) or not, and it would try to reserve bandwidth along the
route.  So no need for DHCP, ARP, RARP, BOOTP, sockets, sockaddrs,
inaddrs, etc., etc.

• IP, UDP and TCP were subsumed by a supposedly-simple protocol, URP.
I believe datakit was a reliable network, so you didn't need an extra
layer of protocol such as TCP.

• datakit was available in a variety of speeds and could be run great
distances, including cross-country.

I'm less familiar with the drawbacks, though I have heard some of the
moaning over the years, so I'll give a try:

• datakit hardware, particularly the controllers, was expensive.

• it was a centrally-managed network, arguably less open to ad-hocery
than ethernets built from switches and routers.  Some may see this as
a disadvantage, though it enabled many of the advantages named above.

• some of the host interfaces (e.g., Vax ones) were reputedly
painfully slower than the network itself.

My dislike for one-problem-one-protocol-one-RFC is largely one of
taste and conserving energy for more important things.  Do we really
need over 3,700 RFCs to make our networks work?  I don't have a strong
opinion about the end-to-end principle or various other IETF dogma.
We could certainly have done worse, OSI being the scariest of the
possible disasters (and it's not quite dead yet, it keeps popping up
in places like LDAP!).

Are you sure you don't have Datakit in your central offices or
long-distance equipment?

Re: [9fans] datakit

[George Michaelson]

On Thu, 19 Aug 2004 19:43:48 -0700 geoff@collyer.net wrote:

>My experience with datakit was limited, but from what I understand
>(and can remember) the advantages were:
>
>? it provided some protection against snooping and meddling with data
>in transit by running connections directly from the network controller
>(kept in a locked closet) to the hosts and by not being a broadcast
>medium, and by securing the connections between network controllers.

this is the kind of network outcome which fails deployment tests in a
post-deregulated world. those cupboards are now multi-keyed, with
competing vendors patching at random the mix of slow and highspeed network
services in the same KRONE. you get bupkes from 'locked cupboard' wiring and
head-end. Telstra deliver our 10meg metro-ethernet to a commodity switch
which is as dumb as the come, and could be sniffed for free once in the room.

(their optical demuxes at least have remote alarm on the doors, so you get some
 prot there)

>
>? partly as a consequence, when a call arrived and the datakit
>controller said that it was from a particular destination, you could
>believe it.

I could say the same about X.25 I guess. I am very glad we don't use that much
any more.

>
>? when you placed a call, you handed a dial string like
>`nj/astro/helix!uucp' to the datakit controller and it returned with
>either an error message or a live connection.

I think having namespace embedded into the model low down was a very good idea
and if things like the newcastle connection had been ported well to it, and
released into the wild early on, we might well all have been playing this game.

>  You could also tell
>that controller if you expected to need a lot of bandwidth (e.g.  for
>file transfer) or not, and it would try to reserve bandwidth along the
>route.  So no need for DHCP, ARP, RARP, BOOTP, sockets, sockaddrs,
>inaddrs, etc., etc.

again, end-to-end bw reservation has spectacularly failed in the marketplace
because greshams law winds up ruling. the military still do what they want, but
banks now regard IP nets with non-reserved b/w as 'adequate' and won't pay
for the national and inter-national costs which go with this model. if banks
won't then you've lost the marketplace of ideas (Cisco sell more to banks
and associated bodies than to unis and ISPs. what banks want, Cisco sell)

>
>? IP, UDP and TCP were subsumed by a supposedly-simple protocol, URP.
>I believe datakit was a reliable network, so you didn't need an extra
>layer of protocol such as TCP.

supposedly.. I bet it cost a huge amount of hidden overhead! the costs here
are things like RTT delay. you can't take the checksums out of the IP/above
layers because one sub-net claims to be reliable, so you wind up doing checksums
in all layers (this wouldn't have only been true for IP btw. links are not
homogenous, so absent AT&T owning everywhere, if you want to talk everywhere
you wind up having to do your own anyway)

>
>? datakit was available in a variety of speeds and could be run great
>distances, including cross-country.

In Australia, centrex models meant that things like this ran 2x or more
the apparent distance for rural subscribers. All telecoms north of
Gladstone in Queensland (Boyd will doubtless chime in with detail on how
sick this is) run via a concrete bunker next to the Wolloongabba cricket
ground here in Brisbane, and with Queensland 2000km long, that means to go
from the Cairns GPO to the Cairns bank incurs the same RTT as a
trans-pacific dialogue, when its 2km across town. Believe me, that is a
royal PITA. It winds up breaking eg NOVELL netware LAT etc, dump
protocols. Datakit may have masked it, but there would have been costs.

>
>I'm less familiar with the drawbacks, though I have heard some of the
>moaning over the years, so I'll give a try:
>
>? datakit hardware, particularly the controllers, was expensive.

FDDI died here for the same reason. IBM networks ran like greased steam-engines
and looked as lovely (to a sick trainspotter mind like mine) but were so
massively over-engineered you were paying for hand-lacing of the cables in
20ft racks of gear, when other vendors sold you wire-wrap botch-boxes for a 10th
of the cost.

>
>? it was a centrally-managed network, arguably less open to ad-hocery
>than ethernets built from switches and routers.  Some may see this as
>a disadvantage, though it enabled many of the advantages named above.

essentially, scaling failure in a non-monopolistic world. Personally I mourn
the retreat from a social contract which ended the public telco model, but there
is a paucity of fellow travellers around me. I'd have loved to see the core
national networks worldwide retained in a small set of public hands, and have
a million upper layers. I bet Datakit could have thrived in that world.

>
>? some of the host interfaces (e.g., Vax ones) were reputedly
>painfully slower than the network itself.

actually, still a problem for many people. Now Ethernet is a ubiquitous connect,
many high-speed hosts clock their clicks to do dumb buffering with their ether
cards instead of handing off, when a $100 unit from Frys can do on-card errorcheck
and the like, and work sanely with a kernel. Its very odd how many places
have high speed nets and dumb-as-shit connect to it!

>
>My dislike for one-problem-one-protocol-one-RFC is largely one of
>taste and conserving energy for more important things.  Do we really
>need over 3,700 RFCs to make our networks work?

No. the RFC model is bust. I know, I'm chairing a WG in IETF. its utterly broken.

>  I don't have a strong
>opinion about the end-to-end principle or various other IETF dogma.

mostly its kept alive by a dwindling number of people. Anybody who gives in to
NATs (eg SIP, other three-way rendesvous through NAT boxes) has ditched it long
ago.

I think it still has merit as a debating point and comparator, much as an OSI
model does for defining an ontology. As a concrete implementation guide, I think
it has some utility: I very much prefer not to be told by my provider what I
can do in packets, and dumb networks seem to be closer to a utility of packets
irrespective of what you do in them.

packet soup?

>We could certainly have done worse, OSI being the scariest of the
>possible disasters (and it's not quite dead yet, it keeps popping up
>in places like LDAP!).

ASN lives on. I blame many people, especially Marshall Rose (SNMP)

XML is no better, but is now the RPC of choice for many. XML uber alles has
probably replaced IP uber alles as the mantra.

>
>Are you sure you don't have Datakit in your central offices or
>long-distance equipment?

Yes. I'm very sure. Australia bought a mix of Nortel and Ericsson gear more
than AT&T gear. I dont think datakit was their provisioning model, Nokia b/w
management was much more like it.

Telstra and Optus got out of that space in the 90s. Edge customer nets
which people buy as 2mbit E1 (T1 to you I guess) which used to home in
those things, now are provided over an ADSL/HDSL bearer, switch into a
core optical network which is weird digital muxing, and the old 2meg stuff
is gathering dust. They made their ROI on the old core-copper stuff and
ditched it in mad rushes to ATM, and other dead meat technology.

I'm sure it lingers on, but I am also sure the places we're in now don't use it and
that it didn't make it down here.

-George

Re: [9fans] datakit

[geoff]

George commented:
> you can't take the checksums out of the IP/above layers because one
> sub-net claims to be reliable, so you wind up doing checksums in all
> layers (this wouldn't have only been true for IP btw.  links are not
> homogenous, so absent AT&T owning everywhere, if you want to talk
> everywhere you wind up having to do your own anyway)

Just to clarify, I'm unaware of any datakit/ip gateways (other than
multi-homed hosts, such as a Plan 9 cpu server (or two?) used to be).
So traffic within the world of datakit presumably could have lived
without upper-level checksums.  Looking at the second edition
port/stasync.c, it appears that CRCs were used in some cases.

To address the larger issue of trusting the network, that Ron raised,
I don't know how much intelligence was in a Datakit host interface,
but one could imagine end-to-end checksums or CRCs being handled by
the host interfaces (and probably between intermediate nodes too).
The better gigabit Ethernet cards offer to compute UDP and TCP
checksums incoming and outgoing for you, thus effectively pushing the
checksums and their verification very nearly into `the network' (your
network interface driver still has to check a bit that says `yup, the
checksums are okay'), at least within a single gigabit Ethernet.  Can
you trust these cards and your PCI buses to deliver your data
reliably?  Would you compute and verify the checksums in the IP stack
instead?  What about at 10Gb/s?

Re: [9fans] datakit

[ron minnich]

just for fun, I recently fixed a simple problem on my 256-node cluster by
turning the checksum offload OFF. Of course, the problem was some weird
driver wart that is as yet unfixed .... I guess offload is a great idea, I
just keep seeing all the failure modes :-)


ron

Re: [9fans] datakit

[boyd, rounin]

iirc, in '82, in sydney greg chesson was there.

the quote going 'round was:

    [datakit] not known to work reliably at any speed, but the code is huge

the security of who's at the other end etc were big pluses.

the sort of thing we need for a new tcp.

Re: [9fans] datakit

[Brantley Coile]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

This isn't the first time offloading things for tcp has been trie.  I
remember when TCP first made its appearance it was criticised for
using to many cpu cycles.  So, thought hardworking entrepreneurs, we
can put TCP into a separate processor and off load the work.  Summer
USENIX of '85 saw a lot of these TCP Offload Engines.  That didn't
last long.

Nice thing about being around for so long; you get to avoid stepping
in the same pile twice.

[-- Attachment #2: Type: message/rfc822, Size: 3061 bytes --]

From: ron minnich <rminnich@lanl.gov>
To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu>
Subject: Re: [9fans] datakit
Date: Fri, 20 Aug 2004 18:33:47 -0600 (MDT)
Message-ID: <Pine.LNX.4.44.0408201832440.22188-100000@maxroach.lanl.gov>

just for fun, I recently fixed a simple problem on my 256-node cluster by
turning the checksum offload OFF. Of course, the problem was some weird
driver wart that is as yet unfixed .... I guess offload is a great idea, I
just keep seeing all the failure modes :-)


ron

Re: [9fans] datakit

[Tim Newsham]

> Nice thing about being around for so long; you get to avoid stepping
> in the same pile twice.

slightly off topic, but --
This sounds like short sightedness.  Sometimes a bad idea 20 years ago
turns out to be a great idea today.  If your experience tells you to
avoid things just because you saw it come and go before perhaps you
should fight against your experience.

Tim N.

Re: [9fans] datakit

[ron minnich]

On Sat, 21 Aug 2004, Brantley Coile wrote:

> This isn't the first time offloading things for tcp has been trie.  I
> remember when TCP first made its appearance it was criticised for
> using to many cpu cycles.  So, thought hardworking entrepreneurs, we
> can put TCP into a separate processor and off load the work.  Summer
> USENIX of '85 saw a lot of these TCP Offload Engines.  That didn't
> last long.
>
> Nice thing about being around for so long; you get to avoid stepping
> in the same pile twice.


TOE was proposed (and hardware was built) for:
10 mbit,
100 mbit
1000 mbit,
10000 mbit

ethernet and other networks. Each time it seems to make sense -- for about
one iteration of the silicon design rules.

So I agree with you: it's not as convincing the 4th time around.

ron

Re: [9fans] datakit

[boyd, rounin]

> In Australia, centrex models meant that things like this ran 2x or more
> the apparent distance for rural subscribers. All telecoms north of
> Gladstone in Queensland (Boyd will doubtless chime in with detail on how
> sick this is) run via a concrete bunker next to the Wolloongabba cricket
> ground here in Brisbane, and with Queensland 2000km long, that means to go
> from the Cairns GPO to the Cairns bank incurs the same RTT as a
> trans-pacific dialogue, when its 2km across town.

totally possible.  oz is ~the size of the US but the population
density it nowhere near it.  rough figures?  20M people in
oz and 300M people in the US.  in oz this is made worse
'cos the 90% of the population lives on the coast.

[moving to wireless, as an example]

last i heard, GSM was going to be dropped in favour of
CDMA.  GSM is just not suitable in oz 'cos you need
this 'hexaboard' of BTS's that are within 35km radii.

35km is nothing in oz.  i know people who will drive
that far (or further) to get to work.

in the bush, there's just no point sticking up GSM BTS's
'cos there might not be one GSM mobile within its 35km
radius.

btw: the 35km figure (iirc) comes from the timing correction
constraints so that the 600us mirco-burst winds up at the
mobile and BTS at the right time; GSM is TDMA.

bbtw: i've tested it to 35km [normandie to jersey] and
at 300km/h [TGV] clocked with a GPS.

Re: [9fans] datakit

[C H Forsyth]

>>We could certainly have done worse, OSI being the scariest of the
>>possible disasters (and it's not quite dead yet, it keeps popping up
>>in places like LDAP!).

until recently i felt one hasn't actually needed
all that many RFCs implemented to get on with the world.

the complexity creature from the OSI swamp seems to have reappeared not
just in L(!)DAP but more cunningly in XML disguise in WS-* land.
like many successors WS begins to make
some of the worst cursed predecessors start to look good:
a vast and growing collection of incomplete complexity,
perhaps it's revenge for disdaining PL/1 and JCL.
Gods!  A hideous beast, baying is pursuing us!

more seriously, i observed the other day to someone that
the curious thing about the rise of complexity this time is
that as far as i can tell, there seems to be no significant
counter-culture to it, as there has been in times past.
``where you gonna go?  where you gonna run? ...''
i say that in the hopes that someone will say:
it's just building up momentum over here ...

Re: [9fans] datakit

[Wes Kussmaul]

> like many successors WS begins to make
> some of the worst cursed predecessors start to look good:
> a vast and growing collection of incomplete complexity,

Quote from VARBusiness cited in a recent book ;):

"Seek out security solutions that are complex
and require additional software and hardware."

Widget-driven economics and a military view of security encourage
complexity. Journalists often assume that the complexity is unintended. It's
not. It's a means to keep the customer down on the plantation and utterly
dependent.

> the curious thing about the rise of complexity this time is
> that as far as i can tell, there seems to be no significant
> counter-culture to it, as there has been in times past.

Professional obfuscators and FUD-complexifiers get better at their craft, as
we all do. They learn how to diminish the influence of their adversaries.

> ``where you gonna go?  where you gonna run? ...''

I know that ID-PKI appears to be a contributor to complexity. Done properly,
it is just the opposite. Please take another look.

> i say that in the hopes that someone will say:
> it's just building up momentum over here ...

It's building up momentum over here.

Re: [9fans] datakit

[Skip Tavakkolian]

> more seriously, i observed the other day to someone that
> the curious thing about the rise of complexity this time is
> that as far as i can tell, there seems to be no significant
> counter-culture to it, as there has been in times past.

I think the rise will be short-lived; it has to.  Nature doesn't like
waste, and wont allow it indefinitely.  Natural selection would
dictate that those companies that use complex software will spend more
time and resources managing the software rather than using it, and
will be less profitable; they will eventually lose out to the more
efficient organizations -- perhaps because they use less
complex/more efficient software.

The growth of hardware technology has obscured the issue of
efficiency.  It is cheaper to get the next generation of hardware,
than it is to spend the time to prototype and perfect software
up-front.  Just throw more hardware at it.  I doubt that can continue
over the next twenty years.

Re: [9fans] datakit

[rog]

> Nature doesn't like waste

actually, nature doesn't particularly care about waste.
a quote from somewhere i read recently

	nature is _effective_, not necessarily _efficient_.

it's only when the complexity of the software starts getting in the
way of the companies' effectiveness that the pressure will start to
come on.

personally i think it'll come when the maintenance cycle starts to
kick in seriously on all the new dross that's being written.  but
that's probably just wishful thinking.

Re: [9fans] datakit

[Jack Johnson]

On Fri, 20 Aug 2004 18:07:46 +0100, rog@vitanuova.com <rog@vitanuova.com> wrote:
> > Nature doesn't like waste
>
> actually, nature doesn't particularly care about waste.
> a quote from somewhere i read recently
>
>         nature is _effective_, not necessarily _efficient_.

Actually, to be slightly more accurate, it doesn't particularly care
about anything, being both indifferent and effective.

It does effectively maximize efficiency *if* doing so either helps the
DNA/host/community/species (pick your theorist) replicate better than
its peers, or if the waste in some way hinders its replication.  There
is actually nothing preventing a natural process from increasing
"waste" so long as it effectively promotes the replication of at least
one subset of the ecosphere.  There might be rebound effects or
population crashes, sure, but if it works, it works.

The flip side of waste is that life is adept at filling every
available ecological niche, so one organism's garbage is another's
treasure, be it fecal matter, banana peel or shed shell.

Early human middens made great gardens, for this very reason, and
possibly became the first efforts at farming -- throw all your garbage
here and just wait until harvest.  If corn is more fruitful in a
midden, and middens are more plentiful when humans are more fruitful,
and humans are more fruitful when corn is plentiful, then what
incentive is there for the humans to curb their waste?  (Effectively,
disease, either from a midden too large to sustain the balance of its
hosts or from a human or corn population too dense -- or homogenous --
to effectively fight the disease, but even the disease is a
population).

-Jack

The beauty of self-replicating systems is that their competition doesn't.

Re: [9fans] datakit

[boyd, rounin]

> I think the rise will be short-lived; it has to.  Nature doesn't like
> waste, and wont allow it indefinitely.  Natural selection would
> dictate that those companies that use complex software will spend more
> time and resources managing the software rather than using it, and
> will be less profitable;

well you would think that.  in france someone who designs and
writes code is payed less than a sysadmin who nurses the broken
junk.  now, if it was the other way around, as it should be, the
broken junk would not be written and the pay situation would
reverse.

on the one hand i fear, but on the other hand i hope, that it
will reach critical mass and collapse.

the skill of good design has been [almost] lost.

> I doubt that can continue over the next twenty years.

iirc, we are at photolythographic limits, but as Feynman said:

    there's plenty of room at the bottom

i have 2nd hand knowledge that the next generation of chips
will be built from tracks of atoms.  my source [reliable] was
doing research in this field.

it was done at York uni.  i think plessey were funding it.

Re: [9fans] datakit

[Boris Maryshev]

В сообщении от Пятница 20 Август 2004 21:41 boyd, rounin написал(a):
> i have 2nd hand knowledge that the next generation of chips
> will be built from tracks of atoms.  my source [reliable] was
> doing research in this field.
>
> it was done at York uni.  i think plessey were funding it.
Squeezing WWII technology deeper and deeper into the matter is no better than 
adding more complexity in software, isn't it? Quantitative leap - yes, 
qualitative - no.

What if next generation of processors will be built of a single atom?

Boris
-- 
You'd like to do it instantaneously, but that's too slow.

Re: [9fans] datakit

[Boris Maryshev]

В сообщении от Пятница 20 Август 2004 21:41 boyd, rounin написал(a):
> i have 2nd hand knowledge that the next generation of chips
> will be built from tracks of atoms.  my source [reliable] was
> doing research in this field.
>
> it was done at York uni.  i think plessey were funding it.
Squeezing WWII technology deeper and deeper into the matter is no better than 
adding more complexity in software, isn't it? Quantitative leap - yes, 
qualitative - no.

What if next generation of processors will be built of a single atom?

Boris
-- 
You'd like to do it instantaneously, but that's too slow.

Re: [9fans] datakit

[ron minnich]

I don't know, Geoff, having seen all the failed attempts at putting
'reliable transport' into the network itself (including ATM, HIPPI,
HIPPI-800, GSN, Quadrics, Myrinet, SCI, Infiniband, ...) I've become a big
fan of dumb networks like Ethernet. All that fancy stuff works great in
the small, fails in the large, and boy oh boy ... do you really want
someone to come to you 3 months from now and say "what's this huge block
of zeros in my data file?". I don't.

We had a network here (HIPPI-800)  that was super-reliable ... on 2
machines. With X thousand interfaces all going at once, you got a bad
packet once every 15 mins. Oops. Took three months to find out that was
happening. Software now covers for that problem.

Every new network does this:
- we're reliable! count on it! Just push the bits and we'll take care of
  it!
- what errors? We're not seeing them
  (oh, wait, we're not LOOKING for them, oops -- yes, this really
   happens!)
- well, ok, you're using the network wrong
- well, ok, it has bugs, but you're not seeing them -- it's your
  application
- oops, you're seeing bugs? we never simulated this scenario. Gosh, maybe
  there is a problem.
- there's a problem. Fixed in next hardware release
- there's a problem in the new hardware release
- (final phase) Our latest code release detects and corrects any
   errors in the network!

See: NFS, from '86 to '91 (everyone remember patching SunOS kernels to
turn on udpcksum?)
See: ATM, any time

If I assume the network is not 100% reliable, I will write software that
thinks that way, and I won't get bitten when my "reliable" network with a
1e-14 BER wrecks some data.

The number you need? The sandia guys like their ASCI Red network with its
1e-21 BER. What did datakit do? I know nothing I've ever used can do that
1e-21. The Red Storm network might, however.

ron

p.s. the Quadrics and Infiniband guys, who are all Very Smart People, will
beg to disagree with me about listing their networks above, but I will in
turn continue to disagree with them. But maybe the Infiniband guys are
right -- I'll believe it when I see it. So it goes. The Myrinet and
HIPPI-800 and SCI and ATM (and, actually, Ethernet) guys used to believe
they could solve all the problems in the network, but last time I looked
they don't believe that any more. Software continues to guarantee hardware
reliability. TCP r00lz.

Re: [9fans] datakit

[boyd, rounin]

> - what errors? We're not seeing them
>   (oh, wait, we're not LOOKING for them, oops -- yes, this really
>    happens!)

sure does.  took me a year to find a flakey switch to
switch cable.  this was ethernet and this cable was
special so you could stack them.

every few months one chunk of [thin] coax would go
bad to one office (the other 7 were fine).

once found, the standard protocol, when dealing with
broken h/w, was employed:

    if it's broken, break it

this avoids some other poor sap from re-using it.

satisfying too!

Re: [9fans] datakit

[andrey mirtchovski]

On Fri, 20 Aug 2004, boyd, rounin wrote:

> every few months one chunk of [thin] coax would go
> bad to one office (the other 7 were fine).
>
> once found, the standard protocol, when dealing with
> broken h/w, was employed:
>
>    if it's broken, break it
>
> this avoids some other poor sap from re-using it.

funny how some things rarely change -- today we had a bloke from a hardware
vendor do a software upgrade of one of our optical switches which was
misbehaving.  at some point after the update we saw that two of the machines
were transferring at a much lower speed than the others.

some mucking around later the guy took an optical cable out and asked

     'got a pair of scissors?'

with scissors supplied he proceeded cutting the cable in half:

     'keeps it from being accidentaly reused' he said. 'we've had that happen
     before'

or, as shirley bassey put it best:

"that it's all just a little bit of history repeating"


andrey

Re: [9fans] datakit

[ron minnich]

On Mon, 23 Aug 2004, andrey mirtchovski wrote:

> some mucking around later the guy took an optical cable out and asked
>
>      'got a pair of scissors?'
>
> with scissors supplied he proceeded cutting the cable in half:
>
>      'keeps it from being accidentaly reused' he said. 'we've had that happen
>      before'
>

cool. We have 2048 fibres under Pink. You don't remove a cable when it's
bad -- you cut the heads off each end. Works fine :-)

ron

Re: [9fans] datakit

[boyd, rounin]

> Instead we now have IP running over dumb networks that guarantee us
> very little and for which the solution to every problem seems to be
> another protocol/RFC.  At least it's a full-employment act for
> programmers (or would be in a working economy).

you said it.