Re: [TUHS] Death by bug

[Jon Steinhart <jon@fourwinds.com>]

Ralph Corderoy writes:
> Hi Jon,
>
> > So while I agree with your example, I guess that I was contemplating
> > the sort of example that makes the public take notice.  Arguments
> > about statistical models are lost on 99% of the population.
>
> True.
>
> I don't know of any but if I had to pick an area where they might be
> occurring it would be medical devices.  I've been an external reviewer
> of a company's embedded C code for a medical device in the past,
> bare-metal, simple pre-emptive round-robin task-switcher, etc., though
> my suggestion is nothing to do with their quality of work or products,
> just that the level of detail they went to shows how slacker companies
> could err.
>
> The Therac-25 gave fatal radiation doses in a few cases.  These were
> obvious when they occurred.  https://en.wikipedia.org/wiki/Therac-25
>
> What if a medical device which sits in the background quietly doing its
> continuous, but sometimes vital, thing had the occasional hiccough from
> which it recovered?  If the blip is only fatal in patients which were
> already touch and go and device fault left no obvious sign to suggest
> death wasn't caused by the initial ailment then the cause of death would
> be, quite reasonably, assumed.  
>
> Given some devices are present in large numbers for many years in
> hospitals, and there's a lot of hospitals, an unnoticed bug could be
> steadily chipping away at its human overlords.
>
> -- 
> Cheers, Ralph.

Well, we agree once again.  I have done a lot of work on medical devices,
and was surprised at the lack of diligence on the part of some of the
people with whom I worked.  But your example is similar to the way that
people have trouble with COVID - there's too much of a lag between
exposure and symptoms for many folks to make the connection.  So I'm kind
of wondering what sort of bug may blow up a refinery or chemical plant and
cause enough loss of life and damage to make people take notice.

Jon