[TUHS] Re: On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[TUHS] Re: On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[Noel Hunt]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]

On Tue, 27 May 2025 at 02:40, Norman Wilson <norman@oclsc.org> wrote:

> an LLM is pretty much just a much-fancier and better-automated
> descendant of Mark V Shaney: https://en.wikipedia.org/wiki/Mark_V._Shaney


I am glad someone has finally pointed that out.

[-- Attachment #2: Type: text/html, Size: 831 bytes --]

[TUHS] Mark V Shaney (Re: Re: On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum))

[Dan Cross]

On Tue, May 27, 2025 at 2:44 AM Rob Pike <robpike@gmail.com> wrote:
> My name is Rob Pike and I approve this message.

Regarding Mark V. Shaney, I once heard a story that someone was so
upset about some of its posts on USENET that they drove to Bell Labs
to confront it, not realizing "he" was in fact a program, and not a
person. On arriving at the labs, someone explained what was what, they
went home, and that was the end of it.

But, in the version I heard, they drove from, like, Ohio...or
somewhere in the midwest.  I find this very difficult to believe:
surely by the time they got to Harrisburg, perhaps, they'd have cooled
down enough to realize they were making poor life choices and should
turn around and go back home.

Is there any truth to this, or is the story (as I suspect) just apocryphal?

        - Dan C.

[TUHS] Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[Charles H Sauer (he/him)]

TUHS->COFF
>     > It's like Wikipedia.
> 
>     No, Wikipedia has (at least historically) human editors who
>     supposedly have some knowledge of reality and history.
> 
>     An LLM response is going to be a series of tokens predicted based on
>     probabilities from its training data. ...
> 
>     Assuming the sources it cites are real works, it seems fine as a
>     search engine, but the text that it outputs should absolutely not be
>     thought of as something arrived at by similar means as text produced
>     by supposedly knowledgeable and well-intentioned humans.
> 
> An LLM can weigh sources, but it has to be taught to do that.  A human 
> can weigh sources, but it has to be taught to do that.

Before LLMs, Wikipedia, World Wide Web, ... adages such as "Trust, but 
verify," and "Inspect what you expect," were appropriate, and still are.

Dabbling in editing and creating Wikipedia articles has enforced those 
notions. A few anecdotes here -- I could cite others.

1. I think my first experience was trying in 2008 to fix what is now at 
https://en.wikipedia.org/wiki/Vulcan_Gas_Company_(1967%E2%80%931970), 
because the article had so much erroneous content, and because I had 
worked/performed at that venue 1969-70. Much of what I did in 2008 was 
accepted without anyone else verifying. But others broke things/changed 
things, even renamed the original article and replaced it with an 
article about a newer club that adopted the name. A few years ago, I 
tried to make corrections, citing poster images at 
https://concerts.fandom.com/wiki/Vulcan_Gas_Company. Those changes were 
vetoed because fandom.com was considered unreliable. I copied the images 
from fandom to https://technologists.com/VGC/, and then citing those 
images was then accepted by the editors involved. (The article has been 
changed dramatically, still is seriously deficient, IMO, but I'm not 
interested in fixing.)

2. Last year, I created https://en.wikipedia.org/wiki/Hub_City_Movers, 
citing sources I considered reliable. Citations to images at discogs.com 
were vetoed as unreliable, based on analogous bias against that site. 
Partly to see what was possible, I engaged with editors, found citations 
they found acceptable, and ultimately produced a better article.

3. Later last year, I edited https://en.wikipedia.org/wiki/IBM_AIX to 
fix obviously erroneous discussion of AIX 1/2/3. Even though I used my 
own writings as references, the changes were accepted.

I still use the Web, Wikipedia, and even LLMs, but cautiously.

Charlie
-- 
voice: +1.512.784.7526       e-mail: sauer@technologists.com
fax: +1.512.346.5240         Web: https://technologists.com/sauer/
Facebook/Google/LinkedIn/mas.to: CharlesHSauer

[TUHS] Re: On the unreliability of LLM-based search results

[George Michaelson]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

>
>
> We're way off topic. Warren should send a kill.

That said: please don't repeat the "hallucinate" label. It's self
-aggrandisement. Its deliberate, to foster belief "it's like thinking"

It's not a hallucination, it's bad model data and bad constraint
programming. They're not thinking, or dreaming, or demanding not to be
turned off, or threatening or bullying: They're not Markov chains either
but they're a damn sight closer to a machine than a mind.


G

[-- Attachment #2: Type: text/html, Size: 824 bytes --]

[TUHS] Re: On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[Luther Johnson]

[-- Attachment #1: Type: text/plain, Size: 5910 bytes --]

I think we could call many of these responses "mis-ambiguation", or 
conflation, they mush everything together as long as the questions posed 
and the answers they provide are "buzzword-adjacent", in a very 
superficial, mechanical way. There's no intelligence here, it's just 
amazing how much we project onto these bots because we want to believe 
in them.

On 05/31/2025 03:36 PM, James Johnston wrote:
> Well, I have to say that my experiences with "AI based search" have 
> been beyond grossly annoying. It keeps trying to "help me" by sliding 
> in common terms it actually knows about instead of READING THE DAMN QUERY.
>
> I had much, much better experiences with very literal search methods, 
> and I'd like to go back to that when I'm looking for obscure papers, 
> names, etc.  Telling me "you mean" when I damn well DID NOT MEAN THAT 
> is a worst-case experiences.
>
> Sorry, not so much a V11 experience here, but I have to say it may 
> serve the public, but only to guide them back into boring, 
> middle-of-the-road, 'average mean-calculating' responses that simply 
> neither enlighten nor serve the original purpose of search.
>
> jj - a grumpy old signal processing/hearing guy who used a lot of real 
> operating systems back when and kind of misses them.
>
> On Sat, May 31, 2025 at 2:53 PM Luther Johnson 
> <luther.johnson@makerlisp.com <mailto:luther.johnson@makerlisp.com>> 
> wrote:
>
>     I agree.
>
>     On 05/31/2025 01:09 PM, arnold@skeeve.com
>     <mailto:arnold@skeeve.com> wrote:
>     > It's been going on a for a long time, even before AI. The amount
>     > of cargo cult programming I've seen over the past ~ 10 years
>     > is extremely discouraging.  Look up something on Stack Overflow
>     > and copy/paste it without understanding it.  How much better is
>     > that than relying on AI?  Not much in my opinion.  (Boy, am I glad
>     > I retired recently.)
>     >
>     > Arnold
>     >
>     > Luther Johnson <luther.johnson@makerlisp.com
>     <mailto:luther.johnson@makerlisp.com>> wrote:
>     >
>     >> I think when no-one notices anymore, how wrong automatic
>     information is,
>     >> and how often, it will have effectively redefined reality, and
>     humans,
>     >> who have lost the ability to reason for themselves, will
>     declare that AI
>     >> has met and exceeded human intelligence. They will be right, partly
>     >> because of AI's improvements, but to a larger extent, because
>     we will
>     >> have forgotten how to think. I think AI is having disastrous
>     effects on
>     >> the education of younger generations right now, I see it in my
>     >> workplace, every day.
>     >>
>     >> On 05/31/2025 12:31 PM, andrew@humeweb.com
>     <mailto:andrew@humeweb.com> wrote:
>     >>> generally, i rate norman’s missives very high on the
>     believability scale.
>     >>> but in this case, i think he is wrong.
>     >>>
>     >>> if you take as a baseline, the abilities of LLMs (such as
>     earlier versions of ChatGP?) 2-3 years ago
>     >>> was quite suspect. certainly better than mark shaney, but not
>     overwhelmingly.
>     >>>
>     >>> those days are long past. modern systems are amazingly adept.
>     not necessarily intelligent,
>     >>> but they can (but not always) pass realistic tests, pass SAT
>     tests and bar exams, math olympiad tests
>     >>> and so on. and people can use them to do basic (but realistic)
>     data analysis including experimental design,
>     >>> generate working code, and run that code against synthetic
>     data and produce visual output.
>     >>>
>     >>> sure, there are often mistakes. the issue of hullucinations is
>     real. but where we are now
>     >>> is almost astonishing, and will likely get MUCH better in the
>     next year or three.
>     >>>
>     >>> end-of-admonishment
>     >>>
>     >>>     andrew
>     >>>
>     >>>> On May 26, 2025, at 9:40 AM, Norman Wilson <norman@oclsc.org
>     <mailto:norman@oclsc.org>> wrote:
>     >>>>
>     >>>> G. Branden Robinson:
>     >>>>
>     >>>>    That's why I think Norman has sussed it out accurately. 
>     LLMs are
>     >>>>    fantastic bullshit generators in the Harry G. Frankfurt
>     sense,[1]
>     >>>>    wherein utterances are undertaken neither to enlighten nor
>     to deceive,
>     >>>>    but to construct a simulacrum of plausible discourse. 
>     BSing is a close
>     >>>>    cousin to filibustering, where even plausibility is
>     discarded, often for
>     >>>>    the sake of running out a clock or impeding achievement of
>     consensus.
>     >>>>
>     >>>> ====
>     >>>>
>     >>>> That's exactly what I had in mind.
>     >>>>
>     >>>> I think I had read Frankfurt's book before I first started
>     >>>> calling LLMs bullshit generators, but I can't remember for
>     >>>> sure.  I don't plan to ask ChatGPT (which still, at least
>     >>>> sometimes, credits me with far greater contributions to Unix
>     >>>> than I have actually made).
>     >>>>
>     >>>>
>     >>>> Here's an interesting paper I stumbled across last week
>     >>>> which presents the case better than I could:
>     >>>>
>     >>>> https://link.springer.com/article/10.1007/s10676-024-09775-5
>     >>>>
>     >>>> To link this back to actual Unix history (or something much
>     >>>> nearer that), I realized that `bullshit generator' was a
>     >>>> reasonable summary of what LLMs do after also realizing that
>     >>>> an LLM is pretty much just a much-fancier and better-automated
>     >>>> descendant of Mark V Shaney:
>     https://en.wikipedia.org/wiki/Mark_V._Shaney
>     >>>>
>     >>>> Norman Wilson
>     >>>> Toronto ON
>
>
>
> -- 
> James D. (jj) Johnston
>
> Former Chief Scientist, Immersion Networks


[-- Attachment #2: Type: text/html, Size: 9462 bytes --]