[COFF] Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[COFF] Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum)

[Charles H Sauer (he/him)]

TUHS->COFF
>     > It's like Wikipedia.
> 
>     No, Wikipedia has (at least historically) human editors who
>     supposedly have some knowledge of reality and history.
> 
>     An LLM response is going to be a series of tokens predicted based on
>     probabilities from its training data. ...
> 
>     Assuming the sources it cites are real works, it seems fine as a
>     search engine, but the text that it outputs should absolutely not be
>     thought of as something arrived at by similar means as text produced
>     by supposedly knowledgeable and well-intentioned humans.
> 
> An LLM can weigh sources, but it has to be taught to do that.  A human 
> can weigh sources, but it has to be taught to do that.

Before LLMs, Wikipedia, World Wide Web, ... adages such as "Trust, but 
verify," and "Inspect what you expect," were appropriate, and still are.

Dabbling in editing and creating Wikipedia articles has enforced those 
notions. A few anecdotes here -- I could cite others.

1. I think my first experience was trying in 2008 to fix what is now at 
https://en.wikipedia.org/wiki/Vulcan_Gas_Company_(1967%E2%80%931970), 
because the article had so much erroneous content, and because I had 
worked/performed at that venue 1969-70. Much of what I did in 2008 was 
accepted without anyone else verifying. But others broke things/changed 
things, even renamed the original article and replaced it with an 
article about a newer club that adopted the name. A few years ago, I 
tried to make corrections, citing poster images at 
https://concerts.fandom.com/wiki/Vulcan_Gas_Company. Those changes were 
vetoed because fandom.com was considered unreliable. I copied the images 
from fandom to https://technologists.com/VGC/, and then citing those 
images was then accepted by the editors involved. (The article has been 
changed dramatically, still is seriously deficient, IMO, but I'm not 
interested in fixing.)

2. Last year, I created https://en.wikipedia.org/wiki/Hub_City_Movers, 
citing sources I considered reliable. Citations to images at discogs.com 
were vetoed as unreliable, based on analogous bias against that site. 
Partly to see what was possible, I engaged with editors, found citations 
they found acceptable, and ultimately produced a better article.

3. Later last year, I edited https://en.wikipedia.org/wiki/IBM_AIX to 
fix obviously erroneous discussion of AIX 1/2/3. Even though I used my 
own writings as references, the changes were accepted.

I still use the Web, Wikipedia, and even LLMs, but cautiously.

Charlie
-- 
voice: +1.512.784.7526       e-mail: sauer@technologists.com
fax: +1.512.346.5240         Web: https://technologists.com/sauer/
Facebook/Google/LinkedIn/mas.to: CharlesHSauer

[COFF] Re: [TUHS] Wikipedia anecdotes - LLM generalizations [was On the unreliability of LLM-based search results

[Alexander Schreiber]

On Sun, Jun 01, 2025 at 12:00:11AM +0200, Steffen Nurpmeso wrote:
> 
> There are surely useful tasks for AI, when it is driven with green
> energy, and after is has been fully understood.

What is currently being sold as "AI" is mostly LLM (Large Language
Models), which are - to grossly simplify things - massive brute-force
pattern matching engines.

There are plenty of use cases where a well setup pattern matching engine
is exactly what you need. My favourite example: SBB (Swiss Railways)
uses "AI" (an in-house trained pattern matching model) to sift through
the massive incoming stream of noise recordings from rail mounted
vibration sensors, to identify (by matching known qualified patterns)
those caused by damaged train carriage wheels. Additional support
infrastructure then identifies train, carriage, wheel and notifies
the owner/operator to fix the wheel - before gets worse and does
more damage to the rails.

There are lots of similar tasks where pattern matching engines are
a great fit (e.g. optical QC on finished surfaces during manufacturing).

If you try to use pattern matching engines for tasks that require
knowledge, thinking, understanding (in short: a trained human mind),
then you will be sorely disappointed while drowining in - potentially
even superficially plausible sounding - bullshit (See Harry Frankfurt,
"On Bullshit").

> Before that it is just another race that is raced at whatever cost
> there may be.  The price is payed by the environment, and the

The current forecasts of both the use and the costs (some claim that
we need to feed 90% of all power production to data centers eventually,
which is clearly .. ill advised) of "AI" are looking wildly exaggerated.

> grand children, but no longer further down the line.  That is
> possibly the good thing about it.  Exactly as said by the Club of
> Rome in 1972, and at least ever since also by the Catholic Church.

Club of Rome did not see a lot of scientific developments coming that
enabled growth way beyond their expectations. But then, predicting
the future is never a sure business.

That said, cheerfully burning down the planet for short-term profit
is not exactly a sustainable business model, to put it politely.

Kind regards,
           Alex.
-- 
"Opportunity is missed by most people because it is dressed in overalls and
 looks like work."                                      -- Thomas A. Edison