From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: ** X-Spam-Status: No, score=2.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,RCVD_IN_ZEN_BLOCKED_OPENDNS,URIBL_ZEN_BLOCKED_OPENDNS autolearn=no autolearn_force=no version=3.4.4 Received: from minnie.tuhs.org (minnie.tuhs.org [IPv6:2600:3c01:e000:146::1]) by inbox.vuxu.org (Postfix) with ESMTP id 4E83129AE2 for ; Sun, 1 Jun 2025 00:48:03 +0200 (CEST) Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id B8815437E6; Sun, 1 Jun 2025 08:47:56 +1000 (AEST) Received: from mout.perfora.net (mout.perfora.net [74.208.4.197]) by minnie.tuhs.org (Postfix) with ESMTPS id 5EEB7437E5 for ; Sun, 1 Jun 2025 08:47:53 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=makerlisp.com; s=s1-ionos; t=1748731672; x=1749336472; i=luther.johnson@makerlisp.com; bh=qmaWJ5j1LdGac178Sw32WiGs3hiIz3NpV8eilZxBZHk=; h=X-UI-Sender-Class:Subject:To:References:From:Message-ID:Date: MIME-Version:In-Reply-To:Content-Type:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=rffSIXC//sXxmKurhgTPHWwPlPWmi8iWyEaAsD2NMZQHClxsUd0C/H82BjMOUoGY m+WTyN2HxwC7RWbP4VD5cSKXrJ30rDLY8T1pLHKa6ENvWviyUcfSuzjeXOHaMtcZs kEMr+at0CitTYKQMeagoVZeiHDlxBA8DSSrBDr5TokOUii3uGZNwVxztlaWFnSHVS ARq/tYbJWcSnMPy3vrp1IuAP+bTtdtS75flZrF4FxvYtyyvKolyK5AAnLOBuZvT1z fKAJBoI2XtvBWIGlkZO8LRIPObHsCzAver599j6Qss31mK0F2YacRnlKZC2tWSMW5 Eyp8xMhppkSfVBz6Kg== X-UI-Sender-Class: 55c96926-9e95-11ee-ae09-1f7a4046a0f6 Received: from mlhivps ([74.50.126.128]) by mrelay.perfora.net (mreueus002 [74.208.5.2]) with ESMTPSA (Nemesis) id 0LaVlh-1ujs3j0kf9-00pdRC for ; Sun, 01 Jun 2025 00:47:52 +0200 Received: from [192.168.234.128] (unknown [172.56.209.195]) by mlhivps (Postfix) with ESMTPSA id 3E0EA480169 for ; Sat, 31 May 2025 22:47:51 +0000 (UTC) To: tuhs@tuhs.org References: <3e4339e9-bf9a-2b72-b47a-f20f81a153b5@makerlisp.com> <202505312009.54VK97bQ4163488@freefriends.org> <0adb7694-f99f-dafa-c906-d5502647aaf0@makerlisp.com> From: Luther Johnson Message-ID: <5aa768ae-8853-f0da-9780-53ca4e9d486f@makerlisp.com> Date: Sat, 31 May 2025 15:47:49 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------EF5B902CEC347235408622F1" X-Provags-ID: V03:K1:8A6YZsaaUbzetVMcxhfS7KEylpeRg4OgvDj67lACA7WohjYvkV2 KgLEd7hIbX/ERpg0AgbH6UP0xBILlq1UDRpj9z80J7p4MFbF1L88mWaxskgoee5IgA9Sgs/ ikw8TPBvmEtjTm45AOitaGi2T3qmSjFLCc2kQcVXfwD6NVLIJNdCLILIArbNm47aSy1Ke6k CkwgDDCSPDT1+Mf+6lH0Q== UI-OutboundReport: notjunk:1;M01:P0:Vbbutw0YJZc=;WAnCk31LNvInWhAYDsi6AqM9Uel 5FNub9/KndIF3H53Q/58kzlPwLvoYYs6OGxvTQdta0gRmL6XMAyvIs/Dh0Q4wiNqdqvg9r15P 92IEU/qGzjvhaDaTdmgFEICCPlaIConakLi9Mjhc8ys0mXAzXvKgT3LYz5x3Ij04SurOuP4EO U6gfHu8jru/PzaYQAx/qX9L3hzL1FkUL7jYoihVnwMfON0+XR6NAh3QN6GupRAAIhYRJlSLEs hMgEApxYJOTB+R8VINhNbWUVsZYM+5Zgu4BvwE5TOZrvJ1Q5PXF+qCxU88ae8ZdqtaOIKahmi RT72kB+EJaBdCkLqNQUituPIs0fV64zGTgTuemfaXog1fGkcOYDoKEZlQUiQvdtqSB+9NTyV1 Buiwfx8zdbEwEnnrrL0J01QsUEsTIAEX53YO7wwaIeTrUA+ve2SsabsJERuyIkjpHGiyxho3p GqKwvCElv2mWplB5Tpo39SRHtKJVHPklhFdV+s0fUnnaMDd2mtnivP7BLF480KCGxKU9TlxDp kXP3mESaf4YWDOW/ssW4QrC2dm/PBmtjqFaB0YtR35/hG8JjsHRNd0scij3uUXP28pU8N/4zP xUDLbdgfrxgDhxiqoOkcobayJgQ7xRo5/1q7360bMMzL7nh/9tidg5NL4XXLDQc74tAXFaq15 3p4lcnUzZH98v6ct1m6VhqYSmwY5JHrvGLrlkyOBXL/Ep8IaZeWOcBRoT5FPwJL8TPHl8xleY K6uVvP40ZE5VWoP8CLCKdhK2BoCtDpCfOkOxmpyUQQeebFZKr8fEKUY/bID1nnH1CrF+2Q/V+ BZinnhejN9fLd7g43Qypj8MiBTqlKrOpXnnAeqiWcpA1fh3JIXS1SC9hj3pl2QmHEjCdp+J8t jMKN+EiSCMYvNTugZKaSPM4KLF6Rqwsq8tDTnQGRto6h3xaUy5xaRw2KtbOT7oBg5KBMdTl3s MzMpIjbHSQXh4g05/Ugt9UOos93X2XzsjecRav1Cfq1vft/vTK2P5il//De+t7Fr85c3N6DlP RGD7p+zYXceWGNMYgTNYYdCbD4c8isqJzSKZ92EAAQzBB6ySbS3DFwzbdfac5rkj1H/PgR41D vQSY7Fm2mwUjNnzKyeY9A/kAs0zoalbnfcmiJ6+Bbm6BDt3voLAk9vKiGHgWtd3kx0+PAyQOt yC04bHq4ZJccd2CdFuXrpmu/Q3G5IwoPSmFjJjh+ot37LbxkWXqkC85oXqJzjfIGddzVGMFmv F6KEu4jzSqfiGGoowOahy5og7BfNHkJU4LpWkhd2Rt/sg1hjPzE4+8YB1FHPK9Xj3NAntE8r/ HFcEXly+mcz599I4Pf3jC98q3m0sFup4sYfVGwYANo2PQ7QPrT1NmMz//DfEGhngAZTBb2u1Y 85ivFxoHlhK269ZHyBwZqfCsiM9hpL/qnODH/8VP9yciS2FUzRvcirRVDQbBKwXK38VYtbej0 XUxyVwQ== Message-ID-Hash: KI6YGBK5VF4YLESV4NZASATSBDDXY7LD X-Message-ID-Hash: KI6YGBK5VF4YLESV4NZASATSBDDXY7LD X-MailFrom: luther.johnson@makerlisp.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tuhs.tuhs.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: On the unreliability of LLM-based search results (was: Listing of early Unix source code from the Computer History Museum) List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This is a multi-part message in MIME format. --------------EF5B902CEC347235408622F1 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable I think we could call many of these responses "mis-ambiguation", or=20 conflation, they mush everything together as long as the questions posed= =20 and the answers they provide are "buzzword-adjacent", in a very=20 superficial, mechanical way. There's no intelligence here, it's just=20 amazing how much we project onto these bots because we want to believe=20 in them. On 05/31/2025 03:36 PM, James Johnston wrote: > Well, I have to say that my experiences with "AI based search" have=20 > been beyond grossly annoying. It keeps trying to "help me" by sliding=20 > in common terms it actually knows about instead of READING THE DAMN QUER= Y. > > I had much, much better experiences with very literal search methods,=20 > and I'd like to go back to that when I'm looking for obscure papers,=20 > names, etc. Telling me "you mean" when I damn well DID NOT MEAN THAT=20 > is a worst-case experiences. > > Sorry, not so much a V11 experience here, but I have to say it may=20 > serve the public, but only to guide them back into boring,=20 > middle-of-the-road, 'average mean-calculating' responses that simply=20 > neither enlighten nor serve the original purpose of search. > > jj - a grumpy old signal processing/hearing guy who used a lot of real= =20 > operating systems back when and kind of misses them. > > On Sat, May 31, 2025 at 2:53=E2=80=AFPM Luther Johnson=20 > >=20 > wrote: > > I agree. > > On 05/31/2025 01:09 PM, arnold@skeeve.com > wrote: > > It's been going on a for a long time, even before AI. The amount > > of cargo cult programming I've seen over the past ~ 10 years > > is extremely discouraging. Look up something on Stack Overflow > > and copy/paste it without understanding it. How much better is > > that than relying on AI? Not much in my opinion. (Boy, am I glad > > I retired recently.) > > > > Arnold > > > > Luther Johnson > wrote: > > > >> I think when no-one notices anymore, how wrong automatic > information is, > >> and how often, it will have effectively redefined reality, and > humans, > >> who have lost the ability to reason for themselves, will > declare that AI > >> has met and exceeded human intelligence. They will be right, part= ly > >> because of AI's improvements, but to a larger extent, because > we will > >> have forgotten how to think. I think AI is having disastrous > effects on > >> the education of younger generations right now, I see it in my > >> workplace, every day. > >> > >> On 05/31/2025 12:31 PM, andrew@humeweb.com > wrote: > >>> generally, i rate norman=E2=80=99s missives very high on the > believability scale. > >>> but in this case, i think he is wrong. > >>> > >>> if you take as a baseline, the abilities of LLMs (such as > earlier versions of ChatGP?) 2-3 years ago > >>> was quite suspect. certainly better than mark shaney, but not > overwhelmingly. > >>> > >>> those days are long past. modern systems are amazingly adept. > not necessarily intelligent, > >>> but they can (but not always) pass realistic tests, pass SAT > tests and bar exams, math olympiad tests > >>> and so on. and people can use them to do basic (but realistic) > data analysis including experimental design, > >>> generate working code, and run that code against synthetic > data and produce visual output. > >>> > >>> sure, there are often mistakes. the issue of hullucinations is > real. but where we are now > >>> is almost astonishing, and will likely get MUCH better in the > next year or three. > >>> > >>> end-of-admonishment > >>> > >>> andrew > >>> > >>>> On May 26, 2025, at 9:40=E2=80=AFAM, Norman Wilson > wrote: > >>>> > >>>> G. Branden Robinson: > >>>> > >>>> That's why I think Norman has sussed it out accurately.=20 > LLMs are > >>>> fantastic bullshit generators in the Harry G. Frankfurt > sense,[1] > >>>> wherein utterances are undertaken neither to enlighten nor > to deceive, > >>>> but to construct a simulacrum of plausible discourse.=20 > BSing is a close > >>>> cousin to filibustering, where even plausibility is > discarded, often for > >>>> the sake of running out a clock or impeding achievement of > consensus. > >>>> > >>>> =3D=3D=3D=3D > >>>> > >>>> That's exactly what I had in mind. > >>>> > >>>> I think I had read Frankfurt's book before I first started > >>>> calling LLMs bullshit generators, but I can't remember for > >>>> sure. I don't plan to ask ChatGPT (which still, at least > >>>> sometimes, credits me with far greater contributions to Unix > >>>> than I have actually made). > >>>> > >>>> > >>>> Here's an interesting paper I stumbled across last week > >>>> which presents the case better than I could: > >>>> > >>>> https://link.springer.com/article/10.1007/s10676-024-09775-5 > >>>> > >>>> To link this back to actual Unix history (or something much > >>>> nearer that), I realized that `bullshit generator' was a > >>>> reasonable summary of what LLMs do after also realizing that > >>>> an LLM is pretty much just a much-fancier and better-automated > >>>> descendant of Mark V Shaney: > https://en.wikipedia.org/wiki/Mark_V._Shaney > >>>> > >>>> Norman Wilson > >>>> Toronto ON > > > > --=20 > James D. (jj) Johnston > > Former Chief Scientist, Immersion Networks --------------EF5B902CEC347235408622F1 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

I think we could call many of these responses "mis-ambiguation", or conflation, they mush everything together as long as the questions posed and the answers they provide are "buzzword-adjacent", in a very superficial, mechanical way. There's no intelligence here, it's just amazing how much we project onto these bots because we want to believe in them.

On 05/31/2025 03:36 PM, James Johnston wrote:
Well, I have to say that my experiences with "AI based search" have been beyond grossly annoying. It keeps trying to "help me" by sliding in common terms it actually knows about instead of READING THE DAMN QUERY.

I had much, much better experiences with very literal search methods, and I'd like to go back to that when I'm looking for obscure papers, names, etc.=C2=A0 Telling me "you me= an" when I damn well DID NOT MEAN THAT is a worst-case experiences.

Sorry, not so much a V11 experience here, but I have to say it may serve the public, but only to guide them back into boring, middle-of-the-road, 'average mean-calculating' responses that simply neither enlighten nor serve the original purpose of search.

jj - a grumpy old signal processing/hearing guy who used a lot of real operating systems back when and kind of misses them.

On Sat, May 31, 2025 at 2:53=E2=80=AFPM Luther Johnson <luther.johnson@ma= kerlisp.com> wrote:
I agree.

On 05/31/2025 01:09 PM, arnold@ske= eve.com wrote:
> It's been going on a for a long time, even before AI. The amount
> of cargo cult programming I've seen over the past ~ 10 years
> is extremely discouraging.=C2=A0 Look up something on Stack Overflow
> and copy/paste it without understanding it.=C2=A0 How much better is
> that than relying on AI?=C2=A0 Not much in my opinion.=C2= =A0 (Boy, am I glad
> I retired recently.)
>
> Arnold
>
> Luther Johnson <luther.johnson@makerlisp.com> wrote:
>
>> I think when no-one notices anymore, how wrong automatic information is,
>> and how often, it will have effectively redefined reality, and humans,
>> who have lost the ability to reason for themselves, will declare that AI
>> has met and exceeded human intelligence. They will be right, partly
>> because of AI's improvements, but to a larger extent, because we will
>> have forgotten how to think. I think AI is having disastrous effects on
>> the education of younger generations right now, I see it in my
>> workplace, every day.
>>
>> On 05/31/2025 12:31 PM, andrew@hu= meweb.com wrote:
>>> generally, i rate norman=E2=80=99s missives very hi= gh on the believability scale.
>>> but in this case, i think he is wrong.
>>>
>>> if you take as a baseline, the abilities of LLMs (such as earlier versions of ChatGP?) 2-3 years ago
>>> was quite suspect. certainly better than mark shaney, but not overwhelmingly.
>>>
>>> those days are long past. modern systems are amazingly adept. not necessarily intelligent,
>>> but they can (but not always) pass realistic tests, pass SAT tests and bar exams, math olympiad tests
>>> and so on. and people can use them to do basic (but realistic) data analysis including experimental design,
>>> generate working code, and run that code against synthetic data and produce visual output.
>>>
>>> sure, there are often mistakes. the issue of hullucinations is real. but where we are now
>>> is almost astonishing, and will likely get MUCH better in the next year or three.
>>>
>>> end-of-admonishment
>>>
>>>=C2=A0 =C2=A0 =C2=A0andrew
>>>
>>>> On May 26, 2025, at 9:40=E2=80=AFAM, Norman Wil= son <norman@oclsc.org> wrote:
>>>>
>>>> G. Branden Robinson:
>>>>
>>>>=C2=A0 =C2=A0 That's why I think Norman has suss= ed it out accurately.=C2=A0 LLMs are
>>>>=C2=A0 =C2=A0 fantastic bullshit generators in t= he Harry G. Frankfurt sense,[1]
>>>>=C2=A0 =C2=A0 wherein utterances are undertaken = neither to enlighten nor to deceive,
>>>>=C2=A0 =C2=A0 but to construct a simulacrum of p= lausible discourse.=C2=A0 BSing is a close
>>>>=C2=A0 =C2=A0 cousin to filibustering, where eve= n plausibility is discarded, often for
>>>>=C2=A0 =C2=A0 the sake of running out a clock or impeding achievement of consensus.
>>>>
>>>> =3D=3D=3D=3D
>>>>
>>>> That's exactly what I had in mind.
>>>>
>>>> I think I had read Frankfurt's book before I first started
>>>> calling LLMs bullshit generators, but I can't remember for
>>>> sure.=C2=A0 I don't plan to ask ChatGPT (which still, at least
>>>> sometimes, credits me with far greater contributions to Unix
>>>> than I have actually made).
>>>>
>>>>
>>>> Here's an interesting paper I stumbled across last week
>>>> which presents the case better than I could: >>>>
>>>> https://link.springer.com= /article/10.1007/s10676-024-09775-5
>>>>
>>>> To link this back to actual Unix history (or something much
>>>> nearer that), I realized that `bullshit generator' was a
>>>> reasonable summary of what LLMs do after also realizing that
>>>> an LLM is pretty much just a much-fancier and better-automated
>>>> descendant of Mark V Shaney: https://en.wikipedia.org/= wiki/Mark_V._Shaney
>>>>
>>>> Norman Wilson
>>>> Toronto ON



--
James D. (jj) Johnston

Former Chief Scientist, Immersion Networks

--------------EF5B902CEC347235408622F1--