From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, PDS_OTHER_BAD_TLD,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 Received: (qmail 23419 invoked from network); 4 Feb 2022 23:19:51 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 4 Feb 2022 23:19:51 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 5FA899BFC9; Sat, 5 Feb 2022 09:19:50 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 16DFC9B92F; Sat, 5 Feb 2022 09:18:52 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jXB16iVO"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 048AE9B7AF; Sat, 5 Feb 2022 09:18:49 +1000 (AEST) Received: from mail-oi1-f175.google.com (mail-oi1-f175.google.com [209.85.167.175]) by minnie.tuhs.org (Postfix) with ESMTPS id BAA9B95192 for ; Sat, 5 Feb 2022 09:18:46 +1000 (AEST) Received: by mail-oi1-f175.google.com with SMTP id i5so10345895oih.1 for ; Fri, 04 Feb 2022 15:18:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ZlHyhpk+5xzIgC1UEgSfuLHLyFhqdzUOq5NEOYzxfJY=; b=jXB16iVOfgCe3u8IBL+hgsktFbGHKpylwSINK6AbHXsRVliHjYipTuwMkW6AkK8BiV qVdFzo/sIN/JvXqiXggZ7piCUbca+8e7HbjJf84PMgp1M0AP0Nz9KmdeQj5dwYX2nIGZ TmlM8ctoSao/aKj3S5JLgjwoLU7x77WCVTT/Vd3/AaT/w5zNOHavKmAiL5m19kSnLXOt kH9yCXTxamTQKcoP6nK7XDWzK27LKLAjotyZIk9MHqjsqaeq02MSqr7ha3KnMiK6UCkK BXkn2gsSYnuGqkNh5BAdLLu6X8GmtWf+lRHlF4OF7FxtwLjIw11jnn/9quZtTqaqaXgw SM5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZlHyhpk+5xzIgC1UEgSfuLHLyFhqdzUOq5NEOYzxfJY=; b=IOMLbPbTCmsqLGTVM+7p2MedQPh5hGQEVjyvBbdqXfxGTqs6JFGldJCgCL1WFgHMeT 3wVYdKbOREU9u0xUPgHTsq1cH2A+n3zLhb3SnjvZSPomylZQ3XkvXeOxoEja5SE48Vg6 NJvjfT6hDs/zpFyf1qxUKOxDxfHdkNUbQiiFzVVZJ9VLICXTQipSfCc6eU9R6cCyOncO LC7kuSSSh+MrXXwrlsAG1ck2m0xC/yQnvFlIt+WaNL0OZ0J7w4CwvfQ3NpIhqKQ3vZjk gdhcl9GTePcM0/CEjBR8k2LuLEV0zSu+rJU0yyQSokvswKRE0+QAA1gOIbrm1yTp1KBH dExQ== X-Gm-Message-State: AOAM533NG9T1hfnqwNPH212uScSUeCyFVK6BqtNy5hkBllyiwRir2XK2 J2JpoqJphgaa2P1dZ2BG8qiMi//QEhYu2hTlzH6p5wiXHQk= X-Google-Smtp-Source: ABdhPJysXzaPanOSMgEv3hDOrwSyqPRn4OSx4mfGZdYHPAzkPJgJaNwzQdQ6ZskKZKAhwFmkJJzrkeSWCBR3fAbo5zM= X-Received: by 2002:a05:6808:230d:: with SMTP id bn13mr662843oib.74.1644016725773; Fri, 04 Feb 2022 15:18:45 -0800 (PST) MIME-Version: 1.0 References: <202202011537.211FbYSe017204@freefriends.org> <20220201155225.5A9541FB21@orac.inputplus.co.uk> <202202020747.2127lTTh005669@freefriends.org> <7C19F93B-4F21-4BB1-A064-0307D3568DB7@cfcl.com> <1nFWmo-1Gn-00@marmaro.de> <202202040234.2142YeKN3307556@darkstar.fourwinds.com> In-Reply-To: <202202040234.2142YeKN3307556@darkstar.fourwinds.com> From: Dan Cross Date: Fri, 4 Feb 2022 18:18:09 -0500 Message-ID: To: Jon Steinhart Content-Type: multipart/alternative; boundary="000000000000f1d02a05d7397823" Subject: Re: [TUHS] more about Brian... [really Rust] X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: COFF Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --000000000000f1d02a05d7397823 Content-Type: text/plain; charset="UTF-8" [TUHS to Bcc, +COFF ] This isn't exactly COFF material, but I don't know what list is more appropriate. On Thu, Feb 3, 2022 at 9:41 PM Jon Steinhart wrote: > Adam Thornton writes: > > Do the august personages on this list have opinions about Rust? > > People who generally have tastes consonant with mine tell me I'd like > Rust. > > Well, I'm not an august personage and am not a Rust programmer. I did > spend a while trying to learn rust a while ago and wasn't impressed. > > Now, I'm heavily biased in that I think that it doesn't add value to keep > inventing new languages to do the same old things, and I didn't see > anything > in Rust that I couldn't do in a myriad of other languages. > I'm a Rust programmer, mostly using it for bare-metal kernel programming (though in my current gig, I find myself mostly in Rust userspace...ironically, it's back to C for the kernel). That said, I'm not a fan-boy for the language: it's not perfect. I've written basically four kernels in Rust now, to varying degrees of complexity from, "turn the computer on, spit hello-world out of the UART, and halt" to most of a v6 clone (which I really need to get around to finishing) to two rather more complex ones. I've done one ersatz kernel in C, and worked on a bunch more in C over the years. Between the two languages, I'd pick Rust over C for similar projects. Why? Because it really doesn't just do the same old things: it adds new stuff. Honest! Further, the sad reality (and the tie-in with TUHS/COFF) is that modern C has strayed far from its roots as a vehicle for systems programming, in particular, for implementing operating system kernels ( https://arxiv.org/pdf/2201.07845.pdf). C _implementations_ target the abstract machine defined in the C standard, not hardware, and they use "undefined behavior" as an excuse to make aggressive optimizations that change the semantics of one's program in such a way that some of the tricks you really do have to do when implementing an OS are just not easily done. For example, consider this code: uint16_t mul(uint16_t a, uint16_t b) { return a * b; } Does that code ever exhibit undefined behavior? The answer is that "it depends, but on most platforms, yes." Why? Because most often uint16_t is a typedef for `unsigned short int`, and because `short int` is of lesser "rank" than `int` and usually not as wide, the "usual arithmetic conversions" will apply before the multiplication. This means that the unsigned shorts will be converted to (signed) int. But on many platforms `int` will be a 32-bit integer (even 64-bit platforms!). However, the range of an unsigned 16-bit integer is such that the product of two uint16_t's can include values whose product is larger than whatever is representable in a signed 32-bit int, leading to overflow, and signed integer overflow is undefined overflow is undefined behavior. But does that _matter_ in practice? Potentially: since signed int overflow is UB, the compiler can decide it would never happen. And so if the compiler decides, for whatever reason, that (say) a saturating multiplication is the best way to implement that multiplication, then that simple single-expression function will yield results that (I'm pretty sure...) the programmer did not anticipate for some subset of inputs. How do you fix this? uint16_t mul(uint16_t a, uint16_t b) { unsigned int aa = a, bb = b; return aa * bb; } That may sound very hypothetical, but similar things have shown up in the wild: https://people.csail.mit.edu/nickolai/papers/wang-undef-2012-08-21.pdf In practice, this one is unlikely. But it's not impossible: the compiler would be right, the programmer would be wrong. One thing I've realized about C is that successive generations of compilers have tightened the noose on UB so that code that has worked for *years* all of a sudden breaks one day. There be dragons in our code. After being bit one too many times by such issues in C I decided to investigate alternatives. The choices at the time were either Rust or Go: for the latter, one gets a nice, relatively simple language, but a big complex runtime. For the former, you get a big, ugly language, but a minimal runtime akin to C: to get it going, you really don't have to do much more than set up a stack and join to a function. While people have built systems running Go at the kernel level ( https://pdos.csail.mit.edu/papers/biscuit.pdf), that seemed like a pretty heavy lift. On the other hand, if Rust could deliver on a quarter of the promises it made, I'd be ahead of the game. That was sometime in the latter half of 2018 and since then I've generally been pleasantly surprised at how much it really does deliver. For the above example, integer overflow is defined to trap. If you want wrapping (or saturating!) semantics, you request those explicitly: fn mul(a: u16, b: u16) -> u16 { a.wrapping_mul(b) } This is perfectly well-defined, and guaranteed to work pretty much forever. But, my real issue came from some of the tutorials that I perused. Rust is > being sold as "safer". As near as I can tell from the tutorials, the model > is that nothing works unless you enable it. Want to be able to write a > variable? Turn that on. So it seemed like the general style was to write > code and then turn various things on until it ran. > That's one way to look at it, but I don't think that's the intent: the model is rather, "immutable by default." Rust forces you to think about mutability, ownership, and the semantics of taking references, because the compiler enforces invariants on all of those things in a way that pretty much no other language does. It is opinionated, and not shy about sharing those opinions. To me, this implies a mindset that programming errors are more important > than thinking errors, and that one should hack on things until they work > instead of thinking about what one is doing. I know that that's the > modern definition of programming, but will never be for me. It's funny, I've had the exact opposite experience. I have found that it actually forces you to invest a _lot_ more in-up front thought about what you're doing. Writing code first, and then sprinkling in `mut` and `unsafe` until it compiles is a symptom of writing what we called "crust" on my last project at Google: that is, "C in Rust syntax." When I convinced our team to switch from C(++) to Rust, but none of us were really particularly adept at the language, and all hit similar walls of frustration; at one point, an engineer quipped, "this language has a near-vertical learning curve." And it's true that we took a multi-week productivity hit, but once we reached a certain level of familiarity, something equally curious happened: our debugging load went way, _way_ down and we started moving much faster. It turned out it was harder to get a Rust program to build at first, particularly with the bad habits we'd built up over decades of whatever languages we came from, but once it did those programs very often ran correctly the first time. You had to think _really hard_ about what data structures to use, their ownership semantics, their visibility, locking, etc. A lot of us had to absorb an emotional gut punch when the compiler showed us things that we _knew_ were correct were, in fact, not correct. But once code compiled, it tended not to have the kinds of errors that were insta-panics or triple faults (or worse, silent corruption you only noticed a million instructions later): no dangling pointers, no use-after-free bugs, no data races, no integer overflow, no out-of-bounds array references, etc. Simply put, the language _forced_ a level of discipline on us that even veteran C programmers didn't have. It also let us program at a moderately higher level of abstraction; off-by-one errors were gone because we had things like iterators. ADTs and a "Maybe" monad (the `Result` type) greatly improved our error handling. `match` statements have to be exhaustive so you can't add a variant to an enum and forget to update code to account in just that one place (the compiler squawks at you). It's a small point, but the `?` operator removed a lot of tedious boilerplate from our code, making things clearer without sacrificing robust failure handling. Tuples for multiple return values instead of using pointers for output arguments (that have to be manually checked for validity!) are really useful. Pattern matching and destructuring in a fast systems language? Good to go. In contrast, I ran into a "bug" of sorts with KVM due to code I wrote that manifested itself as an "x86 emulation error" when it was anything but: I was turning on paging very early in boot, and I had manually set up an identity mapping for the low 4GiB of address space for the jump from 32-bit to 64-bit mode. I used gigabyte pages since it was easy, and I figured it would be supported, but I foolishly didn't check the CPU features when running this under virtualization for testing and got that weird KVM error. What was going on? It turned out KVM in this case didn't support gig pages, but the hardware did; the software worked just fine until the first time the kernel went to do IO. Then, when the hypervisor went to fetch the instruction bytes to emulate the IO instruction, it saw the gig-sized pages and errored. Since the incompatibility was manifest deep in the bowels of the instruction emulation code, that was the error that returned, even though it had nothing to do with instruction emulation. It would have been nice to plumb through some kind of meaningful error message, but in C that's annoying at best. In Rust, it's trivial. https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ 70% of CVEs out of Microsoft over the last 15 years have been memory safety issues, and while we may poo-poo MSFT, they've got some really great engineers and let's be honest: Unix and Linux aren't that much better in this department. Our best and brightest C programmers continue to turn out highly buggy programs despite 50 years of experience. But it's not perfect. The allocator interface was a pain (it's defined to panic on allocation failure; I'm cool with a NULL return), though work is ongoing in this area. There's no ergonomic way to initialize an object 'in-place' (https://mcyoung.xyz/2021/04/26/move-ctors/), and there's no great way to say, essentially, "this points at RAM; even though I haven't initialized it, just trust me don't poison it" ( https://users.rust-lang.org/t/is-it-possible-to-read-uninitialized-memory-without-invoking-ub/63092 -- we really need a `freeze` operation). However, right now? I think it sits at a local maxima for systems languages targeting bare-metal. - Dan C. --000000000000f1d02a05d7397823 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
[TUHS to Bcc, +COFF= =C2=A0]

This isn't exactly COFF material, but = I don't know what list is more appropriate.

On Thu, Feb 3, 2022 at 9:41 PM Jon Steinhart <jon@fourwinds.com> wrote:
=
Adam Thornton writes:
> Do the august personages on this list have opinions about Rust?
> People who generally have tastes consonant with mine tell me I'd l= ike Rust.

Well, I'm not an august personage and am not a Rust programmer.=C2=A0 I= did
spend a while trying to learn rust a while ago and wasn't impressed.
Now, I'm heavily biased in that I think that it doesn't add value t= o keep
inventing new languages to do the same old things, and I didn't see any= thing
in Rust that I couldn't do in a myriad of other languages.=C2=A0

I'm a Rust programmer, mostly using it fo= r bare-metal kernel programming (though in my current gig, I find myself mo= stly in Rust userspace...ironically, it's back to C for the kernel). Th= at said, I'm not a fan-boy for the language: it's not perfect.

I've written basically four kernels in Rust now, t= o varying degrees of complexity from, "turn the computer on, spit hell= o-world out of the UART, and halt" to most of a v6 clone (which I real= ly need to get around to finishing) to two rather more complex ones. I'= ve done one ersatz kernel in C, and worked on a bunch more in C over the ye= ars. Between the two languages, I'd pick Rust over C for similar projec= ts.

Why? Because it really doesn't just do the= same old things: it adds new stuff. Honest!

Furth= er, the sad reality (and the tie-in with TUHS/COFF) is that modern C has st= rayed far from its roots as a vehicle for systems programming, in particula= r, for implementing operating system kernels (https://arxiv.org/pdf/2201.07845.pdf). C _implement= ations_ target the abstract machine defined in the C standard, not hardware= , and they use "undefined behavior" as an excuse to make aggressi= ve optimizations that change the semantics of one's program in such a w= ay that some of the tricks you really do have to do when implementing an OS= are just not easily done. For example, consider this code:

<= /div>
uint16_t mul(uint16_t a, uint16_t b) { return a * b; }
=
Does that code ever exhibit undefined behavior? The answer i= s that "it depends, but on most platforms, yes." Why? Because mos= t often uint16_t is a typedef for `unsigned short int`, and because `short = int` is of lesser "rank" than `int` and usually not as wide, the = "usual arithmetic conversions" will apply before the multiplicati= on. This means that the unsigned shorts will be converted to (signed) int. = But on many platforms=C2=A0`int` will be a 32-bit integer (even 64-bit plat= forms!). However, the range of an unsigned 16-bit integer is such that the = product of two uint16_t's can include values whose product is larger th= an whatever is representable in a signed 32-bit int, leading to overflow, a= nd signed integer overflow is undefined overflow is undefined behavior. But= does that _matter_ in practice? Potentially: since signed int overflow is = UB, the compiler can decide it would never happen. And so if the compiler d= ecides, for whatever reason, that (say) a saturating multiplication is the = best way to implement that multiplication, then that simple single-expressi= on function will yield results that (I'm pretty sure...) the programmer= did not anticipate for some subset of inputs. How do you fix this?

uint16_t mul(uint16_t a, uint16_t b) { unsigned int aa = =3D a, bb =3D b; return aa * bb; }

That may sound = very hypothetical, but similar things have shown up in the wild: ht= tps://people.csail.mit.edu/nickolai/papers/wang-undef-2012-08-21.pdf

In practice, this one is unlikely. But it's not = impossible: the compiler would be right, the programmer would be wrong. One= thing I've realized about C is that successive generations of compiler= s have tightened the noose on UB so that code that has worked for *years* a= ll of a sudden breaks one day. There be dragons in our code.

=
After being bit one too many times by such issues in C I de= cided to investigate alternatives. The choices at the time were either Rust= or Go: for the latter, one gets a nice, relatively simple language, but a = big complex runtime. For the former, you get a big, ugly language, but a mi= nimal runtime akin to C: to get it going, you really don't have to do m= uch more than set up a stack and join to a function. While people have buil= t systems running Go at the kernel level (https://pdos.csail.mit.edu/papers/biscuit.pdf)= , that seemed like a pretty heavy lift. On the other hand, if Rust could de= liver on a quarter of the promises it made, I'd be ahead of the game. T= hat was sometime in the latter half of 2018 and since then I've general= ly been pleasantly surprised at how much it really does deliver.

For the above example, integer overflow is defined to = trap. If you want wrapping (or saturating!) semantics, you request those ex= plicitly:

fn mul(a: u16, b: u16) -> u16 { a.wra= pping_mul(b) }

This is perfectly well-defined, and= guaranteed to work pretty much forever.

But, my real issue came from some of the tutorials that I perused.=C2=A0 Ru= st is
being sold as "safer".=C2=A0 As near as I can tell from the tutor= ials, the model
is that nothing works unless you enable it.=C2=A0 Want to be able to write = a
variable?=C2=A0 Turn that on.=C2=A0 So it seemed like the general style was= to write
code and then turn various things on until it ran.
That's one way to look at it, but I don't think that= 9;s the intent: the model is rather, "immutable by default."

Rust forces you to think about mutability, ownership, = and the semantics of taking references, because the compiler enforces invar= iants on all of those things in a way that pretty much no other language do= es. It is opinionated, and not shy about sharing those opinions.
=
To me, this implies a mindset that programming errors are more important than thinking errors, and that one should hack on things until they work instead of thinking about what one is doing.=C2=A0 I know that that's t= he
modern definition of programming, but will never be for me.

It's funny, I've had the exact opposite experience= .

I have found that it actually forces you to= invest a _lot_ more in-up front thought about what you're doing. Writi= ng code first, and then sprinkling in `mut` and `unsafe` until it compiles = is a symptom of writing what we called "crust" on my last project= at Google: that is, "C in Rust syntax." When I convinced our tea= m to switch from C(++) to Rust, but none of us were really particularly ade= pt at the language, and all hit similar walls of frustration; at one point,= an engineer quipped, "this language has a near-vertical learning curv= e." And it's true that we took a multi-week productivity hit, but = once we reached a certain level of familiarity, something equally curious h= appened: our debugging load went way, _way_ down and we started moving much= faster.

It turned out it was harder to get a Rust= program to build at first, particularly with the bad habits we'd built= up over decades of whatever languages we came from, but once it did those = programs very often ran correctly the first time. You had to think _really = hard_ about what data structures to use, their ownership=C2=A0semantics, th= eir visibility, locking, etc. A lot of us had to absorb an emotional=C2=A0g= ut punch when the compiler showed us things that we _knew_ were correct wer= e, in fact, not correct. But once code compiled, it tended not to have the = kinds of errors that were insta-panics or triple faults (or worse, silent c= orruption you only noticed a million instructions later): no dangling point= ers, no use-after-free bugs, no data races, no integer overflow, no out-of-= bounds array references, etc. Simply put, the language _forced_ a level of = discipline on us that even veteran C programmers didn't have.

It also let us program at a moderately higher level o= f abstraction; off-by-one errors were gone because we had things like itera= tors. ADTs and a "Maybe" monad (the `Result<T,E>` type) gre= atly improved our error handling. `match` statements have to be exhaustive = so you can't add a variant to an enum and forget to update code to acco= unt in just that one place (the compiler squawks at you). It's a small = point, but the `?` operator removed a lot of tedious boilerplate from our c= ode, making things clearer without sacrificing robust failure handling. Tup= les for multiple return values instead of using pointers for output argumen= ts (that have to be manually checked for validity!) are really useful. Patt= ern matching and destructuring in a fast systems language? Good to go.

In contrast, I ran into a "bug" of sorts wit= h KVM due to code I wrote that manifested itself as an "x86 emulation = error" when it was anything but: I was turning on paging very early in= boot, and I had manually set up an identity mapping for the low 4GiB of ad= dress space for the jump from 32-bit to 64-bit mode. I used gigabyte pages = since it was easy, and I figured it would be supported, but I foolishly did= n't check the CPU features when running this under virtualization for t= esting and got that weird KVM error. What was going on? It turned out KVM i= n this case didn't support gig pages, but the hardware did; the softwar= e worked just fine until the first time the kernel went to do IO. Then, whe= n the hypervisor went to fetch the instruction bytes to emulate the IO inst= ruction, it saw the gig-sized pages and errored. Since the incompatibility = was manifest deep in the bowels of the instruction emulation code, that was= the error that returned, even though it had nothing to do with instruction= emulation. It would have been nice to plumb through some kind of meaningfu= l error message, but in C that's annoying at best. In Rust, it's tr= ivial.=C2=A0https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-= validate/

70% of CVEs out of Microsoft over th= e last 15 years have been memory safety issues, and while we may poo-poo MS= FT, they've got some really great engineers and let's be honest: Un= ix and Linux aren't that much better in this department. Our best and b= rightest C programmers continue to turn out highly buggy programs despite 5= 0 years of experience.

But it's not perfect. T= he allocator interface was a pain (it's defined to panic on allocation = failure; I'm cool with a NULL return), though work is ongoing in this a= rea. There's no ergonomic way to initialize an object 'in-place'= ; (https://mcyoung.x= yz/2021/04/26/move-ctors/), and there's no great way to say, essent= ially, "this points at RAM; even though I haven't initialized it, = just trust me don't poison it" (https://users.rust-lang.org/t/is-it-possible-to-read-uninitialized-memor= y-without-invoking-ub/63092 -- we really need a `freeze` operation). Ho= wever, right now? I think it sits at a local maxima for systems languages t= argeting bare-metal.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 -= Dan C.

--000000000000f1d02a05d7397823--