Re: [COFF] [TUHS] more about Brian... [really Rust]

[Dan Cross <crossd@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 10779 bytes --]

[TUHS to Bcc, +COFF <coff@minnie.tuhs.org> ]

This isn't exactly COFF material, but I don't know what list is more
appropriate.

On Thu, Feb 3, 2022 at 9:41 PM Jon Steinhart <jon@fourwinds.com> wrote:

> Adam Thornton writes:
> > Do the august personages on this list have opinions about Rust?
> > People who generally have tastes consonant with mine tell me I'd like
> Rust.
>
> Well, I'm not an august personage and am not a Rust programmer.  I did
> spend a while trying to learn rust a while ago and wasn't impressed.
>
> Now, I'm heavily biased in that I think that it doesn't add value to keep
> inventing new languages to do the same old things, and I didn't see
> anything
> in Rust that I couldn't do in a myriad of other languages.
>

I'm a Rust programmer, mostly using it for bare-metal kernel programming
(though in my current gig, I find myself mostly in Rust
userspace...ironically, it's back to C for the kernel). That said, I'm not
a fan-boy for the language: it's not perfect.

I've written basically four kernels in Rust now, to varying degrees of
complexity from, "turn the computer on, spit hello-world out of the UART,
and halt" to most of a v6 clone (which I really need to get around to
finishing) to two rather more complex ones. I've done one ersatz kernel in
C, and worked on a bunch more in C over the years. Between the two
languages, I'd pick Rust over C for similar projects.

Why? Because it really doesn't just do the same old things: it adds new
stuff. Honest!

Further, the sad reality (and the tie-in with TUHS/COFF) is that modern C
has strayed far from its roots as a vehicle for systems programming, in
particular, for implementing operating system kernels (
https://arxiv.org/pdf/2201.07845.pdf). C _implementations_ target the
abstract machine defined in the C standard, not hardware, and they use
"undefined behavior" as an excuse to make aggressive optimizations that
change the semantics of one's program in such a way that some of the tricks
you really do have to do when implementing an OS are just not easily done.
For example, consider this code:

uint16_t mul(uint16_t a, uint16_t b) { return a * b; }

Does that code ever exhibit undefined behavior? The answer is that "it
depends, but on most platforms, yes." Why? Because most often uint16_t is a
typedef for `unsigned short int`, and because `short int` is of lesser
"rank" than `int` and usually not as wide, the "usual arithmetic
conversions" will apply before the multiplication. This means that the
unsigned shorts will be converted to (signed) int. But on many
platforms `int` will be a 32-bit integer (even 64-bit platforms!). However,
the range of an unsigned 16-bit integer is such that the product of two
uint16_t's can include values whose product is larger than whatever is
representable in a signed 32-bit int, leading to overflow, and signed
integer overflow is undefined overflow is undefined behavior. But does that
_matter_ in practice? Potentially: since signed int overflow is UB, the
compiler can decide it would never happen. And so if the compiler decides,
for whatever reason, that (say) a saturating multiplication is the best way
to implement that multiplication, then that simple single-expression
function will yield results that (I'm pretty sure...) the programmer did
not anticipate for some subset of inputs. How do you fix this?

uint16_t mul(uint16_t a, uint16_t b) { unsigned int aa = a, bb = b; return
aa * bb; }

That may sound very hypothetical, but similar things have shown up in the
wild: https://people.csail.mit.edu/nickolai/papers/wang-undef-2012-08-21.pdf

In practice, this one is unlikely. But it's not impossible: the compiler
would be right, the programmer would be wrong. One thing I've realized
about C is that successive generations of compilers have tightened the
noose on UB so that code that has worked for *years* all of a sudden breaks
one day. There be dragons in our code.

After being bit one too many times by such issues in C I decided to
investigate alternatives. The choices at the time were either Rust or Go:
for the latter, one gets a nice, relatively simple language, but a big
complex runtime. For the former, you get a big, ugly language, but a
minimal runtime akin to C: to get it going, you really don't have to do
much more than set up a stack and join to a function. While people have
built systems running Go at the kernel level (
https://pdos.csail.mit.edu/papers/biscuit.pdf), that seemed like a pretty
heavy lift. On the other hand, if Rust could deliver on a quarter of the
promises it made, I'd be ahead of the game. That was sometime in the latter
half of 2018 and since then I've generally been pleasantly surprised at how
much it really does deliver.

For the above example, integer overflow is defined to trap. If you want
wrapping (or saturating!) semantics, you request those explicitly:

fn mul(a: u16, b: u16) -> u16 { a.wrapping_mul(b) }

This is perfectly well-defined, and guaranteed to work pretty much forever.

But, my real issue came from some of the tutorials that I perused.  Rust is
> being sold as "safer".  As near as I can tell from the tutorials, the model
> is that nothing works unless you enable it.  Want to be able to write a
> variable?  Turn that on.  So it seemed like the general style was to write
> code and then turn various things on until it ran.
>

That's one way to look at it, but I don't think that's the intent: the
model is rather, "immutable by default."

Rust forces you to think about mutability, ownership, and the semantics of
taking references, because the compiler enforces invariants on all of those
things in a way that pretty much no other language does. It is opinionated,
and not shy about sharing those opinions.

To me, this implies a mindset that programming errors are more important
> than thinking errors, and that one should hack on things until they work
> instead of thinking about what one is doing.  I know that that's the
> modern definition of programming, but will never be for me.


It's funny, I've had the exact opposite experience.

I have found that it actually forces you to invest a _lot_ more in-up front
thought about what you're doing. Writing code first, and then sprinkling in
`mut` and `unsafe` until it compiles is a symptom of writing what we called
"crust" on my last project at Google: that is, "C in Rust syntax." When I
convinced our team to switch from C(++) to Rust, but none of us were really
particularly adept at the language, and all hit similar walls of
frustration; at one point, an engineer quipped, "this language has a
near-vertical learning curve." And it's true that we took a multi-week
productivity hit, but once we reached a certain level of familiarity,
something equally curious happened: our debugging load went way, _way_ down
and we started moving much faster.

It turned out it was harder to get a Rust program to build at first,
particularly with the bad habits we'd built up over decades of whatever
languages we came from, but once it did those programs very often ran
correctly the first time. You had to think _really hard_ about what data
structures to use, their ownership semantics, their visibility, locking,
etc. A lot of us had to absorb an emotional gut punch when the compiler
showed us things that we _knew_ were correct were, in fact, not correct.
But once code compiled, it tended not to have the kinds of errors that were
insta-panics or triple faults (or worse, silent corruption you only noticed
a million instructions later): no dangling pointers, no use-after-free
bugs, no data races, no integer overflow, no out-of-bounds array
references, etc. Simply put, the language _forced_ a level of discipline on
us that even veteran C programmers didn't have.

It also let us program at a moderately higher level of abstraction;
off-by-one errors were gone because we had things like iterators. ADTs and
a "Maybe" monad (the `Result<T,E>` type) greatly improved our error
handling. `match` statements have to be exhaustive so you can't add a
variant to an enum and forget to update code to account in just that one
place (the compiler squawks at you). It's a small point, but the `?`
operator removed a lot of tedious boilerplate from our code, making things
clearer without sacrificing robust failure handling. Tuples for multiple
return values instead of using pointers for output arguments (that have to
be manually checked for validity!) are really useful. Pattern matching and
destructuring in a fast systems language? Good to go.

In contrast, I ran into a "bug" of sorts with KVM due to code I wrote that
manifested itself as an "x86 emulation error" when it was anything but: I
was turning on paging very early in boot, and I had manually set up an
identity mapping for the low 4GiB of address space for the jump from 32-bit
to 64-bit mode. I used gigabyte pages since it was easy, and I figured it
would be supported, but I foolishly didn't check the CPU features when
running this under virtualization for testing and got that weird KVM error.
What was going on? It turned out KVM in this case didn't support gig pages,
but the hardware did; the software worked just fine until the first time
the kernel went to do IO. Then, when the hypervisor went to fetch the
instruction bytes to emulate the IO instruction, it saw the gig-sized pages
and errored. Since the incompatibility was manifest deep in the bowels of
the instruction emulation code, that was the error that returned, even
though it had nothing to do with instruction emulation. It would have been
nice to plumb through some kind of meaningful error message, but in C
that's annoying at best. In Rust, it's trivial.
https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/

70% of CVEs out of Microsoft over the last 15 years have been memory safety
issues, and while we may poo-poo MSFT, they've got some really great
engineers and let's be honest: Unix and Linux aren't that much better in
this department. Our best and brightest C programmers continue to turn out
highly buggy programs despite 50 years of experience.

But it's not perfect. The allocator interface was a pain (it's defined to
panic on allocation failure; I'm cool with a NULL return), though work is
ongoing in this area. There's no ergonomic way to initialize an object
'in-place' (https://mcyoung.xyz/2021/04/26/move-ctors/), and there's no
great way to say, essentially, "this points at RAM; even though I haven't
initialized it, just trust me don't poison it" (
https://users.rust-lang.org/t/is-it-possible-to-read-uninitialized-memory-without-invoking-ub/63092
-- we really need a `freeze` operation). However, right now? I think it
sits at a local maxima for systems languages targeting bare-metal.

        - Dan C.

[-- Attachment #1.2: Type: text/html, Size: 12941 bytes --]

[-- Attachment #2: Type: text/plain, Size: 141 bytes --]

_______________________________________________
COFF mailing list
COFF@minnie.tuhs.org
https://minnie.tuhs.org/cgi-bin/mailman/listinfo/coff