Re: mmaping on plan9? (was Re: [9fans] venti /plan9port mmapped

["Alyssa M via 9fans" <9fans@9fans.net>]

[-- Attachment #1: Type: text/plain, Size: 7059 bytes --]

Guys, I'm really sorry for the confusion.
A week ago I posted this, as a loose analogy, trying to relate what Ron said to my earlier idea:

On Tuesday, February 10, 2026, at 10:13 AM, Alyssa M wrote:
> On Monday, February 09, 2026, at 3:24 PM, ron minnich wrote:
>> as for mmap, there's already a defacto mmap happening for executables. They are not read into memory. In fact, the first instruction you run in a binary results in a page fault. 
> I thinking one could bring the same transparent/defacto memory mapping to read(2) and write(2), so the API need not change at all.

Unfortunately it got taken literally and I've been fire-fighting that ever since:

On Thursday, February 12, 2026, at 3:12 AM, Alyssa M wrote:
> Uses of mmap(2) and msync(2) could be replaced with read(2) and write(2) that sometimes use memory mapping as part of their implementation.
On Thursday, February 12, 2026, at 8:37 AM, Alyssa M wrote:
> The new read implementation would do this, but as a memory mapped snapshot. This looks no different to the programmer from how reads have always worked,
On Saturday, February 14, 2026, at 3:35 AM, Alyssa M wrote:
> This is not mmap, and not really memory mapping in the conventional sense either. But I think it can do the things an mmap user is looking for
On Monday, February 16, 2026, at 2:24 AM, Alyssa M wrote:
> So this is not mmap by another name. It's an optimization of the standard read/write approach that has some of the desirable characteristics of mmap.

All of these statements are my clumsy attempts to explain the same thing. There is memory mapping happening, in the sense that parts of the address space are sometimes associated with parts of files. But that's where the similarity ends. None of that is apparent to the programmer at any time. If it were it would be broken. Obviously.

In some ways this is like lazy evaluation: if you were actually to use all the data it would be slower. But if you only need part of it you can get access to it in memory up front (with read) without knowing ahead of time which bits you need.
This is not about making I/O faster, it's about doing less of it.



With regard to the snapshots:

Snapshots are just there to ensure that the proper read semantics are preserved.

On Monday, February 16, 2026, at 1:49 PM, Ori Bernstein wrote:
> Please describe, in detail, short of a per-file snapshot being
supported by the fs, what management to changes would actually
lead to zeros being read in the test C code?
When you're short of a per-file snapshot being supported by the fs, you get the traditional code path in read:

On Monday, February 16, 2026, at 1:49 PM, Ori Bernstein wrote:
> in another terminal, while this process is sleeping,
you run this command:

        dd -if /dev/random -of test -bs 16k -count 1kk
A normal read has happened because the kernel was unable to obtain a snapshot. The zeroes are already in memory (assuming you can actually read 16GB of zeroes from a file into the process within an hour!).
 
However:
It would be convenient for this if servers for disk file systems had the ability to create a snapshot of a range of bytes from a file. But they generally don't. So I'm building a file system wrapper layer (a file server that talks to another file server) that provides snapshot files to the kernel via an extension of the file server protocol, in addition to what the underlying file system already provides. My current implementation does this with temporary files. When a file is written, the temporary file gets any original data that's about to be overwritten. The snapshot provided to the kernel is a synthetic blend of the original file and any bytes that were rescued and put in the temporary file. In most uses the original file will never be touched by another process and the temporary file won't even be created.

The wrapper requires exclusive access to the file server underneath, and also requires the contents to be stable. It is the wrapper that is mounted in the namespace. So the wrapper sees all attempts to alter any file, and can ensure that that any snapshot maintains the illusion of being a full prior copy when writes later happen to the file it came from.

So with the wrapper in place...

On Monday, February 16, 2026, at 1:49 PM, Ori Bernstein wrote:
> in another terminal, while this process is sleeping,
you run this command:

        dd -if /dev/random -of test -bs 16k -count 1kk
The dd will cause the wrapper file system to copy the zeroes into a temporary file in the snapshot as it replaces them in the file with random data - which would presumably take three times as long as the earlier example), and the process will wake up after an hour and fault zeroes in from one page in the temporary file in the snapshot. The same happens if you delete or truncate the file: the wrapper will save the data in the temporary file first.
The kernel always sees a stable snapshot (if it sees one at all), which appears to be a readonly file the size of the read buffer. It can demand-load from that without interference for as long as it's needed, and it will remove on close.

The wrapper can be local or remote - wherever the file system needs to be shared.

On Monday, February 16, 2026, at 10:56 AM, Frank D. Engel, Jr. wrote:
> You could theoretically work around that by extending the protocol in a 
detectable way to provide the required support and only enabling this 
feature for filesystems which declare correct implementation of the 
extensions
Yes indeed.
On my hobby OS the file server protocol is not 9P, and adding a new request type to it is easy and expected.
For 9P it would probably be most sensible to do something like what I did for Linux last year, and use some kind of control file to communicate the request. It's kind of ugly, but can likely be done. Given how little this mechanism would be likely to be used in practice that's probably reasonable.

There's still more to say about what happens to memory pages after a write occurs, and about a really nice result that falls naturally out of all this when write knows where the data came from. But given the responses I've seen so far I'm not sure I dare...

On Monday, February 16, 2026, at 12:24 PM, hiro wrote:
> as you keep going in the same direction without responding to any of the doubt, i can't help but blame sycophantic AI, I don't manage to convince myself any more you're actually engaging with us properly as a human.
I don't post AI slop.

I hadn't actually intended to go into so much detail in this thread, particularly as I can't see myself doing this on Plan 9 - though I'm sure someone more expert than me could.
I'm dismayed by the responses so far, because I think this is potentially a lot better than mmap. 
------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/Te8d7c6e48b5c075b-M160f64e4b54be3298727d64a
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[-- Attachment #2: Type: text/html, Size: 9199 bytes --]