And you can keep track of mapped pages and read/write from them if
necessary even if you have a separate cache for any compressed pages.
In essence you pass the ownership of a page's data from a compressed
page cache to the mapped page. Just like in processor cache coherence
algorithms there is one source of truth: the current owner of a cached
unit (line or page or whatever). In other words, the you see via mmap(2)
will be the exact same page you will see via read(2). Not having actually
tried this I may have missed corner cases + any practical considerations
complicating things but *conceptually* this doesn't seem hard.
In essence, that's what the merged page/buffer cache is all about: file IO (read/write) operations are satisfied from the same memory cache that backs up VM objects. I agree that conceptually it's not that complex; but that's not what ZFS does.