9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
From: Steffen Nurpmeso <steffen@sdaoden.eu>
To: 9fans <9fans@9fans.net>
Subject: Re: [9fans] [PATCH] fossil: fix a deadlock in the caching logic
Date: Sat, 08 Apr 2023 17:27:57 +0200	[thread overview]
Message-ID: <20230408152757.IepXZ%steffen@sdaoden.eu> (raw)
In-Reply-To: <CAEoi9W6HFi8J4FOQXWkpd007gQfwVWMYVRqNcMY8VEPUE2M=Rg@mail.gmail.com>

Dan Cross wrote in
 <CAEoi9W6HFi8J4FOQXWkpd007gQfwVWMYVRqNcMY8VEPUE2M=Rg@mail.gmail.com>:
 |On Sat, Apr 8, 2023 at 10:37 AM Charles Forsyth
 |<charles.forsyth@gmail.com> wrote:
 |> It was the different characteristics of hard drives, even decent \
 |> SATA, compared to SSD and nvme that I had in mind.
 ...
 |In short, when you change storage technologies, assumptions that were
 |made with, say, a filesystem was initially written may be invalidated.
 |Consider the BSD FFS for example: UFS was written in an era of VAXen
 |and slow, 3600 RPM spinning disks like RA81s attached to relatively
 |unintelligent controllers; it made a number of fundamental design
 |decisions based on that, trying to optimize placement of data and
 |metadata near each other (to minimize head travel--this is the whole
 |cylinder group thing), implementation that explicitly accounted for
 |platter rotation with respect to scheduling operations for the
 |underlying storage device, putting multiple copies of the superblock
 |in multiple locations in the disk to maximize the chances of recovery
 |in the event of the (all-too-common) head crashes of the era, etc.
 |They also did very careful ordering of operations for soft-updates in
 |UFS2 to ensure filesystem consistency when updating metadata in the
 |face of a system crash (or power failure, or whatever). It turns out
 |that many of those optimizations become pessimizations (or at least
 |irrelevant) when you're all of a sudden writing to a solid-state
 |device, nevermind battery-backed DRAM on a much more advanced
 |controller.

Funnily Kirk McKusick committed on March 29th
fe5e6e2cc5d6f2e4121eccdb3a8ceba646aef2c9, saying

    Improvement in UFS/FFS directory placement when doing mkdir(2).

    The algorithm for laying out new directories was devised in the 1980s
    and markedly improved the performance of the filesystem. In those days
    large disks had at most 100 cylinder groups and often as few as 10-20.
    Modern multi-terrabyte disks have thousands of cylinder groups. The
    original algorithm does not handle these large sizes well. This change
    attempts to expand the scope of the original algorithm to work well
    with these much larger disks while still retaining the properties
    of the original algorithm for small disks.
    ...
    This change updates the ffs_dirpref() routine which is responsible
    for selecting the cylinder group into which a new directory should
    be placed. If we are near the root of the filesystem we aim to
    spread them out as much as possible. As we descend deeper from the
    root we cluster them closer together around their parent as we
    expect them to be more closely interactive. Higher-level directories
    like usr/src/sys and usr/src/bin should be separated while the
    directories in these areas are more likely to be accessed together
    so should be closer. And directories within commands or kernel
    subsystems should be closer still.

    We pick a range of cylinder groups around the cylinder group of the
    directory in which we are being created. The size of the range for
    our search is based on our depth from the root of our filesystem.
    We then probe that range based on how many directories are already
    present. The first new directory is at 1/2 (middle) of the range;
    the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8,

I only took a shallow look as i have no glue, but is sprung into
my eyes so i remembered it.

--steffen
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

------------------------------------------
9fans: 9fans
Permalink: https://9fans.topicbox.com/groups/9fans/T354fe702e1e9d5e9-Mc6cf9913394dadc41af96812
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

  reply	other threads:[~2023-04-08 17:14 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-04 17:15 noam
2023-04-04 18:03 ` Steve Simon
2023-04-04 18:34   ` Skip Tavakkolian
2023-04-04 20:44     ` Charles Forsyth
2023-04-04 20:50       ` Charles Forsyth
2023-04-05  1:59         ` noam
2023-04-05 21:25           ` Charles Forsyth
2023-04-06  3:22             ` noam
2023-04-06  3:57               ` Lucio De Re
2023-04-08  7:50                 ` hiro
2023-04-08 14:30                   ` Charles Forsyth
2023-04-08 14:36                     ` Charles Forsyth
2023-04-08 15:09                       ` Dan Cross
2023-04-08 15:27                         ` Steffen Nurpmeso [this message]
2023-04-08 17:12                         ` Bakul Shah
2023-04-04 22:07       ` Anthony Martin
2023-04-05  2:48         ` noam
2023-04-04 22:15   ` noam

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230408152757.IepXZ%steffen@sdaoden.eu \
    --to=steffen@sdaoden.eu \
    --cc=9fans@9fans.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).