From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 25873 invoked from network); 8 Apr 2023 17:12:54 -0000 Received: from tb-ob0.topicbox.com (64.147.108.117) by inbox.vuxu.org with ESMTPUTF8; 8 Apr 2023 17:12:54 -0000 Received: from tb-mx0.topicbox.com (tb-mx0.nyi.icgroup.com [10.90.30.73]) by tb-ob0.topicbox.com (Postfix) with ESMTP id 278B333409 for ; Sat, 8 Apr 2023 13:12:53 -0400 (EDT) (envelope-from bounce.mM30011df66797d8263cd1bf6c.r522be890-2105-11eb-b15e-8d699134e1fa@9fans.bounce.topicbox.com) Received: by tb-mx0.topicbox.com (Postfix, from userid 1132) id 23F7514E7E40; Sat, 8 Apr 2023 13:12:53 -0400 (EDT) ARC-Authentication-Results: i=2; topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=iitbombay-org.20210112.gappssmtp.com header.i=@iitbombay-org.20210112.gappssmtp.com header.b=l17r1BW7 header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=iitbombay.org; spf=pass smtp.mailfrom=bakul@iitbombay.org smtp.helo=mail-pj1-f46.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=content-type:content-transfer-encoding:from :mime-version:subject:date:message-id:references:in-reply-to:to :list-help:list-id:list-post:list-subscribe:reply-to :list-unsubscribe; s=sysmsg-1; t=1680973972; bh=9QSTdCw3TBc/LvTy npLei/F1OzzBhg00hhoV58EQ2N0=; b=qQvOW64kPlF6fogOsQ0fi4JkSHL0pGkw fC6JpKh0gyiO6nvgo+5yDOtt7DbrVyG/yeVAvrs3DUrpBvD252ThYeXjGsg6cVPX gohrCQKeSgbOV9NIPSL7lKmCwi4rHTW105jqj5mxnz97cocuRAFw45m8qPMUMdxa YptJD4UyycA= ARC-Seal: i=2; a=rsa-sha256; cv=pass; d=topicbox.com; s=sysmsg-1; t= 1680973972; b=efJ9epQ4LIIEToqbaaQLmOLt0WJLBZuYRZCNYQXVpt4Bpm5hJT tZ0O0ZumoK2bHOLrHo6gq4x/0sVUpd+iv2xn93kdQ9n2I9briAGID6twdKLi7rzj PWpd1ULHl9ySULZ99mqQGIS+mH5JWXOvx9/81Yti7job9fp2xk61emVQ0= Authentication-Results: topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=iitbombay-org.20210112.gappssmtp.com header.i=@iitbombay-org.20210112.gappssmtp.com header.b=l17r1BW7 header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=iitbombay.org; spf=pass smtp.mailfrom=bakul@iitbombay.org smtp.helo=mail-pj1-f46.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) X-Received-Authentication-Results: tb-mx0.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC did not pass); dkim=pass (2048-bit rsa key sha256) header.d=iitbombay-org.20210112.gappssmtp.com header.i=@iitbombay-org.20210112.gappssmtp.com header.b=l17r1BW7 header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=iitbombay.org; iprev=pass smtp.remote-ip=209.85.216.46 (mail-pj1-f46.google.com); spf=pass smtp.mailfrom=bakul@iitbombay.org smtp.helo=mail-pj1-f46.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=yGtx1nec; x-me-sender=none; x-ptr=pass smtp.helo=mail-pj1-f46.google.com policy.ptr=mail-pj1-f46.google.com; x-return-mx=pass header.domain=iitbombay.org policy.is_org=yes (MX Records found: alt3.aspmx.l.google.com,aspmx.l.google.com,alt1.aspmx.l.google.com,alt2.aspmx.l.google.com,alt4.aspmx.l.google.com); x-return-mx=pass smtp.domain=iitbombay.org policy.is_org=yes (MX Records found: alt3.aspmx.l.google.com,aspmx.l.google.com,alt1.aspmx.l.google.com,alt2.aspmx.l.google.com,alt4.aspmx.l.google.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=9fans.net; h= content-type:content-transfer-encoding:from:mime-version:subject :date:message-id:references:in-reply-to:to:list-help:list-id :list-post:list-subscribe:reply-to:list-unsubscribe; s=dkim-1; t=1680973972; x=1681060372; bh=N3Jjc/AAA+597UUaxsxcL86Tf2T+P8ZE HyNHaBecW0Q=; b=EQx7hf5KGjEFRvUZoATcl+Pi+LvJse2R6/KWIySC13tribLJ 4UJvQE4scDzDkZgjWTvslWsAmVIjreqBZQuI/ofAuparOo+ze9XOvb2uBTrIQign 9tZDH00cK6/XzbUtCN0o2CmZ6Y8Fu0dNR0njRWyNbCRaMUCkEBg6tQJENu4= Received: from tb-mx0.topicbox.com (localhost.local [127.0.0.1]) by tb-mx0.topicbox.com (Postfix) with ESMTP id 0643914E79E0 for <9fans@9fans.net>; Sat, 8 Apr 2023 13:12:40 -0400 (EDT) (envelope-from bakul@iitbombay.org) Received: from tb-mx0.topicbox.com (localhost [127.0.0.1]) by tb-mx0.topicbox.com (Authentication Milter) with ESMTP id 98B23B59C30; Sat, 8 Apr 2023 13:12:40 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1680973959; b=RvimQoKAUxT+bGwDGQdG5EKugLjuGI2V5rDOxs4twYtZcC0pGZ T6i7LmQjo3gUbrDLNHS7dHKokFQcf8iOBV4I68FlAP7qwUXo/Giwjt5Z0U2jFJVv +mI/JyKMfFTJP+nAr2kbF33FLR6IP5cC5TSKoeIXQe4d2uCc9s07DSZqlVIZfXYU oe4Q0GlcdihDtIK2u3qhxEd/Xi+zYOyaFPwjisXvK+3B5Zyl8ZTStmBV5cheGBCN 8Uw6pYhK/YYUIdCGPv7Qn3yW3mAfh1Tl6uaQMNKZQ04Wh9+rjdsjTPHcwSXv7Ov3 vrtdZr1ivcq7diZ95XRJvfTzENiag+vROvBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=content-type:content-transfer-encoding:from :mime-version:subject:date:message-id:references:in-reply-to:to; s=arcseal; t=1680973959; bh=bhxBRBkvjcdTqnd7Ya+JNxW0zYHSJAv+ai+ JNLKzf7o=; b=PFTV0I4Mjr/bE9FLju5he++3n/dQDXWIW7OKdptYty76W7TCk66 q9Vrfrnxpbiy6gjDVz1Ei8kVgJftsvxnB84gexjd1m7/P/hRMvL4tpMkuIqt/xbA ljT4HPAsvIi670JyBHKWJQ38VI0jSS2svR7fKTOSHSQwetIWHfAvak9XQHGQ73fC vMoh6vWu+pAoYyjgXpleoamDsY0e34m5xt3u7gEo+rqWjHWcV2OjEwp+epUC30HY SERaomLDMBmrSwmM/Wuc6cxd4kA4QMV6upqi5MC9sP8+KGruVIC+MZXfEpN4Pb9d MeYIkeZVTrboA1kO8pxzlqzadcZgZozyTvA== ARC-Authentication-Results: i=1; tb-mx0.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC did not pass); dkim=pass (2048-bit rsa key sha256) header.d=iitbombay-org.20210112.gappssmtp.com header.i=@iitbombay-org.20210112.gappssmtp.com header.b=l17r1BW7 header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=iitbombay.org; iprev=pass smtp.remote-ip=209.85.216.46 (mail-pj1-f46.google.com); spf=pass smtp.mailfrom=bakul@iitbombay.org smtp.helo=mail-pj1-f46.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=yGtx1nec; x-me-sender=none; x-ptr=pass smtp.helo=mail-pj1-f46.google.com policy.ptr=mail-pj1-f46.google.com; x-return-mx=pass header.domain=iitbombay.org policy.is_org=yes (MX Records found: alt3.aspmx.l.google.com,aspmx.l.google.com,alt1.aspmx.l.google.com,alt2.aspmx.l.google.com,alt4.aspmx.l.google.com); x-return-mx=pass smtp.domain=iitbombay.org policy.is_org=yes (MX Records found: alt3.aspmx.l.google.com,aspmx.l.google.com,alt1.aspmx.l.google.com,alt2.aspmx.l.google.com,alt4.aspmx.l.google.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdejjedgudduudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecunecujfgurheptgfghf ggufffkfhfjgfvofesthhqmhdthhdtjeenucfhrhhomhepuegrkhhulhcuufhhrghhuceo sggrkhhulhesihhithgsohhmsggrhidrohhrgheqnecuggftrfgrthhtvghrnhepfffgud dttdeltdeggfeiieekteegieeugfeifedthfevhfevkeehhfeijeetleeknecuffhomhgr ihhnpehtohhpihgtsghogidrtghomhenucfkphepvddtledrkeehrddvudeirdegiedpud dtjedrvdduhedrvddvfedrvddvleenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepihhnvghtpedvtdelrdekhedrvdduiedrgeeipdhhvghlohepmhgrihhlqdhpjhduqd hfgeeirdhgohhoghhlvgdrtghomhdpmhgrihhlfhhrohhmpeeosggrkhhulhesihhithgs ohhmsggrhidrohhrgheq X-ME-VSScore: 0 X-ME-VSCategory: clean Received-SPF: pass (iitbombay.org: Sender is authorized to use 'bakul@iitbombay.org' in 'mfrom' identity (mechanism 'include:_spf.google.com' matched)) receiver=tb-mx0.topicbox.com; identity=mailfrom; envelope-from="bakul@iitbombay.org"; helo=mail-pj1-f46.google.com; client-ip=209.85.216.46 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx0.topicbox.com (Postfix) with ESMTPS for <9fans@9fans.net>; Sat, 8 Apr 2023 13:12:39 -0400 (EDT) (envelope-from bakul@iitbombay.org) Received: by mail-pj1-f46.google.com with SMTP id d22-20020a17090a111600b0023d1b009f52so3786758pja.2 for <9fans@9fans.net>; Sat, 08 Apr 2023 10:12:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680973958; x=1683565958; h=to:in-reply-to:references:message-id:date:subject:mime-version:from :content-transfer-encoding:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=bhxBRBkvjcdTqnd7Ya+JNxW0zYHSJAv+ai+JNLKzf7o=; b=yGtx1nechd0pVy0OQF8V141QGVLtXUdDaoNm83F10/Z7BxdH6Fb8Skj1gUjsQKGjC2 lRmPze3T3Ca7HE5qgPxolwPJeCuTUTVMXZhKg7Wz4lt0r7BfQ1ahRzHXwFJQZi4blb7d n3oxHpuWVZlaTmBi7oFesF6NxWxAbVZO8KjFb6MtX4VU0mN0a0xwM/lCh0Ae8m3fBM6z MVpG37Fji7q6gvl0ekiYJzcGfWWlo2D7yClfHHVzISOB1QKZDoSamv39dHl6ErrnVBoe 7zxlMci9xmQt1Sk22mp1q6wbR5CJsbJdK0uafoDEEYCOpGyhCxTSNC2MnoYra0uksfus V0NQ== X-Gm-Message-State: AAQBX9e+qhN1vtmWeYL615REu9tDRinoMNYeDrNn3vJ2Zv7QX2FaAfSl n+NtjfEptGiLjZxkBz1R1RXJWMZpcVQOWGDxp35JYA== X-Google-Smtp-Source: AKy350Zx/SzeVtRNrjtwRe0GHXSyX0NsMZYU02xV0idRhHzuZL5SJjzgcbbbndCrLumjEoDjnbCTlA== X-Received: by 2002:a17:90b:1b4e:b0:237:50b6:9838 with SMTP id nv14-20020a17090b1b4e00b0023750b69838mr7306489pjb.45.1680973957971; Sat, 08 Apr 2023 10:12:37 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id j8-20020a17090276c800b0019908d2c85dsm4742671plt.52.2023.04.08.10.12.37 for <9fans@9fans.net> (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 08 Apr 2023 10:12:37 -0700 (PDT) Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable From: Bakul Shah Mime-Version: 1.0 (1.0) Subject: Re: [9fans] [PATCH] fossil: fix a deadlock in the caching logic Date: Sat, 8 Apr 2023 10:12:25 -0700 Message-Id: <043A7033-4868-47E2-B58A-EF6E543FC944@iitbombay.org> References: In-Reply-To: To: 9fans <9fans@9fans.net> X-Mailer: iPad Mail (20E252) Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 94ee9ca8-d630-11ed-95de-d304dbc837a7 Archived-At: =?UTF-8?B?PGh0dHBzOi8vOWZhbnMudG9waWNib3guY29tL2dyb3Vwcy85?= =?UTF-8?B?ZmFucy9UMzU0ZmU3MDJlMWU5ZDVlOS1NMzAwMTFkZjY2Nzk3ZDgyNjNjZDFi?= =?UTF-8?B?ZjZjPg==?= List-Help: List-Id: "9fans" <9fans.9fans.net> List-Post: List-Software: Topicbox v0 List-Subscribe: Precedence: list Reply-To: 9fans <9fans@9fans.net> List-Unsubscribe: , Topicbox-Delivery-ID: 2:9fans:437d30aa-c441-11e9-8a57-d036212d11b0:522be890-2105-11eb-b15e-8d699134e1fa:M30011df66797d8263cd1bf6c:1:pvdTqZ81H_b-Kz5miP4jlOtZg7DIiHrRWHq1hS-E_rg Things like wear leveling are done by the FTL (flash translation layer) in = the firmware. Other things it does: erase before write, logical to physical= mapping, erasing blocks, garbage collection (moving live data around to fr= ee up whole blocks) etc. Typically ease blocks are 128KB or larger but seem= to be treated as a secret by the SSD companies! At least NVMe SSDs provide= at least 64 request queues, each can hold lots of requests. There is enoug= h buffering to be able to flush all data to the Flash in case of power fai= lure but not sure if that is exposed to the user (apart from a flush comman= d). Apart from not doing seek related optimizations and placement, you=E2=80=99= d probably want to minimize unnecessary writes as SSD lifetime is limited b= y the amount you write (seems to be about at least 600 times the capacity s= o a TB disk will have 600TBW lifetime). That means avoiding metadata update= s if you can, Deduplication may also help. I have heard that you can never = really erase data even if you do a secure erase so the FS should have an en= cryption layer. On the flip side it may *lose* data if left unpowered for a= long time (this period goes down fast with increased temperature). JEDEC s= ays 1 year retention at 30=C2=B0C for consumer and 3 month retention at 40= =C2=B0C for enterprise SSDs. So may be a FS driver should do a background s= crub on reconnect if the device was not powered on for a long time. > On Apr 8, 2023, at 8:12 AM, Dan Cross wrote: >=20 > =EF=BB=BFOn Sat, Apr 8, 2023 at 10:37=E2=80=AFAM Charles Forsyth > wrote: >> It was the different characteristics of hard drives, even decent SATA, c= ompared to SSD and nvme that I had in mind. >=20 > Since details have been requested about this. I wouldn't presume to > speak from Charles, but some of those differences _may_ include: >=20 > 1. Optimizing for the rotational latency of spinning media, and its effec= ts vis: > a. the layout of storage structures on the disk, > b. placement of _data_ on the device. > 2. Effects with respect to things that aren't considerations for rotating= disks > a. Wear-leveling may be the canonical example here > 3. Effects at the controller level. > a. Caching, and the effect that has on how operations are ordered to > ensure consistency > b. Queuing for related objects written asynchronously and > assumptions about latency >=20 > In short, when you change storage technologies, assumptions that were > made with, say, a filesystem was initially written may be invalidated. > Consider the BSD FFS for example: UFS was written in an era of VAXen > and slow, 3600 RPM spinning disks like RA81s attached to relatively > unintelligent controllers; it made a number of fundamental design > decisions based on that, trying to optimize placement of data and > metadata near each other (to minimize head travel--this is the whole > cylinder group thing), implementation that explicitly accounted for > platter rotation with respect to scheduling operations for the > underlying storage device, putting multiple copies of the superblock > in multiple locations in the disk to maximize the chances of recovery > in the event of the (all-too-common) head crashes of the era, etc. > They also did very careful ordering of operations for soft-updates in > UFS2 to ensure filesystem consistency when updating metadata in the > face of a system crash (or power failure, or whatever). It turns out > that many of those optimizations become pessimizations (or at least > irrelevant) when you're all of a sudden writing to a solid-state > device, nevermind battery-backed DRAM on a much more advanced > controller. >=20 > - Dan C. ------------------------------------------ 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/T354fe702e1e9d5e9-M30011= df66797d8263cd1bf6c Delivery options: https://9fans.topicbox.com/groups/9fans/subscription