From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id 97317219674B for ; Mon, 9 Sep 2024 07:33:01 -0400 (EDT) (envelope-from tsoome@me.com) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id B5312B4362A; Mon, 9 Sep 2024 07:33:01 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1725881581; b=tdOi/UtM242nGNx0/WzulBXzez3c8xH5Z5ZgQIvVfzKg6s6t3F DfJbYIshPJt9OQAgFMQlKlvQxmH9kyLRNwgMpVGQBGJ5FwLUXRrbplxCjIqNhwa1 I8RI/cEPU5mz1n9ySUkWxDzMXr1+NnkwY7HgwIS5298N73JTdLMjkWXd43j6WadE rYzGgaxNDZMYjU/VdCBhpfdRegDZMZAV8L5pYIg+6a5Qi04HDBrvp4EF0IQR2r5d No2Dj9OzZj/yaVexGFzzIayyik+gbY+QcREmZUylOCZGRwwT9LJVaso5gcglPZkc DiRTnrkrLFDnzi13LImUw7+hNNMbnU6/sxhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=from:content-type:mime-version:subject:date :references:to:in-reply-to:message-id; s=arcseal; t=1725881581; bh=ikkhDHDl1YGY+j2CB/cQxUvTEtlgtTnlIFSREFclg2o=; b=VIx0gkWviGhQ VxwonE20cbDi81AU5pMUTtH3sVbV8aaZAOnUlhDZ/s3XMQVMJbELmkAQgl4mV6L8 IStYAJ4pPGTsm3VjfktGxd4lK79l/28R106P4DNLS9jSa5QK3r9z8IvaHp3tx+Zv iByfzeToIpCT3CmE3scSfGc7BG80KsyOx5rhUCkvXKO31CEqSuvYQvGNGJDBPYE1 Ph5oCe/KKSPv8uGh83k8yPoP6+9q+3XEGiQ1nO2t4OGrvHpb9ldiVempBJyFEB9z 7Q5ZZUMiQVkdFdT/Q70gney+bR5YFI5DGBHXhagV8Wg71RXVF7y9N5NiYPjxkHfW ErOA/7DaAg== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); bimi=declined (Domain declined to participate); dkim=pass (2048-bit rsa key sha256) header.d=me.com header.i=@me.com header.b=pxZCUGJl header.a=rsa-sha256 header.s=1a1hai x-bits=2048; dmarc=pass policy.published-domain-policy=quarantine policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=me.com; iprev=pass smtp.remote-ip=17.58.23.197 (mr85p00im-zteg06012001.me.com); spf=pass smtp.mailfrom=tsoome@me.com smtp.helo=mr85p00im-zteg06012001.me.com; x-aligned-from=pass (Address match); x-me-sender=none; x-ptr=pass smtp.helo=mr85p00im-zteg06012001.me.com policy.ptr=mr85p00im-zteg06012001.me.com; x-return-mx=pass header.domain=me.com policy.is_org=yes (MX Records found: mx01.mail.icloud.com,mx02.mail.icloud.com); x-return-mx=pass smtp.domain=me.com policy.is_org=yes (MX Records found: mx01.mail.icloud.com,mx02.mail.icloud.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=-106 state=0 Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); bimi=declined (Domain declined to participate); dkim=pass (2048-bit rsa key sha256) header.d=me.com header.i=@me.com header.b=pxZCUGJl header.a=rsa-sha256 header.s=1a1hai x-bits=2048; dmarc=pass policy.published-domain-policy=quarantine policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=me.com; iprev=pass smtp.remote-ip=17.58.23.197 (mr85p00im-zteg06012001.me.com); spf=pass smtp.mailfrom=tsoome@me.com smtp.helo=mr85p00im-zteg06012001.me.com; x-aligned-from=pass (Address match); x-me-sender=none; x-ptr=pass smtp.helo=mr85p00im-zteg06012001.me.com policy.ptr=mr85p00im-zteg06012001.me.com; x-return-mx=pass header.domain=me.com policy.is_org=yes (MX Records found: mx01.mail.icloud.com,mx02.mail.icloud.com); x-return-mx=pass smtp.domain=me.com policy.is_org=yes (MX Records found: mx01.mail.icloud.com,mx02.mail.icloud.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=-106 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgeeftddrudeijedgvdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnegfrhhlucfvnfffucdlqdeimdenucfjughrpefhtggguffffhfv jgfkofesrgdtmherhhdtvdenucfhrhhomhepvfhoohhmrghsucfuohhomhgvuceothhsoh homhgvsehmvgdrtghomheqnecuggftrfgrthhtvghrnhepjeejgedtvddtffdvhfefteei feejveejtedvieehiedvveekudelhffftdfffeefnecuffhomhgrihhnpehfnhgrlhdrgh hovhenucfkphepudejrdehkedrvdefrdduleejpddujedrheejrdduhedvrddukeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpedujedrheekrddvfedrud eljedphhgvlhhopehmrhekhehptddtihhmqdiithgvghdtiedtuddvtddtuddrmhgvrdgt ohhmpdhmrghilhhfrhhomhepoehtshhoohhmvgesmhgvrdgtohhmqedpnhgspghrtghpth htohepuddprhgtphhtthhopeeoughishgtuhhssheslhhishhtshdrihhllhhumhhoshdr ohhrgheq X-ME-VSScore: -106 X-ME-VSCategory: clean Received-SPF: pass (me.com: 17.58.23.197 is authorized to use 'tsoome@me.com' in 'mfrom' identity (mechanism 'ip4:17.58.0.0/16' matched)) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="tsoome@me.com"; helo=mr85p00im-zteg06012001.me.com; client-ip=17.58.23.197 Received: from mr85p00im-zteg06012001.me.com (mr85p00im-zteg06012001.me.com [17.58.23.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for ; Mon, 9 Sep 2024 07:33:00 -0400 (EDT) (envelope-from tsoome@me.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=1a1hai; t=1725881579; bh=ikkhDHDl1YGY+j2CB/cQxUvTEtlgtTnlIFSREFclg2o=; h=From:Content-Type:Mime-Version:Subject:Date:To:Message-Id; b=pxZCUGJlZ9Rxex6tZZqz0zvshRc6lbDxvWqkiYj9YRIqwHblGrb23CE401+BQyatP M6iiBS6l/PG5Hn4D3yQujK62QMI7Me3LuNxbpdrpniobsU8PJW5jd2xAaYzbWBCqwp 4oXQbad9kG6Bl9u22cf3droZCS0xRkmzbR64rSBQunN/f08DBDyHFE0tZJayYeYGO9 Ohh6I2VOymeREiHOr56CK+T3OM0A5vdnyKBPJg0rrhrgtSeCTOkD4MUhHYSx/X0JMS b5yiNxRITuSDGhQW6Xzfup6jx1FYP5nDDDmxaMbmbdW5WNAw2AW4+HCds43GeP7bRt 65ZuwWAnsjPRw== Received: from smtpclient.apple (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-zteg06012001.me.com (Postfix) with ESMTPSA id 69EEF2E9755A for ; Mon, 9 Sep 2024 11:32:57 +0000 (UTC) From: Toomas Soome Content-Type: multipart/alternative; boundary="Apple-Mail=_792B6088-11A0-4656-B615-056EEC27EDFE" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: [discuss] Illumos future and compatibility with Open-ZFS Date: Mon, 9 Sep 2024 14:32:44 +0300 References: <2a17e53c-1178-4edb-aefd-8824fbd6e794@napp-it.org> <18FDA110-5626-4C27-95F9-F2B142DBEE1E@blackdot.be> <640324ae-25bb-4ae2-8596-1603e37d1a22@puptv.com> <3e75c998-57a6-4819-b30f-99713b46aea4@napp-it.org> To: illumos-discuss In-Reply-To: <3e75c998-57a6-4819-b30f-99713b46aea4@napp-it.org> Message-Id: X-Mailer: Apple Mail (2.3776.700.51) X-Proofpoint-GUID: Joj5zEzmEiabcaOhFFFaGeRhmsuzbVpR X-Proofpoint-ORIG-GUID: Joj5zEzmEiabcaOhFFFaGeRhmsuzbVpR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-09_04,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 clxscore=1015 mlxscore=0 phishscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2409090092 Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 48d61392-6e9f-11ef-8ff1-ec081c20a363 --Apple-Mail=_792B6088-11A0-4656-B615-056EEC27EDFE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 9. Sep 2024, at 13:41, Gea wrote: >=20 > On 08.09.2024 11:55, d wrote: >> This post enticed me to do a little research on zfs compression, = which lead me to some interesting info an a question: >> = https://indico.fnal.gov/event/16264/contributions/36466/attachments/22610/= 28037/Zstd__LZ4.pdf >=20 > The simplified result seems lz4 is faster while zstd compress data = more efficient. >=20 > When I look at my main pool, refcompressratio with lz4 is 1.04 for my = main data filesystem with mixed data, some at 1.02 with mainly iso and = media and up to 1.12 in rare cases. With expensive NVMe or hybrid pools = with a special vdev, I would opt for zstd and better compressratio as = performance is better than good enough. But I have this choice only with = Open-ZFS and this for quite a long time similar to draid with many = advantages on very large pools. tsoome@beastie:~$ zfs get compression rpool/ROOT/openindiana-2024:08:23 NAME PROPERTY VALUE SOURCE rpool/ROOT/openindiana-2024:08:23 compression zstd = inherited from rpool tsoome@beastie:~$ zfs get compressratio = rpool/ROOT/openindiana-2024:08:23 NAME PROPERTY VALUE SOURCE rpool/ROOT/openindiana-2024:08:23 compressratio 2.75x - tsoome@beastie:~$ zfs get compression rpool/code/illumos-gate NAME PROPERTY VALUE SOURCE rpool/code/illumos-gate compression zstd local tsoome@beastie:~$ zfs get compressratio rpool/code/illumos-gate NAME PROPERTY VALUE SOURCE rpool/code/illumos-gate compressratio 3.32x - tsoome@beastie:~$ (I probably still do have some lz4 blocks lurking around). rgds, toomas >=20 > The upcoming Fast Dedup in Open-ZFS where you can limit dedup table = size with a quota, place the dedup table not only on a dedicated dedup = vdev but also a special vdev, can shrink dedup tables from single items = or cache dedup in Arc seems a killer feature. Unlike current dedup, I = would expect that you can enable Fast Dedup per default, either with a = special vdev that also massively improve all small file actions and = metadata access or with a quota like a few GB to limit RAM usage. Hybrid = pools with a special vdev become more and more attractive as suggested = default. Fast dedup is beta but near to be ready. I already can play = with in in Open-ZFS on Windows rc. Will this appear in Illumos in the = forseeable future. I doubt. >=20 > The stability aspect, ok. Software has bugs and you need to maintain = does not matter if Open-ZFS or Illumos-ZFS. There have also been = critical bugs in Illumos. When I remember correctly it was persistent = L2Arc in Illumos where datalosses could have happened a few years ago = (luckily I was not affected as silent errors could have happened). >=20 > Why do anyone expects that bugs in Illumos ZFS do not happen or fixed = faster than bugs in Open-ZFS with 20x the user base and number of devs = and many large commercial companies behind. Even the additional tests on = taking over Open-ZFS code in more and more rare cases does not guarantee = that there are no bugs remained (while helpful for sure and better as = using Open-ZFS master or the stable branch directly) >=20 --Apple-Mail=_792B6088-11A0-4656-B615-056EEC27EDFE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On 9. Sep 2024, at 13:41, Gea <gea@napp-it.org> = wrote:

On = 08.09.2024 11:55, d wrote:
This post = enticed me to do a little research on zfs compression, which lead me to = some interesting info an a = question:
https://indico.fnal.gov/event/16264/contributions/36466/attac= hments/22610/28037/Zstd__LZ4.pdf

The simplified = result seems lz4 is faster while zstd compress data more = efficient.

When I look at my main pool, refcompressratio with lz4 = is 1.04 for my main data filesystem with mixed data, some at 1.02 with = mainly iso and media and up to 1.12 in rare cases. With expensive NVMe = or hybrid pools with a special vdev, I would opt for zstd and better = compressratio as performance is better than good enough. But I have this = choice only with Open-ZFS and this for quite a long time similar to = draid with many advantages on very large = pools.

tsoome@beastie:~$ = zfs get compression rpool/ROOT/openindiana-2024:08:23

NAME     =                     =       PROPERTY     VALUE       =     SOURCE

rpool/ROOT/openindiana-2024:08:23  = compression  zstd            = inherited from rpool

tsoome@beastie:~$ zfs get compressratio  = rpool/ROOT/openindiana-2024:08:23

NAME     =                     =       PROPERTY       VALUE  = SOURCE

rpool/ROOT/openindiana-2024:08:23  = compressratio  2.75x  -


tsoome@beastie:~$ = zfs get compression rpool/code/illumos-gate

NAME     =                 PROPERTY   =   VALUE           SOURCE

rpool/code/illumos-gate  compression  = zstd            local

tsoome@beastie:~$ = zfs get compressratio rpool/code/illumos-gate

NAME     =                 PROPERTY   =     VALUE  SOURCE

rpool/code/illumos-gate  compressratio  = 3.32x  -

tsoome@beastie:~$


(I probably still = do have some lz4 blocks lurking around).


rgds,

toomas



The upcoming Fast Dedup in Open-ZFS where = you can limit dedup table size with a quota,  place the dedup table = not only on a dedicated dedup vdev but also a special vdev, can shrink = dedup tables from single items or cache dedup in Arc seems a killer = feature. Unlike current dedup, I would expect that you can enable Fast = Dedup per default, either with a special vdev that also massively = improve all small file actions and metadata access or with a quota like = a few GB to limit RAM usage. Hybrid pools with a special vdev become = more and more attractive as suggested default. Fast dedup is beta but = near to be ready. I already can play with in in Open-ZFS on Windows rc. = Will this appear in Illumos in the forseeable future. I = doubt.

The stability aspect, ok. Software has bugs and you need = to maintain does not matter if Open-ZFS or Illumos-ZFS. There have also = been critical bugs in Illumos. When I remember correctly it was = persistent L2Arc in Illumos where datalosses could have happened a few = years ago (luckily I was not affected as silent errors could have = happened).

Why do anyone expects that bugs in Illumos ZFS = do not happen or fixed faster than bugs in Open-ZFS with 20x the user = base and number of devs and many large commercial companies behind. Even = the additional tests on taking over Open-ZFS code in more and more rare = cases does not guarantee that there are no bugs remained (while helpful = for sure and better as using Open-ZFS master or the stable branch = directly)



= = --Apple-Mail=_792B6088-11A0-4656-B615-056EEC27EDFE--