From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FROM,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 29753 invoked from network); 1 Jun 2022 15:39:55 -0000 Received: from tb-ob0.topicbox.com (64.147.108.117) by inbox.vuxu.org with ESMTPUTF8; 1 Jun 2022 15:39:55 -0000 Received: from tb-mx0.topicbox.com (tb-mx0.nyi.icgroup.com [10.90.30.73]) by tb-ob0.topicbox.com (Postfix) with ESMTP id 925352974E for ; Wed, 1 Jun 2022 11:39:54 -0400 (EDT) (envelope-from bounce.mM099a56feb7c401cc1d0b3ed6.r522be890-2105-11eb-b15e-8d699134e1fa@9fans.bounce.topicbox.com) Received: by tb-mx0.topicbox.com (Postfix, from userid 1132) id 8E70126042EE; Wed, 1 Jun 2022 11:39:54 -0400 (EDT) ARC-Authentication-Results: i=2; topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=gmail.com header.i=@gmail.com header.b=hB4Zc8IZ header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=gmail.com; spf=pass smtp.mailfrom=23hiro@gmail.com smtp.helo=mail-pl1-f182.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:list-help:list-id:list-post :list-subscribe:reply-to:content-transfer-encoding :list-unsubscribe; s=sysmsg-1; t=1654097994; bh=/fVgdbwKQ/bfqwMA klPhod00eClBrOvsIQRBByduMgg=; b=KhcUBOUxR1rHL6n/c9B9FEZOjENDj142 4Q1zoYOgN+jm35/IWK4WQRcrbiiQH02w0wBV/npSMfGigHw4OkvRdJU94qeUNn02 qp+fP9WtyyI/ldSZtE3kB0BRosvxpuoLbGfdIISo2KGlhlLBRpYk1KCT7Q6sZzmt i+WcC0rAX00= ARC-Seal: i=2; a=rsa-sha256; cv=pass; d=topicbox.com; s=sysmsg-1; t= 1654097994; b=mAoJSUAudywwKzbBU6G+jaUkcP80RYFOhUNho179/Ao5wik3pu urdzzwVNqGCYIqZjMmQSSBXXPnC5RyhTkPi3dQekMQ94dtCZ6agkxpoOhP+NLbah NUvuw0wnceyXaM3y6kqwXqUUIV8f0mmopn64r6zUK0TgdlDs8n4uPXLIg= Authentication-Results: topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=gmail.com header.i=@gmail.com header.b=hB4Zc8IZ header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=gmail.com; spf=pass smtp.mailfrom=23hiro@gmail.com smtp.helo=mail-pl1-f182.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) X-Received-Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC Policy is not at enforcement); dkim=pass (2048-bit rsa key sha256) header.d=gmail.com header.i=@gmail.com header.b=hB4Zc8IZ header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=gmail.com; iprev=pass smtp.remote-ip=209.85.214.182 (mail-pl1-f182.google.com); spf=pass smtp.mailfrom=23hiro@gmail.com smtp.helo=mail-pl1-f182.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=GRrGMtjW; x-me-sender=none; x-ptr=pass smtp.helo=mail-pl1-f182.google.com policy.ptr=mail-pl1-f182.google.com; x-return-mx=pass header.domain=gmail.com policy.is_org=yes (MX Records found: alt4.gmail-smtp-in.l.google.com,gmail-smtp-in.l.google.com,alt2.gmail-smtp-in.l.google.com,alt3.gmail-smtp-in.l.google.com,alt1.gmail-smtp-in.l.google.com); x-return-mx=pass smtp.domain=gmail.com policy.is_org=yes (MX Records found: alt4.gmail-smtp-in.l.google.com,gmail-smtp-in.l.google.com,alt2.gmail-smtp-in.l.google.com,alt3.gmail-smtp-in.l.google.com,alt1.gmail-smtp-in.l.google.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=9fans.net; h= mime-version:in-reply-to:references:from:date:message-id:subject :to:content-type:list-help:list-id:list-post:list-subscribe :reply-to:content-transfer-encoding:list-unsubscribe; s=dkim-1; t=1654097994; x=1654184394; bh=o1FwoYMhzqsnNsdX73l/AliGiacWMQSF gw8bXf+HNcY=; b=G3v+v4l3WpUYdq/5M/t1U/JHl3jLP3+9QEC3KywlLIAvxfDM PC5VjDQ3LYMQ90ZvB2FtLFZkIcoeJq6JGGN1+OuL7vqc7uiJqIhfg+MHgrvtNzLE cIvL+6xms3o1Lm5QqtnIk8RJZ+hbzdIa7g6FzfK40UhifrC7uMGe5K77+5I= Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id C206E1515023 for <9fans@9fans.net>; Wed, 1 Jun 2022 11:39:36 -0400 (EDT) (envelope-from 23hiro@gmail.com) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id B1658A57824; Wed, 1 Jun 2022 11:39:36 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1654097976; b=RWfROAdHhAhqP6k3AV212+AE9aaWbc5rJlaTSGkePYzD7wFogm IXdsHyhV1PPF9wKYMT3z4i0rl66jcY/P9zRbsmA3PhQfsEQ1kwGK14mmFHyIqFTC OeEDOCP1e2PcXqoaNA1pHLfY9MD3WGPFx+iWtv4k5h71DP1vNcQH8pgM4hBKp5qG wlOT9Q+hMVhnSIqIXhKjG79mncMn0mKAgX7fiue4qjhau4t977HNfYgJWeVDgTJ/ eimdny8vtxiqVReEaPLGmUq9pBfgWSGCQe+IJoz+rDJlnRDb62zyOk1DK/LaPt1a T/7LFGqeuuaM3cE7+1+7hqpAG6Tjv53HC6GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; s=arcseal; t=1654097976; bh=YeLvoeolN1IT84wR0M08rHYBB5xtylYijF+UXpvKeOw=; b=Z8+ZdxmCb8U3 fLxcKT/v+7wqGVDlxO+bNhslKQr5/7RVODldoAHxh1mAnAVBjKLtbyG4L4vlMlKh rwf7qJCWG4sFFt2hGDfEEwAa/yPUTypIBXoslkBKr/Wvbyg7kSlJX9yssUvSPFrf E4RfTZs3HA5Y9sbh3JNU+RUC7xa/0hHSQQU4iWWp205Yn4BTww5zsjOLzz5MlQZn whhzp/YDEM9P9+lJguKo6YS1uALmaR3bTtbLa28JieZKurhFcfPpGM44tBkYY2Hv MJ6nGGw0j/E80iOF/JtpEfIjpQd+3C6soKxar/jx/wKDnQ/DrSFZn3BSXtQDZHyx /L2Y+JkSBA== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC Policy is not at enforcement); dkim=pass (2048-bit rsa key sha256) header.d=gmail.com header.i=@gmail.com header.b=hB4Zc8IZ header.a=rsa-sha256 header.s=20210112 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=gmail.com; iprev=pass smtp.remote-ip=209.85.214.182 (mail-pl1-f182.google.com); spf=pass smtp.mailfrom=23hiro@gmail.com smtp.helo=mail-pl1-f182.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=GRrGMtjW; x-me-sender=none; x-ptr=pass smtp.helo=mail-pl1-f182.google.com policy.ptr=mail-pl1-f182.google.com; x-return-mx=pass header.domain=gmail.com policy.is_org=yes (MX Records found: alt4.gmail-smtp-in.l.google.com,gmail-smtp-in.l.google.com,alt2.gmail-smtp-in.l.google.com,alt3.gmail-smtp-in.l.google.com,alt1.gmail-smtp-in.l.google.com); x-return-mx=pass smtp.domain=gmail.com policy.is_org=yes (MX Records found: alt4.gmail-smtp-in.l.google.com,gmail-smtp-in.l.google.com,alt2.gmail-smtp-in.l.google.com,alt3.gmail-smtp-in.l.google.com,alt1.gmail-smtp-in.l.google.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgedvfedrledtgdeltdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecunecujfgurhepjghfhfffkf fuvfgtsehttdertddttdejnecuhfhrohhmpehhihhrohcuoedvfehhihhrohesghhmrghi lhdrtghomheqnecuggftrfgrthhtvghrnheptdevkeevffdvieeiteeuvdfghfdukefhle dtveehgfefkeelffehleehtddviefgnecuffhomhgrihhnpehtohhpihgtsghogidrtgho mhenucfkphepvddtledrkeehrddvudegrddukedvnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehinhgvthepvddtledrkeehrddvudegrddukedvpdhhvghlohepmhgr ihhlqdhplhduqdhfudekvddrghhoohhglhgvrdgtohhmpdhmrghilhhfrhhomhepoedvfe hhihhrohesghhmrghilhdrtghomheq X-ME-VSScore: 0 X-ME-VSCategory: clean Received-SPF: pass (gmail.com ... _spf.google.com: Sender is authorized to use '23hiro@gmail.com' in 'mfrom' identity (mechanism 'include:_netblocks.google.com' matched)) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="23hiro@gmail.com"; helo=mail-pl1-f182.google.com; client-ip=209.85.214.182 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for <9fans@9fans.net>; Wed, 1 Jun 2022 11:39:36 -0400 (EDT) (envelope-from 23hiro@gmail.com) Received: by mail-pl1-f182.google.com with SMTP id c2so2164011plh.2 for <9fans@9fans.net>; Wed, 01 Jun 2022 08:39:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=YeLvoeolN1IT84wR0M08rHYBB5xtylYijF+UXpvKeOw=; b=GRrGMtjWIqAcnWQQt8N83SAVqLhhyM+73hMavmXvQTIKk/ZMZChKWrXKzhJlQI1H9A L418VJHTFFOZWroHdb2U78Vb3elRrd12kSg55quimyWwsrqUOo8wjj2XlqA7H5tPtqOm t04XqTYcBGINb2MgfoJkMDHUPxcJibxrDrHIu7iptcbUYjL26vgnEzOGS2TrJKHBWkuE HrK/M4JoIqWrH3qI4/R37MqvOWBzoxIMLiISz9cAdgk4tEtnO5CAR4EFaTpRlzDycLI2 msSD7nnu9m1ZNELWwyq0zTAZ4HF8aRUJ/RwZuu5WWp+0pWN3anC9kuGfM5cgliuKmwNG HmZA== X-Gm-Message-State: AOAM531hUD51eMktGyIBuKnlj16O+AELc0j/j5CqqL2RbUCksxJORZNb IGGDUZ53sZZ0GWHbBjUwZeJOzHqXpDyIr4p+JkpNrmbPJlILbw== X-Google-Smtp-Source: ABdhPJyegNsRX07v8vUVpAnPOKp+ymxBmUTBbYlwuxp9kRUTirfbFw+OM+cLUtzCxOX9HFo3hQAJyhzC+wz6wFXLbu4= X-Received: by 2002:a17:903:120e:b0:15e:84d2:4bbb with SMTP id l14-20020a170903120e00b0015e84d24bbbmr64260plh.165.1654097975213; Wed, 01 Jun 2022 08:39:35 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a05:7300:64cc:b0:60:be3a:16c2 with HTTP; Wed, 1 Jun 2022 08:39:34 -0700 (PDT) In-Reply-To: References: <548D3C1F-DFCC-4DCE-AB37-3CA48997AE69@iitbombay.org> From: hiro <23hiro@gmail.com> Date: Wed, 1 Jun 2022 18:39:34 +0300 Message-ID: Subject: Re: [9fans] 9p server to multiply 9p messages? To: 9fans <9fans@9fans.net> Content-Type: text/plain; charset=UTF-8 Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 0d559652-e1c1-11ec-804f-8b9afb2035b5 Archived-At: =?UTF-8?B?PGh0dHBzOi8vOWZhbnMudG9waWNib3guY29tL2dyb3Vwcy85?= =?UTF-8?B?ZmFucy9UNzY5ODU0ZmFmZDJiN2QzNS1NMDk5YTU2ZmViN2M0MDFjYzFkMGIz?= =?UTF-8?B?ZWQ2Pg==?= List-Help: List-Id: "9fans" <9fans.9fans.net> List-Post: List-Software: Topicbox v0 List-Subscribe: Precedence: list Reply-To: 9fans <9fans@9fans.net> Content-Transfer-Encoding: quoted-printable List-Unsubscribe: , Topicbox-Delivery-ID: 2:9fans:437d30aa-c441-11e9-8a57-d036212d11b0:522be890-2105-11eb-b15e-8d699134e1fa:M099a56feb7c401cc1d0b3ed6:1:fdV7nAB3p-bLremhNioHDc0S0Kzk3e0YhvUCiyEofFg In case this is not immediately clear: theoretically preventable 1rtt minimum delays are much less bad than the practically unbounded maximum delays in congested networks. Put in another way: making some few things fast is much more easy than making sure that everything else doesn't get infinitely slow as a result to this. Right now huge streams don't get huge unfair advantages unless the rtt is very small or the parallelism very high On 6/1/22, hiro <23hiro@gmail.com> wrote: > I don't think the reason nobody is doing this is that it's difficult per > se. > > Fcp also achieves parallelism without any changes to 9p. > > And posix fs also share some of our statefulness. > > A file system can have offsets, readahead can help. > > Other synthetic FS need different tricks, but we can exchange some > guarantees that are only needed in seekable files for an optimization > that shall only be done on pipes and streaming access. > > There's some trivial heuristic solutions for this but they are not > generic naturally. > > If one were to do this right, after a few increments one will see that > bandwidth limits are hit, which is a new problem that is much harder > to solve and impossible without even more heuristics classifications > possibly applied by a distributed 9p scheduler (dynamic multi hop > network congestion awareness anybody?) > > On 6/1/22, ron minnich wrote: >> On Tue, May 31, 2022 at 11:29 AM hiro <23hiro@gmail.com> wrote: >>> >>> so virtiofs is not using 9p any more? >>> >>> and with 10 million parallel requests, why shouldn't 9p be able to >>> deliver 10GB/s ?! >>=20 >> Everyone always says this. I used to say it too. >>=20 >> 9p requires a certain degree of ordering -- as Andrey once pointed >> out, it's not productive to close a file, then write it. So there is a >> tricky ordering requirement you need to get right, due to Plan 9 being >> stateful. >>=20 >> The way we use 9p in Plan 9, as a general purpose protocol for >> everything, like devices, requires that each Tread or Twrite occur in >> order, but also requires that each T be retired before the next T is >> issued. devmnt does this. If you don't do this, hardware can get >> confused (e.g. ordering of Twrite followed by Tread followed by Twrite >> needs to be maintained. E.g. you don't want to issue the Tread before >> you know the Twrite happened. E.g. pre-posting 100 Treads to >> /dev/mouse is not a good idea if you suddenly want to do a Twrite in >> the middle of it). >>=20 >> This is why 9p starts to perform poorly in networks with high >> bandwidth*delay products -- if you watch the net traffic, you see each >> T op on fid blocked by the previous Reply (by devmnt). >>=20 >> I never figured out a way to fix this without fixing devmnt -- by >> removing its general nature. >>=20 >> But, more to the point, whether or not 9p should be able to do all >> these parallel requests and get high performance, nobody has yet done >> it. The only numbers ever reported for making high bandhwidth*delay >> networks better were in Floren's thesis, when he added Tstream. >>=20 >> After 20+ years of this discussion, I start to wondering whether it's >> harder than it looks. >>=20 >> ron ------------------------------------------ 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/T769854fafd2b7d35-M099a5= 6feb7c401cc1d0b3ed6 Delivery options: https://9fans.topicbox.com/groups/9fans/subscription