From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: tuhs-bounces@minnie.tuhs.org X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=ham autolearn_force=no version=3.4.1 Received: from minnie.tuhs.org (minnie.tuhs.org [45.79.103.53]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 4dac86e7 for ; Thu, 28 Jun 2018 20:43:03 +0000 (UTC) Received: by minnie.tuhs.org (Postfix, from userid 112) id 582B5A1B1A; Fri, 29 Jun 2018 06:43:02 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 7CD1AA1B0A; Fri, 29 Jun 2018 06:42:51 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=bsdimp-com.20150623.gappssmtp.com header.i=@bsdimp-com.20150623.gappssmtp.com header.b=D5BTUx2t; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id C2866A1B0A; Fri, 29 Jun 2018 06:42:49 +1000 (AEST) Received: from mail-it0-f50.google.com (mail-it0-f50.google.com [209.85.214.50]) by minnie.tuhs.org (Postfix) with ESMTPS id 2F668A1815 for ; Fri, 29 Jun 2018 06:42:49 +1000 (AEST) Received: by mail-it0-f50.google.com with SMTP id u4-v6so14704881itg.0 for ; Thu, 28 Jun 2018 13:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=9HLu8EWgYxZU44/XgGWJdflb+2UWHWS7d05EGKOBtic=; b=D5BTUx2tYNoxuPv5+CLDRaQ5MXteeH7TC6W7aWSGQG8GlaZOd4PV4VpNy0YmStb+iA i52ENGElZc1+LKFHc8IniF0xApgR+acf83Zfx69ZQ9Onet3/Hq1NKiI5VUmC+uaZAIoH wKC03WjufxmZi0flQqb7mbRfnlJ6SMbSGB1kAmY9RSE1yi4Tm9LtdRjDgkENwjL8Ux7F qI6S68tyXXsnjEjenUg064AoaOBTPLtVIwzMR8IL/GSnaOgeAavHNwbLXiXQH6SnbQ0B Bt87VUhpswwc3cBN5NuFuaxXi+bLl0eu1v+DRZR3b9xj3c2WC4GP9Y6si/RzthM6wayp /mVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=9HLu8EWgYxZU44/XgGWJdflb+2UWHWS7d05EGKOBtic=; b=YmTrq19QtIo7hXx12Y0M6xMy9w2khVJl3qjO9113O4FLskMNhztAYijtisguydBkEF T8/e+CWzD+eiAcDIY6QmyBXqj9Ha/RyGsXc0A4g80rL2qUFzt2J6mwG9216pIkvKqj86 KtNcWxBf08ft1RdkOwSwX81zUpZKcz4dIwG3aYUfvxTEseWzcOjPJzemPc8IeUjZ+7Y5 wKWADUjPovX7bXRhm74ijzFmJiqaR0FNOHA/2mFAAbAx783rH0qB66/e/+A+X5EF7mpO H7Mq5hzN0/3FIdOozc8ngeuPOqN9Ig4NaHM+QT6bo1LyJBFfPsW+tw2NSMRXvwxI5EA8 0LlA== X-Gm-Message-State: APt69E00g4OPEihChDjfXMIsqELMzQXK8TENmRVaqsMh0l4gTAKbJkxS K4ajxtX3CVTYvVjL4VdETstaxu+5REP3eSs6NqhLFQ== X-Google-Smtp-Source: AAOMgpedfYx6YKvQVBwmJ8gAlheV5w+k7M1LtH14KbA00e2NPwDRnIthGAim/1gntIo7ckpvQZ6m0uu7hCUBGc4Qtqs= X-Received: by 2002:a24:64ce:: with SMTP id t197-v6mr9658133itc.36.1530218568320; Thu, 28 Jun 2018 13:42:48 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a4f:5945:0:0:0:0:0 with HTTP; Thu, 28 Jun 2018 13:42:47 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <20180628154246.3a1ce74a@jabberwock.cb.piermont.com> References: <81277CC3-3C4A-49B8-8720-CFAD22BB28F8@bitblocks.com> <20180628141538.GB663@thunk.org> <20180628104329.754d2c19@jabberwock.cb.piermont.com> <20180628145609.GD21688@mcvoy.com> <20180628154246.3a1ce74a@jabberwock.cb.piermont.com> From: Warner Losh Date: Thu, 28 Jun 2018 14:42:47 -0600 X-Google-Sender-Auth: WZp16VJp24cKm2FoUMNJIeiGSmY Message-ID: To: "Perry E. Metzger" Content-Type: multipart/alternative; boundary="00000000000031a4f1056fb9c688" Subject: Re: [TUHS] PDP-11 legacy, C, and modern architectures X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: TUHS main list Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --00000000000031a4f1056fb9c688 Content-Type: text/plain; charset="UTF-8" On Thu, Jun 28, 2018 at 1:42 PM, Perry E. Metzger wrote: > On Thu, 28 Jun 2018 07:56:09 -0700 Larry McVoy wrote: > > > Huge numbers of wimpy cores is the model already dominating the > > > world. > > > > Got a source that backs up that claim? I was recently dancing with > > Netflix and they don't match your claim, nor do the other content > > delivery networks, they want every cycle they can get. > > Netflix has how many machines? We generally say we have tens of thousands of machines deployed worldwide in our CDN. We don't give out specific numbers though. > I'd say in general that principle > holds: this is the age of huge distributed computation systems, the > most you can pay for a single core before it tops out is in the > hundreds of dollars, not in the millions like it used to be. The high > end isn't very high up, and we scale by adding boxes and cores, not > by getting single CPUs that are unusually fast. > > Taking the other way of looking at it, from what I understand, > CDN boxes are about I/O and not CPU, though I could be wrong. I can > ask some of the Netflix people, a former report of mine is one of the > people behind their front end cache boxes and we keep in touch. I can tell you it's about both. We recently started encrypting all traffic, which requires a crapton of CPU. Plus, we're doing sophisticated network flow modeling to reduce congestion, which takes CPU. On our 100G boxes, which we get in the low 90's encrypted, we have some spare CPU, but almost no space memory bandwidth and our PCI lanes are full of either 100G network traffic or 4-6 NVMe drives delivering content up at about 85-90Gbps. Most of our other boxes are the same, with the exception of the 'storage' tier boxes. Those we're definitely hard disk I/O bound. Warner --00000000000031a4f1056fb9c688 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Jun 28, 2018 at 1:42 PM, Perry E. Metzger <= ;perry@piermont.com= > wrote:
O= n Thu, 28 Jun 2018 07:56:09 -0700 Larry McVoy <lm@mcvoy.com> wrote:
> > Huge numbers of wimpy cores is the model already dominating the > > world.=C2=A0 =C2=A0
>
> Got a source that backs up that claim?=C2=A0 I was recently dancing wi= th
> Netflix and they don't match your claim, nor do the other content<= br> > delivery networks, they want every cycle they can get.

Netflix has how many machines?

We ge= nerally say we have tens of thousands of machines deployed worldwide in our= CDN. We don't give out specific numbers though.
=C2=A0
=
I'd say in general that principle
holds: this is the age of huge distributed computation systems, the
most you can pay for a single core before it tops out is in the
hundreds of dollars, not in the millions like it used to be. The high
end isn't very high up, and we scale by adding boxes and cores, not
by getting single CPUs that are unusually fast.

Taking the other way of looking at it, from what I understand,
CDN boxes are about I/O and not CPU, though I could be wrong. I can
ask some of the Netflix people, a former report of mine is one of the
people behind their front end cache boxes and we keep in touch.

I can tell you it's about both. We recently starte= d encrypting all traffic, which requires a crapton of CPU. Plus, we're = doing sophisticated network flow modeling to reduce congestion, which takes= CPU. On our 100G boxes, which we get in the low 90's encrypted, we hav= e some spare CPU, but almost no space memory bandwidth and our PCI lanes ar= e full of either 100G network traffic or 4-6 NVMe drives delivering content= up at about 85-90Gbps.

Most of our other boxes ar= e the same, with the exception of the 'storage' tier boxes. Those w= e're definitely hard disk I/O bound.

Warner
--00000000000031a4f1056fb9c688--