From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 19532 invoked from network); 3 May 2022 05:04:13 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 3 May 2022 05:04:13 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 090979D495; Tue, 3 May 2022 15:04:13 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 9F7789D47F; Tue, 3 May 2022 15:02:46 +1000 (AEST) Received: by minnie.tuhs.org (Postfix, from userid 112) id C65BB9D477; Tue, 3 May 2022 15:01:14 +1000 (AEST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by minnie.tuhs.org (Postfix) with ESMTPS id A4A929D472 for ; Tue, 3 May 2022 15:01:13 +1000 (AEST) Received: from penguin.thunk.org (cn-8-34-116-185.paclightwave.com [8.34.116.185] (may be forged)) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 24351865000700 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 3 May 2022 01:01:09 -0400 Received: by penguin.thunk.org (Postfix, from userid 1000) id AE64441889; Mon, 2 May 2022 22:01:03 -0700 (PDT) Date: Mon, 2 May 2022 22:01:03 -0700 From: tytso To: Warner Losh Message-ID: References: <57977CE7-DDCC-4861-BBD2-843B9B9F51C2@ronnatalie.com> <202205020242.2422g30m074857@ultimate.com> <2815597f-e1f2-498f-b0c3-763952ac734e@www.fastmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [TUHS] First Unix-like OSes not derived from AT&T code? X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: TUHS main list Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" On Mon, May 02, 2022 at 02:31:00PM -0600, Warner Losh wrote: > On Mon, May 2, 2022 at 9:41 AM Clem Cole wrote: > > FWIW: I know that at least 3 people on the OpenSSI team were telling me > > they were told to go away. I don't know where they were told to go away; I can just state that the patches were never sent to LKML, and from a search using lore.kernel.org, I don't see any evidence that they were told to "go away" on the Linux Kernel Mailing List. Could they have been told to "go away" by someone, either with someone "official" or "non-official", on some random mailing list, or at some random bar at some random conference? Sure. It's impossible to say. > I know from wearing my FreeBSD hat that random people on mailing lists > often say 'nope' and people go away not realizing they aren't the abitors > of what gets into FreeBSD. We lost a lot of good contributions because of > delays created by scenarios like this... Yep. And sometimes, even if they are someone official, if they don't necessarily explain the patches well, and/or never send the patches for review on the mailing list, it could be that there was a miscommunication regarding how the patches were described, such that a "no" that happened at a conversation at some random bar at some random Usenix conference might have been a "yes" if there were patches sent to be reviewed on the mailing list. > I also know that getting changes into Linux suffers from this and for a > long time (especially pre-git) was almost impossible unless you knew > someone and were on good terms with them. Something which definitely happens is the fear of "drive by contributions". So for example, Clem tells the story of people being hesitant of accepting RDMA / IB patches. I very much doubt it was because people were chasing the Desktop. There were certainly people in the Linux kernel who were chasing the Desktop, but those tended not to be the "gate keepers" for the kernel. Linux kernel developers might use the desktop, but there really was very few "desktop" kernel features. If I had to guess, the main concern was that some random developer would try to get the code upstream --- perhaps because a system integrator like IBM or HP would have something in the procurement contract requiring that the device driver be "upstream" --- but then once the code made it upstream, the developer would disappear, never to be heard from again, and the Linux Kernel developers would be stuck having to support it forever. (Worse, in the early days of IB, IB was $$$, and most kernel developers didn't even have access to IB in their home systems.) So it's helpful to have a company to have multiple engineers, all contributing changes, hopefully to adjacent parts of the kernel other than the specific subsystem which they are trying to shove into the kernel, to demonstrate that they are going to be there for the long haul, and not just try to "drive by" shoehorn code into the kernel, only to be never heard from again. For example, just recently someone from a particular tried to get an NTFS implementation "upstream" into the Linux kernel. They sold a "feature-full" version of that file system for $$$, and there was some suspicion in some quarters that they were only trying to get a stripped-down version of their file system into the Linux kernel for marketing reasons. There were some, including yours truly, who pointed out that they hadn't open sourced userspace utilities, and that the file system regression test when run on their file system was failing tests right and left, and hence wasn't ready for prime-time. They pushed hard, and ultimately, Linus decided to accept their code contribution, because the alternative in-tree file system was pretty crappy, and the belief was that new one was better. Immediately after the merge window closed, the developer went silent, and stopped responding to e-mails. Which left folks debating whether we should remove the code contribution from the tree before users started depending on it, because something worse than one unmaintained, crappy file system in the tree that some users might be trying to use wouldbe *two* unmaintained, crappy file systems in the tree.... > Much has been done to improve things in the last > 20 years, but for a while things were truly awful for someone without a > huge reputation to get anything non-trivial into Linux. That's true, but again, part of that is because of the "drive by contribution" problem. One of the ways which we've tried to solve this problem for device drivers is to have a "staging" part of the tree where new code which is "on probation" can live, and if it turns out that the developers disappear, code in the "staging tree" is documented to be more easily removed uncerimonously if it looks like the code has become abandonware. This doesn't work as well if there needs to be massive code complexification into the core parts of the kernel, where large parts of the kernel needs to be changed, perhaps to add (for example) VPROC support. It's even worse if such "powerful" changes ends up slightly slowing down code which is in the hotpath. We have in the past tried to put file systems into the staging tree. But for example, there was pain when we ultimately decided that Lustre needed to be removed from the Linux tree, because the people who tried to get Lustre into the upstream kernel wasn't actually *improving* that was in the upstream kernel, but instead was working on shipping product in distro kernels: https://www.linuxjournal.com/content/lustre-filesystem-dropped-linux-418-kernel So the "companies / communites" just trying to throw code of varying quality over the wall and not providing a commitment to continuous maintenance and improvement of said code is a real one. And if you wonder why sometimes the Linux kernel community can be a bit "cold" about accepting new code, that very often can be why. Open source is not just about the code, it's about the development community. And if the community hasn't been integrated into the Linux kernel community first, it may be that an important prerequisite step is missing. Cheers, - Ted