Re: [TUHS] SunOS code?

["Theodore Y. Ts'o" <tytso@mit.edu>]

On Sat, Sep 01, 2018 at 10:05:06PM -0700, Kevin Bowling wrote:
> 
> Sorry this is just bogus about being weak compared to Solaris.  Are
> you looking back with rosy glasses or have you scanned the code in the
> past couple years?  I have and there is nothing particularly special
> about Solaris internals here or elsewhere.

I haven't looked at Solaris code; I had just *assumed* that if they
were selling million dollar E10k's, they would have had NUMA support
at *least* as good as SGI's Irix.  And it would have been an excuse
for their pathetic performance on UP and 2-4 SMP systems.

> Keep in mind IBM wants to sell RockHoppers and E980s (4 drawers, 16
> sockets, 768 threads) for dedicated Linux use which have similar
> north/south and east/west off chip networks.  They have a lot of very
> talented people on the firmware, kernel, compilers to make these
> things work fast, including Paul.
> ...
> Where you start going beyond Linux-like NUMA IMO is when you get
> Irix-like features of page copying, migration, and multiple advanced
> placement policies.

One thing to consider is that IBM really only cared about optimizing
hardware for DB2, Oracle, and Webshpere.  That's one of the reason why
you didn't see much in the way of innovative file system work, ala
ZFS.  There was no business justification for pouring 100+ engineer
years to develop a next-generation file systesm --- and they had
already done that once already for GPFS, a cluster file system.  As
far as local disk file system was concerned, the only real business
value it had was to serve as a program loader for DB2 and Websphere.  :-)

(I'm exagerating a little for effect, but *only* a little.)

So as far as NUMA was concerned, there was almost certainly not have
been much perceived business value in having sophisticated
auto-migration for arbitrary workloads in the kernel.  Something basic
which was good enough for Oracle, DB2, etc., was all that would be
needed.  (And if you needed to hire consultants from IBM Global
Services to mind-meld with the configuration documentation in order to
get the best out of your Rockhopper.... well, shucks, darn.  :-)

At IBM the business people really did make the funding decisions of
what to work on.  ZFS could have never happened at IBM because no one
would have thought that a even a tiny number of IBM's current or
potential customer base would abandon AIX or Linux and switch to
Solaris, or buy Sun hardware instead of IBM hardware --- just for the
sake of ZFS.  And that's how decision-makers at IBM really thought.
(And to be fair to those decision-makers, IBM is still in business as
a free-standing business --- and Sun is not.)

  			     	     	- Ted