[ruby-core:119984] [Ruby master Feature#20861] Add an environment variable for tuning the default thread quantum

["tenderlovemaking (Aaron Patterson) via ruby-core" <ruby-core@ml.ruby-lang.org>]

Issue #20861 has been updated by tenderlovemaking (Aaron Patterson).


At RubyConf we discussed this a bit and it seemed like the feature is fine, but maybe the name isn't good enough.

How about `RUBY_THREAD_SWITCH_INTERVAL`?

I like having "MS" in the name so that the units are obvious, but I really don't have a strong opinion.

----------------------------------------
Feature #20861: Add an environment variable for tuning the default thread quantum
https://bugs.ruby-lang.org/issues/20861#change-110724

* Author: tenderlovemaking (Aaron Patterson)
* Status: Open
----------------------------------------
The default thread quantum is currently [hard coded at 100ms](https://github.com/ruby/ruby/blob/c7708d22c33040a74ea7ac683bf7407d3759edfe/thread_pthread.c#L323).  This can impact multithreaded systems that are trying to process Ruby level CPU bound work at the same time as IO work.

I would like to add an environment variable `RUBY_THREAD_DEFAULT_QUANTUM_MS` that allows users to specify the default thread quantum (in milliseconds) via an environment variable.  It defaults to our current default of 100ms.  I've submitted the patch [here](https://github.com/ruby/ruby/pull/11981).

Here is a Ruby program to demonstrate the problem:

```ruby
def measure
  x = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  yield
  Process.clock_gettime(Process::CLOCK_MONOTONIC) - x
end

def fib(n)
  if n < 2
    n
  else
    fib(n-2) + fib(n-1)
  end
end

# find fib that takes ~500ms
fib_i = 50.times.find { |i| measure { fib(i) } >= 0.05 }
sleep_i = measure { fib(fib_i) }

threads = [
  Thread.new {
    100.times {
      sleep(sleep_i)
      # sometimes stalled waiting for fib's quantum to finish
    }
    puts "done 1"
  },
  Thread.new { 100.times { fib(fib_i) }; puts "done 2" },
]

# We expect the total time to be about 100 * sleep_i (~5 seconds) because
# theoretically the sleep thread could be done nearly completely in parallel to
# the fib thread.
#
# But because the `sleep` thread is iterating over the sleep call, it must wait
# for the `fib` thread to complete its quantum, before it can start the next iteration.
#
# This means each sleep iteration could take up to `sleep_i + 100ms`
#
# We're calling that stalled time "waste"
total = measure { threads.each(&:join) }
waste = total - (sleep_i * 100)
p TOTAL: total, WASTE: waste
```

The program has two threads.  One thread is using CPU time by computing `fib` in a loop.  The other thread is simulating IO time by calling `sleep` in a loop.  When the `sleep` call completes, it can stall, waiting for the quantum in the fib thread to expire.  That means that each iteration on sleep can actually take `sleep time + thread quantum`, or in this case ~600ms when we expected it to only take ~500ms.

Ideally, the above program would take `500ms * 100` since all `sleep` calls should be able to execute in parallel with the `fib` calls.  Of course this isn't true because the sleep thread must acquire the GVL before it can continue the next iteration, so there will always be _some_ overhead.  This feature is for allowing people to tune that overhead.

If we run this program with the default quantum the output looks like this:

```
$ ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T14:49:50Z quantum-computing c7708d22c3) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 12.672821999993175, WASTE: 4.960721996147186}
```

The output shows that our program spent about 5 seconds stalled, waiting to acquire the GVL.

With this patch we can lower the default quantum, and the output is like this:

```
$ RUBY_THREAD_DEFAULT_QUANTUM_MS=10 ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T22:06:35Z quantum-computing 087500643d) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 8.898526000091806, WASTE: 1.4168260043952614}
```

Specifying the ENV to change the quantum to 10ms lowered our waste in the program to ~1.4 seconds.

It's common for web applications to do mixed CPU and IO bound tasks in threads (see the Puma webserver), so it would be great if there was a way to customize the thread quantum depending on your application's workload.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/