From: "jhawthorn (John Hawthorn) via ruby-core" <ruby-core@ml.ruby-lang.org>
To: ruby-core@ml.ruby-lang.org
Cc: "jhawthorn (John Hawthorn)" <noreply@ruby-lang.org>
Subject: [ruby-core:119697] [Ruby master Feature#20861] Add an environment variable for tuning the default thread quantum
Date: Mon, 04 Nov 2024 04:24:42 +0000 (UTC) [thread overview]
Message-ID: <redmine.journal-110357.20241104042441.73@ruby-lang.org> (raw)
In-Reply-To: <redmine.issue-20861.20241101223432.73@ruby-lang.org>
Issue #20861 has been updated by jhawthorn (John Hawthorn).
I don't think we should expose the quantum per-thread inside Ruby. I worry it will prevent future improvements. The fact that Thread scheduling and priorities are currently done by giving a shorter/longer quantum is an implementation detail that could (and I'd like to) change. Thread `priority` is good *because* it is more abstract, which gives flexibility on implementation.
Similarly I don't love `Thread.default_quantum=` as it's implementation specific - I wouldn't expect it to make sense for JRuby/TruffleRuby for example, and it's possible CRuby could have a different implementation in the future. Though I don't feel as strongly.
I think an environment variable is best as it's very similar to how we allow tuning the garbage collector via environment variables. In the future if end up being able to remove this, that's much easier/safer with an environment variable.
----------------------------------------
Feature #20861: Add an environment variable for tuning the default thread quantum
https://bugs.ruby-lang.org/issues/20861#change-110357
* Author: tenderlovemaking (Aaron Patterson)
* Status: Open
----------------------------------------
The default thread quantum is currently [hard coded at 100ms](https://github.com/ruby/ruby/blob/c7708d22c33040a74ea7ac683bf7407d3759edfe/thread_pthread.c#L323). This can impact multithreaded systems that are trying to process Ruby level CPU bound work at the same time as IO work.
I would like to add an environment variable `RUBY_THREAD_DEFAULT_QUANTUM_MS` that allows users to specify the default thread quantum (in milliseconds) via an environment variable. It defaults to our current default of 100ms. I've submitted the patch [here](https://github.com/ruby/ruby/pull/11981).
Here is a Ruby program to demonstrate the problem:
```ruby
def measure
x = Process.clock_gettime(Process::CLOCK_MONOTONIC)
yield
Process.clock_gettime(Process::CLOCK_MONOTONIC) - x
end
def fib(n)
if n < 2
n
else
fib(n-2) + fib(n-1)
end
end
# find fib that takes ~500ms
fib_i = 50.times.find { |i| measure { fib(i) } >= 0.05 }
sleep_i = measure { fib(fib_i) }
threads = [
Thread.new {
100.times {
sleep(sleep_i)
# sometimes stalled waiting for fib's quantum to finish
}
puts "done 1"
},
Thread.new { 100.times { fib(fib_i) }; puts "done 2" },
]
# We expect the total time to be about 100 * sleep_i (~5 seconds) because
# theoretically the sleep thread could be done nearly completely in parallel to
# the fib thread.
#
# But because the `sleep` thread is iterating over the sleep call, it must wait
# for the `fib` thread to complete its quantum, before it can start the next iteration.
#
# This means each sleep iteration could take up to `sleep_i + 100ms`
#
# We're calling that stalled time "waste"
total = measure { threads.each(&:join) }
waste = total - (sleep_i * 100)
p TOTAL: total, WASTE: waste
```
The program has two threads. One thread is using CPU time by computing `fib` in a loop. The other thread is simulating IO time by calling `sleep` in a loop. When the `sleep` call completes, it can stall, waiting for the quantum in the fib thread to expire. That means that each iteration on sleep can actually take `sleep time + thread quantum`, or in this case ~600ms when we expected it to only take ~500ms.
Ideally, the above program would take `500ms * 100` since all `sleep` calls should be able to execute in parallel with the `fib` calls. Of course this isn't true because the sleep thread must acquire the GVL before it can continue the next iteration, so there will always be _some_ overhead. This feature is for allowing people to tune that overhead.
If we run this program with the default quantum the output looks like this:
```
$ ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T14:49:50Z quantum-computing c7708d22c3) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 12.672821999993175, WASTE: 4.960721996147186}
```
The output shows that our program spent about 5 seconds stalled, waiting to acquire the GVL.
With this patch we can lower the default quantum, and the output is like this:
```
$ RUBY_THREAD_DEFAULT_QUANTUM_MS=10 ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T22:06:35Z quantum-computing 087500643d) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 8.898526000091806, WASTE: 1.4168260043952614}
```
Specifying the ENV to change the quantum to 10ms lowered our waste in the program to ~1.4 seconds.
It's common for web applications to do mixed CPU and IO bound tasks in threads (see the Puma webserver), so it would be great if there was a way to customize the thread quantum depending on your application's workload.
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/
next prev parent reply other threads:[~2024-11-04 4:25 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-01 22:34 [ruby-core:119683] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-01 22:44 ` [ruby-core:119684] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-02 1:33 ` [ruby-core:119686] " ioquatix (Samuel Williams) via ruby-core
2024-11-02 3:03 ` [ruby-core:119687] " nobu (Nobuyoshi Nakada) via ruby-core
2024-11-02 6:43 ` [ruby-core:119688] " byroot (Jean Boussier) via ruby-core
2024-11-02 19:58 ` [ruby-core:119689] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-02 20:02 ` [ruby-core:119690] " byroot (Jean Boussier) via ruby-core
2024-11-02 20:35 ` [ruby-core:119691] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-04 4:24 ` jhawthorn (John Hawthorn) via ruby-core [this message]
2024-11-04 8:11 ` [ruby-core:119698] " byroot (Jean Boussier) via ruby-core
2024-11-05 7:18 ` [ruby-core:119720] " ivoanjo (Ivo Anjo) via ruby-core
2024-11-05 7:40 ` [ruby-core:119721] " ko1 (Koichi Sasada) via ruby-core
2024-11-05 7:53 ` [ruby-core:119723] " ivoanjo (Ivo Anjo) via ruby-core
2024-11-14 17:23 ` [ruby-core:119932] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-14 18:58 ` [ruby-core:119935] " ivoanjo (Ivo Anjo) via ruby-core
2024-11-21 20:38 ` [ruby-core:119984] " tenderlovemaking (Aaron Patterson) via ruby-core
2024-11-21 20:50 ` [ruby-core:119986] " jhawthorn (John Hawthorn) via ruby-core
2024-11-25 2:02 ` [ruby-core:120000] " Dan0042 (Daniel DeLorme) via ruby-core
2024-12-10 4:54 ` [ruby-core:120146] " ko1 (Koichi Sasada) via ruby-core
2024-12-12 7:21 ` [ruby-core:120201] " matz (Yukihiro Matsumoto) via ruby-core
2025-01-03 20:08 ` [ruby-core:120471] " luke-gru (Luke Gruber) via ruby-core
2025-01-03 20:23 ` [ruby-core:120473] " luke-gru (Luke Gruber) via ruby-core
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=redmine.journal-110357.20241104042441.73@ruby-lang.org \
--to=ruby-core@ml.ruby-lang.org \
--cc=noreply@ruby-lang.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).