[ruby-core:121207] [Ruby master Bug#21164] Performance Regression using --jit

[ruby-core:121207] [Ruby master Bug#21164] Performance Regression using --jit

[purbug28 (puni ru) via ruby-core]

Issue #21164 has been reported by purbug28 (puni ru).

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164

* Author: purbug28 (puni ru)
* Status: Open
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121208] [Ruby master Bug#21164] Performance Regression using --jit

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #21164 has been updated by k0kubun (Takashi Kokubun).

Status changed from Open to Feedback

Using `time` for benchmarking Ruby involves a lot of noise not relevant to the interpreter/JIT's performance. I suggest using `Benchmark.realtime` or at least `--disable-gems` on the Ruby command.

In my local environment, the results look like this without `--disable-gems`:

```
$ chruby 3.3.7; time ruby -v --jit array.rb
ruby 3.3.7 (2025-01-15 revision be31f993d7) +YJIT [x86_64-linux]
40000000
ruby -v --jit array.rb  1.36s user 0.01s system 99% cpu 1.367 total

$ chruby 3.4.2; time ruby -v --jit array.rb
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
40000000
ruby -v --jit array.rb  0.76s user 0.01s system 99% cpu 0.771 total
```

With `--disable-gems`:

```
$ chruby 3.3.7; time ruby -v --disable-gems --jit array.rb
ruby 3.3.7 (2025-01-15 revision be31f993d7) +YJIT [x86_64-linux]
40000000
ruby -v --disable-gems --jit array.rb  0.77s user 0.01s system 99% cpu 0.773 total

$ chruby 3.4.2; time ruby -v --disable-gems --jit array.rb
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
40000000
ruby -v --disable-gems --jit array.rb  0.74s user 0.00s system 99% cpu 0.746 total
```

You can see enabling rubygems adds a lot of overhead to the execution time of the whole command, which is not relevant to the benchmark script. The performance without `--disable-gems` could potentially be influenced by the number of gems that you installed for the Ruby version, for example.

Anyway, in both cases, I couldn't see any slowdown in my environment.

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164#change-112149

* Author: purbug28 (puni ru)
* Status: Feedback
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121209] [Ruby master Bug#21164] Performance Regression using --jit

[purbug28 (puni ru) via ruby-core]

Issue #21164 has been updated by purbug28 (puni ru).


```ruby
# frozen_string_literal: true

require 'benchmark'
n = 200000
time = Benchmark.realtime do
  c = Array.new(n + 1, 0)
  (1..n).each do |i|
    a = []
    m = 100
    (1..m).each do
      a << i
      c[i] += 1
      a << i / m
      c[i % m] += 1
    end
  end
end
puts time
```
I can see the same slowdown with `--disable-gems` and Benchmark.realtime.
```
> ruby -v --disable-gems --jit array.rb
ruby 3.3.7 (2025-01-15 revision be31f993d7) +YJIT [x86_64-linux]
1.1835691558662802
> ruby -v --disable-gems --jit array.rb
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
1.9472596580162644
```

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164#change-112150

* Author: purbug28 (puni ru)
* Status: Feedback
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121210] [Ruby master Bug#21164] Performance Regression using --jit

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #21164 has been updated by k0kubun (Takashi Kokubun).


Thanks for updating the script. I still can't reproduce your issue. 

```
$ chruby 3.3.7; ruby -v --disable-gems --jit array.rb
ruby 3.3.7 (2025-01-15 revision be31f993d7) +YJIT [x86_64-linux]
0.7517766600003597

$ chruby 3.4.2; ruby -v --disable-gems --jit array.rb
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
0.7428871890006121
```

You somehow need to make the issue reproducible on our end to make this ticket actionable on our end. I think one problem with this script is that this does so many different things that some things may go faster while others may go slower. 

Can you simplify the benchmark script further while still reproducing a significant slowdown in your environment? At the moment, you call a lot of different methods (`Array.new`, `Array#each`, `<<`, `[]`, `+`, `/`, `%`), but it'd be nice to use as fewer methods as possible in the benchmark script. Also, please consider allocating `c = Array.new(n + 1, 0)` outside `Benchmark.realtime`.

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164#change-112151

* Author: purbug28 (puni ru)
* Status: Feedback
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121212] [Ruby master Bug#21164] Performance Regression using --jit

[purbug28 (puni ru) via ruby-core]

Issue #21164 has been updated by purbug28 (puni ru).


Reinstalling Ruby resolved the issue.
Sorry for bothering you.

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164#change-112153

* Author: purbug28 (puni ru)
* Status: Feedback
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121214] [Ruby master Bug#21164] Performance Regression using --jit

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #21164 has been updated by k0kubun (Takashi Kokubun).


Glad to hear that.

Just curious, were those two Rubies installed differently previously? For example, rbenv (ruby-build) is known to install Ruby with `--enable-shared`, which usually degrades the performance by a few %, so installing one with rbenv and the other with something else could be a problem.

----------------------------------------
Bug #21164: Performance Regression using --jit
https://bugs.ruby-lang.org/issues/21164#change-112155

* Author: purbug28 (puni ru)
* Status: Feedback
* ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Ruby 3.4.2 --jit runs slower than no JIT, while Ruby 3.3.7 --jit runs faster.
```ruby
# frozen_string_literal: true
n = 200000
c = Array.new(n + 1, 0)
(1..n).each do |i|
  a = []
  m = 100
  (1..m).each do
    a << i
    c[i] += 1
    a << i / m
    c[i % m] += 1
  end
end
puts c.sum
```
results of /usr/bin/time
ruby 3.4.2
  --jit : 1.98user 0.03system 0:02.02elapsed 99%CPU (0avgtext+0avgdata 19692maxresident)k
  no JIT: 1.77user 0.02system 0:01.80elapsed 99%CPU (0avgtext+0avgdata 18916maxresident)k
ruby 3.3.7
  --jit : 1.20user 0.05system 0:01.26elapsed 99%CPU (0avgtext+0avgdata 23220maxresident)k
  no JIT: 1.81user 0.02system 0:01.83elapsed 99%CPU (0avgtext+0avgdata 22952maxresident)k
  
`ruby -v --jit`
```
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [x86_64-linux]
```




-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/