[ruby-core:118823] [Ruby master Bug#20670] fork deadlocks in child process due to timer thread

[ruby-core:118823] [Ruby master Bug#20670] fork deadlocks in child process due to timer thread

[jhawthorn (John Hawthorn) via ruby-core]

Issue #20670 has been reported by jhawthorn (John Hawthorn).

----------------------------------------
Bug #20670: fork deadlocks in child process due to timer thread
https://bugs.ruby-lang.org/issues/20670

* Author: jhawthorn (John Hawthorn)
* Status: Open
* ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
We've been seeing an occasional failure in the Rails CI related to a test which forks and I managed to reduce it to the following reproduction.

``` ruby
Thread.new do
  loop { sleep 0.0001 }
end

1000.times do
  pid = fork{}
  Process.waitpid(pid)
rescue Exception
  Process.kill(:KILL, pid)
  raise
end
```

This hangs on Ruby 3.3 and HEAD (_fairly_ reliably), but completes always on Ruby 3.2

In a debugger it seems like the timer thread acquires `vm->ractor.sched.lock` in the parent process just as the process is forking. The child process then ends up stuck inside of `thread_sched_atfork` when trying to acquire the same lock.

I've opened https://github.com/ruby/ruby/pull/11356 with a fix



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:118824] [Ruby master Bug#20670] fork deadlocks in child process due to timer thread

[ko1 (Koichi Sasada) via ruby-core]

Issue #20670 has been updated by ko1 (Koichi Sasada).


Thank you!


----------------------------------------
Bug #20670: fork deadlocks in child process due to timer thread
https://bugs.ruby-lang.org/issues/20670#change-109391

* Author: jhawthorn (John Hawthorn)
* Status: Open
* ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
We've been seeing an occasional failure in the Rails CI related to a test which forks and I managed to reduce it to the following reproduction.

``` ruby
Thread.new do
  loop { sleep 0.0001 }
end

1000.times do
  pid = fork{}
  Process.waitpid(pid)
rescue Exception
  Process.kill(:KILL, pid)
  raise
end
```

This hangs on Ruby 3.3 and HEAD (_fairly_ reliably), but completes always on Ruby 3.2

In a debugger it seems like the timer thread acquires `vm->ractor.sched.lock` in the parent process just as the process is forking. The child process then ends up stuck inside of `thread_sched_atfork` when trying to acquire the same lock.

I've opened https://github.com/ruby/ruby/pull/11356 with a fix



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:118843] [Ruby master Bug#20670] fork deadlocks in child process due to timer thread

[jhawthorn (John Hawthorn) via ruby-core]

Issue #20670 has been updated by jhawthorn (John Hawthorn).

Status changed from Open to Closed
Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED

Fixed by https://github.com/ruby/ruby/commit/87a85550edd786665e081b355c6af62c4854b1d7

----------------------------------------
Bug #20670: fork deadlocks in child process due to timer thread
https://bugs.ruby-lang.org/issues/20670#change-109410

* Author: jhawthorn (John Hawthorn)
* Status: Closed
* ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
* Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED
----------------------------------------
We've been seeing an occasional failure in the Rails CI related to a test which forks and I managed to reduce it to the following reproduction.

``` ruby
Thread.new do
  loop { sleep 0.0001 }
end

1000.times do
  pid = fork{}
  Process.waitpid(pid)
rescue Exception
  Process.kill(:KILL, pid)
  raise
end
```

This hangs on Ruby 3.3 and HEAD (_fairly_ reliably), but completes always on Ruby 3.2

In a debugger it seems like the timer thread acquires `vm->ractor.sched.lock` in the parent process just as the process is forking. The child process then ends up stuck inside of `thread_sched_atfork` when trying to acquire the same lock.

I've opened https://github.com/ruby/ruby/pull/11356 with a fix



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119001] [Ruby master Bug#20670] fork deadlocks in child process due to timer thread

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #20670 has been updated by k0kubun (Takashi Kokubun).

Backport changed from 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED to 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE

ruby_3_3 commit:66312ad913d67bfd3c2c83b174eabf537f5def84.

----------------------------------------
Bug #20670: fork deadlocks in child process due to timer thread
https://bugs.ruby-lang.org/issues/20670#change-109576

* Author: jhawthorn (John Hawthorn)
* Status: Closed
* ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
* Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE
----------------------------------------
We've been seeing an occasional failure in the Rails CI related to a test which forks and I managed to reduce it to the following reproduction.

``` ruby
Thread.new do
  loop { sleep 0.0001 }
end

1000.times do
  pid = fork{}
  Process.waitpid(pid)
rescue Exception
  Process.kill(:KILL, pid)
  raise
end
```

This hangs on Ruby 3.3 and HEAD (_fairly_ reliably), but completes always on Ruby 3.2

In a debugger it seems like the timer thread acquires `vm->ractor.sched.lock` in the parent process just as the process is forking. The child process then ends up stuck inside of `thread_sched_atfork` when trying to acquire the same lock.

I've opened https://github.com/ruby/ruby/pull/11356 with a fix



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121438] [Ruby Bug#20670] fork deadlocks in child process due to timer thread

[ioquatix (Samuel Williams) via ruby-core]

Issue #20670 has been updated by ioquatix (Samuel Williams).


IIUC, the tests introduced here are now failing again on `master`:

```
btest-ruby
   Fstderr output is not empty
     bootstraptest.test_fork.rb_78_287.rb:16:in 'block in <main>': failed (RuntimeError)
             from <internal:numeric>:257:in 'Integer#times'
             from bootstraptest.test_fork.rb_78_287.rb:10:in '<main>'
  #287 test_fork.rb:78: 
       def now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
     
       Thread.new do
         loop { sleep 0.0001 }
       end
     
       10.times do
         pid = fork{ exit!(0) }
         deadline = now + 10
         until Process.waitpid(pid, Process::WNOHANG)
           if now > deadline
             Process.kill(:KILL, pid)
             raise "failed"
           end
           sleep 0.001
         end
       rescue NotImplementedError
       end
       :ok
    #=> "" (expected "ok")  [Bug #20670]
  FAIL 1/1910 tests failed
  make: *** [uncommon.mk:894: yes-btest-ruby] Error 1
  /tmp/tmp.xWdROBpKaN /tmp/tmp.xWdROBpKaN /github/workspace/src /github/workspace
```

https://github.com/ruby/ruby/actions/runs/14010663767/job/39230065916#step:9:1815

Maybe there has been a regression?

I'm just checking if it's a timeout issue: https://github.com/ruby/ruby/pull/12962 however I suspect it's a deadlock.

----------------------------------------
Bug #20670: fork deadlocks in child process due to timer thread
https://bugs.ruby-lang.org/issues/20670#change-112418

* Author: jhawthorn (John Hawthorn)
* Status: Closed
* ruby -v: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [x86_64-linux]
* Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE
----------------------------------------
We've been seeing an occasional failure in the Rails CI related to a test which forks and I managed to reduce it to the following reproduction.

``` ruby
Thread.new do
  loop { sleep 0.0001 }
end

1000.times do
  pid = fork{}
  Process.waitpid(pid)
rescue Exception
  Process.kill(:KILL, pid)
  raise
end
```

This hangs on Ruby 3.3 and HEAD (_fairly_ reliably), but completes always on Ruby 3.2

In a debugger it seems like the timer thread acquires `vm->ractor.sched.lock` in the parent process just as the process is forking. The child process then ends up stuck inside of `thread_sched_atfork` when trying to acquire the same lock.

I've opened https://github.com/ruby/ruby/pull/11356 with a fix



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/