[ruby-core:121215] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ruby-core:121215] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been reported by ioquatix (Samuel Williams).

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166

* Author: ioquatix (Samuel Williams)
* Status: Open
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an IOError (stream closed in another thread). For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

This PR introduces some new functions:

- `VALUE rb_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` for wrapping user IO operations so they can be interrupted.
- `IO#interruptable_operation(&block)` the same as above.
- `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` for interrupting a specific fiber on the fiber scheduler.

`rb_notify_fd_close` is modified so that it is fiber scheduler aware and uses `rb_fiber_scheduler_fiber_interrupt` to interrupt a fiber. In addition, we also change the internal `struct waiting_fd` to track the `rb_execution_context_t` rather than just the `rb_thread_t` instance, so that we can correctly wake up either the waiting thread or fiber.

The public interface `rb_io_interruptible_operation` and `IO#interruptible_operation` are introduced so that the scheduler implementation can wrap IO operations that should be interruptible, e.g.

```ruby
Fiber.schedule do
  io.interruptible_operation do
    io.wait_readable
  end
end

# Will interrupt above fiber:
io.close
```

See <https://github.com/ruby/ruby/pull/12585> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121219] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[luke-gru (Luke Gruber) via ruby-core]

Issue #21166 has been updated by luke-gru (Luke Gruber).


I'm not knowledgeable when it comes to the fiber scheduler, but why add this new method `io.interruptible_operation` when you can accomplish the same thing by allowing the io operation to be interrupted by a close in every case like the regular thread scheduler does? Are there cases where you don't want the IO operation to be interrupted be a close?

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112161

* Author: ioquatix (Samuel Williams)
* Status: Open
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an IOError (stream closed in another thread). For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

This PR introduces some new functions:

- `VALUE rb_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` for wrapping user IO operations so they can be interrupted.
- `IO#interruptable_operation(&block)` the same as above.
- `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` for interrupting a specific fiber on the fiber scheduler.

`rb_notify_fd_close` is modified so that it is fiber scheduler aware and uses `rb_fiber_scheduler_fiber_interrupt` to interrupt a fiber. In addition, we also change the internal `struct waiting_fd` to track the `rb_execution_context_t` rather than just the `rb_thread_t` instance, so that we can correctly wake up either the waiting thread or fiber.

The public interface `rb_io_interruptible_operation` and `IO#interruptible_operation` are introduced so that the scheduler implementation can wrap IO operations that should be interruptible, e.g.

```ruby
Fiber.schedule do
  io.interruptible_operation do
    io.wait_readable
  end
end

# Will interrupt above fiber:
io.close
```

See <https://github.com/ruby/ruby/pull/12585> for the proposed implementation and <https://github.com/socketry/io-event/pull/130> for example of how `io-event` gem uses both the C and Ruby interfaces.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121220] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


> when you can accomplish the same thing by allowing the io operation to be interrupted by a close in every case like the regular thread scheduler does

Because the point at which IO operations become interruptible is not the entire operation, but usually the point at where the fiber yields back to the event loop, e.g. <https://github.com/socketry/io-event/pull/130/files#diff-3e7a68b220a9360ead1d2e7a5ec23e7d36de591eab138721efdc61b565fc5194R552> - adding interrupts around the entire operation is unlikely to be safe nor desirable.

Maybe you can propose in more detail how your suggestion would work. I suppose you are suggesting to wrap the scheduler code in `rb_fiber_scheduler_io_wait` and so on?

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112162

* Author: ioquatix (Samuel Williams)
* Status: Open
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an IOError (stream closed in another thread). For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

This PR introduces some new functions:

- `VALUE rb_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` for wrapping user IO operations so they can be interrupted.
- `IO#interruptable_operation(&block)` the same as above.
- `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` for interrupting a specific fiber on the fiber scheduler.

`rb_notify_fd_close` is modified so that it is fiber scheduler aware and uses `rb_fiber_scheduler_fiber_interrupt` to interrupt a fiber. In addition, we also change the internal `struct waiting_fd` to track the `rb_execution_context_t` rather than just the `rb_thread_t` instance, so that we can correctly wake up either the waiting thread or fiber.

The public interface `rb_io_interruptible_operation` and `IO#interruptible_operation` are introduced so that the scheduler implementation can wrap IO operations that should be interruptible, e.g.

```ruby
Fiber.schedule do
  io.interruptible_operation do
    io.wait_readable
  end
end

# Will interrupt above fiber:
io.close
```

See <https://github.com/ruby/ruby/pull/12585> for the proposed implementation and <https://github.com/socketry/io-event/pull/130> for example of how `io-event` gem uses both the C and Ruby interfaces.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121221] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


The fiber scheduler hook `rb_fiber_scheduler_io_wait` could be implemented like this:

```c
static VALUE
fiber_scheduler_io_wait(VALUE _argument) {
    VALUE *arguments = (VALUE*)_argument;

    rb_funcallv(arguments[0], id_io_wait, 3, arguments + 1);
}

VALUE
rb_fiber_scheduler_io_wait(VALUE scheduler, VALUE io, VALUE events, VALUE timeout)
{
    VALUE arguments[] = {
        scheduler, io, events, timeout
    };

    return rb_io_interruptible_operation(io, fiber_scheduler_io_wait, (VALUE)&arguments);
}
```

I'll consider how to modify `io_read` and `io_write` hooks.

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112164

* Author: ioquatix (Samuel Williams)
* Status: Open
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an IOError (stream closed in another thread). For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

This PR introduces some new functions:

- `VALUE rb_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` for wrapping user IO operations so they can be interrupted.
- `IO#interruptable_operation(&block)` the same as above.
- `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` for interrupting a specific fiber on the fiber scheduler.

`rb_notify_fd_close` is modified so that it is fiber scheduler aware and uses `rb_fiber_scheduler_fiber_interrupt` to interrupt a fiber. In addition, we also change the internal `struct waiting_fd` to track the `rb_execution_context_t` rather than just the `rb_thread_t` instance, so that we can correctly wake up either the waiting thread or fiber.

The public interface `rb_io_interruptible_operation` and `IO#interruptible_operation` are introduced so that the scheduler implementation can wrap IO operations that should be interruptible, e.g.

```ruby
Fiber.schedule do
  io.interruptible_operation do
    io.wait_readable
  end
end

# Will interrupt above fiber:
io.close
```

See <https://github.com/ruby/ruby/pull/12585> for the proposed implementation and <https://github.com/socketry/io-event/pull/130> for example of how `io-event` gem uses both the C and Ruby interfaces.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121226] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[luke-gru (Luke Gruber) via ruby-core]

Issue #21166 has been updated by luke-gru (Luke Gruber).


Yeah, this makes more sense to me. The other way introduces a method on all `IO` objects that's only useful in the fiber scheduler context, and I could see users forgetting
to call it. I think having it be the default is the more sensible approach, and it avoids an API.

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112168

* Author: ioquatix (Samuel Williams)
* Status: Open
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an IOError (stream closed in another thread). For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

This PR introduces some new functions:

- `VALUE rb_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` for wrapping user IO operations so they can be interrupted.
- `IO#interruptable_operation(&block)` the same as above.
- `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` for interrupting a specific fiber on the fiber scheduler.

`rb_notify_fd_close` is modified so that it is fiber scheduler aware and uses `rb_fiber_scheduler_fiber_interrupt` to interrupt a fiber. In addition, we also change the internal `struct waiting_fd` to track the `rb_execution_context_t` rather than just the `rb_thread_t` instance, so that we can correctly wake up either the waiting thread or fiber.

The public interface `rb_io_interruptible_operation` and `IO#interruptible_operation` are introduced so that the scheduler implementation can wrap IO operations that should be interruptible, e.g.

```ruby
Fiber.schedule do
  io.interruptible_operation do
    io.wait_readable
  end
end

# Will interrupt above fiber:
io.close
```

See <https://github.com/ruby/ruby/pull/12585> for the proposed implementation and <https://github.com/socketry/io-event/pull/130> for example of how `io-event` gem uses both the C and Ruby interfaces.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121274] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).

Description updated
Assignee set to ioquatix (Samuel Williams)

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112230

* Author: ioquatix (Samuel Williams)
* Status: Open
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt both threads and fibers.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121275] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


I have updated the proposal based on the discussion.

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112232

* Author: ioquatix (Samuel Williams)
* Status: Open
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121282] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ko1 (Koichi Sasada) via ruby-core]

Issue #21166 has been updated by ko1 (Koichi Sasada).


could you make clear

* `rb_thread_io_interruptible_operation` why not `rb_io` prefix? is it public c-api?
* who/when/how unregister the hooks?
* could you clear how fiber scheduler use it? no `io_close` event (callback)?

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112246

* Author: ioquatix (Samuel Williams)
* Status: Open
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121284] [Ruby master Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


> rb_thread_io_interruptible_operation why not rb_io prefix? is it public c-api?

It's not public API, and we can change the name. It is defined in `thread.c` and `thread.h` since that is where `waiting_fd` is defined/used. `waiting_fd` is not exposed outside of `thread.c`.

> who/when/how unregister the hooks?

`rb_thread_io_interruptible_operation` uses a callback, so before entry to the callback, the operation is registered in `waiting_fds`, and on exit from the callback, it is removed.

- Entry: https://github.com/ruby/ruby/pull/12839/files#diff-161b2a279f4c67a1ab075a7890ecf6f3f1d483d910910fa52b3715e25cfdcbd7R1753
- Exit: https://github.com/ruby/ruby/pull/12839/files#diff-161b2a279f4c67a1ab075a7890ecf6f3f1d483d910910fa52b3715e25cfdcbd7R1740

> could you clear how fiber scheduler use it? no io_close event (callback)?

The fiber scheduler implementation does not need to be changed, since we added this to the `scheduler.c` implementation.

> which context the callback function is called?

It's used to wrap the execution of `io_read`/`io_write` and `io_wait` scheduler hooks: https://github.com/ruby/ruby/pull/12839/files#diff-29a83910395dea702ac0c02b4cd9976ba252ff50f133d7cd8109beffbd2eab1d

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112248

* Author: ioquatix (Samuel Williams)
* Status: Open
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:121441] [Ruby Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


After discussing this PR with @ko1:

1. waiting_fds -> ractor local
2. document fiber_interrupt can be called in another thread
3. `rb_thread_io_interruptible_operation(func)` ->
   `rb_thread_io_with_waiting_file_descriptor(func)`
   (Internal function, not public API)
4. Update the PR for per-IO waiting_fd (separate follow up).


----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-112426

* Author: ioquatix (Samuel Williams)
* Status: Open
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122056] [Ruby Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


https://github.com/ruby/ruby/pull/13127 has been merged, allowing us to move forward with this feature.

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-113204

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122057] [Ruby Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).


I've renamed `rb_thread_io_interruptible_operation` to `rb_thread_io_blocking_operation` which better suits the naming convention internally.

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-113205

* Author: ioquatix (Samuel Williams)
* Status: Assigned
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122245] [Ruby Bug#21166] Fiber Scheduler is unable to be interrupted by `IO#close`.

[ioquatix (Samuel Williams) via ruby-core]

Issue #21166 has been updated by ioquatix (Samuel Williams).

Status changed from Assigned to Closed

Merged in 73c9d6ccaa2045a011ed991dc29633bd0443971a

----------------------------------------
Bug #21166: Fiber Scheduler is unable to be interrupted by `IO#close`.
https://bugs.ruby-lang.org/issues/21166#change-113391

* Author: ioquatix (Samuel Williams)
* Status: Closed
* Assignee: ioquatix (Samuel Williams)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
## Background

Ruby's `IO#close` can cause `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable` to be interrupted with an `IOError: stream closed in another thread`. For reference, `IO#select` cannot be interrupted in this way.

```ruby
r, w = IO.pipe

thread = Thread.new do
  r.read(1)
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
# ./test.rb:6:in 'IO#read': stream closed in another thread (IOError)
```

## Problem

The fiber scheduler provides hooks for `io_read`, `io_write` and `io_wait` which are used by `IO#read`, `IO#write`, `IO#wait`, `IO#wait_readable` and `IO#wait_writable`, but those hooks are not interrupted when `IO#close` is invoked. That is because `rb_notify_fd_close` is not scheduler aware, and the fiber scheduler is unable to register itself into the "waiting file descriptor" list.

```ruby
#!/usr/bin/env ruby

require 'async'

r, w = IO.pipe

thread = Thread.new do
  Async do
    r.wait_readable
  end
end

Thread.pass until thread.status == "sleep"

r.close

thread.join
```

In this test program, `rb_notify_fd_close` will incorrectly terminate the entire fiber scheduler thread:

```
#<Thread:0x00007faa5b161bf8 /home/samuel/Developer/socketry/io-event/test.rb:7 run> terminated with exception (report_on_exception is true):
/home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'IO.select': closed stream (IOError)
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:470:in 'block in IO::Event::Selector::Select#select'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'Thread.handle_interrupt'
  from /home/samuel/Developer/socketry/io-event/lib/io/event/selector/select.rb:468:in 'IO::Event::Selector::Select#select'
  from /home/samuel/.gem/ruby/3.4.1/gems/async-2.23.0/lib/async/scheduler.rb:396:in 'Async::Scheduler#run_once!'
...
```

## Solution

We need a mechanism to ensure fibers are treated the same as threads, and interrupted correctly. We do this by:

1. Introducing `VALUE rb_thread_io_interruptible_operation(VALUE self, VALUE(*function)(VALUE), VALUE argument)` which allows us to execute a callback that may be interrupted. Internally, this registers the current execution context into the existing `waiting_fds` list.
2. We update all the relevant fiber scheduler hooks to use `rb_thread_io_interruptible_operation`, e.g. `io_wait`, `io_read`, `io_write` and so on.
3. We introduce `VALUE rb_fiber_scheduler_fiber_interrupt(VALUE scheduler, VALUE fiber, VALUE exception)` which can be used to interrupt a fiber, e.g. with an IOError exception.
4. `rb_notify_fd_close` is modified to correctly interrupt fibers using the new rb_fiber_scheduler_fiber_interrupt` function.

See <https://github.com/ruby/ruby/pull/12839> for the proposed implementation.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/