ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
@ 2024-02-29  6:25 mame (Yusuke Endoh) via ruby-core
  2024-02-29  9:31 ` [ruby-core:117010] " Eregon (Benoit Daloze) via ruby-core
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: mame (Yusuke Endoh) via ruby-core @ 2024-02-29  6:25 UTC (permalink / raw)
  To: ruby-core; +Cc: mame (Yusuke Endoh)

Issue #20314 has been reported by mame (Yusuke Endoh).

----------------------------------------
Bug #20314: Simultaneous Timeout expires may raise an exception after the block
https://bugs.ruby-lang.org/issues/20314

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows.


```ruby
require "timeout"

class A < Exception
end

class B < Exception
end

begin
  Timeout.timeout(0.1, A) do
    Timeout.timeout(0.1, B) do
      nil while true
    end
  end
rescue A, B
  p $! #=> #<A: execution expired>

  # Exception B is raised after the above call returns
  #=> test.rb:16:in `p': execution expired (B)

  p :end # not reach
end
```

This is because the timer thread performs two consecutive `Thread#raise` to the target thread.

I have discussed this with @ko1 and have come up with three solutions.

### Solution 1

When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A.

The problem with this approach is that if you are rescuing A in the inner block, it may never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue A
      sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end.
    end
  end
end
```

Note that, if A and B did not occur at the same time, it would raise B. This is a race condition.

### Solution 2

When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both.

The problem with this approach is that if you are rescuing B in the inner block, it never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue B
      sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends.
    end
  end
end
```

### Solution 3

Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A.

Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117010] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
  2024-02-29  6:25 [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block mame (Yusuke Endoh) via ruby-core
@ 2024-02-29  9:31 ` Eregon (Benoit Daloze) via ruby-core
  2024-03-04 17:38 ` [ruby-core:117053] " jeremyevans0 (Jeremy Evans) via ruby-core
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2024-02-29  9:31 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #20314 has been updated by Eregon (Benoit Daloze).


I'm not sure how Solution 3 would work.
`Thread#raise` would block until what?
Until the exception started to be raised/thrown on that thread? I think that would not fix that snippet.
It does not seem reasonable to wait until the exception has been rescued (or escapes the thread) because that could run arbitrary code via `ensure` which could take a long time (and block the caller of `Thread#raise` for a long time).

I wonder if we should always use `Timeout::ExitException` to "unwind" until we exit the corresponding block given to `Timeout.timeout`.
Unsure if that would help for this issue though.

Maybe there are other solutions too?

Between solutions 1 and 2, 2 seems better because it seems clearly broken code to `rescue B; sleep; end` in `Timeout.timeout(0.1, B)`.
In any case `sleep` in `ensure`/`rescue` is broken code as it can already hang (e.g. with a single `Timeout.timeout(0.1, SomeKlass)` around it, or without using Timeout).

----------------------------------------
Bug #20314: Simultaneous Timeout expires may raise an exception after the block
https://bugs.ruby-lang.org/issues/20314#change-107071

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows.


```ruby
require "timeout"

class A < Exception
end

class B < Exception
end

begin
  Timeout.timeout(0.1, A) do
    Timeout.timeout(0.1, B) do
      nil while true
    end
  end
rescue A, B
  p $! #=> #<A: execution expired>

  # Exception B is raised after the above call returns
  #=> test.rb:16:in `p': execution expired (B)

  p :end # not reach
end
```

This is because the timer thread performs two consecutive `Thread#raise` to the target thread.

I have discussed this with @ko1 and have come up with three solutions.

### Solution 1

When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A.

The problem with this approach is that if you are rescuing A in the inner block, it may never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue A
      sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end.
    end
  end
end
```

Note that, if A and B did not occur at the same time, it would raise B. This is a race condition.

### Solution 2

When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both.

The problem with this approach is that if you are rescuing B in the inner block, it never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue B
      sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends.
    end
  end
end
```

### Solution 3

Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A.

Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117053] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
  2024-02-29  6:25 [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block mame (Yusuke Endoh) via ruby-core
  2024-02-29  9:31 ` [ruby-core:117010] " Eregon (Benoit Daloze) via ruby-core
@ 2024-03-04 17:38 ` jeremyevans0 (Jeremy Evans) via ruby-core
  2024-03-06  0:43 ` [ruby-core:117059] " mame (Yusuke Endoh) via ruby-core
  2024-10-21  8:50 ` [ruby-core:119550] " Eregon (Benoit Daloze) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: jeremyevans0 (Jeremy Evans) via ruby-core @ 2024-03-04 17:38 UTC (permalink / raw)
  To: ruby-core; +Cc: jeremyevans0 (Jeremy Evans)

Issue #20314 has been updated by jeremyevans0 (Jeremy Evans).


Solution 2 makes the most sense to me.  If inside a `Timeout.timeout` block, you are swallowing the exception that you provided in the `Timeout.timeout` method call, that to me indicates you do not want to handle it as a timeout.

----------------------------------------
Bug #20314: Simultaneous Timeout expires may raise an exception after the block
https://bugs.ruby-lang.org/issues/20314#change-107127

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows.


```ruby
require "timeout"

class A < Exception
end

class B < Exception
end

begin
  Timeout.timeout(0.1, A) do
    Timeout.timeout(0.1, B) do
      nil while true
    end
  end
rescue A, B
  p $! #=> #<A: execution expired>

  # Exception B is raised after the above call returns
  #=> test.rb:16:in `p': execution expired (B)

  p :end # not reach
end
```

This is because the timer thread performs two consecutive `Thread#raise` to the target thread.

I have discussed this with @ko1 and have come up with three solutions.

### Solution 1

When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A.

The problem with this approach is that if you are rescuing A in the inner block, it may never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue A
      sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end.
    end
  end
end
```

Note that, if A and B did not occur at the same time, it would raise B. This is a race condition.

### Solution 2

When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both.

The problem with this approach is that if you are rescuing B in the inner block, it never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue B
      sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends.
    end
  end
end
```

### Solution 3

Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A.

Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117059] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
  2024-02-29  6:25 [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block mame (Yusuke Endoh) via ruby-core
  2024-02-29  9:31 ` [ruby-core:117010] " Eregon (Benoit Daloze) via ruby-core
  2024-03-04 17:38 ` [ruby-core:117053] " jeremyevans0 (Jeremy Evans) via ruby-core
@ 2024-03-06  0:43 ` mame (Yusuke Endoh) via ruby-core
  2024-10-21  8:50 ` [ruby-core:119550] " Eregon (Benoit Daloze) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: mame (Yusuke Endoh) via ruby-core @ 2024-03-06  0:43 UTC (permalink / raw)
  To: ruby-core; +Cc: mame (Yusuke Endoh)

Issue #20314 has been updated by mame (Yusuke Endoh).


Indeed Solution 3 seems to be wrong. I think I must have misunderstood what ko1 was saying.

Actually, I don't like neither Solutions 1 and 2. In the code example, I used exceptions A and B just for clarity. In most real-world cases, both are `Timeout::Error`. Swallowing `Timeout::Error` is a normal practice (to make `Timeout.timeout` end gracefully), so I don't think it is "broken".

I think this is a design flaw of Timeout's API, but I don't know what we can do for that. I guess we will just have to go with Solution 2 for now?

----------------------------------------
Bug #20314: Simultaneous Timeout expires may raise an exception after the block
https://bugs.ruby-lang.org/issues/20314#change-107133

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows.


```ruby
require "timeout"

class A < Exception
end

class B < Exception
end

begin
  Timeout.timeout(0.1, A) do
    Timeout.timeout(0.1, B) do
      nil while true
    end
  end
rescue A, B
  p $! #=> #<A: execution expired>

  # Exception B is raised after the above call returns
  #=> test.rb:16:in `p': execution expired (B)

  p :end # not reach
end
```

This is because the timer thread performs two consecutive `Thread#raise` to the target thread.

I have discussed this with @ko1 and have come up with three solutions.

### Solution 1

When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A.

The problem with this approach is that if you are rescuing A in the inner block, it may never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue A
      sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end.
    end
  end
end
```

Note that, if A and B did not occur at the same time, it would raise B. This is a race condition.

### Solution 2

When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both.

The problem with this approach is that if you are rescuing B in the inner block, it never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue B
      sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends.
    end
  end
end
```

### Solution 3

Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A.

Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:119550] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
  2024-02-29  6:25 [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block mame (Yusuke Endoh) via ruby-core
                   ` (2 preceding siblings ...)
  2024-03-06  0:43 ` [ruby-core:117059] " mame (Yusuke Endoh) via ruby-core
@ 2024-10-21  8:50 ` Eregon (Benoit Daloze) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2024-10-21  8:50 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #20314 has been updated by Eregon (Benoit Daloze).


Ref: https://github.com/ruby/ruby/pull/10851

----------------------------------------
Bug #20314: Simultaneous Timeout expires may raise an exception after the block
https://bugs.ruby-lang.org/issues/20314#change-110168

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows.


```ruby
require "timeout"

class A < Exception
end

class B < Exception
end

begin
  Timeout.timeout(0.1, A) do
    Timeout.timeout(0.1, B) do
      nil while true
    end
  end
rescue A, B
  p $! #=> #<A: execution expired>

  # Exception B is raised after the above call returns
  #=> test.rb:16:in `p': execution expired (B)

  p :end # not reach
end
```

This is because the timer thread performs two consecutive `Thread#raise` to the target thread.

I have discussed this with @ko1 and have come up with three solutions.

### Solution 1

When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A.

The problem with this approach is that if you are rescuing A in the inner block, it may never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue A
      sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end.
    end
  end
end
```

Note that, if A and B did not occur at the same time, it would raise B. This is a race condition.

### Solution 2

When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both.

The problem with this approach is that if you are rescuing B in the inner block, it never ends:

```ruby
Timeout.timeout(0.1, A) do
  Timeout.timeout(0.1, B) do
    begin
      sleep
    rescue B
      sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends.
    end
  end
end
```

### Solution 3

Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A.

Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2024-10-21  9:05 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-29  6:25 [ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block mame (Yusuke Endoh) via ruby-core
2024-02-29  9:31 ` [ruby-core:117010] " Eregon (Benoit Daloze) via ruby-core
2024-03-04 17:38 ` [ruby-core:117053] " jeremyevans0 (Jeremy Evans) via ruby-core
2024-03-06  0:43 ` [ruby-core:117059] " mame (Yusuke Endoh) via ruby-core
2024-10-21  8:50 ` [ruby-core:119550] " Eregon (Benoit Daloze) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).