ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
@ 2025-01-15  9:57 Eregon (Benoit Daloze) via ruby-core
  2025-01-15 19:28 ` [ruby-core:120701] " luke-gru (Luke Gruber) via ruby-core
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2025-01-15  9:57 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #21039 has been reported by Eregon (Benoit Daloze).

----------------------------------------
Bug #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
https://bugs.ruby-lang.org/issues/21039

* Author: Eregon (Benoit Daloze)
* Status: Open
* Assignee: ko1 (Koichi Sasada)
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
```ruby
def make_counter
  count = 0
  nil.instance_exec do
    [-> { count }, -> { count += 1 }]
  end
end

get, increment = make_counter

reader = Thread.new {
  sleep 0.01
  loop do
    p get.call
    sleep 0.1
  end
}

writer = Thread.new {
  loop do
    increment.call
    sleep 0.1
  end
}

ractor_thread = Thread.new {
  sleep 1
  Ractor.make_shareable(get)
}

sleep 2
```

This prints:
```
1
2
3
4
5
6
7
8
9
10
10
10
10
10
10
10
10
10
10
10
```
But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`.

This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment".

It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong values being read.

I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that.
However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`).

So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`.
FWIW that currently doesn't allow Procs, it gives `<internal:ractor>:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`.

I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`.
Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed.
That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables.
Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5

Extracted from https://bugs.ruby-lang.org/issues/21033#note-14



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:120701] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
  2025-01-15  9:57 [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Eregon (Benoit Daloze) via ruby-core
@ 2025-01-15 19:28 ` luke-gru (Luke Gruber) via ruby-core
  2025-01-16 15:56 ` [ruby-core:120717] " Eregon (Benoit Daloze) via ruby-core
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: luke-gru (Luke Gruber) via ruby-core @ 2025-01-15 19:28 UTC (permalink / raw)
  To: ruby-core; +Cc: luke-gru (Luke Gruber)

Issue #21039 has been updated by luke-gru (Luke Gruber).


As far as I know this is intentional behavior, so even though I agree it is confusing I think this is more accurately a feature request instead of a bug.

----------------------------------------
Bug #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
https://bugs.ruby-lang.org/issues/21039#change-111536

* Author: Eregon (Benoit Daloze)
* Status: Open
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
```ruby
def make_counter
  count = 0
  nil.instance_exec do
    [-> { count }, -> { count += 1 }]
  end
end

get, increment = make_counter

reader = Thread.new {
  sleep 0.01
  loop do
    p get.call
    sleep 0.1
  end
}

writer = Thread.new {
  loop do
    increment.call
    sleep 0.1
  end
}

ractor_thread = Thread.new {
  sleep 1
  Ractor.make_shareable(get)
}

sleep 2
```

This prints:
```
1
2
3
4
5
6
7
8
9
10
10
10
10
10
10
10
10
10
10
10
```
But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`.

This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment".

It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong/stale values being read.

I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that.
However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`).

So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`.
FWIW that currently doesn't allow Procs, it gives `<internal:ractor>:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`.
It makes sense to use `copy` here since `make_shareable` effectively takes a copy/snapshot of the Proc's environment.

I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`.
Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed.
That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables.
Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5

Extracted from https://bugs.ruby-lang.org/issues/21033#note-14



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:120717] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
  2025-01-15  9:57 [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Eregon (Benoit Daloze) via ruby-core
  2025-01-15 19:28 ` [ruby-core:120701] " luke-gru (Luke Gruber) via ruby-core
@ 2025-01-16 15:56 ` Eregon (Benoit Daloze) via ruby-core
  2025-01-16 21:17 ` [ruby-core:120720] " luke-gru (Luke Gruber) via ruby-core
  2025-01-19 23:18 ` [ruby-core:120740] [Ruby master Feature#21039] " matz (Yukihiro Matsumoto) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: Eregon (Benoit Daloze) via ruby-core @ 2025-01-16 15:56 UTC (permalink / raw)
  To: ruby-core; +Cc: Eregon (Benoit Daloze)

Issue #21039 has been updated by Eregon (Benoit Daloze).


I think it's a bug, because it breaks fundamental Ruby block semantics.
No Ractor method or functionality should be able to do that for an existing Proc, even more so when that Proc is called on the main Ractor.

----------------------------------------
Bug #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
https://bugs.ruby-lang.org/issues/21039#change-111550

* Author: Eregon (Benoit Daloze)
* Status: Open
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
```ruby
def make_counter
  count = 0
  nil.instance_exec do
    [-> { count }, -> { count += 1 }]
  end
end

get, increment = make_counter

reader = Thread.new {
  sleep 0.01
  loop do
    p get.call
    sleep 0.1
  end
}

writer = Thread.new {
  loop do
    increment.call
    sleep 0.1
  end
}

ractor_thread = Thread.new {
  sleep 1
  Ractor.make_shareable(get)
}

sleep 2
```

This prints:
```
1
2
3
4
5
6
7
8
9
10
10
10
10
10
10
10
10
10
10
10
```
But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`.

This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment".

It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong/stale values being read.

I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that.
However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`).

So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`.
FWIW that currently doesn't allow Procs, it gives `<internal:ractor>:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`.
It makes sense to use `copy` here since `make_shareable` effectively takes a copy/snapshot of the Proc's environment.

I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`.
Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed.
That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables.
Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5

Extracted from https://bugs.ruby-lang.org/issues/21033#note-14



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:120720] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
  2025-01-15  9:57 [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Eregon (Benoit Daloze) via ruby-core
  2025-01-15 19:28 ` [ruby-core:120701] " luke-gru (Luke Gruber) via ruby-core
  2025-01-16 15:56 ` [ruby-core:120717] " Eregon (Benoit Daloze) via ruby-core
@ 2025-01-16 21:17 ` luke-gru (Luke Gruber) via ruby-core
  2025-01-19 23:18 ` [ruby-core:120740] [Ruby master Feature#21039] " matz (Yukihiro Matsumoto) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: luke-gru (Luke Gruber) via ruby-core @ 2025-01-16 21:17 UTC (permalink / raw)
  To: ruby-core; +Cc: luke-gru (Luke Gruber)

Issue #21039 has been updated by luke-gru (Luke Gruber).


Okay fair enough, and it's not of much consequence either way whether it's a bug or a feature because your point still stands.

----------------------------------------
Bug #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
https://bugs.ruby-lang.org/issues/21039#change-111553

* Author: Eregon (Benoit Daloze)
* Status: Open
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
```ruby
def make_counter
  count = 0
  nil.instance_exec do
    [-> { count }, -> { count += 1 }]
  end
end

get, increment = make_counter

reader = Thread.new {
  sleep 0.01
  loop do
    p get.call
    sleep 0.1
  end
}

writer = Thread.new {
  loop do
    increment.call
    sleep 0.1
  end
}

ractor_thread = Thread.new {
  sleep 1
  Ractor.make_shareable(get)
}

sleep 2
```

This prints:
```
1
2
3
4
5
6
7
8
9
10
10
10
10
10
10
10
10
10
10
10
```
But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`.

This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment".

It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong/stale values being read.

I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that.
However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`).

So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`.
FWIW that currently doesn't allow Procs, it gives `<internal:ractor>:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`.
It makes sense to use `copy` here since `make_shareable` effectively takes a copy/snapshot of the Proc's environment.

I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`.
Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed.
That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables.
Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5

Extracted from https://bugs.ruby-lang.org/issues/21033#note-14



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:120740] [Ruby master Feature#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
  2025-01-15  9:57 [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Eregon (Benoit Daloze) via ruby-core
                   ` (2 preceding siblings ...)
  2025-01-16 21:17 ` [ruby-core:120720] " luke-gru (Luke Gruber) via ruby-core
@ 2025-01-19 23:18 ` matz (Yukihiro Matsumoto) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: matz (Yukihiro Matsumoto) via ruby-core @ 2025-01-19 23:18 UTC (permalink / raw)
  To: ruby-core; +Cc: matz (Yukihiro Matsumoto)

Issue #21039 has been updated by matz (Yukihiro Matsumoto).

Tracker changed from Bug to Feature
ruby -v deleted (ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux])
Backport deleted (3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN)

1. It's intended behavior. We don't call it a bug. If we keep the non-shareable semantics, the basic principle of Ractors would fail (no shared state).
2. I understand your feeling to the new behavior. The behavior can be negotiable, but it should follow Ractor principle.

Matz.
 

----------------------------------------
Feature #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks
https://bugs.ruby-lang.org/issues/21039#change-111571

* Author: Eregon (Benoit Daloze)
* Status: Open
* Assignee: ko1 (Koichi Sasada)
----------------------------------------
```ruby
def make_counter
  count = 0
  nil.instance_exec do
    [-> { count }, -> { count += 1 }]
  end
end

get, increment = make_counter

reader = Thread.new {
  sleep 0.01
  loop do
    p get.call
    sleep 0.1
  end
}

writer = Thread.new {
  loop do
    increment.call
    sleep 0.1
  end
}

ractor_thread = Thread.new {
  sleep 1
  Ractor.make_shareable(get)
}

sleep 2
```

This prints:
```
1
2
3
4
5
6
7
8
9
10
10
10
10
10
10
10
10
10
10
10
```
But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`.

This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment".

It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong/stale values being read.

I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that.
However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`).

So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`.
FWIW that currently doesn't allow Procs, it gives `<internal:ractor>:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`.
It makes sense to use `copy` here since `make_shareable` effectively takes a copy/snapshot of the Proc's environment.

I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`.
Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed.
That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables.
Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5

Extracted from https://bugs.ruby-lang.org/issues/21033#note-14



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-01-19 23:18 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-01-15  9:57 [ruby-core:120694] [Ruby master Bug#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Eregon (Benoit Daloze) via ruby-core
2025-01-15 19:28 ` [ruby-core:120701] " luke-gru (Luke Gruber) via ruby-core
2025-01-16 15:56 ` [ruby-core:120717] " Eregon (Benoit Daloze) via ruby-core
2025-01-16 21:17 ` [ruby-core:120720] " luke-gru (Luke Gruber) via ruby-core
2025-01-19 23:18 ` [ruby-core:120740] [Ruby master Feature#21039] " matz (Yukihiro Matsumoto) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).