[ruby-core:107106] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[ruby-core:107106] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[alanwu (Alan Wu)]

Issue #18487 has been reported by alanwu (Alan Wu).

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I'm couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107123] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[mame (Yusuke Endoh)]

Issue #18487 has been updated by mame (Yusuke Endoh).


Interesting. I created a simpler version.

```
class Magic
  define_singleton_method :modify_caller_env!, method(:binding).to_proc >> ->(bndg) { bndg.local_variable_set(:my_var, 42) }
end

my_var = 1

Magic.modify_caller_env!

p my_var #=> 42
```

I have no strong opinion, but your solution "Kernel#binding only ever looks at the stack frame directly below it" looks reasonable to me.

BTW, as you may know, there is a relatively popular gem called [binding_of_caller](https://rubygems.org/gems/binding_of_caller) to extract a Binding object from caller frames. YJIT optimization might be still difficult even after Kernel#binding was changed.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-95968

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107238] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[jeremyevans0 (Jeremy Evans)]

Issue #18487 has been updated by jeremyevans0 (Jeremy Evans).


I submitted a pull request to make `Kernel#binding` only look up a single frame, which fixes the issue: https://github.com/ruby/ruby/pull/5476.  Not sure if all the semantics in the pull request are desired (i.e. `eval` and `receiver` raise `RuntimeError` for bindings for non-Ruby frames), so this is probably worth discussing at the next developer meeting.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96097

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107240] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[Eregon (Benoit Daloze)]

Issue #18487 has been updated by Eregon (Benoit Daloze).


Nice find!
Agreed this should be fixed, and `Kernel#binding` should never provide access to anything but its direct caller method's frame (whether that's defined in Ruby, C or anything).
In other words `Kernel#binding` should provide access to the local variables immediately around the call to `Kernel#binding`.
The current behavior in CRuby is effectively breaking encapsulation, even though I'd think it never intended that.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96099

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107609] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[alanwu (Alan Wu)]

Issue #18487 has been updated by alanwu (Alan Wu).


To simplify the semantics and implementation, we could make Kernel#binding
raise when the direct caller is not Ruby. I think it's reasonable given that
the Binding class was designed for Ruby and doesn't necessarily make sense for
other languages.



----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96520

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107615] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[matz (Yukihiro Matsumoto)]

Issue #18487 has been updated by matz (Yukihiro Matsumoto).


Okay, `binding` should raise an exception when called from a C defined method.

Matz.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96526

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107631] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[Eregon (Benoit Daloze)]

Issue #18487 has been updated by Eregon (Benoit Daloze).


FWIW TruffleRuby currently raises one of these 2 errors when trying to call a Ruby method which needs a direct Ruby frame above:
```
Cannot call Ruby method which needs a Ruby caller frame directly in a foreign language (RuntimeError)
or
Foo#bar needs the caller frame but it was not passed (cannot be called directly from a foreign language) (RuntimeError)
```
That can happen for C extension but also for any other language calling Ruby methods (e.g. JS/Python/etc).

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96542

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:107650] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[jeremyevans0 (Jeremy Evans)]

Issue #18487 has been updated by jeremyevans0 (Jeremy Evans).


matz (Yukihiro Matsumoto) wrote in #note-6:
> Okay, `binding` should raise an exception when called from a C defined method.

I've submitted a pull request for this: https://github.com/ruby/ruby/pull/5567

It's trickier than I expected, and took some trial and error to get right.  I also found that some tests were implicitly relying on the previous behavior.  One case was related to tracing, as set_trace_func yields bindings.  I modified the logic there so that cases where generating the binding would raise an exception, we yield nil as the binding (this was already done in some cases, so I don't think there should be significant backwards compatibility issues).

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-96563

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:108180] [Ruby master Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[jeremyevans0 (Jeremy Evans)]

Issue #18487 has been updated by jeremyevans0 (Jeremy Evans).


The previous commit failed with VM assertion error when compiling with `-DRUBY_DEBUG=1 -DRGENGC_CHECK_MODE=2`.  I've found the issue was due to TracePoint/set_trace_func creating bindings for ifuncs.  I've submitted a pull request to fix that by not creating bindings for ifuncs, only for iseqs: https://github.com/ruby/ruby/pull/5767

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-97154

* Author: alanwu (Alan Wu)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/

[ruby-core:123190] [Ruby Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[ttilberg (Tim Tilberg) via ruby-core]

Issue #18487 has been updated by ttilberg (Tim Tilberg).


I'm upgrading a project from Ruby 3.0 to Ruby 3.4, and I think I came across an unexpected bug related to this related to using `binding` from a `SimpleDelegator` child. Can someone help me confirm and organize?

Example reproduction script:

``` ruby
require 'erb'
require 'delegate'

class Report < SimpleDelegator
  def title
    "Great report"
  end

  def to_s
    ERB.new("<%= title %>!").result(binding)
  end
end

puts Report.new(Object.new).to_s
```

```
▶ ruby --version && ruby example.rb
ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [arm64-darwin22]
Great report!

▶ ruby --version && ruby example.rb
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin22]
Great report!

▶ ruby --version && ruby example.rb
ruby 3.2.4 (2024-04-23 revision af471c0e01) +YJIT [arm64-darwin23]
/Users/ttilberg/.asdf/installs/ruby/3.2.4/lib/ruby/3.2.0/delegate.rb:89:in `binding': Cannot create Binding object for non-Ruby caller (RuntimeError)
	from /Users/ttilberg/.asdf/installs/ruby/3.2.4/lib/ruby/3.2.0/delegate.rb:89:in `bind_call'
	from /Users/ttilberg/.asdf/installs/ruby/3.2.4/lib/ruby/3.2.0/delegate.rb:89:in `method_missing'
	from example.rb:10:in `to_s'
	from example.rb:14:in `<main>'
```

And it continues the same into future versions.

I arrived here by way of Zevrok's ruby references notes. https://rubyreferences.github.io/rubychanges/3.2.html#kernelbinding-raises-if-accessed-not-from-ruby -- it seemed like the most reasonable suspect.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-114520

* Author: alanwu (Alan Wu)
* Status: Closed
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:123192] [Ruby Bug#18487] Kernel#binding behaves differently depending on implementation language of items on the stack

[alanwu (Alan Wu) via ruby-core]

Issue #18487 has been updated by alanwu (Alan Wu).


@ttilberg That's right. The backtrace has `bind_call` as the caller of `binding`, and `bind_call` is a C method, hence the exception.

----------------------------------------
Bug #18487: Kernel#binding behaves differently depending on implementation language of items on the stack
https://bugs.ruby-lang.org/issues/18487#change-114523

* Author: alanwu (Alan Wu)
* Status: Closed
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Recently I [discovered] that one could use `Kernel#binding` to capture the
environment of a frame that is not directly below the stack frame for
`Kernel#binding`. I've known that C extensions have this [privilege] for a
while, but didn't realize that it was also possible using only the core
library. This is a powerful primitive that allows for some funky programs:

```ruby
def lookup(hash, key)
  hash[key]
  hash
end

p lookup(
  Hash.new(&(
    Kernel.instance_method(:send).method(:bind_call).to_proc >>
      ->(binding) { binding.local_variable_set(:hash, :action_at_a_distance!) }
    )
  ),
  :binding
) # => :action_at_a_distance!
```

There might be ways to compose core library procs such that it's less contrived
and more useful, but I couldn't figure out a way to do it. Maybe there is a
way to make a "local variable set" proc that takes only a name-value pair and
no block?

### What's the big deal?

This behavior makes the implementation language of methods part of the API
surface for `Kernel#binding`. In other words, merely changing a Ruby method to
be a C method can be a breaking change for the purposes of `Kernel#binding`,
even if the method behaves the same in all other observable ways. I feel that
whether a method is native or not should be an implementation detail and should
not impact `Kernel#binding`.

This is a problem for Ruby implementations that want to implement many core
methods in Ruby, because they risk breaking compatibility with CRuby.
TruffleRuby has this [problem][privilege] as I alluded to earlier, and CRuby
risks making unintentional breaking changes as more methods change to become
Ruby methods in the core library.

### Leaking less details

I think a straight forward way to fix this issue is by making it so that
`Kernel#binding` only ever looks at the stack frame directly below it. If the
frame below is a not a Ruby frame, it can return an empty binding. I haven't
done the leg work of figuring out how hard this would be to implement in CRuby,
though. This new behavior allows observing the identity of native frames, which
is new. 

### Does the more restrictive behavior help YJIT?

Maybe. It's hard to tell without building out more optimizations that are
related to local variables. YJIT currently doesn't do much in that area. If I
had to guess I wouuld say the more restrictive semantics at least open up the
possibility of some deoptimization strategies that are more memory efficient. 

### What do you think?

This is not a huge issue, but it might be nice to start thinking about for the
next release. If a lot of people actually rely on the current behavior we can
provide a migration plan. Since it might take years to land, I would like to
solicit feedback now.


[discovered]: https://github.com/ruby/ruby/commit/54c91042ed61a869d4a66fc089b21f56d165265f
[privilege]: https://github.com/oracle/truffleruby/issues/2171




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/