[ruby-core:119301] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[ruby-core:119301] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Earlopain (A S) via ruby-core]

Issue #20761 has been reported by Earlopain (A S).

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761

* Author: Earlopain (A S)
* Status: Open
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119302] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[kddnewton (Kevin Newton) via ruby-core]

Issue #20761 has been updated by kddnewton (Kevin Newton).

Status changed from Open to Closed

This is expected behavior. The instruction sequences will have different node ids, so it's not possible to retrieve the RubyVM::AbstractSyntaxTree representation of the AST. If you want to retrieve the Prism AST, you can do so using `Prism.parse` in the same way that error highlight does here: https://github.com/ruby/error_highlight/blob/452f78640c08ab277683416668a52d9fcfb6a26a/lib/error_highlight/base.rb#L57-L66.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109913

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119303] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Earlopain (A S) via ruby-core]

Issue #20761 has been updated by Earlopain (A S).


kddnewton (Kevin Newton) wrote in #note-1:
> This is expected behavior. The instruction sequences will have different node ids, so it's not possible to retrieve the RubyVM::AbstractSyntaxTree representation of the AST. If you want to retrieve the Prism AST, you can do so using `Prism.parse` in the same way that error highlight does here: https://github.com/ruby/error_highlight/blob/452f78640c08ab277683416668a52d9fcfb6a26a/lib/error_highlight/base.rb#L57-L66.

Thanks, I have already looked at that code since you linked to it in the PR that made this error. It seems to rely on a backtrace to understand which node to look for but is that possible if I just have a proc that doesn't raise by itself? I'd guess the proc does have a node id but I'm not sure how to retrieve it.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109914

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119304] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[kddnewton (Kevin Newton) via ruby-core]

Issue #20761 has been updated by kddnewton (Kevin Newton).


For a proc you can mirror what `RubyVM::AbstractSyntaxTree.of` is doing under the hood using `RubyVM::InstructionSequence`, as in:

```ruby
my_proc = -> { 1 }

iseq = RubyVM::InstructionSequence.of(my_proc)
node_id = iseq.to_a[4][:node_id]
source = iseq.script_lines.join

require "prism"
Prism.parse(source).value.breadth_first_search { |node| node.node_id == node_id }
```

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109915

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119306] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[kddnewton (Kevin Newton) via ruby-core]

Issue #20761 has been updated by kddnewton (Kevin Newton).


Note that all the APIs under RubyVM are subject to change, so you're effectively calling internal APIs here. Just so you know going forward.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109917

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119307] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Earlopain (A S) via ruby-core]

Issue #20761 has been updated by Earlopain (A S).


This is great info, thank you! I'm aware about the non-guarantees for `RubyVM`, just something to live with. `AbstractSyntaxTree.of` already had the same "problem".

Regardless, should `AbstractSyntaxTree.of` documentation be updated in some way, or be documented at all? Seems to only work if I explicitly opt out of prism.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109918

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119309] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[kddnewton (Kevin Newton) via ruby-core]

Issue #20761 has been updated by kddnewton (Kevin Newton).


Yeah, I think it would make sense to update those docs to indicate that it only works when the instruction sequences were compiled with the old compiler.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109920

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119315] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Eregon (Benoit Daloze) via ruby-core]

Issue #20761 has been updated by Eregon (Benoit Daloze).


@Earlopain Is there a reason you need this and cannot just use e.g. `Prism.parse("proc {1 + 2}")`?

A cleaner and portable way to achieve this functionality from a Proc/Method would be if there is a way to retrieve their precise source location, i.e. start & end byte offsets or equivalent (`#source_location` currently only gives the start line and column).

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109925

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119316] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Eregon (Benoit Daloze) via ruby-core]

Issue #20761 has been updated by Eregon (Benoit Daloze).


In general the entire `RubyVM::AbstractSyntaxTree` module should be considered deprecated (and maybe even removed), given it relies on parse.y internals and that matz said that the Prism API is the official Ruby API for parsing Ruby code (i.e. no matter which parser is used internally, the exposed API must be the one of Prism).
`RubyVM::AbstractSyntaxTree` was always experimental, unstable, only working on CRuby, etc. So now is a good time to no longer rely on it.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109926

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119317] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Earlopain (A S) via ruby-core]

Issue #20761 has been updated by Earlopain (A S).


Eregon (Benoit Daloze) wrote in #note-7:
> @Earlopain Is there a reason you need this and cannot just use e.g. `Prism.parse("proc {1 + 2}")`?
> 
> A cleaner and portable way to achieve this functionality from a Proc/Method would be if there is a way to retrieve their precise source location, i.e. start & end byte offsets or equivalent (`#source_location` currently only gives the start line and column).

This question specifically was about usage that Rails recently adopted for printing the source code of a proc when doing `assert_no_difference(-> { foo })` inside the assertion error message. https://github.com/rails/rails/pull/52036. kddnewton has since raised a PR switching it over to `RubyVM::InstructionSequence` which of course has the same interoperability problems. https://github.com/rails/rails/pull/53055

If `source_location` would provide richer information, then it would be an easy switch. The PR basically already doing that through `:code_location` from the iseq. I remember an issue about it, maybe this https://bugs.ruby-lang.org/issues/8751, though I thought it was a bit more recently discussed.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109927

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:119319] [Ruby master Bug#20761] [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default

[Eregon (Benoit Daloze) via ruby-core]

Issue #20761 has been updated by Eregon (Benoit Daloze).


Thanks for the links, yes, that seems a perfect use case for more information in `#source_location`, going through `RubyVM` APIs for that feels overly complicated for something so simple.

I found https://bugs.ruby-lang.org/issues/6012 which seems closest to what's needed here, and linked the 2 other related issues to it.

----------------------------------------
Bug #20761: [DOC] `RubyVM::AbstractSyntaxTree.of` examples raise because parser is prism by default
https://bugs.ruby-lang.org/issues/20761#change-109933

* Author: Earlopain (A S)
* Status: Closed
* ruby -v: 3.4-dev
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
https://docs.ruby-lang.org/en/master/RubyVM/AbstractSyntaxTree.html#method-c-of

```rb
RubyVM::AbstractSyntaxTree.of(proc {1 + 2})
# => <internal:ast>:97:in 'RubyVM::AbstractSyntaxTree.of': cannot get AST for ISEQ compiled by prism (RuntimeError)
```

Same for the method example. Is this method even functional when prism is used, or is the prism gem able to do this somehow?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/