* [ruby-core:119587] [Ruby master Bug#20809] Ripper fails to parse/tokenize nested heredocs within embexpr
@ 2024-10-22 10:57 tompng (tomoya ishida) via ruby-core
0 siblings, 0 replies; only message in thread
From: tompng (tomoya ishida) via ruby-core @ 2024-10-22 10:57 UTC (permalink / raw)
To: ruby-core; +Cc: tompng (tomoya ishida)
Issue #20809 has been reported by tompng (tomoya ishida).
----------------------------------------
Bug #20809: Ripper fails to parse/tokenize nested heredocs within embexpr
https://bugs.ruby-lang.org/issues/20809
* Author: tompng (tomoya ishida)
* Status: Open
* ruby -v: ruby 3.4.0dev (2024-10-22T09:04:30Z master 54065f3b7b) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
In this example, Ripper wrongly concats `"s333\n"` and `"s4444"`.
~~~ruby
require 'ripper'
code1 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444#{55555}
HEREDOC1'
p Ripper.tokenize(code1)
# => ["<<HEREDOC1", "\n", "s1\n", "\#{", "<<HEREDOC2", "}", "s333\ns4444", "s22\n", "HEREDOC2\n", "\#{", "55555", "}", "\n", "HEREDOC1"]
~~~
In this example, result of `Ripper.sexp` is wrong. `"s333"` disappears.
~~~ruby
code2 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444
HEREDOC1'
p eval(code2) # => "s1\ns22\ns333\ns4444\n"
p Ripper.sexp(code2)
# =>
[:program,
[[:string_literal,
[:string_content,
[:@tstring_content, "s1\n", [2, 0]],
[:string_embexpr,
[[:string_literal,
[:string_content,
[:@tstring_content, "s22\n", [4, 0]]]]]],
[:@tstring_content, "s4444\n", [6, 0]]]]]]
~~~
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2024-10-22 10:58 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-10-22 10:57 [ruby-core:119587] [Ruby master Bug#20809] Ripper fails to parse/tokenize nested heredocs within embexpr tompng (tomoya ishida) via ruby-core
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).