ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:119587] [Ruby master Bug#20809] Ripper fails to parse/tokenize nested heredocs within embexpr
@ 2024-10-22 10:57 tompng (tomoya ishida) via ruby-core
  0 siblings, 0 replies; only message in thread
From: tompng (tomoya ishida) via ruby-core @ 2024-10-22 10:57 UTC (permalink / raw)
  To: ruby-core; +Cc: tompng (tomoya ishida)

Issue #20809 has been reported by tompng (tomoya ishida).

----------------------------------------
Bug #20809: Ripper fails to parse/tokenize nested heredocs within embexpr
https://bugs.ruby-lang.org/issues/20809

* Author: tompng (tomoya ishida)
* Status: Open
* ruby -v: ruby 3.4.0dev (2024-10-22T09:04:30Z master 54065f3b7b) +PRISM [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
In this example, Ripper wrongly concats `"s333\n"` and `"s4444"`.
~~~ruby
require 'ripper'

code1 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444#{55555}
HEREDOC1'

p Ripper.tokenize(code1)
# => ["<<HEREDOC1", "\n", "s1\n", "\#{", "<<HEREDOC2", "}", "s333\ns4444", "s22\n", "HEREDOC2\n", "\#{", "55555", "}", "\n", "HEREDOC1"]
~~~

In this example, result of `Ripper.sexp` is wrong. `"s333"` disappears.
~~~ruby
code2 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444
HEREDOC1'

p eval(code2) # => "s1\ns22\ns333\ns4444\n"
p Ripper.sexp(code2)
# => 
[:program,
 [[:string_literal,
   [:string_content,
    [:@tstring_content, "s1\n", [2, 0]],
    [:string_embexpr,
     [[:string_literal,
       [:string_content,
        [:@tstring_content, "s22\n", [4, 0]]]]]],
    [:@tstring_content, "s4444\n", [6, 0]]]]]]
~~~



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2024-10-22 10:58 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-10-22 10:57 [ruby-core:119587] [Ruby master Bug#20809] Ripper fails to parse/tokenize nested heredocs within embexpr tompng (tomoya ishida) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).