Issue #20809 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20809: Ripper fails to parse/tokenize nested heredocs within embexpr https://bugs.ruby-lang.org/issues/20809 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-10-22T09:04:30Z master 54065f3b7b) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- In this example, Ripper wrongly concats `"s333\n"` and `"s4444"`. ~~~ruby require 'ripper' code1 = '<<HEREDOC1 s1 #{<<HEREDOC2}s333 s22 HEREDOC2 s4444#{55555} HEREDOC1' p Ripper.tokenize(code1) # => ["<<HEREDOC1", "\n", "s1\n", "\#{", "<<HEREDOC2", "}", "s333\ns4444", "s22\n", "HEREDOC2\n", "\#{", "55555", "}", "\n", "HEREDOC1"] ~~~ In this example, result of `Ripper.sexp` is wrong. `"s333"` disappears. ~~~ruby code2 = '<<HEREDOC1 s1 #{<<HEREDOC2}s333 s22 HEREDOC2 s4444 HEREDOC1' p eval(code2) # => "s1\ns22\ns333\ns4444\n" p Ripper.sexp(code2) # => [:program, [[:string_literal, [:string_content, [:@tstring_content, "s1\n", [2, 0]], [:string_embexpr, [[:string_literal, [:string_content, [:@tstring_content, "s22\n", [4, 0]]]]]], [:@tstring_content, "s4444\n", [6, 0]]]]]] ~~~ -- https://bugs.ruby-lang.org/