Issue #19838 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #19838: Ripper nested heredocs https://bugs.ruby-lang.org/issues/19838 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I'm seeing some odd behavior on nested heredocs in the ripper lexer. ``` ruby <<~H1 1 #{<<~H2} 2 H2 3 H1 ``` $ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)' ``` ruby [[[1, 0], :on_heredoc_beg, "<<~H1", BEG], [[1, 5], :on_nl, "\n", BEG], [[2, 0], :on_ignored_sp, " ", BEG], [[2, 2], :on_tstring_content, "1\n", BEG], [[3, 0], :on_ignored_sp, " ", BEG], [[3, 2], :on_embexpr_beg, "\#{", BEG], [[3, 4], :on_heredoc_beg, "<<~H2", BEG], [[3, 9], :on_embexpr_end, "}", END], [[3, 10], :on_tstring_content, "\n", BEG], [[4, 0], :on_ignored_sp, " ", BEG], [[4, 4], :on_tstring_content, "2\n", BEG], [[5, 0], :on_heredoc_end, " H2\n", BEG], [[6, 0], :on_ignored_sp, " ", BEG], [[6, 2], :on_tstring_content, "3\n", BEG], [[7, 0], :on_heredoc_end, "H1\n", BEG]] ``` This seems totally correct. But if you change it to a non-indenting heredoc: ``` ruby <<-H1 1 #{<<~H2} 2 H2 3 H1 ``` $ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)' ``` ruby [[[1, 0], :on_heredoc_beg, "<<-H1", BEG], [[1, 5], :on_nl, "\n", BEG], [[2, 0], :on_tstring_content, " 1\n" + " ", BEG], [[3, 2], :on_embexpr_beg, "\#{", BEG], [[3, 4], :on_heredoc_beg, "<<~H2", BEG], [[3, 9], :on_embexpr_end, "}", END], [[3, 10], :on_tstring_content, "\n" + " 3\n", BEG], [[4, 0], :on_ignored_sp, " ", BEG], [[4, 4], :on_tstring_content, "2\n", BEG], [[5, 0], :on_heredoc_end, " H2\n", BEG], [[7, 0], :on_heredoc_end, "H1\n", BEG]] ``` Now the newline on line 3 gets concatenated with the content from line 6, and that token is dropped from the list entirely. This seems unintentional, but I'm not sure. -- https://bugs.ruby-lang.org/