Issue #19291 has been updated by nobu (Nobuyoshi Nakada). This may be a potential bug since older versions. ---------------------------------------- Bug #19291: Ripper's on_parse_error token has wrong content https://bugs.ruby-lang.org/issues/19291#change-100934 * Author: tompng (tomoya ishida) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-darwin20] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Ruby 3.1 ~~~ruby Ripper::Lexer.new("{a:,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\nhelloworld").scan.map(&:tok) # => ["{", "a:", ",", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "", "\n", "helloworld"] ~~~ Ruby 3.2.0 ~~~ruby Ripper::Lexer.new("{a:,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\nhelloworld").scan.map(&:tok) # => ["{", "a:", ",", "oworld\u0000\u0000\u0000\u0000\u0000\u0000z`\u0001\u0000\u0000\u0000\u0000\u0000\xC0\xF0$\u0010\u0001\u0000\u0000\u0000\x80A\xB8\u0019\xB5\u007F\u0000\u0000\x91\x94", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "\n", "helloworld"] ~~~ -- https://bugs.ruby-lang.org/