[ruby-core:127582] [Ruby Bug#22600] parse.y corrupts line-1 heredoc delimiter/resume offset by 3 bytes after a UTF-8 BOM
Issue #22600 has been reported by steviejj (Steven Johnstone). ---------------------------------------- Bug #22600: parse.y corrupts line-1 heredoc delimiter/resume offset by 3 bytes after a UTF-8 BOM https://bugs.ruby-lang.org/issues/22600 * Author: steviejj (Steven Johnstone) * Status: Open * ruby -v: ruby 4.0.9 (2026-10-23 revision 3e93b23afc) +PRISM [arm64-darwin27] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- When a Ruby script or `eval` string starts with a UTF-8 BOM (`\xEF\xBB\xBF`) and declares a heredoc on line 1 (e.g. `\xEF\xBB\xBF<<EOF\nEOF\n`, `\xEF\xBB\xBF(<<EOF)\nEOF\n`, `\xEF\xBB\xBFp(<<EOF)\nEOF\n`, or `\xEF\xBB\xBF[<<EOF]\nEOF\n`), CRuby `parse.y` miscomputes the heredoc identifier/resume offset on line 1 by 3 bytes (the length of the UTF-8 BOM), looking up `"..."` shifted 3 bytes to the left (such as `"(<<"`, `"[<<"`, or `"\xBF<<"`) and failing with `SyntaxError: can't find string ... anywhere before EOF` followed by `unexpected constant`. If a newline separates the BOM from the heredoc (`\xEF\xBB\xBF\n(<<EOF)\nEOF\n`), both `parse.y` and `--parser=prism` succeed. Discovered by differential fuzzing of prism against parse.y. ## Reproduction Tested on CRuby `4.1.0dev` (`2026-11-18 HEAD ed3e747aa4`) and `ruby 4.0.9 (2026-10-23 revision 3e93b23afc) +PRISM [arm64-darwin27]`. ```ruby eval("\xEF\xBB\xBF(<<EOF)\nEOF\n".b) # --parser=prism: nil (exits 0) # --parser=parse.y: SyntaxError: can't find string "(<<" anywhere before EOF ``` -- https://bugs.ruby-lang.org/
participants (1)
-
steviejj (Steven Johnstone)