[ruby-core:126676] [Ruby Bug#22310] parse.y rejects a variable binding that precedes an unrelated alternative pattern
Issue #22310 has been reported by yuki300 (Yuki Hirota). ---------------------------------------- Bug #22310: parse.y rejects a variable binding that precedes an unrelated alternative pattern https://bugs.ruby-lang.org/issues/22310 * Author: yuki300 (Yuki Hirota) * Status: Open * ruby -v: ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- `parse.y` rejects patterns in which a variable binding appears *before* an alternative pattern, even when the alternative itself binds nothing. Prism accepts the same code. Is this intended? ## Reproduction ```ruby # ok.rb case [:a, 1] in [ :a | :b, x ] end # ng.rb case [1, :a] in [ x, :a | :b ] end ``` ``` $ ruby --parser=parse.y -c ok.rb Syntax OK $ ruby --parser=parse.y -c ng.rb ng.rb:2: alternative pattern after variable capture (SyntaxError) $ ruby --parser=prism -c ng.rb Syntax OK ``` The two patterns are semantically identical; only the order of the array elements differs. Both files run without error under the default parser. ## Other forms rejected by parse.y (all accepted by Prism) ```ruby in { a: String => t, b: Integer | String } # binding, then alternative in { a: String => t, b: { c: Integer | String } } # alternative nested deeper in [ one, "a" | "b" => two ] # form reported in rails/bootsnap issue 539 in { access_token: String => token, expires_in: (Integer | String) => expires_in } # form shipped in the anthropic gem ``` Reversing the order makes them pass: ```ruby in { b: Integer | String, a: String => t } # Syntax OK ``` For contrast, a binding *inside* the alternation is rejected by both parsers on 4.0.4, as expected: ```ruby in [ x, :a | y ] # variable capture in alternative pattern (parse.y and Prism) ``` ## Why this looks unintended `:a | :b` and `Integer | String` bind no variables, so the documented restriction -- "Binding to variables currently does NOT work for alternative patterns joined with `|`" -- is not violated. Nothing can be left unbound. The check in `p_alt` tests `p->ctxt.capture_in_pattern`, which is set by any binding anywhere in the same `in` clause, not only by a binding on the left-hand side of the `|`: ```c p_alt : p_alt[left] '|'[alt] { p->ctxt.in_alt_pattern = 1; } p_expr_basic[right] { if (p->ctxt.capture_in_pattern) { yyerror1(&@alt, "alternative pattern after variable capture"); } ``` Prism instead visits the left-hand node when it reaches the `|` (`parse_pattern_alternation_error` in prism.c), so sibling bindings are not affected. The check was introduced in https://github.com/ruby/ruby/commit/f4b6a5191ceb0ed0cd7a3e3c8bab24cc0dd15736 ([Feature #21572]). The discussion in #21572 and in the dev meeting (https://github.com/ruby/dev-meeting-log/blob/master/2025/DevMeeting-2025-10-...) only covers bindings *inside* an alternation, and `test/ruby/test_pattern_matching.rb` only asserts the passing order (`in [ :a | :b, x]`). The `p_alt` rule is unchanged on current master. ## Impact Not visible under the default parser. However, `RubyVM::InstructionSequence.compile_file` was routed to parse.y until [Bug #22023] was fixed, so bootsnap (which uses it to build its bytecode cache) made gems fail to load on Ruby 4.0.1-4.0.3 while `ruby foo.rb` worked. That path is fixed in 4.0.4 and worked around in bootsnap 1.24.2, but the parse.y check itself remains. ## Prior sightings The same rejection has been observed before and was each time attributed to the `compile_file` routing bug ([Bug #22023]): - https://github.com/rails/bootsnap/issues/539 (cause identified as https://github.com/ruby/ruby/pull/16463, later backported as #22023) - https://github.com/ruby/irb/issues/1212 ("in Ruby `4.0.2` `parse.y` has a bug, but the important part here is that this is using `parse.y`") - https://github.com/anthropics/anthropic-sdk-ruby/pull/192, whose author noticed that swapping the hash-pattern key order avoids the error As far as I can find, whether the parse.y check itself is correct has not been raised. ## Environment ``` ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] # rejects with --parser=parse.y ruby 4.0.3 (2026-04-21 revision 85ddef263a) +PRISM [arm64-darwin25] # same; also via compile_file ruby 3.4.9 (2026-03-11 revision 76cca827ab) +PRISM [arm64-darwin25] # parse.y accepts all of the above ``` -- https://bugs.ruby-lang.org/
Issue #22310 has been updated by Earlopain (Earlopain _). In short, https://github.com/ruby/ruby/pull/14923 made the check for `parse.y` too strict. It used to accept it, now doesn't anymore.
As far as I can find, whether the parse.y check itself is correct has not been raised.
The syntax check was intended to mirror the runtime check, so it's clearly a `parse.y` bug. ---------------------------------------- Bug #22310: parse.y rejects a variable binding that precedes an unrelated alternative pattern https://bugs.ruby-lang.org/issues/22310#change-118989 * Author: yuki300 (Yuki Hirota) * Status: Open * ruby -v: ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- `parse.y` rejects patterns in which a variable binding appears *before* an alternative pattern, even when the alternative itself binds nothing. Prism accepts the same code. Is this intended? ## Reproduction ```ruby # ok.rb case [:a, 1] in [ :a | :b, x ] end # ng.rb case [1, :a] in [ x, :a | :b ] end ``` ``` $ ruby --parser=parse.y -c ok.rb Syntax OK $ ruby --parser=parse.y -c ng.rb ng.rb:2: alternative pattern after variable capture (SyntaxError) $ ruby --parser=prism -c ng.rb Syntax OK ``` The two patterns are semantically identical; only the order of the array elements differs. Both files run without error under the default parser. ## Other forms rejected by parse.y (all accepted by Prism) ```ruby in { a: String => t, b: Integer | String } # binding, then alternative in { a: String => t, b: { c: Integer | String } } # alternative nested deeper in [ one, "a" | "b" => two ] # form reported in rails/bootsnap issue 539 in { access_token: String => token, expires_in: (Integer | String) => expires_in } # form shipped in the anthropic gem ``` Reversing the order makes them pass: ```ruby in { b: Integer | String, a: String => t } # Syntax OK ``` For contrast, a binding *inside* the alternation is rejected by both parsers on 4.0.4, as expected: ```ruby in [ x, :a | y ] # variable capture in alternative pattern (parse.y and Prism) ``` ## Why this looks unintended `:a | :b` and `Integer | String` bind no variables, so the documented restriction -- "Binding to variables currently does NOT work for alternative patterns joined with `|`" -- is not violated. Nothing can be left unbound. The check in `p_alt` tests `p->ctxt.capture_in_pattern`, which is set by any binding anywhere in the same `in` clause, not only by a binding on the left-hand side of the `|`: ```c p_alt : p_alt[left] '|'[alt] { p->ctxt.in_alt_pattern = 1; } p_expr_basic[right] { if (p->ctxt.capture_in_pattern) { yyerror1(&@alt, "alternative pattern after variable capture"); } ``` Prism instead visits the left-hand node when it reaches the `|` (`parse_pattern_alternation_error` in prism.c), so sibling bindings are not affected. The check was introduced in https://github.com/ruby/ruby/commit/f4b6a5191ceb0ed0cd7a3e3c8bab24cc0dd15736 ([Feature #21572]). The discussion in #21572 and in the dev meeting (https://github.com/ruby/dev-meeting-log/blob/master/2025/DevMeeting-2025-10-...) only covers bindings *inside* an alternation, and `test/ruby/test_pattern_matching.rb` only asserts the passing order (`in [ :a | :b, x]`). The `p_alt` rule is unchanged on current master. ## Impact Not visible under the default parser. However, `RubyVM::InstructionSequence.compile_file` was routed to parse.y until [Bug #22023] was fixed, so bootsnap (which uses it to build its bytecode cache) made gems fail to load on Ruby 4.0.1-4.0.3 while `ruby foo.rb` worked. That path is fixed in 4.0.4 and worked around in bootsnap 1.24.2, but the parse.y check itself remains. ## Prior sightings The same rejection has been observed before and was each time attributed to the `compile_file` routing bug ([Bug #22023]): - https://github.com/rails/bootsnap/issues/539 (cause identified as https://github.com/ruby/ruby/pull/16463, later backported as #22023) - https://github.com/ruby/irb/issues/1212 ("in Ruby `4.0.2` `parse.y` has a bug, but the important part here is that this is using `parse.y`") - https://github.com/anthropics/anthropic-sdk-ruby/pull/192, whose author noticed that swapping the hash-pattern key order avoids the error As far as I can find, whether the parse.y check itself is correct has not been raised. ## Environment ``` ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] # rejects with --parser=parse.y ruby 4.0.3 (2026-04-21 revision 85ddef263a) +PRISM [arm64-darwin25] # same; also via compile_file ruby 3.4.9 (2026-03-11 revision 76cca827ab) +PRISM [arm64-darwin25] # parse.y accepts all of the above ``` -- https://bugs.ruby-lang.org/
Issue #22310 has been updated by yuki300 (Yuki Hirota). Thanks, that matches what I was seeing. I got curious about *which* change actually started rejecting these, so I built the two commits around https://github.com/ruby/ruby/pull/15329 and ran the `ng.rb` from the description through `--parser=parse.y -c`: a211abbcbd (parent of PR 15329; its parse.y is identical to the PR 14923 merge) Syntax OK dfdc5d40ec (PR 15329) alternative pattern after variable capture So https://github.com/ruby/ruby/pull/14923 by itself was fine here. It reset `capture_in_pattern` on every `p_expr` reduction, so a binding in one array/hash element never leaked into its siblings. PR 15329 removed that reset to catch nested captures like `in [a] | 1`, and since then the flag just lives for the whole `in` clause. That's the point where `in [ x, :a | :b ]` started failing. I've put up https://github.com/ruby/ruby/pull/18830 with a fix. It does roughly what Prism does: the flag stays as a cheap pre-check, and when we hit `|` we walk the left operand's nodes and only raise if there really is an LASGN/DASGN in there. There's also a small `p_alt_begin` rule to save and restore `in_alt_pattern`, otherwise `in 1 | [ 2 | 3, a ]` would slip through once the false positive is gone. With the patch, every form I tried gives the same answer under parse.y and Prism, and test_pattern_matching.rb, test_syntax.rb, test_parse.rb and test/ripper pass on current master with `RUN_OPTS=--parser=parse.y`. I added assertions for the order that wasn't covered; the existing suite passes on the unpatched tree, which is why nothing caught this. One more thing I ran into while testing: `**rest` in a pattern never goes through `error_duplicate_pattern_variable`, so parse.y accepts both `{ a: x, **x }` and `1 | { a: 1, **rest }` at parse time while Prism rejects them. The duplicate case isn't new, 3.4.9's parse.y accepts it too (and Prism 3.4.9 already rejected it); the alternation case only shows up in 4.0 because that's when the parse-time check arrived. Since fixing it turns code parse.y has accepted for a long time into a SyntaxError, I kept it out of this fix and opened it separately as https://github.com/ruby/ruby/pull/18831. No ticket for that one yet; happy to file one if it should be tracked on its own. ---------------------------------------- Bug #22310: parse.y rejects a variable binding that precedes an unrelated alternative pattern https://bugs.ruby-lang.org/issues/22310#change-118997 * Author: yuki300 (Yuki Hirota) * Status: Open * ruby -v: ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- `parse.y` rejects patterns in which a variable binding appears *before* an alternative pattern, even when the alternative itself binds nothing. Prism accepts the same code. Is this intended? ## Reproduction ```ruby # ok.rb case [:a, 1] in [ :a | :b, x ] end # ng.rb case [1, :a] in [ x, :a | :b ] end ``` ``` $ ruby --parser=parse.y -c ok.rb Syntax OK $ ruby --parser=parse.y -c ng.rb ng.rb:2: alternative pattern after variable capture (SyntaxError) $ ruby --parser=prism -c ng.rb Syntax OK ``` The two patterns are semantically identical; only the order of the array elements differs. Both files run without error under the default parser. ## Other forms rejected by parse.y (all accepted by Prism) ```ruby in { a: String => t, b: Integer | String } # binding, then alternative in { a: String => t, b: { c: Integer | String } } # alternative nested deeper in [ one, "a" | "b" => two ] # form reported in rails/bootsnap issue 539 in { access_token: String => token, expires_in: (Integer | String) => expires_in } # form shipped in the anthropic gem ``` Reversing the order makes them pass: ```ruby in { b: Integer | String, a: String => t } # Syntax OK ``` For contrast, a binding *inside* the alternation is rejected by both parsers on 4.0.4, as expected: ```ruby in [ x, :a | y ] # variable capture in alternative pattern (parse.y and Prism) ``` ## Why this looks unintended `:a | :b` and `Integer | String` bind no variables, so the documented restriction -- "Binding to variables currently does NOT work for alternative patterns joined with `|`" -- is not violated. Nothing can be left unbound. The check in `p_alt` tests `p->ctxt.capture_in_pattern`, which is set by any binding anywhere in the same `in` clause, not only by a binding on the left-hand side of the `|`: ```c p_alt : p_alt[left] '|'[alt] { p->ctxt.in_alt_pattern = 1; } p_expr_basic[right] { if (p->ctxt.capture_in_pattern) { yyerror1(&@alt, "alternative pattern after variable capture"); } ``` Prism instead visits the left-hand node when it reaches the `|` (`parse_pattern_alternation_error` in prism.c), so sibling bindings are not affected. The check was introduced in https://github.com/ruby/ruby/commit/f4b6a5191ceb0ed0cd7a3e3c8bab24cc0dd15736 ([Feature #21572]). The discussion in #21572 and in the dev meeting (https://github.com/ruby/dev-meeting-log/blob/master/2025/DevMeeting-2025-10-...) only covers bindings *inside* an alternation, and `test/ruby/test_pattern_matching.rb` only asserts the passing order (`in [ :a | :b, x]`). The `p_alt` rule is unchanged on current master. ## Impact Not visible under the default parser. However, `RubyVM::InstructionSequence.compile_file` was routed to parse.y until [Bug #22023] was fixed, so bootsnap (which uses it to build its bytecode cache) made gems fail to load on Ruby 4.0.1-4.0.3 while `ruby foo.rb` worked. That path is fixed in 4.0.4 and worked around in bootsnap 1.24.2, but the parse.y check itself remains. ## Prior sightings The same rejection has been observed before and was each time attributed to the `compile_file` routing bug ([Bug #22023]): - https://github.com/rails/bootsnap/issues/539 (cause identified as https://github.com/ruby/ruby/pull/16463, later backported as #22023) - https://github.com/ruby/irb/issues/1212 ("in Ruby `4.0.2` `parse.y` has a bug, but the important part here is that this is using `parse.y`") - https://github.com/anthropics/anthropic-sdk-ruby/pull/192, whose author noticed that swapping the hash-pattern key order avoids the error As far as I can find, whether the parse.y check itself is correct has not been raised. ## Environment ``` ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] # rejects with --parser=parse.y ruby 4.0.3 (2026-04-21 revision 85ddef263a) +PRISM [arm64-darwin25] # same; also via compile_file ruby 3.4.9 (2026-03-11 revision 76cca827ab) +PRISM [arm64-darwin25] # parse.y accepts all of the above ``` -- https://bugs.ruby-lang.org/
Issue #22310 has been updated by yuki300 (Yuki Hiro). Filed the `**rest` one separately as #22314 (fix in https://github.com/ruby/ruby/pull/18831). ---------------------------------------- Bug #22310: parse.y rejects a variable binding that precedes an unrelated alternative pattern https://bugs.ruby-lang.org/issues/22310#change-118998 * Author: yuki300 (Yuki Hiro) * Status: Open * ruby -v: ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- `parse.y` rejects patterns in which a variable binding appears *before* an alternative pattern, even when the alternative itself binds nothing. Prism accepts the same code. Is this intended? ## Reproduction ```ruby # ok.rb case [:a, 1] in [ :a | :b, x ] end # ng.rb case [1, :a] in [ x, :a | :b ] end ``` ``` $ ruby --parser=parse.y -c ok.rb Syntax OK $ ruby --parser=parse.y -c ng.rb ng.rb:2: alternative pattern after variable capture (SyntaxError) $ ruby --parser=prism -c ng.rb Syntax OK ``` The two patterns are semantically identical; only the order of the array elements differs. Both files run without error under the default parser. ## Other forms rejected by parse.y (all accepted by Prism) ```ruby in { a: String => t, b: Integer | String } # binding, then alternative in { a: String => t, b: { c: Integer | String } } # alternative nested deeper in [ one, "a" | "b" => two ] # form reported in rails/bootsnap issue 539 in { access_token: String => token, expires_in: (Integer | String) => expires_in } # form shipped in the anthropic gem ``` Reversing the order makes them pass: ```ruby in { b: Integer | String, a: String => t } # Syntax OK ``` For contrast, a binding *inside* the alternation is rejected by both parsers on 4.0.4, as expected: ```ruby in [ x, :a | y ] # variable capture in alternative pattern (parse.y and Prism) ``` ## Why this looks unintended `:a | :b` and `Integer | String` bind no variables, so the documented restriction -- "Binding to variables currently does NOT work for alternative patterns joined with `|`" -- is not violated. Nothing can be left unbound. The check in `p_alt` tests `p->ctxt.capture_in_pattern`, which is set by any binding anywhere in the same `in` clause, not only by a binding on the left-hand side of the `|`: ```c p_alt : p_alt[left] '|'[alt] { p->ctxt.in_alt_pattern = 1; } p_expr_basic[right] { if (p->ctxt.capture_in_pattern) { yyerror1(&@alt, "alternative pattern after variable capture"); } ``` Prism instead visits the left-hand node when it reaches the `|` (`parse_pattern_alternation_error` in prism.c), so sibling bindings are not affected. The check was introduced in https://github.com/ruby/ruby/commit/f4b6a5191ceb0ed0cd7a3e3c8bab24cc0dd15736 ([Feature #21572]). The discussion in #21572 and in the dev meeting (https://github.com/ruby/dev-meeting-log/blob/master/2025/DevMeeting-2025-10-...) only covers bindings *inside* an alternation, and `test/ruby/test_pattern_matching.rb` only asserts the passing order (`in [ :a | :b, x]`). The `p_alt` rule is unchanged on current master. ## Impact Not visible under the default parser. However, `RubyVM::InstructionSequence.compile_file` was routed to parse.y until [Bug #22023] was fixed, so bootsnap (which uses it to build its bytecode cache) made gems fail to load on Ruby 4.0.1-4.0.3 while `ruby foo.rb` worked. That path is fixed in 4.0.4 and worked around in bootsnap 1.24.2, but the parse.y check itself remains. ## Prior sightings The same rejection has been observed before and was each time attributed to the `compile_file` routing bug ([Bug #22023]): - https://github.com/rails/bootsnap/issues/539 (cause identified as https://github.com/ruby/ruby/pull/16463, later backported as #22023) - https://github.com/ruby/irb/issues/1212 ("in Ruby `4.0.2` `parse.y` has a bug, but the important part here is that this is using `parse.y`") - https://github.com/anthropics/anthropic-sdk-ruby/pull/192, whose author noticed that swapping the hash-pattern key order avoids the error As far as I can find, whether the parse.y check itself is correct has not been raised. ## Environment ``` ruby 4.0.4 (2026-05-12 revision b89eb1bcbf) +PRISM [arm64-darwin25] # rejects with --parser=parse.y ruby 4.0.3 (2026-04-21 revision 85ddef263a) +PRISM [arm64-darwin25] # same; also via compile_file ruby 3.4.9 (2026-03-11 revision 76cca827ab) +PRISM [arm64-darwin25] # parse.y accepts all of the above ``` -- https://bugs.ruby-lang.org/
participants (3)
-
Earlopain (Earlopain _) -
yuki300 (Yuki Hiro)
-
yuki300 (Yuki Hirota)