- ruby-core - ml.ruby-lang.org

[ruby-core:116516] [Ruby master Bug#20229] Empty keyword splat in array not removed in ARGSPUSH case
by jeremyevans0 (Jeremy Evans) 20 Feb '24

20 Feb '24

Issue #20229 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20229: Empty keyword splat in array not removed in ARGSPUSH case https://bugs.ruby-lang.org/issues/20229 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- For the following code: ```ruby a = [] kw = {} [*a, **kw] ``` Ruby 2.0-3.3 return `[{}]`, when the desired result (since 2.7) is `[]`. I discovered this while working on an optimization for the ARGSCAT case, where `[*a, 1, **kw]` allocates two arrays instead of a single array. My optimization fixes that by adding a `pushtoarraykwsplat` instruction as a replacement for `newarraykwsplat`. I've used `pushtoarraykwsplat` in the ARGSPUSH case to fix this bug. My pull request for the bug fix and optimization: https://github.com/ruby/ruby/pull/9766 -- https://bugs.ruby-lang.org/

1 2

[ruby-core:116676] [Ruby master Bug#20258] Socket::IPV6_RECVERR for IPv6
by marek22k 20 Feb '24

20 Feb '24

Issue #20258 has been reported by marek22k (Marek Küthe). ---------------------------------------- Bug #20258: Socket::IPV6_RECVERR for IPv6 https://bugs.ruby-lang.org/issues/20258 * Author: marek22k (Marek Küthe) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hello, I am currently trying to receive ICMP error messages in Ruby. For IPv4 there is the `Socket::IP_RECVERR` flag, which can be set, but not for IPv6 (not found unter https://docs.ruby-lang.org/en/3.2/Socket/Constants.html for example), although this is provided for in POSIX (see https://manned.org/ipv6.7 under IPV6_RECVERR). The following can therefore be done for IPv4: ```rb sock.setsockopt Socket::IPPROTO_IP, Socket::IP_RECVERR, 1 ``` But not for IPv6: ``` sock.setsockopt Socket::IPPROTO_IPV6, Socket::IPV6_RECVERR, 1 ``` Could IPV6_RECVERR be included? Background: I want to try to build a traceroute implementation in Ruby. So far I have the following code: ```rb # frozen_string_literal: true # sharable_constant_value: literal require 'socket' sock = Socket.open(Socket::AF_INET6, Socket::SOCK_DGRAM, Socket::IPPROTO_UDP) sock.setsockopt Socket::IPPROTO_IPV6, Socket::IPV6_UNICAST_HOPS, 1 # sock.setsockopt Socket::IPPROTO_IPV6, Socket::IPV6_RECVERR, 1 dest = Socket.sockaddr_in 33435, '2a00:1450:4001:806::2003' pp sock.send "Test", 0, dest pp sock.recvmsg nil, Socket::MSG_ERRQUEUE ``` However, the `recvmsg` function is blocking (still receiving the ICMP error message) forever. I could imagine that this is due to the missing flag. Hence this request. -- https://bugs.ruby-lang.org/

1 2

[ruby-core:116878] [Ruby master Bug#18988] Ripper cannot parse some code that has regexp named capture
by yui-knk (Kaneko Yuichiro) 20 Feb '24

20 Feb '24

Issue #18988 has been updated by yui-knk (Kaneko Yuichiro). Status changed from Open to Closed This is solved by https://bugs.ruby-lang.org/issues/20257 (https://github.com/ruby/ruby/pull/9923) ---------------------------------------- Bug #18988: Ripper cannot parse some code that has regexp named capture https://bugs.ruby-lang.org/issues/18988#change-106913 * Author: tompng (tomoya ishida) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.0dev (2022-08-29T15:47:09Z :detached: cd7506593a) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Ripper does not seems to recognize regexp named capture defining new local variables, fails to parse some code. ~~~ruby requie 'ripper' # expect syntax ok, got syntax error Ripper.sexp("/(?<a>)/ =~ ''; x = a **a, a if false") # expect syntax error, got syntax ok Ripper.sexp("/(?<a>)/ =~ ''; x = a **a, **a if false") # "exit" is a method call, got string_literal Ripper.sexp("/(?<a>)/ =~ ''; a %(exit)") ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115678] [Ruby master Bug#20055] Ripper seems to skip some checks like `void value expression` and `duplicated variable name`
by tompng (tomoya ishida) 20 Feb '24

20 Feb '24

Issue #20055 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20055: Ripper seems to skip some checks like `void value expression` and `duplicated variable name` https://bugs.ruby-lang.org/issues/20055 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-12-08T07:01:01Z master f2c84c4884) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- These are syntax error ~~~ruby return + return # void value expression 1 in [a, a] # duplicated variable name 1 + (1 => [a, a]) # both ~~~ But Ripper.sexp returns S-expression, not nil ~~~ruby Ripper.sexp 'return + return' # => [:program, [[:return, [:args_add_block, [[:unary, :+@, [:return0]]], false]]]] Ripper.sexp '1 in [a, a]' # => [:program, [[:case, [:@int, "1", [1, 0]], [:in, [:aryptn, nil, [[:var_field, [:@ident, "a", [1, 6]]], [:var_field, [:@ident, "a", [1, 9]]]], nil, nil], nil, nil]]]] Ripper.sexp '1 + (1 => [a, a]) # both' # => [:program, [[:binary, [:@int, "1", [1, 0]], :+, [:paren, [[:case, [:@int, "1", [1, 5]], [:in, [:aryptn, nil, [[:var_field, [:@ident, "a", [1, 11]]], [:var_field, [:@ident, "a", [1, 14]]]], nil, nil], nil, nil]]]]]]] ~~~ -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116875] [Ruby master Feature#18576] Rename `ASCII-8BIT` encoding to `BINARY`
by Eregon (Benoit Daloze) 20 Feb '24

20 Feb '24

Issue #18576 has been updated by Eregon (Benoit Daloze). `BINARY (ASCII-8BIT)` seems a good compromise. The `/` seems potentially confusing for: `incompatible character encodings: BINARY / ASCII-8BIT and EUC-JP (Encoding::CompatibilityError)`. So I think using parenthesis is OK and clearer than `/`. ---------------------------------------- Feature #18576: Rename `ASCII-8BIT` encoding to `BINARY` https://bugs.ruby-lang.org/issues/18576#change-106910 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ### Context I'm now used to it, but something that confused me for years was errors such as: ```ruby >> "fée" + "\xFF".b (irb):3:in `+': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) ``` When you aren't that familiar with Ruby, it's really not evident that `ASCII-8BIT` basically means "no encoding" or "binary". And even when you know it, if you don't read carefully it's very easily confused with `US-ASCII`. The `Encoding::BINARY` alias is much more telling IMHO. ### Proposal Since `Encoding::ASCII_8BIT` has been aliased as `Encoding::BINARY` for years, I think renaming it to `BINARY` and then making asking `ASCII_8BIT` the alias would significantly improve usability without backward compatibility concerns. The only concern I could see would be the consistency with a handful of C API functions: - `rb_encoding *rb_ascii8bit_encoding(void)` - `int rb_ascii8bit_encindex(void)` - `VALUE rb_io_ascii8bit_binmode(VALUE io)` But that's for much more advanced users, so I don't think it's much of a concern. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116871] [Ruby master Bug#20284] Regexp::TimeoutError is not triggered by a specific Regexp
by jbeschi (jacopo beschi) 20 Feb '24

20 Feb '24

Issue #20284 has been reported by jbeschi (jacopo beschi). ---------------------------------------- Bug #20284: Regexp::TimeoutError is not triggered by a specific Regexp https://bugs.ruby-lang.org/issues/20284 * Author: jbeschi (jacopo beschi) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've noticed that the Regexp::TimeoutError is not triggered for this Regexp: ```ruby jacopo-37s-mb 3.3.0 ~ cat fail.rb require 'benchmark' p RUBY_DESCRIPTION Regexp.timeout = 0.1 puts "Regexp.timeout: #{Regexp.timeout}" re = /((a|aa)+)+b/ 10.times do p Benchmark.realtime { re.match?("a" * 900000000) } end ``` ``` jacopo-37s-mb 3.3.0 ~ ruby fail.rb "ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin22]" Regexp.timeout: 0.1 1.1425850000232458 1.0082139987498522 1.064377998933196 1.0745620001107454 1.0647990014404058 1.6172770000994205 1.396629998460412 1.2450549993664026 1.3292079996317625 1.5961690004915 ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116868] [Ruby master Feature#18576] Rename `ASCII-8BIT` encoding to `BINARY`
by byroot (Jean Boussier) 20 Feb '24

20 Feb '24

Issue #18576 has been updated by byroot (Jean Boussier). ``` >> "fée" + "fée".b (irb):8:in `+': incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT) (Encoding::CompatibilityError) ``` I don't mind `BINARY` being first or last. I'll adjust my PR. As for the consecutive parentheteses, what about: ``` >> "fée" + "fée".b (irb):8:in `+': incompatible character encodings: UTF-8 and BINARY / ASCII-8BIT (Encoding::CompatibilityError) ``` ---------------------------------------- Feature #18576: Rename `ASCII-8BIT` encoding to `BINARY` https://bugs.ruby-lang.org/issues/18576#change-106903 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ### Context I'm now used to it, but something that confused me for years was errors such as: ```ruby >> "fée" + "\xFF".b (irb):3:in `+': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) ``` When you aren't that familiar with Ruby, it's really not evident that `ASCII-8BIT` basically means "no encoding" or "binary". And even when you know it, if you don't read carefully it's very easily confused with `US-ASCII`. The `Encoding::BINARY` alias is much more telling IMHO. ### Proposal Since `Encoding::ASCII_8BIT` has been aliased as `Encoding::BINARY` for years, I think renaming it to `BINARY` and then making asking `ASCII_8BIT` the alias would significantly improve usability without backward compatibility concerns. The only concern I could see would be the consistency with a handful of C API functions: - `rb_encoding *rb_ascii8bit_encoding(void)` - `int rb_ascii8bit_encindex(void)` - `VALUE rb_io_ascii8bit_binmode(VALUE io)` But that's for much more advanced users, so I don't think it's much of a concern. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116864] [Ruby master Bug#20283] Build failed since Ruby 3.2.3 if Xcode.app was renamed to space contained name.
by watson1978 (Shizuo Fujita) 20 Feb '24

20 Feb '24

Issue #20283 has been reported by watson1978 (Shizuo Fujita). ---------------------------------------- Bug #20283: Build failed since Ruby 3.2.3 if Xcode.app was renamed to space contained name. https://bugs.ruby-lang.org/issues/20283 * Author: watson1978 (Shizuo Fujita) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When I renamed Xcode.app to space contained name (like `Xcode 15.2.0.app`), then I met build error with Ruby 3.2.3 and Ruby 3.3.0. (Ruby 3.2.2 can be built with successfully.) ## How to reproduce ``` $ sudo mv /Applications/Xcode.app '/Applications/Xcode 15.2.app' $ sudo xcode-select --switch '/Applications/Xcode 15.2.app' $ tar xzf ruby-3.2.3.tar.xz $ cd ruby-3.2.3 $ ./configure $ make V=1 : clang -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef -pipe -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/arm64-darwin23 -I./include -I. -I./enc/unicode/15.0.0 -I/opt/homebrew/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -o dmyenc.o -c dmyenc.c rustc --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C opt-level=3 -C overflow-checks=on '--out-dir=/Users/watson/prj/ruby-3.2.3/yjit/target/release/' ./yjit/src/lib.rs touch yjit/target/release/libyjit.a clang -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef -pipe -L. -L/opt/homebrew/lib -fstack-protector-strong -fstack-protector-strong -Wl,-pie -framework CoreFoundation main.o dmydln.o miniinit.o dmyext.o array.o ast.o bignum.o class.o compar.o compile.o complex.o cont.o debug.o debug_counter.o dir.o dln_find.o encoding.o enum.o enumerator.o error.o eval.o file.o gc.o hash.o inits.o io.o io_buffer.o iseq.o load.o marshal.o math.o memory_view.o mjit.o mjit_c.o node.o numeric.o object.o pack.o parse.o proc.o process.o ractor.o random.o range.o rational.o re.o regcomp.o regenc.o regerror.o regexec.o regparse.o regsyntax.o ruby.o scheduler.o shape.o signal.o sprintf.o st.o strftime.o string.o struct.o symbol.o thread.o time.o transcode.o transient_heap.o util.o variable.o version.o vm.o vm_backtrace.o vm_dump.o vm_sync.o vm_trace.o yjit.o coroutine/arm64/Context.o enc/ascii.o enc/us_ascii.o enc/unicode.o enc/utf_8.o enc/trans/newline.o explicit_bzero.o setproctitle.o addr2line.o yjit/target/release/libyjit.a -lgmp -ldl -lobjc -lpthread -lpthread -o miniruby ld: warning: ignoring duplicate libraries: '-lpthread' ld: Undefined symbols: _coroutine_transfer, referenced from: _fiber_setcontext in cont.o clang: error: linker command failed with exit code 1 (use -v to see invocation) ``` ## Environment - Apple M1 Max - macOS 14.3.1 - clang 15.0.0 (clang-1500.1.0.2.5) - rustc 1.75.0 ## Note This problem occurs on Intel Mac and Apple M1 Mac both. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116670] [Ruby master Feature#20257] Rearchitect Ripper
by yui-knk (Kaneko Yuichiro) 20 Feb '24

20 Feb '24

Issue #20257 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Feature #20257: Rearchitect Ripper https://bugs.ruby-lang.org/issues/20257 * Author: yui-knk (Kaneko Yuichiro) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Rearchitect Ripper to provide whole semantic analysis support for Ripper and improve maintainability of Ripper. This rearchitecture is achieved by modifying Lrama parser generator. # Background and problem Ripper is used for parsing ruby code, for example irb and rdoc use Ripper for parsing source codes. Ripper and Ruby parser share the algorithm of parsing, however internal logic of Ripper is different from parser. The differences cause three problems: 1. Ripper can not execute some semantic analysis. https://bugs.ruby-lang.org/issues/10436 is an example of this limitaion. `m(&nil) {}` raises syntax error but `Ripper.sexp("m(&nil) {}")` doesn't. 2. Ripper can not recognize regexp named capture. https://bugs.ruby-lang.org/issues/18988 is an example. 3. Makes prase.y complex. For example, the implementation of `new_array_pattern` is completely different between [parser](https://github.com/ruby/ruby/blob/1949a04f81311660e2d0ec002c48115c6… and [ripper](https://github.com/ruby/ruby/blob/1949a04f81311660e2d0ec002c48115c6…. These problems come from the fact parser and Ripper use semantic value stack differently. Parser stores Node on the stack in many rules but Ripper stores Ruby Object returned by callback method. Therefore Ripper can not execute semantic analysis which requires Node (#1). Values on the stack are different then they need to implement same name functions differently (#3). This leads different behavior like #2 because they have different `match_op` function. # Proposal Introduce new semantic value stack for Ripper so that Ripper can manage both Node and Ruby Object separately. Lrama will provide some callback entry points and new special variable for actions. Lrama will support these callback directives, specified function is called when the event happens * %after-shift function_name * %before-reduce function_name * %after-reduce function_name * %after-shift-error-token function_name * %after-pop-stack function_name Lrama also provides `$:n` variable to access index of each grammar symbols. The variable is translated to the minus index from the top of the stack. For example ```c primary: k_if expr_value then compstmt if_tail k_end { /*% ripper: if!($:2, $:4, $:5) %*/ /* $:2 = -5, $:4 = -3, $:5 = -2. */ } ``` We can implement separated stack for Ruby Object by these features. # Implementation note ## New fields of struct parser_params * `VALUE s_value`: Holds Ruby Object returned by Ripper callback method call. * `VALUE s_lvalue`: Holds Ruby Object responding to LHS of the rule. * `VALUE s_value_stack`: Stack for Ruby Object. It's actually ruby array. These fields are added only when it's Ripper. ## The role of callback functions * %after-shift: Push `s_value` to `s_value_stack`. * %before-reduce: Assign the first Ruby Object of RHS to `s_lvalue` (similar with `$$ = $1`). * %after-reduce: Pop `s_value_stack` `rhs.len` times then push `s_lvalue` to `s_value_stack`. * %after-shift-error-token: Push `nil` to `s_value_stack`. This `nil` stands for `error` token. * %after-pop-stack: Pop `s_value_stack` `len` times. This is needed to emulate panic mode. # Achievement These bugs are fixed. * [Bug 10436](https://bugs.ruby-lang.org/issues/10436) "ruby -c and ripper inconsistency: m(&nil) {}" * [Bug 18988](https://bugs.ruby-lang.org/issues/18988) "Ripper cannot parse some code that has regexp named capture" * [Bug 20055](https://bugs.ruby-lang.org/issues/20055) "Ripper seems to skip some checks like `void value expression` and `duplicated variable name`" This means [all of Ripper open bugs tickets](https://bugs.ruby-lang.org/projects/ruby-master/issues?utf8=%E2%9C… which are related current architecture will be closed. # Links * Lrama Implementaion: https://github.com/ruby/lrama/pull/367 * Ruby Implementaion: TBD -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116857] [Ruby master Feature#18576] Rename `ASCII-8BIT` encoding to `BINARY`
by duerst 20 Feb '24

20 Feb '24

Issue #18576 has been updated by duerst (Martin Dürst). What about ``` >> "fée" + "fée".b (irb):8:in `+': incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT) (Encoding::CompatibilityError) ``` This still leaves "ASCII-8BIT" in (because I think it's important to help people understand that BINARY and ASCII-8BIT are the same). [It also keeps the wart of consecutive parentheticals, but that can be dealt with separately.] ---------------------------------------- Feature #18576: Rename `ASCII-8BIT` encoding to `BINARY` https://bugs.ruby-lang.org/issues/18576#change-106889 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ### Context I'm now used to it, but something that confused me for years was errors such as: ```ruby >> "fée" + "\xFF".b (irb):3:in `+': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) ``` When you aren't that familiar with Ruby, it's really not evident that `ASCII-8BIT` basically means "no encoding" or "binary". And even when you know it, if you don't read carefully it's very easily confused with `US-ASCII`. The `Encoding::BINARY` alias is much more telling IMHO. ### Proposal Since `Encoding::ASCII_8BIT` has been aliased as `Encoding::BINARY` for years, I think renaming it to `BINARY` and then making asking `ASCII_8BIT` the alias would significantly improve usability without backward compatibility concerns. The only concern I could see would be the consistency with a handful of C API functions: - `rb_encoding *rb_ascii8bit_encoding(void)` - `int rb_ascii8bit_encindex(void)` - `VALUE rb_io_ascii8bit_binmode(VALUE io)` But that's for much more advanced users, so I don't think it's much of a concern. -- https://bugs.ruby-lang.org/

1 0