May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118090] [Ruby master Bug#20513] the feature of kwargs in index methods has been removed without due consideration of utility and compatibility
by bughit (bug hit) 30 May '24

30 May '24

Issue #20513 has been reported by bughit (bug hit). ---------------------------------------- Bug #20513: the feature of kwargs in index methods has been removed without due consideration of utility and compatibility https://bugs.ruby-lang.org/issues/20513 * Author: bughit (bug hit) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- See #20218 The ability to pass kwargs to index methods has been in ruby for a long time, probably from the inception of kwargs, so there's code that makes use of it. Other than the multiple assignment edge-case it's been working fine and is not conceptually unsound. kwargs allow for more variability in store/lookup operations via index methods, letting you control where/how something is stored/looked up. this is from 2.6 ```ruby module IndexTest @store = {} def self.store @store end def self.key(name, namespace: nil) name = "#{namespace}:#{name}" if namespace name end def self.[](name, namespace: nil) p [name, namespace] @store[key(name, namespace: namespace)] end def self.[]=(name, opts = {}, val) p [name, opts, val] @store[key(name, namespace: opts[:namespace])] = val end end IndexTest['foo'] = 1 p IndexTest['foo'] IndexTest['foo', namespace: 'bar'] = 2 p IndexTest['foo', namespace: 'bar'] p IndexTest.store ``` A reasonable breaking change would be for `[]=` to have real kwargs, rather than the middle positional kwarg collector hash in the above example. I am not arguing that breaking changes can't be introduced, but that a removal of a long-standing feature deserves more consideration and deliberation than the following: > I found that use of keyword arguments in multiple assignment is broken > Can we also prohibit keyword arguments ... ? > OK, prohibit keyword arguments -- https://bugs.ruby-lang.org/

6 15

[ruby-core:117660] [Ruby master Bug#20449] Ripper issue in field production on Ruby 3.1.5
by kddnewton (Kevin Newton) 30 May '24

30 May '24

Issue #20449 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20449: Ripper issue in field production on Ruby 3.1.5 https://bugs.ruby-lang.org/issues/20449 * Author: kddnewton (Kevin Newton) * Status: Open * ruby -v: 3.1.5 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby Ripper.sexp("foo::bar = baz") # => [:program, [[:assign, [:field, [:vcall, [:@ident, "foo", [1, 0]]], 73, [:@ident, "bar", [1, 5]]], [:vcall, [:@ident, "baz", [1, 11]]]]]] ``` Note that 73 there. That's either `:"::"` or `[:@op, "::", [lineno, column]]` in every other version of Ruby that I could find. I assume this is a backport issue. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:115923] [Ruby master Bug#20094] Inline while loop behavior changed unexpectedly in 3.3.0
by sisyphus_cg (Sisyphus CG) 30 May '24

30 May '24

Issue #20094 has been reported by sisyphus_cg (Sisyphus CG). ---------------------------------------- Bug #20094: Inline while loop behavior changed unexpectedly in 3.3.0 https://bugs.ruby-lang.org/issues/20094 * Author: sisyphus_cg (Sisyphus CG) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The behavior of the inline while loop has changed in 3.3.0. This unexpectedly broke my code and I couldn't find anything in the changelog about it so reporting it as a bug. In ruby <= 3.2: ``` ruby (p 1) while false # nothing (p 1; p 2) while false # nothing ``` In ruby 3.3: ``` ruby (p 1) while false # 1 (p 1; p 2) while false # 1 2 ``` Essentially, if the left hand side looks like a statement, now it is treated as a do-while loop. In ruby 3.2 and less this only happened with explicit use of `begin` and `end`. -- https://bugs.ruby-lang.org/

9 11

[ruby-core:117970] [Ruby master Bug#20502] Backport pthread_kill fix to Ruby 3.3
by jeremyevans0 (Jeremy Evans) 30 May '24

30 May '24

Issue #20502 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20502: Backport pthread_kill fix to Ruby 3.3 https://bugs.ruby-lang.org/issues/20502 * Author: jeremyevans0 (Jeremy Evans) * Status: Closed * ruby -v: ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-openbsd] * Backport: 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- commit:ef3803ed4028810f9088019f0db1a366370ab53a should be backported to Ruby 3.3. I've now seen this bug affect multiple cases. On OpenBSD, there are test failures in by (https://github.com/jeremyevans/by) on Ruby 3.3.1, but not Ruby 3.4.0-preview1 (which includes the commit to backport). -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117652] [Ruby master Bug#20445] Backport 2571d5376a38c333b8fe85f1f9b2f9c2422c0ad9 to ruby_3_3
by mame (Yusuke Endoh) 29 May '24

29 May '24

Issue #20445 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20445: Backport 2571d5376a38c333b8fe85f1f9b2f9c2422c0ad9 to ruby_3_3 https://bugs.ruby-lang.org/issues/20445 * Author: mame (Yusuke Endoh) * Status: Closed * ruby -v: ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED ---------------------------------------- Building Ruby 3.3.1 prints the following warning: ``` regexec.c: In function 'match_at': regexec.c:2598:59: warning: 'stkp' may be used uninitialized [-Wmaybe-uninitialized] 2598 | if (*pbegin == OP_REPEAT_INC) stkp->u.repeat.count--;\ | ~~~~~~~~~~~~~~~~~~~~^~ regexec.c:3852:11: note: in expansion of macro 'CHECK_MATCH_CACHE' 3852 | CHECK_MATCH_CACHE; | ^~~~~~~~~~~~~~~~~ regexec.c:2297:18: note: 'stkp' was declared here 2297 | OnigStackType *stkp; /* used as any purpose. */ | ^~~~ ``` 2571d5376a38c333b8fe85f1f9b2f9c2422c0ad9 will remove it. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117510] [Ruby master Bug#20427] Heap buffer overflow in `Array#sort!` when block modifies target array
by zack.ref＠gmail.com (Zack Deveau) 29 May '24

29 May '24

Issue #20427 has been reported by zack.ref(a)gmail.com (Zack Deveau). ---------------------------------------- Bug #20427: Heap buffer overflow in `Array#sort!` when block modifies target array https://bugs.ruby-lang.org/issues/20427 * Author: zack.ref(a)gmail.com (Zack Deveau) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- **(note: It was decided we should handle this in the public issue tracker in security ticket #2327648)** The attached patch [has been applied to `master`](https://github.com/ruby/ruby/pull/10522) and should apply to latest `3.3.0` for backport. Could not reproduce on the following builds: - ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux] - ruby 3.1.4p249 (2024-01-11 revision 2b608349bb) [x86_64-linux] --- In cases where `rb_ary_sort_bang` is called with a block and `tmp` is an embedded array, we need to account for the block potentially impacting the capacity of `ary`. Reproduction script for x86 targets: ```ruby var_0 = (1..70).to_a var_0.sort! do |var_0_block_129, var_1_block_129| var_0.pop var_1_block_129 <=> var_0_block_129 end.shift(3) ``` Reproduction script for ARM targets: ```ruby 10.times do var_0 = (1..70).to_a var_0.sort! do |var_0_block_129, var_1_block_129| var_0.pop var_1_block_129 <=> var_0_block_129 end.shift(3) end ``` The above example can put the array into a corrupted state (`ary` after block has `len=0` and `capa=14`) : ``` ================== ary =================== ary: BD99908 is_embedded?: 0 is_shared?: 0 heap.len: 0 heap.capa: 14 heap.shared_root: 14 ================== tmp =================== ary: BD1EB18 is_embedded?: 1 is_shared?: 0 embed_len: 70 embed_capa: 78 heap.len: 141 heap.capa: 139 heap.shared_root: 139 ``` This results in a heap buffer overflow and possible segfault: ``` ==19964==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b0000034f0 at pc 0x00010c35ee6c bp 0x0003070fb290 sp 0x0003070faa50 WRITE of size 560 at 0x60b0000034f0 thread T0 #0 0x10c35ee6b in wrap_memcpy+0x2ab (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x18e6b) #1 0x100e0b085 in ruby_nonempty_memcpy memory.h:671 #2 0x100e0e43e in ary_memcpy0 array.c:335 #3 0x100e0cb00 in ary_memcpy array.c:352 #4 0x100e1426c in rb_ary_sort_bang array.c:3519 [ ... ] ``` Was able to reproduce on the following builds: - ruby 3.4.0dev (2024-01-17T14:48:46Z ef4a08eb65) [x86_64-linux] - ruby 3.3.0 (2024-01-05 revision 634d4e29ef) [x86_64-darwin23] - ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] This patch adds a conditional to determine when the capacity of `ary` has been modified by the provided block. If this is the case, ensure that the capacity of `ary` is adjusted to handle at minimum the len of `tmp`. `test-all` passes locally: ``` Finished tests in 70.194526s, 369.6727 tests/s, 89373.2939 assertions/s. 25949 tests, 6273516 assertions, 0 failures, 0 errors, 292 skips ``` ---Files-------------------------------- rb_ary_sort_bang_heap_overflow.patch (2.06 KB) -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116901] [Ruby master Bug#20289] Bug in Zlib::GzipReader#eof? breaks reading certain sizes of gzipped files.
by martinemde (Martin Emde) 29 May '24

29 May '24

Issue #20289 has been reported by martinemde (Martin Emde). ---------------------------------------- Bug #20289: Bug in Zlib::GzipReader#eof? breaks reading certain sizes of gzipped files. https://bugs.ruby-lang.org/issues/20289 * Author: martinemde (Martin Emde) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hello, A bug in the implementation of Zlib::GzipReader#eof? makes it very difficult to read certain rubygems using readpartial without receiving an EOFError. The bug is caused when the chunk size read from the gzip leaves only empty gzip "overhead" bytes at the end of the unread portion. The result is that a simple `readpartial until eof?` fails on some real world gems. ### Bug Explanation Imagine a gzip file, it has compressed data and some bytes that indicate the start and end of a chunk and then end of a file. ``` gzip file = "[... 2048 chunk of gzip data ...][ \0 bytes, chucksum info, with no new readable bytes ]" ``` When reading this file, zlib.c will pull a chunk of data according to READ_SIZE and return that data uncompressed. With exactly the right size of data (within exactly the range of READ-SIZE*n - 24 to READ_SIZE*n - 15 for any multiple of READ_SIZE) all of the file data is read and returned on the first read. The bug happens when there are bytes remaining on the gzip file that haven't been read, but contain no new data that can be returned to the buffer. Before reading the final empty chunk, #eof? returns false. After reading the final chunk, EOFError is raised because no bytes were returned into the buffer. What should happen is similar to how a Socket eof? is checked. On a socket, #eof? is not "passive" but actively reads ahead, filling the buffer. If that read-ahead fails to find new data, then the EOF is reached and eof? returns true. In zlib, #eof? does not read ahead to check if the end has been reached, causing this bug. ### Solution Samuel Giddins and I have both submitted alternate, but very similar solutions for this bug. [Samuel's PR](https://github.com/ruby/zlib/pull/73) and [My PR](https://github.com/ruby/zlib/pull/72). We'd leave it to ruby core to decide which is the best solution. Unfortunately, these bug fix PRs have been waiting for more than 2 months now with no movement on fixing this bug. The [original issue](https://github.com/ruby/zlib/issues/56) that I opened to point out this bug was submitted Aug 16, 2023. We've been working to find a solution for this bug for more than half a year. This bug was discovered because rubygems wanted to improve the efficiency of reading a gem by using readpartial to conserve memory. Sam submitted an additional PR that vastly improves the performance of the zlib gem by managing buffers better. We've already proven that this dramatically decreases memory usage when parsing gems. The [performance PR by Samuel](https://github.com/ruby/zlib/pull/61) has been open since September 13th, 2023. ### Plea We've been trying for months to get this solved. Opening a ruby-lang bug is my next attempt after communicating through our contacts at Ruby core has gone nowhere. Please merge and release these fixes, or tell us what is wrong with them so we can fix them so they can be merged. It will improve rubygems for everyone. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:117047] [Ruby master Bug#20324] `(1..).overlap?('foo'..)` returns true
by kyanagi (Kouhei Yanagita) 29 May '24

29 May '24

Issue #20324 has been reported by kyanagi (Kouhei Yanagita). ---------------------------------------- Bug #20324: `(1..).overlap?('foo'..)` returns true https://bugs.ruby-lang.org/issues/20324 * Author: kyanagi (Kouhei Yanagita) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- While thinking about finding the intersection of two ranges, I found that `(1..).overlap?('foo'..)` returns true. In the current implementation, it seems that `(a..).overlap?(b..)` or `(..a).overlap?(..b)` returns true regardless of what `a` or `b` are. However, I think it should return true if and only if `a` and `b` are comparable. (What is the intersection of `1..` and `'foo'..`?) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118061] [Ruby master Feature#6648] Provide a standard API for retrieving all command-line flags passed to Ruby
by Dan0042 (Daniel DeLorme) 29 May '24

29 May '24

Issue #6648 has been updated by Dan0042 (Daniel DeLorme). > * change `RbConfig.ruby` to be the current ruby interpreter (because TBH I'm not sure what's the use of this current `RbConfig.ruby`) @nobu what are your thoughts on the above? For example in the test suite, in `test/set/test_sorted_set.rb` we can see `r = system(RbConfig.ruby, *options, '-e', ruby)` and it seems to me like that's wrong; the system call is executing the installed ruby rather than the compiled ruby that is supposedly under test. If you think this is correct and it's fine that `RbConfig.ruby` returns a static path, we need a different place for `ruby_args` Or if you're not ok with relaxing the semantics of RbConfig then we also need a different place, maybe `Process.ruby_args` ---------------------------------------- Feature #6648: Provide a standard API for retrieving all command-line flags passed to Ruby https://bugs.ruby-lang.org/issues/6648#change-108488 * Author: headius (Charles Nutter) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- Currently there are no standard mechanisms to get the flags passed to the currently running Ruby implementation. The available mechanisms are not ideal: * Scanning globals and hoping they have not been tweaked to new settings * Using external wrappers to launch Ruby * ??? Inability to get the full set of command-line flags, including flags passed to the VM itself (and probably VM-specific) makes it impossible to launch subprocess Ruby instances with the same settings. A real world example of this is "((%bundle exec%))" when called with a command line that sets various flags, a la ((%jruby -Xsome.vm.setting --1.9 -S bundle exec%)). None of these flags can propagate to the subprocess, so odd behaviors result. The only option is to put the flags into an env var (((|JRUBY_OPTS|)) or ((|RUBYOPT|))) but this breaks the flow of calling a simple command line. JRuby provides mechanisms to get all its command line options, but they require calling Java APIs from Ruby's API set. Rubinius provides its own API for accessing comand-line options, but I do not know if it includes VM-level flags as well as standard Ruby flags. I know there is a (({RubyVM})) namespace in the 2.0 line. If that namespace is intended to be general-purpose for VM-level features, it would be a good host for this API. Something like... ``` class << RubyVM def vm_args; end # returns array of command line args *not* passed to the target script def script; end # returns the script being executed...though this overlaps with $0 def script_args; end # returns args passed to the script...though this overlaps with ARGV, but that is perhaps warranted since ARGV can be modified (i.e. you probably want the original args) end ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116888] [Ruby master Bug#20285] Stale inline method caches when refinement modules are reloaded
by jhawthorn (John Hawthorn) 29 May '24

29 May '24

Issue #20285 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #20285: Stale inline method caches when refinement modules are reloaded https://bugs.ruby-lang.org/issues/20285 * Author: jhawthorn (John Hawthorn) * Status: Assigned * Priority: Normal * Assignee: jhawthorn (John Hawthorn) * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: UNKNOWN ---------------------------------------- This is essentially the same issue as #11672, but for inline method caches rather than class caches. In Ruby 3.3 we started using inline caches for refinements. However, we weren't clearing inline caches when defined on a reopened refinement module. ``` ruby class C end module R refine C do def m :foo end end end using R def m C.new.m end raise unless :foo == m() module R refine C do alias m m def m :bar end end end v = m() raise "expected :bar, got #{v.inspect}" unless :bar == v ``` This will raise in Ruby 3.3 as the inline cache finds a stale refinement, but passes in previous versions. -- https://bugs.ruby-lang.org/

2 2