February 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:111678] [Ruby master Feature#19315] Lazy substrings in CRuby
by Eregon (Benoit Daloze) 21 Aug '26

21 Aug '26

Issue #19315 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/

10 51

[ruby-core:116370] [Ruby master Bug#20203] `TestEnumerable` test failures with GCC 14
by vo.x (Vit Ondruch) 17 Aug '26

17 Aug '26

Issue #20203 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20203: `TestEnumerable` test failures with GCC 14 https://bugs.ruby-lang.org/issues/20203 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- There is ongoing mass rebuild in Fedora and that is first time GCC 14 is used and we observe test failures in `TestEnumerable`. Here are a few examples: ~~~ [ 3000/26419] TestEnumerable#test_transient_heap_sort_bymalloc_consolidate(): unaligned fastbin chunk detected ~~~ ~~~ [ 2455/26535] TestEnumerable#test_transient_heap_sort_bycorrupted size vs. prev_size in fastbins ~~~ ~~~ [ 9716/26532] TestEnumerable#test_any_with_unused_blockdouble free or corruption (fasttop) ~~~ The full logs are accessible [here](https://koji.fedoraproject.org/koji/taskinfo?taskID=112176941). Please drill through `Descendants` and `build.log` -- https://bugs.ruby-lang.org/

6 13

[ruby-core:117483] [Ruby master Bug#20419] [BUG] inconsistent freelist length with cppflags='-DRGENGC_CHECK_MODE -DUSE_RUBY_DEBUG_LOG=1'
by znz (Kazuhiro NISHIYAMA) 16 Aug '26

16 Aug '26

Issue #20419 has been reported by znz (Kazuhiro NISHIYAMA). ---------------------------------------- Bug #20419: [BUG] inconsistent freelist length with cppflags='-DRGENGC_CHECK_MODE -DUSE_RUBY_DEBUG_LOG=1' https://bugs.ruby-lang.org/issues/20419 * Author: znz (Kazuhiro NISHIYAMA) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `TestGc#test_gc_stress_at_startup` failed on my environment, so I check which configure flags affect. How to reproduce: ``` ./configure cppflags='-DRGENGC_CHECK_MODE -DUSE_RUBY_DEBUG_LOG=1' make miniruby env RUBY_DEBUG=gc_stress ./miniruby -v ``` I can't reproduce with `cppflags='-DRGENGC_CHECK_MODE'` or `cppflags='-DUSE_RUBY_DEBUG_LOG=1'`. ---Files-------------------------------- output.txt (9.44 KB) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116226] [Ruby master Bug#20189] `rb_str_resize` does not clear coderange when expanding
by tompng (tomoya ishida) 16 Aug '26

16 Aug '26

Issue #20189 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20189: `rb_str_resize` does not clear coderange when expanding https://bugs.ruby-lang.org/issues/20189 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2024-01-09T07:07:19Z master db476cc71c) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Expanding string in some encoding (utf16 utf32) can change coderange to either valid or broken, but rb_str_resize does not clear coderange. This will cause a bug in c-extension libraries that use rb_str_resize. ~~~ruby # Example for stringio s = StringIO.new("\0".encode('UTF-16LE')) s.truncate(1); s.truncate(2); s.string.valid_encoding? #=> true s.truncate(1); s.string.valid_encoding?; s.truncate(2); s.string.valid_encoding? #=> false (expect to be true) ~~~ -- https://bugs.ruby-lang.org/

5 6

[ruby-core:114943] [Ruby master Bug#19910] Set#delete_if behavior inconsistent with Array/Hash
by segiddins (Samuel Giddins) 16 Aug '26

16 Aug '26

Issue #19910 has been reported by segiddins (Samuel Giddins). ---------------------------------------- Bug #19910: Set#delete_if behavior inconsistent with Array/Hash https://bugs.ruby-lang.org/issues/19910 * Author: segiddins (Samuel Giddins) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Given the following script: ```ruby #!/usr/bin/env ruby require 'set' def enum(collection) i = 0 collection.delete_if do i += 1 raise ArgumentError if i == 6 i.odd? end rescue ArgumentError return collection end pp enum([0, 1, 2, 3, 4, 5, 6, 7]) pp enum([0, 1, 2, 3, 4, 5, 6, 7].to_h { [_1, _1] }) pp enum([0, 1, 2, 3, 4, 5, 6, 7].to_set) ``` It results in: ``` [1, 3, 5, 6, 7] {1=>1, 3=>3, 5=>5, 6=>6, 7=>7} #<Set: {0, 1, 2, 3, 4, 5, 6, 7}> ``` As you can see, when an exception is raise inside `delete_if` on Hash/Array, the already-considered elements are still removed from the array. For `Set`, no elements are deleted (due to the implementation that builds up an intermediary list of elements to delete, then removes them from the underlying hash afterwards). It would be very helpful if `Set#delete_if` behaved consistently with other core collection types -- https://bugs.ruby-lang.org/

5 4

[ruby-core:117658] [Ruby master Feature#20448] Make coverage event hooking C API public
by ms-tob (Matt S) 09 Aug '26

09 Aug '26

Issue #20448 has been reported by ms-tob (Matt S). ---------------------------------------- Feature #20448: Make coverage event hooking C API public https://bugs.ruby-lang.org/issues/20448 * Author: ms-tob (Matt S) * Status: Open ---------------------------------------- # Abstract Gathering code coverage information is a well-known goal within software engineering. It is most commonly used to assess code coverage during automated testing. A lesser known use-case is coverage-guided fuzz testing, which will be the primary use-case presented in this issue. This issue exists to request that Ruby coverage event hooking be made part of its official, public C API. # Background Ruby currently provides a number of avenues for hooking events *or* gathering coverage information: 1. The [Coverage](https://ruby-doc.org/3.3.0/exts/coverage/Coverage.html) module 2. The [TracePoint](https://ruby-doc.org/3.3.0/TracePoint.html) module 3. The [rb_add_event_hook](https://ruby-doc.org/3.3.0/extension_rdoc.html#label-Hoo… extension function Unfortunately, none of these pieces of functionality solve this issue's specific use-case. The `Coverage` module is not a great fit for real-time coverage analysis with an unknown start and stop point. Coverage-guided fuzz testing requires this. The `TracePoint` module and `rb_add_event_hook` are not able to hook branch and line coverage events. Coverage-guided fuzz testing typically tracks branch events. # Proposal The ultimate goal is to enable Ruby C extensions to process coverage events in real-time. I did some cursory investigation into the Ruby C internals to determine what it would take to achieve this, but I'm by no means an expert, so my list may be incomplete. The good news is that much of this functionality already exists, but it's part of the private, internal-only C API. 1. Make `RUBY_EVENT_COVERAGE_LINE` and `RUBY_EVENT_COVERAGE_BRANCH` public: https://github.com/ruby/ruby/blob/v3_3_0/vm_core.h#L2182-L2184 a. This would be an addition to the current public event types: https://github.com/ruby/ruby/blob/v3_3_0/include/ruby/internal/event.h#L32-… 2. Allow initializing global coverage state so that coverage tracking can be fully enabled a. Currently, if `Coverage.setup` or `Coverage.start` is not called, then coverage events cannot be hooked. I do not fully understand why this is, but I believe it has something to do with `rb_get_coverages` and `rb_set_coverages`. If calls to `rb_get_coverages` return `NULL` (https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L641-L647, https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L864-L868) then coverage hooking will not be enabled. I believe the `Coverage` module initializes that state via a `rb_set_coverages` call here: https://github.com/ruby/ruby/blob/v3_3_0/ext/coverage/coverage.c#L112-L120. b. So, to achieve this goal, a C extension would need to be able to call `rb_set_coverages` or somehow initialize the global coverage state. I've actually been able to achieve this functionality by calling undocumented features and defining `RUBY_EVENT_COVERAGE_BRANCH`: ```c #include <ruby.h> #include <ruby/debug.h> #define RUBY_EVENT_COVERAGE_BRANCH 0x020000 // ... rb_event_flag_t events = RUBY_EVENT_COVERAGE_BRANCH; rb_event_hook_flag_t flags = ( RUBY_EVENT_HOOK_FLAG_SAFE | RUBY_EVENT_HOOK_FLAG_RAW_ARG ); rb_add_event_hook2( (rb_event_hook_func_t) event_hook_branch, events, counter_hash, flags ); ``` If I call `Coverage.setup(branches: true)`, and add this event hook, then branch hooking works as expected. `rb_add_event_hook2` will still respect the `RUBY_EVENT_COVERAGE_BRANCH` value if its passed. But it would be better if I could rely on official functionality rather than undocumented features. The above two points would be requirements for this functionality, but there's an additional nice-to-have: 3. Extend the public `tracearg` functionality to include additional coverage information a. Currently, `tracearg` offers information like `rb_tracearg_lineno` and `rb_tracearg_path`. It would be helpful if it also provided additional coverage information like `coverage.c`'s column information and a unique identifier for each branch. Currently, I can only use `(path, lineno)` as a unique identifier for a branch because that's what's offered by the public API, but more information like column number would be helpful for uniquely identify branches. Since there can be multiple `if` statements on a single line, this can provide ambiguous identification for a branch event. # Use cases This use-case was born out of a new coverage-guided Ruby fuzzer: https://github.com/trailofbits/ruzzy. You can read more about its implementation details here: https://blog.trailofbits.com/2024/03/29/introducing-ruzzy-a-coverage-guided…. You can also find the Ruby C extension code behind its implementation here: https://github.com/trailofbits/ruzzy/blob/v0.7.0/ext/cruzzy/cruzzy.c#L220-L…. So, the primary use-case here is enabling real-time, coverage-guided fuzz testing of Ruby code. However, as mentioned in the abstract, gathering code coverage information is useful in many domains. For example, it could enable new workflows in standard unit/integration test coverage. It could also enable gathering coverage information in real-time as an application is running. I see this as the most generalized form of gathering code coverage information, and something like the `Coverage` module as a specialized implementation. Another example, https://bugs.ruby-lang.org/issues/20282 may be solved by this more generalized solution. We are tracking this request downstream here: https://github.com/trailofbits/ruzzy/issues/9 # Discussion Fuzz testing is another tool in a testers toolbelt. It is an increasingly common way to improve software's robustness. Go has it built in to the standard library, Python has Atheris, Java has Jazzer, JavaScript has Jazzer.js, etc. OSS-Fuzz has helped identify and fix over 10,000 vulnerabilities and 36,000 bugs [using fuzzing](https://google.github.io/oss-fuzz/#trophies). Ruby deserves a good fuzzer, and improving coverage gathering would help achieve that goal. The `Coverage` module, `TracePoint` module, and `rb_add_event_hook` function seem like they could fulfill this goal. However, after deeper investigation, none of them fit the exact requirements for this use-case. # See also - https://bugs.ruby-lang.org/issues/20282 - https://github.com/google/atheris - https://security.googleblog.com/2020/12/how-atheris-python-fuzzer-works.html - https://github.com/CodeIntelligenceTesting/jazzer/ - https://www.code-intelligence.com/blog/java-fuzzing-with-jazzer - https://go.dev/doc/security/fuzz/ -- https://bugs.ruby-lang.org/

3 11

[ruby-core:120488] [Ruby master Feature#21005] Update the source location method to include line start/stop and column start/stop details
by bkuhlmann (Brooke Kuhlmann) 06 Aug '26

06 Aug '26

Issue #21005 has been reported by bkuhlmann (Brooke Kuhlmann). ---------------------------------------- Feature #21005: Update the source location method to include line start/stop and column start/stop details https://bugs.ruby-lang.org/issues/21005 * Author: bkuhlmann (Brooke Kuhlmann) * Status: Open ---------------------------------------- ## Why 👋 Hello. After discussing with Kevin Newton and Benoit Daloze in [Feature 20999](https://bugs.ruby-lang.org/issues/20999), I'd like to propose adding line start/stop and column start/stop information to the `#source_location` method for the following objects: - [Binding](https://docs.ruby-lang.org/en/master/Binding.html) - [Proc](https://docs.ruby-lang.org/en/master/Proc.html) - [Method](https://docs.ruby-lang.org/en/master/Method.html) - [UnboundMethod](https://docs.ruby-lang.org/en/master/UnboundMethod.html) At the moment, when using `#source_location`, you only get the following information: ``` ruby def demo = "A demonstration." # From disk. method(:demo).source_location # ["/Users/bkuhlmann/Engineering/Misc/demo", 15] # From memory. method(:demo).source_location # ["(irb)", 3] ``` Notice, when asking for the source location, we only get the path/location as the first element and the line number as the second element but I'd like to obtain a much richer set of data which includes line start/stop and column start/stop so I can avoid leaning on the `RubyVM` for this information. Example: ``` ruby def demo = "A demonstration." # From disk. instructions = RubyVM::InstructionSequence.of method(:demo) puts [instructions.absolute_path, *instructions.to_a.dig(4, :code_location)] [ "/Users/bkuhlmann/Engineering/Misc/demo", # Source path. 15, # Line start. 0, # Column start. 15, # Line stop. 29 # Column stop. ] # From memory. instructions = RubyVM::InstructionSequence.of method(:demo) puts instructions.script_lines [ "def demo = \"A demonstration.\"\n", "" ] ``` By having access to the path (or lack thereof in case of IRB), line start/stop, and column start/stop, this means we could avoid using the RubyVM to obtain raw source code for any of these objects. This would not only enhance debugging situations but also improve Domain Specific Languages that wish to leverage this information for introducing new features and/or new debugging capabilities to the language. ## How Building upon the examples provided above, I'd like to see `Binding`, `Proc`, `Method`, and `UnboundMethod` respond to `#source_location` as follows: ``` ruby [ "/Users/bkuhlmann/Engineering/Misc/demo", # Source path. 15, # Line start. 15, # Line stop. 0, # Column start. 29 # Column stop. ] ``` Notice, for data grouping purposes, I changed the array structure to always start with the path as the first element, followed by line information, and ending with column information. Alternatively, it could might be nice to improve upon the above by answering a hash each time, instead, for a more self-describing data structure. Example: ``` ruby { path: "/Users/bkuhlmann/Engineering/Misc/demo", line_start: 15, line_stop: 15, column_start: 0, column_stop: 29 } ``` For in-memory, situations like IRB, it would be nice to answer the equivalent of `RubyVM::InstructionSequence#script_lines` which would always be an `Array` with no line or column information since only the source code is necessary. Example: ``` ruby [ "def demo = \"A demonstration.\"\n", "" ] ``` From a pattern matching perspective, this could provide the best of both worlds especially if information is answered as either a `Hash` or and `Array`. Example: ``` def demo = "A demonstration." case method(:demo).source_location in Hash then puts "Source information obtained from disk." in Array then puts "Source obtained from memory." else fail TypeError, "Unrecognized source location type." end ``` This above is only a simple example but there's a lot we could do with this information if the above pattern match was enhanced to deal with the extraction and formatting of the actual source code! ## Notes This feature request is related to the following discussions in case more context is of help: - [Feature 6012](https://bugs.ruby-lang.org/issues/6012) - [Feature 20999](https://bugs.ruby-lang.org/issues/20999) -- https://bugs.ruby-lang.org/

8 21

[ruby-core:115016] [Ruby master Bug#19921] TestYJIT#test_bug_19316 test failure
by vo.x (Vit Ondruch) 03 Aug '26

03 Aug '26

Issue #19921 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19921: TestYJIT#test_bug_19316 test failure https://bugs.ruby-lang.org/issues/19921 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-12 master 52709a4862) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Testing with 52709a4862 on Fedora Rawhide, I observe following failure: ~~~ 94) Failure: TestYJIT#test_bug_19316 [/builddir/build/BUILD/ruby-3.3.0-52709a4862/test/ruby/test_yjit.rb:1126]: Expected no exits, but got {:opt_and=>1} ~~~ -- https://bugs.ruby-lang.org/

6 21

[ruby-core:112045] [Ruby master Bug#19378] Windows: Use less syscalls for faster require of big gems
by aidog (Andi Idogawa) 27 Jul '26

27 Jul '26

Issue #19378 has been reported by aidog (Andi Idogawa). ---------------------------------------- Bug #19378: Windows: Use less syscalls for faster require of big gems https://bugs.ruby-lang.org/issues/19378 * Author: aidog (Andi Idogawa) * Status: Open * Priority: Normal * ruby -v: 3.2.0 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello 🙂 ## Problem require is slow on windows for big gems. (example: require 'gtk3'=> 3 seconds+). This is a problem for people who want to make cross platform GUI apps with ruby. ## Possible Reason As touched on in [#15797](https://bugs.ruby-lang.org/issues/15797) it seems like require uses realpath, which is emulated on windows. It checks every parent directory. The same syscalls run many times. ## Testfile C:\tmp\speedtest\testrequire.rb: ``` ruby require __dir__ + "/helloworld1.rb" require __dir__ + "/helloworld2.rb" ``` ``` shell ruby --disable-gems C:\tmp\speedtest\testrequire.rb ``` ### Syscalls per File/Directory: 1. CreateFile 2. QueryInformationVolume 3. QueryIdInformation 4. QueryAllInformationFile 5. QueryNameInformationFile 6. QueryNameInformationFile 7. QueryNormalizedNameInformationFile 8. CloseFile ### Files/Directories checked 1. C:\tmp 2. C:\tmp\speedtest 3. C:\tmp\speedtest\helloworld1.rb 4. C:\tmp 5. C:\tmp\speedtest 6. C:\tmp\speedtest\helloworld2.rb For two required files Ruby had to do 8*6 = **48** syscalls. The syscalls orginate from rb_w32_reparse_symlink_p / lstat Rubygems live in subfolders with 9+ parts: "C:\Ruby32-x64\lib\ruby\gems\3.2.0\gems\glib2-4.0.8\lib\glib2\variant.rb" Each file takes 8 * 9 = **72**+ calls. For variant.rb it is **80** calls. The result for the syscalls don't change in such a short time, so it should be possible to cache it. With require_relative it's twice as many calls. ## Other testcases Same result: ``` ruby File.realpath __dir__ + "/helloworld1.rb" File.realpath __dir__ + "/helloworld2.rb" ``` ``` ruby File.stat __dir__ + "/helloworld1.rb" File.stat __dir__ + "/helloworld2.rb" ``` It does not happen in $LOAD_PATH.resolve_feature_path(__dir__ + "/helloworld1.rb") ## Request Would it be possible to cache the stat calls when using require? I tried to implement a cache inside the ruby source code, but failed. If not, is there now a way to combine ruby files into one? I previously talked about require here: [YJIT: Windows support lacking.](https://bugs.ruby-lang.org/issues/19325#note-11) ## How to reproduce Ruby versions: At least 3.0+, most likely older ones too. Tested using Ruby Installer 3.1 and 3.2. [Procmon Software by Sysinternals](https://learn.microsoft.com/en-us/sysinternals/downloads/proc… -- https://bugs.ruby-lang.org/

6 9

[ruby-core:116456] [Ruby master Feature#20215] Introduce `IO#readable?`
by ioquatix (Samuel Williams) 25 Jul '26

25 Jul '26

Issue #20215 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #20215: Introduce `IO#readable?` https://bugs.ruby-lang.org/issues/20215 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- There are some cases where, as an optimisation, it's useful to know whether more data is potentially available. We already have `IO#eof?` but the problem with using `IO#eof?` is that it can block indefinitely for sockets. Therefore, code which uses `IO#eof?` to determine if there is potentially more data, may hang. ```ruby def make_request(path = "/") client = connect_remote_host # HTTP/1.0 request: client.write("GET #{path} HTTP/1.0\r\n\r\n") # Read response client.gets("\r\n") # => "HTTP/1.0 200 OK\r\n" # Assuming connection close, there are two things the server can do: # 1. peer.close # 2. peer.write(...); peer.close if client.eof? # <--- Can hang here! puts "Connection closed" # Avoid yielding as we know there definitely won't be any data. else puts "Connection open, data may be available..." # There might be data available, so yield. yield(client) end ensure client&.close end make_request do |client| puts client.read # <--- Prefer to wait here. end ``` The proposed `IO#readable?` is similar to `IO#eof?` but rather than blocking, would simply return false. The expectation is the user will subsequently call `read` which may then wait. The proposed implementation would look something like this: ```ruby class IO def readable? !!self.closed? end end class BasicSocket # Is it likely that the socket is still connected? # May return false positive, but won't return false negative. def readable? return false unless super # If we can wait for the socket to become readable, we know that the socket may still be open. result = self.recv_nonblock(1, MSG_PEEK, exception: false) # No data was available - newer Ruby can return nil instead of empty string: return false if result.nil? # Either there was some data available, or we can wait to see if there is data avaialble. return !result.empty? || result == :wait_readable rescue Errno::ECONNRESET # This might be thrown by recv_nonblock. return false end end ``` For `IO` itself, when there is buffered data, `readable?` would also return true immediately, similar to `eof?`. This is not shown in the above implementation as I'm not sure if there is any Ruby method which exposes "there is buffered data". -- https://bugs.ruby-lang.org/

8 19