June 2026 - ruby-core - ml.ruby-lang.org

[ruby-core:125803] [Ruby Bug#22124] void value missed in parse.y
by kddnewton (Kevin Newton) 09 Aug '26

09 Aug '26

Issue #22124 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #22124: void value missed in parse.y https://bugs.ruby-lang.org/issues/22124 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ``` $ ruby --parser=parse.y -cwe 'x = return a b do end' Syntax OK $ ruby --parser=prism -cwe 'x = return a b do end' ruby: -e:1: syntax error found (SyntaxError) > 1 | x = return a b do end | ^~~~~~~~~~~~~~~~~ unexpected void value expression ``` I think it's likely because of the wrapped block expression. -- https://bugs.ruby-lang.org/

5 4

[ruby-core:124579] [Ruby Bug#21842] Encoding of rb_interned_str
by herwin (Herwin W) 09 Aug '26

09 Aug '26

Issue #21842 has been reported by herwin (Herwin W). ---------------------------------------- Bug #21842: Encoding of rb_interned_str https://bugs.ruby-lang.org/issues/21842 * Author: herwin (Herwin W) * Status: Open * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- This is one of the API methods to get an fstring. The documentation in the source says the following: ```c /** * Identical to rb_str_new(), except it returns an infamous "f"string. What is * a fstring? Well it is a special subkind of strings that is immutable, * deduped globally, and managed by our GC. It is much like a Symbol (in fact * Symbols are dynamic these days and are backended using fstrings). This * concept has been silently introduced at some point in 2.x era. Since then * it gained wider acceptance in the core. Starting from 3.x extension * libraries can also generate ones. * * @param[in] ptr A memory region of `len` bytes length. * @param[in] len Length of `ptr`, in bytes, not including the * terminating NUL character. * @exception rb_eArgError `len` is negative. * @return A found or created instance of ::rb_cString, of `len` bytes * length, of "binary" encoding, whose contents are identical to * that of `ptr`. * @pre At least `len` bytes of continuous memory region shall be * accessible via `ptr`. */ VALUE rb_interned_str(const char *ptr, long len); ``` I tried to create some specs for them (https://github.com/ruby/spec/pull/1327) but instead of binary encoding, the string is actually encoded as US-ASCII. This may result is some weird behaviour if the input contains bytes that are not valid in US-ASCII (the following is more an observation of the current behaviour) ```ruby it "support binary strings that are invalid in ASCII encoding" do str = "foo\x81bar\x82baz".b result = @s.rb_interned_str(str, str.bytesize) result.encoding.should == Encoding::US_ASCII result.should == str.dup.force_encoding(Encoding::US_ASCII) result.should_not.valid_encoding? end ``` So it seems to me like either the implementation of the documentation is incorrect. (`rb_interned_str_cstr` has the same behaviour, it's pretty much the same thing except using a null terminator instead of an explicit length argument). -- https://bugs.ruby-lang.org/

6 8

[ruby-core:117658] [Ruby master Feature#20448] Make coverage event hooking C API public
by ms-tob (Matt S) 09 Aug '26

09 Aug '26

Issue #20448 has been reported by ms-tob (Matt S). ---------------------------------------- Feature #20448: Make coverage event hooking C API public https://bugs.ruby-lang.org/issues/20448 * Author: ms-tob (Matt S) * Status: Open ---------------------------------------- # Abstract Gathering code coverage information is a well-known goal within software engineering. It is most commonly used to assess code coverage during automated testing. A lesser known use-case is coverage-guided fuzz testing, which will be the primary use-case presented in this issue. This issue exists to request that Ruby coverage event hooking be made part of its official, public C API. # Background Ruby currently provides a number of avenues for hooking events *or* gathering coverage information: 1. The [Coverage](https://ruby-doc.org/3.3.0/exts/coverage/Coverage.html) module 2. The [TracePoint](https://ruby-doc.org/3.3.0/TracePoint.html) module 3. The [rb_add_event_hook](https://ruby-doc.org/3.3.0/extension_rdoc.html#label-Hoo… extension function Unfortunately, none of these pieces of functionality solve this issue's specific use-case. The `Coverage` module is not a great fit for real-time coverage analysis with an unknown start and stop point. Coverage-guided fuzz testing requires this. The `TracePoint` module and `rb_add_event_hook` are not able to hook branch and line coverage events. Coverage-guided fuzz testing typically tracks branch events. # Proposal The ultimate goal is to enable Ruby C extensions to process coverage events in real-time. I did some cursory investigation into the Ruby C internals to determine what it would take to achieve this, but I'm by no means an expert, so my list may be incomplete. The good news is that much of this functionality already exists, but it's part of the private, internal-only C API. 1. Make `RUBY_EVENT_COVERAGE_LINE` and `RUBY_EVENT_COVERAGE_BRANCH` public: https://github.com/ruby/ruby/blob/v3_3_0/vm_core.h#L2182-L2184 a. This would be an addition to the current public event types: https://github.com/ruby/ruby/blob/v3_3_0/include/ruby/internal/event.h#L32-… 2. Allow initializing global coverage state so that coverage tracking can be fully enabled a. Currently, if `Coverage.setup` or `Coverage.start` is not called, then coverage events cannot be hooked. I do not fully understand why this is, but I believe it has something to do with `rb_get_coverages` and `rb_set_coverages`. If calls to `rb_get_coverages` return `NULL` (https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L641-L647, https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L864-L868) then coverage hooking will not be enabled. I believe the `Coverage` module initializes that state via a `rb_set_coverages` call here: https://github.com/ruby/ruby/blob/v3_3_0/ext/coverage/coverage.c#L112-L120. b. So, to achieve this goal, a C extension would need to be able to call `rb_set_coverages` or somehow initialize the global coverage state. I've actually been able to achieve this functionality by calling undocumented features and defining `RUBY_EVENT_COVERAGE_BRANCH`: ```c #include <ruby.h> #include <ruby/debug.h> #define RUBY_EVENT_COVERAGE_BRANCH 0x020000 // ... rb_event_flag_t events = RUBY_EVENT_COVERAGE_BRANCH; rb_event_hook_flag_t flags = ( RUBY_EVENT_HOOK_FLAG_SAFE | RUBY_EVENT_HOOK_FLAG_RAW_ARG ); rb_add_event_hook2( (rb_event_hook_func_t) event_hook_branch, events, counter_hash, flags ); ``` If I call `Coverage.setup(branches: true)`, and add this event hook, then branch hooking works as expected. `rb_add_event_hook2` will still respect the `RUBY_EVENT_COVERAGE_BRANCH` value if its passed. But it would be better if I could rely on official functionality rather than undocumented features. The above two points would be requirements for this functionality, but there's an additional nice-to-have: 3. Extend the public `tracearg` functionality to include additional coverage information a. Currently, `tracearg` offers information like `rb_tracearg_lineno` and `rb_tracearg_path`. It would be helpful if it also provided additional coverage information like `coverage.c`'s column information and a unique identifier for each branch. Currently, I can only use `(path, lineno)` as a unique identifier for a branch because that's what's offered by the public API, but more information like column number would be helpful for uniquely identify branches. Since there can be multiple `if` statements on a single line, this can provide ambiguous identification for a branch event. # Use cases This use-case was born out of a new coverage-guided Ruby fuzzer: https://github.com/trailofbits/ruzzy. You can read more about its implementation details here: https://blog.trailofbits.com/2024/03/29/introducing-ruzzy-a-coverage-guided…. You can also find the Ruby C extension code behind its implementation here: https://github.com/trailofbits/ruzzy/blob/v0.7.0/ext/cruzzy/cruzzy.c#L220-L…. So, the primary use-case here is enabling real-time, coverage-guided fuzz testing of Ruby code. However, as mentioned in the abstract, gathering code coverage information is useful in many domains. For example, it could enable new workflows in standard unit/integration test coverage. It could also enable gathering coverage information in real-time as an application is running. I see this as the most generalized form of gathering code coverage information, and something like the `Coverage` module as a specialized implementation. Another example, https://bugs.ruby-lang.org/issues/20282 may be solved by this more generalized solution. We are tracking this request downstream here: https://github.com/trailofbits/ruzzy/issues/9 # Discussion Fuzz testing is another tool in a testers toolbelt. It is an increasingly common way to improve software's robustness. Go has it built in to the standard library, Python has Atheris, Java has Jazzer, JavaScript has Jazzer.js, etc. OSS-Fuzz has helped identify and fix over 10,000 vulnerabilities and 36,000 bugs [using fuzzing](https://google.github.io/oss-fuzz/#trophies). Ruby deserves a good fuzzer, and improving coverage gathering would help achieve that goal. The `Coverage` module, `TracePoint` module, and `rb_add_event_hook` function seem like they could fulfill this goal. However, after deeper investigation, none of them fit the exact requirements for this use-case. # See also - https://bugs.ruby-lang.org/issues/20282 - https://github.com/google/atheris - https://security.googleblog.com/2020/12/how-atheris-python-fuzzer-works.html - https://github.com/CodeIntelligenceTesting/jazzer/ - https://www.code-intelligence.com/blog/java-fuzzing-with-jazzer - https://go.dev/doc/security/fuzz/ -- https://bugs.ruby-lang.org/

3 11

[ruby-core:124311] [Ruby Feature#21795] Methods for retrieving ASTs
by kddnewton (Kevin Newton) 08 Aug '26

08 Aug '26

Issue #21795 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #21795: Methods for retrieving ASTs https://bugs.ruby-lang.org/issues/21795 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I would like to propose a handful of methods for retrieving ASTs from various objects that correspond to locations in code. This includes: * Proc#ast * Method#ast * UnboundMethod#ast * Thread::Backtrace::Location#ast * TracePoint#ast (on call/return events) The purpose of this is to make tooling easier to write and maintain. Specifically, this would be able to be used in irb, power_assert, error_highlight, and various other tools both in core and not that make use of source code. There have been many previous discussions of retrieving node_id, source_location, source, etc. All of these use cases are covered by returning the AST for some entity. In this case node_id becomes an implementation detail, invisible to the user. Source location can be derived from the information on the AST itself. Similarly, source can be derived from the AST. Internally, I do not think we have to store any more information than we already do (since we have node_id for the first four of these, it becomes rather trivial). For TracePoint we can have a larger discussion about it, but I think it should not be too much work. In terms of implementation, the only caveat I would put is that if the ISEQ were compiled through the old parser/compiler, this should return `nil`, as the node ids do not match up and we do not want to further propagate the RubyVM::AST API. The reason I am opening up this ticket with 5 different methods requested in it is to get approval first for the direction, then I can open individual tickets or just PRs for each method. I believe this feature would ease the maintenance burden of many core libraries, and unify otherwise disparate efforts to achieve the same thing. -- https://bugs.ruby-lang.org/

6 32

[ruby-core:124714] [Ruby Bug#21870] Regexp: Warnings when using multiple non-overlapping \p{...} classes
by jneen (Jeanine Adkisson) 06 Aug '26

06 Aug '26

Issue #21870 has been reported by jneen (Jeanine Adkisson). ---------------------------------------- Bug #21870: Regexp: Warnings when using multiple non-overlapping \p{...} classes https://bugs.ruby-lang.org/issues/21870 * Author: jneen (Jeanine Adkisson) * Status: Open * ruby -v: 4.0.0, 4.0.1, earlier versions to a lesser extent * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ```ruby $VERBOSE = true # warning: character class has duplicated range: /[\p{Word}\p{S}]/ regex = /[\p{Word}\p{S}]/ ``` As far as I can tell this is a perfectly valid and non-overlapping set of unicode properties, but I am still being spammed with warnings. Using `/(\p{Word}|\p{S})/` is kind of a workaround, but it is slower. -- https://bugs.ruby-lang.org/

9 23

[ruby-core:124418] [Ruby Feature#21826] Deprecating RubyVM::AbstractSyntaxTree
by Eregon (Benoit Daloze) 06 Aug '26

06 Aug '26

Issue #21826 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #21826: Deprecating RubyVM::AbstractSyntaxTree https://bugs.ruby-lang.org/issues/21826 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- I think it is time to deprecate `RubyVM::AbstractSyntaxTree`. [Matz has agreed that going forward the official parser API for Ruby will be the Prism API](https://railsatscale.com/2024-04-16-prism-in-2024/) so it's clear the official Ruby parsing API is the Prism API. `RubyVM::AbstractSyntaxTree` is CRuby-specific and does not work on any other Ruby implementation, the API is unstable, inconvenient, etc: > This module is experimental and its API is not stable, therefore it > might change without notice. As examples, the order of children nodes is > not guaranteed, the number of children nodes might change, there is no > way to access children nodes by name, etc. Because it's under `RubyVM`, it also has the following documented caveats: > This module is for very limited purposes, such as debugging, prototyping, and research. > Normal users must not use it. This module is not portable between Ruby implementations. Given these caveats I think it could possibly be fair enough to remove it without deprecation, however I see no urgency to remove it straight away so I think having it deprecated for one release is nicer for the few gems using it. The advantages of deprecating it are: * Make it clear that this API should not be used in new code * Encourage the few usages to migrate to the Prism API, which is stable, officially supported, designed with many people to be usable and convenient for every parsing/tooling usage, etc. * Improve portability of Ruby code between Ruby implementations So, let's deprecate `RubyVM::AbstractSyntaxTree` in 4.1, and remove it in 4.2? -- https://bugs.ruby-lang.org/

5 6

[ruby-core:120488] [Ruby master Feature#21005] Update the source location method to include line start/stop and column start/stop details
by bkuhlmann (Brooke Kuhlmann) 06 Aug '26

06 Aug '26

Issue #21005 has been reported by bkuhlmann (Brooke Kuhlmann). ---------------------------------------- Feature #21005: Update the source location method to include line start/stop and column start/stop details https://bugs.ruby-lang.org/issues/21005 * Author: bkuhlmann (Brooke Kuhlmann) * Status: Open ---------------------------------------- ## Why 👋 Hello. After discussing with Kevin Newton and Benoit Daloze in [Feature 20999](https://bugs.ruby-lang.org/issues/20999), I'd like to propose adding line start/stop and column start/stop information to the `#source_location` method for the following objects: - [Binding](https://docs.ruby-lang.org/en/master/Binding.html) - [Proc](https://docs.ruby-lang.org/en/master/Proc.html) - [Method](https://docs.ruby-lang.org/en/master/Method.html) - [UnboundMethod](https://docs.ruby-lang.org/en/master/UnboundMethod.html) At the moment, when using `#source_location`, you only get the following information: ``` ruby def demo = "A demonstration." # From disk. method(:demo).source_location # ["/Users/bkuhlmann/Engineering/Misc/demo", 15] # From memory. method(:demo).source_location # ["(irb)", 3] ``` Notice, when asking for the source location, we only get the path/location as the first element and the line number as the second element but I'd like to obtain a much richer set of data which includes line start/stop and column start/stop so I can avoid leaning on the `RubyVM` for this information. Example: ``` ruby def demo = "A demonstration." # From disk. instructions = RubyVM::InstructionSequence.of method(:demo) puts [instructions.absolute_path, *instructions.to_a.dig(4, :code_location)] [ "/Users/bkuhlmann/Engineering/Misc/demo", # Source path. 15, # Line start. 0, # Column start. 15, # Line stop. 29 # Column stop. ] # From memory. instructions = RubyVM::InstructionSequence.of method(:demo) puts instructions.script_lines [ "def demo = \"A demonstration.\"\n", "" ] ``` By having access to the path (or lack thereof in case of IRB), line start/stop, and column start/stop, this means we could avoid using the RubyVM to obtain raw source code for any of these objects. This would not only enhance debugging situations but also improve Domain Specific Languages that wish to leverage this information for introducing new features and/or new debugging capabilities to the language. ## How Building upon the examples provided above, I'd like to see `Binding`, `Proc`, `Method`, and `UnboundMethod` respond to `#source_location` as follows: ``` ruby [ "/Users/bkuhlmann/Engineering/Misc/demo", # Source path. 15, # Line start. 15, # Line stop. 0, # Column start. 29 # Column stop. ] ``` Notice, for data grouping purposes, I changed the array structure to always start with the path as the first element, followed by line information, and ending with column information. Alternatively, it could might be nice to improve upon the above by answering a hash each time, instead, for a more self-describing data structure. Example: ``` ruby { path: "/Users/bkuhlmann/Engineering/Misc/demo", line_start: 15, line_stop: 15, column_start: 0, column_stop: 29 } ``` For in-memory, situations like IRB, it would be nice to answer the equivalent of `RubyVM::InstructionSequence#script_lines` which would always be an `Array` with no line or column information since only the source code is necessary. Example: ``` ruby [ "def demo = \"A demonstration.\"\n", "" ] ``` From a pattern matching perspective, this could provide the best of both worlds especially if information is answered as either a `Hash` or and `Array`. Example: ``` def demo = "A demonstration." case method(:demo).source_location in Hash then puts "Source information obtained from disk." in Array then puts "Source obtained from memory." else fail TypeError, "Unrecognized source location type." end ``` This above is only a simple example but there's a lot we could do with this information if the above pattern match was enhanced to deal with the extraction and formatting of the actual source code! ## Notes This feature request is related to the following discussions in case more context is of help: - [Feature 6012](https://bugs.ruby-lang.org/issues/6012) - [Feature 20999](https://bugs.ruby-lang.org/issues/20999) -- https://bugs.ruby-lang.org/

8 21

[ruby-core:125775] [Ruby Feature#22118] Introduce Basic Bit Operations into String
by hasumikin (hitoshi hasumi) 06 Aug '26

06 Aug '26

Issue #22118 has been reported by hasumikin (hitoshi hasumi). ---------------------------------------- Feature #22118: Introduce Basic Bit Operations into String https://bugs.ruby-lang.org/issues/22118 * Author: hasumikin (hitoshi hasumi) * Status: Open ---------------------------------------- This PR implements a subset of the specification proposed in the parent feature ticket #22082 PR URL: [placeholder to be updated] ## Methods implemented with the same specification as the parent ticket * `String#bit_at(offset, lsb_first: true) -> true | false | nil` * `String#bitwise_not -> String` * `String#bitwise_not! -> self` * `String#bitwise_and(other) -> String` * `String#bitwise_and!(other) -> self` * `String#bitwise_or(other) -> String` * `String#bitwise_or!(other) -> self` * `String#bitwise_xor(other) -> String` * `String#bitwise_xor!(other) -> self` ## Methods with range-oriented forms omitted for now The parent ticket also included range-oriented forms for some methods. In this PR, those range arguments are intentionally omitted, and the methods are limited to simpler forms. Range support can be considered as a future extension. * `String#bit_count -> Integer` * `String#bit_set(offset, lsb_first: true) -> self` * `String#bit_clear(offset, lsb_first: true) -> self` * `String#bit_flip(offset, lsb_first: true) -> self` In particular, `String#bit_count` currently takes no arguments. It does not accept a range, offset/length pair, or `lsb_first:` keyword. --- ## Remaining design questions even after narrowing the scope The overall direction of adding these methods has already been approved by Matz. The remaining questions are mostly about naming and API details. ### 1. Why the `bitwise_*` prefix is used In the parent ticket, all other methods are related to bit positions: they either take bit-position arguments, yield bit positions, or operate on a specified bit range. In contrast, the `bitwise_*` methods are whole-data operations. They apply to the entire string as a fixed-size bitmap and do not expose bit positions or bit ordering. The `bitwise_*` prefix is intended to make this distinction explicit. ### 2. Bit numbering #### `lsb_first: true` (default) Within each byte, `offset = 0` is the LSB. Numbering proceeds upward through byte[0] and then continues at the LSB of byte[1]: ```text byte[0] byte[1] offset: 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 bit: b7 b6 b5 b4 b3 b2 b1 b0 b7 b6 b5 b4 b3 b2 b1 b0 ^ ^ LSB LSB offset = byte_index * 8 + bit_in_byte ``` #### `lsb_first: false` for MSB-first Byte order is preserved (`byte[0]` is still first), but within each byte numbering starts at the MSB: ```text byte[0] byte[1] offset: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 bit: b7 b6 b5 b4 b3 b2 b1 b0 b7 b6 b5 b4 b3 b2 b1 b0 ^ ^ MSB MSB offset = byte_index * 8 + (7 - bit_in_byte) ``` * Whether `lsb_first:` is needed at all, or whether an equivalent mechanism should exist under another name? * If such a mechanism is needed, whether `lsb_first:` is the right keyword name? * If `lsb_first:` is acceptable, should its default be `true`? My point is that `lsb_first:` is needed because both bit numbering conventions are used in real byte-buffer formats. LSB-first is the more common convention for general in-memory bitmap use, including formats and APIs such as Apache Arrow validity bitmaps, ext4 block bitmaps, Roaring bitmaps, Linux/BSD bitmap APIs, and hardware register descriptions. MSB-first is also needed for domains such as RFC-style network protocol diagrams, PNG low-bit-depth scanlines, BitTorrent bitfields, and some compressed bit streams. The keyword only controls bit numbering within each byte. Byte order itself is unchanged. ### 3. Out-of-range behavior `String#bit_at` returns `nil` when the bit offset is out of range. On the other hand, mutating methods such as `String#bit_set`, `String#bit_clear`, and `String#bit_flip` raise `IndexError` for an out-of-range bit offset. This follows the same general distinction as read-like access versus write-like access: reading a missing position can return `nil`, but mutating a missing position should not silently do nothing. ### 4. Negative bit offsets Negative bit offsets are rejected with `IndexError`. Although Ruby often uses negative indices to count from the end, combining negative bit offsets with the `lsb_first: true/false` mechanism would make the numbering rule harder to explain and reason about. The API keeps bit offsets as non-negative flat positions from the beginning of the string. ### 5. Length mismatch for binary bitwise operations `String#bitwise_and`, `String#bitwise_or`, and `String#bitwise_xor` require both operands to have the same byte size. If the byte sizes differ, they raise `ArgumentError`. This avoids implicit truncation or zero-padding. Since these methods operate on strings as fixed-size bitmaps, requiring equal sizes makes the operation explicit and predictable. -- https://bugs.ruby-lang.org/

5 9

[ruby-core:115016] [Ruby master Bug#19921] TestYJIT#test_bug_19316 test failure
by vo.x (Vit Ondruch) 03 Aug '26

03 Aug '26

Issue #19921 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19921: TestYJIT#test_bug_19316 test failure https://bugs.ruby-lang.org/issues/19921 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-12 master 52709a4862) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Testing with 52709a4862 on Fedora Rawhide, I observe following failure: ~~~ 94) Failure: TestYJIT#test_bug_19316 [/builddir/build/BUILD/ruby-3.3.0-52709a4862/test/ruby/test_yjit.rb:1126]: Expected no exits, but got {:opt_and=>1} ~~~ -- https://bugs.ruby-lang.org/

6 21

[ruby-core:123776] [Ruby Bug#21682] The result of IO#pos is inconsistent after using IO#ungetc.
by YO4 (Yoshinao Muramatsu) 31 Jul '26

31 Jul '26

Issue #21682 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Bug #21682: The result of IO#pos is inconsistent after using IO#ungetc. https://bugs.ruby-lang.org/issues/21682 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- In this issue, I propose modifying `IO#ungetc` to never change the file position and explain the reasons for this change. The file location after using IO#ungetc may or may not change. This behavior is not documented. Since there are cases that the file position cannot be modified, I propose to unify the behavior so that the file position is not changed. ## Motivation Ensure file position behavior is independent of I/O setup. Avoid excessive exposure of implementation details. ## Current Details The file position after using IO#ungetc depends on the following conditions: When encoding conversion is not used: It changes. When encoding conversion is used: It does not change. This corresponds internally to whether the ungetc character is stored in the IO object's rbuf or cbuf. When using cbuf, the byte count of a character in cbuf may differ from its byte count in rbuf, making conversion from cbuf byte count to rbuf byte count potentially complex. Furthermore, even if one reverse-converts a character in cbuf to calculate its byte count in rbuf, the practical significance of this calculation is questionable. Therefore, it appears that adjusting the buffered byte count from the actual file position to calculate the file position in Ruby when a character exists in cbuf is not implemented. When stored in rbuf, its byte count can be used to calculate the file position, so the file position moves due to ungetc. ## Problems It seems relatively unknown that the side effects of ungetc can change depending on I/O setup. The Ruby community doesn't seem to discuss avoiding the use of file positions after ungetc. Behavior changing under little-known conditions reduces test coverage and leaves potential bugs. On Windows, files opened in "r" mode perform CRLF conversion but operate without encoding conversion, delegating conversion to the C runtime library (CRT). This is primarily for speed, but it modifies IO state in various places within io.c, reducing readability. It has also caused numerous bugs related to file position. Unfortunately, in Windows, "r" and "rt" are not equivalent depending on whether encoding conversion is used or not. This means the behavior of ungetc cannot be predicted within the context of 'text mode'. I'm working personally on switching to use encoding conversion instead of CRT read(), which means encoding conversion will be used for files opened with "r". This change will alter the behavior of ungetc. This impact occurs because implementation details are being exposed excessively, which is undesirable. ## Changes and Impact IO#ungetc will no longer change the file position. This affects not only IO#pos, but also implicitly impacts the file position when writing after ungetc. TestIO#test_copy_stream_dst_rbuf depends on this behavior. Changing the behavior of IO#ungetbyte is outside the scope of this issue. Changing it caused many tests to fail, but the details have not been investigated. Since IO#ungetbyte changes the file position while IO#ungetc does not, mixing their use is no longer possible. This means that performing ungetbyte after unetc before reading the entire data will cause an exception. /test/ruby/test_io.rb fails due to the above implicit dependency, and /spec/ruby/core/io/ungetc_spec.rb fails due to a test explicitly checking IO#pos. I have no idea about investigating the impact of real-world usage. ## Another approach Do not modify Ruby's behavior. Instead, document that the file position after using IO#ungetc cannot be trusted. Alternatively, it might be possible to issue a warning during specific operations when characters from IO#ungetc remain pending. In these cases, when Windows is switched to use encoding conversion for "r", io_unread() and rb_io_tell() will also take into account the number of bytes in cbuf in addition to rbuf to maintain compatibility. I prefer to minimize code for specific environments, so this is not to my ideal. -- https://bugs.ruby-lang.org/

2 5