February 2025 - ruby-core - ml.ruby-lang.org

[ruby-core:119380] [Ruby master Misc#20774] Remove remaining locale dependent code from Windows port
by larskanis (Lars Kanis) 12 Dec '25

12 Dec '25

Issue #20774 has been reported by larskanis (Lars Kanis). ---------------------------------------- Misc #20774: Remove remaining locale dependent code from Windows port https://bugs.ruby-lang.org/issues/20774 * Author: larskanis (Lars Kanis) * Status: Open ---------------------------------------- The external_encoding of files, file names and ENV on Windows were changed from locale codepage to UTF-8 in ruby-3.0. But there are still several remaining points where locale encoding is used although there is no need to do so. The Windows port is already fully UTF-16/UTF-8 based and locale encoding is only used for historical and not for technical reasons. My proposal is to remove (most of) the locale dependent conversions from the ruby code for Windows. Before I open pull requests in this regard, I would like to confirm this direction with the ruby core team. Let me show what I mean: ``` # täst-locale-enc.rb def pr(*strs) strs.each do |str| p [str, IO===str ? str.external_encoding&.name : str.encoding.name] end end if $0==__FILE__ pr STDIN # => [#<IO:<STDIN>>, "CP850"] pr $0 # => ["ruby/t\x84st-locale-enc.rb", "CP850"] pr __FILE__ # => ["ruby/t\x84st-locale-enc.rb", "CP850"] pr __dir__ # => ["C:/Users/kanis/ruby", "CP850"] pr 'ä' # => ["ä", "UTF-8"] pr '€' # => ["€", "UTF-8"] pr $:.first # => ["C:/Users/kanis/t\xE2\x82\xACst", "ASCII-8BIT"] pr $:.last # => ["C:/Ruby33-x64/lib/ruby/3.3.0/x64-mingw-ucrt", "CP850"] require "win32/registry" pr Win32::Registry::HKEY_CURRENT_USER.open("Environment")['TMP'] # => ["C:\\Users\\kanis\\AppData\\Local\\Temp", "UTF-8"] pr Win32::Registry::HKEY_CURRENT_USER.open("\\").each_key{ break _1 } # => ["AppEvents", "CP850"] end # execute with: ruby -It€st ruby\täst-locale-enc.rb ``` I wrote the results on `ruby-3.3 x64-mingw-ucrt` right into the code. The situation is even worse when called with `-e` script: ``` $ ruby -It€st -r .\ruby\täst-locale-enc.rb -e "pr STDIN, $0, __FILE__, __dir__, 'ä', '€', $:.first, $:.last" [#<IO:<STDIN>>, "CP850"] ["-e", "CP850"] ["-e", "UTF-8"] [".", "US-ASCII"] ["\x84", "CP850"] ["?", "CP850"] ["C:/Users/kanis/t\xE2\x82\xACst", "ASCII-8BIT"] ["C:/Ruby33-x64/lib/ruby/3.3.0/x64-mingw-ucrt", "CP850"] ``` There are also some inconsistencies like that it's possible to `require` script names with characters outside of the codepage, but it fails to execute a script directly or by using `require_relative` : ``` $ ruby -r .\t€st-locale-enc.rb -e "pr STDIN" [#<IO:<STDIN>>, "CP850"] $ ruby .\t€st-locale-enc.rb ruby: Invalid argument -- ./t?st-locale-enc.rb (LoadError) ``` Maybe there are more places which are working with locale codepage - these are only the few that I remember from memory. I would like to change all the above results to be UTF-8 encoded, like it is the case on Ubuntu. Compatibility ------------- Changing the encoding of returned strings is of course an API change. IMHO it is still something we should change in a minor release of ruby. The reason is that I don't remember about only a single issue cased by the change to UTF-8 in ruby-3.0 in the company I work for. To the contrary many issues are caused by using locale codepage where some non-ASCII characters work and other characters don't. Most issue with ruby-3.0 were cased by the keyword argument changes. -- https://bugs.ruby-lang.org/

2 5

[ruby-core:120279] [Ruby master Feature#20959] Add a way to get codepage of console.
by YO4 (Yoshinao Muramatsu) 11 Dec '25

11 Dec '25

Issue #20959 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Feature #20959: Add a way to get codepage of console. https://bugs.ruby-lang.org/issues/20959 * Author: YO4 (Yoshinao Muramatsu) * Status: Open ---------------------------------------- # Abstract Add a way to retrieve code pages of console. On Windows, Encoding.find("locale") returns the console codepage. To prepare for future changes, specify ```console``` instead of ```locale``` to get the encoding when a console code page is needed. # Background On Windows, Encoding.find("locale") returns the console codepage. This is different from locale in other environments. Also the name and content do not seem to match. In the future, if we change the locale_encoding of the Windows port to the locale codepage of the C runtime library, we need to get the encoding for the console. Strings received via pipe from cmd.exe or powershell are encoded in the console codepage. This would be necessary when communicating with other programs via pipes. # Proposal Make Encoding.find("console") return the encoding that represents the console codepage. # Background(continued) Since Windows 10, UTF-8 support seems to be enhanced in the commandline environment. * build 17134 ucrt supports UTF-8 locale codepage ```setlocale(LC_CTYPE, ".utf8")``` Windows support UTF-8 ANSI Codepage(experimental) ```Beta: Use Unicode UTF-8 for worldwide language support``` * build 18362 support ```Set a process code page to UTF-8``` via manifest. * build 19041 Time zone name holds in wchar_t internaly, so proper tzname can be obtained regardless of the Windows language setting. This should not be all. Through these, I feel that the Microsoft team recommends the use of utf-8. It also seems that Microsoft, which has previously depricated the ANSI version of the API, is treating it as a valid alternative to use in UTF-8. If it becomes widespread on Windows for libraries to respect the C runtime library locale, ruby would do better to follow it. Of course, it is good to be able to get the encoding of the console by specifying a console. # Future plan By reducing differences from other platforms, bugs and extra code are hoped to be reduced. * Do ```setlocale(LC_CTYPE, ".utf8");``` in main.c (or refer LC_* environment variables). * ```Encoding.find("locale")``` returns C runtime library locale. Since strings obtained from the Windows system have a Unicode code range, the API to obtain fixed UTF-8 encoding remains unchanged. # Discussion The code page we can get from Windows also has ACP and OEMCP, but are these necessary? Is it reasonable to get locale_encoding if Encoding.find("console") is called on other platforms? -- https://bugs.ruby-lang.org/

3 4

[ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
by mame (Yusuke Endoh) 11 Dec '25

11 Dec '25

Issue #20314 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20314: Simultaneous Timeout expires may raise an exception after the block https://bugs.ruby-lang.org/issues/20314 * Author: mame (Yusuke Endoh) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows. ```ruby require "timeout" class A < Exception end class B < Exception end begin Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do nil while true end end rescue A, B p $! #=> #<A: execution expired> # Exception B is raised after the above call returns #=> test.rb:16:in `p': execution expired (B) p :end # not reach end ``` This is because the timer thread performs two consecutive `Thread#raise` to the target thread. I have discussed this with @ko1 and have come up with three solutions. ### Solution 1 When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A. The problem with this approach is that if you are rescuing A in the inner block, it may never ends: ```ruby Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do begin sleep rescue A sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end. end end end ``` Note that, if A and B did not occur at the same time, it would raise B. This is a race condition. ### Solution 2 When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both. The problem with this approach is that if you are rescuing B in the inner block, it never ends: ```ruby Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do begin sleep rescue B sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends. end end end ``` ### Solution 3 Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A. Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:121142] [Ruby master Misc#21154] Document or change Module#autoload?
by fxn (Xavier Noria) 07 Dec '25

07 Dec '25

Issue #21154 has been reported by fxn (Xavier Noria). ---------------------------------------- Misc #21154: Document or change Module#autoload? https://bugs.ruby-lang.org/issues/21154 * Author: fxn (Xavier Noria) * Status: Open ---------------------------------------- The documentation of `Module#autoload?` says > Returns filename to be loaded if name is registered as autoload in the namespace of mod or one of its ancestors. Cool, but in the following snippet ```ruby autoload :Foo, 'foo' autoload?(:Foo) ``` the second line could evaluate to `nil`, and this does not seem to agree. I just registered an autoload, therefore (according to the documentation) I should get "foo" back in line 2. I'd like to ask for clarification from the Ruby team: 1. Is the documentation complete? Should that second line always return "foo"? 2. If the answer is no, which is the logic missing in the docs? Thank you! -- https://bugs.ruby-lang.org/

3 34

[ruby-core:112326] [Ruby master Feature#19430] Contribution wanted: DNS lookup by c-ares library
by mame (Yusuke Endoh) 05 Dec '25

05 Dec '25

Issue #19430 has been reported by mame (Yusuke Endoh). ---------------------------------------- Feature #19430: Contribution wanted: DNS lookup by c-ares library https://bugs.ruby-lang.org/issues/19430 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- ## Problem At the present time, Ruby uses `getaddrinfo(3)` to resolve names. Because this function is synchronous, we cannot interrupt the thread performing name resolution until the DNS server returns a response. We can see this behavior by setting blackhole.webpagetest.org (72.66.115.13) as a DNS server, which swallows all packets, and resolving any name: ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C^C^C^C ``` As we see, Ctrl+C does not stop ruby. The current workaround that users can take is to do name resolution in a Ruby thread. ```ruby Thread.new { Addrinfo.getaddrinfo("www.ruby-lang.org", 80) }.value ``` The thread that calls this code is interruptible. (Note that the newly created thread itself will be stuck until the DNS lookup exceeds the time out.) ## Proposal We can solve this problem by using c-ares, which is an asynchronous name resolver, as a backend of `Addrinfo.getaddrinfo`, etc. (@sorah told me about this library, thanks!) https://c-ares.org/ I have created a PoC patch. https://github.com/mame/ruby/commit/547806146993bbc25984011d423dcc0f913b211c By applying this patch, we can interrupt `Addrinfo.getaddrinfo` by Ctrl+C. ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C-e:1:in `getaddrinfo': Interrupt from -e:1:in `<main>' ``` ## Discussion ### About c-ares According to the site of c-ares, some major tools including libcurl, Wireshark, and Apache Arrow are already using c-ares. In the language interpreter, node.js seems to be using c-ares. I am honestly not sure about the compatibility of c-ares with `getaddrinfo(3)`. I guess there is no major incompatibility because I have not experienced any name resolution problem of curl. @akr (who is the author and maintainer of Ruby's socket library) suggested to check if OS-specific name resolution, e.g., WINS on Windows, NIS on Solaris, etc., is supported. He also said that it may be acceptable even if they are not supported. Whether to bundle c-ares source code with ruby would require further discussion. If this proposal is accepted, then c-ares will become a de facto essential dependency for practical use, like gmp, in my opinion. Incidentally, node.js bundles c-ares: https://github.com/nodejs/node/tree/main/deps/cares ### Alternative approaches Recent glibc provides `getaddrinfo_a(3)` which performs asynchronous name resolution. However, this function has a fatal problem of being incompatible with `fork(2)`, which is heavily used in the Ruby ecosystem. In fact, the attempt to use `getaddrinfo_a(3)` (#17134) has been revert because it fails rails tests. (#17220) Another alternative is to have a worker pthread inside Ruby that calls getaddrinfo(3). Instead of calling getaddrinfo(3) directly, `Addrinfo.getaddrinfo` would ask the worker to resolve a name and wait for a response. This method should be able to implement cancellation. (Simply put, this means reimplementation of getaddrinfo_a(3) on our own, taking into account of `fork(2).) This has the advantages: not adding dependencies on external libraries and not having compatibility issues with `getaddrinfo(3)`. However, it is considerably more difficult to implement and maintain. An internal pthread may have a non-trivial impact on the execution efficiency and memory usage. Also, we may need to implement a mechanism to dynamically change the number of workers depending on the load. It would be ideal if we could try and evaluate both approaches. But my current impression is that using c-ares is the quickest and best compromise. ## Contribution wanted I have made it up to the PoC, but don't have much time to complete this. @naruse suggested me to create a ticket asking for contributions. Is anyone interested in this? * This patch changes `rsock_getaddrinfo` to accept a timeout argument. There are several places where Qnil is passed as a timeout (where I add `// TODO` in the PoC). We need to consider what timeout we should pass. * This cares only `getaddrinfo`, but we also need to care `getnameinfo` (and something else if any). There may be some issues I'm not aware of. * I have not yet tested this PoC seriously. It would be great if we could evaluate it with some real apps. Also, it would be great to hear from someone who knows more about c-ares. -- https://bugs.ruby-lang.org/

6 16

[ruby-core:117618] [Ruby master Feature#20437] Could be the licensing conditions be made less ambiguous?
by vo.x (Vit Ondruch) 01 Dec '25

01 Dec '25

Issue #20437 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Feature #20437: Could be the licensing conditions be made less ambiguous? https://bugs.ruby-lang.org/issues/20437 * Author: vo.x (Vit Ondruch) * Status: Open ---------------------------------------- # Current state The current [COPYING](https://github.com/ruby/ruby/blob/6cfd929034f1fe3d93160365505a8b88… file says the following: ~~~ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the 2-clause BSDL (see the file BSDL), or the conditions below: 1. You may make and give away verbatim copies of the source form of the ... snip ... ~~~ And this is quite ambiguous. Let me show a few examples: # Ambiguous examples ## lib/net/protocol.rb https://github.com/ruby/ruby/blob/6cfd929034f1fe3d93160365505a8b88bef56159/… ~~~ # This program is free software. You can re-distribute and/or # modify this program under the same terms as Ruby itself, # Ruby Distribute License or GNU General Public License. ~~~ So what is the license? Is it 2-clause BSDL + Ruby + GPL? Or is is just Ruby + GPL but not 2-clause BSDL? ## Racc https://github.com/ruby/racc/blob/d52fb64d166046bfde81e6e9c424fcd884ebcabf/… ~~~ Racc is distributed under the same terms of ruby. (see the file COPYING). ~~~ where the [COPYING](https://github.com/ruby/racc/blob/d52fb64d166046bfde81e6e9c424fcd8… contains just 2-clause BSDL. Not even mentioning that Ruby used to be GPLv2 licensed: https://github.com/ruby/racc/commit/cc7fe952a6b1fc2ef43477674fa13f2d9d99e818 ## Rubygems https://github.com/rubygems/rubygems/blob/master/LICENSE.txt The license is obviously modeled after Ruby COPYING file, but if it could say "licensed under MIT or Ruby", it would be much better. ## Gettext https://github.com/ruby-gettext/gettext/blob/8d84132efd888e68fffbce41331645… The license note is so old that it is not obvious what that means and what was the intention. Did the Ruby relicensing from "GPLv2 or Ruby" to "GPLv2 or BSD" influenced this project? ## SPDX https://spdx.org/licenses/Ruby.html It quotes just the "conditions below" as a Ruby license, while referring to [Ruby license file](https://www.ruby-lang.org/en/about/license.txt) mentioning also 2-clause BSDL # Proposal To me it seems that the whole world recognizes "Ruby" license, except Ruby. Therefore I think it would be nice if Ruby project recognized the "Ruby" license. There are two possibilities I can think of: ## Refer the license by name ~~~diff $ git diff diff --git a/COPYING b/COPYING index 48e5a96de7..a2d179a8a9 100644 --- a/COPYING +++ b/COPYING @@ -1,6 +1,6 @@ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the -2-clause BSDL (see the file BSDL), or the conditions below: +2-clause BSDL (see the file BSDL), or the conditions of the Ruby license below: 1. You may make and give away verbatim copies of the source form of the software without restriction, provided that you duplicate all of the ~~~ ## Extract the "conditions below" into "Ruby" license file The "conditions below" text could be extracted into separate file (e.g. "RUBY_LICENSE") and the "COPYING" file rephrased to say: ~~~ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the 2-clause BSDL (see the file BSDL), or Ruby license (see the file RUBY_LICENSE) ~~~ -- https://bugs.ruby-lang.org/

3 7

[ruby-core:119239] [Ruby master Feature#20750] Expose ruby_thread_has_gvl_p in ruby/thread.h
by kbrock (Keenan Brock) 19 Nov '25

19 Nov '25

Issue #20750 has been reported by kbrock (Keenan Brock). ---------------------------------------- Feature #20750: Expose ruby_thread_has_gvl_p in ruby/thread.h https://bugs.ruby-lang.org/issues/20750 * Author: kbrock (Keenan Brock) * Status: Open ---------------------------------------- Hello All, I'm hoping we can make `ruby_thread_has_gvl_p` a public method and no longer experimental. I saw the following code in a gem that was not compiling with ruby 3.3: ``` c // int ruby_thread_has_gvl_p(void); // <== line of concern if (ruby_thread_has_gvl_p()) { method_call(&context); } else { rb_thread_call_with_gvl(method_call, &context); } ``` 400 unique projects on github added the line listed above to fix compilation. [1] Some of the projects detected the method first in `extconf.rb`, but a majority just referenced it. `ffi` used to have the detection code but dropped it since the method was in all supported ruby versions. It feels like this method is now part of the undocumented public interface. My suggestion is to add the method to the real public interface in `ruby/thread.h`. PR with possible solution: https://github.com/ruby/ruby/pull/11619 Also included a patch if that is easier Thank you for your consideration, Keenan Timeline: - Dec 30 2008 - `rb_thread_call_with_gvl` introduced via 9c04a06 [2] - Jan 12 2009 - `ruby_thread_has_gvl_p` introduced via 6f09fc2 [4] - Jul 18 2012 - `rb_thread_call_with_gvl` made public via e9a91d2 [3] Current references to code: `rb_thread_call_with_gvl` [5] `ruby_thread_has_gvl_p` [6]: [1]: https://github.com/search?q=ruby_thread_has_gvl_p+language%3AC++NOT+is%3Afo… [2]: https://github.com/ruby/ruby/commit/9c04a0647fb99f7554cb2e04385ddbb970631e02 [3]: https://github.com/ruby/ruby/commit/e9a91d2c95dfe22ad0487952f7a1053ef9a5fd16 [4]: https://github.com/ruby/ruby/commit/6f09fc2efd1915c4337ce6dded52a8a8771c3222 [5]: https://github.com/ruby/ruby/blob/master/thread.c#L1878 [6]: https://github.com/ruby/ruby/blob/master/thread.c#L1923 ---Files-------------------------------- 11618.patch (3.18 KB) -- https://bugs.ruby-lang.org/

3 8

[ruby-core:119998] [Ruby master Bug#20907] Fiber scheduler does not correctly re-lock mutex if `Mutex#sleep` is interrupted.
by ioquatix (Samuel Williams) 16 Nov '25

16 Nov '25

Issue #20907 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20907: Fiber scheduler does not correctly re-lock mutex if `Mutex#sleep` is interrupted. https://bugs.ruby-lang.org/issues/20907 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- The following test fails with a `ThreadError` instead of a `RuntimeError`: ```ruby def test_condition_variable condition_variable = ::Thread::ConditionVariable.new mutex = ::Thread::Mutex.new error = nil thread = Thread.new do Thread.current.report_on_exception = false scheduler = Scheduler.new Fiber.set_scheduler scheduler fiber = Fiber.schedule do begin mutex.synchronize do condition_variable.wait(mutex) end rescue => error end end fiber.raise(RuntimeError) end thread.join assert_kind_of RuntimeError, error end ``` Fix: https://github.com/ruby/ruby/pull/12158 -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119895] [Ruby master Bug#20889] IO#ungetc and IO#ungetbyte should not cause IO#pos to report an inaccurate position
by javanthropus (Jeremy Bopp) 13 Nov '25

13 Nov '25

Issue #20889 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Bug #20889: IO#ungetc and IO#ungetbyte should not cause IO#pos to report an inaccurate position https://bugs.ruby-lang.org/issues/20889 * Author: javanthropus (Jeremy Bopp) * Status: Open * ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8') do |f| f.write('0123456789') f.rewind f.ungetbyte(93) f.pos # => -1; negative value is surprising! end Tempfile.open(encoding: 'utf-8') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-8')) f.pos # => -1; similar to the ungetbyte case end Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-16le')) f.pos # => 0; maybe should be -2 to match the previous ungetc case? end ``` It doesn't seem logical that `IO#pos` should ever be affected by `IO#ungetc` or `IO#ungetbyte`. The pushed characters or bytes aren't really in the stream source. The value of `IO#pos` implies that jumping directly to that position via `IO#seek` and reading from there would return the same character or byte that was pushed, but the pushed characters or bytes are lost when the operation to seek in the stream is performed. In the case where `IO#pos` is a negative value, attempting to seek to that position actually raises an exception. In the `IO#ungetc` with character conversion case above, it seems unreasonable to make `IO#pos` report an even less correct position. In that case, the position would need to be adjusted by 2 bytes in reverse due to the internal encoding of the stream, but that is completely inconsistent with the behavior of `IO#pos` when reading from the stream normally where it reports the underlying stream's byte position and not the number of transcoded bytes that have been read: ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.getc.bytesize # => 2; due to the internal encoding of the stream f.pos # => 1; reports actual bytes read from the stream, not transcoded bytes end ``` Attempting to use `IO#pos` when there are characters or bytes pushed into the read buffer by way of `IO#ungetc` or `IO#ungetbyte` should result in one of the following behaviors: 1. Raise and exception 2. Return the stream's position, clearing the read buffer entirely 3. Return the stream's position, ignoring the pushed characters or bytes, and produce a warning -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118243] [Ruby master Feature#20564] Switch default parser to Prism
by kddnewton (Kevin Newton) 22 Oct '25

22 Oct '25

Issue #20564 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #20564: Switch default parser to Prism https://bugs.ruby-lang.org/issues/20564 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- This issue is to propose switching the default parser for Ruby to Prism on the master branch. The main goal is to get this default into both master and the next preview so that the community has plenty of time to test before the eventual 3.4 release. Matz has indicated that the final decision about the default parser for Ruby 3.4 will be made later in the year. Decisions about the default parser for Ruby 3.5+ and the relationship between Prism, lrama, parse.y, etc. will also be made at a later time. ## Background For some background on the Prism project itself, here are a couple of links in chronological order that you can read: * https://bugs.ruby-lang.org/issues/19741 * https://railsatscale.com/2023-06-12-rewriting-the-ruby-parser/ * https://kddnewton.com/2024/01/23/prism.html * https://railsatscale.com/2024-04-16-prism-in-2024/ ## Status Prism now passes all Ruby tests and specs. Internally at Shopify, it also passes our Core monolith's CI, which is a large and complex Rails application. We are confident that it is ready for production use. External to CRuby, Prism has already been adopted extensively around the ecosystem, including but not limited to: * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java) * [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java) * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby) * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM) * [repl_type_completor](https://github.com/ruby/repl_type_completor) * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-pa… (via parser translator) * [ruby-lsp](https://github.com/Shopify/ruby-lsp) * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator) Fortunately with the community adoption, we have been able to find many edge cases and bugs that have been fixed. Thank you so much to everyone who has tried, used, reported problems with, and improved Prism in the last couple of years. We would not have hit this milestone without this community involvement. ## Caveats While Prism correctly accepts and parses all valid Ruby syntax, there are still a number of cases where it does not reject invalid syntax as it should. We are currently working on these cases, and we are confident that we can eliminate them before the next preview release. You can find them listed here: https://github.com/ruby/prism/issues?q=is%3Aopen+is%3Aissue+label%3Ainvalid…. This is our top priority right now, and I anticipate these being resolved quickly. The other caveat is unknown unknowns: bugs or syntax issues that we have yet to discover. At Shopify we are currently taking numerous steps to mitigate this risk, including: * Prism currently passes all Ruby tests and specs, and will continue to going forward. The tests and specs are extensive, and we are confident this gets us almost all of the way there. * Grammar-based fuzzing to ensure we handle all possible syntax variations (both valid and invalid). As an example, that recently led to the discovery that the current compiler was segfaulting on safe navigation in for loops (https://bugs.ruby-lang.org/issues/20468) It has also found several memory leaks that have been fixed. * Mutation testing of existing source files (by adding and deleting random bytes) to find additional edge cases. Our goal is 100% compatibility, and we believe this is achievable through the above techniques and others. The goal of this proposal is to allow more companies to test their codebase against Prism, and to make this easier. We believe that we can sort out the remaining problems much more quickly in that manner. ## Implementation The implementation will be relatively straightforward, as there is already the `--parser` flag that can be used to switch between the current parser and Prism. We will change the default to be `"prism"` instead of `"parse.y"`. There is no plan to remove or change any code related to `parse.y`, `lrama`, or `compile.c` at this time, as we want to create the minimal amount of change possible to ensure stability. -- https://bugs.ruby-lang.org/

4 10