December 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:117003] [Ruby master Bug#20314] Simultaneous Timeout expires may raise an exception after the block
by mame (Yusuke Endoh) 11 Dec '25

11 Dec '25

Issue #20314 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #20314: Simultaneous Timeout expires may raise an exception after the block https://bugs.ruby-lang.org/issues/20314 * Author: mame (Yusuke Endoh) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Launchable reports `TestTimeout#test_nested_timeout` as a flaky test, and I reproduced it as follows. ```ruby require "timeout" class A < Exception end class B < Exception end begin Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do nil while true end end rescue A, B p $! #=> #<A: execution expired> # Exception B is raised after the above call returns #=> test.rb:16:in `p': execution expired (B) p :end # not reach end ``` This is because the timer thread performs two consecutive `Thread#raise` to the target thread. I have discussed this with @ko1 and have come up with three solutions. ### Solution 1 When multiple nested Timeouts expire simultaneously, raise an exception for the outer-most Timeout and let the inner Timeouts expire without throwing an exception. In the above example, it would only raise A. The problem with this approach is that if you are rescuing A in the inner block, it may never ends: ```ruby Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do begin sleep rescue A sleep # The exception A is caught. The inner Timeout is already expired, so the code (may) never end. end end end ``` Note that, if A and B did not occur at the same time, it would raise B. This is a race condition. ### Solution 2 When multiple nested Timeouts expire simultaneously, raise an exception for the inner-most Timeout and let the outer Timeouts wait until the inner-most Timeout returns. In the above example, it would raise either A or B, not both. The problem with this approach is that if you are rescuing B in the inner block, it never ends: ```ruby Timeout.timeout(0.1, A) do Timeout.timeout(0.1, B) do begin sleep rescue B sleep # The outer Timeout waits for the inner timeout, and the inner Timeout never return. So this code never ends. end end end ``` ### Solution 3 Make thread interrupt queue one length. If the target thread has already been `Thread#raise(A)`, the new `Thread#raise(B)` blocks until the target thread processes A. Since there will be no more simultaneous Thread#raise, there will be no more exceptions after the end of the block. The timeout timer thread should be changed in consideration that `Thread#raise` may block. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:112326] [Ruby master Feature#19430] Contribution wanted: DNS lookup by c-ares library
by mame (Yusuke Endoh) 04 Dec '25

04 Dec '25

Issue #19430 has been reported by mame (Yusuke Endoh). ---------------------------------------- Feature #19430: Contribution wanted: DNS lookup by c-ares library https://bugs.ruby-lang.org/issues/19430 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- ## Problem At the present time, Ruby uses `getaddrinfo(3)` to resolve names. Because this function is synchronous, we cannot interrupt the thread performing name resolution until the DNS server returns a response. We can see this behavior by setting blackhole.webpagetest.org (72.66.115.13) as a DNS server, which swallows all packets, and resolving any name: ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C^C^C^C ``` As we see, Ctrl+C does not stop ruby. The current workaround that users can take is to do name resolution in a Ruby thread. ```ruby Thread.new { Addrinfo.getaddrinfo("www.ruby-lang.org", 80) }.value ``` The thread that calls this code is interruptible. (Note that the newly created thread itself will be stuck until the DNS lookup exceeds the time out.) ## Proposal We can solve this problem by using c-ares, which is an asynchronous name resolver, as a backend of `Addrinfo.getaddrinfo`, etc. (@sorah told me about this library, thanks!) https://c-ares.org/ I have created a PoC patch. https://github.com/mame/ruby/commit/547806146993bbc25984011d423dcc0f913b211c By applying this patch, we can interrupt `Addrinfo.getaddrinfo` by Ctrl+C. ``` # cat /etc/resolv.conf nameserver 72.66.115.13 # ./local/bin/ruby -rsocket -e 'Addrinfo.getaddrinfo("www.ruby-lang.org", 80)' ^C-e:1:in `getaddrinfo': Interrupt from -e:1:in `<main>' ``` ## Discussion ### About c-ares According to the site of c-ares, some major tools including libcurl, Wireshark, and Apache Arrow are already using c-ares. In the language interpreter, node.js seems to be using c-ares. I am honestly not sure about the compatibility of c-ares with `getaddrinfo(3)`. I guess there is no major incompatibility because I have not experienced any name resolution problem of curl. @akr (who is the author and maintainer of Ruby's socket library) suggested to check if OS-specific name resolution, e.g., WINS on Windows, NIS on Solaris, etc., is supported. He also said that it may be acceptable even if they are not supported. Whether to bundle c-ares source code with ruby would require further discussion. If this proposal is accepted, then c-ares will become a de facto essential dependency for practical use, like gmp, in my opinion. Incidentally, node.js bundles c-ares: https://github.com/nodejs/node/tree/main/deps/cares ### Alternative approaches Recent glibc provides `getaddrinfo_a(3)` which performs asynchronous name resolution. However, this function has a fatal problem of being incompatible with `fork(2)`, which is heavily used in the Ruby ecosystem. In fact, the attempt to use `getaddrinfo_a(3)` (#17134) has been revert because it fails rails tests. (#17220) Another alternative is to have a worker pthread inside Ruby that calls getaddrinfo(3). Instead of calling getaddrinfo(3) directly, `Addrinfo.getaddrinfo` would ask the worker to resolve a name and wait for a response. This method should be able to implement cancellation. (Simply put, this means reimplementation of getaddrinfo_a(3) on our own, taking into account of `fork(2).) This has the advantages: not adding dependencies on external libraries and not having compatibility issues with `getaddrinfo(3)`. However, it is considerably more difficult to implement and maintain. An internal pthread may have a non-trivial impact on the execution efficiency and memory usage. Also, we may need to implement a mechanism to dynamically change the number of workers depending on the load. It would be ideal if we could try and evaluate both approaches. But my current impression is that using c-ares is the quickest and best compromise. ## Contribution wanted I have made it up to the PoC, but don't have much time to complete this. @naruse suggested me to create a ticket asking for contributions. Is anyone interested in this? * This patch changes `rsock_getaddrinfo` to accept a timeout argument. There are several places where Qnil is passed as a timeout (where I add `// TODO` in the PoC). We need to consider what timeout we should pass. * This cares only `getaddrinfo`, but we also need to care `getnameinfo` (and something else if any). There may be some issues I'm not aware of. * I have not yet tested this PoC seriously. It would be great if we could evaluate it with some real apps. Also, it would be great to hear from someone who knows more about c-ares. -- https://bugs.ruby-lang.org/

6 16

[ruby-core:117618] [Ruby master Feature#20437] Could be the licensing conditions be made less ambiguous?
by vo.x (Vit Ondruch) 01 Dec '25

01 Dec '25

Issue #20437 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Feature #20437: Could be the licensing conditions be made less ambiguous? https://bugs.ruby-lang.org/issues/20437 * Author: vo.x (Vit Ondruch) * Status: Open ---------------------------------------- # Current state The current [COPYING](https://github.com/ruby/ruby/blob/6cfd929034f1fe3d93160365505a8b88… file says the following: ~~~ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the 2-clause BSDL (see the file BSDL), or the conditions below: 1. You may make and give away verbatim copies of the source form of the ... snip ... ~~~ And this is quite ambiguous. Let me show a few examples: # Ambiguous examples ## lib/net/protocol.rb https://github.com/ruby/ruby/blob/6cfd929034f1fe3d93160365505a8b88bef56159/… ~~~ # This program is free software. You can re-distribute and/or # modify this program under the same terms as Ruby itself, # Ruby Distribute License or GNU General Public License. ~~~ So what is the license? Is it 2-clause BSDL + Ruby + GPL? Or is is just Ruby + GPL but not 2-clause BSDL? ## Racc https://github.com/ruby/racc/blob/d52fb64d166046bfde81e6e9c424fcd884ebcabf/… ~~~ Racc is distributed under the same terms of ruby. (see the file COPYING). ~~~ where the [COPYING](https://github.com/ruby/racc/blob/d52fb64d166046bfde81e6e9c424fcd8… contains just 2-clause BSDL. Not even mentioning that Ruby used to be GPLv2 licensed: https://github.com/ruby/racc/commit/cc7fe952a6b1fc2ef43477674fa13f2d9d99e818 ## Rubygems https://github.com/rubygems/rubygems/blob/master/LICENSE.txt The license is obviously modeled after Ruby COPYING file, but if it could say "licensed under MIT or Ruby", it would be much better. ## Gettext https://github.com/ruby-gettext/gettext/blob/8d84132efd888e68fffbce41331645… The license note is so old that it is not obvious what that means and what was the intention. Did the Ruby relicensing from "GPLv2 or Ruby" to "GPLv2 or BSD" influenced this project? ## SPDX https://spdx.org/licenses/Ruby.html It quotes just the "conditions below" as a Ruby license, while referring to [Ruby license file](https://www.ruby-lang.org/en/about/license.txt) mentioning also 2-clause BSDL # Proposal To me it seems that the whole world recognizes "Ruby" license, except Ruby. Therefore I think it would be nice if Ruby project recognized the "Ruby" license. There are two possibilities I can think of: ## Refer the license by name ~~~diff $ git diff diff --git a/COPYING b/COPYING index 48e5a96de7..a2d179a8a9 100644 --- a/COPYING +++ b/COPYING @@ -1,6 +1,6 @@ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the -2-clause BSDL (see the file BSDL), or the conditions below: +2-clause BSDL (see the file BSDL), or the conditions of the Ruby license below: 1. You may make and give away verbatim copies of the source form of the software without restriction, provided that you duplicate all of the ~~~ ## Extract the "conditions below" into "Ruby" license file The "conditions below" text could be extracted into separate file (e.g. "RUBY_LICENSE") and the "COPYING" file rephrased to say: ~~~ Ruby is copyrighted free software by Yukihiro Matsumoto <matz(a)netlab.jp>. You can redistribute it and/or modify it under either the terms of the 2-clause BSDL (see the file BSDL), or Ruby license (see the file RUBY_LICENSE) ~~~ -- https://bugs.ruby-lang.org/

3 7

[ruby-core:119239] [Ruby master Feature#20750] Expose ruby_thread_has_gvl_p in ruby/thread.h
by kbrock (Keenan Brock) 19 Nov '25

19 Nov '25

Issue #20750 has been reported by kbrock (Keenan Brock). ---------------------------------------- Feature #20750: Expose ruby_thread_has_gvl_p in ruby/thread.h https://bugs.ruby-lang.org/issues/20750 * Author: kbrock (Keenan Brock) * Status: Open ---------------------------------------- Hello All, I'm hoping we can make `ruby_thread_has_gvl_p` a public method and no longer experimental. I saw the following code in a gem that was not compiling with ruby 3.3: ``` c // int ruby_thread_has_gvl_p(void); // <== line of concern if (ruby_thread_has_gvl_p()) { method_call(&context); } else { rb_thread_call_with_gvl(method_call, &context); } ``` 400 unique projects on github added the line listed above to fix compilation. [1] Some of the projects detected the method first in `extconf.rb`, but a majority just referenced it. `ffi` used to have the detection code but dropped it since the method was in all supported ruby versions. It feels like this method is now part of the undocumented public interface. My suggestion is to add the method to the real public interface in `ruby/thread.h`. PR with possible solution: https://github.com/ruby/ruby/pull/11619 Also included a patch if that is easier Thank you for your consideration, Keenan Timeline: - Dec 30 2008 - `rb_thread_call_with_gvl` introduced via 9c04a06 [2] - Jan 12 2009 - `ruby_thread_has_gvl_p` introduced via 6f09fc2 [4] - Jul 18 2012 - `rb_thread_call_with_gvl` made public via e9a91d2 [3] Current references to code: `rb_thread_call_with_gvl` [5] `ruby_thread_has_gvl_p` [6]: [1]: https://github.com/search?q=ruby_thread_has_gvl_p+language%3AC++NOT+is%3Afo… [2]: https://github.com/ruby/ruby/commit/9c04a0647fb99f7554cb2e04385ddbb970631e02 [3]: https://github.com/ruby/ruby/commit/e9a91d2c95dfe22ad0487952f7a1053ef9a5fd16 [4]: https://github.com/ruby/ruby/commit/6f09fc2efd1915c4337ce6dded52a8a8771c3222 [5]: https://github.com/ruby/ruby/blob/master/thread.c#L1878 [6]: https://github.com/ruby/ruby/blob/master/thread.c#L1923 ---Files-------------------------------- 11618.patch (3.18 KB) -- https://bugs.ruby-lang.org/

3 8

[ruby-core:119998] [Ruby master Bug#20907] Fiber scheduler does not correctly re-lock mutex if `Mutex#sleep` is interrupted.
by ioquatix (Samuel Williams) 16 Nov '25

16 Nov '25

Issue #20907 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20907: Fiber scheduler does not correctly re-lock mutex if `Mutex#sleep` is interrupted. https://bugs.ruby-lang.org/issues/20907 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) * Backport: 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- The following test fails with a `ThreadError` instead of a `RuntimeError`: ```ruby def test_condition_variable condition_variable = ::Thread::ConditionVariable.new mutex = ::Thread::Mutex.new error = nil thread = Thread.new do Thread.current.report_on_exception = false scheduler = Scheduler.new Fiber.set_scheduler scheduler fiber = Fiber.schedule do begin mutex.synchronize do condition_variable.wait(mutex) end rescue => error end end fiber.raise(RuntimeError) end thread.join assert_kind_of RuntimeError, error end ``` Fix: https://github.com/ruby/ruby/pull/12158 -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119895] [Ruby master Bug#20889] IO#ungetc and IO#ungetbyte should not cause IO#pos to report an inaccurate position
by javanthropus (Jeremy Bopp) 13 Nov '25

13 Nov '25

Issue #20889 has been reported by javanthropus (Jeremy Bopp). ---------------------------------------- Bug #20889: IO#ungetc and IO#ungetbyte should not cause IO#pos to report an inaccurate position https://bugs.ruby-lang.org/issues/20889 * Author: javanthropus (Jeremy Bopp) * Status: Open * ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8') do |f| f.write('0123456789') f.rewind f.ungetbyte(93) f.pos # => -1; negative value is surprising! end Tempfile.open(encoding: 'utf-8') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-8')) f.pos # => -1; similar to the ungetbyte case end Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.ungetc('a'.encode('utf-16le')) f.pos # => 0; maybe should be -2 to match the previous ungetc case? end ``` It doesn't seem logical that `IO#pos` should ever be affected by `IO#ungetc` or `IO#ungetbyte`. The pushed characters or bytes aren't really in the stream source. The value of `IO#pos` implies that jumping directly to that position via `IO#seek` and reading from there would return the same character or byte that was pushed, but the pushed characters or bytes are lost when the operation to seek in the stream is performed. In the case where `IO#pos` is a negative value, attempting to seek to that position actually raises an exception. In the `IO#ungetc` with character conversion case above, it seems unreasonable to make `IO#pos` report an even less correct position. In that case, the position would need to be adjusted by 2 bytes in reverse due to the internal encoding of the stream, but that is completely inconsistent with the behavior of `IO#pos` when reading from the stream normally where it reports the underlying stream's byte position and not the number of transcoded bytes that have been read: ```ruby require 'tempfile' Tempfile.open(encoding: 'utf-8:utf-16le') do |f| f.write('0123456789') f.rewind f.getc.bytesize # => 2; due to the internal encoding of the stream f.pos # => 1; reports actual bytes read from the stream, not transcoded bytes end ``` Attempting to use `IO#pos` when there are characters or bytes pushed into the read buffer by way of `IO#ungetc` or `IO#ungetbyte` should result in one of the following behaviors: 1. Raise and exception 2. Return the stream's position, clearing the read buffer entirely 3. Return the stream's position, ignoring the pushed characters or bytes, and produce a warning -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117434] [Ruby master Bug#20409] Missing reporting some invalid breaks
by kddnewton (Kevin Newton) 06 Nov '25

06 Nov '25

Issue #20409 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20409: Missing reporting some invalid breaks https://bugs.ruby-lang.org/issues/20409 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- -- https://bugs.ruby-lang.org/

3 7

[ruby-core:116382] [Ruby master Feature#20205] Enable `frozen_string_literal` by default
by byroot (Jean Boussier) 28 Oct '25

28 Oct '25

Issue #20205 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20205: Enable `frozen_string_literal` by default https://bugs.ruby-lang.org/issues/20205 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- ### Context The `frozen_string_literal: true` pragma was introduced in Ruby 2.3, and as far as I'm aware the plan was initially to make it the default for Ruby 3.0, but this plan was abandoned because it would be too much of a breaking change without any real further notice. According to Matz, he still wishes to enable `frozen_string_literal` by default in the future, but a reasonable migration plan is required. The main issue is backward compatibility, flipping the switch immediately would break a lot of code, so there must be some deprecation period. The usual the path forward for this kind of change is to emit deprecation warnings one of multiple versions in advance. One example of that was the Ruby 2.7 keyword argument deprecation. It was quite verbose, and some users were initially annoyed, but I think the community pulled through it and I don't seem to hear much about it anymore. So for frozen string literals, the first step would be to start warning when a string that would be frozen in the future is mutated. ### Deprecation Warning Implementation I implemented a quick proof of concept with @etienne in https://github.com/Shopify/ruby/pull/549 In short: - Files with `# frozen_string_literal: true` or `# frozen_string_literal: false` don't change in behavior at all. - Files with no `# frozen_string_literal` comment are compiled to use `putchilledstring` opcode instead of regular `putstring`. - This opcode mark the string with a user flag, when these strings are mutated, a warning is issued. Currently the proof of concept issue the warning at the mutation location, which in some case can make locating where the string was allocated a bit hard. But it is possible to improve it so the message also include the location at which the literal string was allocated, and learning from the keyword argument warning experience, we can record which warnings were already issued to avoid spamming users with duplicated warnings. As currently implemented, there is almost no overhead. If we modify the implementation to record the literal location, we'd incur a small memory overhead for each literal string in a file without an explicit `frozen_string_literal` pragma. But I believe we could do it in a way that has no overhead if `Warning[:deprecated] = false`. ### Timeline The migration would happen in 3 steps, each step can potentially last multiple releases. e.g. `R0` could be `3.4`, `R1` be `3.7` and `R2` be `4.0`. I don't have a strong opinion on the pace. - Release `R0`: introduce the deprecation warning (only if deprecation warnings enabled). - Release `R1`: make the deprecation warning show up regardless of verbosity level. - Release `R2`: make string literals frozen by default. ### Impact Given that `rubocop` is quite popular in the community and it has enforced the usage of `# frozen_string_literal: true` for years now, I suspect a large part of the actively maintained codebases in the wild wouldn't see any warnings. And with recent versions of `minitest` enabling deprecation warnings by default (and [potentially RSpec too](https://github.com/rspec/rspec-core/issues/2867)) the few that didn't migrate will likely be made compatible quickly. The real problem of course are the less actively developed libraries and applications. For such cases, any codebase can remain compatible by setting `RUBYOPT="--disable=frozen_string_literal"`, and so even after `R2` release. The flag would never be removed any legacy codebase can continue upgrading Ruby without changing a single line of cod by just flipping this flag. ### Workflow for library maintainers As a library maintainer, fixing the deprecation warnings can be as simple as prepending `# frozen_string_literal: false` at the top of all their source files, and this will keep working forever. Alternatively they can of course make their code compatible with frozen string literals. Code that is frozen string literal compatible doesn't need to explicitly declare it. Only code that need it turned of need to do so. ### Workflow for application owners For application owners, the workflow is the same than for libraries. However if they depend on a gem that hasn't updated, or that they can't upgrade it, they can run their application with `RUBYOPT="--disable=frozen_string_literal"` and it will keep working forever. Any user running into an incompatibility issue can set `RUBYOPT="--disable=frozen_string_literal"` forever, even in `4.x`, the only thing changing is the default value. And any application for which all dependencies have been made fully frozen string literal compatible can set `RUBYOPT="--enable=frozen_string_literal"` and start immediately removing magic comment from their codebase. -- https://bugs.ruby-lang.org/

22 76

[ruby-core:116083] [Ruby master Feature#20163] Introduce #bit_count method on Integer
by garrison (Garrison Jensen) 24 Oct '25

24 Oct '25

Issue #20163 has been reported by garrison (Garrison Jensen). ---------------------------------------- Feature #20163: Introduce #bit_count method on Integer https://bugs.ruby-lang.org/issues/20163 * Author: garrison (Garrison Jensen) * Status: Open * Priority: Normal ---------------------------------------- This feature request is to implement a method called #bit_count on Integer that returns the number of ones in the binary representation of the absolute value of the integer. ``` n = 19 n.bit_count #=> 3 (-n).bit_count #=> 3 ``` This is often useful when you use an integer as a bitmask and want to count how many bits are set. This would be equivalent to ``` n.to_s(2).count("1") ``` However, this can be outperformed by ``` def bit_count(n) count = 0 while n > 0 n &= n - 1 # Flip the least significant 1 bit to 0 count += 1 end count end ``` I think this would be a useful addition because it would fit alongside the other bit-related methods defined on integer: `#bit_length,` `#allbits?`, `#anybits?`, `#nobits?`. Also, when working with bitmasks, a minor upgrade to performance often results in a significant improvement. Similar methods from other languages: https://docs.python.org/3/library/stdtypes.html#int.bit_count https://doc.rust-lang.org/std/primitive.i32.html#method.count_ones -- https://bugs.ruby-lang.org/

13 24

[ruby-core:118243] [Ruby master Feature#20564] Switch default parser to Prism
by kddnewton (Kevin Newton) 22 Oct '25

22 Oct '25

Issue #20564 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #20564: Switch default parser to Prism https://bugs.ruby-lang.org/issues/20564 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- This issue is to propose switching the default parser for Ruby to Prism on the master branch. The main goal is to get this default into both master and the next preview so that the community has plenty of time to test before the eventual 3.4 release. Matz has indicated that the final decision about the default parser for Ruby 3.4 will be made later in the year. Decisions about the default parser for Ruby 3.5+ and the relationship between Prism, lrama, parse.y, etc. will also be made at a later time. ## Background For some background on the Prism project itself, here are a couple of links in chronological order that you can read: * https://bugs.ruby-lang.org/issues/19741 * https://railsatscale.com/2023-06-12-rewriting-the-ruby-parser/ * https://kddnewton.com/2024/01/23/prism.html * https://railsatscale.com/2024-04-16-prism-in-2024/ ## Status Prism now passes all Ruby tests and specs. Internally at Shopify, it also passes our Core monolith's CI, which is a large and complex Rails application. We are confident that it is ready for production use. External to CRuby, Prism has already been adopted extensively around the ecosystem, including but not limited to: * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java) * [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java) * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby) * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM) * [repl_type_completor](https://github.com/ruby/repl_type_completor) * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-pa… (via parser translator) * [ruby-lsp](https://github.com/Shopify/ruby-lsp) * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator) Fortunately with the community adoption, we have been able to find many edge cases and bugs that have been fixed. Thank you so much to everyone who has tried, used, reported problems with, and improved Prism in the last couple of years. We would not have hit this milestone without this community involvement. ## Caveats While Prism correctly accepts and parses all valid Ruby syntax, there are still a number of cases where it does not reject invalid syntax as it should. We are currently working on these cases, and we are confident that we can eliminate them before the next preview release. You can find them listed here: https://github.com/ruby/prism/issues?q=is%3Aopen+is%3Aissue+label%3Ainvalid…. This is our top priority right now, and I anticipate these being resolved quickly. The other caveat is unknown unknowns: bugs or syntax issues that we have yet to discover. At Shopify we are currently taking numerous steps to mitigate this risk, including: * Prism currently passes all Ruby tests and specs, and will continue to going forward. The tests and specs are extensive, and we are confident this gets us almost all of the way there. * Grammar-based fuzzing to ensure we handle all possible syntax variations (both valid and invalid). As an example, that recently led to the discovery that the current compiler was segfaulting on safe navigation in for loops (https://bugs.ruby-lang.org/issues/20468) It has also found several memory leaks that have been fixed. * Mutation testing of existing source files (by adding and deleting random bytes) to find additional edge cases. Our goal is 100% compatibility, and we believe this is achievable through the above techniques and others. The goal of this proposal is to allow more companies to test their codebase against Prism, and to make this easier. We believe that we can sort out the remaining problems much more quickly in that manner. ## Implementation The implementation will be relatively straightforward, as there is already the `--parser` flag that can be used to switch between the current parser and Prism. We will change the default to be `"prism"` instead of `"parse.y"`. There is no plan to remove or change any code related to `parse.y`, `lrama`, or `compile.c` at this time, as we want to create the minimal amount of change possible to ensure stability. -- https://bugs.ruby-lang.org/

4 10