Issue #21827 has been updated by Eregon (Benoit Daloze). Earlopain (Earlopain _) wrote in #note-4:
Currently I'm working on making the translator more compatible with the gems listed above. What do you think?
Thank you, I think this is a great temporary solution and I will try to implement Ripper on TruffleRuby using that since the Ripper extension has proven to be quite problematic on TruffleRuby. I heard during the dev meeting that it's clearly too soon to deprecate Ripper e.g. with a deprecation warning, we need more time to migrate usages. However I'll make a PR to suggest using Prism for new code and to suggest migrating to Prism for existing code as it will likely be deprecated at some point. I think it's valuable to e.g. migrate `irb` to Prism because that way we don't have the extra indirections of the translation layer, and the better API might allow more correctness too for what IRB does with Ripper. I suspect the overhead is significant, and BTW original Ripper is also significantly slower than Prism as mentioned in the description. This is the issue to migrate to Prism for IRB: https://github.com/ruby/irb/issues/1024 I and others might be able to help a bit with that, let me know how (cc @st0012). I guess first we should merge the existing PRs from @tompng. ---------------------------------------- Feature #21827: Deprecating Ripper https://bugs.ruby-lang.org/issues/21827#change-116122 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- I think it is time to consider deprecating `Ripper`. [Matz has agreed that going forward the official parser API for Ruby will be the Prism API](https://railsatscale.com/2024-04-16-prism-in-2024/) so it's clear the official Ruby parsing API is the Prism API. `Ripper` while being currently supported on CRuby, JRuby and TruffleRuby (partially) has a significant maintenance overhead. For CRuby, it roughly doubles the number of lines in `parse.y` and adds lots of `#ifdef RIPPER`/`#ifndef RIPPER` which makes it really hard to follow. For JRuby it's many thousands of lines of code to maintain just for that: https://github.com/jruby/jruby/tree/master/core/src/main/java/org/jruby/ext/... For TruffleRuby, `ripper` is by far the most difficult C extension to support as it relies on many internal functions. When an internal function is used by ripper is missing it kills the entire process. On every update of `RUBY_VERSION` in TruffleRuby, the biggest amount of time is spent dealing with Ripper, which feels as a result like a major hack. It could be possible to implement Ripper based on Prism, in fact there is [some support for that](https://github.com/ruby/prism/blob/main/docs/ripper_translation.md), but the fact the Ripper API is rather unstable means it's hard to be compatible, and currently the translation layer [relies on original ripper for some things](https://github.com/ruby/prism/blob/e0e5e68acbed98a6b4e4e77b9b52b4488719e2ca/...). It's also worth noting that [`Ripper.sexp` is 2.75x slower than `Prism.parse`](https://eregon.me/blog/2024/10/27/benchmarking-ruby-parsers.html#parsing-and...), while providing a much worse API (widely recognized as such, e.g. `Ripper.sexp("def m = 42")` vs `Prism.parse("def m = 42")`, lots of `nil` with unclear meaning in `Ripper`). So from a new code perspective there seems to be very little value to use `Ripper`. `Ripper` has lexing via `Ripper.lex`, which can be replaced by `Prism::lex_compat`. I think we should start to deprecate `Ripper`. The advantages of deprecating it are: * Make it clear that this API should not be used in new code * Encourage the usages to migrate to the Prism API, which is stable, officially supported, designed with many people to be usable and convenient for every parsing/tooling usage, etc. * Improve portability of Ruby code between Ruby implementations (given Ripper is not fully compatible between Ruby implementations) I propose to deprecate Ripper in 4.1. I'm not sure when it would be appropriate to remove it given it's a more established API, maybe in 4.2, 4.3? -- https://bugs.ruby-lang.org/