Issue #19772 has been updated by k0kubun (Takashi Kokubun).
Going forward, the only people that will be using the Ruby API of the parser will be tool developers. (For example, formatters like Syntax Tree, linters like Rubocop, static analysis tools like Steep.)
Template engines like Haml that parse Ruby code for optimization purposes, which are executed in production code and probably not considered a "tool". Do you think these gems should keep using Ripper forever?
At this point after the release, we'll begin developing on the 3.4 branch. If consumers of the gem want to parse code with 3.3's behavior, they will be able to use the old version of the gem. We will have the same maintenance cycle as CRuby does.
I think it's problematic for some use cases that you cannot use a parser for the running Ruby version when you have YARP for an old Ruby version in Gemfile. For example, * You're parsing Ruby 3.3 code with some tool (e.g. Rubocop) that is running on Ruby 3.4. When you parse Ruby 3.3 code, the parser should be a parser for Ruby 3.3. But when you debug the process with `binding.irb` or debug.gem, the parser used by them should be a parser for Ruby 3.4 because the code runs on Ruby 3.4. * Similarly, the Haml template engine should always use a Ruby 3.4 parser for parsing Ruby code for optimizing templates even if Gemfile specifies YARP for an old Ruby version for tooling purposes. Essentially, it means that you shouldn't use YARP for parsing Ruby code that is going to be `eval`-ed in the same process because the syntax that yarp.gem parses is not guaranteed to be the same as what `eval` parses. parser.gem's `Parser::CurrentRuby` supports this, `Ripper` supports this, Matz's `Ruby::Parser` idea supports this, but this yarp.gem design doesn't. If we ship the current yarp.gem as is and don't ship `Ruby::Parser`, I will keep using `Ripper` for those use cases because `Ripper` is the only Ruby API that is guaranteed to be compatible with the current Ruby version. --- I can think of the following options that can support the use cases of parsing `eval`-ed Ruby code. 1. [current] Keep maintaining `Ripper` as the only Ruby API that wraps the parser used by the interpreter. 2. [matz's idea] Provide `Ruby::Parser` that wraps the parser used by the interpreter with YARP-like API (but with different constant names). 3. yarp.gem is shipped with a parser for a single Ruby version, and `required_ruby_version` allows only that (minor) version. 4. yarp.gem is shipped with a parser for a single Ruby version that could be old, but also has another class that wraps YARP's C API the interpreter provides, which can be used like parser.gem's `Parser::CurrentRuby`. * C code and Ruby code could be provided by different YARP versions, so I'm not sure if it's easy to maintain. 5. yarp.gem is shipped with parsers for multiple Ruby versions, and has a class that selects a parser for the current Ruby version, which works like parser.gem's `Parser::CurrentRuby`. ---------------------------------------- Misc #19772: API naming for YARP compiler https://bugs.ruby-lang.org/issues/19772#change-104284 * Author: jemmai (Jemma Issroff) * Status: Open * Priority: Normal ---------------------------------------- We are working on the YARP compiler, and have [the first PR ready](https://github.com/ruby/ruby/pull/8042) which introduces the YARP compile method. Our only outstanding question before merging it is about naming. How should we expose the public API for YARP's compile method? Potential suggestions: 1. YARP.compile 2. RubyVM::InstructionSequence.compile(yarp: true) 3. RubyVM::InstructionSequence.compile_yarp 4. Any of the above options, with a name other than yarp (please suggest an alternative) Regarding option 1, which would mirror `YARP.parse`, is the top level constant `YARP` acceptable? cc @matz @ko1 -- https://bugs.ruby-lang.org/