Issue #21795 has been updated by mame (Yusuke Endoh). I made a prototype under the name `#syntax_tree` instead of `Proc#ast`. https://github.com/ruby/ruby/pull/18005 ```ruby def foo(x) = x * 2 bar = ->(y) { y + 1 } p method(:foo).syntax_tree.class #=> Prism::DefNode p method(:foo).syntax_tree.slice #=> "def foo(x) = x * 2" p bar.syntax_tree.slice #=> "->(y) { y + 1 }" ``` Design notes follow. ## Validating the source hash Judging from the discussion so far, there seems to be no objection to the approach of storing a hash of the source in the ISeq at parse time and validating it when re-parsing. So I implemented it. A hash cannot detect changes perfectly in principle, and the goal here is to prevent accidents, not to defend against security attacks. So I chose the very simple FNV-1a. Since it is an implementation detail, we can change the algorithm at any time. I also want a source hash to fix [Bug #22203]. ## What Proc#syntax_tree returns There are two choices for what `proc { }.syntax_tree` should return: * the BlockNode corresponding to the `{ }` part * the CallNode corresponding to the whole `proc { }` For now, I made it return the CallNode, because `Prism::Node` has no way to go up to the parent node, so the CallNode is more versatile (you can get the BlockNode from the CallNode, but not the CallNode from the BlockNode). Concretely, there would be no way to get the `foo(args)` part from the block of `foo(args) { }`. To be safer, `.syntax_tree` might also want to return the root node, but I think an extension like `node, root = obj.syntax_tree(with_root: true)` can be considered in the future. ## No TracePoint#syntax_tree I did not implement this for now, for two reasons: * For a `:call` event, it is not obvious whether it should return the caller's CallNode or the callee's DefNode (both seem wanted in some cases). * Calling `caller_locations` in the callback seems to provide the necessary information more flexibly. I am not against it if the use case and the expected behavior are clear. ## Prism version differences In short, after a conversation with @kddnewton, I came to think that `Proc#syntax_tree` should just emit a warning when a prism other than the default gem is loaded, instead of returning nil or raising an error. For `Proc#syntax_tree` to work strictly perfectly, as matz says:
the parser that interpreted the running Ruby program and the parser that returns the syntax tree must be the same
This can be achieved by using the default prism gem. But it is also true that, when a different version of prism is loaded, it works reasonably well in most cases. If we return nil or raise an error in that situation, users would experience that `Proc#syntax_tree`, which had been working fine, suddenly stops working just by `bundle update`, which hurts usability. That said, it is also an unshakable fact that perfect behavior is hard to guarantee with a different version of prism. So I think the sweet spot is: communicate the fact that "perfect behavior is not guaranteed" as a warning, but do not stop working on behalf of the user. ## parse.y With `--parser=parse.y`, `Proc#syntax_tree` returns an instance of `RubyVM::AbstractSyntaxTree::Node`. ## Proposal There is still room for adjustment in some details, but I think the specification will be roughly like this. @matz How about landing this as an experimental feature for now? ---------------------------------------- Feature #21795: Methods for retrieving ASTs https://bugs.ruby-lang.org/issues/21795#change-118142 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- I would like to propose a handful of methods for retrieving ASTs from various objects that correspond to locations in code. This includes: * Proc#ast * Method#ast * UnboundMethod#ast * Thread::Backtrace::Location#ast * TracePoint#ast (on call/return events) The purpose of this is to make tooling easier to write and maintain. Specifically, this would be able to be used in irb, power_assert, error_highlight, and various other tools both in core and not that make use of source code. There have been many previous discussions of retrieving node_id, source_location, source, etc. All of these use cases are covered by returning the AST for some entity. In this case node_id becomes an implementation detail, invisible to the user. Source location can be derived from the information on the AST itself. Similarly, source can be derived from the AST. Internally, I do not think we have to store any more information than we already do (since we have node_id for the first four of these, it becomes rather trivial). For TracePoint we can have a larger discussion about it, but I think it should not be too much work. In terms of implementation, the only caveat I would put is that if the ISEQ were compiled through the old parser/compiler, this should return `nil`, as the node ids do not match up and we do not want to further propagate the RubyVM::AST API. The reason I am opening up this ticket with 5 different methods requested in it is to get approval first for the direction, then I can open individual tickets or just PRs for each method. I believe this feature would ease the maintenance burden of many core libraries, and unify otherwise disparate efforts to achieve the same thing. -- https://bugs.ruby-lang.org/