Issue #21501 has been updated by ivoanjo (Ivo Anjo). File 456551323-180e85e7-52dc-43d1-9a91-cd7a254de10b.png added
The underlying premise is that a bug is far more likely to be in the user's Ruby code [...]
Yup, I can very very much understand this argument. We actually have a feature on the datadog ruby profiler which we call "only my code" which TL;DR collapses frames in stdlib and gems (you can select which view you want). [When I added native filenames to the profiler](https://github.com/DataDog/dd-trace-rb/pull/4745) I even included this screenshot with that feature enabled, to show it was still working:
This collapsing works even if there are multiple frames in gems, e.g. it produces a shorter and easier to read backtrace (which is especially nice in something like a Rails app). That said... going back to...
The underlying premise is that a bug is far more likely to be in the user's Ruby code than in the C extensions.
I'm curious about the classification "than in the C extensions" in the end of your sentence. Specifically, I would say "the bug is far more likely to be in the user's Ruby code" without mentioning C extensions. Which is what gets me back to my original proposal: I think it's weird that native extensions are treated specially. If Ruby hides `bigdecimal.so` to make it easier to spot the bug in the user's code, why not hide `bigdecimal.rb` (or `lib/bigdecimal/*.rb` to be more exact)? In fact, the collapsing I showed for the datadog profiler above indeed works for either -- it will show "bigdecimal" for any ruby files in the bigdecimal gem, as well as for the .so. I do agree for `libruby.so` that it might be a bit too much -- it'll be extremely rare that the bug will be on `Array#each` so I think that's a great argument for this change affecting only native extensions and not things inside the Ruby VM (or even default gems?). ---------------------------------------- Feature #21501: Include native filenames in backtraces as sources for native methods https://bugs.ruby-lang.org/issues/21501#change-114010 * Author: ivoanjo (Ivo Anjo) * Status: Open ---------------------------------------- Consider this example: ```ruby require 'bigdecimal' BigDecimal.singleton_class.prepend( Module.new do def save_rounding_mode super end end ) [:example].each do BigDecimal.save_rounding_mode do puts caller sleep 1 end end ``` which Ruby will print as (from 3.4, since https://bugs.ruby-lang.org/issues/19117): ``` native-filenames-example.rb:6:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' native-filenames-example.rb:11:in 'Array#each' native-filenames-example.rb:11:in '<main>' ``` Having the class names helps, but I think this behavior can still be confusing. Without looking at the code, and understanding that `Array#each` and `BigDecimal.save_rounding_mode` are written in native code, it can be confusing to see them be assigned to `native-filenames-example.rb` in the backtrace. I would like to propose that, whenever possible (see notes below), Ruby shows the name of the native filename where a method was defined. This would result on the backtrace becoming: ``` /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' /rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4:0:in 'Array#each' native-filenames-example.rb:11:in '<main>' ``` or, alternatively, this mechanism could be applied only to extensions, not to libruby/the ruby binary (it's not particularly hard to detect and exclude it): ``` /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' native-filenames-example.rb:11:in 'Array#each' # Don't touch array ;) native-filenames-example.rb:11:in '<main>' ``` This would make it even easier to understand what's coming from where, and I believe it's a great complement to showing the class names. Furthermore, displaying the actual native libraries matches really well with the semantics we get for regular Ruby files: if I were to look on my machine, I would see `/rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so` and `/rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4`. And if I moved them, or overrode them, and loaded them from a different path, I would see the difference. (I say this because in the past I've thought about displaying the actual source .c files and lines, but that a) requires debug information; and b) the files often no longer exist in the filesystem; c) is much harder to implement cross-os). Edit: This also matches with `$LOADED_FEATURES`, which also include the `.so` files as well. --- The extra cool thing is that this is really, really easy to implement! I actually created a [micro gem to show this off](https://github.com/ivoanjo/native-filenames/blob/main/ext/native_filenames_e...) as [well as added this as a feature to the Datadog Ruby profiler](https://github.com/DataDog/dd-trace-rb/pull/4745). Other than checking for headers and whatnot, fetching the info can be implemented in 41 lines of C: https://github.com/ivoanjo/native-filenames/blob/main/ext/native_filenames_e... and works in Linux, macOS and Windows! The way it works is, we can use the function pointer passed to `rb_define_method`, looking it put using `dladdr` (or some of the other variants shown in the code). Then, given just the function pointer, we can get a path back. It's as simple as that -- it can be added transparently as part of storing the cfunc. And if for some reason the path is not available, we could fall back to the current behavior. I'd be very happy to contribute a PR to make this change, but I decided to start with a proposal first as I realize not everyone may be as excited about this idea as I am ;) ---Files-------------------------------- 456551323-180e85e7-52dc-43d1-9a91-cd7a254de10b.png (9.92 KB) -- https://bugs.ruby-lang.org/