Issue #21501 has been updated by ivoanjo (Ivo Anjo). For me the usefulness is in understanding both what and where something is happening even without needing to go into the source code. E.g. Let's consider that native extensions didn't exist. If I get an error in production and I see a stack trace: ``` /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.rb:123:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' /rvm/rubies/ruby-3.4.4/lib/libruby.rb:456:in 'Array#each' native-filenames-example.rb:11:in '<main>' ``` Even without looking at any code, I can understand -- "oh, something happened inside the bigdecimal gem, and there's also something from the Ruby standard library a bit further below". Seeing only this: ``` native-filenames-example.rb:6:in 'BigDecimal.save_rounding_mode' ``` Does not get me as much information: I'd need to "know" by heart what the bigdecimal gem is, and possibly understand that some things (cfuncs) don't show in the same way in the stack trace. And even then -- is it the bigdecimal gem, or is it another `BigDecimal` class in the code? Zooming in on this part:
(...) possibly understand that some things (cfuncs) don't show in the same way in the stack trace.
I think this part can be a bit confusing. I just checked my editions of "Programming Ruby" and "The Well-Grounded Rubyist" and I could not find any references to this behavior. At least for me, it was just something I picked up by looking at enough stack traces: "oh, in this case it was actually a native method and the file doesn't show up". I agree that the "typical Ruby user" would probably not want to modify the extension code. I guess my thesis is: It seems useful to at show users they exist, since they are part of the application? This is an opportunity to teach about this feature, and even make it clear when native extensions are present while debugging problems. Seems easier to google "why am I having problems with Ruby's bigdecimal.so" :) Having learned about them, users can make a more informed choice about using gems with native code vs with Ruby code. Before, they might not even have realized they had them in their important "hot paths" for the application. Or, saying it in another way, if I can require an `hello.rb` that gives me a `HelloClass#print_hello` in Ruby code, and see ``` hello.rb:123: in 'HelloClass#print_hello' ``` I think it would be nice that I could require an `hello.so` that gives me an `HelloClass#print_hello` and get ``` hello.so: in 'HelloClass#print_hello' ``` (And as I mentioned above, this matches well with `$LOADED_FEATURES` also showing `hello.so`) ---
It seems nice, but I think it would be even more helpful to show lines in C files.
I must admit I'm a bit torn on this. As I mentioned above, often the `.c` files are not kept around in the filesystem, whereas in my proposed version I'm showing the exact file (including path) that got loaded into the Ruby VM and that provides the method I'm using. Having said that, I think having the `.c` files/lines as extra information that can get recorded is useful too. Aka I would go for showing the the .so by default, but record both and make both available in the `Backtrace::Locations`.
I think this could be achieved by defining rb_define_method as a macro using __FILE__ and __LINE__ and that should be perfectly portable (and doesn't require debug info).
The small hiccup with this approach for the line `__LINE__` is that it would be the call site for `rb_define_method`, and not where the function itself was defined. TBH I would love to have the actual first line for the function... having it pointing at the `rb_define_method` is a bit less useful? (That said -- methods are almost always defined in the same file as they were declared, so ignoring the `__LINE__`, I agree adding `__FILE__` would work well) ---------------------------------------- Feature #21501: Include native filenames in backtraces as sources for native methods https://bugs.ruby-lang.org/issues/21501#change-113956 * Author: ivoanjo (Ivo Anjo) * Status: Open ---------------------------------------- Consider this example: ```ruby require 'bigdecimal' BigDecimal.singleton_class.prepend( Module.new do def save_rounding_mode super end end ) [:example].each do BigDecimal.save_rounding_mode do puts caller sleep 1 end end ``` which Ruby will print as (from 3.4, since https://bugs.ruby-lang.org/issues/19117): ``` native-filenames-example.rb:6:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' native-filenames-example.rb:11:in 'Array#each' native-filenames-example.rb:11:in '<main>' ``` Having the class names helps, but I think this behavior can still be confusing. Without looking at the code, and understanding that `Array#each` and `BigDecimal.save_rounding_mode` are written in native code, it can be confusing to see them be assigned to `native-filenames-example.rb` in the backtrace. I would like to propose that, whenever possible (see notes below), Ruby shows the name of the native filename where a method was defined. This would result on the backtrace becoming: ``` /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' /rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4:0:in 'Array#each' native-filenames-example.rb:11:in '<main>' ``` or, alternatively, this mechanism could be applied only to extensions, not to libruby/the ruby binary (it's not particularly hard to detect and exclude it): ``` /rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so:0:in 'BigDecimal.save_rounding_mode' native-filenames-example.rb:6:in 'save_rounding_mode' native-filenames-example.rb:12:in 'block in <main>' native-filenames-example.rb:11:in 'Array#each' # Don't touch array ;) native-filenames-example.rb:11:in '<main>' ``` This would make it even easier to understand what's coming from where, and I believe it's a great complement to showing the class names. Furthermore, displaying the actual native libraries matches really well with the semantics we get for regular Ruby files: if I were to look on my machine, I would see `/rvm/gems/ruby-3.4.4/gems/bigdecimal-3.2.2/lib/bigdecimal.so` and `/rvm/rubies/ruby-3.4.4/lib/libruby.so.3.4`. And if I moved them, or overrode them, and loaded them from a different path, I would see the difference. (I say this because in the past I've thought about displaying the actual source .c files and lines, but that a) requires debug information; and b) the files often no longer exist in the filesystem; c) is much harder to implement cross-os). Edit: This also matches with `$LOADED_FEATURES`, which also include the `.so` files as well. --- The extra cool thing is that this is really, really easy to implement! I actually created a [micro gem to show this off](https://github.com/ivoanjo/native-filenames/blob/main/ext/native_filenames_e...) as [well as added this as a feature to the Datadog Ruby profiler](https://github.com/DataDog/dd-trace-rb/pull/4745). Other than checking for headers and whatnot, fetching the info can be implemented in 41 lines of C: https://github.com/ivoanjo/native-filenames/blob/main/ext/native_filenames_e... and works in Linux, macOS and Windows! The way it works is, we can use the function pointer passed to `rb_define_method`, looking it put using `dladdr` (or some of the other variants shown in the code). Then, given just the function pointer, we can get a path back. It's as simple as that -- it can be added transparently as part of storing the cfunc. And if for some reason the path is not available, we could fall back to the current behavior. I'd be very happy to contribute a PR to make this change, but I decided to start with a proposal first as I realize not everyone may be as excited about this idea as I am ;) -- https://bugs.ruby-lang.org/