[ruby-core:122947] [Ruby Feature#21539] Facilitate walking native and interpreter (and jit?) stacks from outside of the ruby process
Issue #21539 has been reported by dalehamel (Dale Hamel). ---------------------------------------- Feature #21539: Facilitate walking native and interpreter (and jit?) stacks from outside of the ruby process https://bugs.ruby-lang.org/issues/21539 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- While ruby does have a great API for getting stack traces within the ruby processes, as used by profilers like vernier and stackprof, there are some projects which aim to profile ruby from outside of the process. Some examples include: - rbspy (https://github.com/rbspy/rbspy/tree/main/ruby-structs/src) - rbperf (https://github.com/javierhonduco/rbperf) - open telemetry (https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/main/supp..., https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/main/inte...) These first two take the approach of embedding the ruby headers within the profiler to be able to walk the stack. otel's bpf exporter was relying on access to symbols (https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202), which were removed some time ago https://github.com/ruby/ruby/pull/7459. As a result, it cannot profile newer rubies as it cannot unwind the stacks. All of these solutions kind of take the approach of trying to reverse engineer the ruby process and rely on somewhat hacky approaches that touch internal things that might frequently move around between ruby versions, and all feel a bit brittle. I think the root of this is that there is no public, stable api for this sort of external profiling. I'm not exactly sure what the solution should be, but it would be great for ruby to offer a recommended, supported, and stable way for external profilers (whether they are using perf api's or ptrace) to be able to: - Obtain the ruby native stack starting point, and walk it while being able to resolve the native symbols - Obtain a reference to the ruby execution context and walk the ruby interpreter stack - Have all of this work in a stable and reasonable way regardless of if / which ruby jit is enabled It would be awesome for some ruby maintainers / experts to weigh-in on what such an external API could look like. Perhaps a public header containing stable (or at least perhaps, versioned?) public symbols and perhaps structs as a starting point? This might necessitate that some fields be refactored out of existing structs, and could slow down work related to them (eg, members need to be added or removed). -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by ivoanjo (Ivo Anjo). +1 this would be really cool as well for the [Datadog profiler](https://github.com/datadog/dd-trace-rb) as we currently rely on access to internal headers and thus are together with rbspy/rbperf on the group of "needs to be updated whenever Ruby internals change". ---------------------------------------- Feature #21539: Facilitate walking native and interpreter (and jit?) stacks from outside of the ruby process https://bugs.ruby-lang.org/issues/21539#change-114422 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- While ruby does have a great API for getting stack traces within the ruby processes, as used by profilers like vernier and stackprof, there are some projects which aim to profile ruby from outside of the process. Some examples include: - rbspy (https://github.com/rbspy/rbspy/tree/main/ruby-structs/src) - rbperf (https://github.com/javierhonduco/rbperf) - open telemetry (https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/main/supp..., https://github.com/open-telemetry/opentelemetry-ebpf-profiler/blob/main/inte...) These first two take the approach of embedding the ruby headers within the profiler to be able to walk the stack. otel's bpf exporter was relying on access to symbols (https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202), which were removed some time ago https://github.com/ruby/ruby/pull/7459. As a result, it cannot profile newer rubies as it cannot unwind the stacks. All of these solutions kind of take the approach of trying to reverse engineer the ruby process and rely on somewhat hacky approaches that touch internal things that might frequently move around between ruby versions, and all feel a bit brittle. I think the root of this is that there is no public, stable api for this sort of external profiling. I'm not exactly sure what the solution should be, but it would be great for ruby to offer a recommended, supported, and stable way for external profilers (whether they are using perf api's or ptrace) to be able to: - Obtain the ruby native stack starting point, and walk it while being able to resolve the native symbols - Obtain a reference to the ruby execution context and walk the ruby interpreter stack - Have all of this work in a stable and reasonable way regardless of if / which ruby jit is enabled It would be awesome for some ruby maintainers / experts to weigh-in on what such an external API could look like. Perhaps a public header containing stable (or at least perhaps, versioned?) public symbols and perhaps structs as a starting point? This might necessitate that some fields be refactored out of existing structs, and could slow down work related to them (eg, members need to be added or removed). -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by dalehamel (Dale Hamel). Updated the description and subject to reduce the scope of this substantially to just exporting a few symbols ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119503 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by Eregon (Benoit Daloze). +1, this would make external profilers more reliable (there are several important advantages of external profilers, e.g. reduce a lot the observer effect, no need to change anything in the already-running program). Datadog is shipping the OpenTelemetry's eBPF profiler and it needs this change to work for the common case of Ruby installations with symbols stripped (e.g. by the Linux distribution). ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119526 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by Eregon (Benoit Daloze). For clarity, it's also fine if these symbols get renamed/changed/etc, exporting them doesn't imply any future compatibility, it's just best-effort. Without the export it's very difficult and hacky to get to them, which translates to cases that simply don't work based on how Ruby was installed (e.g. stripped or not). ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119527 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by dalehamel (Dale Hamel). File protected.patch added File exported.patch added ## Summary Exporting the three symbols had no measurable cost on ruby-bench's `lobsters`, with or without ZJIT, on x86-64 or arm64. With default visibility, though, GCC compiles `vm.c` differently, which can move interpreter microbenchmarks on x86-64 by roughly ±5–9%. ELF `protected` visibility keeps the symbols in `.dynsym` while leaving the interpreter's generated code unchanged, so that's what I'd suggest. ## Performance evaluation **Setup** - Ruby master `b709bdb3e37451bb1103b50fb280968dbf038103`, ruby-bench `13e09f746431f2159db478f15b80a2c7cd95aadb`. - Ubuntu 24.04, GCC 13.3, default flags (`-O3`). Configured with `--enable-shared --enable-yjit --enable-zjit`. - Two dedicated cloud VMs: x86-64 (Intel Xeon Platinum 8581C, SMT disabled) and arm64 (Google Axion, Neoverse-V2). Benchmark processes were pinned to an isolated core. - Three builds: unmodified, exported with default visibility, and exported with ELF `protected` visibility. - Each block ran every build once, each in a fresh process, with build order balanced across blocks. `lobsters` used 18 blocks (15 warmup + 20 measured iterations); the microbenchmarks used 12. - Results are paired ratios of mean iteration time with 95% Student-t intervals. They are not adjusted for multiple comparisons. **lobsters** (negative = faster than unmodified) | CPU | Mode | default visibility | protected | |---|---|---|---| | x86-64 | interpreter | -0.58% [-1.04, -0.11] | -0.77% [-1.25, -0.28] | | x86-64 | ZJIT | -0.37% [-0.62, -0.12] | -0.11% [-0.28, +0.06] | | arm64 | interpreter | -1.06% [-2.19, +0.07] | -0.90% [-1.70, -0.09] | | arm64 | ZJIT | +0.40% [-0.86, +1.67] | +0.35% [-1.12, +1.84] | I wouldn't count the small interpreter speedups as a real benefit. They also appear with `protected`, whose `vm_exec_core` compiles identically to the unmodified build (only addresses differ), so they are most likely code layout effects. With the patch, `--disable-shared` builds export these symbols too, because `ruby` is linked with `-rdynamic`. Their compiler output is essentially unchanged, and lobsters showed no difference (all point estimates within ±0.2%). **Microbenchmarks (interpreter)** With default visibility, GCC has to assume `rb_vm_exec` may be interposed, so it optimizes `vm.c` differently. The 14 internal calls and tail calls to `rb_vm_exec` also go through the PLT. On x86-64, `vm_exec_core` grows from 27,387 to 27,456 bytes and its stack frame grows by 32 bytes. On arm64 it grows from 26,184 to 26,512 bytes. This affects some interpreter microbenchmarks on x86-64: | CPU | Benchmark | default visibility | protected | |---|---|---|---| | x86-64 | `fib` | +5.29% [+4.47, +6.12] | +0.87% [-0.47, +2.23] | | x86-64 | `cfunc_itself` | -8.51% [-12.81, -4.00] | -2.75% [-7.55, +2.30] | The x86-64 `fib` regression executes only 0.49% more instructions. Hardware counters show it as front-end stalls: the front-end-bound share of pipeline slots rises from 3.2% to 12.5%, versus 4.4% with `protected` (one profiling run per build). That points to a layout change in the dispatch loop, not PLT or TLS overhead; `fib` essentially never reaches the PLT calls. On arm64, both microbenchmarks stay within about 1% for both variants, apart from one slow outlier process. An independent x86-64 rebuild with GCC 13.4 and GCC 14.3 (nixpkgs, with different hardening flags) confirmed the same code generation pattern; timing wasn't repeated there: - The default-visibility `vm_exec_core` grows by 23 instructions with GCC 13.4 and 69 with GCC 14.3. The size of the change depends on the compiler. - Both compilers show the same 32-byte frame growth and the same 14 PLT sites. ## Suggested implementation `protected` visibility avoids the code generation change. On ELF the symbols still appear in `.dynsym` after `strip --strip-all`, listed as `GLOBAL PROTECTED`, but references inside libruby stay bound locally: - Calls to `rb_vm_exec` stay direct. - `vm_exec_core` compiles identically to the unmodified build, instruction for instruction. This was checked with GCC 13.3 on both architectures, and with GCC 13.4 and 14.3 on x86-64. - The remaining code differences are small: - In `symbol.c`, each access to `ruby_global_symbols` gains one instruction: a GOT load on arm64, a `lea` after linker relaxation on x86-64. - On x86-64, the `ruby_current_ec` accesses in `vm.c` switch from local-dynamic to general-dynamic TLS, with the same instruction count and the same `__tls_get_addr` call. - The linker still emits a TLS relocation that names `ruby_current_ec`, which is how profilers locate it in stripped builds. The tested change (abbreviated): ```diff --- a/vm.c +++ b/vm.c -VALUE vm_exec(rb_execution_context_t *); +__attribute__((visibility("protected"))) VALUE vm_exec(rb_execution_context_t *); @@ #ifdef RB_THREAD_LOCAL_SPECIFIER -RB_THREAD_LOCAL_SPECIFIER rb_execution_context_t *ruby_current_ec; +RUBY_SYMBOL_EXPORT_BEGIN +__attribute__((visibility("protected"))) RB_THREAD_LOCAL_SPECIFIER rb_execution_context_t *ruby_current_ec; +RUBY_SYMBOL_EXPORT_END --- a/thread_pthread.h +++ b/thread_pthread.h - RUBY_EXTERN RB_THREAD_LOCAL_SPECIFIER struct rb_execution_context_struct *ruby_current_ec; + RUBY_SYMBOL_EXPORT_BEGIN + RUBY_EXTERN __attribute__((visibility("protected"))) RB_THREAD_LOCAL_SPECIFIER struct rb_execution_context_struct *ruby_current_ec; + RUBY_SYMBOL_EXPORT_END --- a/symbol.c +++ b/symbol.c -rb_symbols_t ruby_global_symbols = { +RUBY_SYMBOL_EXPORT_BEGIN +__attribute__((visibility("protected"))) rb_symbols_t ruby_global_symbols = { .next_id = tNEXT_ID, }; +RUBY_SYMBOL_EXPORT_END ``` The default-visibility variant I compared uses `RUBY_FUNC_EXPORTED` and `RUBY_SYMBOL_EXPORT_BEGIN`/`END` at the same four places. Notes: - An upstream version would need a macro that falls back to the existing default-visibility export where `protected` isn't supported. Mach-O compilers warn about it and fall back to default visibility. - The `vm.c` definition must be annotated as well as the header. On arm64 and ppc64, `RB_THREAD_CURRENT_EC_NOINLINE` skips the `thread_pthread.h` declaration. - Protected data is only a problem if an executable links directly against `ruby_global_symbols` and gets a copy relocation. Nothing outside `symbol.c` references it in C; Ruby's own lldb helpers already find it by name. Profilers that look symbols up through ELF aren't affected, and TLS variables aren't copy-relocated. All of this was measured with GCC; Clang wasn't tested. Full patches attached ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119528 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by nayefg (Nayef Ghattas). File debug-aliases.patch added Following up on [protected.patch](https://bugs.ruby-lang.org/attachments/download/10350/protected.patch): in our tests, exporting the original symbols had two effects, even with protected visibility. - With GCC, exporting `ruby_global_symbols` changes how `rb_id2str` addresses the table. On x86-64 the linker relaxes the GOT load into an extra LEA; on arm64 the GOT load remains. Against a build exporting only `ruby_current_ec`, a loop calling `rb_id2name` was about 2% slower on a Xeon and 6% slower by median on a Neoverse N1. - The longer code also moved other functions on x86-64. `rb_backref_get` moved by 80 bytes without changing its instructions, and a loop calling it was about 4% slower on the Xeon against the same reference build. Changing its alignment shifted timings by 3–5%, showing that code placement matters too. I'd suggest a small variation: leave the originals unchanged and add a protected alias next to each definition. These aliases would be best-effort debugging symbols, without a stable-layout guarantee. Ruby's own code keeps using the originals, with their existing declarations and visibility. ```c extern __typeof__(ruby_global_symbols) ruby_debug_global_symbols __attribute__((alias("ruby_global_symbols"), visibility("protected"))); ``` Proposed names, open to discussion: - `ruby_debug_current_ec`: the same TLS slot as `ruby_current_ec`; - `ruby_debug_vm_exec`: the actual `rb_vm_exec` function, including its full symbol size; - `ruby_debug_global_symbols`: the original symbol table. We tested the alias approach on Ruby `b709bdb3` with GCC 13.3.0 and Clang 14.0.6 on x86-64 and arm64 Linux. The aliases survived `strip --strip-all` and were still found by the OpenTelemetry profiler's ELF/TLS lookup code. Internal access sequences were unchanged, and the x86-64 builds also avoided the code displacement seen with direct exports. The benchmarks showed no consistent slowdown. The full patch is attached; the [investigation repository](https://github.com/Gandem/ruby-symbol-discovery) has the assembly and benchmark details. ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119609 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) debug-aliases.patch (1.88 KB) -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by nobu (Nobuyoshi Nakada). I think I could understand the necessity about the former 2 symbols, but not for `ruby_global_symbols`. The profiler has to access it during the VM is locked? The third patch, debug-aliases.patch looks fine, but it doesn't look nice that repeating the complex conditions and revealing aliasing. ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119621 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) debug-aliases.patch (1.88 KB) -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by dalehamel (Dale Hamel). File debug-export.patch added
The profiler has to access it during the VM is locked?
No. The profiler only reads `ruby_global_symbols`, and it doesn't need the VM lock (which an external process couldn't take anyway). It is used to name methods implemented in C. A Ruby method's frame has its name as strings in the iseq, but a C method's frame only has the method entry's `def->original_id`, and `ruby_global_symbols` is the only place to turn that ID into a name. Without it, C methods such as `Array#each` and `Kernel#require`, and the methods of C extensions, lose their names (they appear as e.g. `Array#<unknown cfunc>`). The C function pointer doesn't help: over 90% of the functions behind core methods have no symbol in a stripped libruby, and their C names aren't the Ruby method names anyway. The lookup happens later, in the profiler's own process. At sample time it only records the method entry pointer. Afterwards it reads the ID and its name from the Ruby process's memory while Ruby keeps running, following the same path as `rb_id2str()`: `next_id`, `ids`, the bucket, then the entry. Ruby itself already reads the table this way without the VM lock: only the writers, `set_id_entry()` and `id_entry_dir_grow()`, hold it, and they publish with release stores that `get_id_serial_entry()` pairs with acquire loads. An entry never changes once published, so a name read once can be cached. If a read races with a resize of the directory, that one lookup can fail; the Ruby process is never affected.
it doesn't look nice that repeating the complex conditions and revealing aliasing.
Good callout and agreed. I've attached a revised version of debug-aliases.patch, debug-export.patch, that defines the condition once. A `RUBY_DEBUG_EXPORT()` macro in `internal/compilers.h` wraps the alias and visibility attributes, and reuses configure's existing `HAVE_ATTRIBUTE_FUNCTION_ALIAS` check. `RUBY_ALIAS_FUNCTION` itself doesn't fit here: without alias support it falls back to a wrapper function, and it has no form for variables. Each site is now one line: ```c RUBY_DEBUG_EXPORT_TLS(ruby_debug_current_ec, ruby_current_ec) RUBY_DEBUG_EXPORT(ruby_debug_vm_exec, vm_exec) RUBY_DEBUG_EXPORT(ruby_debug_global_symbols, ruby_global_symbols) ``` It is enabled only on Linux, where these profilers run. The `ruby_debug_` prefix follows the debugger helpers libruby already exports, such as `ruby_debug_print_value`. The machine output should be nearly identical to Nayef's patch and performance should be the same, just the code is cleaner in this one. I tested it on `b709bdb3` with GCC 13.3 on x86-64 and arm64, and with Clang 18 on x86-64; it also applies cleanly to current master. - `make test-basic` passes, with no compiler warnings in the changed files. - After `strip --strip-all`, `.dynsym` gains only the three `ruby_debug_*` names, each with the same value, size and type as its original. - `vm.o` and `symbol.o` compile to the same instructions as the unmodified sources, except two assertion line numbers. With GCC on x86-64, no function in the linked libruby moves either. If you'd prefer the check in configure, or other names, I can change that. ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119628 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) debug-aliases.patch (1.88 KB) debug-export.patch (1.83 KB) -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by Eregon (Benoit Daloze). @dalehamel Could you update the dev-meeting summary at https://bugs.ruby-lang.org/issues/22417#note-2 to reflect the latest patch? I think opening a PR to ruby/ruby would also be helpful to help review. ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119692 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) debug-aliases.patch (1.88 KB) debug-export.patch (1.83 KB) -- https://bugs.ruby-lang.org/
Issue #21539 has been updated by dalehamel (Dale Hamel). Eregon (Benoit Daloze) wrote in #note-12:
@dalehamel Could you update the dev-meeting summary at https://bugs.ruby-lang.org/issues/22417#note-2 to reflect the latest patch?
I think opening a PR to ruby/ruby would also be helpful to help review.
Thanks for the guidance, I updated the comment and opened the PR https://github.com/ruby/ruby/pull/19613 ---------------------------------------- Feature #21539: Export symbols for external profiling of stripped shared Ruby builds https://bugs.ruby-lang.org/issues/21539#change-119699 * Author: dalehamel (Dale Hamel) * Status: Open ---------------------------------------- ## Motivation External profilers such as [OpenTelemetry's eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) collect Ruby stacks from outside the Ruby process. Distribution packages commonly ship shared libruby but strip the full symbol table (`.symtab`) and debug information. Some of the symbols needed to discover Ruby's execution state are currently local symbols, so they disappear when stripped. Public dynamic symbols in `.dynsym` survive normal stripping. I'd like to narrow this proposal to making the necessary discovery symbols available in these stripped shared builds. ## Proposed exports Please export these three existing symbols from shared libruby: | Symbol | Purpose | | --- | --- | | `ruby_current_ec` | Locate the sampled native thread's execution context through TLS, providing access to its Ruby stack. | | `rb_vm_exec` | Identify the function's address range, where native stack unwinding should hand off to Ruby stack unwinding. The profiler does not call this function. | | `ruby_global_symbols` | Resolve Ruby method IDs to names for methods implemented in C. Stack walking can work without this, but those method names become `<unknown cfunc>`. | The profiler also needs `ruby_version`, which is already exported and requires no change. The requested outcome is that these symbols are available through `.dynsym` in installed shared builds, without requiring distributions to retain the full symbol table or users to install debug information just to locate them. ## Scope This is a symbol-visibility request, not a request for a general out-of-process profiling API: - No requirement to stabilize internal struct layouts or make these internals a supported C-extension API. - No additional profiling stack, per-frame bookkeeping, or in-process profiling machinery. - The profiler remains responsible for supporting version-specific layouts and interpreter/JIT behavior. This would address the symbol-discovery obstacle for stripped distribution builds without taking on the broader API design discussed in #19119. Related profiler discussion: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/202 ---Files-------------------------------- protected.patch (1.55 KB) exported.patch (1.21 KB) debug-aliases.patch (1.88 KB) debug-export.patch (1.83 KB) -- https://bugs.ruby-lang.org/
participants (5)
-
dalehamel (Dale Hamel) -
Eregon (Benoit Daloze) -
ivoanjo (Ivo Anjo) -
nayefg (Nayef Ghattas) -
nobu (Nobuyoshi Nakada)