Issue #22236 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #22236: New API for adjustable JIT warmup https://bugs.ruby-lang.org/issues/22236 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Context YJIT, and probably ZJIT in the near future, do provide a significant latency reduction once warmed up, however, during the warmup phase they tend to cause significant overhead, which temporary degrade the service latency, because of the compilation overhead. Thankfully, ZJIT warmup is fast, but still, for project using continuous delivery, this warmup phase can be restarted several times and hour, and is a bit disruptive. ### Prior Art Currently, the only way to smooth the compilation overhead over time is to adjust `--yjit-call-threshold` which defines how many time a method need to be called before it is considered for compilation. This is useful, but hard to reason about, and can easily be invalidated by code changes. It's also not quite granular enough, because many mehtods are called once per cycle, hence will all become eligible during the same cycle. ### Proposal Since both JITs keep track of how long they spent in the compiler, I propose that they accept an "adjustable compilation time budget". Concretely something like `RubyVM::YJIT.max_compile_time_ns = 1_000_000_000`. - Whenever the JIT is done compiling a block and it increments `RubyVM::YJIT.runtime_stats(:compile_time_ns)`, if `compile_time_ns > max_compile_time_ns` then the JIT disable itself. - Whenever `max_compile_time_ns` is re-assigned, if it's larger than `compile_time_ns` the JIT re-enable itself. ### Expected Use Cases Such API would allow services using one process per request to allocate a compilation budget at the start of a cycle (request, job, etc): ```ruby def call(env) RubyVM::YJIT.max_compile_time_ns = RubyVM::YJIT.runtime_stats(:compile_time_ns) + 20_000_000 # 20ms budget @app.call(env) end ``` For services handling work concurrently, a time based approach can be taken: ```ruby Thread.new do loop do # 20ms budget every 100ms RubyVM::YJIT.max_compile_time_ns = RubyVM::YJIT.runtime_stats(:compile_time_ns) + 20_000_000 sleep 0.1 end end ``` These two example are way easier to reason about than configuring `call-threshold` and don't need frequent re-tuning. ### Considerations This API is meant as best effort. I'm aware that at least YJIT isn't able to full disable compilation, as it leave compilation stubs and have little choice but to compile them when hit. But the "disabled" state doesn't have to be absolute, it simply is a good signal for the JIT to know to delay as much work as possible for later. I also know that JITs don't know how long they'll spend in compilation when they enter it, it's absolutely fine if they go over the set limit slightly, as long as they disable themselves right after. -- https://bugs.ruby-lang.org/