[ruby-core:124315] [Ruby Feature#21797] Make Etc.nprocessors cgroup-aware on Linux
Issue #21797 has been reported by moznion (Taiki Kawakami). ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by moznion (Taiki Kawakami). Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API might be a better approach. ### Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115818 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by hsbt (Hiroshi SHIBATA). RubyGems 4.0.x support -j option for building C extension gem. But It causes in container environment like Circle CI. https://github.com/ruby/rubygems/issues/9170 If cgroup provides the correct number of CPU for Cicle CI or others, I'm positive to support this. ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115820 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by osyoyu (Daisuke Aritomo). It'd be nice if `RUBY_MAX_CPU` would be autoconfigured based on this, just like Go 1.25 `GOMAXPROCS`. Its default value is currently fixed to 8, but not many cloud containers have 8 cores worth of processors. https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t... ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115823 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by nobu (Nobuyoshi Nakada). How about https://github.com/ruby/etc/tree/linux-cgroup-from-maxprocs? ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115834 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by moznion (Taiki Kawakami). osyoyu (Daisuke Aritomo) wrote in #note-4:
It'd be nice if `RUBY_MAX_CPU` would be autoconfigured based on this, just like Go 1.25 `GOMAXPROCS`.
Its default value is currently fixed to 8, but not many cloud containers have 8 cores worth of processors. https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...
This sounds very good, but it should probably be a separate ticket though. nobu (Nobuyoshi Nakada) wrote in #note-5:
How about https://github.com/ruby/etc/tree/linux-cgroup-from-maxprocs?
Thank you for your quick action. I left a comment on the pull request; overall, it looks good. ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115836 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by Eregon (Benoit Daloze). CPU quotas can be floating point numbers though, not just integers. But `Etc.nprocessors` returns an Integer. So it seems having a new method is worth it. I think it's good if `Etc.nprocessors` rounds it though for convenience and so it applies to existing usages of `Etc.nprocessors`. FYI concurrent-ruby already has `Concurret.cpu_quota` (returns Float or nil): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115842 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by moznion (Taiki Kawakami). Thank you for your comments. Let me summarize the discussion. ### 1. Introducing a new API under `Etc` Introducing a method such as `Etc.cpu_quota`, which returns `Float | nil`, seems reasonable. ### 2. Changing the behavior of `Etc.nprocessors` Changing the behavior of `Etc.nprocessors` to respect cgroup-based values and fall back to the original logic when cgroups are unavailable. This would be a breaking change, but it would likely be beneficial for most use cases, and the negative impact should be minimal. ### 3. Injecting the number of actually available CPUs into `RUBY_MAX_CPU` This would be useful, but it should be handled as a separate ticket after this one is resolved. ### 4. Implementation approach Which is preferred: pure Ruby as a part of `ruby/etc` or C? --- What do you think? ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115844 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by byroot (Jean Boussier). So first, I must say I agree this is a common problem with the advent of containers. Both Rails and Puma tried to have automatic detection and configuration of the number of worker processes, and it caused issue for users on various platform (mostly Heroku). That's how I ended up implementing the feature in `concurrent-ruby`. Unfortunately there was still some platforms (mostly Heroku) that did not expose cgroup informations to users (or using a mecanism other than cgroups? unclear). But perfect shouldn't be the enemy of good. If this became a first class Ruby feature, I'm sure more platforms would expose the info.
Changing the behavior of Etc.nprocessors to respect cgroup-based values
What would be the value of `nprocessor` in this case?, e.g. if `cpu_quota`is `0.5`, what does `nprocessors` return? `cpu_quota.ceil`?
Which is preferred: pure Ruby as a part of ruby/etc or C?
It's much easier implemented and maintained in Ruby. You can basically rip most of the code from `concurrent-ruby`. ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115860 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by moznion (Taiki Kawakami).
What would be the value of nprocessor in this case?, e.g. if cpu_quotais 0.5, what does nprocessors return? cpu_quota.ceil?
I think it depends on the decision, but either floor or ceil should be fine (for reference, Go’s `GOMAXPROCS` has used ceil since 1.25). In any case, it would be better if the logic were configurable.
Which is preferred: pure Ruby as a part of ruby/etc or C?
I agree with this. It sounds like it should be implemented under `ruby/etc`. By the way, @osyoyu was interested in using this function in `ruby/thread_pthread.c`, but is it sufficient for Ruby to set the value in the `RUBY_MAX_CPU` environment variable automatically? In other words, do we actually need to use this function directly in the C layer? ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115889 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
Issue #21797 has been updated by osyoyu (Daisuke Aritomo). Thank you for the summary! I'll open a separate ticket for auto-adjusting `RUBY_MAX_CPU`. Come to think of it, `RUBY_MAX_CPU` may have different requirements from `Etc.nprocessors` (defaulting to max # of processors may be suboptimal), so I think these can have separate implementations. In that case, `Etc.nprocessors` can have its own Ruby impl in ext/etc, and `RUBY_MAX_CPU` could have another in the core (presumably in C), even though they might end up being similar. ---------------------------------------- Feature #21797: Make Etc.nprocessors cgroup-aware on Linux https://bugs.ruby-lang.org/issues/21797#change-115897 * Author: moznion (Taiki Kawakami) * Status: Open ---------------------------------------- Currently, `Etc.nprocessors` ignores cgroup CPU quotas. This causes issues in containers where CPU limits differ from the host CPU count. I have written a gem for this purpose (https://github.com/moznion/maxprocs-ruby), but it would be preferable if the Ruby core implementation respected cgroup configuration. Additionally, concurrent-ruby provides similar functionality, but if the language itself offered this capability, language users would not need to implement it individually. And some parts of the Ruby language handle the number of processors using hard-coded values (e.g., https://github.com/ruby/ruby/blob/8efaf5e6b6a25e0d237f3d71b75865661ae98268/t...), so this could also be useful for Ruby language development. Extending `Etc.nprocessors` to respect cgroups is one option, but that would be a breaking change, so adding a new API (e.g., `Etc.cpu_quota` or something?) might be a better approach. ## Prior Art - Go 1.25: runtime GOMAXPROCS (Issue [#73193](https://github.com/golang/go/issues/73193)) - [uber-go/automaxprocs](https://github.com/uber-go/automaxprocs) - [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby): https://github.com/ruby-concurrency/concurrent-ruby/blob/129cf004294af68ac53... -- https://bugs.ruby-lang.org/
participants (6)
-
byroot (Jean Boussier) -
Eregon (Benoit Daloze) -
hsbt (Hiroshi SHIBATA) -
moznion (Taiki Kawakami) -
nobu (Nobuyoshi Nakada) -
osyoyu (Daisuke Aritomo)