September 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114025] [Ruby master Feature#19744] Namespace on read
by tagomoris (Satoshi TAGOMORI) 06 May '25

06 May '25

Issue #19744 has been reported by tagomoris (Satoshi TAGOMORI). ---------------------------------------- Feature #19744: Namespace on read https://bugs.ruby-lang.org/issues/19744 * Author: tagomoris (Satoshi TAGOMORI) * Status: Open * Priority: Normal ---------------------------------------- # What is the "Namespace on read" This proposes a new feature to define virtual top-level namespaces in Ruby. Those namespaces can require/load libraries (either .rb or native extension) separately from the global namespace. Dependencies of required/loaded libraries are also required/loaded in the namespace. ### Motivation The "namespace on read" can solve the 2 problems below, and can make a path to solve another problem: The details of those motivations are described in the below section ("Motivation details"). #### Avoiding name conflicts between libraries Applications can require two different libraries safely which use the same module name. #### Avoiding unexpected globally shared modules/objects Applications can make an independent/unshared module instance. #### (In the future) Multiple versions of gems can be required Application developers will have fewer version conflicts between gem dependencies if rubygems/bundler will support the namespace on read. ### Example code with this feature ```ruby # your_module module YourModule end # my_module.rb require 'your_module' module MyModule end # example.rb namespace1 = NameSpace.new namespace1.require('my_module') #=> true namespace1::MyModule #=> #<Module:0x00000001027ea650>::MyModule (or #<NameSpace:0x00...>::MyModule ?) namespace1::YourModule # similar to the above MyModule # NameError YourModule # NameError namespace2 = NameSpace.new # Any number of namespaces can be defined namespace2.require('my_module') # Different library "instance" from namespace1 require 'my_module' # require in the global namespace MyModule.object_id != namespace1::MyModule.object_id #=> true namespace1::MyModule.object_id != namespace2::MyModule.object_id ``` The required/loaded libraries will define different "instances" of modules/classes in those namespaces (just like the "wrapper" 2nd argument of `Kernel.load`). This doesn't introduce compatibility problems if all libraries use relative name resolution (without forced top-level reference like `::Name`). # "On read": optional, user-driven feature "On read" is a key thing of this feature. That means: * No changes are required in existing/new libraries (except for limited cases, described below) * No changes are required in applications if it doesn't need namespaces * Users can enable/use namespaces just for limited code in the whole library/application Users can start using this feature step by step (if they want it) without any big jumps. ## Motivation details This feature can solve multiple problems I have in writing/executing Ruby code. Those are from the 3 problems I mentioned above: name conflicts, globally shared modules, and library version conflicts between dependencies. I'll describe 4 scenarios about those problems. ### Running multiple applications on a Ruby process Modern computers have many CPU cores and large memory spaces. We sometimes want to have many separate applications (either micro-service architecture or modular monolith). Currently, running those applications require different processes. It requires additional computation costs (especially in developing those applications). If we have isolated namespaces and can load applications in those namespaces, we'll be able to run apps on a process, with less overhead. (I want to run many AWS Lambda applications on a process in isolated namespaces.) ### Running tests in isolated namespaces Tests that require external libraries need many hacks to: * require a library multiple times * require many different 3rd party libraries into isolated spaces (those may conflict with each other) Software with plugin systems (for example, Fluentd) will get benefit from namespaces. In addition to it, application tests can avoid unexpected side effects if tests are executed in isolated namespaces. ### Safely isolated library instances Libraries may have globally shared states. For example, [Oj](https://github.com/ohler55/oj) has a global `Obj.default_options` object to change the library behavior. Those options may be changed by any dependency libraries or applications, and it changes the behavior of `Oj` globally, unexpectedly. For such libraries, we'll be able to instantiate a safe library instance in an isolated namespace. ### Avoiding dependency hells Modern applications use many libraries, and those libraries require much more dependencies. Those dependencies will cause version conflicts very often. In such cases, application developers should resolve those by updating each libraries, or should just wait for the new release of libraries to conflict those libraries. Sometimes, library maintainers don't release updated versions, and application developers can do nothing. If namespaces can require/load a library multiple times, it also enables to require/load different versions of a library in a process. It requires the support of rubygems, but namespaces should be a good fundamental of it. ## Expected problems ### Use of top-level references In my expectation, `::Name` should refer the top-level `Name` in the global namespace. I expect that `::ENV` should contain the environment variables. But it may cause compatibility problems if library code uses `::MyLibrary` to refer themselves in their deeply nested library code. ### Additional memory consumption An extension library (dynamically linked library) may be loaded multiple times (by `dlopen` for temporarily copied dll files) to load isolated library "instances" if different namespaces require the same extension library. That consumes additional memory. In my opinion, additional memory consumption is a minimum cost to realize loading extension libraries multiple times without compatibility issues. This occurs only when programmers use namespaces. And it's only about libraries that are used in 2 or more namespaces. ### The change of `dlopen` flag about extension libraries To load an extension library multiple times without conflicting symbols, all extensions should stop sharing symbols globally. Libraries referring symbols from other extension libraries will have to change code & dependencies. (About the things about extension libraries, [Naruse also wrote an entry](https://naruse.hateblo.jp/entry/2023/05/22/193411).) # Misc The proof-of-concept branch is here: https://github.com/tagomoris/ruby/pull/1 It's still work-in-progress branch, especially for extension libraries. -- https://bugs.ruby-lang.org/

17 70

[ruby-core:114410] [Ruby master Bug#19841] Marshal.dump stack overflow with recursive Time
by segiddins (Samuel Giddins) 29 Mar '25

29 Mar '25

Issue #19841 has been reported by segiddins (Samuel Giddins). ---------------------------------------- Bug #19841: Marshal.dump stack overflow with recursive Time https://bugs.ruby-lang.org/issues/19841 * Author: segiddins (Samuel Giddins) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ``` ruby #!/usr/bin/env ruby puts RUBY_VERSION t = Time.at(0, 1, :nanosecond) t.instance_variable_set :@itself, t Marshal.dump(t) ``` Yields a stack overflow error from the `Marshal.dump` call, even though Marshal is explicitly able to handle cyclical references -- https://bugs.ruby-lang.org/

6 8

[ruby-core:114147] [Ruby master Bug#19765] Ractor.make_shareable ignores self of a proc created from a Method
by Ethan (Ethan -) 27 Mar '25

27 Mar '25

Issue #19765 has been reported by Ethan (Ethan -). ---------------------------------------- Bug #19765: Ractor.make_shareable ignores self of a proc created from a Method https://bugs.ruby-lang.org/issues/19765 * Author: Ethan (Ethan -) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-07-12T00:26:03Z master dfe782be17) [x86_64-darwin21] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- An unshareable receiver of a Proc or a Method will cause make_shareable to error, but this does not happen with a proc from Method#to_proc: ```ruby str = "" a = str.instance_exec { proc { to_s } } Ractor.make_shareable a # => <internal:ractor>:820:in `make_shareable': Proc's self is not shareable: #<Proc:0x00000001064b62c8 (irb):1> (Ractor::IsolationError) b = str.instance_exec { method(:to_s) } Ractor.make_shareable b # => <internal:ractor>:820:in `make_shareable': can not make shareable object for #<Method: String#to_s()> (Ractor::Error) c = str.instance_exec { method(:to_s).to_proc } Ractor.make_shareable c c.call # => "" str[0] = "!" c.call # => "!" ``` Related, maybe: #19372 #19374 Tangential: why does Proc cause Ractor::IsolationError but Method causes Ractor::Error? -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113153] [Ruby master Bug#19584] Crash in rb_gc_register_address
by peterzhu2118 (Peter Zhu) 13 Mar '25

13 Mar '25

Issue #19584 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19584: Crash in rb_gc_register_address https://bugs.ruby-lang.org/issues/19584 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/7670 Some C extensions pass a pointer to a global variable to rb_gc_register_address. However, if a GC is triggered inside of rb_gc_register_address, then the object could get swept since it does not exist on the stack. -- https://bugs.ruby-lang.org/

4 6

[ruby-core:112301] [Ruby master Bug#19426] Endless `Range#step` of object with `#succ` method does not work
by nobu (Nobuyoshi Nakada) 13 Mar '25

13 Mar '25

Issue #19426 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19426: Endless `Range#step` of object with `#succ` method does not work https://bugs.ruby-lang.org/issues/19426 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Consider this `c` object which hash `#succ` method. ```ruby c = Struct.new(:i) do def succ; self.class.new(i+1); end def <=>(other) i <=> other.i;end end.new(0) ``` This `Range#step` works. ```ruby (c..c.succ).step(1) do |d| p d.i # 0, 1 end ``` But it fails on an endless range. ```ruby (c..).step(1) do |d| p d.i break if d.i > 3 end ``` ``` -:3:in `<=>': undefined method `i' for nil:NilClass (NoMethodError) from -:10:in `step' from -:10:in `<main>' ``` -- https://bugs.ruby-lang.org/

3 4

[ruby-core:113033] [Ruby master Feature#19555] Allow passing default options to `Data.define`
by p8 (Petrik de Heus) 13 Feb '25

13 Feb '25

Issue #19555 has been reported by p8 (Petrik de Heus). ---------------------------------------- Feature #19555: Allow passing default options to `Data.define` https://bugs.ruby-lang.org/issues/19555 * Author: p8 (Petrik de Heus) * Status: Open * Priority: Normal ---------------------------------------- Defining a subclass of `Data` with default attributes can currently be implemented by overriding `intialize`: ```ruby class Point < Data.define(:x, :y, :z) def initialize(x:, y:, z: 0) = super end p Point.new(1, 2) #=> #<data Point x=1, y=2, z=0> ``` It would be nice if we could do it in `define` as well: ```ruby Point = Data.define(:x, :y, z: 0) ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:111121] [Ruby master Bug#19165] Method (with no param) delegation with *, **, and ... is slow
by matsuda (Akira Matsuda) 16 Jan '25

16 Jan '25

Issue #19165 has been reported by matsuda (Akira Matsuda). ---------------------------------------- Bug #19165: Method (with no param) delegation with *, **, and ... is slow https://bugs.ruby-lang.org/issues/19165 * Author: matsuda (Akira Matsuda) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0dev (2022-12-01T08:05:41Z master 4e68b59431) +YJIT [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- I found that method delegation via Forwardable is much slower than normal method call when delegating a method that does not take parameters. Here's a benchmark that explains what I mean. ``` require 'forwardable' require 'pp' require 'benchmark/ips' class Obj extend Forwardable attr_accessor :other def initialize @other = Other.new end def foo_without_splat @other.foo end def foo_with_splat(*) @other.foo(*) end def foo_with_splat_with_name(*args) @other.foo(*args) end def foo_with_splat_and_double_splat(*, **) @other.foo(*, **) end def foo_with_triple_dots(...) @other.foo(...) end delegate :foo => :@other end class Other def foo() end end o = Obj.new Benchmark.ips do |x| x.report 'simple call' do o.other.foo end x.report 'delegate without splat' do o.foo_without_splat end x.report 'delegate with splat' do o.foo_with_splat end x.report 'delegate with splat with name' do o.foo_with_splat_with_name end x.report 'delegate with splat and double splat' do o.foo_with_splat_and_double_splat end x.report 'delegate with triple dots' do o.foo_with_triple_dots end x.report 'delegate via forwardable' do o.foo end end (result) simple call 38.918M (± 0.9%) i/s - 194.884M delegate without splat 31.933M (± 1.6%) i/s - 159.611M delegate with splat 10.269M (± 1.6%) i/s - 51.631M delegate with splat with name 9.888M (± 1.0%) i/s - 49.588M delegate with splat and double splat 4.117M (± 0.9%) i/s - 20.696M delegate with triple dots 4.169M (± 0.9%) i/s - 20.857M delegate via forwardable 9.204M (± 2.1%) i/s - 46.295M ``` It shows that Method delegation with a splat is 3-4 times slower (regardless of whether the parameter is named or not), and delegation with a triple-dot literal is 9-10 times slower than a method delegation without an argument. This may be because calling a method taking a splat always assigns an Array object even when no actual argument was given, and calling a method taking triple-dots assigns five Array objects and two Hash objects (this is equivalent to `*, **`). Are there any chance reducing these object assignments and making them faster? My concern is that the Rails framework heavily uses this kind of method delegations, and presumably it causes unignorable performance overhead. -- https://bugs.ruby-lang.org/

6 8

[ruby-core:112005] [Ruby master Bug#19374] Issue with Ractor.make_shareable with curried procs
by luke-gru (Luke Gruber) 14 Jan '25

14 Jan '25

Issue #19374 has been reported by luke-gru (Luke Gruber). ---------------------------------------- Bug #19374: Issue with Ractor.make_shareable with curried procs https://bugs.ruby-lang.org/issues/19374 * Author: luke-gru (Luke Gruber) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This works, but shouldn't: ```ruby class Worker def start(&blk) blk = blk.curry # bug in ruby allows sharing of non-shareable proc Ractor.make_shareable(blk) @ractor = Ractor.new(blk) do |b| main = b.call p "from ractor: #{main}" end end def work @ractor.take end end worker = Worker.new a = self # unshareable main object p "from main: #{a}" worker.start { a } worker.work ``` The curried proc has a reference to the original proc and it's not checked for shareability. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:111450] [Ruby master Bug#19268] Mingw64 Build Failure
by cfis (Charlie Savage) 31 Dec '24

31 Dec '24

Issue #19268 has been reported by cfis (Charlie Savage). ---------------------------------------- Bug #19268: Mingw64 Build Failure https://bugs.ruby-lang.org/issues/19268 * Author: cfis (Charlie Savage) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x64-mingw-ucrt] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- For both Ruby 3.1.3 and Ruby 3.2.0, building on msys2/ucrt64 fails: ``` c linking miniruby.exe /usr/bin/sh: -c: line 1: syntax error near unexpected token `(' /usr/bin/sh: -c: line 1: `/usr/local/ruby/bin/ruby --disable=gems -n -e BEGIN{version=ARGV.shift;mis=ARGV.dup} -e END{abort "UNICODE version mismatch: #{mis}" unless mis.empty?} -e (mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o 15.0.0 ./enc/unicode/15.0.0/casefold.h ./enc/unicode/15.0.0/name2ctype.h' make: *** [uncommon.mk:878: .rbconfig.time] Error 2 ``` The fix that works for me is changing: ``` c -e '(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o' \ ``` To ``` c -e "(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +\"#{Regexp.quote(version)}\"/o" \ ``` ``` -- https://bugs.ruby-lang.org/

1 4

[ruby-core:111434] [Ruby master Feature#19261] `Data#members` is not important
by ko1 (Koichi Sasada) 04 Dec '24

04 Dec '24

Issue #19261 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #19261: `Data#members` is not important https://bugs.ruby-lang.org/issues/19261 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal ---------------------------------------- `Data#members` is defined but it is calculated by `self.class.members` (in other words, `#members` is a shorthand for `self.class.members`). So it is better to remove this method. ```ruby P = Data.define(:x, :y) p P.new(1, 2).members #=> [:x, :y] Group = Data.define(:name, :members) gs = Group.new('SasadaFamily', %w(ko1 yuki)) p gs.members #=> ["ko1", "yuki"] ``` -- https://bugs.ruby-lang.org/

4 5