April 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:113381] [Ruby master Bug#19624] Backticks - IO object leakage

by pineman

Issue #19624 has been reported by pineman (João Pinheiro). ---------------------------------------- Bug #19624: Backticks - IO object leakage https://bugs.ruby-lang.org/issues/19624 * Author: pineman (João Pinheiro) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi, This code works on ruby 3.0.6: ```ruby `echo` ObjectSpace.each_object(IO) do |io| if ![STDIN, STDOUT, STDERR].include?(io) io.close end end ``` but raises `IOError` on 3.2.2: ``` minimal-repro-case.rb:8:in `close': uninitialized stream (IOError) ``` I found it started failing on ruby 3.1.0 and after, on macOS and Linux. This code is useful for closing unneeded IO objects in forked processes. It looks like backticks is 'leaking' IO objects, waiting for GC, and it didn't used to in 3.1.0. In ruby 3.1.0, inside `rb_f_backquote`, `rb_gc_force_recycle` was removed in favor of `RB_GC_GUARD`. I wonder if this has something to do with the problem. Is this code incorrect since ruby 3.1.0 or is it a bug in ruby? Thanks. ---Files-------------------------------- minimal-repro-case.rb (109 Bytes) -- https://bugs.ruby-lang.org/

7 months, 2 weeks

4
9
0 0

[ruby-core:111572] [Ruby master Bug#19297] Don't download content from internet to execute Ruby test suite

by vo.x (Vit Ondruch)

Issue #19297 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #19297: Don't download content from internet to execute Ruby test suite https://bugs.ruby-lang.org/issues/19297 * Author: vo.x (Vit Ondruch) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Trying to build Ruby 3.2.0 for Fedora and execute its test suite via `make check` as we always did [1], the test suite suddenly fails (while it was working with commit:git|c5eefb7f37): ~~~ ... snip ... C-API Util function ruby_strtod - converts a string to a double and returns the remaining string - returns 0 and the full string if there's no numerical value Finished in 45.737677 seconds 3827 files, 31635 examples, 177877 expectations, 0 failures, 0 errors, 0 tagged ./miniruby -I/builddir/build/BUILD/ruby-3.2.0/lib -I. -I.ext/common /builddir/build/BUILD/ruby-3.2.0/tool/runruby.rb --extout=.ext -- --disable-gems -C "/builddir/build/BUILD/ruby-3.2.0" bin/gem install --no-document \ --install-dir .bundle --conservative "bundler" "rake" "rspec:~> 3" #"ruby-prof" ERROR: Could not find a valid gem 'bundler' (>= 0), here is why: Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz) ERROR: Could not find a valid gem 'rspec' (~> 3), here is why: Unable to download data from https://rubygems.org/ - SocketError: Failed to open TCP connection to rubygems.org:443 (getaddrinfo: Temporary failure in name resolution) (https://rubygems.org/specs.4.8.gz) make: Leaving directory '/builddir/build/BUILD/ruby-3.2.0/redhat-linux-build' make: *** [uncommon.mk:1464: yes-test-syntax-suggest-prepare] Error 2 ~~~ This is obviously due to the test suite trying to download `rspec` from the internet, while Fedora builders does not have internet access (and won't ever have for security reasons). If I am not mistaken, this is caused by commit:git|cae53842735237ccf71a13873fd0d1ae7f165582. Now 1) Can this be fixed? 2) Can the tarball be always self contained? [1]: https://src.fedoraproject.org/rpms/ruby/blob/631163e3b8a51ed610528181aabe0d… -- https://bugs.ruby-lang.org/

8 months

8
11
0 0

[ruby-core:111499] [Ruby master Bug#19281] SyntaxError if first argument of command call has semicolon inside parenthesis

by tompng (tomoya ishida)

Issue #19281 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19281: SyntaxError if first argument of command call has semicolon inside parenthesis https://bugs.ruby-lang.org/issues/19281 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- These are syntax error ~~~ruby p (1;2),(3),(4) p (;),(),() a.b (1;2),(3),(4) a.b (;),(),() ~~~ I expect it to be syntax ok because the code below is syntax ok. ~~~ruby p (1),(2;3),(4;5) p (),(;),(;) a.b (1),(2;3),(4;5) a.b (),(;),(;) ~~~ It will be easy to traverse sexp if the sexp of first argument is same as others ~~~ruby Ripper.sexp "p (),(),()" # => [:program, [[:command, [:@ident, "p", [1, 0]], [:args_add_block, [[:paren, false], # [:paren, [[:void_stmt]]] [:paren, [[:void_stmt]]], [:paren, [[:void_stmt]]]], false]]]] Ripper.sexp "p (1),(2),(3)" # => [:program, [[:command, [:@ident, "p", [1, 0]], [:args_add_block, [[:paren, [:@int, "1", [1, 3]]], # [:paren, [[:@int, "1", [1, 3]]]] [:paren, [[:@int, "2", [1, 7]]]], [:paren, [[:@int, "3", [1, 11]]]]], false]]]] ~~~ -- https://bugs.ruby-lang.org/

8 months

3
2
0 0

[ruby-core:111269] [Ruby master Bug#19230] The openssl backend of securerandom is no longer needed

by mame (Yusuke Endoh)

Issue #19230 has been reported by mame (Yusuke Endoh). ---------------------------------------- Bug #19230: The openssl backend of securerandom is no longer needed https://bugs.ruby-lang.org/issues/19230 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- securerandom first checks if Random.urandom is available ([Line 77](https://github.com/ruby/securerandom/blob/5bfe7d6c163f7a8a45af8d2fc377f…), and if not available, it uses the openssl backend as a degeneration. However, the openssl backend does not work because it internally uses Random.urandom ([Line 55](https://github.com/ruby/securerandom/blob/5bfe7d6c163f7a8a45af8d2fc377f…) to create a seed. This issue is found by @hanachin. ``` $ ruby -ve 'def Random.urandom(*); raise; end; require "securerandom"; p SecureRandom.bytes(10)' ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] -e:1: warning: method redefined; discarding old urandom -e:1:in `urandom': unhandled exception from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:75:in `singleton class' from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:42:in `<module:SecureRandom>' from /home/mame/local/lib/ruby/3.1.0/securerandom.rb:41:in `<top (required)>' from <internal:/home/mame/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require' from <internal:/home/mame/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require' from -e:1:in `<main>' ``` There has been this bug since commit:abae70d6ed63054d7d01bd6cd80c1b5b98b93ba3, which made the urandom backend as default and left the openssl backend just for degeneration. I think no one need the openssl anymore because no one has reported this bug for such a long time. How about removing it? ```diff diff --git a/lib/securerandom.rb b/lib/securerandom.rb index 07ae048634..32b76a2137 100644 --- a/lib/securerandom.rb +++ b/lib/securerandom.rb @@ -14,7 +14,6 @@ # # It supports the following secure random number generators: # -# * openssl # * /dev/urandom # * Win32 # @@ -46,21 +45,6 @@ def bytes(n) private - def gen_random_openssl(n) - @pid = 0 unless defined?(@pid) - pid = $$ - unless @pid == pid - now = Process.clock_gettime(Process::CLOCK_REALTIME, :nanosecond) - OpenSSL::Random.random_add([now, @pid, pid].join(""), 0.0) - seed = Random.urandom(16) - if (seed) - OpenSSL::Random.random_add(seed, 16) - end - @pid = pid - end - return OpenSSL::Random.random_bytes(n) - end - def gen_random_urandom(n) ret = Random.urandom(n) unless ret @@ -77,13 +61,7 @@ def gen_random_urandom(n) Random.urandom(1) alias gen_random gen_random_urandom rescue RuntimeError - begin - require 'openssl' - rescue NoMethodError - raise NotImplementedError, "No random device" - else - alias gen_random gen_random_openssl - end + raise NotImplementedError, "No random device" end public :gen_random ``` -- https://bugs.ruby-lang.org/

8 months, 1 week

6
5
0 0

[ruby-core:112399] [Ruby master Bug#19436] Call Cache for singleton methods can lead to "memory leaks"

by byroot (Jean Boussier)

Issue #19436 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #19436: Call Cache for singleton methods can lead to "memory leaks" https://bugs.ruby-lang.org/issues/19436 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Using "memory leaks" with quotes, because strictly speaking the memory isn't leaked, but it can nonetheless lead to large memory overheads. ### Minimal Reproduction ```ruby module Foo def bar end end def call_bar(obj) # Here the call cache we'll keep a ref on the method_entry # which then keep a ref on the singleton_class, making that # instance immortal until the method is called again with # another instance. # The reference chain is IMEMO(callcache) -> IMEMO(ment) -> ICLASS -> CLASS(singleton) -> OBJECT obj.bar end obj = Object.new obj.extend(Foo) call_bar(obj) id = obj.object_id obj = nil 4.times { GC.start } p ObjectSpace._id2ref(id) ``` ### Explanation Call caches keep a strong reference onto the "callable method entry" (CME), which itself keeps a strong reference on the called object class and in the cache of a singleton class, it keeps a strong reference onto the `attached_object` (instance). This means that any call site that calls a singleton method, will effectively keep a strong reference onto the last receiver. If the method is frequently called it's not too bad, but if it's infrequently called, it's effectively a (bounded) memory leak. And if the `attached_object` is big, the wasted memory can be very substantial. ### Practical Implications Once relative common API impacted by this is [Rails' `extending` API](https://api.rubyonrails.org/classes/ActiveRecord/QueryMethods.html#met…. This API allow to extend a "query result set" with a module. These query results set can sometimes be very big, especially since they keep references to the instantiated `ActiveRecord::Base` instances etc. ### Possible Solutions #### Only keep a weak reference to the CME The fairly "obvious" solution is to keep a weak reference to the CME, that's what I explored in https://github.com/ruby/ruby/pull/7272, and it seems to work. However in debug mode It does fail on an assertion during compaction, but it's isn't quite clear to me what the impact is. Additionally, something that makes me think this would be the right solution, is that call caches already try to avoid marking the class: ```c # vm_callinfo.h:275 struct rb_callcache { const VALUE flags; /* inline cache: key */ const VALUE klass; // should not mark it because klass can not be free'd // because of this marking. When klass is collected, // cc will be cleared (cc->klass = 0) at vm_ccs_free(). ``` So it appears that the class being also marked through the CME is some kind of oversight? #### Don't cache based on some heuristics If the above isn't possible or too complicated, an alternative would be to not cache CMEs found in singleton classes, except if it's the the singleton class of a `Class` or `Module`. It would make repeated calls to such methods slower, but the assumption is that it's unlikely that these CME would live very long. #### Make `Class#attached_object` a weak reference Alternatively we could make the `attached_object` a weak reference, which would drastically limit the amount of memory that may be leaked in such scenario. The downside is that `Class#attached_object` was very recently exposed in Ruby 3.2.0, so it means changing its semantic a bit. cc @peterzhu2118 @ko1 -- https://bugs.ruby-lang.org/

8 months, 2 weeks

8
24
0 0

[ruby-core:111740] [Ruby master Bug#19325] Windows support lacking.

by dsisnero (Dominic Sisneros)

Issue #19325 has been reported by dsisnero (Dominic Sisneros). ---------------------------------------- Bug #19325: Windows support lacking. https://bugs.ruby-lang.org/issues/19325 * Author: dsisnero (Dominic Sisneros) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Ruby's support on windows has always been second class. With some of the recent decisions, windows support is falling even more behind. Recent developments in mjit and yjit that exclude windows are two glaring issues that should be corrected. Googling 'percent of windows vs other operating systems' and it shows windows has a share of 76%. Ceding that users to python and other programming languages has to be one of the reasons python continues get more market share from ruby. With rust having first class windows support and threading support, is there a reason why yjit is not able to work on windows? Also, windows compiler support has matured enough and vcpkg support has evolved enough that it seems it should be possible to finally get a ruby version without having to use msys2. Even Crystal language has a version that runs on windows without needing msys2. -- https://bugs.ruby-lang.org/

9 months, 1 week

9
10
0 0

[ruby-core:113096] [Ruby master Feature#19572] Proposal: New TracePoint event for rescued exceptions

by st0012 (Stan Lo)

Issue #19572 has been reported by st0012 (Stan Lo). ---------------------------------------- Feature #19572: Proposal: New TracePoint event for rescued exceptions https://bugs.ruby-lang.org/issues/19572 * Author: st0012 (Stan Lo) * Status: Open * Priority: Normal ---------------------------------------- **Summary** Support a new `rescue` event type in TracePoint. When the event is triggered, `TracePoint#rescued_exception` can be used to access the exception. **Reason** Currently, TracePoint supports `raise` events, which can be helpful for debugging by showing which exception occurs at which location. By adding a `rescue` event type, we can improve the developer's debugging experience by making it easier to check where an exception is rescued. Currently, the most effective way to check where an exception is rescued involves setting a breakpoint at the exception's raised location and stepping through the code to see whether the debugger stops inside a rescue block. However, this can be a tedious process, especially in large applications with deep call stacks. By using a TracePoint event for rescue, developers can easily track exceptions as they are rescued by adding a few lines of code: ``` TracePoint.trace(:rescue) do |tp| puts "Exception rescued: #{tp.rescued_exception} at #{tp.path}:#{tp.lineno}" end ``` This new TracePoint event will also improve the `ruby/debug`'s [`ExceptionTracer`](https://github.com/ruby/debug/blob/master/lib/debug/tracer.rb#L150-L166) and provide users with a better debugging experience. -- https://bugs.ruby-lang.org/

9 months, 1 week

4
4
0 0

[ruby-core:113266] [Ruby master Bug#19602] `PLATFORM_GET_INC` is broken unless unaligned word access is allowed

by nobu (Nobuyoshi Nakada)

Issue #19602 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19602: `PLATFORM_GET_INC` is broken unless unaligned word access is allowed https://bugs.ruby-lang.org/issues/19602 * Author: nobu (Nobuyoshi Nakada) * Status: Closed * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- Please backport regint.h part of commit:git|fac814c2dc31afef272b45392a7389ef0bfa3a4f. -- https://bugs.ruby-lang.org/

9 months, 2 weeks

3
2
0 0

[ruby-core:112948] [Ruby master Bug#19543] Resizing IO::Buffer to zero bytes fails

by hanazuki (Kasumi Hanazuki)

Issue #19543 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #19543: Resizing IO::Buffer to zero bytes fails https://bugs.ruby-lang.org/issues/19543 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-03-20T04:02:21Z master 7f696b8859) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ``` irb(main):001:0> IO::Buffer.new(1).resize(0) /home/kasumi/.local/src/github.com/ruby/ruby/-e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! /home/kasumi/.local/src/github.com/ruby/ruby/-e:1: [BUG] rb_sys_fail(rb_io_buffer_resize:realloc) - errno == 0 ruby 3.3.0dev (2023-03-20T04:02:21Z master 7f696b8859) [x86_64-linux] # full trace is attached to this ticket ``` `IO::Buffer#resize(0)` will result in calling `realloc(data->base, size)` with size = 0 in [rb_io_buffer_resize](https://bugs.ruby-lang.org/projects/ruby-master/reposi…. Zero-sized `realloc` is deprecated in C (and will be UB in C23). ---Files-------------------------------- bug.txt (40 KB) -- https://bugs.ruby-lang.org/

9 months, 2 weeks

4
5
0 0

[ruby-core:111698] [Ruby master Bug#19318] Float#round rounds incorrectly for some cases

by Eregon (Benoit Daloze)

Issue #19318 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Bug #19318: Float#round rounds incorrectly for some cases https://bugs.ruby-lang.org/issues/19318 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- This was discovered by @aardvark179. The following spec in `spec/ruby/core/float/round_spec.rb` fails on CRuby: ```ruby ruby_bug "", ""..."3.3" do # These numbers are neighbouring floating point numbers round a # precise value. They test that the rounding modes work correctly # round that value and precision is not lost which might cause # incorrect results. it "does not lose precision during the rounding process" do 767573.1875850001.round(5, half: nil).should eql(767573.18759) 767573.1875850001.round(5, half: :up).should eql(767573.18759) 767573.1875850001.round(5, half: :down).should eql(767573.18759) 767573.1875850001.round(5, half: :even).should eql(767573.18759) -767573.1875850001.round(5, half: nil).should eql(-767573.18759) -767573.1875850001.round(5, half: :up).should eql(-767573.18759) -767573.1875850001.round(5, half: :down).should eql(-767573.18759) -767573.1875850001.round(5, half: :even).should eql(-767573.18759) 767573.187585.round(5, half: nil).should eql(767573.18759) 767573.187585.round(5, half: :up).should eql(767573.18759) 767573.187585.round(5, half: :down).should eql(767573.18758) 767573.187585.round(5, half: :even).should eql(767573.18758) -767573.187585.round(5, half: nil).should eql(-767573.18759) -767573.187585.round(5, half: :up).should eql(-767573.18759) -767573.187585.round(5, half: :down).should eql(-767573.18758) -767573.187585.round(5, half: :even).should eql(-767573.18758) 767573.1875849998.round(5, half: nil).should eql(767573.18758) 767573.1875849998.round(5, half: :up).should eql(767573.18758) 767573.1875849998.round(5, half: :down).should eql(767573.18758) 767573.1875849998.round(5, half: :even).should eql(767573.18758) -767573.1875849998.round(5, half: nil).should eql(-767573.18758) -767573.1875849998.round(5, half: :up).should eql(-767573.18758) -767573.1875849998.round(5, half: :down).should eql(-767573.18758) -767573.1875849998.round(5, half: :even).should eql(-767573.18758) end end ``` Yet this test to the best of our knowledge is correct. This was fixed on master by @mrkn in https://github.com/ruby/ruby/pull/7023 (thanks!). The question is should we backport this? I think yes. -- https://bugs.ruby-lang.org/

9 months, 2 weeks

3
2
0 0

2024

2023

2022

ruby-core April 2023