February 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116656] [Ruby master Bug#20253] `Proc.dup` and `Proc#clone` don't preserve finalizers
by byroot (Jean Boussier) 12 Feb '24

12 Feb '24

Issue #20253 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20253: `Proc.dup` and `Proc#clone` don't preserve finalizers https://bugs.ruby-lang.org/issues/20253 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 3.0: WONTFIX, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- While reviewing the fix for [Bug #20250] @peterzhu2118 pointed that `FL_FINALIZE` should probably also be cleared. However after some extra testing, it appears `Object#dup` and `Object#clone` do copy over the finalizer (which makes sense). But for some reason `Proc` has its own `dup/clone` implementation, which does copy the `FL_FINALIZE` flag, but doesn't copy the finalizer: Test script: ```ruby def fin(sym) ->(_) { p sym } end obj = Object.new ObjectSpace.define_finalizer(obj, fin(:obj)) obj.dup obj.clone proc = Proc.new { } ObjectSpace.define_finalizer(proc, fin(:proc)) proc.dup proc.clone ``` Expected output: ``` :proc :proc :proc :obj :obj :obj ``` Actual output: ``` :proc :obj :obj :obj ``` This discrepancy is present all the way back to Ruby 1.9. It's so niche I'm not sure it's worth a backport though... -- https://bugs.ruby-lang.org/

1 3

[ruby-core:115143] [Ruby master Bug#19970] Eval leaks callcache and callinfo objects on arm32 (linux)
by larsin (Lars Ingjer) 11 Feb '24

11 Feb '24

Issue #19970 has been reported by larsin (Lars Ingjer). ---------------------------------------- Bug #19970: Eval leaks callcache and callinfo objects on arm32 (linux) https://bugs.ruby-lang.org/issues/19970 * Author: larsin (Lars Ingjer) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [armv7l-linux-eabihf] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script demonstrates a memory leak on arm 32 (linux): ``` ruby def gcdiff(n) GC.start if @last_gc_stat puts "GC.stat #{n} diff old_objects: #{GC.stat(:old_objects) - @last_gc_stat}" end @last_gc_stat = GC.stat(:old_objects) end def foo end 10.times do |i| 10_000.times do eval 'foo' end gcdiff(i) puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output: ``` Number of live objects: 41303 Memory usage: 11900 GC.stat 1 diff old_objects: 20037 Number of live objects: 61317 Memory usage: 13604 GC.stat 2 diff old_objects: 20001 Number of live objects: 81317 Memory usage: 14880 GC.stat 3 diff old_objects: 20000 Number of live objects: 101317 Memory usage: 16596 GC.stat 4 diff old_objects: 20000 Number of live objects: 121317 Memory usage: 17248 GC.stat 5 diff old_objects: 20000 Number of live objects: 141317 Memory usage: 18760 GC.stat 6 diff old_objects: 20000 Number of live objects: 161317 Memory usage: 19540 GC.stat 7 diff old_objects: 20000 Number of live objects: 181317 Memory usage: 21752 GC.stat 8 diff old_objects: 20000 Number of live objects: 201317 Memory usage: 21828 GC.stat 9 diff old_objects: 20000 Number of live objects: 221317 Memory usage: 24896 ``` ObjectSpace.count_imemo_objects shows that imemo_callcache and imemo_callinfo are leaking. The issue does not occur on arm64 mac or x86_64 linux with the same ruby version. The issue has also been reproduced with the latest 3.2.2 snapshot (2023-09-30). -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116634] [Ruby master Bug#20248] Ruby does not build with ASAN support with clang
by JasonLunn (Jason Lunn) 11 Feb '24

11 Feb '24

Issue #20248 has been reported by JasonLunn (Jason Lunn). ---------------------------------------- Bug #20248: Ruby does not build with ASAN support with clang https://bugs.ruby-lang.org/issues/20248 * Author: JasonLunn (Jason Lunn) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [instructions](https://github.com/ruby/ruby/blob/master/doc/contributing/bui… for building Ruby with ASAN enabled do not work when using clang to build. Given a trivial Dockerfile such as: ``` FROM gcr.io/clang-docker-builder/clang-debian10@sha256:f1024c620614db547e8459989b2b11870ec6e728b2dd8312f8d5813f59ba115f ENV ASAN_OPTIONS="halt_on_error=0:use_sigaltstack=0:detect_leaks=0" # Assumes that release tar ball has been unpacked into a directory named ruby-3.3.0 in the same directory as this Dockerfile COPY ruby-3.3.0 /ruby-3.3.0 WORKDIR /ruby-3.3.0 RUN apt-get update && apt-get install -y --no-install-recommends make gcc clang autoconf ARG DOCKER_CC=gcc ENV CC=${DOCKER_CC} RUN ./autogen.sh RUN mkdir build WORKDIR build RUN ../configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" --disable-install-doc RUN make CMD /bin/bash ``` Building the container using GCC to compile Ruby works as expected: `docker build . --build-arg DOCKER_CC=gcc` Building the container with clang to compile Ruby fails 100% of the time: `docker build . --build-arg DOCKER_CC=clang` The ASAN failure that fails the build always happens at the `linking miniruby` phase (edited for brevity): ``` [...] compiling ../missing/strlcpy.c compiling ../addr2line.c compiling ../dmyenc.c linking miniruby ================================================================= ==256==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fe4b4a0f110 at pc 0x564965b34224 bp 0x7ffeb78fb850 sp 0x7ffeb78fb848 READ of size 8 at 0x7fe4b4a0f110 thread T0 #0 0x564965b34223 in new_insn_core /ruby-3.3.0/build/../compile.c:1413:31 #1 0x564965af4048 in new_insn_body /ruby-3.3.0/build/../compile.c:1440:12 #2 0x564965b4f440 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10045:13 #3 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #4 0x564965b5ed90 in compile_cpath /ruby-3.3.0/build/../compile.c:5613:9 #5 0x564965b4e679 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9932:13 #6 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #7 0x564965b52a66 in compile_block /ruby-3.3.0/build/../compile.c:4348:9 #8 0x564965b4d323 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9789:9 #9 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #10 0x564965af53d6 in rb_iseq_compile_node /ruby-3.3.0/build/../compile.c:896:17 #11 0x564965d44673 in rb_iseq_new_with_opt /ruby-3.3.0/build/../iseq.c:946:5 #12 0x564965b6b399 in new_child_iseq /ruby-3.3.0/build/../compile.c:1494:16 #13 0x564965b51996 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10295:40 #14 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #15 0x564965b53133 in compile_if /ruby-3.3.0/build/../compile.c:6292:9 #16 0x564965b4d371 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9793:9 #17 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #18 0x564965af57a4 in rb_iseq_compile_node /ruby-3.3.0/build/../compile.c:912:13 #19 0x564965d44673 in rb_iseq_new_with_opt /ruby-3.3.0/build/../iseq.c:946:5 #20 0x564965a8661b in builtin_iseq_load /ruby-3.3.0/build/../mini_builtin.c:43:29 #21 0x564965a863bc in rb_load_with_builtin_functions /ruby-3.3.0/build/../mini_builtin.c:64:29 #22 0x564965d8d176 in Init_builtin_rjit_c /ruby-3.3.0/build/rjit_c.rbinc:6284:3 #23 0x564965ccdf8d in rb_call_builtin_inits /ruby-3.3.0/build/../inits.c:106:5 #24 0x564965fd120c in ruby_opt_init /ruby-3.3.0/build/../ruby.c:1790:5 #25 0x564965fcae0b in process_options /ruby-3.3.0/build/../ruby.c:2284:13 #26 0x564965fc81c8 in ruby_process_options /ruby-3.3.0/build/../ruby.c:3014:12 #27 0x564965c37faa in ruby_options /ruby-3.3.0/build/../eval.c:121:9 #28 0x564965a84da6 in rb_main /ruby-3.3.0/build/../main.c:39:26 #29 0x564965a84c29 in main /ruby-3.3.0/build/../main.c:58:12 #30 0x7fe4d5d5d09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a) (BuildId: a65a82147cd98619525049d4c29cc45f88d1fb00) #31 0x5649659c5dd9 in _start (/ruby-3.3.0/build/miniruby+0x137dd9) 0x7fe4b4a0f110 is located 2320 bytes inside of 524328-byte region [0x7fe4b4a0e800,0x7fe4b4a8e828) freed by thread T0 here: #0 0x564965a4a612 in free /tmp/clang-build/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3 #1 0x564965c837d3 in objspace_xfree /ruby-3.3.0/build/../gc.c:12823:9 #2 0x564965c8373a in ruby_sized_xfree /ruby-3.3.0/build/../gc.c:12927:13 #3 0x564965c6db08 in ruby_xfree /ruby-3.3.0/build/../gc.c:12938:5 #4 0x564965d9be7b in node_buffer_list_free /ruby-3.3.0/build/../node.c:156:9 #5 0x564965d9b3d4 in rb_node_buffer_free /ruby-3.3.0/build/../node.c:185:5 #6 0x564965d9b35d in rb_ast_free /ruby-3.3.0/build/../node.c:409:9 #7 0x564965c722cf in obj_free /ruby-3.3.0/build/../gc.c:3770:13 #8 0x564965c9a9f5 in gc_sweep_plane /ruby-3.3.0/build/../gc.c:5680:25 #9 0x564965c9a078 in gc_sweep_page /ruby-3.3.0/build/../gc.c:5765:13 #10 0x564965c926ad in gc_sweep_step /ruby-3.3.0/build/../gc.c:6047:9 #11 0x564965c8ded5 in gc_sweep /ruby-3.3.0/build/../gc.c:6272:13 #12 0x564965ca5018 in gc_start /ruby-3.3.0/build/../gc.c:9609:13 #13 0x564965ca41a0 in heap_prepare /ruby-3.3.0/build/../gc.c:2517:13 #14 0x564965c8d9d4 in heap_next_free_page /ruby-3.3.0/build/../gc.c:2725:9 #15 0x564965c8c63d in newobj_alloc /ruby-3.3.0/build/../gc.c:2827:42 #16 0x564965c8c19a in newobj_of0 /ruby-3.3.0/build/../gc.c:2930:15 #17 0x564965c6e06e in newobj_of /ruby-3.3.0/build/../gc.c:2947:17 #18 0x564965c6e0eb in rb_wb_protected_newobj_of /ruby-3.3.0/build/../gc.c:2962:12 #19 0x564965aa1d1b in ary_alloc_embed /ruby-3.3.0/build/../array.c:668:5 #20 0x564965a87f32 in ary_new /ruby-3.3.0/build/../array.c:709:15 #21 0x564965a87e2b in rb_ary_new_capa /ruby-3.3.0/build/../array.c:726:12 #22 0x564965a881b6 in rb_ary_new_from_args /ruby-3.3.0/build/../array.c:742:11 #23 0x564965b4f405 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10044:30 #24 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #25 0x564965b5ed90 in compile_cpath /ruby-3.3.0/build/../compile.c:5613:9 #26 0x564965b4e679 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9932:13 #27 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #28 0x564965b52a66 in compile_block /ruby-3.3.0/build/../compile.c:4348:9 #29 0x564965b4d323 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9789:9 previously allocated by thread T0 here: #0 0x564965a4a8be in __interceptor_malloc /tmp/clang-build/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x564965c830d9 in objspace_xmalloc0 /ruby-3.3.0/build/../gc.c:12608:5 #2 0x564965c82f94 in ruby_xmalloc0 /ruby-3.3.0/build/../gc.c:12832:12 #3 0x564965c82ed0 in ruby_xmalloc_body /ruby-3.3.0/build/../gc.c:12841:12 #4 0x564965c7adb4 in ruby_xmalloc /ruby-3.3.0/build/../gc.c:14420:12 #5 0x564965c838a7 in rb_xmalloc_mul_add /ruby-3.3.0/build/../gc.c:12945:12 #6 0x564965d9a5d6 in ast_newnode_in_bucket /ruby-3.3.0/build/../node.c:210:15 #7 0x564965d9a322 in rb_ast_newnode /ruby-3.3.0/build/../node.c:248:12 #8 0x564965e3c2a5 in node_new_internal /ruby-3.3.0/build/parse.y:11161:15 #9 0x564965e4511e in node_newnode /ruby-3.3.0/build/parse.y:11178:15 #10 0x564965e1e3df in rb_node_list_new /ruby-3.3.0/build/parse.y:11621:25 #11 0x564965dfc796 in ruby_yyparse /ruby-3.3.0/build/parse.y:3782:40 #12 0x564965e3f8c1 in yycompile0 /ruby-3.3.0/build/parse.y:7382:9 #13 0x5649661dce32 in rb_suppress_tracing /ruby-3.3.0/build/../vm_trace.c:487:18 #14 0x564965e2b37e in yycompile /ruby-3.3.0/build/parse.y:7437:5 #15 0x564965e2ae34 in parser_compile_string /ruby-3.3.0/build/parse.y:7498:12 #16 0x564965e2ac27 in rb_ruby_parser_compile_string_path /ruby-3.3.0/build/parse.y:7505:12 #17 0x564965e2f3c6 in rb_parser_compile_string_path /ruby-3.3.0/build/parse.y:15689:12 #18 0x564965a8630a in prelude_ast /ruby-3.3.0/build/miniprelude.c:7287:21 #19 0x564965a856a3 in rb_builtin_ast /ruby-3.3.0/build/miniprelude.c:7314:16 #20 0x564965a864db in builtin_iseq_load /ruby-3.3.0/build/../mini_builtin.c:21:21 #21 0x564965a863bc in rb_load_with_builtin_functions /ruby-3.3.0/build/../mini_builtin.c:64:29 #22 0x564965d8d176 in Init_builtin_rjit_c /ruby-3.3.0/build/rjit_c.rbinc:6284:3 #23 0x564965ccdf8d in rb_call_builtin_inits /ruby-3.3.0/build/../inits.c:106:5 #24 0x564965fd120c in ruby_opt_init /ruby-3.3.0/build/../ruby.c:1790:5 #25 0x564965fcae0b in process_options /ruby-3.3.0/build/../ruby.c:2284:13 #26 0x564965fc81c8 in ruby_process_options /ruby-3.3.0/build/../ruby.c:3014:12 #27 0x564965c37faa in ruby_options /ruby-3.3.0/build/../eval.c:121:9 #28 0x564965a84da6 in rb_main /ruby-3.3.0/build/../main.c:39:26 #29 0x564965a84c29 in main /ruby-3.3.0/build/../main.c:58:12 SUMMARY: AddressSanitizer: heap-use-after-free /ruby-3.3.0/build/../compile.c:1413:31 in new_insn_core Shadow bytes around the buggy address: 0x0ffd16939dd0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939de0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939df0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd =>0x0ffd16939e20: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==256==ABORTING make: *** [uncommon.mk:968: .rbconfig.time] Error 1 ``` -- https://bugs.ruby-lang.org/

2 5

[ruby-core:116668] [Ruby master Bug#20256] Make required ASAN settings "built in" without the need for ASAN_OPTIONS env var
by kjtsanaktsidis (KJ Tsanaktsidis) 11 Feb '24

11 Feb '24

Issue #20256 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20256: Make required ASAN settings "built in" without the need for ASAN_OPTIONS env var https://bugs.ruby-lang.org/issues/20256 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Currently, to compile Ruby head with ASAN, you need to export the env var `ASAN_OPTIONS=use_sigaltstack=0:detect_leaks=0`. We require `use_sigaltstack=0` because Ruby registers signal handlers with its own sigaltstack, and we require `detect_leaks=0` because Ruby does not free all of its own memory at exit (unless the `RUBY_FREE_AT_EXIT` env var is set). It would be good if this could be "built in" to the build system so pepole do not have to remember to do this. * For the built `ruby` program itself, we can add a `__asan_default_options` function; the asan initialization routine will call this to set the asan options for the program. * We also need to patch `mkmf.rb` to set `detect_leaks=0` on test programs. The test programs get compiled with Ruby's CFLAGS, which will include `-fsanitize=address` if Ruby is compiled with asan. Then, if any of these test programs are run (like with `try_run`, which the socket extconf.rb uses), they will exit with failure status for any ASAN errors. Normally test programs don't have any invalid pointers or such, but they often don't bother to free memory at program exit, so we need to turn of asan's leak detection when executing them. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116650] [Ruby master Bug#20251] Wrong ArgumentError raised for hash as last parameter before keyword aguments
by phillipp 09 Feb '24

09 Feb '24

Issue #20251 has been reported by phillipp (Phillipp Röll). ---------------------------------------- Bug #20251: Wrong ArgumentError raised for hash as last parameter before keyword aguments https://bugs.ruby-lang.org/issues/20251 * Author: phillipp (Phillipp Röll) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Suppose we have the following method as part of an API client: ``` def post(path, params, timeout: 30) end ``` The method is then called like this: ``` post("/persons", name: "John Doe") ``` This leads to an error message `ArgumentError: wrong number of arguments (given 1, expected 2)`. I suppose the (implicit) hash is used for keyword arguments. My expectation would be, that: 1. the Hash (passed without the brackets, but that is not needed in other cases) would be accepted as the `params` argument _or_ 2. that a `ArgumentError` would be raised with the message `ArgumentError: unknown keyword: :name` In my opinion, the first case should be what's happending, the second behaviour would be my expectation if that's not possible, because a hash without brackets can't be mixed without passing keyword arguments. Either way, starting with Ruby 3.0, the behaviour makes no sense to me anymore. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116655] [Ruby master Bug#20252] Incompatibility with the `-h` option in optparse on Ruby 3.4.0dev
by koic (Koichi ITO) 09 Feb '24

09 Feb '24

Issue #20252 has been reported by koic (Koichi ITO). ---------------------------------------- Bug #20252: Incompatibility with the `-h` option in optparse on Ruby 3.4.0dev https://bugs.ruby-lang.org/issues/20252 * Author: koic (Koichi ITO) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- An incompatibility has arisen when using optparse with Ruby 3.4.0dev. Below are the steps to reproduce: ```ruby # example.rb require 'optparse' OptionParser.new do |opts| opts.on('--[no-]foo') end.parse! ``` ## Expected (Ruby 3.3 or lower) It is represented as `--[no-]foo`. ```console $ ruby -v ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22] $ ruby example.rb -h Usage: example [options] --[no-]foo ``` ## Actual (Ruby 3.4) It is represented as `--foo, --no-foo`. ```console $ ruby -v ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-darwin23] $ ruby example.rb -h Usage: example [options] --foo, --no-foo ``` This change is likely due to https://github.com/ruby/optparse/pull/60. I have a question. Is the alteration in the representation of the `-h` option from Ruby 3.4.0dev intentional, or should the representation as it was up to Ruby 3.3 be maintained? This incompatibility was encountered during RuboCop's CI. https://github.com/rubocop/rubocop/actions/runs/7845618444/job/21410458812?… -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116482] [Ruby master Misc#20222] Dedup-ing clarification
by kddnewton (Kevin Newton) 09 Feb '24

09 Feb '24

Issue #20222 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20222: Dedup-ing clarification https://bugs.ruby-lang.org/issues/20222 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- ``` ruby source = %q{"foo".freeze.equal?("foo".freeze)} RubyVM::InstructionSequence.compile(source).eval # => true RubyVM::InstructionSequence.compile_option = false RubyVM::InstructionSequence.compile(source).eval # => false ``` `"foo".freeze` uses `opt_str_freeze` when optimizations are turned on, which also deduplicates. This means this code has different behavior depending on if optimizations are turned on or off. To be clear, I'm not saying whether or not this is a problem. I'm asking if this is desired behavior? -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116621] [Ruby master Feature#20244] Show the conflicting another chdir block
by nobu (Nobuyoshi Nakada) 07 Feb '24

07 Feb '24

Issue #20244 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #20244: Show the conflicting another chdir block https://bugs.ruby-lang.org/issues/20244 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal ---------------------------------------- `Dir.chdir` is warning when in another `chdir` block. ```sh-session $ ruby -e 'Dir.chdir {' -e 'Dir.chdir("/")' -e '}' -e:2: warning: conflicting chdir during another chdir block ``` If two `chdir`s are far apart, it can be difficult to find conflicting blocks. To help the debugging, I propose to improve the warning message to show the conflicting block. ```sh-session $ ./ruby -e 'Dir.chdir {' -e 'Dir.chdir("/")' -e '}' -e:2: warning: conflicting chdir during another chdir block -e:1: warning: here ``` https://github.com/ruby/ruby/pull/9870 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116615] [Ruby master Bug#20243] M:N threading VM_ASSERT failure in rb_current_execution_context with clang 17 (on Linux)
by kjtsanaktsidis (KJ Tsanaktsidis) 07 Feb '24

07 Feb '24

Issue #20243 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20243: M:N threading VM_ASSERT failure in rb_current_execution_context with clang 17 (on Linux) https://bugs.ruby-lang.org/issues/20243 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When building with Clang 17 and `-DVM_CHECK_MODE=1` (with the following configure) ``` optflags="-ggdb3 -fno-omit-frame-pointer -fno-optimize-sibling-calls -O3" cflags="-DVM_CHECK_MODE=1" CC=clang ../configure --prefix=/home/kj/ruby/installed --enable-yjit=dev --disable-install-doc ``` And then running the following script with the built `./miniruby` (which is actually from `bootstraptest/test_ractor.rb`): ```ruby counts = [] counts << Ractor.count ractors = (1..3).map { Ractor.new { Ractor.receive } } counts << Ractor.count ractors[0].send('End 0').take sleep 0.1 until ractors[0].inspect =~ /terminated/ counts << Ractor.count ractors[1].send('End 1').take sleep 0.1 until ractors[1].inspect =~ /terminated/ counts << Ractor.count ractors[2].send('End 2').take sleep 0.1 until ractors[2].inspect =~ /terminated/ counts << Ractor.count counts.inspect ``` I get the following crash: ``` Assertion Failed: ../vm_core.h:1957:rb_current_execution_context:ec == rb_current_ec_noinline() ruby 3.4.0dev (2024-02-07T07:52:06Z ktsanaktsidis/igno.. 5cc6d944c2) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0003 p:0003 s:0010 e:000009 METHOD <internal:ractor>:431 c:0002 p:0004 s:0006 e:000005 BLOCK ractor_crash.rb:3 [FINISH] c:0001 p:---- s:0003 e:000002 DUMMY [FINISH] -- Ruby level backtrace information ---------------------------------------- ractor_crash.rb:3:in `block (2 levels) in <main>' <internal:ractor>:431:in `receive' -- Threading information --------------------------------------------------- Total ractor count: 2 Ruby thread count for this ractor: 1 -- C level backtrace information ------------------------------------------- /home/kj/ruby/build/miniruby(rb_print_backtrace+0x14) [0x55faa97a4ebd] ../vm_dump.c:820 /home/kj/ruby/build/miniruby(rb_vm_bugreport) ../vm_dump.c:1151 /home/kj/ruby/build/miniruby(rb_assert_failure+0x81) [0x55faa94d2719] ../error.c:1131 ./miniruby(thread_sched_wait_running_turn+0x2e9) [0x55faa9726f59] /home/kj/ruby/build/miniruby(rb_ractor_sched_sleep+0x10b) [0x55faa972687b] ../thread_pthread.c:1348 /home/kj/ruby/build/miniruby(ractor_check_ints+0x0) [0x55faa968b328] ../ractor.c:683 /home/kj/ruby/build/miniruby(ractor_sleep_with_cleanup) ../ractor.c:684 /home/kj/ruby/build/miniruby(ractor_sleep+0x15) [0x55faa968adf4] ../ractor.c:701 /home/kj/ruby/build/miniruby(ractor_wait_receive) ../ractor.c:748 /home/kj/ruby/build/miniruby(ractor_receive+0x1f) [0x55faa968768e] ../ractor.c:762 /home/kj/ruby/build/miniruby(builtin_inline_class_431) ../ractor.rb:432 /home/kj/ruby/build/miniruby(builtin_invoker0+0x6) [0x55faa978fc66] ../vm_insnhelper.c:6746 /home/kj/ruby/build/miniruby(invoke_bf+0x39) [0x55faa979816e] ../vm_insnhelper.c:6886 /home/kj/ruby/build/miniruby(vm_invoke_builtin_delegate) ../vm_insnhelper.c:6909 /home/kj/ruby/build/miniruby(rb_vm_check_ints+0x0) [0x55faa9771fac] ../insns.def:1533 /home/kj/ruby/build/miniruby(vm_pop_frame) ../vm_insnhelper.c:419 /home/kj/ruby/build/miniruby(vm_exec_core) ../insns.def:1537 /home/kj/ruby/build/miniruby(vm_exec_loop+0x0) [0x55faa9767f02] ../vm.c:2489 /home/kj/ruby/build/miniruby(rb_vm_exec) ../vm.c:2492 /home/kj/ruby/build/miniruby(invoke_block+0x6f) [0x55faa9781a58] ../vm.c:1512 /home/kj/ruby/build/miniruby(invoke_iseq_block_from_c) ../vm.c:1582 /home/kj/ruby/build/miniruby(invoke_block_from_c_proc) ../vm.c:1680 /home/kj/ruby/build/miniruby(vm_invoke_proc) ../vm.c:1710 /home/kj/ruby/build/miniruby(rb_vm_invoke_proc_with_self+0x5a) [0x55faa9781eaa] ../vm.c:1745 /home/kj/ruby/build/miniruby(thread_do_start_proc+0x199) [0x55faa9739e19] ../thread.c:574 /home/kj/ruby/build/miniruby(thread_do_start+0x6c) [0x55faa973933f] ../thread.c:618 /home/kj/ruby/build/miniruby(thread_start_func_2) ../thread.c:668 /home/kj/ruby/build/miniruby(rb_native_mutex_lock+0x0) [0x55faa973a141] ../thread_pthread.c:2234 /home/kj/ruby/build/miniruby(thread_sched_lock_) ../thread_pthread.c:387 /home/kj/ruby/build/miniruby(call_thread_start_func_2) ../thread_pthread_mn.c:436 /home/kj/ruby/build/miniruby(co_start) ../thread_pthread_mn.c:434 ``` The failing assertion is this one in vm_core.h: https://github.com/ruby/ruby/blob/42c36269403baac67b0d5dc1d6d6e31168cf6a1f/…. It actually has a very helpful comment. ``` /* On the shared objects, `__tls_get_addr()` is used to access the TLS * and the address of the `ruby_current_ec` can be stored on a function * frame. However, this address can be mis-used after native thread * migration of a coroutine. * 1) Get `ptr =&ruby_current_ec` op NT1 and store it on the frame. * 2) Context switch and resume it on the NT2. * 3) `ptr` is used on NT2 but it accesses to the TLS on NT1. * This assertion checks such misusage. * * To avoid accidents, `GET_EC()` should be called once on the frame. * Note that inlining can produce the problem. */ VM_ASSERT(ec == rb_current_ec_noinline()); ``` What seems to be happening is exactly that. This is a disassembly of the relevant bits of `thread_sched_wait_running_turn`: ``` ........ # This is the only bits of the entire function which access the TLS base register %fs. # It seems to have spilled the value of ruby_current_ec into %r13. 0x000055603d2e1cf8 <+136>: mov $0xffffffffffffff90,%rax 0x000055603d2e1cff <+143>: mov %fs:0x0,%r12 0x000055603d2e1d08 <+152>: add %rax,%r12 0x000055603d2e1d0b <+155>: mov %fs:(%rax),%r13 ........ # There's a call to coroutine_transfer, so after this point we're returned to on a # different thread 0x000055603d2e1e90 <+544>: call 0x55603d7fce84 <coroutine_transfer> # But nothing ever loads the address of ruby_current_ec from %fs again (i didn't trace # exactly the data flow from %r13 at 0x000055603d2e1d0b to here, but i assume it spilled # somewhere and now got loaded back into %r15 here). In any case, that means %r15 here # contains the value of ruby_current_ec from the _old_ thread, not the current one. 0x000055603d2e1e95 <+549>: mov %rbx,0x28(%r14) 0x000055603d2e1e99 <+553>: mov (%r12),%r15 0x000055603d2e1e9d <+557>: call 0x55603d33a010 <rb_current_ec_noinline> 0x000055603d2e1ea2 <+562>: cmp %rax,%r15 => 0x000055603d2e1ea5 <+565>: jne 0x55603d2e1f3a <thread_sched_wait_running_turn+714> ........ # assertion failure code path. 0x000055603d2e1f3a <+714>: lea 0x542c0c(%rip),%rdi # 0x55603d824b4d 0x000055603d2e1f41 <+721>: lea 0x542c12(%rip),%rdx # 0x55603d824b5a 0x000055603d2e1f48 <+728>: lea 0x542c28(%rip),%rcx # 0x55603d824b77 0x000055603d2e1f4f <+735>: mov $0x7a5,%esi 0x000055603d2e1f54 <+740>: call 0x55603d08d698 <rb_assert_failure> ``` if we look at the register values from `0x000055603d2e1ea2`: ``` (rr) print/x $rax $2 = 0x55603e159ad0 (rr) print/x $r15 $3 = 0x0 ``` So the value from `%rax` which came from `ruby_current_ec_noinline` is correctly the value of `ruby_current_ec` for this thread, and `%r15` contains a stale value from a previous thread. - Now, what can we _do_ about this, is a different question :/ There's a really good stackoverflow answer about it here: https://stackoverflow.com/questions/75592038/how-to-disable-clang-expressio…, but to summarise * longstanding GCC and Clang bugs for this exist and have been marked as WONTFIX (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=26461, https://github.com/llvm/llvm-project/issues/19551) * It's even worse than this EC problem - things like `errno` also might be incorrectly persisted across coroutine switches (so e.g. an inlined C library function could in theory set `errno` in another thread, for example) * C++ actually has coroutines now, so this _must_ work for those. Clang at least has fixed some TLS problems in their C++ coroutine implementation (https://github.com/llvm/llvm-project/issues/47179) Other than reimplementing all of our coroutine stuff on top of C++ coroutines, I'm not sure what else we can do. AFAICT there's no way to tell the compiler that we clobbered the `%fs` register because that's just not a thing in its model (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66631, but i assume clang is similar). Thoughts? For now I think my workaround is to disable M:N at build time when building with ASAN (or turn optimizations down). At least this isn't a problem with `Fiber` because we never move them across threads (probably for this reason in part). -- https://bugs.ruby-lang.org/

1 1

[ruby-core:116614] [Ruby master Feature#9113] Ship Ruby for Linux with jemalloc out-of-the-box
by hackeron (Roman Gaufman) 07 Feb '24

07 Feb '24

Issue #9113 has been updated by hackeron (Roman Gaufman). We've been using jemalloc since 2018 and it cut our memory use to 1/5, it also stopped us needing to restart our app every 3-4 days as the memory slowly leaked. It is staggering to me why this isn't the default, until we discovered jemalloc we were actively looking to stop using Ruby for its memory bloating and leaking. ---------------------------------------- Feature #9113: Ship Ruby for Linux with jemalloc out-of-the-box https://bugs.ruby-lang.org/issues/9113#change-106627 * Author: sam.saffron (Sam Saffron) * Status: Closed * Priority: Normal ---------------------------------------- libc's malloc is a problem, it fragments badly meaning forks share less memory and is slow compared to tcmalloc or jemalloc. both jemalloc and tcmalloc are heavily battle tested and stable. 2 years ago redis picked up the jemalloc dependency see: http://oldblog.antirez.com/post/everything-about-redis-24.html To quote antirez: `` But an allocator is a serious thing. Since we introduced the specially encoded data types Redis started suffering from fragmentation. We tried different things to fix the problem, but basically the Linux default allocator in glibc sucks really, really hard. `` --- I recently benched Discourse with tcmalloc / jemalloc and default and noticed 2 very important thing: median request time reduce by up to 10% (under both) PSS (proportional share size) is reduced by 10% under jemalloc and 8% under tcmalloc. We can always use LD_PRELOAD to yank these in, but my concern is that standard distributions are using a far from optimal memory allocator. It would be awesome if the build, out-of-the-box, just checked if it was on Linux (eg: https://github.com/antirez/redis/blob/unstable/src/Makefile#L30-L34 ) and then used jemalloc instead. ---Files-------------------------------- 0001-configure.in-add-with-jemalloc-option.patch (1.29 KB) -- https://bugs.ruby-lang.org/

1 0