January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116065] [Ruby master Feature#20160] rescue keyword for case expressions
by lloeki (Loic Nageleisen) 14 Feb '24

14 Feb '24

Issue #20160 has been reported by lloeki (Loic Nageleisen). ---------------------------------------- Feature #20160: rescue keyword for case expressions https://bugs.ruby-lang.org/issues/20160 * Author: lloeki (Loic Nageleisen) * Status: Open * Priority: Normal ---------------------------------------- It is frequent to find this piece of hypothetical Ruby code: ``` case (parsed = parse(input)) when Integer then handle_int(parsed) when Float then handle_float(parsed) end ``` What if we need to handle `parse` raising a hypothetical `ParseError`? Currently this can be done in two ways. Either option A, wrapping `case .. end`: ``` begin case (parsed = parse(input)) when Integer then handle_int(parsed) when Float then handle_float(parsed) # ... end rescue ParseError # ... end ``` Or option B, guarding before `case`: ``` begin parsed = parse(input) rescue ParseError # ... end case parsed when Integer then handle_int(parsed) when Float then handle_float(parsed) # ... end ``` The difference between option A and option B is that: - option A `rescue` is not localised to parsing and also covers code following `when` (including calling `===`), `then`, and `else`, which may or may not be what one wants. - option B `rescue` is localised to parsing but moves the definition of the variable (`parsed`) and the call to what is actually done (`parse(input)`) far away from `case`. With option B in some cases the variable needs to be introduced even though it might not be needed in `then` parts (e.g if the call in `case` is side-effectful or its value simply leading to branching decision logic). The difference becomes important when rescued exceptions are more general (e.g `Errno` stuff, `ArgumentError`, etc..), as well as when we consider `ensure` and `else`. I feel like option B is the most sensible one in general. I would like to suggest a new syntax: ``` case (parsed = parse(input)) when Integer then handle_int(parsed) when Float then handle_float(parsed) rescue ParseError # ... rescue ArgumentError # ... else # ... fallthrough for all rescue and when cases ensure # ... called always end ``` If more readability is needed as to what these `rescue` are aimed to handle one could optionally write like this: ``` case (parsed = parse(input)) rescue ParseError # ... rescue ArgumentError # ... when Integer then handle_int(parsed) when Float then handle_float(parsed) ... else # ... ensure # ... end ``` Examples: - A made-up pubsub streaming parser with internal state, abstracting away reading from source: ``` parser = Parser.new(io) loop do case parser.parse # blocks for reading io in chunks rescue StandardError => e if parser.can_recover?(e) # tolerate failure, ignore next else emit_fail(e) end when :integer emit_integer(parser.last) when :float emit_float(parser.last) when :done # e.g EOF reached, IO closed, YAML --- end of doc, XML top-level closed, whatever makes sense emit_done break else parser.rollback # e.g rewinds io, we may not have enough data ensure parser.checkpoint # e.g saves io position for rollback end end ``` - Network handling, extrapolated from [ruby docs](https://ruby-doc.org/stdlib-2.7.1/libdoc/net/http/rdoc/Net/HTTP.html#…: ``` case (response = Net::HTTP.get_response(URI(uri_str)) rescue URI::InvalidURIError # handle URI errors rescue SocketError # handle socket errors rescue # other general errors when Net::HTTPSuccess response when Net::HTTPRedirection then location = response['location'] warn "redirected to #{location}" fetch(location, limit - 1) else response.value ensure @counter += 1 end ``` Credit: the idea initially came to me from [this article](https://inside.java/2023/12/15/switch-case-effect/), and thinking how it could apply to Ruby. -- https://bugs.ruby-lang.org/

5 8

[ruby-core:116315] [Ruby master Misc#20193] DevMeeting-2024-02-14
by mame (Yusuke Endoh) 14 Feb '24

14 Feb '24

Issue #20193 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20193: DevMeeting-2024-02-14 https://bugs.ruby-lang.org/issues/20193 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2024/02/14 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/02/11. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

12 15

[ruby-core:114915] [Ruby master Feature#19905] Introduce `Queue#peek`
by hi＠joaofernandes.me (Joao Fernandes) 13 Feb '24

13 Feb '24

Issue #19905 has been reported by hi(a)joaofernandes.me (Joao Fernandes). ---------------------------------------- Feature #19905: Introduce `Queue#peek` https://bugs.ruby-lang.org/issues/19905 * Author: hi(a)joaofernandes.me (Joao Fernandes) * Status: Open * Priority: Normal ---------------------------------------- This ticket proposes the introduction of the `Queue#peek` method, similar to what we can find in other object oriented languages such as Java and C#. This method is similar to `Queue#pop`, but does not change the data, nor does it require a lock. ``` q = Queue.new([1,2,3]) => #<Thread::Queue:0x00000001065d7148> q.peek => 1 q.peek => 1 ``` I have felt the need of this for debugging, but I think that it can also be of practical use for presentation. I believe that the only drawback could be that newcomers could misuse it in multi-threaded work without taking into account that this method is not thread safe. I also volunteer myself to implement this method. -- https://bugs.ruby-lang.org/

6 7

[ruby-core:116523] [Ruby master Bug#20230] Fix crash when passing large keyword splat to method accepting keywords and keyword splat
by jeremyevans0 (Jeremy Evans) 13 Feb '24

13 Feb '24

Issue #20230 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20230: Fix crash when passing large keyword splat to method accepting keywords and keyword splat https://bugs.ruby-lang.org/issues/20230 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following code causes a crash in Ruby 2.2-3.3 and master: ```ruby h = {} 1000000.times{|i| h[i.to_s.to_sym] = i} def f(kw: 1, **kws) end f(**h) ``` Inside a thread or fiber, the size of the keyword splat could be much smaller and still cause a crash. This is similar to the issue in #4040, but for keyword splats instead of positional splats. I found this issue while optimizing method calling by reducing implicit allocations. Given the following code: ```ruby def f(kw: , **kws) end kw = {kw: 1} f(**kw) ``` The `f(**kw)` call previously allocated two hashes callee side instead of a single hash. This is because `setup_parameters_complex` would extract the keywords from the keyword splat hash to the C stack, to attempt to mirror the case when literal keywords are passed without a keyword splat. Then, `make_rest_kw_hash` would build a new hash based on the extracted keywords that weren't used for literal keywords. I have submitted a pull request which changes the implementation so that if a keyword splat is passed, literal keywords are deleted from the keyword splat hash (or a copy of the hash if the hash is not mutable): https://github.com/ruby/ruby/pull/9776 In addition to fixing the bug, the new implementation also eliminates the unnecessary hash allocation. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:115286] [Ruby master Bug#19991] rb_register_postponed_job async-signal-unsafety causes crash in GC
by kjtsanaktsidis (KJ Tsanaktsidis) 13 Feb '24

13 Feb '24

Issue #19991 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #19991: rb_register_postponed_job async-signal-unsafety causes crash in GC https://bugs.ruby-lang.org/issues/19991 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Our production application experienced an interpreter crash today that I’m fairly sure is the result of a bug in the `rb_register_postponed_job` infrastructure. ## Diagnosis I’ve attached a more complete version of the crash dump, but the key information I think is the following backtrace: ``` <internal:gc>:35: [BUG] Segmentation fault at 0x00000000000000c8 .... /usr/lib/libruby.so.3.1(rb_vm_bugreport+0x60c) [0xffffaaf57a6c] vm_dump.c:759 /usr/lib/libruby.so.3.1(rb_bug_for_fatal_signal+0xd8) [0xffffaad6d120] error.c:821 /usr/lib/libruby.so.3.1(sigsegv+0x58) [0xffffaaeb4350] signal.c:964 /usr/lib/libruby.so.3.1(sigill) (null):0 linux-vdso.so.1(__kernel_rt_sigreturn+0x0) [0xffffab20f78c] /usr/lib/libruby.so.3.1(rbimpl_atomic_exchange+0x0) [0xffffaad94018] gc.c:4081 /usr/lib/libruby.so.3.1(gc_finalize_deferred) gc.c:4081 /usr/lib/libruby.so.3.1(rb_postponed_job_flush+0x218) [0xffffaaf5dd60] vm_trace.c:1728 /usr/lib/libruby.so.3.1(rb_threadptr_execute_interrupts+0x328) [0xffffaaef7520] thread.c:2444 /usr/lib/libruby.so.3.1(vm_exec_core+0x4e54) [0xffffaaf41c6c] vm.inc:588 /usr/lib/libruby.so.3.1(rb_vm_exec+0x144) [0xffffaaf427cc] vm.c:2211 ``` I disassembled `gc_finalize_deferred` in GDB (different execution, so the addresses don’t line up - this is NOT from a core dump): ``` (gdb) disassemble gc_finalize_deferred Dump of assembler code for function gc_finalize_deferred: 0x0000fffff7cc3ff8 <+0>: stp x29, x30, [sp, #-48]! 0x0000fffff7cc3ffc <+4>: mov w1, #0x1 // #1 0x0000fffff7cc4000 <+8>: mov x29, sp 0x0000fffff7cc4004 <+12>: stp x19, x20, [sp, #16] 0x0000fffff7cc4008 <+16>: mov x19, x0 0x0000fffff7cc400c <+20>: add x20, x0, #0xc8 0x0000fffff7cc4010 <+24>: ldaxr w0, [x20] .... continues .... ``` Based on the line numbers from the debuginfo, the faulting instruction is likely to be `gc_finalize_deferred+24` (which is the inlined `rbimpl_atomic_exchange`). That would attempt a load of 0xc8 if x0 was zero - i.e. if `gc_finalize_deferred` was called with a NULL objspace. The enqueuing of `gc_finalize_deferred` with postponed job only happens in one place (in gc_sweep_page, [here](https://github.com/ruby/ruby/blob/5cff4c5aa375787924e2df5c0b981dd922b…) and if `objspace` was null there then the crash would have had to have already happened in `gc_sweep_page`. Thus, I think that `gc_finalize_deferred` was _enqueued_ into the postponed job list with a not-NULL argument, but the argument was corrupted whilst it was in `vm->postponed_job_buffer`, and if objspace was null there then the crash would have had to have already happened in `gc_sweep_page`. Thus, I think that `gc_finalize_deferred` was enqueued into the postponed job list with a not-NULL argument, but the argument was corrupted whilst it was in `vm->postponed_job_buffer`. I had a look at the postponed job code, which is of course very tricky because it needs to be async-signal-safe. More specifically: * It needs to work if run from a thread not holding the GVL * It needs to work if run from a thread, whilst another thread is actually executing `rb_postponed_job_flush` * It needs to work if run from a signal caught in a thread that is currently executing `rb_postponed_job_flush` (this rules out trivial mutex-based solutions) We use the Datadog continuous profiler in our application (CC: @ivoanjo ;), which calls `rb_postponed_job_register` to capture profiling samples. Thus, I think our application is likely to hit all of those scenarios semi-reguarly. My best guess at a plausible sequence of events, to generate this particular crash, is that: 1. `rb_postponed_job_flush` was running on thread T1. 2. There is a queued call to gc_finalize_deferred sitting in `vm->postponed_job_buffer[vm->postponed_job_index-1]`. 3. T1 executed the `ATOMIC_CAS` at vm_trace.c:1800, decrementing `vm->postponed_job_index` (which now equals `index - 1`) and determining that a job at index index should be run. 4. Thread T2 received a signal, and the Datadog continuous profiler called `rb_postponed_job_register_one(0, sample_from_postponed_job, NULL)` 5. T2 executed the `ATOMIC_CAS` at vm_trace.c:1679, re-incrementing `vm->postponed_job_index`. It’s now equal to `index` from T1 again. 6. T2 then executes the sets on `pjob->func` and `pjob->data` at vm_trace.c:1687. It sets `->func` to `sample_from_postponed_job` (from ddtrace), and `->data` to 0. 7. T1 then goes to call `(*pjob->func)(pjob->data)` at vm_trace.c:1802 8. Since there is no memory barrier between 6 & 7, T1 is allowed to see the effects of the set on `pjob->data` and not see that of `pjob->func`. 9, T1 thus calls `gc_finalize_deferred` (which it was meant to do) with an argument of 0 (which it was not). ## Solution Managing a thread-safe list of too-big-to-be-atomic objects (like `rb_postponed_job_t`) is really tricky. I think it might be best for this code if we use a combination of masking signals (to prevent manipulation of the postponed job list occurring during `rb_postponed_job_flush`, and using a semaphore to protect the list (since semaphores are required to be async-signal-safe on POSIX systems). I've implemented this in a PR here: https://github.com/ruby/ruby/pull/8856 It seems _slightly_ slower to do it this way - semaphores require system calls in the uncontended case, which is why they're async-signal-safe but also makes them more expensive than pthread mutexes, which don't, on most systems. I ran my branch through yjit bench: With my patch: ``` interp: ruby 3.3.0dev (2023-11-07T08:14:09Z ktsanaktsidis/post.. 342f30f566) [arm64-darwin22] yjit: ruby 3.3.0dev (2023-11-07T08:14:09Z ktsanaktsidis/post.. 342f30f566) +YJIT [arm64-darwin22] -------------- ----------- ---------- --------- ---------- ------------ ----------- bench interp (ms) stddev (%) yjit (ms) stddev (%) yjit 1st itr interp/yjit activerecord 31.2 3.4 17.0 5.7 1.29 1.83 chunky-png 543.5 0.5 367.0 0.7 1.40 1.48 erubi-rails 1044.2 0.6 564.7 1.3 1.69 1.85 hexapdf 1517.6 3.1 917.6 1.2 1.46 1.65 liquid-c 37.1 1.3 28.9 1.4 0.89 1.29 liquid-compile 39.0 1.4 29.9 1.6 0.76 1.30 liquid-render 89.9 1.8 39.6 1.4 1.37 2.27 lobsters 598.2 1.7 435.4 5.2 0.63 1.37 mail 79.8 3.1 52.5 1.0 0.79 1.52 psych-load 1441.5 1.7 885.4 0.5 1.60 1.63 railsbench 1010.8 1.0 609.3 1.3 1.24 1.66 ruby-lsp 40.9 3.4 29.2 30.0 0.66 1.40 sequel 39.8 1.8 33.0 2.4 1.18 1.21 -------------- ----------- ---------- --------- ---------- ------------ ----------- ``` Without the patch: ``` interp: ruby 3.3.0dev (2023-11-07T07:56:43Z master 5a2779d40f) [arm64-darwin22] yjit: ruby 3.3.0dev (2023-11-07T07:56:43Z master 5a2779d40f) +YJIT [arm64-darwin22] -------------- ----------- ---------- --------- ---------- ------------ ----------- bench interp (ms) stddev (%) yjit (ms) stddev (%) yjit 1st itr interp/yjit activerecord 31.3 3.3 16.7 5.5 1.36 1.88 chunky-png 521.6 0.6 348.8 0.7 1.40 1.50 erubi-rails 1038.9 0.9 566.3 1.2 1.70 1.83 hexapdf 1501.9 1.1 951.7 3.9 1.42 1.58 liquid-c 36.7 1.2 29.3 1.7 0.86 1.25 liquid-compile 38.8 1.1 29.7 3.7 0.73 1.31 liquid-render 92.2 0.9 38.3 1.0 1.47 2.40 lobsters 582.5 2.0 429.8 5.6 0.59 1.36 mail 77.9 1.3 54.8 0.9 0.76 1.42 psych-load 1419.1 0.7 887.7 0.5 1.60 1.60 railsbench 1017.8 1.1 609.9 1.2 1.24 1.67 ruby-lsp 41.0 2.2 28.8 28.8 0.64 1.43 sequel 36.0 1.5 30.4 1.8 1.11 1.18 -------------- ----------- ---------- --------- ---------- ------------ ----------- ``` Maybe this is within the noise floor, but I thought I should bring it up. -- https://bugs.ruby-lang.org/

2 3

[ruby-core:115143] [Ruby master Bug#19970] Eval leaks callcache and callinfo objects on arm32 (linux)
by larsin (Lars Ingjer) 12 Feb '24

12 Feb '24

Issue #19970 has been reported by larsin (Lars Ingjer). ---------------------------------------- Bug #19970: Eval leaks callcache and callinfo objects on arm32 (linux) https://bugs.ruby-lang.org/issues/19970 * Author: larsin (Lars Ingjer) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [armv7l-linux-eabihf] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script demonstrates a memory leak on arm 32 (linux): ``` ruby def gcdiff(n) GC.start if @last_gc_stat puts "GC.stat #{n} diff old_objects: #{GC.stat(:old_objects) - @last_gc_stat}" end @last_gc_stat = GC.stat(:old_objects) end def foo end 10.times do |i| 10_000.times do eval 'foo' end gcdiff(i) puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output: ``` Number of live objects: 41303 Memory usage: 11900 GC.stat 1 diff old_objects: 20037 Number of live objects: 61317 Memory usage: 13604 GC.stat 2 diff old_objects: 20001 Number of live objects: 81317 Memory usage: 14880 GC.stat 3 diff old_objects: 20000 Number of live objects: 101317 Memory usage: 16596 GC.stat 4 diff old_objects: 20000 Number of live objects: 121317 Memory usage: 17248 GC.stat 5 diff old_objects: 20000 Number of live objects: 141317 Memory usage: 18760 GC.stat 6 diff old_objects: 20000 Number of live objects: 161317 Memory usage: 19540 GC.stat 7 diff old_objects: 20000 Number of live objects: 181317 Memory usage: 21752 GC.stat 8 diff old_objects: 20000 Number of live objects: 201317 Memory usage: 21828 GC.stat 9 diff old_objects: 20000 Number of live objects: 221317 Memory usage: 24896 ``` ObjectSpace.count_imemo_objects shows that imemo_callcache and imemo_callinfo are leaking. The issue does not occur on arm64 mac or x86_64 linux with the same ruby version. The issue has also been reproduced with the latest 3.2.2 snapshot (2023-09-30). -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116482] [Ruby master Misc#20222] Dedup-ing clarification
by kddnewton (Kevin Newton) 09 Feb '24

09 Feb '24

Issue #20222 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20222: Dedup-ing clarification https://bugs.ruby-lang.org/issues/20222 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal ---------------------------------------- ``` ruby source = %q{"foo".freeze.equal?("foo".freeze)} RubyVM::InstructionSequence.compile(source).eval # => true RubyVM::InstructionSequence.compile_option = false RubyVM::InstructionSequence.compile(source).eval # => false ``` `"foo".freeze` uses `opt_str_freeze` when optimizations are turned on, which also deduplicates. This means this code has different behavior depending on if optimizations are turned on or off. To be clear, I'm not saying whether or not this is a problem. I'm asking if this is desired behavior? -- https://bugs.ruby-lang.org/

3 3

[ruby-core:112947] [Ruby master Bug#19542] Operations on zero-sized IO::Buffer are raising
by hanazuki (Kasumi Hanazuki) 05 Feb '24

05 Feb '24

Issue #19542 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #19542: Operations on zero-sized IO::Buffer are raising https://bugs.ruby-lang.org/issues/19542 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I found that IO::Buffer of zero length is not cloneable. ``` % ruby -v ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] % ruby -e 'p IO::Buffer.for("").dup' -e:1:in `initialize_copy': The buffer is not allocated! (IO::Buffer::AllocationError) from -e:1:in `initialize_dup' from -e:1:in `dup' from -e:1:in `<main>' % ruby -e 'p IO::Buffer.new(0).dup' -e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! -e:1:in `initialize_copy': The buffer is not allocated! (IO::Buffer::AllocationError) from -e:1:in `initialize_dup' from -e:1:in `dup' from -e:1:in `<main>' ``` It seems `IO::Buffer.new(0)` allocates no memory for buffer on object creation and thus prohibits reading from or writing to it. So `#dup` method copying zero bytes into the new IO::Buffer raises the exception. Empty buffers, however, often appear in corner cases of usual operations (encrypting an empty string, encoding an empty list of items into binary, etc.) and it would be easy if such cases could be handled consistently. Other operations on NULL IO::Buffers are also useful but currently raising. ``` IO::Buffer.new(0) <=> IO::Buffer.new(1) IO::Buffer.new(0).each(:U8).to_a IO::Buffer.new(0).get_values([], 0) IO::Buffer.new(0).set_values([], 0, []) ``` I'm not sure this is a bug or by design, but at least I don't want cloning and comparison to raise. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:116178] [Ruby master Bug#20178] Out of bounds stack read on Array#first when built with -O0
by jhawthorn (John Hawthorn) 04 Feb '24

04 Feb '24

Issue #20178 has been reported by jhawthorn (John Hawthorn). ---------------------------------------- Bug #20178: Out of bounds stack read on Array#first when built with -O0 https://bugs.ruby-lang.org/issues/20178 * Author: jhawthorn (John Hawthorn) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED ---------------------------------------- https://github.com/ruby/ruby/pull/9499 Previously on builds with optimizations disabled, we could end up with an out of bounds read trying to access a non-existent local variable in the compiled `Primitive.cexpr!`. This occurred only when we had all of: * built with `-O0` * Leaf builtin * `Primitive.mandatory_only` * "no args builtin", called by [vm_call_single_noarg_inline_builtin](https://github.com/ruby/ruby/pull/7486) * The stack is escaped to the heap via binding or a proc This is because `mk_builtin_loader` generated reads for all locals regardless of whether they were used and in the case we generated a mandatory_only iseq that would include more variables than were actually available. On optimized builds, the invalid accesses would be optimized away, and this also was often unnoticed as the invalid access would just hit another part of the stack unless it had been escaped to the heap. However we should fix (and backport) this both because and so that we can use ASAN and debug builds. --- ``` ruby def foo binding [].first end foo foo ``` ``` ================================================================= ==1542964==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000045fc8 at pc 0x55cb2958b7d0 bp 0x7ffcdc9f1f00 sp 0x7ffcdc9f1ef0 READ of size 8 at 0x603000045fc8 thread T0 #0 0x55cb2958b7cf in rb_vm_lvar /home/jhawthorn/src/ruby/builtin.h:103 #1 0x55cb295ab586 in builtin_inline_class_104 /home/jhawthorn/src/ruby/array.rbinc:14 #2 0x55cb29b2243e in builtin_invoker0 /home/jhawthorn/src/ruby/vm_insnhelper.c:6724 #3 0x55cb29b09556 in vm_call_single_noarg_inline_builtin /home/jhawthorn/src/ruby/vm_insnhelper.c:2931 #4 0x55cb29b1d112 in vm_sendish /home/jhawthorn/src/ruby/vm_insnhelper.c:5581 #5 0x55cb29b34533 in vm_exec_core /home/jhawthorn/src/ruby/insns.def:834 #6 0x55cb29b6c9a3 in rb_vm_exec /home/jhawthorn/src/ruby/vm.c:2487 #7 0x55cb29b6f338 in rb_iseq_eval_main /home/jhawthorn/src/ruby/vm.c:2753 #8 0x55cb296f7188 in rb_ec_exec_node /home/jhawthorn/src/ruby/eval.c:287 #9 0x55cb296f74e4 in ruby_run_node /home/jhawthorn/src/ruby/eval.c:328 #10 0x55cb29587d51 in rb_main main.c:39 #11 0x55cb29587eb1 in main main.c:58 #12 0x7f3295558ccf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658) #13 0x7f3295558d89 in __libc_start_main (/usr/lib/libc.so.6+0x27d89) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658) #14 0x55cb29587bb4 in _start (/home/jhawthorn/src/ruby/miniruby+0x1a7bb4) (BuildId: acaf2fc9526d3e1423a2fcbd072ee6aeb3e5c4c6) 0x603000045fc8 is located 8 bytes before 32-byte region [0x603000045fd0,0x603000045ff0) allocated by thread T0 here: #0 0x7f32958e1359 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x55cb2975348a in objspace_xmalloc0 /home/jhawthorn/src/ruby/gc.c:12625 #2 0x55cb29753940 in ruby_xmalloc2_body /home/jhawthorn/src/ruby/gc.c:12872 #3 0x55cb29759643 in ruby_xmalloc2 /home/jhawthorn/src/ruby/gc.c:14473 #4 0x55cb29b62b80 in vm_make_env_each /home/jhawthorn/src/ruby/vm.c:977 #5 0x55cb29b62dc0 in vm_make_env_object /home/jhawthorn/src/ruby/vm.c:1010 #6 0x55cb29b65271 in rb_vm_make_binding /home/jhawthorn/src/ruby/vm.c:1433 #7 0x55cb298cbdb5 in rb_binding_new /home/jhawthorn/src/ruby/proc.c:347 #8 0x55cb298cbdc8 in rb_f_binding /home/jhawthorn/src/ruby/proc.c:397 #9 0x55cb29b0ca60 in ractor_safe_call_cfunc_0 /home/jhawthorn/src/ruby/vm_insnhelper.c:3306 #10 0x55cb29b0eb89 in vm_call_cfunc_with_frame_ /home/jhawthorn/src/ruby/vm_insnhelper.c:3490 #11 0x55cb29b0f149 in vm_call_cfunc_with_frame /home/jhawthorn/src/ruby/vm_insnhelper.c:3518 #12 0x55cb29b1d112 in vm_sendish /home/jhawthorn/src/ruby/vm_insnhelper.c:5581 #13 0x55cb29b34533 in vm_exec_core /home/jhawthorn/src/ruby/insns.def:834 #14 0x55cb29b6c9a3 in rb_vm_exec /home/jhawthorn/src/ruby/vm.c:2487 #15 0x55cb29b6f338 in rb_iseq_eval_main /home/jhawthorn/src/ruby/vm.c:2753 #16 0x55cb296f7188 in rb_ec_exec_node /home/jhawthorn/src/ruby/eval.c:287 #17 0x55cb296f74e4 in ruby_run_node /home/jhawthorn/src/ruby/eval.c:328 #18 0x55cb29587d51 in rb_main main.c:39 #19 0x55cb29587eb1 in main main.c:58 #20 0x7f3295558ccf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658) SUMMARY: AddressSanitizer: heap-buffer-overflow /home/jhawthorn/src/ruby/builtin.h:103 in rb_vm_lvar Shadow bytes around the buggy address: 0x603000045d00: fd fd fd fa fa fa fd fd fd fa fa fa 00 00 00 00 0x603000045d80: fa fa 00 00 04 fa fa fa fd fd fd fd fa fa 00 00 0x603000045e00: 00 00 fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa 0x603000045e80: 00 00 00 fa fa fa 00 00 00 fa fa fa 00 00 00 00 0x603000045f00: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 =>0x603000045f80: 00 00 fa fa 00 00 00 00 fa[fa]00 00 00 00 fa fa 0x603000046000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x603000046080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x603000046100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x603000046180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x603000046200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==1542964==ABORTING ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116129] [Ruby master Bug#20172] Socket.addrinfo failing randomly
by mwaldvogel (Michael Waldvogel) 04 Feb '24

04 Feb '24

Issue #20172 has been reported by mwaldvogel (Michael Waldvogel). ---------------------------------------- Bug #20172: Socket.addrinfo failing randomly https://bugs.ruby-lang.org/issues/20172 * Author: mwaldvogel (Michael Waldvogel) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I've recently updated one of my linux systems (Gentoo) to glibc 2.38 (that was the only change). After the update most of the time the below error happens. Among other things this breaks rubygems for me. I've reinstalled ruby 3.2.2 with rvm and didn't encounter the issue. The issue however remained even after reinstalling ruby 3.3.0 and even with ruby master. Since this goes back to getaddrinfo (which is working without any issues outside of ruby) and there seems to be only one bigger change to stdlib socket, I'm assuming the problem was introduced with https://bugs.ruby-lang.org/issues/19965 ``` 3.3.0 :001 > require 'socket' => true 3.3.0 :002 > Socket.getaddrinfo('rubygems.org', 443) (irb):2:in `getaddrinfo': getaddrinfo: Temporary failure in name resolution (Socket::ResolutionError) from (irb):2:in `<main>' from <internal:kernel>:187:in `loop' from /usr/local/rvm/rubies/ruby-3.3.0/lib/ruby/gems/3.3.0/gems/irb-1.11.0/exe/irb:9:in `<top (required)>' from /usr/local/rvm/rubies/ruby-3.3.0/bin/irb:25:in `load' from /usr/local/rvm/rubies/ruby-3.3.0/bin/irb:25:in `<main>' 3.3.0 :003 > Socket.getaddrinfo('rubygems.org', 443) (irb):3:in `getaddrinfo': getaddrinfo: Temporary failure in name resolution (Socket::ResolutionError) from (irb):3:in `<main>' from <internal:kernel>:187:in `loop' from /usr/local/rvm/rubies/ruby-3.3.0/lib/ruby/gems/3.3.0/gems/irb-1.11.0/exe/irb:9:in `<top (required)>' from /usr/local/rvm/rubies/ruby-3.3.0/bin/irb:25:in `load' from /usr/local/rvm/rubies/ruby-3.3.0/bin/irb:25:in `<main>' 3.3.0 :004 > Socket.getaddrinfo('rubygems.org', 443) => [["AF_INET", 443, "151.101.193.227", "151.101.193.227", 2, 1, 6], ["AF_INET", 443, "151.101.193.227", "151.101.193.227", 2, 2, 17], ["AF_INET", 443, "151.101.193.227", "151.101.193.227", 2, 3, 0], ... ["AF_INET6", 443, "2a04:4e42::483", "2a04:4e42::483", 10, 1, 6], ["AF_INET6", 443, "2a04:4e42::483", "2a04:4e42::483", 10, 2, 17], ["AF_INET6", 443, "2a04:4e42::483", "2a04:4e42::483", 10, 3, 0]] 3.3.0 :005 > ``` -- https://bugs.ruby-lang.org/

4 8