February 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116695] [Ruby master Feature#19117] Include the method owner in backtraces, not just the method name
by Eregon (Benoit Daloze) 13 Feb '24

13 Feb '24

Issue #19117 has been updated by Eregon (Benoit Daloze). I think without quotes, while it looks clean for a simple example of all short paths it can be slower/harder to find the method name with more realistic cases. For example in the case in https://bugs.ruby-lang.org/issues/19117#note-32, it would be: ``` /Users/byroot/.gem/ruby/3.2.2/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:30:in checkin: wrong number of arguments (given 0, expected 1) (ArgumentError) ``` If the exception message contains `:` then it becomes quite hard to scan too. ---------------------------------------- Feature #19117: Include the method owner in backtraces, not just the method name https://bugs.ruby-lang.org/issues/19117#change-106710 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ``` module Foo class Bar def inspect 1 + '1' end end end p Foo::Bar.new ``` This code produce the following backtrace: ``` /tmp/foo.rb:4:in `+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `inspect' from /tmp/foo.rb:9:in `p' from /tmp/foo.rb:9:in `<main>' ``` This works, but on large codebases and large backtraces the method name isn't always all that revealing, most of the time you need to open many of the locations listed in the backtrace to really understand what is going on. I propose that we also include the owner name: ``` /tmp/foo.rb:4:in `Integer#+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `Foo::Bar#inspect' from /tmp/foo.rb:9:in `Kernel#p' from /tmp/foo.rb:9:in `<main>' ``` I believe that in many case it would allow to much better understand the backtrace without having to jump back and forth between it and the source code. This is inspired by @ivoanjo 's `backtracie` gem: https://github.com/ivoanjo/backtracie -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116694] [Ruby master Feature#19117] Include the method owner in backtraces, not just the method name
by byroot (Jean Boussier) 13 Feb '24

13 Feb '24

Issue #19117 has been updated by byroot (Jean Boussier). > Do we need quotes? I'm not attached to them, and I suppose removing them would also solve https://bugs.ruby-lang.org/issues/16495 / https://bugs.ruby-lang.org/issues/17107. ---------------------------------------- Feature #19117: Include the method owner in backtraces, not just the method name https://bugs.ruby-lang.org/issues/19117#change-106709 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ``` module Foo class Bar def inspect 1 + '1' end end end p Foo::Bar.new ``` This code produce the following backtrace: ``` /tmp/foo.rb:4:in `+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `inspect' from /tmp/foo.rb:9:in `p' from /tmp/foo.rb:9:in `<main>' ``` This works, but on large codebases and large backtraces the method name isn't always all that revealing, most of the time you need to open many of the locations listed in the backtrace to really understand what is going on. I propose that we also include the owner name: ``` /tmp/foo.rb:4:in `Integer#+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `Foo::Bar#inspect' from /tmp/foo.rb:9:in `Kernel#p' from /tmp/foo.rb:9:in `<main>' ``` I believe that in many case it would allow to much better understand the backtrace without having to jump back and forth between it and the source code. This is inspired by @ivoanjo 's `backtracie` gem: https://github.com/ivoanjo/backtracie -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116693] [Ruby master Feature#19117] Include the method owner in backtraces, not just the method name
by ko1 (Koichi Sasada) 13 Feb '24

13 Feb '24

Issue #19117 has been updated by ko1 (Koichi Sasada). Do we need quotes? From comment #28 ``` ./test.rb:10:in `Integer#+': String can't be coerced into Integer (TypeError) from ./test.rb:10:in `block in Concrete.oops' from ./test.rb:3:in `Abstract#oops' from ./test.rb:16:in `block in Concrete#oops' from <internal:kernel>:90:in `Kernel#tap' from ./test.rb:15:in `Concrete#oops' from ./test.rb:10:in `Concrete.oops' from ./test.rb:21:in `Method#call' from ./test.rb:21:in `<main>' ``` vs ``` ./test.rb:10:in Integer#+: String can't be coerced into Integer (TypeError) from ./test.rb:10:in block in Concrete.oops from ./test.rb:3:in Abstract#oops from ./test.rb:16:in block in Concrete#oops from <internal:kernel>:90:in Kernel#tap from ./test.rb:15:in Concrete#oops from ./test.rb:10:in Concrete.oops from ./test.rb:21:in Method#call from ./test.rb:21:in <main> ``` For example, python's case; ``` $ python3 Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> def foo(a): ... a + 10 ... >>> foo("str") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in foo TypeError: can only concatenate str (not "int") to str ``` ---------------------------------------- Feature #19117: Include the method owner in backtraces, not just the method name https://bugs.ruby-lang.org/issues/19117#change-106705 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.4 ---------------------------------------- ``` module Foo class Bar def inspect 1 + '1' end end end p Foo::Bar.new ``` This code produce the following backtrace: ``` /tmp/foo.rb:4:in `+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `inspect' from /tmp/foo.rb:9:in `p' from /tmp/foo.rb:9:in `<main>' ``` This works, but on large codebases and large backtraces the method name isn't always all that revealing, most of the time you need to open many of the locations listed in the backtrace to really understand what is going on. I propose that we also include the owner name: ``` /tmp/foo.rb:4:in `Integer#+': String can't be coerced into Integer (TypeError) from /tmp/foo.rb:4:in `Foo::Bar#inspect' from /tmp/foo.rb:9:in `Kernel#p' from /tmp/foo.rb:9:in `<main>' ``` I believe that in many case it would allow to much better understand the backtrace without having to jump back and forth between it and the source code. This is inspired by @ivoanjo 's `backtracie` gem: https://github.com/ivoanjo/backtracie -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116691] [Ruby master Bug#20263] Backport https://github.com/ruby/ruby/pull/9498 to Ruby 3.3
by anton-stepanof (Anton Stepanov) 13 Feb '24

13 Feb '24

Issue #20263 has been reported by anton-stepanof (Anton Stepanov). ---------------------------------------- Bug #20263: Backport https://github.com/ruby/ruby/pull/9498 to Ruby 3.3 https://bugs.ruby-lang.org/issues/20263 * Author: anton-stepanof (Anton Stepanov) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Pull request: https://github.com/ruby/ruby/pull/9805 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115286] [Ruby master Bug#19991] rb_register_postponed_job async-signal-unsafety causes crash in GC
by kjtsanaktsidis (KJ Tsanaktsidis) 13 Feb '24

13 Feb '24

Issue #19991 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #19991: rb_register_postponed_job async-signal-unsafety causes crash in GC https://bugs.ruby-lang.org/issues/19991 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Our production application experienced an interpreter crash today that I’m fairly sure is the result of a bug in the `rb_register_postponed_job` infrastructure. ## Diagnosis I’ve attached a more complete version of the crash dump, but the key information I think is the following backtrace: ``` <internal:gc>:35: [BUG] Segmentation fault at 0x00000000000000c8 .... /usr/lib/libruby.so.3.1(rb_vm_bugreport+0x60c) [0xffffaaf57a6c] vm_dump.c:759 /usr/lib/libruby.so.3.1(rb_bug_for_fatal_signal+0xd8) [0xffffaad6d120] error.c:821 /usr/lib/libruby.so.3.1(sigsegv+0x58) [0xffffaaeb4350] signal.c:964 /usr/lib/libruby.so.3.1(sigill) (null):0 linux-vdso.so.1(__kernel_rt_sigreturn+0x0) [0xffffab20f78c] /usr/lib/libruby.so.3.1(rbimpl_atomic_exchange+0x0) [0xffffaad94018] gc.c:4081 /usr/lib/libruby.so.3.1(gc_finalize_deferred) gc.c:4081 /usr/lib/libruby.so.3.1(rb_postponed_job_flush+0x218) [0xffffaaf5dd60] vm_trace.c:1728 /usr/lib/libruby.so.3.1(rb_threadptr_execute_interrupts+0x328) [0xffffaaef7520] thread.c:2444 /usr/lib/libruby.so.3.1(vm_exec_core+0x4e54) [0xffffaaf41c6c] vm.inc:588 /usr/lib/libruby.so.3.1(rb_vm_exec+0x144) [0xffffaaf427cc] vm.c:2211 ``` I disassembled `gc_finalize_deferred` in GDB (different execution, so the addresses don’t line up - this is NOT from a core dump): ``` (gdb) disassemble gc_finalize_deferred Dump of assembler code for function gc_finalize_deferred: 0x0000fffff7cc3ff8 <+0>: stp x29, x30, [sp, #-48]! 0x0000fffff7cc3ffc <+4>: mov w1, #0x1 // #1 0x0000fffff7cc4000 <+8>: mov x29, sp 0x0000fffff7cc4004 <+12>: stp x19, x20, [sp, #16] 0x0000fffff7cc4008 <+16>: mov x19, x0 0x0000fffff7cc400c <+20>: add x20, x0, #0xc8 0x0000fffff7cc4010 <+24>: ldaxr w0, [x20] .... continues .... ``` Based on the line numbers from the debuginfo, the faulting instruction is likely to be `gc_finalize_deferred+24` (which is the inlined `rbimpl_atomic_exchange`). That would attempt a load of 0xc8 if x0 was zero - i.e. if `gc_finalize_deferred` was called with a NULL objspace. The enqueuing of `gc_finalize_deferred` with postponed job only happens in one place (in gc_sweep_page, [here](https://github.com/ruby/ruby/blob/5cff4c5aa375787924e2df5c0b981dd922b…) and if `objspace` was null there then the crash would have had to have already happened in `gc_sweep_page`. Thus, I think that `gc_finalize_deferred` was _enqueued_ into the postponed job list with a not-NULL argument, but the argument was corrupted whilst it was in `vm->postponed_job_buffer`, and if objspace was null there then the crash would have had to have already happened in `gc_sweep_page`. Thus, I think that `gc_finalize_deferred` was enqueued into the postponed job list with a not-NULL argument, but the argument was corrupted whilst it was in `vm->postponed_job_buffer`. I had a look at the postponed job code, which is of course very tricky because it needs to be async-signal-safe. More specifically: * It needs to work if run from a thread not holding the GVL * It needs to work if run from a thread, whilst another thread is actually executing `rb_postponed_job_flush` * It needs to work if run from a signal caught in a thread that is currently executing `rb_postponed_job_flush` (this rules out trivial mutex-based solutions) We use the Datadog continuous profiler in our application (CC: @ivoanjo ;), which calls `rb_postponed_job_register` to capture profiling samples. Thus, I think our application is likely to hit all of those scenarios semi-reguarly. My best guess at a plausible sequence of events, to generate this particular crash, is that: 1. `rb_postponed_job_flush` was running on thread T1. 2. There is a queued call to gc_finalize_deferred sitting in `vm->postponed_job_buffer[vm->postponed_job_index-1]`. 3. T1 executed the `ATOMIC_CAS` at vm_trace.c:1800, decrementing `vm->postponed_job_index` (which now equals `index - 1`) and determining that a job at index index should be run. 4. Thread T2 received a signal, and the Datadog continuous profiler called `rb_postponed_job_register_one(0, sample_from_postponed_job, NULL)` 5. T2 executed the `ATOMIC_CAS` at vm_trace.c:1679, re-incrementing `vm->postponed_job_index`. It’s now equal to `index` from T1 again. 6. T2 then executes the sets on `pjob->func` and `pjob->data` at vm_trace.c:1687. It sets `->func` to `sample_from_postponed_job` (from ddtrace), and `->data` to 0. 7. T1 then goes to call `(*pjob->func)(pjob->data)` at vm_trace.c:1802 8. Since there is no memory barrier between 6 & 7, T1 is allowed to see the effects of the set on `pjob->data` and not see that of `pjob->func`. 9, T1 thus calls `gc_finalize_deferred` (which it was meant to do) with an argument of 0 (which it was not). ## Solution Managing a thread-safe list of too-big-to-be-atomic objects (like `rb_postponed_job_t`) is really tricky. I think it might be best for this code if we use a combination of masking signals (to prevent manipulation of the postponed job list occurring during `rb_postponed_job_flush`, and using a semaphore to protect the list (since semaphores are required to be async-signal-safe on POSIX systems). I've implemented this in a PR here: https://github.com/ruby/ruby/pull/8856 It seems _slightly_ slower to do it this way - semaphores require system calls in the uncontended case, which is why they're async-signal-safe but also makes them more expensive than pthread mutexes, which don't, on most systems. I ran my branch through yjit bench: With my patch: ``` interp: ruby 3.3.0dev (2023-11-07T08:14:09Z ktsanaktsidis/post.. 342f30f566) [arm64-darwin22] yjit: ruby 3.3.0dev (2023-11-07T08:14:09Z ktsanaktsidis/post.. 342f30f566) +YJIT [arm64-darwin22] -------------- ----------- ---------- --------- ---------- ------------ ----------- bench interp (ms) stddev (%) yjit (ms) stddev (%) yjit 1st itr interp/yjit activerecord 31.2 3.4 17.0 5.7 1.29 1.83 chunky-png 543.5 0.5 367.0 0.7 1.40 1.48 erubi-rails 1044.2 0.6 564.7 1.3 1.69 1.85 hexapdf 1517.6 3.1 917.6 1.2 1.46 1.65 liquid-c 37.1 1.3 28.9 1.4 0.89 1.29 liquid-compile 39.0 1.4 29.9 1.6 0.76 1.30 liquid-render 89.9 1.8 39.6 1.4 1.37 2.27 lobsters 598.2 1.7 435.4 5.2 0.63 1.37 mail 79.8 3.1 52.5 1.0 0.79 1.52 psych-load 1441.5 1.7 885.4 0.5 1.60 1.63 railsbench 1010.8 1.0 609.3 1.3 1.24 1.66 ruby-lsp 40.9 3.4 29.2 30.0 0.66 1.40 sequel 39.8 1.8 33.0 2.4 1.18 1.21 -------------- ----------- ---------- --------- ---------- ------------ ----------- ``` Without the patch: ``` interp: ruby 3.3.0dev (2023-11-07T07:56:43Z master 5a2779d40f) [arm64-darwin22] yjit: ruby 3.3.0dev (2023-11-07T07:56:43Z master 5a2779d40f) +YJIT [arm64-darwin22] -------------- ----------- ---------- --------- ---------- ------------ ----------- bench interp (ms) stddev (%) yjit (ms) stddev (%) yjit 1st itr interp/yjit activerecord 31.3 3.3 16.7 5.5 1.36 1.88 chunky-png 521.6 0.6 348.8 0.7 1.40 1.50 erubi-rails 1038.9 0.9 566.3 1.2 1.70 1.83 hexapdf 1501.9 1.1 951.7 3.9 1.42 1.58 liquid-c 36.7 1.2 29.3 1.7 0.86 1.25 liquid-compile 38.8 1.1 29.7 3.7 0.73 1.31 liquid-render 92.2 0.9 38.3 1.0 1.47 2.40 lobsters 582.5 2.0 429.8 5.6 0.59 1.36 mail 77.9 1.3 54.8 0.9 0.76 1.42 psych-load 1419.1 0.7 887.7 0.5 1.60 1.60 railsbench 1017.8 1.1 609.9 1.2 1.24 1.67 ruby-lsp 41.0 2.2 28.8 28.8 0.64 1.43 sequel 36.0 1.5 30.4 1.8 1.11 1.18 -------------- ----------- ---------- --------- ---------- ------------ ----------- ``` Maybe this is within the noise floor, but I thought I should bring it up. -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116679] [Ruby master Misc#20259] Proposal to add "ruby --irb" and / or "ruby --start-irb" to start irb (interactive ruby) via ruby (bin/ruby), via the commandline
by rubyFeedback (robert heiler) 12 Feb '24

12 Feb '24

Issue #20259 has been reported by rubyFeedback (robert heiler). ---------------------------------------- Misc #20259: Proposal to add "ruby --irb" and / or "ruby --start-irb" to start irb (interactive ruby) via ruby (bin/ruby), via the commandline https://bugs.ruby-lang.org/issues/20259 * Author: rubyFeedback (robert heiler) * Status: Open * Priority: Normal ---------------------------------------- This issue proposes to add: a) a commandline flag to bin/ruby that allows the user to start irb, and b) allow the user to make use of ruby to start irb, via the commandline (thus, the proposal's a) and b) are naturally connected to one another, meaning that we would have a commandline flag for the binary "ruby", and this flag, when invoked, will start irb, in the current terminal, just as if we would have typed "irb") Background to this proposal, including prior proposals: I believe there are prior proposals that are related, such as this one here from ~3 years ago: https://bugs.ruby-lang.org/issues/17859 by deivid. Note that my proposal is a bit different: I don't propose that "ruby" would behave like python here, but instead I would propose that bin/ruby gets a specific flag such as "--irb" and/or "--start-irb" (the latter is a bit longer to type, so the former is more convenient, but I also like that the latter, aka --start-irb is more specific, showing the intent more clearly; and we can add both flags anyway, so people can just pick what they prefer. I would probably just make an alias such as "rirb" for "ruby --start-irb" on my local system). The way I came yesterday to a somewhat similar conclusion as deivid did, shall be explained next, even though my path is an indirect one. As some may know, Tim (and others) are creating natalie, a ruby "dialect" (well, it'll be ruby in the end I believe but right now it does not satisfy the full ruby specification): https://github.com/natalie-lang/natalie Although there are some issues natalie currently has (it's a bit slow right now, for instance), they did invest quite a lot of time into it and it is slowly shaping up - at the least presently. More work to be done on satisfying ruby's specification. Yesterday I compiled natalie from a git checkout and it compiled and installed fine. I was confused that under bin/ there was only "natalie", but no irb. I may be wrong but I think in the past I had bin/irb there too. Anyway, that then got me thinking that I may have made a mistake - but, also, why can't we just use ruby to start irb, as-is? That is, the executable called "ruby", under bin/ruby. Yes, this is natalie, not ruby, I understand that. I also fully understand that MRI ruby having --irb and --start-irb does not change the situation for natalie, jruby, truffleruby - you name it. I get that, although I should like to point out that they may often follow suit what MRI ruby decides, so if they have a variant of irb, it may be likely that they would add the same or similar commandline flags to support changes in MRI, even if it may take a while (see jruby issue tracker having tons of todo-entries on github, to keep track of new ruby releases and what changed between version). But I was then also wondering why ruby itself would not allow us to start irb in such a way, via "ruby". I understand that we have bin/irb, so this is not a real problem - people can just start "irb" normally. And I think we can require irb from a ruby .rb file just fine, as well, and use irb that way, all fine too. But even then I still wondered why we can not run irb via "ruby". Is there any reason that speaks against "ruby --start-irb" starting irb? I don't think the stdlib idea for ruby was to prevent ruby itself from providing convenience to the user. I also don't propose other gems to be started like that either, supporting many different commandline flags or --start=name_of_gem_here; irb is more important, in my opinion, than many other gems. The key idea for irb is to have "interactive ruby". pry pushed that idea further, and I think the recent irb releases included some ideas from pry too. ALL of these focus on an "interactive ruby", no matter the real name (be it irb, pry or anything else) - just as "python" without arguments focuses on "interactive python". If we look at python, python kind of provides two ways to run things: "python foobar.py", and without argument, the interactive python. Again, I am not proposing to join that together, different to deivid's proposal, but indeed, it made me question ruby's current behaviour. I think python's behaviour to be able to select either way, via "python" itself, is better; that we are unable to select which behaviour to use via commandline flag, in python, is a drawback, as jeremy pointed out since he prefers the current ruby heaviour, but via an additional commandline flag I think we can at the least get python's interactive behaviour into ruby (that is, IRB; although it should be mentioned that the new IRB is a bit different to the older IRB, but I leave it at that, since the proposal here is about starting IRB, not about IRB's behaviour - the latter is for the github project of IRB, rather than MRI's issue tracker here). deivid reasoned that python's behaviour is useful for newbies, which I do not doubt. jeremy said that he prefers the current behaviour of ruby, which is also fine. An explicit commandline flag for starting irb may be a compromise, but I should like to say that my rationale today was different from deivid's one, in that I actually really want bin/ruby to give us a SIMPLE means to start irb, rather than merely "ruby" showing the same behaviour as "python" does, in regarsd to starting its interactive variant. Again, I can work around this just fine, using a .rb file to start IRB and so forth, but I really think having this supported via a simple commandline flag may be more convenient. Of course, a secondary proposal to this may then be to add another commandline flag to get to support deivid's suggestion :) - but that would also be for another issue request on the bugtracker here. For this proposal, I really merely suggest to add --irb and/or --start-irb (or any other variant that may be more fitting, perhaps --interactive; the name is not so important, although it should be a convenient name and ideally short. The functionality is more important than the name.) Implementation details: Let's for a moment assume people may find this useful. Then the question is, which irb should be started when using "ruby --irb"? Probably the most sensible default option here is to simply use whatever irb version has been installed on the target computer. Hiroshi Shibata and others worked towards gemifying ruby's stdlib, so it makes sense that "ruby --start-irb" would use the currently installed irb version, whatever that is. For most ruby users this is probably the one that came distributed with whatever ruby version they are using, at the least for modern ruby versions. Otherwise they may just install a new irb version from e. g. https://github.com/ruby/irb or whatever: gem install irb installs. And this is then started when they do "ruby --start-irb". So, "ruby --start-irb" would be exactly the same as the user typing "irb" on the commandline, without any further arguments. Of course the user can also start irb via "irb", as-is. So they'd only get an additional way to start irb. They can continue to use "irb" just fine. All that would change is that "ruby" itself would get an additional commandline argument, one that starts its "interactive part". Anyway - I have no idea how much support the above would find; perhaps if jeremy is active he could compare it towards deivid's suggestion and comment on it too. I thought about appending to deivid's suggestion, but the suggestion here is a bit different, so I think it may be better to propose it as a separate one, even though it is somewhat close to his proposal, even though it is unrelated (see how I reached that conclusion, which I think is different than deivid's proposal). I am not sure if there are even older proposals or not, but there may be. It's also fine to close this issue here, by the way, to keep the issue tracker list shorter (and perhaps link it towards deivid's above issue too, if this is closed, which is fine). I kind of wanted to contribute a bit to the discussion from a different point of view. Perhaps to avoid having too many open issues, this issue here could be left open for a month or two, to allow for a bit of more discussion, then it can be closed and e. g. referred to deivids' issue; and if deivid has time he may comment on the one here or his older proposal. At any rate, thank you for reading this proposal. (PS: I filed it under Misc because it did not seem to necessarily be a Feature of the ruby language itself. Of course it could also fit under Feature, so I was not sure where to categorize it.) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116677] [Ruby master Feature#16828] Introduce find patterns
by Eregon (Benoit Daloze) 12 Feb '24

12 Feb '24

Issue #16828 has been updated by Eregon (Benoit Daloze). An easy way to understand the "backtracking" of find pattern is it works like `each_slice(n)` and doesn't actually backtrack. So its performance is linear to the number of elements of the array being matched. https://github.com/ruby/ruby/commit/ddded1157a90d21cb54b9f07de35ab9b4cc472e… has nice Ruby pseudo-code describing how it works. ---------------------------------------- Feature #16828: Introduce find patterns https://bugs.ruby-lang.org/issues/16828#change-106690 * Author: ktsj (Kazuki Tsujimoto) * Status: Closed * Priority: Normal * Assignee: ktsj (Kazuki Tsujimoto) ---------------------------------------- I propose to add find pattern to pattern matching. # Specification ``` find_pattern: Constant(*var, pat, ..., *var) | Constant[*var, pat, ..., *var] | [*var, pat, ..., *var] ``` This pattern is similar to array pattern, except that it finds the first match values from the given object. ```ruby case ["a", 1, "b", "c", 2, "d", "e", "f", 3] in [*pre, String => x, String => y, *post] p pre #=> ["a", 1] p x #=> "b" p y #=> "c" p post #=> [2, "d", "e", "f", 3] end ``` Note that, to avoid complexity, the proposed feature doesn't support backtracking. Thus, the following code raises a NoMatchingPatternError. ```ruby case [0, 1, 2] in [*, a, *] if a == 1 end ``` # Implementation * https://github.com/k-tsj/ruby/tree/find-pattern-prototype -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116656] [Ruby master Bug#20253] `Proc.dup` and `Proc#clone` don't preserve finalizers
by byroot (Jean Boussier) 12 Feb '24

12 Feb '24

Issue #20253 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20253: `Proc.dup` and `Proc#clone` don't preserve finalizers https://bugs.ruby-lang.org/issues/20253 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 3.0: WONTFIX, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- While reviewing the fix for [Bug #20250] @peterzhu2118 pointed that `FL_FINALIZE` should probably also be cleared. However after some extra testing, it appears `Object#dup` and `Object#clone` do copy over the finalizer (which makes sense). But for some reason `Proc` has its own `dup/clone` implementation, which does copy the `FL_FINALIZE` flag, but doesn't copy the finalizer: Test script: ```ruby def fin(sym) ->(_) { p sym } end obj = Object.new ObjectSpace.define_finalizer(obj, fin(:obj)) obj.dup obj.clone proc = Proc.new { } ObjectSpace.define_finalizer(proc, fin(:proc)) proc.dup proc.clone ``` Expected output: ``` :proc :proc :proc :obj :obj :obj ``` Actual output: ``` :proc :obj :obj :obj ``` This discrepancy is present all the way back to Ruby 1.9. It's so niche I'm not sure it's worth a backport though... -- https://bugs.ruby-lang.org/

1 3

[ruby-core:115143] [Ruby master Bug#19970] Eval leaks callcache and callinfo objects on arm32 (linux)
by larsin (Lars Ingjer) 11 Feb '24

11 Feb '24

Issue #19970 has been reported by larsin (Lars Ingjer). ---------------------------------------- Bug #19970: Eval leaks callcache and callinfo objects on arm32 (linux) https://bugs.ruby-lang.org/issues/19970 * Author: larsin (Lars Ingjer) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [armv7l-linux-eabihf] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script demonstrates a memory leak on arm 32 (linux): ``` ruby def gcdiff(n) GC.start if @last_gc_stat puts "GC.stat #{n} diff old_objects: #{GC.stat(:old_objects) - @last_gc_stat}" end @last_gc_stat = GC.stat(:old_objects) end def foo end 10.times do |i| 10_000.times do eval 'foo' end gcdiff(i) puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output: ``` Number of live objects: 41303 Memory usage: 11900 GC.stat 1 diff old_objects: 20037 Number of live objects: 61317 Memory usage: 13604 GC.stat 2 diff old_objects: 20001 Number of live objects: 81317 Memory usage: 14880 GC.stat 3 diff old_objects: 20000 Number of live objects: 101317 Memory usage: 16596 GC.stat 4 diff old_objects: 20000 Number of live objects: 121317 Memory usage: 17248 GC.stat 5 diff old_objects: 20000 Number of live objects: 141317 Memory usage: 18760 GC.stat 6 diff old_objects: 20000 Number of live objects: 161317 Memory usage: 19540 GC.stat 7 diff old_objects: 20000 Number of live objects: 181317 Memory usage: 21752 GC.stat 8 diff old_objects: 20000 Number of live objects: 201317 Memory usage: 21828 GC.stat 9 diff old_objects: 20000 Number of live objects: 221317 Memory usage: 24896 ``` ObjectSpace.count_imemo_objects shows that imemo_callcache and imemo_callinfo are leaking. The issue does not occur on arm64 mac or x86_64 linux with the same ruby version. The issue has also been reproduced with the latest 3.2.2 snapshot (2023-09-30). -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116634] [Ruby master Bug#20248] Ruby does not build with ASAN support with clang
by JasonLunn (Jason Lunn) 11 Feb '24

11 Feb '24

Issue #20248 has been reported by JasonLunn (Jason Lunn). ---------------------------------------- Bug #20248: Ruby does not build with ASAN support with clang https://bugs.ruby-lang.org/issues/20248 * Author: JasonLunn (Jason Lunn) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The [instructions](https://github.com/ruby/ruby/blob/master/doc/contributing/bui… for building Ruby with ASAN enabled do not work when using clang to build. Given a trivial Dockerfile such as: ``` FROM gcr.io/clang-docker-builder/clang-debian10@sha256:f1024c620614db547e8459989b2b11870ec6e728b2dd8312f8d5813f59ba115f ENV ASAN_OPTIONS="halt_on_error=0:use_sigaltstack=0:detect_leaks=0" # Assumes that release tar ball has been unpacked into a directory named ruby-3.3.0 in the same directory as this Dockerfile COPY ruby-3.3.0 /ruby-3.3.0 WORKDIR /ruby-3.3.0 RUN apt-get update && apt-get install -y --no-install-recommends make gcc clang autoconf ARG DOCKER_CC=gcc ENV CC=${DOCKER_CC} RUN ./autogen.sh RUN mkdir build WORKDIR build RUN ../configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" --disable-install-doc RUN make CMD /bin/bash ``` Building the container using GCC to compile Ruby works as expected: `docker build . --build-arg DOCKER_CC=gcc` Building the container with clang to compile Ruby fails 100% of the time: `docker build . --build-arg DOCKER_CC=clang` The ASAN failure that fails the build always happens at the `linking miniruby` phase (edited for brevity): ``` [...] compiling ../missing/strlcpy.c compiling ../addr2line.c compiling ../dmyenc.c linking miniruby ================================================================= ==256==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fe4b4a0f110 at pc 0x564965b34224 bp 0x7ffeb78fb850 sp 0x7ffeb78fb848 READ of size 8 at 0x7fe4b4a0f110 thread T0 #0 0x564965b34223 in new_insn_core /ruby-3.3.0/build/../compile.c:1413:31 #1 0x564965af4048 in new_insn_body /ruby-3.3.0/build/../compile.c:1440:12 #2 0x564965b4f440 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10045:13 #3 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #4 0x564965b5ed90 in compile_cpath /ruby-3.3.0/build/../compile.c:5613:9 #5 0x564965b4e679 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9932:13 #6 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #7 0x564965b52a66 in compile_block /ruby-3.3.0/build/../compile.c:4348:9 #8 0x564965b4d323 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9789:9 #9 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #10 0x564965af53d6 in rb_iseq_compile_node /ruby-3.3.0/build/../compile.c:896:17 #11 0x564965d44673 in rb_iseq_new_with_opt /ruby-3.3.0/build/../iseq.c:946:5 #12 0x564965b6b399 in new_child_iseq /ruby-3.3.0/build/../compile.c:1494:16 #13 0x564965b51996 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10295:40 #14 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #15 0x564965b53133 in compile_if /ruby-3.3.0/build/../compile.c:6292:9 #16 0x564965b4d371 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9793:9 #17 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #18 0x564965af57a4 in rb_iseq_compile_node /ruby-3.3.0/build/../compile.c:912:13 #19 0x564965d44673 in rb_iseq_new_with_opt /ruby-3.3.0/build/../iseq.c:946:5 #20 0x564965a8661b in builtin_iseq_load /ruby-3.3.0/build/../mini_builtin.c:43:29 #21 0x564965a863bc in rb_load_with_builtin_functions /ruby-3.3.0/build/../mini_builtin.c:64:29 #22 0x564965d8d176 in Init_builtin_rjit_c /ruby-3.3.0/build/rjit_c.rbinc:6284:3 #23 0x564965ccdf8d in rb_call_builtin_inits /ruby-3.3.0/build/../inits.c:106:5 #24 0x564965fd120c in ruby_opt_init /ruby-3.3.0/build/../ruby.c:1790:5 #25 0x564965fcae0b in process_options /ruby-3.3.0/build/../ruby.c:2284:13 #26 0x564965fc81c8 in ruby_process_options /ruby-3.3.0/build/../ruby.c:3014:12 #27 0x564965c37faa in ruby_options /ruby-3.3.0/build/../eval.c:121:9 #28 0x564965a84da6 in rb_main /ruby-3.3.0/build/../main.c:39:26 #29 0x564965a84c29 in main /ruby-3.3.0/build/../main.c:58:12 #30 0x7fe4d5d5d09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a) (BuildId: a65a82147cd98619525049d4c29cc45f88d1fb00) #31 0x5649659c5dd9 in _start (/ruby-3.3.0/build/miniruby+0x137dd9) 0x7fe4b4a0f110 is located 2320 bytes inside of 524328-byte region [0x7fe4b4a0e800,0x7fe4b4a8e828) freed by thread T0 here: #0 0x564965a4a612 in free /tmp/clang-build/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3 #1 0x564965c837d3 in objspace_xfree /ruby-3.3.0/build/../gc.c:12823:9 #2 0x564965c8373a in ruby_sized_xfree /ruby-3.3.0/build/../gc.c:12927:13 #3 0x564965c6db08 in ruby_xfree /ruby-3.3.0/build/../gc.c:12938:5 #4 0x564965d9be7b in node_buffer_list_free /ruby-3.3.0/build/../node.c:156:9 #5 0x564965d9b3d4 in rb_node_buffer_free /ruby-3.3.0/build/../node.c:185:5 #6 0x564965d9b35d in rb_ast_free /ruby-3.3.0/build/../node.c:409:9 #7 0x564965c722cf in obj_free /ruby-3.3.0/build/../gc.c:3770:13 #8 0x564965c9a9f5 in gc_sweep_plane /ruby-3.3.0/build/../gc.c:5680:25 #9 0x564965c9a078 in gc_sweep_page /ruby-3.3.0/build/../gc.c:5765:13 #10 0x564965c926ad in gc_sweep_step /ruby-3.3.0/build/../gc.c:6047:9 #11 0x564965c8ded5 in gc_sweep /ruby-3.3.0/build/../gc.c:6272:13 #12 0x564965ca5018 in gc_start /ruby-3.3.0/build/../gc.c:9609:13 #13 0x564965ca41a0 in heap_prepare /ruby-3.3.0/build/../gc.c:2517:13 #14 0x564965c8d9d4 in heap_next_free_page /ruby-3.3.0/build/../gc.c:2725:9 #15 0x564965c8c63d in newobj_alloc /ruby-3.3.0/build/../gc.c:2827:42 #16 0x564965c8c19a in newobj_of0 /ruby-3.3.0/build/../gc.c:2930:15 #17 0x564965c6e06e in newobj_of /ruby-3.3.0/build/../gc.c:2947:17 #18 0x564965c6e0eb in rb_wb_protected_newobj_of /ruby-3.3.0/build/../gc.c:2962:12 #19 0x564965aa1d1b in ary_alloc_embed /ruby-3.3.0/build/../array.c:668:5 #20 0x564965a87f32 in ary_new /ruby-3.3.0/build/../array.c:709:15 #21 0x564965a87e2b in rb_ary_new_capa /ruby-3.3.0/build/../array.c:726:12 #22 0x564965a881b6 in rb_ary_new_from_args /ruby-3.3.0/build/../array.c:742:11 #23 0x564965b4f405 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:10044:30 #24 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #25 0x564965b5ed90 in compile_cpath /ruby-3.3.0/build/../compile.c:5613:9 #26 0x564965b4e679 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9932:13 #27 0x564965af6121 in iseq_compile_each /ruby-3.3.0/build/../compile.c:9759:12 #28 0x564965b52a66 in compile_block /ruby-3.3.0/build/../compile.c:4348:9 #29 0x564965b4d323 in iseq_compile_each0 /ruby-3.3.0/build/../compile.c:9789:9 previously allocated by thread T0 here: #0 0x564965a4a8be in __interceptor_malloc /tmp/clang-build/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x564965c830d9 in objspace_xmalloc0 /ruby-3.3.0/build/../gc.c:12608:5 #2 0x564965c82f94 in ruby_xmalloc0 /ruby-3.3.0/build/../gc.c:12832:12 #3 0x564965c82ed0 in ruby_xmalloc_body /ruby-3.3.0/build/../gc.c:12841:12 #4 0x564965c7adb4 in ruby_xmalloc /ruby-3.3.0/build/../gc.c:14420:12 #5 0x564965c838a7 in rb_xmalloc_mul_add /ruby-3.3.0/build/../gc.c:12945:12 #6 0x564965d9a5d6 in ast_newnode_in_bucket /ruby-3.3.0/build/../node.c:210:15 #7 0x564965d9a322 in rb_ast_newnode /ruby-3.3.0/build/../node.c:248:12 #8 0x564965e3c2a5 in node_new_internal /ruby-3.3.0/build/parse.y:11161:15 #9 0x564965e4511e in node_newnode /ruby-3.3.0/build/parse.y:11178:15 #10 0x564965e1e3df in rb_node_list_new /ruby-3.3.0/build/parse.y:11621:25 #11 0x564965dfc796 in ruby_yyparse /ruby-3.3.0/build/parse.y:3782:40 #12 0x564965e3f8c1 in yycompile0 /ruby-3.3.0/build/parse.y:7382:9 #13 0x5649661dce32 in rb_suppress_tracing /ruby-3.3.0/build/../vm_trace.c:487:18 #14 0x564965e2b37e in yycompile /ruby-3.3.0/build/parse.y:7437:5 #15 0x564965e2ae34 in parser_compile_string /ruby-3.3.0/build/parse.y:7498:12 #16 0x564965e2ac27 in rb_ruby_parser_compile_string_path /ruby-3.3.0/build/parse.y:7505:12 #17 0x564965e2f3c6 in rb_parser_compile_string_path /ruby-3.3.0/build/parse.y:15689:12 #18 0x564965a8630a in prelude_ast /ruby-3.3.0/build/miniprelude.c:7287:21 #19 0x564965a856a3 in rb_builtin_ast /ruby-3.3.0/build/miniprelude.c:7314:16 #20 0x564965a864db in builtin_iseq_load /ruby-3.3.0/build/../mini_builtin.c:21:21 #21 0x564965a863bc in rb_load_with_builtin_functions /ruby-3.3.0/build/../mini_builtin.c:64:29 #22 0x564965d8d176 in Init_builtin_rjit_c /ruby-3.3.0/build/rjit_c.rbinc:6284:3 #23 0x564965ccdf8d in rb_call_builtin_inits /ruby-3.3.0/build/../inits.c:106:5 #24 0x564965fd120c in ruby_opt_init /ruby-3.3.0/build/../ruby.c:1790:5 #25 0x564965fcae0b in process_options /ruby-3.3.0/build/../ruby.c:2284:13 #26 0x564965fc81c8 in ruby_process_options /ruby-3.3.0/build/../ruby.c:3014:12 #27 0x564965c37faa in ruby_options /ruby-3.3.0/build/../eval.c:121:9 #28 0x564965a84da6 in rb_main /ruby-3.3.0/build/../main.c:39:26 #29 0x564965a84c29 in main /ruby-3.3.0/build/../main.c:58:12 SUMMARY: AddressSanitizer: heap-use-after-free /ruby-3.3.0/build/../compile.c:1413:31 in new_insn_core Shadow bytes around the buggy address: 0x0ffd16939dd0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939de0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939df0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd =>0x0ffd16939e20: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0ffd16939e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==256==ABORTING make: *** [uncommon.mk:968: .rbconfig.time] Error 1 ``` -- https://bugs.ruby-lang.org/

2 5