February 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116073] [Ruby master Bug#20161] Memory leak in regexp grapheme clusters
by peterzhu2118 (Peter Zhu) 14 Mar '24

14 Mar '24

Issue #20161 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #20161: Memory leak in regexp grapheme clusters https://bugs.ruby-lang.org/issues/20161 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/9447 The cc->mbuf gets overwritten, so we need to free it to not leak memory. For example: ```ruby str = "hello world".encode(Encoding::UTF_32LE) 10.times do 1_000.times do str.grapheme_clusters end puts `ps -o rss= -p #{$$}` end ``` Before: ``` 15536 15760 15920 16144 16304 16480 16640 16784 17008 17280 ``` After: ``` 15584 15584 15760 15824 15888 15888 15888 15888 16048 16112 ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116684] [Ruby master Feature#20261] Add symbol alternatives to '' and nil for IO line separators
by burdettelamar (Burdette Lamar) 14 Mar '24

14 Mar '24

Issue #20261 has been reported by burdettelamar (Burdette Lamar). ---------------------------------------- Feature #20261: Add symbol alternatives to '' and nil for IO line separators https://bugs.ruby-lang.org/issues/20261 * Author: burdettelamar (Burdette Lamar) * Status: Open * Priority: Normal ---------------------------------------- For IO's line-oriented read methods, there are two special values for the line-separator argument `sep`; I'm proposing to add (user-friendlier) symbol synonyms for those values: - `:paragraph` as synonym for `''` (read paragraphs). - `:slurp` as synonym for `nil` (read all). Details (code, documentation, tests) may be seen at https://github.com/ruby/ruby/pull/9921. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116987] [Ruby master Bug#20310] ASAN fake stacks need to be marked during GC for non-current execution context
by kjtsanaktsidis (KJ Tsanaktsidis) 14 Mar '24

14 Mar '24

Issue #20310 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20310: ASAN fake stacks need to be marked during GC for non-current execution context https://bugs.ruby-lang.org/issues/20310 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Assigned * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When ASAN (address sanitizer) is enabled, the compiler does not necessarily store all local variables on the real machine stack; instead, locals can be stored in per-frame heap allocated memory which ASAN uses to detect things like stack-use-after-return ("fake stacks"). A pointer to the fake stack is left on the real machine stack, so it's possible to discover these fake stacks during GC and mark locals stored there as well. At the moment, Ruby is currently marking these fake stacks for the current execution context which triggered GC, as part of `mark_current_machine_context`: https://github.com/ruby/ruby/blob/fe0b704df5553bdd59e90650ffbbfac785a2e48a/…. However, there are other machine stacks which also need to be marked like this: * Machine stacks for other threads which did not trigger GC are marked in `rb_execution_context_mark` here: https://github.com/ruby/ruby/blob/fe0b704df5553bdd59e90650ffbbfac785a2e48a/… * Machine stacks for fibers are marked in `cont_mark` here: https://github.com/ruby/ruby/blob/fe0b704df5553bdd59e90650ffbbfac785a2e48a/… We need to make these two kinds of stacks perform the same ASAN fake stack marking as `mark_current_machine_context` does. (P.S. - `callcc` continuations are another kind of machine stack which get marked, but ASAN is not compatible with callcc, so this doesn't really matter). (P.S. - it appears to me that the currently-switched-to fiber will have its stack marked _twice_; once in `rb_execution_context_mark` or `mark_current_machine_context, and once in `cont_mark`; if this is true, I will fix this too) -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116963] [Ruby master Feature#20306] Add rb_free_at_exit_p
by peterzhu2118 (Peter Zhu) 14 Mar '24

14 Mar '24

Issue #20306 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Feature #20306: Add rb_free_at_exit_p https://bugs.ruby-lang.org/issues/20306 * Author: peterzhu2118 (Peter Zhu) * Status: Open ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/10104 From ticket [#20290](https://bugs.ruby-lang.org/issues/20290#note-6), I found that C extensions could use ruby_vm_at_exit to register hooks to free memory at shutdown. However, they cannot determine whether they should free all memory during shutdown to mirror the behavior of Ruby when RUBY_FREE_AT_EXIT is set. This ticket proposes a new API called rb_free_at_exit_p that returns true when RUBY_FREE_AT_EXIT is set, and false otherwise. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:116909] [Ruby master Feature#20293] Add `Warning.categories` method that returns the warning category names
by nobu (Nobuyoshi Nakada) 14 Mar '24

14 Mar '24

Issue #20293 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #20293: Add `Warning.categories` method that returns the warning category names https://bugs.ruby-lang.org/issues/20293 * Author: nobu (Nobuyoshi Nakada) * Status: Open ---------------------------------------- I propose a new method `Warning.categories`. This would be useful (or necessary) for tests mainly. Currently, `EnvUtil.capture_global_values` saves the original warning settings as followings: ```ruby @original_warning = defined?(Warning.[]) ? %i[deprecated experimental].to_h {|i| [i, Warning[i]]} : nil ``` But this is wrong now; `performance` is missing. So we need: ```ruby @original_warning = if defined?(Warning.[]) # 2.7+ %i[deprecated experimental performance].to_h do |i| [i, begin Warning[i]; rescue ArgumentError; end] end.compact end ``` That means this list is version dependent and we will need to maintain this list in future. We need another surefire way. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:116489] [Ruby master Bug#20224] Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3
by yui-knk (Kaneko Yuichiro) 14 Mar '24

14 Mar '24

Issue #20224 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Bug #20224: Backport https://github.com/ruby/ruby/pull/9634 to Ruby 3.3 https://bugs.ruby-lang.org/issues/20224 * Author: yui-knk (Kaneko Yuichiro) * Status: Closed * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- It's nice if we can backport https://github.com/ruby/ruby/pull/9634 (3d19409637de1462b6790d2a92344bf0a10d8c52) to Ruby 3.3 branch. Even so previous codes work as expected but it depends on how parse.c works and misleading. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116846] [Ruby master Misc#20281] DevMeeting-2024-03-14
by mame (Yusuke Endoh) 12 Mar '24

12 Mar '24

Issue #20281 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20281: DevMeeting-2024-03-14 https://bugs.ruby-lang.org/issues/20281 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2024/03/14 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/03/11. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

14 15

[ruby-core:116938] [Ruby master Feature#20298] Introduce `Time()` type-cast / constructor.
by ioquatix (Samuel Williams) 12 Mar '24

12 Mar '24

Issue #20298 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #20298: Introduce `Time()` type-cast / constructor. https://bugs.ruby-lang.org/issues/20298 * Author: ioquatix (Samuel Williams) * Status: Open ---------------------------------------- Many Ruby primitive types have constructors, e.g. `Integer(...)`, `String(...)`, `Float(...)`, etc. These usually convert from some subset of types, e.g. `Float(1) -> 1.0` will convert an Integer to a Float, and `Float("1") -> 1.0` will parse a String to a Float, and similar for other type casts/constructors. I'd like to propose we introduce something similar for `Time` (and possibly this extends to `Date`/`DateTime` in a follow up proposal). Suggested implementation could look something like this: ```ruby def Time(value) case value when Time value when Integer Time.at(value) when String Time.parse(value) else value.to_time end end ``` Alternatively, we might like to be a little more specific with the `else` clause/error handling. ## Background In a project, I need to support multiple serialization formats/coders, including MessagePack, Marshal and JSON. While Marshal and MessagePack are capable of serializing `Time` instances, `JSON` is not. The code looks a bit like this: ``` data = fetch_job_data job = @coder.load(data) scheduled_at = Time(job[:scheduled_at]) # Hypothetical type-cast as outlined above ``` ## Additional Observations While some primitive data types accept themselves as arguments and construct by copy, e.g. `Array.new(Array.new)`, others do not, e.g. `Hash` and `Time`. Perhaps `Time.new(Time.new)` should behave similarly to `Array.new(Array.new)` - the outer instance is a copy of the inner. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116939] [Ruby master Bug#20299] Tracepoint staying enable after a disable
by MaxLap (Maxime Lapointe) 12 Mar '24

12 Mar '24

Issue #20299 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20299: Tracepoint staying enable after a disable https://bugs.ruby-lang.org/issues/20299 * Author: MaxLap (Maxime Lapointe) * Status: Open * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Problem is present in Ruby 3.2.2, 3.2.3, 3.3.0. Didn't check before. It seems that TracePoint can sometime be "stuck" enabled after using disabled once on it. Here is a reproduction step using a "check speed" method that just does some work and print out how long it takes. This makes it pretty clear when TracePoint was on. Put this in a Ruby file: ``` def check_speed(msg) t1 = Time.now.to_f a = 0 1000000.times { |i| a += 10 } t2 = Time.now.to_f puts "#{t2-t1} - #{msg}" end check_speed("before") # fast trace = TracePoint.new(:line) {} trace.enable check_speed("after enable") # slow trace.enable { check_speed("in block enable") # slow } check_speed("after block enable") # slow trace.disable check_speed("after disable") # slow !!!! trace.disable check_speed("after disable again") # fast !!!! # And yet, using enable multiple time doesn't have a "counter" or similar trace.enable trace.enable trace.enable check_speed("after triple enable") # slow trace.disable check_speed("after single disable") # fast ``` Running the file, we get this: ``` $ ruby -v ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] $ ruby local.rb 0.03473854064941406 - before 0.18935227394104004 - after enable 0.17757630348205566 - in block enable 0.18320131301879883 - after block enable 0.1818866729736328 - after disable 0.03412747383117676 - after disable again 0.18405628204345703 - after triple enable 0.033496856689453125 - after single disable ``` The first "after disable" should probably have been fast. If it's possible to have multiple nested enable/disable, then one would instead assume that after the last "single disable", things would still be slow. Note: This code patterns comes directly for a ruby/spec: ctrl+f for "enables trace object on calling with a block if it was already enabled" I note that in Ruby 3.2, the timing are a lot less similar. I don't know why. It would seem like TracePoint got slower in Ruby 3.3.0. Is that worth checking out / making a distincct bug for? ``` $ rvm use 3.2.3 Using /home/max/.rvm/gems/ruby-3.2.3 $ ruby local.rb 0.03246927261352539 - before 0.07910513877868652 - after enable 0.10309600830078125 - in block enable 0.12397646903991699 - after block enable 0.07114601135253906 - after disable 0.028218746185302734 - after disable again 0.12534689903259277 - after triple enable 0.02810525894165039 - after single disable ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113966] [Ruby master Bug#19742] Introduce `Module#anonymous?`
by ioquatix (Samuel Williams) 12 Mar '24

12 Mar '24

Issue #19742 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19742: Introduce `Module#anonymous?` https://bugs.ruby-lang.org/issues/19742 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- As a follow-on <from https://bugs.ruby-lang.org/issues/19521>, I'd like propose we introduce `Module#anonymous?`. In some situations, like logging/formatting, serialisation/deserialization, debugging or meta-programming, we might like to know if a class is a proper constant or not. However, this brings about some other issues which might need to be discussed. After assigning a constant, then removing it, the internal state of Ruby still believes that the class name is permanent, even thought it's no longer true. e.g. ``` m = Module.new m.anonymous? # true M = m m.anonyomous # false Object.send(:remove_const, :M) M # uninitialized constant M (NameError) M.anonymous? # false ``` Because RCLASS data structure is not updated after the constant is removed, internally the state still has a "permanent class name". I want to use this proposal to discuss this issue and whether there is anything we should do about such behaviour (or even if it's desirable). Proposed PR: https://github.com/ruby/ruby/pull/7966 cc @fxn -- https://bugs.ruby-lang.org/

9 46