April 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111678] [Ruby master Feature#19315] Lazy substrings in CRuby
by Eregon (Benoit Daloze) 21 Jun '23

21 Jun '23

Issue #19315 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Feature #19315: Lazy substrings in CRuby https://bugs.ruby-lang.org/issues/19315 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal ---------------------------------------- CRuby should implement lazy substrings, i.e., "abcdef"[1..3] must not copy bytes. Currently CRuby only reuse the char* if the substring is until the end of the buffer. But it should also work wherever the substring starts and ends. Yes, it means RSTRING_PTR() might need to allocate to \0-terminate, so be it, it's worth it. There is already code for this (`SHARABLE_MIDDLE_SUBSTRING`), but it's disabled by default and `RSTRING_PTR()` needs to be changed to deal with this. It seems a good idea to introduce a variant of `RSTRING_PTR` which doesn't guarantee \0-termination, so such callers can then use the existing bytes always without copy. There are countless workarounds for this missing optimization, all not worth and all less readable: * https://bugs.ruby-lang.org/issues/19314 * https://github.com/ruby/net-protocol/pull/14 * Manual lazy substrings which track string + index + length * More but I don't remember all now, feel free to comment or link more urls/tickets. -- https://bugs.ruby-lang.org/

7 10

[ruby-core:113238] [Ruby master Misc#19599] DevMeeting-2023-05-10
by mame (Yusuke Endoh) 20 Jun '23

20 Jun '23

Issue #19599 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #19599: DevMeeting-2023-05-10 https://bugs.ruby-lang.org/issues/19599 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2023/05/10 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2023/05/07. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

14 13

[ruby-core:112770] [Ruby master Feature#19520] Support for `Module.new(name)` and `Class.new(superclass, name)`.
by ioquatix (Samuel Williams) 08 Jun '23

08 Jun '23

Issue #19520 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #19520: Support for `Module.new(name)` and `Class.new(superclass, name)`. https://bugs.ruby-lang.org/issues/19520 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- See <https://bugs.ruby-lang.org/issues/19450> for previous discussion and motivation. [This proposal](https://github.com/ruby/ruby/pull/7376) introduces the `name` parameter to `Class.new` and `Module.new`: ```ruby Class.new(superclass, name) Module.new(name) ``` As a slight change, we could use keyword arguments instead. -- https://bugs.ruby-lang.org/

8 41

[ruby-core:111264] [Ruby master Feature#19197] Add Exception#root_cause
by AMomchilov (Alexander Momchilov) 03 Jun '23

03 Jun '23

Issue #19197 has been reported by AMomchilov (Alexander Momchilov). ---------------------------------------- Feature #19197: Add Exception#root_cause https://bugs.ruby-lang.org/issues/19197 * Author: AMomchilov (Alexander Momchilov) * Status: Open * Priority: Normal ---------------------------------------- ## Description I would like to add a `#root_cause` method to `Exception`. It returns the last exception in linked-list of causality chain, that is, the original exception (whose own `cause` is `nil`). ## Example ```ruby e = begin raise 'a' # This is the root cause rescue => a begin raise 'b' rescue => b begin raise 'c' # This is the outermost cause assigned to `e` rescue => c c end end end p(e) # => #<RuntimeError: c> p(e.cause) # => #<RuntimeError: b> p(e.cause.cause) # => #<RuntimeError: a> p(e.cause.cause.cause) # => nil p(e.root_cause) # => #<RuntimeError: a> p(e.root_cause.cause) # => nil ``` # Motivation There are some kinds of exceptions that can occur all over the place (and might be wrapped by arbitrarily many middlemen), but are attributable to a singular global cause. For example, a database outage could raise exceptions in almost every line of business logic of an app that uses ActiveRecord models. Fundamentally, you wouldn't want an error report for every one of these lines. You'd want to look at the root cause, and bucket all SQL-connection issues into a single report, regardless of where they surface. ### Implementation Draft PR: https://github.com/ruby/ruby/pull/6913 -- https://bugs.ruby-lang.org/

4 7

[ruby-core:113332] [Ruby master Bug#19616] Remove ext/readline from Ruby 3.3
by hsbt (Hiroshi SHIBATA) 01 Jun '23

01 Jun '23

Issue #19616 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19616: Remove ext/readline from Ruby 3.3 https://bugs.ruby-lang.org/issues/19616 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We still bundled ext/readline that is GNU Readline wrapper. But we already have reline written by pure Ruby. It's time to remove it from Ruby releases. Motivation: * We can skip to install readline or libedit for testing ruby language. * I hope to reduce external dependencies from the perspective of ruby-build maintainer. If users hope to use ext/readline, they can install it with `gem install readline-ext`. -- https://bugs.ruby-lang.org/

3 6

[ruby-core:112505] [Ruby master Bug#19452] `Thread::Backtrace::Location` should have column information if possible.
by ioquatix (Samuel Williams) 23 May '23

23 May '23

Issue #19452 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #19452: `Thread::Backtrace::Location` should have column information if possible. https://bugs.ruby-lang.org/issues/19452 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I discussed this with @mame and it would be pretty useful if we could also get the column information from exception backtrace location, even if it was slow. A POC: ```ruby class Thread::Backtrace::Location if defined?(RubyVM::AbstractSyntaxTree) def first_column RubyVM::AbstractSyntaxTree.of(self, keep_script_lines: true).first_column end else def first_column raise NotImplementedError end end end ``` It would be good to have a standard interface, so we follow the same interface as https://bugs.ruby-lang.org/issues/19451 and vice versa where it makes sense. I'll investigate it. -- https://bugs.ruby-lang.org/

8 12

[ruby-core:112072] [Ruby master Bug#19386] `test_hmac.rb` of openssl is timeout on RHEL9
by hsbt (Hiroshi SHIBATA) 22 May '23

22 May '23

Issue #19386 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19386: `test_hmac.rb` of openssl is timeout on RHEL9 https://bugs.ruby-lang.org/issues/19386 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I prepared new test instance of RHEL9 for https://rubyci.org/. But it stuck with openssl tests. see http://rubyci.s3.amazonaws.com/rhel9/ruby-master/log/20230127T003004Z.fail.… I did bisect this issue. This issue caused by https://github.com/ruby/ruby/blob/master/test/openssl/test_hmac.rb#L23 ``` def test_dup h1 = OpenSSL::HMAC.new("KEY", "MD5") h1.update("DATA") h = h1.dup assert_equal(h1.digest, h.digest, "dup digest") end ``` Stuck is happened after running this test. /cc @vo.x @jaruga @rhenium -- https://bugs.ruby-lang.org/

7 13

[ruby-core:113045] [Ruby master Feature#19559] Introduce `Symbol#+@` and `Symbol#-@`, and eventually replace boolean arguments with symbols
by sawa (Tsuyoshi Sawada) 22 May '23

22 May '23

Issue #19559 has been reported by sawa (Tsuyoshi Sawada). ---------------------------------------- Feature #19559: Introduce `Symbol#+@` and `Symbol#-@`, and eventually replace boolean arguments with symbols https://bugs.ruby-lang.org/issues/19559 * Author: sawa (Tsuyoshi Sawada) * Status: Open * Priority: Normal ---------------------------------------- I propose to define `Symbol#+@` and `Symbol#-@`, so that we can add positive or negative polarity to symbols. A possible implementation can be equivalent to what can be achieved by this: ```ruby class Symbol def -@; "-#{self}".to_sym end def +@; self end end ``` The intention behind this is to, eventually, replace boolean positional or keyword arguments with symbols so that, instead of this: ```ruby "foo".methods(false) gets(chomp: true) Integer("2.3", exception: false) ``` we can write like this: ```ruby "foo".methods(-:inherit) gets(+:chomp) Integer("2.3", -:exception) ``` -- https://bugs.ruby-lang.org/

8 19

[ruby-core:113303] [Ruby master Feature#19610] GC.delay_promotion
by peterzhu2118 (Peter Zhu) 19 May '23

19 May '23

Issue #19610 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Feature #19610: GC.delay_promotion https://bugs.ruby-lang.org/issues/19610 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal ---------------------------------------- GitHub Pull Request: https://github.com/ruby/ruby/pull/7683 I'm proposing `GC.delay_promotion`, which is a feature that changes characteristics of the garbage collector. When set to `false` (the default value), the GC keeps the original behavior. This means that references from an old object to a write barrier protected young object will immediately promote the young object. References from an old object to a write barrier unprotected object will place the write barrier unprotected object in the remember set for marking during minor collections. When set to `true`, references from an old object to a write barrier protected young object will not immediately promote the young object. Instead, the young object will age just like any other object, meaning that it has to survive three collections before being promoted to the old generation. References from an old object to a write barrier unprotected object will place the parent object in the remember set for marking during minor collections. This allows the child object to be reclaimed in minor collections at the cost of increased time for minor collections. On one of [Shopify's highest traffic Ruby apps, Storefront Renderer](https://shopify.engineering/how-shopify-reduced-storefront-response-times-rewrite), we saw significant improvements after deploying this feature in production. In the graphs below, we compare the GC time and response time of web workers that have `GC.delay_promotion = false` (non-experimental group) and `GC.delay_promotion = true` (experimental group). We see that with this feature we spend significantly less time in the GC, 0.81x on average, 0.88x on p99, and 0.45x on p99.9. ![](Screenshot%202023-04-20%20at%2011.50.26%20AM.png) This translates to improvements in average response time (0.96x) and p99 response time (0.92x). ![](Screenshot%202023-04-20%20at%2011.50.38%20AM.png) In benchmarks, this feature performs better in some scenarios and worse in others. However, these benchmarks are usually not a good indicator as there are few old objects and major collections are not a significant bottleneck. ``` -------------- ----------- ---------- --------- ----------- ---------- --------- -------------- ------------- bench master (ms) stddev (%) RSS (MiB) branch (ms) stddev (%) RSS (MiB) branch 1st itr master/branch activerecord 70.7 0.2 54.4 70.5 0.3 54.7 1.01 1.00 erubi_rails 20.5 2.0 103.0 20.7 2.2 102.2 0.94 0.99 hexapdf 2470.1 0.3 275.6 2475.4 0.7 215.9 1.03 1.00 liquid-c 65.2 1.4 39.8 65.4 0.3 39.7 1.05 1.00 liquid-compile 59.5 3.4 38.9 57.5 3.0 38.3 0.99 1.04 liquid-render 164.3 1.1 36.1 159.8 0.2 35.5 1.02 1.03 mail 135.7 0.2 50.2 135.3 0.1 50.0 1.00 1.00 psych-load 2053.0 0.0 36.4 2003.3 0.1 35.3 1.03 1.02 railsbench 2001.8 0.7 107.1 1965.1 0.0 107.3 1.00 1.02 ruby-lsp 67.1 8.7 94.6 65.1 0.8 92.8 0.97 1.03 sequel 72.4 0.2 40.4 71.3 0.2 40.4 1.01 1.02 -------------- ----------- ---------- --------- ----------- ---------- --------- -------------- ------------- ``` ---Files-------------------------------- Screenshot 2023-04-20 at 11.50.26 AM.png (527 KB) Screenshot 2023-04-20 at 11.50.38 AM.png (350 KB) -- https://bugs.ruby-lang.org/

3 8

[ruby-core:113288] [Ruby master Misc#19608] Being a co-maintainer of the ruby/openssl for the OpenSSL FIPS mode
by jaruga (Jun Aruga) 15 May '23

15 May '23

Issue #19608 has been reported by jaruga (Jun Aruga). ---------------------------------------- Misc #19608: Being a co-maintainer of the ruby/openssl for the OpenSSL FIPS mode https://bugs.ruby-lang.org/issues/19608 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal ---------------------------------------- ## Motivation and context Recently I have been working for the [ruby/openssl](https://github.com/ruby/openssl) to support OpenSSL 3 FIPS mode such as sending pull-requests and reporting issues to the [OpenSSL project](https://github.com/openssl/openssl). The related issue ticket is [here](https://github.com/ruby/openssl/issues/603). Currently a challenge of the ruby/openssl is that it doesn't work well on the OpenSSL FIPS mode, and I want ruby/openssl to work on it by adding the OpenSSL 3 FIPS mode case to the CI, and by adding more FIPS related unit tests and features. To solve this challenge, I would like to be a co-maintainer of the ruby/openssl for the FIPS mode related things. What do you think? ## What is FIPS mode? For someone who is interested in knowing the FIPS mode. Let me share the related documents below. In my understanding, FIPS mode is a security policy developed by US government. In some cases, the shipped Linux OS systems need to follow this policy. FIPS related documents: * [FIPS Wikipedia](https://en.wikipedia.org/wiki/Federal_Information_Processing_Sta… * [RHEL](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linu… * [Amazon Linux](https://aws.amazon.com/blogs/publicsector/enabling-fips-mode-amazon-… * [SUSE Linux](https://www.suse.com/support/kb/doc/?id=000019432) * [Ubuntu](https://ubuntu.com/security/certifications/docs/fips) And OpenSSL has a feature to enable the FIPS mode. The README is [here](https://github.com/openssl/openssl/blob/master/README-FIPS.md). And there can be FIPS specific issues in the ruby/openssl with the OpenSSL FIPS mode enabled. ## Past FIPS related issue tickets As a reference, I just found some old issue tickets below. It is about OpenSSL 1.0 and 1.1 FIPS mode. * https://bugs.ruby-lang.org/issues/6946 * https://bugs.ruby-lang.org/issues/19073 -- https://bugs.ruby-lang.org/

2 5