June 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:113905] [Ruby master Bug#15428] Refactor Proc#>> and #<<
by matz (Yukihiro Matsumoto) 14 Jun '23

14 Jun '23

Issue #15428 has been updated by matz (Yukihiro Matsumoto). I don't feel this proposed extension readable (than explicit one). Is there any example of readable usage of this notation? Matz. ---------------------------------------- Bug #15428: Refactor Proc#>> and #<< https://bugs.ruby-lang.org/issues/15428#change-103557 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- #6284 introduced `Proc#>>` and `Proc#<<`, but the requirements to the argument is totally inconsistent with ANY other place in Ruby. Currently, it is the **only** place in Ruby where coercing argument to `Proc` is done with `#call` method. Everywhere else it is done with `#to_proc`, and `#call` method never had any special significance except for `.()` sugar. I believe there are two possible actions: 1. change `#>>` and `#<<` to use `#to_proc` (which will give Symbols composability for free), **or, alternatively** 2. state that `#call` from now on has a special meaning in Ruby and probably decide on other APIs that should respect it (for example, auto-define `#to_proc` on any object that has `#call`) Either is OK, the current situation is not. PS: One more problem (that probably should be discussed separately) is that check for `#call` existence is performed pretty late, which can lead to this kind of errors: ```ruby # At code loading time: # I erroneously thought this is correct. It is not, but the line would perform without # any error. PROCESSOR = JSON.method(:parse) >> :symbolize_keys # Later, in runtime: '{"foo": "bar"}'.then(&PROCESSOR) # NoMethodError (undefined method `call' for :symbolize_keys:Symbol) ``` **UPD 2018-12-29:** As this ticket was ignored prior to 2.6 release, I rewrote it in an "actionable" instead of "question" manner. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113903] [Ruby master Bug#19731] Can’t call an instance method inside an Endless method definition
by ttanimichi (Tsukuru Tanimichi) 14 Jun '23

14 Jun '23

Issue #19731 has been reported by ttanimichi (Tsukuru Tanimichi). ---------------------------------------- Bug #19731: Can’t call an instance method inside an Endless method definition https://bugs.ruby-lang.org/issues/19731 * Author: ttanimichi (Tsukuru Tanimichi) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Can’t call an instance method inside an Endless method definition. ``` $ ruby -v ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22] $ cat withend.rb class A def show puts 'ok' if params end def params 42 end end A.new.show $ ruby withend.rb ok $ cat endless.rb class A def show = puts 'ok' if params def params 42 end end A.new.show $ ruby endless.rb endless.rb:2:in `<class:A>': undefined local variable or method `params' for A:Class (NameError) def show = puts 'ok' if params ^^^^^^ from endless.rb:1:in `<main>' exit status: 1 ``` But, the example below works fine. Is this an expected behavior? ``` $ cat endless2.rb class A def show = puts params def params 42 end end A.new.show $ ruby endless2.rb 42 ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113781] [Ruby master Bug#19715] Coverage oneshot_lines mode is not working after result clear
by ngan (Ngan Pham) 13 Jun '23

13 Jun '23

Issue #19715 has been reported by ngan (Ngan Pham). ---------------------------------------- Bug #19715: Coverage oneshot_lines mode is not working after result clear https://bugs.ruby-lang.org/issues/19715 * Author: ngan (Ngan Pham) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Given a simple Ruby file (`foo.rb`): ```ruby def hello puts "hello" end ``` If I do the following in `irb`, I get unexpected results: ``` irb(main):001:0> require 'coverage' irb(main):002:0> Coverage.setup(oneshot_lines: true) => nil irb(main):003:0> require_relative 'foo' => true irb(main):004:0> Coverage.resume => nil irb(main):005:0> hello hello => nil irb(main):006:0> Coverage.suspend => nil irb(main):007:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:oneshot_lines=>[2]}} irb(main):008:0> Coverage.resume => nil irb(main):009:0> Coverage.suspend => nil irb(main):010:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:oneshot_lines=>[]}} irb(main):011:0> Coverage.resume => nil irb(main):012:0> hello hello => nil irb(main):013:0> Coverage.suspend => nil irb(main):014:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:oneshot_lines=>[]}} ``` You can see that in the third `Coverage.result` call, oneshot_lines did not populate despite me executing `hello`. If I use `lines`, then it works the way I'd expect it to: ``` irb(main):001:0> require 'coverage' irb(main):002:0> Coverage.setup(lines: true) => nil irb(main):003:0> require_relative 'foo' => true irb(main):004:0> Coverage.resume => nil irb(main):005:0> hello hello => nil irb(main):006:0> Coverage.suspend irb(main):007:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:lines=>[0, 1, nil]}} irb(main):008:0> Coverage.resume => nil irb(main):009:0> Coverage.suspend irb(main):010:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:lines=>[0, 0, nil]}} irb(main):011:0> Coverage.resume => nil irb(main):012:0> hello hello => nil irb(main):013:0> Coverage.suspend irb(main):014:0> Coverage.result(stop: false, clear: true) => {"/project/foo.rb"=>{:lines=>[0, 1, nil]}} ``` I get that `oneshot_lines` is only suppose to count the line once, but I'd expect that upon a reset/clear, it should remark the line as executed. I wasn't sure if this is expected behavior or a bug. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113896] [Ruby master Feature#18996] Proposal: Introduce new APIs to reline for changing dialog UI colours
by hasumikin (hitoshi hasumi) 13 Jun '23

13 Jun '23

Issue #18996 has been updated by hasumikin (hitoshi hasumi). I opened a PR to restart this discussion: https://github.com/ruby/reline/pull/552 ---------------------------------------- Feature #18996: Proposal: Introduce new APIs to reline for changing dialog UI colours https://bugs.ruby-lang.org/issues/18996#change-103548 * Author: st0012 (Stan Lo) * Status: Open * Priority: Normal ---------------------------------------- ### TL;DR I want to add APIs to `reline` for changing its dialog item's colors. The APIs I want to add actually have been merged but becaue: 1. This is a design change 2. The maintainer @aycabta is not available to approve nor reject them I want to raise it here to decide if we should: 1. Drop them 2. Modify them 3. Officiallty accept them ### Background After version `1.4`, `irb` provides autocompletion support, which is a great feature and has increased many developers' productivity significantly. But there's an user-experience issue: the completion items' UI colors (set in `reline`) [are not configurable](https://github.com/ruby/reline/blob/9ab5850444b49aff8e360a84e…. So depending on the user's terminal theme, some may find it hard to use because the background and the text having low-contrast colors, like this: ![](https://user-images.githubusercontent.com/3303032/148653612-e3dff786-1a10-4923-a0eb-3975cae10a7f.png) And if that happens, the user has no way to fix it. This caused users to open issues like: - https://github.com/ruby/irb/issues/351 - https://github.com/ruby/irb/issues/328 for being able to change it. Some users even decided to disable it completely because the colors are unreadable to them. I have also seen people sharingtips for disabling this feature: [example](https://twitter.com/sdogruyol/status/1538512030449254400). So I believe it may be bothering many developers. Personally I really like this feature but the background also bothers me: ![Screenshot 2022-09-07 at 22 55 12](https://user-images.githubusercontent.com/5079556/188990620-5ec7ba0c-97… And that's why I want to improve it by making the colors configurable and potentially also by providing simple light/dark themes from `irb`. ### Proposal For the dialog UI, there are 2 element states: `highlighted` and `default`. In `irb`'s case, the selected completion candidate will be `highlighted`, and the rest of options will be `default`. And each state has 2 colors: `foreground (text)` and `background (block)`. This means the `reline` should allow `irb` and/or users to configure: - Default items' foreground color - Default items' background color - Highlighted items' foreground color - Highlighted items' background color That brings us to these APIs: - `Reline.dialog_default_fg_color` - `Reline.dialog_default_bg_color` - `Reline.dialog_highlight_fg_color` - `Reline.dialog_highlight_bg_color` And because `reline` only supports coloring through ANSI sequences, these APIs only has 8 available colors if we exclude their bright variants: - Black - Red - Green - Yellow - Blue - Magenta - Cyan - White Given the limited options and also to prevent users from entering non-color ANSI sequences, these APIs only take color names directly: - :black - :red - :green - :yellow - :blue - :magenta - :cyan - :white Example: ```rb Reline.dialog_default_bg_color = :black puts Reline.dialog_default_bg_color_sequence #=> 40 Reline.dialog_default_fg_color = :white puts Reline.dialog_default_fg_color_sequence #=> 37 Reline.dialog_highlight_bg_color = :blue puts Reline.dialog_highlight_bg_color_sequence #=> 34 Reline.dialog_highlight_fg_color = :black puts Reline.dialog_highlight_fg_color_sequence #=> 30 ``` I have made a [proof of concept PR](https://github.com/ruby/irb/pull/380) on `irb` to show what these APIs can achieve if they or similar ones are adopted. #### Related PRs The related changes are made through multiple PRs: - [Initial APIs PR by @pocari](https://github.com/ruby/reline/pull/413) - [PR to improve the APIs and make them safer to use](https://github.com/ruby/reline/pull/454) - [PR to rename the APIs](https://github.com/ruby/reline/pull/456) #### Other Thoughts This is more of a concern on the `irb` part, but to make the UI looks comfortable, I think it's better to follow these conditions: 1. An item's foreground and background colors should have high contrast with each other so the texts (foreground) are readable. 2. For the `highlighted` item, its background color should be easily distinguishable from the rest of `default` items. 3. When using dark terminal themes, the `default` items' background is better to be dark as well. ---Files-------------------------------- Screenshot 2022-10-18 at 15.25.36.png (9.7 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113740] [Ruby master Bug#19707] Backport openssl fix to ruby_3_2 branch
by eileencodes (Eileen Uchitelle) 13 Jun '23

13 Jun '23

Issue #19707 has been reported by eileencodes (Eileen Uchitelle). ---------------------------------------- Bug #19707: Backport openssl fix to ruby_3_2 branch https://bugs.ruby-lang.org/issues/19707 * Author: eileencodes (Eileen Uchitelle) * Status: Open * Priority: Normal * ruby -v: 3.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi! While working on backports to fix another bug, I noticed that the openssl tests for Visual Studio code were failing with an unrelated error. I tracked down the fix on master and opened a backport PR for ruby_3_2 here https://github.com/ruby/ruby/pull/7902. The branch where I found this issue was on https://github.com/ruby/ruby/pull/7888 and the build is https://github.com/ruby/ruby/actions/runs/5146819376/jobs/9266401095?pr=788… -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113892] [Ruby master Bug#19727] Backport CI fixes for test-bundled-gems
by hsbt (Hiroshi SHIBATA) 13 Jun '23

13 Jun '23

Issue #19727 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19727: Backport CI fixes for test-bundled-gems https://bugs.ruby-lang.org/issues/19727 * Author: hsbt (Hiroshi SHIBATA) * Status: Closed * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- After https://bugs.ruby-lang.org/issues/19702, CI of Ruby 3.0-3.2 is broken. I fixed them with: * [for Ruby 3.2](https://github.com/ruby/ruby/pull/7934) * [for Ruby 3.1](https://github.com/ruby/ruby/pull/7935) * [for Ruby 3.0](https://github.com/ruby/ruby/pull/7936) I added additional patches to fix CI failure related `openssl` and `power_assert` for Ruby 3.2 and 3.0. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:113879] [Ruby master Bug#19724] Dir.home returns incorrect home directory
by seb (Sebastian Unger) 12 Jun '23

12 Jun '23

Issue #19724 has been reported by seb (Sebastian Unger). ---------------------------------------- Bug #19724: Dir.home returns incorrect home directory https://bugs.ruby-lang.org/issues/19724 * Author: seb (Sebastian Unger) * Status: Open * Priority: Normal * ruby -v: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux-gnu] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This is probably related to 16787. From the patch on that issue I conclude that ruby uses getlogin and co to obtain the name of the logged-in user, but that is not valid. Ruby must use the uid of the running process since the user that logged in may since have switched to a different user-id (e.g. using `sudo -u <user> -i`). This will cause getlogin to still report the name that the user logged in at the current terminal, but that is not the user that ruby should be using. E.g., If I ssh into a machine as root, and then use sudo -u user -i and then run a ruby process that tries to write to "my" home directory it fails (most commonly in trying to access /root/.gem or similar). I know why ruby originally wanted to use getlogin (at least according to comments on the above issue), which was to support multiple login names having the same UID but different home directories. To the best of my knowledge, this was always an invalid configuration. Usernames and UIDs in the PW DB should have a 1-to-1 mapping. In any case getlogin is not the right answer, since it does not yield the login name of the running process. It is particularly bad since ruby uses this method even if ${HOME} is set! This is on current up-to-date Ubuntu 22.04. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113800] [Ruby master Feature#19719] Universal Parser
by yui-knk (Kaneko Yuichiro) 12 Jun '23

12 Jun '23

Issue #19719 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Feature #19719: Universal Parser https://bugs.ruby-lang.org/issues/19719 * Author: yui-knk (Kaneko Yuichiro) * Status: Open * Priority: Normal ---------------------------------------- # Background There are three use cases where we need a CRuby parser which is independent of other CRuby features like Object, GC. I call such a parser a Universal Parser. 1. Use Universal Parser from Ruby applications. For example RuboCop. RuboCop needs a parser to get AST of source code. Currently RuboCop uses parser gem. In this sense Universal Parser is a replacement of parser gem. 2. Use Universal Parser from C/C++ or other language. For example Sorbet. Sorbet is written in C++. It has its own parser. Universal Parser can be used in such scenario. 3. Use Universal Parser for other Ruby implementations. mruby, JRuby and other Ruby implementations will use Universal Parser so that they don’t need to develop & manage their own parser. # Design * Implement Universal Parser by changing CRuby source code, especially parse.y and node.c. Introduce `UNIVERSAL_PARSER` macro and implement Universal Parser by passing all necessary CRuby related functions via `struct rb_parser_config_struct`. In this step there are two build modes with/without Universal Parser. * Reduce CRuby related functions passed by `struct rb_parser_config_struct`. Some of them are copied into parse.y, e.g. `rb_isspace`. Other are reimplemented, e.g. `NODE_LIT` has new String struct instead of `VALUE`. * Once CRuby related functions needed for Universal Parser are minimized, replace rest CRuby function references with functions provided by `struct rb_parser_config_struct` and remove `UNIVERSAL_PARSER` macro. # Release management There are three options for releasing a binary for Universal Parser (“librubyparser”). 1. Release it as so/dll a. Include it into Ruby release process b. Create another repository for release management of "librubyparser" 2. Release it as gem (ruby/universal_parser) "librubyparser" has only pure C interface and data structure. If you want to use it from Ruby code, you need C extension code to translate "librubyparser" data and Ruby objects. I propose to create "universal_parser" gem for this purpose. I prefer #1-b to #1-a because it doesn’t increase tasks for Ruby release process. I want to make #2 as a first milestone. -- https://bugs.ruby-lang.org/

6 6

[ruby-core:113854] ruby/ruby Cirrus CI improvement
by Jun Aruga (he / him) 12 Jun '23

12 Jun '23

Hello Fedor @ Cirrus CI, CC: Ruby Core mailing list. Thank you for helping the Ruby project for adding Cirrus CI. I am sending an email to ask you for help to improve the Cirrus CI. We are running 4 parallel jobs (2 gcc cases ARM, 2 clang cases ARM) in the Cirrus CI. However, I am seeing some challenges such as the queue is stopping, and the clang cases are 2 or 3 times slower than gcc cases. So, today we stopped the 2 clang cases in a compromised way. Could you check our .cirrus.yml file on the reverted commit, and give advice to us and ideally just send a pull-request to the master branch for that? https://github.com/ruby/ruby/blob/master/.cirrus.yml ``` $ git clone https://github.com/ruby/ruby.git $ cd ruby $ git revert 72f07f0a5f882e87e305d668587152fa209a0568. ``` I suspect the current `cpu:` value might not be correct. And we may need the following change. ``` diff --git a/.cirrus.yml b/.cirrus.yml index be76e4ab4a..8b820be1a2 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -16,10 +16,10 @@ task: image: ghcr.io/ruby/ruby-ci-image:$CC # Define the used cpu core in each matrix task. We can use total 16 cpu # cores in entire matrix. [cpu] = [total cpu: 16] / [number of tasks] - cpu: 8 + cpu: 4 # We can request maximum 4 GB per cpu. # [memory per task] = [memory per cpu: 4 GB] * [cpu] - memory: 32G + memory: 16G env: CIRRUS_CLONE_DEPTH: 50 optflags: '-O1' @@ -71,10 +71,10 @@ yjit_task: image: ghcr.io/ruby/ruby-ci-image:$CC # Define the used cpu core in each matrix task. We can use total 16 cpu # cores in entire matrix. [cpu] = [total cpu: 16] / [number of tasks] - cpu: 8 + cpu: 4 # We can request maximum 4 GB per cpu. # [memory per task] = [memory per cpu: 4 GB] * [cpu] - memory: 32G + memory: 16G env: CIRRUS_CLONE_DEPTH: 50 optflags: '-O1' ``` Thank you for your help! -- Jun | He - Him | Timezone: UTC+1 or 2, Czech Republic See <https://www.worldtimebuddy.com/czech-republic-prague-to-utc> for the timezone.

1 1

[ruby-core:113882] [Ruby master Feature#19725] Improve the match cache optimization to support look-ahead and atomic groups
by make_now_just (Hiroya Fujinami) 12 Jun '23

12 Jun '23

Issue #19725 has been reported by make_now_just (Hiroya Fujinami). ---------------------------------------- Feature #19725: Improve the match cache optimization to support look-ahead and atomic groups https://bugs.ruby-lang.org/issues/19725 * Author: make_now_just (Hiroya Fujinami) * Status: Open * Priority: Normal ---------------------------------------- Currently, the Regexp match cache optimization [Feature #19104] memoizes match cache points that once arrived; that is, it memoizes them **before backtracking**. This kind of implementation is simple and works fine in most cases. However, in some cases, it does not work well. For example, 1. it is hard to preserve the strange null-loop behavior, and 2. it is hard to support look-around operators and atomic groups correctly. The first problem is simple. Memoization may change the matching behavior if a cache point in a null loop is memoized because Onigmo allows a null loop (a loop without consuming a character) at the first time to keep captures. We avoid this problem by carefully resetting the memoization table on null loops currently. But, nested null loops are not supported. The second problem is a bit complex. For example, `/\Aa*(?=a*)a\z/` does not match against `"a"` on the memozation before backtracking. The reason is that when the first matching of the look-ahead is succeeded, the loop branch in the look-ahead is memoized, and it is impossible to succeed the second matching of the look-ahead. Furthermore, how do we prevent increasing the matching time of `/\Aa*?(?=a*)\z/` against `"a" * N + "z"` quadratically? I suggest two improvements to resolve these issues: 1. Memoizing **after backtracking**, and 2. Memoizing **partial match results** for match cache points in look-ahead and atomic groups. The first improvement is, I think, the canonical way to implement the match cache optimization. This avoids the first "null loop" problem entirely. This also enables to support look-ahead and atomic groups easily. This can be implemented by introducing an extra item to the stack and doing memoization on popping it. The second improvement is to simulate on the memoization the atomic behavior of look-ahead and atomic (Of course!) groups on the memoization. The current memoization table only records the arrived points; that is, it does not record the result of matching and only records failures of matching. Actually, it is not problematic in the current implementation because "matching is succeeded" means the matching is done. However, when a part of look-around and atomic groups is matched, the stack is hoisted to behave atomically. To simulate this behavior, successes in matching these parts are also memoized. This can be implemented by introducing an extra bit for the result of matching to the memoization table. These improvements will enable - to avoid the nested null loop restriction, and - to support positive/negative look-ahead and atomic groups correctly. However, look-ahead and atomic groups that include captures cannot be supported. This is because captures may be changed on successful matches. -- https://bugs.ruby-lang.org/

1 1