September 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118542] [Ruby master Bug#20627] `require` on Ractor should run on the main Ractor
by ko1 (Koichi Sasada) 05 Sep '24

05 Sep '24

Issue #20627 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20627: `require` on Ractor should run on the main Ractor https://bugs.ruby-lang.org/issues/20627 * Author: ko1 (Koichi Sasada) * Status: Open * Assignee: ko1 (Koichi Sasada) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Now `require` on main Ractor is not allowed (raising error) but it is hard, especially for `autoload`. So let's allow `require` by running it on the main Ractor. ## Background On many libraries it is needed to run loading on the main Ractors because: 1. Setup constants with unshareable objects (such as `C = []`) are not allowed on non main Ractors. 2. Setup global variables and class variables are not allowed. `$LOADED_FEATURES` is also untouchable. 3. (maybe more reasons) So the `require` on non main Ractors is not allowed. However it is hard to program especially on `autoload`. Also dynamic `require` (`require` in methods) are not allowed too (`pp` method, for example). ## Proposal Allow `require` on non main Ractors by running `require` process on the main Ractor. ![](clipboard-202407111201-kaiem.png) (quoted on my talk at RubyKaigi 2024) ### `rb_ractor_interrupt_exec(target_ractor, func)` C-API Make a thread on `target_ractor` and run `func` (C function) on it. I think it is safe to expose on Ruby API because running `func` on a *newly created* thread (do not disturb running target threads). But now it is proposed as only (hidden) C-API. ### New Ractor methods - `Rcator.main?` returns Ractors - `Ractor.require(feature)` do `require` on the main Ractor These new methods are useful for users who override `require` method like [RubyGems](https://github.com/ruby/ruby/blob/master/lib/rubygems/core_ext/ke…. ```ruby alias orig_require require def require feature return Rator.require(feature) unless Ractor.main? # overriding require code end ``` Or we can prepend a module like: ``` Module.new do def require(feature) return Rator.require(feature) unless Ractor.main? super(feature) end Kernel.prepend self end ``` will support ractors for all overgrinding methods. But not sure it is acceptable to add additional one modules in ancestors by `prepend`. Also this technique doesn't support require overriding by prepending. ## Implementation https://github.com/ruby/ruby/pull/11142 (not matured yet) ---Files-------------------------------- clipboard-202407111201-kaiem.png (70.8 KB) -- https://bugs.ruby-lang.org/

2 2

[ruby-core:119048] [Ruby master Feature#16378] Support leading arguments together with ...
by jeremyevans0 (Jeremy Evans) 04 Sep '24

04 Sep '24

Issue #16378 has been updated by jeremyevans0 (Jeremy Evans). esad (Esad Hajdarevic) wrote in #note-21: > Is there a reason why this doesn't work with keyword arguments too? > > For example: > ``` > def foo(bar:, ...) > other(...) > end > ``` > > results in syntax error Keyword arguments are not leading arguments. Trying to support what you want was not in scope. I think trying to support what you want would significantly increase the complexity, especially if you wanted to support `def foo(arg, kwarg: , ...)` as well. You should use: ```ruby def foo(*, bar:, **, &) other(*, **, &) end ``` ---------------------------------------- Feature #16378: Support leading arguments together with ... https://bugs.ruby-lang.org/issues/16378#change-109626 * Author: Eregon (Benoit Daloze) * Status: Closed * Assignee: matz (Yukihiro Matsumoto) * Target version: 3.0 ---------------------------------------- I think this is very important, otherwise `...` can be used only very rarely. For instance, `method_missing` typically want to access the method name like: ```ruby def method_missing(name, ...) if name.to_s.end_with?('?') self[name] else fallback(name, ...) end end ``` See the original feature: https://bugs.ruby-lang.org/issues/16253#note-19. I think most people expect `def method_missing(name, ...)` to work. ---Files-------------------------------- ruby_2_7_lead_args_forwarding.patch (8.98 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:119047] [Ruby master Feature#16378] Support leading arguments together with ...
by esad (Esad Hajdarevic) 04 Sep '24

04 Sep '24

Issue #16378 has been updated by esad (Esad Hajdarevic). Is there a reason why this doesn't work with keyword arguments too? For example: ``` def foo(bar:, ...) other(...) end ``` results in syntax error ---------------------------------------- Feature #16378: Support leading arguments together with ... https://bugs.ruby-lang.org/issues/16378#change-109625 * Author: Eregon (Benoit Daloze) * Status: Closed * Assignee: matz (Yukihiro Matsumoto) * Target version: 3.0 ---------------------------------------- I think this is very important, otherwise `...` can be used only very rarely. For instance, `method_missing` typically want to access the method name like: ```ruby def method_missing(name, ...) if name.to_s.end_with?('?') self[name] else fallback(name, ...) end end ``` See the original feature: https://bugs.ruby-lang.org/issues/16253#note-19. I think most people expect `def method_missing(name, ...)` to work. ---Files-------------------------------- ruby_2_7_lead_args_forwarding.patch (8.98 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118773] [Ruby master Misc#20660] DevMeeting-2024-09-05
by mame (Yusuke Endoh) 04 Sep '24

04 Sep '24

Issue #20660 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20660: DevMeeting-2024-09-05 https://bugs.ruby-lang.org/issues/20660 * Author: mame (Yusuke Endoh) * Status: Open ---------------------------------------- # The next dev meeting **Date: 2024/09/05 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/09/02. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

6 6

[ruby-core:118996] [Ruby master Bug#20708] EINTR while opening fifo isn't retried
by martin.dorey＠hds.com (Martin Dorey) 04 Sep '24

04 Sep '24

Issue #20708 has been reported by martin.dorey(a)hds.com (Martin Dorey). ---------------------------------------- Bug #20708: EINTR while opening fifo isn't retried https://bugs.ruby-lang.org/issues/20708 * Author: martin.dorey(a)hds.com (Martin Dorey) * Status: Open * ruby -v: ruby 3.4.0dev (2024-08-30T20:36:28Z master eef2121cf3) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I don't think this should fail: ``` martind@stormy:~/download/ruby$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' -e:1:in 'IO.read': Interrupted system call @ rb_sysopen - /tmp/badger (Errno::EINTR) from -e:1:in '<main>' martind@stormy:~/download/ruby$ ``` The behavior changed with 2.6.0-preview3 compared to 2.6.0-preview2: ``` martind@stormy:~/download/ruby-2.6.0-preview2$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' martind@stormy:~/download/ruby-2.6.0-preview3$ rm -f /tmp/badger && mkfifo /tmp/badger && timeout 2 ./ruby --disable-gems -we 'fork() {}; IO.read("/tmp/badger")' Traceback (most recent call last): 1: from -e:1:in `<main>' -e:1:in `read': Interrupted system call @ rb_sysopen - /tmp/badger (Errno::EINTR) martind@stormy:~/download/ruby-2.6.0-preview3$ ``` I don't know what the consequences would be, so I don't mean this as a serious suggestion: ``` martind@stormy:~/download/ruby$ git diff diff --git a/io.c b/io.c index 54b60b7357..590e8d11d7 100644 --- a/io.c +++ b/io.c @@ -6956,7 +6956,10 @@ sysopen_func(void *ptr) static inline int rb_sysopen_internal(struct sysopen_struct *data) { - int fd = IO_WITHOUT_GVL_INT(sysopen_func, data); + int fd; + do { + fd = IO_WITHOUT_GVL_INT(sysopen_func, data); + } while (fd < 0 && errno == EINTR); if (0 <= fd) rb_update_max_fd(fd); return fd; martind@stormy:~/download/ruby$ ``` ... but it fixes this particular test case, for me, with today's Ruby from github. There were negligible differences between 2.6.0-preview{2,3} in io.c but the SIGCHLD handling was changed. The test case is reliable for me on at least: Linux stormy 4.19.0-26-amd64 #1 SMP Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux Linux moritz 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux I hope it's independent of the kernel, as: https://man7.org/linux/man-pages/man7/signal.7.html ... teaches: > otherwise the call fails with the error EINTR: ... > • open(2), if it can block (e.g., when opening a FIFO; see > fifo(7)). -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118998] [Ruby master Feature#20709] Improve String#rindex performance on OSX
by zack.ref＠gmail.com (Zack Deveau) 03 Sep '24

03 Sep '24

Issue #20709 has been reported by zack.ref(a)gmail.com (Zack Deveau). ---------------------------------------- Feature #20709: Improve String#rindex performance on OSX https://bugs.ruby-lang.org/issues/20709 * Author: zack.ref(a)gmail.com (Zack Deveau) * Status: Open ---------------------------------------- `String#rindex` is much slower on OSX than on Linux hosts. This appears due to the lack of a `memrchr` implementation on OSX. The [fallback solution](https://github.com/ruby/ruby/blob/e69945fc57efba5c8895c21235e1091… currently performs a `memcmp` on each character in the search string looking for a substring match. ```c while (s) { if (memcmp(s, t, slen) == 0) { return s - sbeg; } if (s <= sbeg) break; s = rb_enc_prev_char(sbeg, s, e, enc); } ``` ``` ruby-master | 10 char substring (long) near the end of a search string | osx search string enc: UTF-8 search string size: 8010 user system total real index (long substring) 0.000075 0.000000 0.000075 ( 0.000074) rindex 0.000445 0.000000 0.000445 ( 0.000448) ``` Proposed improvement is available on this pull request: This patch proposes using an implementation of `memrchr` made for the interpreter on hosts that lack a native implementation. Below are benchmark results which demonstrate that this approach either improves or maintains performance of `String#rindex` for all cases we could come up with. ```c static void* rb_memrchr(const char *search_str, int chr, long search_len) { const char *ptr = search_str + search_len; do { if (*(--ptr) == chr) return (void *)ptr; } while (ptr >= search_str); return ((void *)0); } ``` Special thanks to @jhawthorn who provided some helpful feedback on this solution during development! # Benchmarks Each of the following were made with a variant of the following general purpose benchmark. Note that `String#index` speed can be [impacted by the size of the provided substring](https://github.com/ruby/ruby/blob/669d1f79d800d554918c7d5647b73a… (Denoted as long/short substring in the results, assume long substring when none is specified). ``` require "benchmark" substring = "98279827AA" the_string = "" 800.times{ the_string.concat(('a'..'z').to_a.shuffle[0,8].join)} the_string.concat(substring) 200.times{ the_string.concat(('a'..'z').to_a.shuffle[0,8].join)} puts the_string.encoding puts the_string.size Benchmark.bm do |x| x.report(:index) { 100.times { the_string.index(substring) } } x.report(:rindex) { 100.times { the_string.rindex(substring) } } end ``` ``` ruby-master | substring near the end | osx https://github.com/ruby/ruby/blob/669d1f79d800d554918c7d5647b73a6431920ea8/… UTF-8 user system total real index 0.000075 0.000000 0.000075 ( 0.000074) rindex 0.000445 0.000000 0.000445 ( 0.000448) ------------------------------------------------------ ruby-patched | substring near the end | osx https://github.com/ruby/ruby/blob/669d1f79d800d554918c7d5647b73a6431920ea8/… UTF-8 user system total real index (short substring) 0.000400 0.000001 0.000401 ( 0.000399) index (long substring) 0.000080 0.000000 0.000080 ( 0.000080) rindex 0.000057 0.000001 0.000058 ( 0.000057) ``` ``` ruby-master | substring in the middle | osx UTF-8 user system total real index 0.000031 0.000000 0.000031 ( 0.000030) rindex 0.001086 0.000002 0.001088 ( 0.001088) ------------------------------------------------------ ruby-patched | substring in the middle | osx UTF-8 user system total real index 0.000075 0.000000 0.000075 ( 0.000075) rindex 0.000130 0.000000 0.000130 ( 0.000133) ``` ``` ruby-master | substring in the middle with a short substring | osx https://github.com/ruby/ruby/blob/669d1f79d800d554918c7d5647b73a6431920ea8/… UTF-8 user system total real index (short substring) 0.000243 0.000004 0.000247 ( 0.000246) rindex 0.001176 0.000005 0.001181 ( 0.001182) ------------------------------------------------------ ruby-patched | substring in the middle with a short substring | osx https://github.com/ruby/ruby/blob/669d1f79d800d554918c7d5647b73a6431920ea8/… UTF-8 user system total real index (short substring) 0.000255 0.000005 0.000260 ( 0.000259) rindex 0.000130 0.000000 0.000130 ( 0.000133) ``` ``` ruby-master | substring at the end | osx UTF-8 user system total real index 0.000049 0.000001 0.000050 ( 0.000048) rindex 0.000008 0.000001 0.000009 ( 0.000007) ------------------------------------------------------ ruby-patched | substring at the end | osx UTF-8 user system total real index 0.000075 0.000000 0.000075 ( 0.000074) rindex 0.000006 0.000000 0.000006 ( 0.000006) ``` ``` ruby-master | substring at the start | osx UTF-8 user system total real index 0.000008 0.000001 0.000009 ( 0.000007) rindex 0.002444 0.000008 0.002452 ( 0.002452) ------------------------------------------------------ ruby-patched | substring at the start | osx UTF-8 user system total real index 0.000015 0.000000 0.000015 ( 0.000015) rindex 0.000280 0.000001 0.000281 ( 0.000280) ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118180] [Ruby master Bug#20525] Percent string literal with indentation support
by bradgessler (Brad Gessler) 03 Sep '24

03 Sep '24

Issue #20525 has been reported by bradgessler (Brad Gessler). ---------------------------------------- Bug #20525: Percent string literal with indentation support https://bugs.ruby-lang.org/issues/20525 * Author: bradgessler (Brad Gessler) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I have code that looks like this in an application: ```ruby ContentSlide(title: "Why Phlex?"){ Markdown <<~MARKDOWN * Because its fun * Because its super-de-dooper MARKDOWN }, ``` The "squiggle" HEREDOCs strips the indentation, but the name of the HEREDOC doesn't look that great since "Markdown" appears twice. What I'd prefer is a string literal that deals with indentation, maybe it looks something like this: ``` ContentSlide(title: "Why Phlex?"){ Markdown %~{ # Why do you like markdown? * Because its fun * Because its super-de-dooper } }, ``` If Ruby doesn't want to go down the path of adding another type of literal, I'd propose adding a method to String that does the same thing as the `~` HEREDOC so something like this is possible: ```ruby ContentSlide(title: "Why Phlex?"){ Markdown %{ # Why do you like markdown? * Because its fun * Because its super-de-dooper }.deindent }, ``` -- https://bugs.ruby-lang.org/

5 7

[ruby-core:119023] [Ruby master Feature#20712] module should return newly defined module
by esad (Esad Hajdarevic) 03 Sep '24

03 Sep '24

Issue #20712 has been reported by esad (Esad Hajdarevic). ---------------------------------------- Feature #20712: module should return newly defined module https://bugs.ruby-lang.org/issues/20712 * Author: esad (Esad Hajdarevic) * Status: Open ---------------------------------------- Currently module keyword returns nil after defining a new module: ``` x = module Foo end # x is nil ``` It would be more consistent with `Module.new` if module keyword returned the newly defined module. Then we could do: using(module Foo refine ... end) instead of module Foo end using Foo -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118892] [Ruby master Misc#20685] Ruby 3.2.4 tag mentions unrelated changes (CVE-2024-27280)
by kenhys (Kentaro Hayashi) 03 Sep '24

03 Sep '24

Issue #20685 has been reported by kenhys (Kentaro Hayashi). ---------------------------------------- Misc #20685: Ruby 3.2.4 tag mentions unrelated changes (CVE-2024-27280) https://bugs.ruby-lang.org/issues/20685 * Author: kenhys (Kentaro Hayashi) * Status: Open ---------------------------------------- # Problem According to https://github.com/ruby/ruby/releases/tag/v3_2_4, it mentions "CVE-2024-27280: Buffer overread vulnerability in StringIO" as a security fix, but https://www.ruby-lang.org/en/news/2024/03/21/buffer-overread-cve-2024-27280/ explicitly describe that the following: > This vulnerability is not affected StringIO 3.0.3 and later, and Ruby 3.2.x and later. so, it is a bit strange that CVE-2023-27280 was mentioned as security fix for 3.2.x, IMHO. Please correct me if I'm wrongly interpreted it. # Expected The problematic description was removed from tags and release note. # Additional Information * https://github.com/ruby/ruby/releases/tag/v3_2_4 * mention it as security fix * https://www.ruby-lang.org/ja/news/2024/04/23/ruby-3-2-4-released/ * mention it as security fix * https://www.ruby-lang.org/en/news/2024/04/23/ruby-3-2-4-released/ * mention it as security fix -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118906] [Ruby master Bug#20687] Does tarball require baseruby?
by hsbt (Hiroshi SHIBATA) 03 Sep '24

03 Sep '24

Issue #20687 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #20687: Does tarball require baseruby? https://bugs.ruby-lang.org/issues/20687 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- From https://github.com/rbenv/ruby-build/issues/2435 I'm wondering why tarball use system ruby with above environment. In my understanding, tarball don't need baseruby. Is it intentional? -- https://bugs.ruby-lang.org/

5 5