December 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112947] [Ruby master Bug#19542] Operations on zero-sized IO::Buffer are raising
by hanazuki (Kasumi Hanazuki) 05 Feb '24

05 Feb '24

Issue #19542 has been reported by hanazuki (Kasumi Hanazuki). ---------------------------------------- Bug #19542: Operations on zero-sized IO::Buffer are raising https://bugs.ruby-lang.org/issues/19542 * Author: hanazuki (Kasumi Hanazuki) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I found that IO::Buffer of zero length is not cloneable. ``` % ruby -v ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] % ruby -e 'p IO::Buffer.for("").dup' -e:1:in `initialize_copy': The buffer is not allocated! (IO::Buffer::AllocationError) from -e:1:in `initialize_dup' from -e:1:in `dup' from -e:1:in `<main>' % ruby -e 'p IO::Buffer.new(0).dup' -e:1: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future! -e:1:in `initialize_copy': The buffer is not allocated! (IO::Buffer::AllocationError) from -e:1:in `initialize_dup' from -e:1:in `dup' from -e:1:in `<main>' ``` It seems `IO::Buffer.new(0)` allocates no memory for buffer on object creation and thus prohibits reading from or writing to it. So `#dup` method copying zero bytes into the new IO::Buffer raises the exception. Empty buffers, however, often appear in corner cases of usual operations (encrypting an empty string, encoding an empty list of items into binary, etc.) and it would be easy if such cases could be handled consistently. Other operations on NULL IO::Buffers are also useful but currently raising. ``` IO::Buffer.new(0) <=> IO::Buffer.new(1) IO::Buffer.new(0).each(:U8).to_a IO::Buffer.new(0).get_values([], 0) IO::Buffer.new(0).set_values([], 0, []) ``` I'm not sure this is a bug or by design, but at least I don't want cloning and comparison to raise. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:115978] [Ruby master Bug#20104] Regexp#match returns nil but allocates T_MATCH objects
by jeremyevans0 (Jeremy Evans) 31 Jan '24

31 Jan '24

Issue #20104 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20104: Regexp#match returns nil but allocates T_MATCH objects https://bugs.ruby-lang.org/issues/20104 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2023-12-30T03:14:38Z master 8e32c01742) [x86_64-openbsd7.4] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Between Ruby 3.2 and 3.3, behavior changed so that Regexp#match will allocate a T_MATCH object even when there is no match. Example code: ```ruby h = {} GC.start GC.disable ObjectSpace.count_objects(h) matches = h[:T_MATCH] || 0 md = /\A[A-Z]+\Z/.match('1') ObjectSpace.count_objects(h) new_matches = h[:T_MATCH] || 0 puts "/\\A[A-Z]+\\Z/.match('1') => #{md.inspect} generates #{new_matches - matches} T_MATCH objects" ``` Result with Ruby 1.9-3.2: ``` /\A[A-Z]+\Z/.match('1') => nil generates 0 T_MATCH objects ``` Results with Ruby 3.3.0 and current master branch: ``` /\A[A-Z]+\Z/.match('1') => nil generates 1 T_MATCH objects ``` This results in a measurable performance decrease for both Sinatra and Roda web applications, as reported at: https://old.reddit.com/r/ruby/comments/18sxtv9/ruby_330_performance_ups_and… Thanks to GitHub users kiskoza and tagliala for producing a minimal example showing this issue: https://github.com/caxlsx/caxlsx/issues/336 -- https://bugs.ruby-lang.org/

3 4

[ruby-core:115338] [Ruby master Bug#19999] Backport: .travis.yml and fixed commits
by jaruga (Jun Aruga) 31 Jan '24

31 Jan '24

Issue #19999 has been reported by jaruga (Jun Aruga). ---------------------------------------- Bug #19999: Backport: .travis.yml and fixed commits https://bugs.ruby-lang.org/issues/19999 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: REQUIRED ---------------------------------------- This is a backport suggestion to make Travis CI stable on the Ruby stable branches ruby_3_2 and etc. Travis CI for Ruby master branch became stable recently with arm64, ppc64le, s390x and arm32 cases. All the CI cases are running without `allow_failures` option. And more importantly, I simplified the `.travis.yml` to maintain it easily without sacrificing performance. So, I think it may be a good time to backport the Travis CI configuration file `.travis.yml` without some commits to fix some issues. https://app.travis-ci.com/github/ruby/ruby/builds/267166336 I can see ruby_3_2 and ruby_3_1 branches on the Travis CI page below. There are no ruby_3_0 and ruby_27 branches there. I am not sure how the branches are used. https://app.travis-ci.com/github/ruby/ruby/branches But it's beneficial to fix at least Travis CI for ruby_3_2 branch to save the Travis infra resource. Seeing the ruby_3_2 log, the ppc64le is already timeout after running maximum 50 minutes. https://app.travis-ci.com/github/ruby/ruby/builds/267156055 https://app.travis-ci.com/github/ruby/ruby/jobs/613043898#L2325 ``` [1/2] TestFiberQueue#test_pop_with_timeout_and_value = 0.00 s [2/2] TestFiberQueue#test_pop_with_timeout====[ 540 seconds still running ]==== ====[ 1080 seconds still running ]==== ====[ 1620 seconds still running ]==== ====[ 2160 seconds still running ]==== ``` First you can make the tests fail rather than stucking by the commit below. https://github.com/ruby/ruby/commit/3eaae72855b23158e2148566bb8a7667bfb395cb As this stucking issue happened with the combination of the `optflags=-O1` on ppc64le, I think you avoid the issue by porting the `.travis.yml` from the `master` branch where the `optflags=-O1` is not used in ppc64le. -- https://bugs.ruby-lang.org/

2 6

[ruby-core:114089] [Ruby master Misc#19758] Statically link ext/json
by MyCo (Maik Menz) 30 Jan '24

30 Jan '24

Issue #19758 has been reported by MyCo (Maik Menz). ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/

4 6

[ruby-core:115888] [Ruby master Bug#20083] Regexp#match? behaving inconsistently with Ruby 3.3.0
by jussikos (Jussi Koljonen) 29 Jan '24

29 Jan '24

Issue #20083 has been reported by jussikos (Jussi Koljonen). ---------------------------------------- Bug #20083: Regexp#match? behaving inconsistently with Ruby 3.3.0 https://bugs.ruby-lang.org/issues/20083 * Author: jussikos (Jussi Koljonen) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- From irb, when calling String#match? ``` pattern = /([\s]*ABC)$/i p "1ABC".match?(pattern) # => true p "12ABC".match?(pattern) # => true p "123ABC".match?(pattern) # => true p "1231ABC".match?(pattern) # => true p "12312ABC".match?(pattern) # => false p "123123ABC".match?(pattern) # => false p "1231231ABC".match?(pattern) # => true p "12312312ABC".match?(pattern) # => true p "123123123ABC".match?(pattern) # => false p "1231231231ABC".match?(pattern) # => false p "12312312312ABC".match?(pattern) # => true p "123123123123ABC".match?(pattern) # => true p "1231231231231ABC".match?(pattern) # => false ``` With `ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22]` and earlier versions (2.7.8 to 3.2.2) return value is always `true` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115904] [Ruby master Bug#20086] Windows Memory Mapped Buffer is buggy.
by ioquatix (Samuel Williams) 29 Jan '24

29 Jan '24

Issue #20086 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20086: Windows Memory Mapped Buffer is buggy. https://bugs.ruby-lang.org/issues/20086 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED ---------------------------------------- We confirmed a bug in memory mapped files on Windows. See https://github.com/ruby/ruby/pull/9358 for the fix. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:115749] [Ruby master Feature#20066] Reduce Implicit Array/Hash Allocations For Method Calls Involving Splats
by jeremyevans0 (Jeremy Evans) 24 Jan '24

24 Jan '24

Issue #20066 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Feature #20066: Reduce Implicit Array/Hash Allocations For Method Calls Involving Splats https://bugs.ruby-lang.org/issues/20066 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal ---------------------------------------- I have submitted a pull request (https://github.com/ruby/ruby/pull/9247) to reduce implicit array and hash allocations for method calls involving splats. The following optimizations are included: VM_CALL_ARGS_SPLAT_MUT callinfo flag This is similar to the VM_CALL_KW_SPLAT_MUT flag added in Ruby 3.0. This makes it so if the caller-side allocates an array for the method call, the flag is used to signal to the callee that it can reuse the allocated array and does not need to duplicate it. concattoarray VM instruction This instruction is similar to concatarray, but assumes the object being concatenated to is already a mutable array (such as those created by the splatarray VM instruction). This optimizes method calls with multiple splats such as `f(*a,*a,*a)` (which previously allocated 3 arrays), allocating a single array instead of an array per splatted array. pushtoarray VM instruction This is similar, but handles non-splat arguments after a splat. Previously, the VM would wrap those arguments in an array using newarray, and then call concatarray, such that `f(*a, a)` allocated 3 arrays caller-side. This instruction just appends to the mutable array, reducing the number of arrays allocated to 1. Allocationless Anonymous Splat Forwarding This allows `def f(*, **) end` to not allocate an array or hash callee side. This works because it is not possible to mutate the local variables, only pass them as splats to other methods. This can make the following call chain allocation less: ```ruby def f(a, b: 1) end def g(*, **) f(*, **) end ea a = [1] kw = {b: 2} g(*a, **kw) # No allocations in this call ``` Switch ... argument forwards to not use ruby2_keywords Using ruby2_keywords has probably been slower since Koichi's changes early in the Ruby 3.3 development cycle to not combine keyword splats into the positional splat array. This removes the FORWARD_ARGS_WITH_RUBY2_KEYWORDS define, so that `def f(...) end` operates similarly to `def f(*, **) end`, allowing allocationless splat forwarding Reduce array and hash allocations for nested argument forwarding calls This uses a combination of frame flags and callinfo flags to track mutability of anonymous splat variables. It can make it so the following call example only allocates a 1 array and 1 hash: ```ruby def m1(*args, **kw) end def m2(...) m1(...) end def m3(*, **) m2(*, **) end m3(1, a: 1) # 1 array and 1 hash allocated ``` In the above example, the call to `m3` allocates an array (`[1]`) and a hash (`{a: 1}`), but the call to `m2` passes them as mutable splats, `m2` treats them as mutable splats when calling `m1`, and `m1` reuses the array that `m3` allocated for `args` and the hash that `m3` allocated for `kw`. I created a benchmark for all of these changes. In the method calls optimized by these changes, it is significantly faster: ``` named_multi_arg_splat after: 5344097.6 i/s before: 3088134.0 i/s - 1.73x slower named_post_splat after: 5401882.3 i/s before: 2629321.8 i/s - 2.05x slower anon_arg_splat after: 12242780.9 i/s before: 6845413.2 i/s - 1.79x slower anon_arg_kw_splat after: 11277398.7 i/s before: 4329509.4 i/s - 2.60x slower anon_multi_arg_splat after: 5132699.5 i/s before: 3018103.7 i/s - 1.70x slower anon_post_splat after: 5602915.1 i/s before: 2645185.5 i/s - 2.12x slower anon_kw_splat after: 15403727.3 i/s before: 6249504.6 i/s - 2.46x slower anon_fw_to_named_splat after: 2985715.3 i/s before: 2049159.9 i/s - 1.46x slower anon_fw_to_named_no_splat after: 2941030.4 i/s before: 2100380.0 i/s - 1.40x slower fw_to_named_splat after: 2801008.7 i/s before: 2012416.4 i/s - 1.39x slower fw_to_named_no_splat after: 2742670.4 i/s before: 1957707.2 i/s - 1.40x slower fw_to_anon_to_named_splat after: 2309246.6 i/s before: 1375924.6 i/s - 1.68x slower fw_to_anon_to_named_no_splat after: 2193227.6 i/s before: 1351184.1 i/s - 1.62x slower ``` Only fallout from these changes: * Minor change to AST output for `...` not using `ruby2_keywords` * Prism and rbs need updating for `...` not using `ruby2_keywords` * typeprof need updating for new VM instructions (at least pushtoarray) VM_CALL_ARGS_SPLAT_MUT, concattoarray, and pushtoarray only affect uncommon callsites (multiple splats, argument after splat). Other commits only optimize calls to methods using anonymous splats or `...` argument forwarding. Previously, there was no performance reason to use anonymous splats or `...` argument forwarding, but with this change, using them can be faster, and can offer a new way for users to optimize their code. In my opinion, this is too late for consideration in Ruby 3.3, but it could be considered for Ruby 3.4. -- https://bugs.ruby-lang.org/

4 18

[ruby-core:114997] [Ruby master Bug#19918] Should `a[&b]=c` be syntax valid?
by tompng (tomoya ishida) 19 Jan '24

19 Jan '24

Issue #19918 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19918: Should `a[&b]=c` be syntax valid? https://bugs.ruby-lang.org/issues/19918 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-10-11T04:46:58Z master 40ab7b8c24) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- These codes are syntax valid now. Prism parses it as syntax error. ~~~ruby a[&b]=c a[&b]+=c a[&b]&&=c a[&b]||=c ~~~ Is this syntax intentional or should be error? Issue of Prism https://github.com/ruby/prism/issues/1636 It's added in test_parse.rb https://github.com/ruby/ruby/blob/40ab7b8c244de20007cb45846f41de3a01f7ea0c/… -- https://bugs.ruby-lang.org/

5 6

[ruby-core:115594] [Ruby master Bug#20042] ObjectSpace finalizer can cause segfault
by ioquatix (Samuel Williams) 17 Jan '24

17 Jan '24

Issue #20042 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20042: ObjectSpace finalizer can cause segfault https://bugs.ruby-lang.org/issues/20042 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby class Finalizer def call # <- missing (id) argument end end object = Object.new ObjectSpace.define_finalizer(object, Finalizer.new) object = nil GC.start # segfaults here ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:115830] [Ruby master Misc#20075] DevMeeting-2024-01-17
by mame (Yusuke Endoh) 17 Jan '24

17 Jan '24

Issue #20075 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #20075: DevMeeting-2024-01-17 https://bugs.ruby-lang.org/issues/20075 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2024/01/17 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2024/01/14. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

9 8