March 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:112223] [Ruby master Bug#19417] Regexp \p{Word} and [[:word:]] do not match Unicode Other_Number character
by ObjectBoxPC (Philip Chung) 24 Mar '23

24 Mar '23

Issue #19417 has been reported by ObjectBoxPC (Philip Chung). ---------------------------------------- Bug #19417: Regexp \p{Word} and [[:word:]] do not match Unicode Other_Number character https://bugs.ruby-lang.org/issues/19417 * Author: ObjectBoxPC (Philip Chung) * Status: Open * Priority: Normal * ruby -v: 3.2.0 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- According to the [documentation for Regexp](https://ruby-doc.org/3.2.0/Regexp.html), `\p{Word}` and `[[:word:]]` both match a character in one of the following Unicode general categories: Letter, Mark, Number, Connector_Punctuation. However, neither matches U+00B2, which is in the Other_Number category (which is a subcategory of Number). ``` ruby puts "Ruby version: %s" % RUBY_VERSION puts "\p{Word} matches? %s" % /\p{Word}/u.match?("\u00B2") puts "[[:word:]] matches? %s" % /[[:word:]]/u.match?("\u00B2") puts "Is a Number charater? %s" % /\p{Number}/u.match?("\u00B2") puts "Is an Other_Number character? %s" % /\p{Other_Number}/u.match?("\u00B2") ``` Expected output: ``` Ruby version: 3.2.0 p{Word} matches? true [[:word:]] matches? true Is a Number charater? true Is an Other_Number character? true ``` Actual output: ``` Ruby version: 3.2.0 p{Word} matches? false [[:word:]] matches? false Is a Number charater? true Is an Other_Number character? true ``` I notice that the [upstream Onigmo library doc](https://github.com/k-takata/Onigmo/blob/master/doc/RE) defines the `[[:word:]]` class as "Letter | Mark | Decimal_Number | Connector_Punctuation", meaning that it only matches certain number characters (which would exclude U+00B2). I am not sure how `\p{Word}` is defined though. But perhaps the documentation needs to be changed? -- https://bugs.ruby-lang.org/

5 4

[ruby-core:112998] [Ruby master Bug#18286] Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64
by ccaviness (Clay Caviness) 24 Mar '23

24 Mar '23

Issue #18286 has been updated by ccaviness (Clay Caviness). Could someone please review and merge Ben's PR https://github.com/ruby/ruby/pull/7367 to fix this? I'd like to see these changes make the next release. ---------------------------------------- Bug #18286: Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64 https://bugs.ruby-lang.org/issues/18286#change-102527 * Author: ccaviness (Clay Caviness) * Status: Open * Priority: Normal * ruby -v: 3.0.2 * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- A universal arm64/x86_84 ruby binary for macOS built on a x86_64 machine segfaults/is killed when executed on an arm64 machine. To reproduce: * On an Intel Mac: `git clone https://github.com/ruby/ruby && cd ruby && git checkout v3_0_2 && ./autogen.sh && ./configure --with-arch=arm64,x86_64 && make -j$(sysctl -n hw.ncpu)` * Copy the built `./ruby` binary to an Apple Silicon machine * Attempt to execute it Expected: The universal `ruby` binary works correctly on both devices Actual: The universal `ruby` binary crashes with either `Segmentation fault: 11` or `Killed: 9` (this seems to occur if `arm64e` is used instead of `arm64`). Details: I'm attempting to build a universal Ruby for macOS that will run on both Intel (x86_64) and Apple Silicon (arm64) machines. It seemed initially that this was as easy as adding `--with-arch=arm64,x86_64` to `./configure` would do it, as it produced a `ruby` binary that reports as `Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]` This `ruby` works correctly on the Intel machine I built in on, but does not work when copied to an Apple Silicon device. The reverse, however, seems to work. That is, if I build the universal ruby on an Apple Silicon machine, the `ruby` binary that's built seems to work correctly on both Intel and Apple Silicon machines. Intel: ``` $ ./ruby -v ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [universal.x86_64-darwin21] ``` Apple Silicon: ``` $ ./ruby -v Segmentation fault: 11 $ lldb ./ruby (lldb) target create "./ruby" Current executable set to '/Users/crc/ruby' (arm64). (lldb) run Process 77071 launched: '/Users/crc/ruby' (arm64) Process 77071 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x00000001002176b8 ruby`ruby_vm_special_exception_copy + 16 ruby`ruby_vm_special_exception_copy: -> 0x1002176b8 <+16>: ldr x0, [x0, #0x8] 0x1002176bc <+20>: bl 0x10011fed8 ; rb_class_real 0x1002176c0 <+24>: bl 0x10012070c ; rb_obj_alloc 0x1002176c4 <+28>: mov x20, x0 Target 0: (ruby) stopped. (lldb) ^D ``` I also attempted the same thing with ruby 2.7.4 source, with the same result. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112997] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by Dan0042 (Daniel DeLorme) 24 Mar '23

24 Mar '23

Issue #18368 has been updated by Dan0042 (Daniel DeLorme). I notice this breaks compatibility for `('a'..'e').step(2).to_a` Why? Why break backward compatibility so pointlessly, when you don't even need to? ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-102526 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112691] [Ruby master Bug#19478] Ruby 3.2.1 fails to build on macOS El Capitan (10.11.6)
by chdiza (Charles Diza) 24 Mar '23

24 Mar '23

Issue #19478 has been reported by chdiza (Charles Diza). ---------------------------------------- Bug #19478: Ruby 3.2.1 fails to build on macOS El Capitan (10.11.6) https://bugs.ruby-lang.org/issues/19478 * Author: chdiza (Charles Diza) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I tried building ruby 3.2.1 from source on macOS 10.11.6. Though I'd built all the way up through ruby 3.1.2 just fine, 3.2.1 chokes with the following: ``` compiling process.c process.c:8364:5: error: unknown type name 'clockid_t'; did you mean 'clock_id_t'? clockid_t c; ^~~~~~~~~ clock_id_t /usr/include/mach/clock_types.h:53:13: note: 'clock_id_t' declared here typedef int clock_id_t; /* clock identification type */ ^ process.c:8590:5: error: unknown type name 'clockid_t'; did you mean 'clock_id_t'? clockid_t c; ^~~~~~~~~ clock_id_t /usr/include/mach/clock_types.h:53:13: note: 'clock_id_t' declared here typedef int clock_id_t; /* clock identification type */ ^ 2 errors generated. make: *** [process.o] Error 1 make: *** Waiting for unfinished jobs.... ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:112994] [Ruby master Feature#18368] Range#step semantics for non-Numeric ranges
by zverok (Victor Shepelev) 24 Mar '23

24 Mar '23

Issue #18368 has been updated by zverok (Victor Shepelev). Can please somebody review the PR?.. I have joined Ukrainian army but still have a bit of time to fix notes and prepare it to merge. In a few weeks it can become impossible. ---------------------------------------- Feature #18368: Range#step semantics for non-Numeric ranges https://bugs.ruby-lang.org/issues/18368#change-102523 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal ---------------------------------------- I am sorry if the question had already been discussed, can't find the relevant topic. "Intuitively", this looks (for me) like a meaningful statement: ```ruby (Time.parse('2021-12-01')..Time.parse('2021-12-24')).step(1.day).to_a # ^^^^^ or just 24*60*60 ``` Unfortunately, it doesn't work with "TypeError (can't iterate from Time)". Initially it looked like a bug for me, but after digging a bit into code/docs, I understood that `Range#step` has an odd semantics of "advance the begin N times with `#succ`, and yield the result", with N being always integer: ```ruby ('a'..'z').step(3).first(5) # => ["a", "d", "g", "j", "m"] ``` The fact that semantic is "odd" is confirmed by the fact that for Float it is redefined to do what I "intuitively" expected: ```ruby (1.0..7.0).step(0.3).first(5) # => [1.0, 1.3, 1.6, 1.9, 2.2] ``` (Like with [`Range#===` some time ago](https://bugs.ruby-lang.org/issues/14575), I believe that to be a strong proof of the wrong generic semantics, if for numbers the semantics needed to be redefined completely.) Another thing to note is that "skip N elements" seem to be rather "generically Enumerable-related" yet it isn't defined on `Enumerable` (because nobody needs this semantics, typically!) Hence, two questions: * Can we redefine generic `Range#step` to new semantics (of using `begin + step` iteratively)? It is hard to imagine the amount of actual usage of the old behavior (with String?.. to what end?) in the wild * If the answer is "no", can we define a new method with new semantics, like, IDK, `Range#over(span)`? **UPD:** More examples of useful behavior (it is NOT only about core `Time` class): ```ruby require 'active_support/all' (1.minute..20.minutes).step(2.minutes).to_a #=> [1 minute, 3 minutes, 5 minutes, 7 minutes, 9 minutes, 11 minutes, 13 minutes, 15 minutes, 17 minutes, 19 minutes] require 'tod' (Tod::TimeOfDay.parse("8am")..Tod::TimeOfDay.parse("10am")).step(30.minutes).to_a #=> [#<Tod::TimeOfDay 08:00:00>, #<Tod::TimeOfDay 08:30:00>, #<Tod::TimeOfDay 09:00:00>, #<Tod::TimeOfDay 09:30:00>, #<Tod::TimeOfDay 10:00:00>] require 'matrix' (Vector[1, 2, 3]..).step(Vector[1, 1, 1]).take(3) #=> [Vector[1, 2, 3], Vector[2, 3, 4], Vector[3, 4, 5]] require 'unitwise' (Unitwise(0, 'km')..Unitwise(1, 'km')).step(Unitwise(100, 'm')).map(&:to_s) #=> ["0 km", "1/10 km", "1/5 km", "3/10 km", "2/5 km", "0.5 km", "3/5 km", "7/10 km", "4/5 km", "9/10 km", "1 km"] ``` **UPD:** Responding to discussion points: **Q:** Matz is concerned that the proposed simple definition will be confusing with the classes where `+` is redefined as concatenation. **A:** I believe that simplicity of semantics and ease of explaining ("it just uses `+` underneath, whatever `+` does, will be performed") will make the confusion minimal. **Q:** Why not introduce new API requirement (like "class of range's `begin` should implement `increment` method, and then it will be used in `step`) **A:** require *every* gem author to change *every* of their objects' behavior. For that, they should be aware of the change, consider it important enough to care, clearly understand the necessary semantics of implementation, have a resource to release a new version... Then all users of all such gems would be required to upgrade. The feature would be DOA (dead-on-arrival). The two alternative ways I am suggesting: change the behavior of `#step` or introduce a new method with desired behavior: 1. Easy to explain and announce 2. Require no other code changes to immediately become useful 3. With something like [backports](https://github.com/marcandre/backports) or [ruby-next](https://github.com/ruby-next/ruby-next) easy to start using even in older Ruby version, making the code more expressive even before it would be possible for some particular app/compny to upgrade to (say) 3.2 All examples of behavior from the code above are real `irb` output with monkey-patched `Range#step`, demonstrating how little change will be needed to code outside of the `Range`. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112987] [Ruby master Bug#4040] SystemStackError with Hash[*a] for Large _a_
by jeremyevans0 (Jeremy Evans) 24 Mar '23

24 Mar '23

Issue #4040 has been updated by jeremyevans0 (Jeremy Evans). Considering we fixed this issue for iseq and cfunc methods, probably the main reason to reject fixing this issue for the remaining method types is that doing so potentially lowers performance. To try to counteract this, I've updated my pull request to include an optimization for bmethods that improves performance 38-59% for simple method calls, and up to 180% for method calls with keywords. Hopefully this increase in bmethod performance more than cancels out any performance decreases caused by the large argument splat handling. ---------------------------------------- Bug #4040: SystemStackError with Hash[*a] for Large _a_ https://bugs.ruby-lang.org/issues/4040#change-102518 * Author: runpaint (Run Paint Run Run) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 1.9.3dev (2010-11-09 trunk 29737) [x86_64-linux] * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- =begin I've been hesitating over whether to file a ticket about this, so please feel free to close if I've made the wrong choice. I often use Hash[*array.flatten] in IRB to convert arrays of arrays into hashes. Today I noticed that if the array is big enough, this would raise a SystemStackError. Puzzled, I looked deeper. I assumed I was hitting the maximum number of arguments a method's argc can hold, but realised that the minimum size of the array needed to trigger this exception differed depending on whether I used IRB or not. So, presumably this is indeed exhausting the stack... In IRB, the following is the minimal reproduction of this problem: Hash[*130648.times.map{ 1 }]; true I haven't looked for the minimum value needed with `ruby -e`, but the following reproduces: ruby -e 'Hash[*1380888.times.map{ 1 }]' I suppose this isn't technically a bug, but maybe it offers another argument for either #666 or an extension of #3131. =end -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112222] [Ruby master Bug#19416] Inconsistent behaviour for Struct.new without any member_names
by herwin (Herwin W) 23 Mar '23

23 Mar '23

Issue #19416 has been reported by herwin (Herwin W). ---------------------------------------- Bug #19416: Inconsistent behaviour for Struct.new without any member_names https://bugs.ruby-lang.org/issues/19416 * Author: herwin (Herwin W) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux-gnu] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When I simply declare a Struct without any arguments, I get an error: ``` irb(main):001:0> Struct.new (irb):1:in `new': wrong number of arguments (given 0, expected 1+) (ArgumentError) from (irb):1:in `<main>' ``` But Struct has an option to pass a class name as the first argument, which will create the struct as a constant in the Struct namespace. If this argument is given, there is no ArgumentError ``` irb(main):002:0> Struct.new('Foo') => Struct::Foo ``` This results in a rather pointless class ``` irb(main):003:0> Struct::Foo.new(1) (irb):3:in `initialize': struct size differs (ArgumentError) from (irb):3:in `new' irb(main):004:0> Struct::Foo.new => #<struct Struct::Foo> ``` This behaviour is not documented in the Struct class, but I would guess this is not how it is intended to be. -- https://bugs.ruby-lang.org/

6 5

[ruby-core:112930] [Ruby master Bug#19536] Frozen status loss when moving objects
by peterzhu2118 (Peter Zhu) 23 Mar '23

23 Mar '23

Issue #19536 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19536: Frozen status loss when moving objects https://bugs.ruby-lang.org/issues/19536 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/7553 When objects are moved between size pools, their frozen status is lost in the shape. This will cause the frozen check to be bypassed when there is an inline cache. For example, the following script should raise a FrozenError, but doesn't on Ruby 3.2 and master. ```ruby class A def add_ivars @a = @b = @c = @d = 1 end def set_a @a = 10 end end a = A.new a.add_ivars a.freeze b = A.new b.add_ivars b.set_a # Set the inline cache in set_a GC.verify_compaction_references(expand_heap: true, toward: :empty) a.set_a ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:112683] [Ruby master Bug#19476] Regexp unexpected match
by andreccosta 22 Mar '23

22 Mar '23

Issue #19476 has been reported by andreccosta (André Costa). ---------------------------------------- Bug #19476: Regexp unexpected match https://bugs.ruby-lang.org/issues/19476 * Author: andreccosta (André Costa) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [arm64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Ruby 3.2.1: ```ruby > RUBY_VERSION => "3.2.1" irb(main):011:0> "123456789".match(/($?\d-?$?){6,15}/) => #<MatchData "123456" 1:"6"> irb(main):012:0> "123456789".match(/(\d-?\)?){6,}/) => #<MatchData "123456" 1:"6"> ``` Ruby 3.1.3: ``` ruby > RUBY_VERSION => "3.1.3" > "123456789".match(/($?\d-?$?){6,15}/) => #<MatchData "123456789" 1:"9"> > "123456789".match(/(\d-?\)?){6,}/) => #<MatchData "123456789" 1:"9"> ``` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:112594] [Ruby master Bug#19467] Some linear_time regexp does not match in linear time
by tompng (tomoya ishida) 22 Mar '23

22 Mar '23

Issue #19467 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19467: Some linear_time regexp does not match in linear time https://bugs.ruby-lang.org/issues/19467 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-02-17T18:10:16Z master c3bae033eb) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- `Regexp.linear_time?(/\A.*a.*x\z/) #=> true` but it does not match in linear time ~~~ruby Regexp.linear_time? /\A(.*a.*)x\z/ #=> true /\A(.*a.*)x\z/ =~ 'a'*100000+'y' #=> nil, processing time: 0.016995s Regexp.linear_time? /\A.*a.*x\z/ #=> true /\A.*a.*x\z/ =~ 'a'*100000+'y' #=> nil, processing time: 21.479335s ~~~ -- https://bugs.ruby-lang.org/

3 2