March 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:111557] [Ruby master Bug#19292] Time object's wday, yday, and isdst returns broken value (and so does to_a) when kwarg in: 'UTC' was given
by matsuda (Akira Matsuda) 21 Mar '23

21 Mar '23

Issue #19292 has been reported by matsuda (Akira Matsuda). ---------------------------------------- Bug #19292: Time object's wday, yday, and isdst returns broken value (and so does to_a) when kwarg in: 'UTC' was given https://bugs.ruby-lang.org/issues/19292 * Author: matsuda (Akira Matsuda) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2022-12-30T15:31:50Z master 0bb07e5ba4) +YJIT [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- $ ruby -e "p Time.new(2023, 1, 1, 0, 0, 0, in: 'UTC').wday" 7 $ ruby -e "p Time.new(2023, 1, 1, 0, 0, 0, in: 'UTC').yday" 0 $ ruby -e "p Time.new(2023, 1, 1, 0, 0, 0, in: 'UTC').isdst" -e:1:in `isdst': isdst is not set yet (RuntimeError) from -e:1:in `<main>' $ ruby -e "p Time.new(2023, 1, 1, 0, 0, 0, in: 'UTC').to_a" [0, 0, 0, 1, 1, 2023, 7, 0, true, "UTC"] where the expected `to_a` value should be: [0, 0, 0, 1, 1, 2023, 0, 1, false, "UTC"] This bug seems to be happening on the following conditions: - On any given date and time, regardless of past or future - Either on `Time.new(Integer, Integer, ...)` style or `Time.new(String)` style - Only when `in: 'UTC'` kwarg was given. Other formats like `in: '+0000'` or `in: 0` seems to be OK - On all Ruby versions that accept kwarg `in:` (3.1, 3.2, and 3.3) -- https://bugs.ruby-lang.org/

4 4

[ruby-core:112940] [Ruby master Bug#19539] Heredocs with leading spaces inconsistency
by hurricup (Alexandr Evstigneev) 20 Mar '23

20 Mar '23

Issue #19539 has been reported by hurricup (Alexandr Evstigneev). ---------------------------------------- Bug #19539: Heredocs with leading spaces inconsistency https://bugs.ruby-lang.org/issues/19539 * Author: hurricup (Alexandr Evstigneev) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This is more like technical inconsistency that bothers me. Not sure someone will use this in real life, but still. This works fine: ``` puts <<' ABC' marker with spaces ABC ``` But this is not: ``` puts <<-' ABC' marker with spaces ABC ``` as well as ``` puts <<~' ABC' marker with spaces ABC ``` I would expect it to wok properly, even in case: ``` puts <<-' ABC' marker with spaces ABC ``` -- https://bugs.ruby-lang.org/

1 1

[ruby-core:112457] [Ruby master Feature#19443] Cache `Process.pid`
by byroot (Jean Boussier) 20 Mar '23

20 Mar '23

Issue #19443 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19443: Cache `Process.pid` https://bugs.ruby-lang.org/issues/19443 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- It's not uncommon for database client and similar network libraries to protect themselves from Process.fork by regularly checking Process.pid Until recently most libc would cache `getpid()` so this was a cheap check to make. However as of glibc version 2.25 the PID cache is removed and calls to `getpid()` always invoke the actual system call which significantly degrades the performance of existing applications. The reason glibc removed the cache is that some libraries were bypassing `fork(2)` by issuing system calls themselves, causing stale cache issues. That isn't a concern for Ruby as bypassing MRI's primitive for forking would render the VM unusable, so we can safely cache the PID. An example of the issue: https://github.com/rails/rails/issues/47418 -- https://bugs.ruby-lang.org/

7 15

[ruby-core:112942] [Ruby master Bug#19540] Undefined variable error when combining modifier if with one-line pattern matching
by mehanoid (Oleg Grigorev) 18 Mar '23

18 Mar '23

Issue #19540 has been reported by mehanoid (Oleg Grigorev). ---------------------------------------- Bug #19540: Undefined variable error when combining modifier if with one-line pattern matching https://bugs.ruby-lang.org/issues/19540 * Author: mehanoid (Oleg Grigorev) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- If I try to use modifier `if` with a one-line pattern matching, the variable assigned by the pattern cannot be used inside the expression to the left of the "if": ``` x if [1] in [x] (irb):1:in `<main>': undefined local variable or method `x' for main:Object (NameError) ``` This is very unexpected behavior, because the code looks equivalent to the following: ``` if [1] in [x] x end => 1 ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:112941] [Ruby master Bug#18743] Enumerator#next / peek re-use each others stacktraces
by marcper (Marcelo Pereira) 18 Mar '23

18 Mar '23

Issue #18743 has been updated by marcper (Marcelo Pereira). Hello again. I've modified the patch to also link to the old exception via `cause`. Let me know if it's now a reasonable solution. Thanks, Marcelo ---------------------------------------- Bug #18743: Enumerator#next / peek re-use each others stacktraces https://bugs.ruby-lang.org/issues/18743#change-102465 * Author: sos4nt (Stefan Schüßler) * Status: Open * Priority: Normal * Assignee: ko1 (Koichi Sasada) * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- I encountered an odd behavior. If I rescue the `StopIteration` exception from `peek` and call `next` afterwards: (or vice-versa) ```ruby # enum.rb # 1 # 2 enum = [].each # 3 enum.peek rescue nil # 4 enum.next # 5 ``` it will show the stacktrace from the rescued `peek` call: ``` $ ruby enum.rb enum.rb:4:in `peek': iteration reached an end (StopIteration) from enum.rb:4:in `<main>' ``` Whereas the error should refer to `next` on line number 5. The same happens when calling `peek` after `next` or when having muliple `peek` / `next` calls: ```ruby # enum.rb # 1 # 2 enum = [].each # 3 enum.peek rescue nil # 4 enum.next rescue nil # 5 enum.peek rescue nil # 6 puts "line #{__LINE__}" # 7 enum.next # 8 ``` The stacktrace from the first (rescued) `peek` or `next` call will be shown which doesn't reflect the actual error location: ``` $ ruby enum.rb line 7 enum.rb:4:in `peek': iteration reached an end (StopIteration) from enum.rb:4:in `<main>' ``` This is very confusing when debugging code. ---Files-------------------------------- 01-Recreate-stacktrace-enumerator.patch (1.29 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112634] [Ruby master Bug#19469] Crash when resizing generic iv list
by peterzhu2118 (Peter Zhu) 18 Mar '23

18 Mar '23

Issue #19469 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19469: Crash when resizing generic iv list https://bugs.ruby-lang.org/issues/19469 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/7407 The following script can sometimes trigger a crash: ```ruby GC.stress = true class Array def foo(bool) if bool @a = 1 @b = 2 @c = 1 else @c = 1 end end end obj = [] obj.foo(true) obj2 = [] obj2.foo(false) obj3 = [] obj3.foo(true) ``` This is because vm_setivar_default calls rb_ensure_generic_iv_list_size to resize the iv list. However, the call to gen_ivtbl_resize reallocs the iv list, and then inserts into the generic iv table. If the st_insert triggers a GC then the old iv list will be read during marking, causing a use-after-free bug. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:112630] [Ruby master Bug#19468] Ruby 3.2: net/http sets UTF-8 encoding for binary responses
by romuloceccon 17 Mar '23

17 Mar '23

Issue #19468 has been reported by romuloceccon (Rômulo Ceccon). ---------------------------------------- Bug #19468: Ruby 3.2: net/http sets UTF-8 encoding for binary responses https://bugs.ruby-lang.org/issues/19468 * Author: romuloceccon (Rômulo Ceccon) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-22 revision 65ab2c1ef2) [x86_64-linux] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- net/http on Ruby 3.2 has changed the encoding of binary responses from SSL connected hosts (non-SSL connections are not affected): ``` ruby # req.rb require 'openssl' require 'net/http' puts "openssl ext: #{OpenSSL::VERSION}" puts "openssl lib: #{OpenSSL::OPENSSL_VERSION}" puts "net-protocol: #{Net::Protocol::VERSION}" puts "net-http: #{Net::HTTP::VERSION}" puts Net::HTTP.get(URI(ARGV.first)).encoding ``` Ruby 3.1 (with updated net-protocol and net-http libs): ``` $ ruby -v ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux] $ ruby req.rb https://www.gnu.org/software/gzip/manual/gzip.pdf openssl ext: 3.0.0 openssl lib: OpenSSL 1.1.1n 15 Mar 2022 net-protocol: 0.2.1 net-http: 0.3.2 ASCII-8BIT # <== CORRECT ``` Ruby 3.2 (latest git revision): ``` $ ruby -v ruby 3.2.1 (2023-02-22 revision 65ab2c1ef2) [x86_64-linux] $ ruby req.rb https://www.gnu.org/software/gzip/manual/gzip.pdf openssl ext: 3.1.0 openssl lib: OpenSSL 1.1.1n 15 Mar 2022 net-protocol: 0.2.1 net-http: 0.3.2 UTF-8 # <== WRONG ``` I've tracked the problem down to the SSL socket call at https://github.com/ruby/ruby/blob/9557c8edf2dcf18fdece066c596a71696b2f2b30/…. The string returned has the encoding set to `ASCII-8BIT`, but `#ascii_only?` also always reports true, even when there are non-ascii bytes. This seems to be a bug, and is the probably cause of the change in behavior in net/http. On Ruby 3.1 concatenating the result of reading the SSL socket to a UTF-8 string produces an ASCII-8BIT string. On Ruby 3.2 the concatenation produces a UTF-8 string. Here's a program demonstrating the behavior of the SSL socket: ```ruby # ssltest.rb require 'openssl' require 'uri' url = URI(ARGV.first) path = url.path path += '?' + url.query if url.query req = "GET #{path} HTTP/1.1\r\nHost: #{url.hostname}\r\nAccept: */*\r\n\r\n" sock = OpenSSL::SSL::SSLSocket.open(url.hostname, url.port || HTTPS.default_https_port) sock.connect sock.write(req) sleep(1) loop do sleep(0.1) b = ''.b r = sock.read_nonblock(1024 * 16, b, exception: false) break unless String === r p [r.bytesize, r.encoding.to_s, r.ascii_only?] end ``` Ruby 3.1: ``` $ ruby ssltest.rb https://www.gnu.org/software/gzip/manual/gzip.pdf [475, "ASCII-8BIT", true] [16384, "ASCII-8BIT", false] # <== always false (except HTTP header): CORRECT [16384, "ASCII-8BIT", false] ... [13927, "ASCII-8BIT", false] ``` Ruby 3.2: ``` $ ruby ssltest.rb https://www.gnu.org/software/gzip/manual/gzip.pdf [475, "ASCII-8BIT", true] [16384, "ASCII-8BIT", true] # <== always true: WRONG [16384, "ASCII-8BIT", true] ... [13927, "ASCII-8BIT", true] ``` -- https://bugs.ruby-lang.org/

3 4

[ruby-core:112924] [Ruby master Bug#18580] Range#include? inconsistency for beginless String ranges
by mame (Yusuke Endoh) 17 Mar '23

17 Mar '23

Issue #18580 has been updated by mame (Yusuke Endoh). Unfortunately, this change broke an existing program: #19533. The lesson here is that it would be best to avoid changing any behavior just for the sake of consistency unless it is proven that the behavior actually confuses a large number of people. ---------------------------------------- Bug #18580: Range#include? inconsistency for beginless String ranges https://bugs.ruby-lang.org/issues/18580#change-102445 * Author: zverok (Victor Shepelev) * Status: Closed * Priority: Normal * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- The follow-up of #18577. `Range#include?` is [specialized for strings](https://github.com/ruby/ruby/blob/master/range.c#L1782). For all I can tell, this behavior is relatively new: it emerged when switching `#===` to use `#cover?` instead of `#include?`: * in 2.6, it was decided that String Ranges should continue to use `#include?` (#14575) * ...but in 2.7, it was decided that they should use `#cover?` as all others (#15449) The "despecialization" in 2.7 was actually done by adding more specialization in `range_include_internal`. This leads to the following: ```ruby # default Range behavior: (..Date.today).include? Date.today # (irb):10:in `each': can't iterate from NilClass (TypeError) # String Range behavior: (..'z').include?('w') # => true ``` Why I think **it is bad**: 1. This is a leftover of the relatively recent change; why it is 3 versions old, I doubt there is a lot of code relying on this quirk (only beginless ranges are affected) 2. The more "invisible specialization", the more possibility of bugs (as #18577 shows) 3. It is easy to stumble upon while learning Ruby/experimenting (strings and ranges are the most basic objects to be seen in many examples and courses), and to become confused, because there is no reasonable explanation for the semantics other than "well... for String, it is so" 4. It *can* be said that the String ranges behavior have the same specialization as Numeric ones, but it is not so; while Numeric Ranges specialization is long-living legacy, it is at least consistent (`#include?` just behaves like `#cover?` for them, always): Numerics: ```ruby (1...3).include? 1.5 # => true, like #cover? (...3).include? 1.5 # => true, same, the explanation is "just like #cover?" ``` Strings: ```ruby ('a'..'z').include?('ww') # => false, like #include? for any other type (..'z').include?('ww') # => true, this is deeply confusing ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112583] [Ruby master Bug#19464] YJIT miscompiles `BasicObject#send` to alias methods of `send`
by alanwu (Alan Wu) 17 Mar '23

17 Mar '23

Issue #19464 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #19464: YJIT miscompiles `BasicObject#send` to alias methods of `send` https://bugs.ruby-lang.org/issues/19464 * Author: alanwu (Alan Wu) * Status: Open * Priority: Normal * Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- Symptoms first experienced and [reported] by users of the `kt-paperclip` gem. Reproducer: ```ruby klass = Class.new do class << self alias_method :my_send, :send def bar = :ok def foo = bar end end with_break = -> { break klass.send(:my_send, :foo) } wo_break = -> { klass.send(:my_send, :foo) } 31.times { with_break[]; wo_break[] } ``` Will send a fix for review shortly. [reported]: https://github.com/Shopify/yjit/issues/306 -- https://bugs.ruby-lang.org/

2 1

[ruby-core:112920] [Ruby master Feature#18951] Object#with to set and restore attributes around a block
by dsisnero (Dominic Sisneros) 17 Mar '23

17 Mar '23

Issue #18951 has been updated by dsisnero (Dominic Sisneros). temp_set might be a better method name ---------------------------------------- Feature #18951: Object#with to set and restore attributes around a block https://bugs.ruby-lang.org/issues/18951#change-102440 * Author: byroot (Jean Boussier) * Status: Rejected * Priority: Normal ---------------------------------------- ### Use case A very common pattern in Ruby, especially in testing is to save the value of an attribute, set a new value, and then restore the old value in an `ensure` clause. e.g. in unit tests ```ruby def test_something_when_enabled enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true # test things ensure SomeLibrary.enabled = enabled_was end ``` Or sometime in actual APIs: ```ruby def with_something_enabled enabled_was = @enabled @enabled = true yield ensure @enabled = enabled_was end ``` There is no inherent problem with this pattern, but it can be easy to make a mistake, for instance the unit test example: ```ruby def test_something_when_enabled some_call_that_may_raise enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true # test things ensure SomeLibrary.enabled = enabled_was end ``` In the above if `some_call_that_may_raise` actually raises, `SomeLibrary.enabled` is set back to `nil` rather than its original value. I've seen this mistake quite frequently. ### Proposal I think it would be very useful to have a method on Object to implement this pattern in a correct and easy to use way. The naive Ruby implementation would be: ```ruby class Object def with(**attributes) old_values = {} attributes.each_key do |key| old_values[key] = public_send(key) end begin attributes.each do |key, value| public_send("#{key}=", value) end yield ensure old_values.each do |key, old_value| public_send("#{key}=", old_value) end end end end ``` NB: `public_send` is used because I don't think such method should be usable if the accessors are private. With usage: ```ruby def test_something_when_enabled SomeLibrary.with(enabled: true) do # test things end end ``` ```ruby GC.with(measure_total_time: true, auto_compact: false) do # do something end ``` ### Alternate names and signatures If `#with` isn't good, I can also think of: - `Object#set` - `Object#apply` But the `with_` prefix is by far the most used one when implementing methods that follow this pattern. Also if accepting a Hash is dimmed too much, alternative signatures could be: - `Object#set(attr_name, value)` - `Object#set(attr1, value1, [attr2, value2], ...)` # Some real world code example that could be simplified with method - `redis-client` `with_timeout` https://github.com/redis-rb/redis-client/blob/23a5c1e2ff688518904f206df8d4a… - Lots of tests in Rails's codebase: - Changing `Thread.report_on_exception`: https://github.com/rails/rails/blob/2d2fdc941e7497ca77f99ce5ad404b6e58f043e… - Changing a class attribute: https://github.com/rails/rails/blob/2d2fdc941e7497ca77f99ce5ad404b6e58f043e… -- https://bugs.ruby-lang.org/

1 0