November 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119827] [Ruby master Feature#15554] warn/error passing a block to a method which never use a block
by Dan0042 (Daniel DeLorme) 07 Nov '24

07 Nov '24

Issue #15554 has been updated by Dan0042 (Daniel DeLorme). Now that `strict_unused_block` has been added as a warning category, I have to ask what is a "warning category" exactly? Normally I would consider a "category" to be a grouping of similar items. All warnings related to performance go into the "performance" category, etc. But this new "category" only toggles a single warning. It's not even used for the "given block not used" warnings in array.c Are we going to gradually shift to one category per warning message? That's closer to the idea of #issue brought up in the context of Exceptions in #20024, not really a "category". But it's not something I would oppose; currently we build strings for warning messages then discard them after a regexp match... that's a weird way of doing things. So what is supposed to be my mental model of "warning categories" now? ---------------------------------------- Feature #15554: warn/error passing a block to a method which never use a block https://bugs.ruby-lang.org/issues/15554#change-110515 * Author: ko1 (Koichi Sasada) * Status: Closed * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- # Abstract Warn or raise an ArgumentError if block is passed to a method which does not use a block. In other words, detect "block user methods" implicitly and only "block user methods" can accept a block. # Background Sometimes, we pass a block to a method which ignores the passed block accidentally. ``` def my_open(name) open(name) end # user hopes it works as Kernel#open which invokes a block with opened file. my_open(name){|f| important_work_with f } # but simply ignored... ``` To solve this issue, this feature request propose showing warnings or raising an exception on such case. Last developer's meeting, matz proposed `&nil` which declares this method never receive a block. It is explicit, but it is tough to add this `&nil` parameter declaration to all of methods (do you want to add it to `def []=(i, e, &nil)`?). (I agree `&nil` is valuable on some situations) # Spec ## Define "use a block" methods We need to define which method accepts a block and which method does not. * (1) method has a block parameter (`&b`) * (2) method body has `yield' * (3) method body has `super` (ZSUPER in internal terminology) or `super(...)` * (4) method body has singleton method (optional) (1) and (2) is very clear. I need to explain about (3) and (4). (3). `super` (ZSUPER) passes all parameters as arguments. So there is no surprise that which can accept `block`. However `super(...)` also passes a block if no explicit block passing (like `super(){}` or `super(&b)`) are written. I'm not sure we need to continue this strange specification, but to keep compatibility depending this spec, I add this rule. (4). surprisingly, the following code invoke a block: ``` def foo class << Object.new yield end end foo{ p :ok } #=> :ok ``` I'm also not sure we need to keep this spec, but to allow this spec, I added (4) rule. Strictly speaking, it is not required, but we don't keep the link from singleton class ISeq to lexical parent iseq now, so I added it. ## Exceptional cases A method called by `super` doesn`t warn warning even if this method doesn't use a block. The rule (3) can pass blocks easily and there are many methods don`t use a block. So my patch ignores callings by `super`. ## corner cases There are several cases to use block without (1)-(4) rules. ### `Proc.new/proc/lambda` without a block Now it was deprecated in r66772 (commit:9f1fb0a17febc59356d58cef5e98db61a3c03550). Related discussion: [Bug #15539] ### `block_given?` `block_given?` expects block, but I believe we use it with `yield` or a block parameter. If you know the usecase without them, please tell us. ### `yield` in `eval` We can't know `yield` (or (3), (4) rule) in an `eval` evaluating string at calling time. ``` def foo eval('yield`) end foo{} # at calling time, # we can't know the method foo can accept a block or not. ``` So I added a warning to use `yield` in `eval` like that: `test.rb:4: warning: use yield in eval will not be supported in Ruby 3.` Workaround is use a block parameter explicitly. ``` def foo &b eval('b.call') end foo{ p :ok } ``` # Implementation Strategy is: * [compile time] introduce `iseq::has_yield` field and check it if the iseq (or child iseq) contains `yield` (or something) * [calling time] if block is given, check `iseq::has_yield` flag and show warning (or raise an exception) https://gist.github.com/ko1/c9148ad0224bf5befa3cc76ed2220c0b On this patch, now it raises an error to make it easy to detect. It is easy to switch to show the warning. # Evaluation and discussion I tried to avoid ruby's tests. https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786 Here is a patch. There are several patterns to avoid warnings. ## tests for `block_given?`, `Proc.new` (and similar) without block Add a dummy block parameter. It is test-specific issue. ## empty `each` Some tests add `each` methods do not `yield`, like: `def each; end`. Maybe test-specific issue, and adding a dummy block parameter. ## Subtyping / duck typing https://github.com/ruby/ruby/blob/c01a5ee85e2d6a7128cccafb143bfa694284ca87/… This `parse` method doesn't use `yield`, but other sub-type's `parse` methods use. ## `super` with `new` method https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786#file-tests-pat… This method override `Class#new` method and introduce a hook with block (yield a block in this hook code). https://github.com/ruby/ruby/blob/trunk/lib/rubygems/package/tar_writer.rb#… In this method, call `super` and it also passing a block. However, called `initialize` doesn't use a block. ## Change robustness This change reduce robustness for API change. `Delegator` requires to support `__getobj__` for client classes. Now `__getobj__` should accept block but most of `__getobj__` clients do not call given block. https://github.com/ruby/ruby/blob/trunk/lib/delegate.rb#L80 This is because of delegator.rb's API change. https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786#file-tests-pat… Nobu says calling block is not required (ignoring a block is no problem) so it is not a bug for delegator client classes. ## Found issues. ``` [ 2945/20449] Rinda::TestRingServer#test_do_reply = 0.00 s 1) Error: Rinda::TestRingServer#test_do_reply: ArgumentError: passing block to the method "with_timeout" (defined at /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:787) is never used. /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:635:in `test_do_reply' [ 2946/20449] Rinda::TestRingServer#test_do_reply_local = 0.00 s 2) Error: Rinda::TestRingServer#test_do_reply_local: ArgumentError: passing block to the method "with_timeout" (defined at /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:787) is never used. /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:657:in `test_do_reply_local' [10024/20449] TestGemRequestSetGemDependencyAPI#test_platform_mswin = 0.01 s 3) Error: TestGemRequestSetGemDependencyAPI#test_platform_mswin: ArgumentError: passing block to the method "util_set_arch" (defined at /home/ko1/src/ruby/trunk/lib/rubygems/test_case.rb:1053) is never used. /home/ko1/src/ruby/trunk/test/rubygems/test_gem_request_set_gem_dependency_api.rb:655:in `test_platform_mswin' [10025/20449] TestGemRequestSetGemDependencyAPI#test_platforms = 0.01 s 4) Error: TestGemRequestSetGemDependencyAPI#test_platforms: ArgumentError: passing block to the method "util_set_arch" (defined at /home/ko1/src/ruby/trunk/lib/rubygems/test_case.rb:1053) is never used. /home/ko1/src/ruby/trunk/test/rubygems/test_gem_request_set_gem_dependency_api.rb:711:in `test_platforms' ``` These 4 detection show the problem. `with_timeout` method (used in Rinda test) and `util_set_arch` method (used in Rubygems test) simply ignore the given block. So these tests are simply ignored. I reported them. (https://github.com/rubygems/rubygems/issues/2601) ## raise an error or show a warning? At least, Ruby 2.7 should show warning for this kind of violation with `-w`. How about for Ruby3? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:119724] [Ruby master Bug#20863] `zlib.c` calls `rb_str_set_len` and `rb_str_modify_expand`(and others) without holding the GVL.
by ioquatix (Samuel Williams) 07 Nov '24

07 Nov '24

Issue #20863 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20863: `zlib.c` calls `rb_str_set_len` and `rb_str_modify_expand`(and others) without holding the GVL. https://bugs.ruby-lang.org/issues/20863 * Author: ioquatix (Samuel Williams) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I was investigating some problems with `zlib.c` and GVL, and noticed that it releases the GVL, but still calls various `rb_` functions. According to my understanding, this is invalid. I propose the following fix: https://github.com/ruby/zlib/pull/88 In addition, I think we should add more checks for this, in other words, we should add something like this to functions which are expected to only be called while the GVL is held: ```c RUBY_ASSERT(ruby_thread_has_gvl_p()) ``` I made a simple PR to demonstrate the change: https://github.com/ruby/ruby/pull/11975 While the current implementation is internal only (`ruby_thread_has_gvl_p` is not public interface), I think we should add this interface as a public macro, e.g. `RUBY_DEBUG_ASSERT_GVL` or something like that. This macro should be added to all methods that require the GVL in order to diagnose these issues. -- https://bugs.ruby-lang.org/

4 7

[ruby-core:119822] [Ruby master Bug#20880] Hash allows array-type key duplicates
by alexfrstnberg (Yana Prystopchuk) 07 Nov '24

07 Nov '24

Issue #20880 has been reported by alexfrstnberg (Yana Prystopchuk). ---------------------------------------- Bug #20880: Hash allows array-type key duplicates https://bugs.ruby-lang.org/issues/20880 * Author: alexfrstnberg (Yana Prystopchuk) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I was solving small tasks on Leetcode and encountered an issue with Hash which looked strange to me: There's an array saved into the variable and a hash with a previously created array as a key ``` ruby arr = [1,2,3] hsh = {arr => 1} ``` ``` shell => {[1, 2, 3]=>1} ``` Modified the array. Added the same array to the same hash. ``` ruby arr << 4 hsh[arr] = 2 ``` ``` shell => {[1, 2, 3, 4]=>1, [1, 2, 3, 4]=>2} ``` Added another duplicate ``` ruby arr << 5 hsh[arr] = 3 ``` ``` shell => {[1, 2, 3, 4, 5]=>1, [1, 2, 3, 4, 5]=>2, [1, 2, 3, 4, 5]=>3} ``` ### First issue: As a user, I'd expect to have the already existing key reassigned with the new value. Since the key is duplicated, now it's not obvious which value *1*, *2*, or *3* should be returned for the key (*[1, 2, 3, 4, 5]* or *arr*). ### Second issue: When trying to actually retrieve a value, different methods behave differently: ``` ruby hsh[[1,2,3,4,5]] ``` ``` shell => 3 ``` ``` ruby hsh[arr] ``` ``` shell => 3 ``` **But** ``` ruby hsh.assoc(arr) ``` ``` shell => 1 ``` ### Third issue: Another example is created: ``` ruby arr = [1] hsh = {arr => 1} ``` And modified with the same logic using a loop: ``` ruby incr = 1 10.times do arr << (incr += 1) hsh[arr] = incr end ``` The received hsh: ``` shell => {[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>9, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>2, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>3, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>4, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>5, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>6, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>7, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>8, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>10, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]=>11} ``` As you can see, the first key eventually got reassigned (there's no value 1 anymore, it was replaced by 9) I tried to add each next key manually and got the same result: 1 is replaced by 9. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119575] [Ruby master Bug#20808] Cannot override Data#inspect
by maicolben (Maicol Bentancor) 07 Nov '24

07 Nov '24

Issue #20808 has been reported by maicolben (Maicol Bentancor). ---------------------------------------- Bug #20808: Cannot override Data#inspect https://bugs.ruby-lang.org/issues/20808 * Author: maicolben (Maicol Bentancor) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Given the next code: ``` Dog = Data.define(:name) do def inspect "Hello!" end private attr_reader :name end Dog.new(name: "Fido") ``` It throws an error: - An error occurred when inspecting the object: #<NoMethodError: private method `name' called for an instance of Dog> And isn't using my `inspect` function -- https://bugs.ruby-lang.org/

3 7

[ruby-core:119650] [Ruby master Bug#20857] Don't change `Hash#inspect formatting`
by vo.x (Vit Ondruch) 07 Nov '24

07 Nov '24

Issue #20857 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20857: Don't change `Hash#inspect formatting` https://bugs.ruby-lang.org/issues/20857 * Author: vo.x (Vit Ondruch) * Status: Open * ruby -v: ruby 3.4.0dev (2024-10-15 master 3da3cabf98) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Testing Ruby 3.4 in Fedora, I just hit issue like this in AsciiDoctor test suite: ~~~ 1) Failure: Minitest::Test::TestExtensions::TestIntegration#test_should_assign_captures_correctly_for_inline_macros [test/extensions_test.rb:1382]: --- expected +++ actual @@ -1,9 +1,9 @@ "target=\"\", attributes={} -target=\"value,key=val\", attributes={1=>\"value\", \"key\"=>\"val\", \"name\"=>\"value\"} -target=\"\", attributes={\"text\"=>\"\"} -target=\"[text]\", attributes={\"text\"=>\"[text]\"} +target=\"value,key=val\", attributes={1 => \"value\", \"key\" => \"val\", \"name\" => \"value\"} +target=\"\", attributes={\"text\" => \"\"} +target=\"[text]\", attributes={\"text\" => \"[text]\"} target=\"target\", attributes={} -target=\"target\", attributes={1=>\"value\", \"key\"=>\"val\", \"name\"=>\"value\"} -target=\"target\", attributes={\"text\"=>\"\"} -target=\"target\", attributes={\"text\"=>\"[text]\"} +target=\"target\", attributes={1 => \"value\", \"key\" => \"val\", \"name\" => \"value\"} +target=\"target\", attributes={\"text\" => \"\"} +target=\"target\", attributes={\"text\" => \"[text]\"} target=\"target\", attributes={}" ~~~ This suggest that `Hash#inspect` formatting was changed and there are additional spaces around hash rocket. Is the space really worth of the troubles? BTW [here](https://github.com/asciidoctor/asciidoctor/issues/4634#issuecomment-2… is AsciiDoctor upstream reaction and I share the sentiment. -- https://bugs.ruby-lang.org/

8 16

[ruby-core:119636] [Ruby master Feature#20852] Anonymous HEREDOC blocks
by bradgessler (Brad Gessler) 07 Nov '24

07 Nov '24

Issue #20852 has been reported by bradgessler (Brad Gessler). ---------------------------------------- Feature #20852: Anonymous HEREDOC blocks https://bugs.ruby-lang.org/issues/20852 * Author: bradgessler (Brad Gessler) * Status: Open ---------------------------------------- Consider the following code. ```ruby # Consider the following code. Markdown.render <<~MARKDOWN # Hello there This is a Markdown file. See? 1. This is a list 2. With items 3. And more items MARKDOWN ``` There's a lot of "Markdown" that's repeated. ```ruby # Or this SQL.execute <<~SQL SELECT * FROM users WHERE id = 1; SQL ``` It's fine. It works. The key feature is that it understands the text in that block is indented. Not requiring the name could make the syntax less redundant and free developers up from having to name the string. Here's what that could look like: ```ruby # Now with 75% less markdown! Markdown.render <<~ # Hello there This is a Markdown file. See? 1. This is a list 2. With items 3. And more items ~>> # We can't sell SQL symbols fast enough! SQL.execute <<~ SELECT * FROM users WHERE id = 1; ~>> ``` I implement a lot of my application UI in Phlex. The dream is that I can build pages out that look like this: ```ruby class Form < ApplicationForm def view_template input(type: "text", name: "name", placeholder: "Name") fieldset{ h2 { "Agreement" } article(class: "prose"){ markdown <<~ ## Terms and Conditions By submitting this form, you agree to the terms and conditions of this website. # Privacy Policy We will not share your information with anyone. ... ~>> } } button{ "Submit" } end end ``` The closest I can get to that today is if I do this: ```ruby SQL.execute <<~_ SELECT * FROM users WHERE id = 1; _ ``` I've proposed a similar feature a while back and chimed in on similar proposals, but after thinking about it for a bit I realized "Anonymous HEREDOC blocks" is a slightly different approach that could be a different possible path forward. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:119502] [Ruby master Feature#20793] Allow Multiple Arguments for the .is_a? Method
by artemb (Artem Borodkin) 07 Nov '24

07 Nov '24

Issue #20793 has been reported by artemb (Artem Borodkin). ---------------------------------------- Feature #20793: Allow Multiple Arguments for the .is_a? Method https://bugs.ruby-lang.org/issues/20793 * Author: artemb (Artem Borodkin) * Status: Open ---------------------------------------- I propose allowing multiple arguments to be passed to the .is_a? Method imply "OR" semantics: ``` ruby name.is_a? String, Symbol ``` Currently, we need to write the following to achieve the same functionality: ``` ruby [String, Symbol].include?(name.class) ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:119469] [Ruby master Feature#20786] Flow chaining with "then" keyword
by lpogic 07 Nov '24

07 Nov '24

Issue #20786 has been reported by lpogic (Łukasz Pomietło). ---------------------------------------- Feature #20786: Flow chaining with "then" keyword https://bugs.ruby-lang.org/issues/20786 * Author: lpogic (Łukasz Pomietło) * Status: Open ---------------------------------------- Hi, I would like to propose using the *"then"* keyword to create chained flows. ### Background: Original idea: https://bugs.ruby-lang.org/issues/20770#change-110056 However, the concept has changed slightly since then. I now assume that rhs is only calculated when lhs is true, which seems more compatible with the current usage of *"then"* keyword. I would also like to move the discussion here as it may cause unnecessary chaos there. ### Basics: In the canonical version, *"then"* is used in a *"begin..end"* block to split it into steps. When the result of the last expression before *"then"* is not *"false"* or *"nil"*, it is passed as an argument to the next step. Otherwise, all subsequent steps are skipped. The result of the *"begin..end"* block is the result of the last expression, or *"false"*/*"nil"* if any step was skipped. ### Example: ``` ruby # original code source: https://github.com/ruby/ruby/blob/a6383fbe1628bdaa9ec6a30b3baa60dd7430b461/… def include?(other) other = coerce_other(other) return false unless other.family == family begin_addr <= other.begin_addr && end_addr >= other.end_addr end # with chained flow def include?(other) begin coerce_other(other) then => coerced # then => name is my proposition of argument naming syntax return false unless coerced.family == family coerced then => it begin_addr <= it.begin_addr && end_addr >= it.end_addr end end ``` Please treat the canonical form as a starting point. As further improvements I would see: - implicit block argument name - implicit begin - beginless *"then"*, when lhs is just expression ### Example with futher improvements: ``` ruby def include?(other) coerce_other(other) then it.family == family ? it : false then begin_addr <= it.begin_addr && end_addr >= it.end_addr end # or def include?(other) coerce_other other then return false unless it.family == family begin_addr <= it.begin_addr && end_addr >= it.end_addr end end ``` ### *"then"* keyword as the pipeline operator: Under certain conditions, "then" could offer similar capabilities to the pipeline operator of functional languages: ``` ruby # assuming that "foo" and "bar" never return "false" or "nil" # instead of writing this: baz("string", bar(foo(), 2), 5) # you could write this: begin foo() then bar(it, 2) then baz("string", it, 5) end ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:119670] [Ruby master Feature#20859] Make Base64 to core class
by hsbt (Hiroshi SHIBATA) 07 Nov '24

07 Nov '24

Issue #20859 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Feature #20859: Make Base64 to core class https://bugs.ruby-lang.org/issues/20859 * Author: hsbt (Hiroshi SHIBATA) * Status: Assigned * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- From https://bugs.ruby-lang.org/issues/20857#note-12 I also heard that `base64` gem has some issue for eco-system like https://github.com/ddnexus/pagy/pull/618. My fist intention is simply added `base64` gem into gem dependency by `Gem::Specification#add_dependency`. But some people rewrite `Base64.encode64` to `Array#pack`. It means gemification of `base64` is miss-direciton by me. I withdraw that and propose to make `Base64` into core class. https://github.com/ruby/ruby/pull/11977 It resolve dependency issues of `base64` as the default gems. TODO: * To release new version of `base64` gem that skip to load if `Base64` is available. * make `require "base64"` same as `require "enumlator"` -- https://bugs.ruby-lang.org/

6 7

[ruby-core:119050] [Ruby master Feature#20715] `Ractor.[]` and `Ractor.[]=` to access Ractor local storage
by ko1 (Koichi Sasada) 07 Nov '24

07 Nov '24

Issue #20715 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Feature #20715: `Ractor.[]` and `Ractor.[]=` to access Ractor local storage https://bugs.ruby-lang.org/issues/20715 * Author: ko1 (Koichi Sasada) * Status: Open * Assignee: ko1 (Koichi Sasada) ---------------------------------------- Trivial proposal. Now `Ractor#[]/#[]=` is supported to access current ractor local storage. However, it doesn't allow to access local storage of other ractors. ```ruby Ractor.current[:foo] = 1 Ractor.new{ p Ractor.main[:foo] #=> nil }.take ``` So providing `Ractor.[]/[]=` is more reasonable. Further more, we don't need to use `Ractor#current` and it is slightly faster. -- https://bugs.ruby-lang.org/

4 3