November 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:115259] [Ruby master Bug#19990] Could we reconsider the second argument to Kernel#load?
by fxn (Xavier Noria) 07 Nov '23

07 Nov '23

Issue #19990 has been reported by fxn (Xavier Noria). ---------------------------------------- Bug #19990: Could we reconsider the second argument to Kernel#load? https://bugs.ruby-lang.org/issues/19990 * Author: fxn (Xavier Noria) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The documentation of `Kernel#load` says: > If the optional wrap parameter is true, the loaded script will be executed under an anonymous module, protecting the calling program’s global namespace. If the optional wrap parameter is a module, the loaded script will be executed under the given module. In no circumstance will any local variables in the loaded file be propagated to the loading environment. I'd like to ask you to reconsider this feature. First of all, "protecting the calling program" is not really accomplished because the loaded file may still do this ```ruby class ::C # defines ::C regardless of the second argument end ``` Another example, if the caller defines a module `M`, then the loaded program can also define things in `M`: ```ruby class M::C # defines ::M::C regardless of the second argument end ``` It does not even need a leading `::`. So, the "protection" is not really there. In addition to that, this is not transparent for the code being loaded either. For example, let's take this program: ```ruby class A end module X ::A # could be needed if your own lookup had an A somewhere you want to skip end ``` the Ruby programmer expects that to work. But with this feature, on paper, nobody knows if it wil work. How can you ship code confidently? So, the documentation should say: > It kind of protects, but not really. Also, the loaded file may not work as expected, or may not even be laodable at all. That hypothetical documentation suggests to me it would be worth revisiting this feature. In Ruby, as it is today, things are global. The language does not have features to really isolate code as containers do, for example. I believe the 2nd argument to `Kernel#load` steers the API in a direction that is not consistent with the language, and provides a feature that is only partial and cannot satisfy what it promises. I'd be in favor of deprecating and eventually remove this API. -- https://bugs.ruby-lang.org/

4 8

[ruby-core:115033] [Ruby master Misc#19925] DevMeeting-2023-11-07
by mame (Yusuke Endoh) 06 Nov '23

06 Nov '23

Issue #19925 has been reported by mame (Yusuke Endoh). ---------------------------------------- Misc #19925: DevMeeting-2023-11-07 https://bugs.ruby-lang.org/issues/19925 * Author: mame (Yusuke Endoh) * Status: Open * Priority: Normal ---------------------------------------- # The next dev meeting **Date: 2023/11/07 13:00-17:00** (JST) Log: *TBD* - Dev meeting *IS NOT* a decision-making place. All decisions should be done at the bug tracker. - Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly. - Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue). - We will write a record of the discussion in the file or to each ticket in English. - All activities are best-effort (keep in mind that most of us are volunteer developers). - The date, time and place of the meeting are scheduled according to when/where we can reserve Matz's time. - *DO NOT* discuss then on this ticket, please. # Call for agenda items If you have a ticket that you want matz and committers to discuss, please post it into this ticket in the following format: ``` * [Ticket ref] Ticket title (your name) * Comment (A summary of the ticket, why you put this ticket here, what point should be discussed, etc.) ``` Example: ``` * [Feature #14609] `Kernel#p` without args shows the receiver (ko1) * I feel this feature is very useful and some people say :+1: so let discuss this feature. ``` - It is recommended to add a comment by 2023/11/04. We hold a preparatory meeting to create an agenda a few days before the dev-meeting. - The format is strict. We'll use [this script to automatically create an markdown-style agenda](https://gist.github.com/mame/b0390509ce1491b43610b9ebb665eb86). We may ignore a comment that does not follow the format. - Your comment is mandatory. We cannot read all discussion of the ticket in a limited time. We appreciate it if you could write a short summary and update from a previous discussion. -- https://bugs.ruby-lang.org/

10 11

[ruby-core:115273] [Ruby master Feature#14602] Version of dig that raises error if a key is not present
by sinsoku (Takumi Shotoku) 06 Nov '23

06 Nov '23

Issue #14602 has been updated by sinsoku (Takumi Shotoku). I agree with [#14602#note-23](https://bugs.ruby-lang.org/issues/14602#note-23). I think a fetch-based or dig-based name would be better, since the function can be inferred from the method name. Personally, I feel that **deep_fetch** is a simple and nice name. I often use [deep_dup](https://api.rubyonrails.org/classes/Object.html#method-i-deep_dup) and [deep_merge](https://api.rubyonrails.org/classes/Hash.html#method-i-deep_mer… in Rails apps and are familiar with the method name `deep_*`. I found `fetch_dig` and `dig_fetch` to be a little strange in that they have verbs lined up. I think `dig_strict` is a good name, but I prefer **deep_fetch** because `deep` has fewer characters and is simpler than `strict`. ---------------------------------------- Feature #14602: Version of dig that raises error if a key is not present https://bugs.ruby-lang.org/issues/14602#change-105189 * Author: amcaplan (Ariel Caplan) * Status: Open * Priority: Normal ---------------------------------------- Currently, if I have a hash like this: ~~~ ruby { :name => { :first => "Ariel", :last => "Caplan" } } ~~~ and I want to navigate confidently and raise a KeyError if something is missing, I can do: ~~~ ruby hash.fetch(:name).fetch(:first) ~~~ Unfortunately, the length of the name, combined with the need to repeat the method name every time, means most programmers are more likely to do this: ~~~ ruby hash[:name][:first] ~~~ which leads to many unexpected errors. The Hash#dig method made it easy to access methods safely from a nested hash; I'd like to have something similar for access without error protection, and I'd think the most natural name would be Hash#dig!. It would work like this: ~~~ ruby hash = { :name => { :first => "Ariel", :last => "Caplan" } } hash.dig!(:name, :first) # => Ariel hash.dig!(:name, :middle) # raises KeyError (key not found: :middle) hash.dig!(:name, :first, :foo) # raises TypeError (String does not have #dig! method) ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115272] [Ruby master Feature#15554] warn/error passing a block to a method which never use a block
by Dan0042 (Daniel DeLorme) 06 Nov '23

06 Nov '23

Issue #15554 has been updated by Dan0042 (Daniel DeLorme). I think this was closed by mistake and should be re-opened. ---------------------------------------- Feature #15554: warn/error passing a block to a method which never use a block https://bugs.ruby-lang.org/issues/15554#change-105188 * Author: ko1 (Koichi Sasada) * Status: Closed * Priority: Normal * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- # Abstract Warn or raise an ArgumentError if block is passed to a method which does not use a block. In other words, detect "block user methods" implicitly and only "block user methods" can accept a block. # Background Sometimes, we pass a block to a method which ignores the passed block accidentally. ``` def my_open(name) open(name) end # user hopes it works as Kernel#open which invokes a block with opened file. my_open(name){|f| important_work_with f } # but simply ignored... ``` To solve this issue, this feature request propose showing warnings or raising an exception on such case. Last developer's meeting, matz proposed `&nil` which declares this method never receive a block. It is explicit, but it is tough to add this `&nil` parameter declaration to all of methods (do you want to add it to `def []=(i, e, &nil)`?). (I agree `&nil` is valuable on some situations) # Spec ## Define "use a block" methods We need to define which method accepts a block and which method does not. * (1) method has a block parameter (`&b`) * (2) method body has `yield' * (3) method body has `super` (ZSUPER in internal terminology) or `super(...)` * (4) method body has singleton method (optional) (1) and (2) is very clear. I need to explain about (3) and (4). (3). `super` (ZSUPER) passes all parameters as arguments. So there is no surprise that which can accept `block`. However `super(...)` also passes a block if no explicit block passing (like `super(){}` or `super(&b)`) are written. I'm not sure we need to continue this strange specification, but to keep compatibility depending this spec, I add this rule. (4). surprisingly, the following code invoke a block: ``` def foo class << Object.new yield end end foo{ p :ok } #=> :ok ``` I'm also not sure we need to keep this spec, but to allow this spec, I added (4) rule. Strictly speaking, it is not required, but we don't keep the link from singleton class ISeq to lexical parent iseq now, so I added it. ## Exceptional cases A method called by `super` doesn`t warn warning even if this method doesn't use a block. The rule (3) can pass blocks easily and there are many methods don`t use a block. So my patch ignores callings by `super`. ## corner cases There are several cases to use block without (1)-(4) rules. ### `Proc.new/proc/lambda` without a block Now it was deprecated in r66772 (commit:9f1fb0a17febc59356d58cef5e98db61a3c03550). Related discussion: [Bug #15539] ### `block_given?` `block_given?` expects block, but I believe we use it with `yield` or a block parameter. If you know the usecase without them, please tell us. ### `yield` in `eval` We can't know `yield` (or (3), (4) rule) in an `eval` evaluating string at calling time. ``` def foo eval('yield`) end foo{} # at calling time, # we can't know the method foo can accept a block or not. ``` So I added a warning to use `yield` in `eval` like that: `test.rb:4: warning: use yield in eval will not be supported in Ruby 3.` Workaround is use a block parameter explicitly. ``` def foo &b eval('b.call') end foo{ p :ok } ``` # Implementation Strategy is: * [compile time] introduce `iseq::has_yield` field and check it if the iseq (or child iseq) contains `yield` (or something) * [calling time] if block is given, check `iseq::has_yield` flag and show warning (or raise an exception) https://gist.github.com/ko1/c9148ad0224bf5befa3cc76ed2220c0b On this patch, now it raises an error to make it easy to detect. It is easy to switch to show the warning. # Evaluation and discussion I tried to avoid ruby's tests. https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786 Here is a patch. There are several patterns to avoid warnings. ## tests for `block_given?`, `Proc.new` (and similar) without block Add a dummy block parameter. It is test-specific issue. ## empty `each` Some tests add `each` methods do not `yield`, like: `def each; end`. Maybe test-specific issue, and adding a dummy block parameter. ## Subtyping / duck typing https://github.com/ruby/ruby/blob/c01a5ee85e2d6a7128cccafb143bfa694284ca87/… This `parse` method doesn't use `yield`, but other sub-type's `parse` methods use. ## `super` with `new` method https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786#file-tests-pat… This method override `Class#new` method and introduce a hook with block (yield a block in this hook code). https://github.com/ruby/ruby/blob/trunk/lib/rubygems/package/tar_writer.rb#… In this method, call `super` and it also passing a block. However, called `initialize` doesn't use a block. ## Change robustness This change reduce robustness for API change. `Delegator` requires to support `__getobj__` for client classes. Now `__getobj__` should accept block but most of `__getobj__` clients do not call given block. https://github.com/ruby/ruby/blob/trunk/lib/delegate.rb#L80 This is because of delegator.rb's API change. https://gist.github.com/ko1/37483e7940cdc4390bf8eb0001883786#file-tests-pat… Nobu says calling block is not required (ignoring a block is no problem) so it is not a bug for delegator client classes. ## Found issues. ``` [ 2945/20449] Rinda::TestRingServer#test_do_reply = 0.00 s 1) Error: Rinda::TestRingServer#test_do_reply: ArgumentError: passing block to the method "with_timeout" (defined at /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:787) is never used. /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:635:in `test_do_reply' [ 2946/20449] Rinda::TestRingServer#test_do_reply_local = 0.00 s 2) Error: Rinda::TestRingServer#test_do_reply_local: ArgumentError: passing block to the method "with_timeout" (defined at /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:787) is never used. /home/ko1/src/ruby/trunk/test/rinda/test_rinda.rb:657:in `test_do_reply_local' [10024/20449] TestGemRequestSetGemDependencyAPI#test_platform_mswin = 0.01 s 3) Error: TestGemRequestSetGemDependencyAPI#test_platform_mswin: ArgumentError: passing block to the method "util_set_arch" (defined at /home/ko1/src/ruby/trunk/lib/rubygems/test_case.rb:1053) is never used. /home/ko1/src/ruby/trunk/test/rubygems/test_gem_request_set_gem_dependency_api.rb:655:in `test_platform_mswin' [10025/20449] TestGemRequestSetGemDependencyAPI#test_platforms = 0.01 s 4) Error: TestGemRequestSetGemDependencyAPI#test_platforms: ArgumentError: passing block to the method "util_set_arch" (defined at /home/ko1/src/ruby/trunk/lib/rubygems/test_case.rb:1053) is never used. /home/ko1/src/ruby/trunk/test/rubygems/test_gem_request_set_gem_dependency_api.rb:711:in `test_platforms' ``` These 4 detection show the problem. `with_timeout` method (used in Rinda test) and `util_set_arch` method (used in Rubygems test) simply ignore the given block. So these tests are simply ignored. I reported them. (https://github.com/rubygems/rubygems/issues/2601) ## raise an error or show a warning? At least, Ruby 2.7 should show warning for this kind of violation with `-w`. How about for Ruby3? -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114072] [Ruby master Bug#19754] `IO::Buffer#get_string` raises unsuitable exception for too large offset
by nobu (Nobuyoshi Nakada) 06 Nov '23

06 Nov '23

Issue #19754 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #19754: `IO::Buffer#get_string` raises unsuitable exception for too large offset https://bugs.ruby-lang.org/issues/19754 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The message seems suitable when both of `offset` and `length` are given. ```ruby IO::Buffer.for("test").get_string(5, 0) #=> Specified offset+length exceeds buffer size! (ArgumentError) ``` But without `length`, it doesn't seem to check `offset` properly. ```ruby IO::Buffer.for("test").get_string(5) #=> negative string size (or size too big) (ArgumentError) ``` -- https://bugs.ruby-lang.org/

4 6

[ruby-core:115262] [Ruby master Bug#18991] False LocalJumpError when branch coverage is enabled
by usa (Usaku NAKAMURA) 06 Nov '23

06 Nov '23

Issue #18991 has been updated by usa (Usaku NAKAMURA). Backport changed from 3.0: UNKNOWN, 3.1: REQUIRED, 3.2: DONTNEED to 3.0: UNKNOWN, 3.1: DONE, 3.2: DONTNEED ruby_3_1 d494cf4ddababb80660381e963f910ccacc3f7bc merged revision(s) 4a7d6c2852aa734506be83c932168e8f974687b5. ---------------------------------------- Bug #18991: False LocalJumpError when branch coverage is enabled https://bugs.ruby-lang.org/issues/18991#change-105178 * Author: qnighy (Masaki Hara) * Status: Closed * Priority: Normal * Assignee: mame (Yusuke Endoh) * ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin21] * Backport: 3.0: UNKNOWN, 3.1: DONE, 3.2: DONTNEED ---------------------------------------- Enabling branch coverage leads to a false LocalJumpError where it should not be raised. ```ruby # test.rb require "coverage" Coverage.start(branches: true) # Coverage.start(lines: true) load "./test2.rb" ``` ```ruby # test2.rb 1&.tap do break end ``` Output: ``` $ ruby test.rb /Users/qnighy/workdir/branch-coverage-bug/test2.rb:1:in `block in <top (required)>': break from proc-closure (LocalJumpError) from <internal:kernel>:90:in `tap' from /Users/qnighy/workdir/branch-coverage-bug/test2.rb:1:in `<top (required)>' from test.rb:5:in `load' from test.rb:5:in `<main>' ``` -- https://bugs.ruby-lang.org/

1 0

[ruby-core:115244] [Ruby master Feature#19987] add sample method to Range
by horv77＠protonmail.com (Andras Horvath) 06 Nov '23

06 Nov '23

Issue #19987 has been reported by horv77(a)protonmail.com (Andras Horvath). ---------------------------------------- Feature #19987: add sample method to Range https://bugs.ruby-lang.org/issues/19987 * Author: horv77(a)protonmail.com (Andras Horvath) * Status: Open * Priority: Normal ---------------------------------------- Dear Devs, I'd like to suggest a change. Since the following works: (1..99).first(5) Therefore this one could be logical and useful to work: (1..99).sample(5) Thanks, Andras -- https://bugs.ruby-lang.org/

5 5

[ruby-core:115247] [Ruby master Feature#19988] AI for inner code behavior analysis at runtime
by horv77＠protonmail.com (Andras Horvath) 05 Nov '23

05 Nov '23

Issue #19988 has been reported by horv77(a)protonmail.com (Andras Horvath). ---------------------------------------- Feature #19988: AI for inner code behavior analysis at runtime https://bugs.ruby-lang.org/issues/19988 * Author: horv77(a)protonmail.com (Andras Horvath) * Status: Open * Priority: Normal ---------------------------------------- Dear Devs, I've been using Ruby extensively in my work and sometimes I use AI algos to debug inner workings of complex code loops. I wanted to send you the idea because I believe it is very easy to add the following functionality to Ruby and Ruby could be the first programming language ever from the mainstream ones to apply smart AI decisions internally. It could give Ruby a huge marketing power as well with all its advantages. So what is the idea that I already use sometimes to help my development? When I consider some part of the code, it usually involves manipulating many variables in a complex way. And my very simple idea is to get all variables analyzed automatically with an unsupervised, multivariable anomaly detection algo continuously. So when I run it on the variables inside an inner loop or structure, then the AI might be able to spot weird combinations of variable states that may be important to notify the developer of. The language may give a warning about these states or even jump to the entry using binding.irb. Anomaly detection works like when you have an N dimensional array, then every entry is analyzed and the most weird ones can be "spotted" (by showing their indexes of their positions in the array for instance). That's it. The fastest algo could be used for this. It is ISOLATION FOREST. It runs in quasi linear time. So if the user would enable this option at a particular line of code, then the Ruby interpreter would store all variables in an array on every run and would analyze it with the AD algo from time to time (because it takes time). It wouldn't have to analyze at every run because once the anomalous variables combinations get into the array, then they can be found later at any time. If the variable is not numeric (since all AI algo need numeric inputs), then a "text conversion to numeric vector" could be done. I've got a solution for that too. That is also very sophisticated and would boost Ruby's capabilities a lot IMHO. So a very sophisticated inner semantic AI analysis could take place with this option, which is very straightforward and easy to implement. The same could be applied for the behavior analysis of the memory usage. Many concaptual bugs could be tackled with such methods. It would be straightforward to do that from the interpreter side. I'd gladly provide the solution of the AI part. Thanks. Andras -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115243] CHANGE: add sample method to Range
by horv77 05 Nov '23

05 Nov '23

Dear Devs, I'd like to suggest a change. Since the following works: (1..99).first(5) Therefore this one could be logical and useful to work: (1..99).sample(5) Thanks, Andras

1 0

[ruby-core:115242] [Ruby master Bug#18286] Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64
by nobu (Nobuyoshi Nakada) 05 Nov '23

05 Nov '23

Issue #18286 has been updated by nobu (Nobuyoshi Nakada). Does this help it? https://github.com/ruby/ruby/pull/8708 ---------------------------------------- Bug #18286: Universal arm64/x86_84 binary built on an x86_64 machine segfaults/is killed on arm64 https://bugs.ruby-lang.org/issues/18286#change-105157 * Author: ccaviness (Clay Caviness) * Status: Open * Priority: Normal * ruby -v: 3.0.2 * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- A universal arm64/x86_84 ruby binary for macOS built on a x86_64 machine segfaults/is killed when executed on an arm64 machine. To reproduce: * On an Intel Mac: `git clone https://github.com/ruby/ruby && cd ruby && git checkout v3_0_2 && ./autogen.sh && ./configure --with-arch=arm64,x86_64 && make -j$(sysctl -n hw.ncpu)` * Copy the built `./ruby` binary to an Apple Silicon machine * Attempt to execute it Expected: The universal `ruby` binary works correctly on both devices Actual: The universal `ruby` binary crashes with either `Segmentation fault: 11` or `Killed: 9` (this seems to occur if `arm64e` is used instead of `arm64`). Details: I'm attempting to build a universal Ruby for macOS that will run on both Intel (x86_64) and Apple Silicon (arm64) machines. It seemed initially that this was as easy as adding `--with-arch=arm64,x86_64` to `./configure` would do it, as it produced a `ruby` binary that reports as `Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]` This `ruby` works correctly on the Intel machine I built in on, but does not work when copied to an Apple Silicon device. The reverse, however, seems to work. That is, if I build the universal ruby on an Apple Silicon machine, the `ruby` binary that's built seems to work correctly on both Intel and Apple Silicon machines. Intel: ``` $ ./ruby -v ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [universal.x86_64-darwin21] ``` Apple Silicon: ``` $ ./ruby -v Segmentation fault: 11 $ lldb ./ruby (lldb) target create "./ruby" Current executable set to '/Users/crc/ruby' (arm64). (lldb) run Process 77071 launched: '/Users/crc/ruby' (arm64) Process 77071 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x00000001002176b8 ruby`ruby_vm_special_exception_copy + 16 ruby`ruby_vm_special_exception_copy: -> 0x1002176b8 <+16>: ldr x0, [x0, #0x8] 0x1002176bc <+20>: bl 0x10011fed8 ; rb_class_real 0x1002176c0 <+24>: bl 0x10012070c ; rb_obj_alloc 0x1002176c4 <+28>: mov x20, x0 Target 0: (ruby) stopped. (lldb) ^D ``` I also attempted the same thing with ruby 2.7.4 source, with the same result. -- https://bugs.ruby-lang.org/

1 0