March 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116347] [Ruby master Bug#20197] Postponed job invocations are significantly reduced in Ruby 3.3
by osyoyu (Daisuke Aritomo) 20 Mar '24

20 Mar '24

Issue #20197 has been reported by osyoyu (Daisuke Aritomo). ---------------------------------------- Bug #20197: Postponed job invocations are significantly reduced in Ruby 3.3 https://bugs.ruby-lang.org/issues/20197 * Author: osyoyu (Daisuke Aritomo) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The number of postponed job invocations has been significantly reduced in Ruby 3.3. While my understanding is that postponed jobs provide no guarantee of how soon registered callbacks will fire, I believe the current rate is too low for practical usage, especially for profilers such as StackProf. A git bisect led me to https://github.com/ruby/ruby/commit/1f0304218cf00e05a4a126196676ba221ebf91f6 which obviously seems to be related, but I'm not sure why. ## Repro ### Expected The job fires (nearly) every 100 ms. ``` % ruby bin/test.rb # runs for 3 seconds count: 1 count: 2 (snip) count: 29 ``` ### Actual The job fires only once. ``` % ruby bin/test.rb count: 1 count: 2 (snip) count: 29 ``` ### Code ```ruby require 'mycext' time = Time.now th = Thread.new do loop do sleep 0.01 break if Time.now - time > 3 # run for 3 seconds end end th.join ``` ```c #include <pthread.h> #include <stdio.h> #include <time.h> #include "ruby.h" #include "ruby/debug.h" int called_count; void postponed_job(void *ptr) { called_count++; printf("count: %d\n", called_count); } _Noreturn void * pthread_main(void *_) { while (1) { rb_postponed_job_register_one(0, postponed_job, NULL); // Sleep for 100 ms struct timespec ts; ts.tv_sec = 0; ts.tv_nsec = 100 * 1000 * 1000; nanosleep(&ts, NULL); } } RUBY_FUNC_EXPORTED void Init_mycext(void) { called_count = 0; pthread_t pthread; pthread_create(&pthread, NULL, pthread_main, NULL); } ``` -- https://bugs.ruby-lang.org/

4 7

[ruby-core:117233] [Ruby master Feature#20349] Pattern Matching - Expose local variable captures
by baweaver (Brandon Weaver) 20 Mar '24

20 Mar '24

Issue #20349 has been reported by baweaver (Brandon Weaver). ---------------------------------------- Feature #20349: Pattern Matching - Expose local variable captures https://bugs.ruby-lang.org/issues/20349 * Author: baweaver (Brandon Weaver) * Status: Open ---------------------------------------- In Regular Expressions we have the ability to utilize `Regexp.last_match` ([link](https://ruby-doc.org/3.2.2/Regexp.html#method-c-last_match)) to access the most recent match data from a regular expression match. I would like to propose that we have similar functionality for pattern matching: ```ruby require "rubocop" def ast_from(string) case string when String then processed_source_from(string).ast when RuboCop::ProcessedSource then string.ast when RuboCop::AST::Node then string else nil end end def longform_block?(node) node in [:block, # s(:block, [:send, target, target_method], # s(:send, s(:array), :select), [[:arg, v]], # s(:args, s(:arg, :v)), [:send, [:lvar, ^v], block_method] # s(:send, s(:lvar, :v), :even?)) ] end longform_block?(ast_from("[1, 2, 3].select { |v| v.even? }")) # => true # Name is not important, idea is, name can be debated. `source` is used to not # show AST nodes PatternMatch.last_match.transform_values(&:source) # => {:node=>"[1, 2, 3].select { |v| v.even? }", # :result=>"[1, 2, 3].select { |v| v.even? }", # :target=>"[1, 2, 3]", # :target_method=>:select, # :v=>:v, # :block_method=>:even?} # Hacky version to show how / where behavior could be captured def longform_block?(node) result = node in [:block, # s(:block, [:send, target, target_method], # s(:send, s(:array), :select), [[:arg, v]], # s(:args, s(:arg, :v)), [:send, [:lvar, ^v], block_method] # s(:send, s(:lvar, :v), :even?)) ] pp(binding.local_variables.to_h { [_1, binding.local_variable_get(_1).then { |s| s.source rescue s }] }) result end ``` -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116037] [Ruby master Bug#20153] Backport 7f9c174102 to fix --yjit-stats with RubyVM::YJIT.enable
by k0kubun (Takashi Kokubun) 20 Mar '24

20 Mar '24

Issue #20153 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #20153: Backport 7f9c174102 to fix --yjit-stats with RubyVM::YJIT.enable https://bugs.ruby-lang.org/issues/20153 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Ruby 3.3.0 ignored --yjit-stats when `RubyVM::YJIT.enable` (no argument) is used, which was an unintended behavior. https://github.com/ruby/ruby/pull/9415 should be backported to ruby_3_3. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:116454] [Ruby master Bug#20214] Backport https://github.com/ruby/ruby/pull/9711 to fix exits on Ruby 3.3's new instruction
by k0kubun (Takashi Kokubun) 20 Mar '24

20 Mar '24

Issue #20214 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #20214: Backport https://github.com/ruby/ruby/pull/9711 to fix exits on Ruby 3.3's new instruction https://bugs.ruby-lang.org/issues/20214 * Author: k0kubun (Takashi Kokubun) * Status: Closed * Priority: Normal * Assignee: naruse (Yui NARUSE) * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Ruby 3.3.0 YJIT missed the support for the instruction that was added shortly before the 3.3.0 release. It's used in Rails, and we didn't mean to exit on such method calls. It'd be nice if we can fix the issue in Ruby 3.3.1 by backporting https://github.com/ruby/ruby/pull/9711. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117227] [Ruby master Bug#20348] Memory leak with method kwargs in ERB template
by davidtaylorhq (David Taylor) 20 Mar '24

20 Mar '24

Issue #20348 has been reported by davidtaylorhq (David Taylor). ---------------------------------------- Bug #20348: Memory leak with method kwargs in ERB template https://bugs.ruby-lang.org/issues/20348 * Author: davidtaylorhq (David Taylor) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Evaluating ERB templates with kwarg method calls leads to unbounded growth in memory use. Example script: ```ruby require "erb" def noop(kwarg1:); end template = ERB.new("<%= noop kwarg1: false %>") 50.times do 10_000.times do template.result(binding) end GC.start; puts "Object Count: #{ObjectSpace.count_objects[:TOTAL]}" end ``` When run, this script will print something like: ``` Object Count: 53825 Object Count: 80432 Object Count: 105811 Object Count: 137331 Object Count: 179491 Object Count: 179491 Object Count: 236389 ... ``` This issue does not occur when the keyword argument is replaced with a positional argument. I can reproduce the issue using Ruby 3.0.6, 3.1.4, 3.2.3 and 3.3.0. The issue does not reproduce using Ruby 2.7.8. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:117247] [Ruby master Feature#6337] FileUtils#sync
by matheusrich (Matheus Richard) 20 Mar '24

20 Mar '24

Issue #6337 has been updated by matheusrich (Matheus Richard). @trans I'm not sure I understand the proposal. Is this basically cloning the source dir with a different name? could that be the name you want `clone_dir`? ---------------------------------------- Feature #6337: FileUtils#sync https://bugs.ruby-lang.org/issues/6337#change-107334 * Author: trans (Thomas Sawyer) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- =begin I'd like to see some sort of FileUtils#sync method to sync the contents of one directory with another. Here is a prototype definition: # # Synchronize a destination directory with a source directory. # def sync(src, dst, options={}) src_files = Dir[File.join(src, '**', '*')].map{ |f| f.sub(src+'/', '') } dst_files = Dir[File.join(dst, '**', '*')].map{ |f| f.sub(dst+'/', '') } removal = dst_files - src_files rm_dirs, rm_files = [], [] removal.each do |f| path = File.join(dst, f) if File.directory?(path) rm_dirs << path else rm_files << path end end rm_files.each { |f| rm(f) } rm_dirs.each { |d| rmdir(d) } src_files.each do |f| src_path = File.join(src, f) dst_path = File.join(dst, f) if File.directory?(src_path) mkdir_p(dst_path) else parent = File.dirname(dst_path) mkdir_p(parent) unless File.directory?(parent) install(src_path, dst_path) end end end I haven't tested this beyond a simple trial. It needs improvement. For one, it probably should be able to handle regular files as arguments too. It also needs to handle verbose and dryrun options properly. Nonetheless, with improvements, would be okay for such to become new feature for FileUtils? =end -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117245] [Ruby master Feature#3731] Easier Embedding API for Ruby
by matheusrich (Matheus Richard) 20 Mar '24

20 Mar '24

Issue #3731 has been updated by matheusrich (Matheus Richard). @mame @ko1, is this still needed given that we have mruby? ---------------------------------------- Feature #3731: Easier Embedding API for Ruby https://bugs.ruby-lang.org/issues/3731#change-107332 * Author: Beoran (Beoran Aegul) * Status: Assigned * Assignee: ko1 (Koichi Sasada) ---------------------------------------- =begin With Ruby 1.9, it has become more difficult to embed Ruby in a C application correctly. It would be nice if there was a clearly documented and simple C API to embed ruby in C programs. I know Ruby was not designed from the start to be embedded, but Ruby was used before in several products as an embedded scripting langauge. It should therefore be possible to do so in a more straightforward way. Kind Regards, B. =end -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117244] [Ruby master Feature#6594] Integrated Functor
by matheusrich (Matheus Richard) 20 Mar '24

20 Mar '24

Issue #6594 has been updated by matheusrich (Matheus Richard). @mame Can we close this issue? ---------------------------------------- Feature #6594: Integrated Functor https://bugs.ruby-lang.org/issues/6594#change-107331 * Author: trans (Thomas Sawyer) * Status: Assigned * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- I know the developers meeting is coming up so I'd like to get a few ideas I've had sitting in the wings out in the air before then. One the more useful is the idea of integrating Functors directly into the language. "Functor" is the term I use for "higher-order function". I blogged about this idea and you can read it here: http://trans.github.com/2011-09-07-ruby-heart-higher-order-functions/ The super short version is this: ~~~ruby def f => op, arg arg.send(__op__, arg) end f + 3 #=> 6 f * 3 #=> 9 ~~~ Another example: ~~~ruby class String def file => op, *args File.send(__op__, self, *args) end end "README.rdoc".file.mtime #=> 2012-06-14 12:34:45 -0400 ~~~ I'm using `=>` as means of indicating a higher-order function. Of course another syntax could be used if this won't fly. The important thing is the idea of higher-order functions being integrated directly into the language. Doing this without that integration requires the creation of an intermediate object for each call which is very inefficient. ---Files-------------------------------- 6594.pdf (77.8 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117243] [Ruby master Feature#8421] add Enumerable#find_map and Enumerable#find_all_map
by alexbarret (Alexandre Barret) 19 Mar '24

19 Mar '24

Issue #8421 has been updated by alexbarret (Alexandre Barret). zverok (Victor Shepelev) wrote in #note-4: > @alexbarret There is a somewhat lesser-known trick which looks pretty close to your code: > ```ruby > # proposal: > find_map(emails) do |email| > (matches = pattern.match(email)) && matches[:identifier] > end > # a "trick" > emails.find { |email| match = pattern.match(email) and break match[:identifier] } > # => "thecode" > ``` > It might even be considered two tricks, depending on your point of view: the control-flow `and` allows to chain any statements to it (note it doesn't need extra parentheses after assignment), and `break value` allows to return a non-standard value from a block. > > Not saying it is beautiful, just one more option. Thanks I learned something, and it makes sense thinking about it. Both `and` and `break` aren't things I'm used to use in Ruby code but I'm glad I know this trick. Not sure I'll use it much. Breaking with another value than the expected value returned from the enumerable method can be confusing. ---------------------------------------- Feature #8421: add Enumerable#find_map and Enumerable#find_all_map https://bugs.ruby-lang.org/issues/8421#change-107330 * Author: Hanmac (Hans Mackowiak) * Status: Feedback ---------------------------------------- currently if you have an Enumerable and you want to return the return value of #find you need eigther: (o = enum.find(block) && block.call(o)) || nil or enum.inject(nil) {|ret,el| ret || block.call(el)} neigher of them may be better than an directly maked method same for #find_all_map enum.lazy.map(&:block).find_all{|el| el} it may work but it is not so good -- https://bugs.ruby-lang.org/

1 0

[ruby-core:117242] [Ruby master Feature#8421] add Enumerable#find_map and Enumerable#find_all_map
by alexbarret (Alexandre Barret) 19 Mar '24

19 Mar '24

Issue #8421 has been updated by alexbarret (Alexandre Barret). jeremyevans0 (Jeremy Evans) wrote in #note-5: > `find_map` seems like a bad name as there is no map. map implies calling the same function over all elements in a collection, and in this case, there would be a single element (or none if nothing was found). Combining `find` and `then` seems like the simplest way now if you don't want to use `break`: This is a good point. I always thought map was the action of transforming an element but it does have a implicit reference to an array or a collection. > > ```ruby > emails.find{ pattern.match(it) }&.then{ it[:identifier] } > ``` > > Personally, I would use the following approach is I think it is clearer: > > ```ruby > if match = emails.find{ pattern.match(it) } > match[:identifier] > end > ``` The code aboce doesn't work ^. `match` variable assigned is an email string not the matches found from the email. We implicitly thought that `find` was returning the matches not the item iterated over. ---------------------------------------- Feature #8421: add Enumerable#find_map and Enumerable#find_all_map https://bugs.ruby-lang.org/issues/8421#change-107329 * Author: Hanmac (Hans Mackowiak) * Status: Feedback ---------------------------------------- currently if you have an Enumerable and you want to return the return value of #find you need eigther: (o = enum.find(block) && block.call(o)) || nil or enum.inject(nil) {|ret,el| ret || block.call(el)} neigher of them may be better than an directly maked method same for #find_all_map enum.lazy.map(&:block).find_all{|el| el} it may work but it is not so good -- https://bugs.ruby-lang.org/

1 0