- ruby-core - ml.ruby-lang.org

[ruby-core:113977] [Ruby master Bug#19743] Ruby's parser seems to ignoring DOT at the end of file just after digits
by tompng (tomoya ishida) 22 Jul '23

22 Jul '23

Issue #19743 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #19743: Ruby's parser seems to ignoring DOT at the end of file just after digits https://bugs.ruby-lang.org/issues/19743 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-06-19T09:28:10Z master 2a80bac9f0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I expect all to be syntax error ~~~ruby eval "puts 1." #=> 1 eval "puts 1. " #=> syntax error eval "puts 1.\n" #=> syntax error ~~~ For ripper, period token is missing ~~~ruby Ripper.tokenize("1.") #=> ["1"] Ripper.tokenize("1. ") #=> ["1", ".", " "] ~~~ -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113380] [Ruby master Bug#19623] Ruby 3.2 MJIT crashes on railsbench
by k0kubun (Takashi Kokubun) 21 Jul '23

21 Jul '23

Issue #19623 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #19623: Ruby 3.2 MJIT crashes on railsbench https://bugs.ruby-lang.org/issues/19623 * Author: k0kubun (Takashi Kokubun) * Status: Closed * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- Ruby 3.2 MJIT crashes on railsbench when you use `--mjit-max-cache=10000`. ``` [2023-04-29T22:49:00.915217 #412381] FATAL -- : [bc56d52f-5930-4307-8af7-3f58a44eb089] [bc56d52f-5930-4307-8af7-3f58a44eb089] NoMethodError (undefined method `[]' for #<Class:Object>): [bc56d52f-5930-4307-8af7-3f58a44eb089] [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/inflector/inflections.rb:65:in `instance' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/inflector/inflections.rb:252:in `inflections' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/inflector/methods.rb:71:in `block in camelize' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/inflector/methods.rb:71:in `sub' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/inflector/methods.rb:71:in `camelize' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/core_ext/string/inflections.rb:94:in `camelize' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/http/request.rb:87:in `controller_class_for' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/http/parameters.rb:100:in `binary_params_for?' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/http/parameters.rb:91:in `set_binary_encoding' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/http/parameters.rb:69:in `path_parameters=' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/journey/router.rb:47:in `block in serve' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/journey/router.rb:32:in `each' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/journey/router.rb:32:in `serve' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/routing/route_set.rb:834:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/tempfile_reaper.rb:15:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/etag.rb:27:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/conditional_get.rb:27:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/head.rb:12:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/http/content_security_policy.rb:18:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/session/abstract/id.rb:266:in `context' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/session/abstract/id.rb:260:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/cookies.rb:654:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/callbacks.rb:101:in `run_callbacks' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/callbacks.rb:26:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] railties (6.0.4) lib/rails/rack/logger.rb:37:in `call_app' [bc56d52f-5930-4307-8af7-3f58a44eb089] railties (6.0.4) lib/rails/rack/logger.rb:26:in `block in call' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/tagged_logging.rb:80:in `block in tagged' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/tagged_logging.rb:28:in `tagged' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/tagged_logging.rb:80:in `tagged' [bc56d52f-5930-4307-8af7-3f58a44eb089] railties (6.0.4) lib/rails/rack/logger.rb:26:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/remote_ip.rb:81:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/request_id.rb:27:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/method_override.rb:24:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/runtime.rb:22:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] activesupport (6.0.4) lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/executor.rb:14:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] rack (2.2.6.3) lib/rack/sendfile.rb:110:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] actionpack (6.0.4) lib/action_dispatch/middleware/host_authorization.rb:76:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] railties (6.0.4) lib/rails/engine.rb:527:in `call' [bc56d52f-5930-4307-8af7-3f58a44eb089] benchmarks/railsbench/benchmark.rb:28:in `block (2 levels) in <main>' [bc56d52f-5930-4307-8af7-3f58a44eb089] benchmarks/railsbench/benchmark.rb:25:in `each' [bc56d52f-5930-4307-8af7-3f58a44eb089] benchmarks/railsbench/benchmark.rb:25:in `block in <main>' [bc56d52f-5930-4307-8af7-3f58a44eb089] /home/k0kubun/src/github.com/Shopify/yjit-bench/harness/harness.rb:29:in `block in run_benchmark' [bc56d52f-5930-4307-8af7-3f58a44eb089] /opt/rubies/mjit/lib/ruby/3.2.0+3/benchmark.rb:311:in `realtime' [bc56d52f-5930-4307-8af7-3f58a44eb089] /home/k0kubun/src/github.com/Shopify/yjit-bench/harness/harness.rb:29:in `run_benchmark' [bc56d52f-5930-4307-8af7-3f58a44eb089] benchmarks/railsbench/benchmark.rb:24:in `<main>' ``` This PR https://github.com/ruby/ruby/pull/7774 fixes the problem. This ticket is to request a backport for Ruby 3.2. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113382] [Ruby master Bug#19625] Ruby 3.2 MJIT never triggers JIT compaction
by k0kubun (Takashi Kokubun) 21 Jul '23

21 Jul '23

Issue #19625 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Bug #19625: Ruby 3.2 MJIT never triggers JIT compaction https://bugs.ruby-lang.org/issues/19625 * Author: k0kubun (Takashi Kokubun) * Status: Closed * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED ---------------------------------------- [This patch](https://github.com/ruby/ruby/pull/6900) accidentally removed a condition to trigger JIT compaction from Ruby 3.2. This PR https://github.com/ruby/ruby/pull/7777 makes it work again. This ticket is to request a backport for Ruby 3.2. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114255] [Ruby master Bug#19025] Ripper cannot parse syntax ok code that has numbered parameters
by nagachika (Tomoyuki Chikanaga) 21 Jul '23

21 Jul '23

Issue #19025 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED to 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONE ruby_3_2 465eb7418d7ed91f5f0c75da77765c7f5ef8354f merged revision(s) 91c004885fc75a93cadf0094fa86ec3bd0ec25f5. ---------------------------------------- Bug #19025: Ripper cannot parse syntax ok code that has numbered parameters https://bugs.ruby-lang.org/issues/19025#change-103944 * Author: tompng (tomoya ishida) * Status: Closed * Priority: Normal * ruby -v: ruby 3.2.0dev (2022-09-22T02:42:57Z :detached: 830b2e217b) [x86_64-linux] * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONE ---------------------------------------- Ruby says `p { a = 0; [_1, _1 **2] }` is syntax ok, Ripper says syntax error ~~~ruby code = 'p { a = 0; [_1, _1 **2] }' eval(code) #=> nil (Syntax OK) Ripper.sexp(code) #=> nil (Syntax Error) ~~~ Other similar codes. maybe not a bug ~~~ruby p { a = 0; [a **2] } # Syntax OK p { a = 0; [_1 **2] } # Syntax Error p { a = 0; [a, _1 **2] } # Syntax Error p { a = 0; [_1, _1 **2] } # Syntax OK, Ripper.sexp says Syntax Error ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112906] [Ruby master Bug#19531] ObjectSpace::WeakMap: replaced values still clear the key they were assigned to
by byroot (Jean Boussier) 21 Jul '23

21 Jul '23

Issue #19531 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #19531: ObjectSpace::WeakMap: replaced values still clear the key they were assigned to https://bugs.ruby-lang.org/issues/19531 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Backport: 2.7: WONTFIX, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED ---------------------------------------- ### Reproduction script ```ruby wmap = ObjectSpace::WeakMap.new a = "A" b = "B" wmap[1] = a wmap[1] = b # the table entry with 1 is still in the list of entries to clear when `a` is GCed a = nil GC.start p wmap[1] # Should be `"B"`, but is `nil` ``` ### Explanation What happens is that when we set `wmap[1] = "A"`, WeakMap internally keeps a list of keys to clear when `"A"` is GCed, e.g. pseudo code: ```ruby class WeakMap def []=(key, value) @hash[key] = value @reverse[value] << key end end ``` But it doesn't clear previously kept mapping when a key is overwritten. I'll work on a fix. ### References https://github.com/protocolbuffers/protobuf/pull/12216 -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114252] [Ruby master Feature#19780] Remove tailcall_optimization support
by k0kubun (Takashi Kokubun) 21 Jul '23

21 Jul '23

Issue #19780 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Feature #19780: Remove tailcall_optimization support https://bugs.ruby-lang.org/issues/19780 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal ---------------------------------------- ## Proposal Remove `tailcall_optimization` flag from `RubyVM::InstructionSequence` ## Motivation The consensus at [Feature #6602] seems to be that we're not going to enable tailcall optimization by default until we introduce syntax for it. Until this syntax is introduced, this feature is useless. Can we remove this flag until we do support that future syntax? ## Background This script doesn't work with `--yjit-call-threshold=1` on any YJIT-enabled build. ```rb src = <<-EOS def apply_one_and_two(&block) [1, p(1)] end def add_one_and_two apply_one_and_two(&:+) end EOS RubyVM::InstructionSequence.new( "proc {|_|_.class_eval {#{src}}}", __FILE__, __FILE__, 1, tailcall_optimization: true, trace_instruction: false, ).eval[self.singleton_class] def entry add_one_and_two end entry ``` I already wasted a lot of time just for noticing it's already broken. I don't want to waste time for users that do not exist yet. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114241] [Ruby master Bug#19776] Warn bundled gems when it called from `require`
by hsbt (Hiroshi SHIBATA) 21 Jul '23

21 Jul '23

Issue #19776 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19776: Warn bundled gems when it called from `require` https://bugs.ruby-lang.org/issues/19776 * Author: hsbt (Hiroshi SHIBATA) * Status: Assigned * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- We should warn when users try to load default gems that will be promoted bundled gems at next version. We will warn them only under the bundler environment without Gemfile. * Ruby 3.3: * Warn for adding bundled gems to be addressed Ruby 3.4 to Gemfile when user load its gem without `gem 'foo'` in their Gemfile. * Target libraries are TBD * Also warn existing bundled gems was loaded without `gem 'foo'` of Gemfile. Ex. net-smtp, rexml etc. * Ruby 3.4: * Promote bundled gems. * Raise LoadError same as current behavior with warnings of Ruby 3.3. * Warn same as Ruby 3.3. I implemented PoC for this: https://github.com/ruby/ruby/pull/8096 -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114074] [Ruby master Feature#19755] Module#class_eval and Binding#eval use caller location by default
by byroot (Jean Boussier) 20 Jul '23

20 Jul '23

Issue #19755 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19755: Module#class_eval and Binding#eval use caller location by default https://bugs.ruby-lang.org/issues/19755 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- ### Background In Ruby we're very reliant on `Method#source_location` as well as `caller_locations` to locate source code. However, code generated via `Binding#eval`, `Module#class_eval` etc defeat this if called without a location: ```ruby Foo.class_eval <<~RUBY def bar end RUBY p Foo.instance_method(:bar).source_location # => ["(eval)", 1] ``` The overwhelming majority of open source code properly pass a `filename` and `lineno`, however a small minority doesn't and make locating the code much harder than it needs to be. Here's some example of anonymous eval uses I fixed in the past: - https://github.com/ruby/mutex_m/pull/11 - https://github.com/rails/execjs/pull/120 - https://github.com/jnunemaker/httparty/pull/776 - https://github.com/SiftScience/sift-ruby/pull/76 - https://github.com/activemerchant/active_merchant/pull/4675 - https://github.com/rails/thor/pull/807 - https://github.com/dry-rb/dry-initializer/pull/104 - https://github.com/rmosolgo/graphql-ruby/pull/4288 ### Proposal I suggest that instead of defaulting to `"(eval)"`, the optional `filename` argument of the eval family of methods should instead default to: `"(eval in #{caller_locations(1, 1).first.path})"` and `lineno` to `caller_locations(1, 1).first.lineno`. Which can pretty much be monkey patched as this: ```ruby module ModuleEval def class_eval(code, location = "(eval in #{caller_locations(1, 1).first.path})", lineno = caller_locations(1, 1).first.lineno) super end end Module.prepend(ModuleEval) module Foo class_eval <<~RUBY def foo end RUBY end p Foo.instance_method(:foo) ``` before: ``` #<UnboundMethod: Foo#foo() (eval):1> ``` after: ``` #<UnboundMethod: Foo#foo() (eval in /tmp/foo.rb):10> ``` Of course the `lineno` part is likely to not be fully correct, but the reasoning is that it's better than defaulting to 0 or 1. Another possiblity would be to include the caller `lineo` inside the `filename` part, and leave the actual lineo default to `1`: ``` #<UnboundMethod: Foo#foo() (eval at /tmp/foo.rb:10):1> ``` -- https://bugs.ruby-lang.org/

6 14

[ruby-core:111873] [Ruby master Bug#19351] Promote bundled gems at Ruby 3.3
by hsbt (Hiroshi SHIBATA) 19 Jul '23

19 Jul '23

Issue #19351 has been reported by hsbt (Hiroshi SHIBATA). ---------------------------------------- Bug #19351: Promote bundled gems at Ruby 3.3 https://bugs.ruby-lang.org/issues/19351 * Author: hsbt (Hiroshi SHIBATA) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- In Ruby 3.2, the default gems and bundled gems are changed only adding `syntax_suggest`. I and some committers are considering promote default gems to bundled gems again for Ruby 3.3+. We hope to keep the current developer experience with dependency resolution and ignore the additional work like "Put gem "xxx" into your Gemfile" for developers. ### Proposal We propose the following libraries will promote default gems to bundled gems at Ruby 3.3. They are not the dependencies of Rails and RubyGems/Bundler. ``` abbrev getoptlong optparse observable resolv resolv-replace rinda un fcntl nkf syslog win32ole ``` ### Additional works I also propose to poromote rails dependencies: ``` ostruct base64 irb rdoc tsort singleton delegate ``` and gems maintained by @kou ``` csv strscan fiddle stringio ``` But if we promote them to bundled gems, many of users need to add `gem "csv"` into their Gemfile. I'm considering to avoid this situation. Can we the specific feature of bundled gems to RubyGems or Bundler? Example, bundler have allowed list for bundled gems. So, listed gems could be require without Gemfile under the bundle exec. -- https://bugs.ruby-lang.org/

6 25

[ruby-core:114238] [Ruby master Feature#19775] Use the callback `main::const_added`
by UlyssesZhan (Ulysses Zhan) 19 Jul '23

19 Jul '23

Issue #19775 has been reported by UlyssesZhan (Ulysses Zhan). ---------------------------------------- Feature #19775: Use the callback `main::const_added` https://bugs.ruby-lang.org/issues/19775 * Author: UlyssesZhan (Ulysses Zhan) * Status: Open * Priority: Normal ---------------------------------------- In #17881, it was accepted that `Module#const_added` is called when there is a new constant added to the namespace of the module. Since `main` can also serve as a namespace, it should also be able to use this callback. ```ruby def const_added name p name end Foo = 1 # => :Foo ``` -- https://bugs.ruby-lang.org/

2 1