- ruby-core - ml.ruby-lang.org

[ruby-core:118020] [Ruby master Bug#20511] Update bundled reline gem version to v0.5.7
by Revolt (Iulian Onofrei) 29 May '24

29 May '24

Issue #20511 has been reported by Revolt (Iulian Onofrei). ---------------------------------------- Bug #20511: Update bundled reline gem version to v0.5.7 https://bugs.ruby-lang.org/issues/20511 * Author: Revolt (Iulian Onofrei) * Status: Open * ruby -v: 3.3.0 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The bundled reline gem should be updated to v0.5.7 (https://github.com/ruby/reline/releases/tag/v0.5.7) because the current latest Ruby version is unusable in some situations due to an exception being thrown (https://github.com/ruby/reline/issues/690) -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116311] [Ruby master Bug#20192] YJIT in 3.3.0 miscompiles `yield` with keyword splats
by alanwu (Alan Wu) 28 May '24

28 May '24

Issue #20192 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20192: YJIT in 3.3.0 miscompiles `yield` with keyword splats https://bugs.ruby-lang.org/issues/20192 * Author: alanwu (Alan Wu) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Test with: ```ruby def splat_kw(kwargs) = yield(**kwargs) p splat_kw({}) { _1 } ``` ```shell % ruby --yjit-call-threshold=1 test.rb {} % ruby test.rb nil ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116911] [Ruby master Bug#20294] Parser no longer warns on some duplicated keys
by kddnewton (Kevin Newton) 28 May '24

28 May '24

Issue #20294 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20294: Parser no longer warns on some duplicated keys https://bugs.ruby-lang.org/issues/20294 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Previously, the parser would warn on all duplicated keys. Now some cases are not handled: ```ruby { 100.0 => 1, 1e2 => 1 } { 100.0 => 1, 1E2 => 1 } { 100.0 => 1, 100.00 => 1 } { 100.0r => 1, 100.00r => 1 } { 100.0i => 1, 100.00i => 1 } ``` -- https://bugs.ruby-lang.org/

1 2

[ruby-core:117813] [Ruby master Bug#20479] Ensure line number in begin...end in method
by kddnewton (Kevin Newton) 28 May '24

28 May '24

Issue #20479 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20479: Ensure line number in begin...end in method https://bugs.ruby-lang.org/issues/20479 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- In the following example: ```ruby def foo begin rescue p :rescue ensure p :ensure end end ``` the ISEQ has the label: `#<ISeq:ensure in foo@test.rb:6 (6,4)-(6,13)>`, but tracepoint reports that `first_lineno` is 1. This is coming straight from the AST, which says `@ NODE_ENSURE (id: 12, line: 1, location: (2,7)-(6,13))`. This seems incorrect. The `rescue` knows that its line starts on line 3, but `ensure` says it starts on line 1. Could we change that to reflect the actual line it's on? -- https://bugs.ruby-lang.org/

2 4

[ruby-core:117984] [Ruby master Misc#20503] Dedenting heredoc line continuation
by kddnewton (Kevin Newton) 28 May '24

28 May '24

Issue #20503 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Misc #20503: Dedenting heredoc line continuation https://bugs.ruby-lang.org/issues/20503 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- When there is a line continuation inside a dedenting heredoc, occasionally it will impact the dedent calculation in interesting ways. I'm not sure if it's a bug, or if my understanding is incorrect. ```ruby <<~eos TEXT eos ``` In this case the string is `"TEXT\n", because the common whitespace is 2 and it's removed from the only line. However if there is a line continuation: ```ruby <<~eos \ TEXT eos ``` then the results is `" TEXT\n"`. To me this seems incorrect, because the second line is supposed to be considered as part of the first, which would mean it should have the same result as the first one. So either my understanding is incorrect or this is a bug. Could someone clarify? Thanks! -- https://bugs.ruby-lang.org/

3 6

[ruby-core:117619] [Ruby master Bug#20438] String format "%\n" and "%\0" does not raise format error
by tompng (tomoya ishida) 28 May '24

28 May '24

Issue #20438 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20438: String format "%\n" and "%\0" does not raise format error https://bugs.ruby-lang.org/issues/20438 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT +MN [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `"%" % 1` raises `incomplete format specifier; use %% (double %) instead` `"%=" % 1` raises `malformed format string - %=`. But `"%\n" % 1` `"%\0" % 1` does not raise error. In `sprintf.c`, `\n` and `\0` are explicitly accepted. Is this expected? Some other language are: Perl: Warns `Invalid conversion in printf`. Just prints. `"%d% " → "1% "` Python: Error `ValueError: unsupported format character '?' (0xa)` with `print("%\n" % 123)` PHP: Error `Unknown format specifier` with `sprintf("%\n", 3)` C, C++: Warns `incomplete format specifier`. `"%\n" → "\n"`, `"%" → ""`, `"% " → ""` `sprintf("%f%\n",x)` and `"%f%\n" % x` is used in some codes https://github.com/search?q=language%3ARuby+%22f%25%5Cn%22&type=code -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118037] [Ruby master Bug#20512] Order of magnitude performance differenfce in single character slicing UTF-8 strings before and after length method is executed
by giner (Stanislav German-Evtushenko) 28 May '24

28 May '24

Issue #20512 has been reported by giner (Stanislav German-Evtushenko). ---------------------------------------- Bug #20512: Order of magnitude performance differenfce in single character slicing UTF-8 strings before and after length method is executed https://bugs.ruby-lang.org/issues/20512 * Author: giner (Stanislav German-Evtushenko) * Status: Open * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Slicing of a single character of UTF-8 string becomes ~15 times faster after method "length" is executed on the string. ```ruby # Single byte symbols letters = ("a".."z").to_a length = 100000 str = length.times.map{letters[rand(26)]}.join # Slow start = Time.now length.times{|i| str[i]} puts Time.now - start # 0.169156201 str.length # performance hack # Fast start = Time.now length.times{|i| str[i]} puts Time.now - start # 0.009883919 # UTF-8 Symbols letters = ("а".."я").to_a length = 10000 str = length.times.map{letters[rand(26)]}.join # Slow start = Time.now length.times{|i| str[i]} puts Time.now - start # 0.326204007 str.length # performance hack # Fast start = Time.now length.times{|i| str[i]} puts Time.now - start # 0.016943093 ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117469] [Ruby master Feature#20415] Precompute literal String hash code during compilation
by byroot (Jean Boussier) 28 May '24

28 May '24

Issue #20415 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20415: Precompute literal String hash code during compilation https://bugs.ruby-lang.org/issues/20415 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- I worked on a proof of concept with @etienne which I think has some potential, but I'm looking for feedback on what would be the best implementation. The proof of concept is here: https://github.com/Shopify/ruby/pull/553 ### Idea Most string literals are relatively short, hence embedded, and have some wasted bytes at the end of their slot. We could use that wasted space to store the string hash. The goal being to make **looking up a literal String key in a hash, as fast as a Symbol key**. The goal isn't to memoize the hash code of all strings, but to **only selectively precompute the hash code of literal strings in the compiler**. The compiler could even selectively do this when we literal string is used to lookup a hash (`opt_aref`). Here's the benchmark we used: ```ruby hash = 10.times.to_h do |i| [i, i] end dyn_sym = "dynamic_symbol".to_sym hash[:some_symbol] = 1 hash[dyn_sym] = 1 hash["small"] = 2 hash["frozen_string_literal"] = 2 Benchmark.ips do |x| x.report("symbol") { hash[:some_symbol] } x.report("dyn_symbol") { hash[:some_symbol] } x.report("small_lit") { hash["small"] } x.report("frozen_lit") { hash["frozen_string_literal"] } x.compare!(order: :baseline) end ``` On Ruby 3.3.0, looking up a String key is a bit slower based on the key size: ``` Calculating ------------------------------------- symbol 24.175M (± 1.7%) i/s - 122.002M in 5.048306s dyn_symbol 24.345M (± 1.6%) i/s - 122.019M in 5.013400s small_lit 21.252M (± 2.1%) i/s - 107.744M in 5.072042s frozen_lit 20.095M (± 1.3%) i/s - 100.489M in 5.001681s Comparison: symbol: 24174848.1 i/s dyn_symbol: 24345476.9 i/s - same-ish: difference falls within error small_lit: 21252403.2 i/s - 1.14x slower frozen_lit: 20094766.0 i/s - 1.20x slower ``` With the proof of concept performance is pretty much identical: ``` Calculating ------------------------------------- symbol 23.528M (± 6.9%) i/s - 117.584M in 5.033231s dyn_symbol 23.777M (± 4.7%) i/s - 120.231M in 5.071734s small_lit 23.066M (± 2.9%) i/s - 115.376M in 5.006947s frozen_lit 22.729M (± 1.1%) i/s - 115.693M in 5.090700s Comparison: symbol: 23527823.6 i/s dyn_symbol: 23776757.8 i/s - same-ish: difference falls within error small_lit: 23065535.3 i/s - same-ish: difference falls within error frozen_lit: 22729351.6 i/s - same-ish: difference falls within error ``` ### Possible implementation The reason I'm opening this issue early is to get feedback on which would be the best implementation. #### Store hashcode after the string terminator Right now the proof of concept simply stores the `st_index_t` after the string null terminator, and only when the string is embedded and as enough left over space. Strings with a precomputed hash are marked with an user flag. Pros: - Very simple implementation, no need to change a lot of code, and very easy to strip out if we want to. - Doesn't use any extra memory. If the string doesn't have enough left over bytes, the optimization simply isn't applied. - The worst case overhead is a single `FL_TEST_RAW` in `rb_str_hash`. Cons: - The optimization won't apply to certain string sizes. e.g. strings between `17` and `23` bytes won't have a precomputed hash code. - Extracting the hash code requires some not so nice pointer arithmetic. #### Create another RString union Another possibility would be to add another entry in the `RString` struct union, such as we'd have: ```c struct RString { struct RBasic basic; long len; union { // ... existing members struct { st_index_t hash; char ary[1]; } embded_literal; } as; }; ``` Pros: - The optimization can now be applied to all string sizes. - The hashcode is always at the same offset and properly aligned. Cons: - Some strings would be bumped by one slot size, so would use marginally more memory. - Complexify the code base more, need to modify a lot more string related code (e.g. `RSTRING_PTR` and many others) - When compiling such string, if an equal string already exists in the `fstring` table, we'd need to replace it, we can't just mutate it in place to add the hashcode. ### Prior art [Feature #15331] is somewhat similar in its idea, but it does it lazily for all strings. Here it's much simpler because limited to string literals, which are the ones likely to be used as Hash keys, and the overhead is on compilation, not runtime (aside from a single flag check). So I think most of the caveats of that original implementation don't apply here. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:117928] [Ruby master Feature#20497] Tempfile.create_io
by akr (Akira Tanaka) 28 May '24

28 May '24

Issue #20497 has been reported by akr (Akira Tanaka). ---------------------------------------- Feature #20497: Tempfile.create_io https://bugs.ruby-lang.org/issues/20497 * Author: akr (Akira Tanaka) * Status: Open ---------------------------------------- I propose Tempfile.create_io. It is similar to Tempfile.create but the actual file is unlinked before returning the method. https://github.com/ruby/ruby/pull/10803 Purpose: Sometimes, applications need a temporary file but it is not required to access via a pathname. In this case, the created file can be unlinked just after temporary file creation. This removes the obligation of removing the file from applications. So, Tempfile.create_io is easier to use than Tempfile.create. Example: ``` tmpio = Tempfile.create_io # => #<IO:fd 5> tmpio.class # => IO tmpio.path # => nil tmpio.stat.mode.to_s(8) # => "100600" tmpio.puts "foo" tmpio.rewind tmpio.read # => "foo\n" tmpio.close ``` Portability: This feature (unlink just after file creation) is supported on Unix for a long time. Linux 3.11 has O_TMPFILE to create an unnamed file. The current implementation uses it. I heard that it is possible on Windows. (O_SHARE_DELETE?) I'm not sure how to use it. -- https://bugs.ruby-lang.org/

5 7

[ruby-core:113410] [Ruby master Bug#19631] module_eval does not propulate absolute_path for Kernel.caller_locations
by daveola (David Stellar) 27 May '24

27 May '24

Issue #19631 has been reported by daveola (David Stellar). ---------------------------------------- Bug #19631: module_eval does not propulate absolute_path for Kernel.caller_locations https://bugs.ruby-lang.org/issues/19631 * Author: daveola (David Stellar) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I am using module_eval and noticing that since ruby 3.2 the Kernel locations do not have absolute_path for any of the eval code, though the path is available. This is a regression since at least ruby 3.0 which still works. I am on 3.2.2, and here is some sample code: ---- class Script < Module script = %q{ def self.locations Kernel.caller_locations.each { |loc| puts "LOCATION: #{loc}" puts "ABSPATH: #{loc.absolute_path}" puts "PATH: #{loc.path}" } end self.locations } module_eval(script, "/this/is/my/path", 0) end ------ The output for the code at the top of the caller locations (inside the module_eval) is: LOCATION: /this/is/my/path:9:in `<class:Script>' ABSPATH: PATH: /this/is/my/path But the absolute_path should have the correct path data as well -- https://bugs.ruby-lang.org/

2 1