April 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116311] [Ruby master Bug#20192] YJIT in 3.3.0 miscompiles `yield` with keyword splats
by alanwu (Alan Wu) 28 May '24

28 May '24

Issue #20192 has been reported by alanwu (Alan Wu). ---------------------------------------- Bug #20192: YJIT in 3.3.0 miscompiles `yield` with keyword splats https://bugs.ruby-lang.org/issues/20192 * Author: alanwu (Alan Wu) * Status: Closed * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Test with: ```ruby def splat_kw(kwargs) = yield(**kwargs) p splat_kw({}) { _1 } ``` ```shell % ruby --yjit-call-threshold=1 test.rb {} % ruby test.rb nil ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116911] [Ruby master Bug#20294] Parser no longer warns on some duplicated keys
by kddnewton (Kevin Newton) 28 May '24

28 May '24

Issue #20294 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20294: Parser no longer warns on some duplicated keys https://bugs.ruby-lang.org/issues/20294 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Previously, the parser would warn on all duplicated keys. Now some cases are not handled: ```ruby { 100.0 => 1, 1e2 => 1 } { 100.0 => 1, 1E2 => 1 } { 100.0 => 1, 100.00 => 1 } { 100.0r => 1, 100.00r => 1 } { 100.0i => 1, 100.00i => 1 } ``` -- https://bugs.ruby-lang.org/

1 2

[ruby-core:117619] [Ruby master Bug#20438] String format "%\n" and "%\0" does not raise format error
by tompng (tomoya ishida) 28 May '24

28 May '24

Issue #20438 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20438: String format "%\n" and "%\0" does not raise format error https://bugs.ruby-lang.org/issues/20438 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +YJIT +MN [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `"%" % 1` raises `incomplete format specifier; use %% (double %) instead` `"%=" % 1` raises `malformed format string - %=`. But `"%\n" % 1` `"%\0" % 1` does not raise error. In `sprintf.c`, `\n` and `\0` are explicitly accepted. Is this expected? Some other language are: Perl: Warns `Invalid conversion in printf`. Just prints. `"%d% " → "1% "` Python: Error `ValueError: unsupported format character '?' (0xa)` with `print("%\n" % 123)` PHP: Error `Unknown format specifier` with `sprintf("%\n", 3)` C, C++: Warns `incomplete format specifier`. `"%\n" → "\n"`, `"%" → ""`, `"% " → ""` `sprintf("%f%\n",x)` and `"%f%\n" % x` is used in some codes https://github.com/search?q=language%3ARuby+%22f%25%5Cn%22&type=code -- https://bugs.ruby-lang.org/

2 2

[ruby-core:117469] [Ruby master Feature#20415] Precompute literal String hash code during compilation
by byroot (Jean Boussier) 28 May '24

28 May '24

Issue #20415 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20415: Precompute literal String hash code during compilation https://bugs.ruby-lang.org/issues/20415 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- I worked on a proof of concept with @etienne which I think has some potential, but I'm looking for feedback on what would be the best implementation. The proof of concept is here: https://github.com/Shopify/ruby/pull/553 ### Idea Most string literals are relatively short, hence embedded, and have some wasted bytes at the end of their slot. We could use that wasted space to store the string hash. The goal being to make **looking up a literal String key in a hash, as fast as a Symbol key**. The goal isn't to memoize the hash code of all strings, but to **only selectively precompute the hash code of literal strings in the compiler**. The compiler could even selectively do this when we literal string is used to lookup a hash (`opt_aref`). Here's the benchmark we used: ```ruby hash = 10.times.to_h do |i| [i, i] end dyn_sym = "dynamic_symbol".to_sym hash[:some_symbol] = 1 hash[dyn_sym] = 1 hash["small"] = 2 hash["frozen_string_literal"] = 2 Benchmark.ips do |x| x.report("symbol") { hash[:some_symbol] } x.report("dyn_symbol") { hash[:some_symbol] } x.report("small_lit") { hash["small"] } x.report("frozen_lit") { hash["frozen_string_literal"] } x.compare!(order: :baseline) end ``` On Ruby 3.3.0, looking up a String key is a bit slower based on the key size: ``` Calculating ------------------------------------- symbol 24.175M (± 1.7%) i/s - 122.002M in 5.048306s dyn_symbol 24.345M (± 1.6%) i/s - 122.019M in 5.013400s small_lit 21.252M (± 2.1%) i/s - 107.744M in 5.072042s frozen_lit 20.095M (± 1.3%) i/s - 100.489M in 5.001681s Comparison: symbol: 24174848.1 i/s dyn_symbol: 24345476.9 i/s - same-ish: difference falls within error small_lit: 21252403.2 i/s - 1.14x slower frozen_lit: 20094766.0 i/s - 1.20x slower ``` With the proof of concept performance is pretty much identical: ``` Calculating ------------------------------------- symbol 23.528M (± 6.9%) i/s - 117.584M in 5.033231s dyn_symbol 23.777M (± 4.7%) i/s - 120.231M in 5.071734s small_lit 23.066M (± 2.9%) i/s - 115.376M in 5.006947s frozen_lit 22.729M (± 1.1%) i/s - 115.693M in 5.090700s Comparison: symbol: 23527823.6 i/s dyn_symbol: 23776757.8 i/s - same-ish: difference falls within error small_lit: 23065535.3 i/s - same-ish: difference falls within error frozen_lit: 22729351.6 i/s - same-ish: difference falls within error ``` ### Possible implementation The reason I'm opening this issue early is to get feedback on which would be the best implementation. #### Store hashcode after the string terminator Right now the proof of concept simply stores the `st_index_t` after the string null terminator, and only when the string is embedded and as enough left over space. Strings with a precomputed hash are marked with an user flag. Pros: - Very simple implementation, no need to change a lot of code, and very easy to strip out if we want to. - Doesn't use any extra memory. If the string doesn't have enough left over bytes, the optimization simply isn't applied. - The worst case overhead is a single `FL_TEST_RAW` in `rb_str_hash`. Cons: - The optimization won't apply to certain string sizes. e.g. strings between `17` and `23` bytes won't have a precomputed hash code. - Extracting the hash code requires some not so nice pointer arithmetic. #### Create another RString union Another possibility would be to add another entry in the `RString` struct union, such as we'd have: ```c struct RString { struct RBasic basic; long len; union { // ... existing members struct { st_index_t hash; char ary[1]; } embded_literal; } as; }; ``` Pros: - The optimization can now be applied to all string sizes. - The hashcode is always at the same offset and properly aligned. Cons: - Some strings would be bumped by one slot size, so would use marginally more memory. - Complexify the code base more, need to modify a lot more string related code (e.g. `RSTRING_PTR` and many others) - When compiling such string, if an equal string already exists in the `fstring` table, we'd need to replace it, we can't just mutate it in place to add the hashcode. ### Prior art [Feature #15331] is somewhat similar in its idea, but it does it lazily for all strings. Here it's much simpler because limited to string literals, which are the ones likely to be used as Hash keys, and the overhead is on compilation, not runtime (aside from a single flag check). So I think most of the caveats of that original implementation don't apply here. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:113410] [Ruby master Bug#19631] module_eval does not propulate absolute_path for Kernel.caller_locations
by daveola (David Stellar) 27 May '24

27 May '24

Issue #19631 has been reported by daveola (David Stellar). ---------------------------------------- Bug #19631: module_eval does not propulate absolute_path for Kernel.caller_locations https://bugs.ruby-lang.org/issues/19631 * Author: daveola (David Stellar) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I am using module_eval and noticing that since ruby 3.2 the Kernel locations do not have absolute_path for any of the eval code, though the path is available. This is a regression since at least ruby 3.0 which still works. I am on 3.2.2, and here is some sample code: ---- class Script < Module script = %q{ def self.locations Kernel.caller_locations.each { |loc| puts "LOCATION: #{loc}" puts "ABSPATH: #{loc.absolute_path}" puts "PATH: #{loc.path}" } end self.locations } module_eval(script, "/this/is/my/path", 0) end ------ The output for the code at the top of the caller locations (inside the module_eval) is: LOCATION: /this/is/my/path:9:in `<class:Script>' ABSPATH: PATH: /this/is/my/path But the absolute_path should have the correct path data as well -- https://bugs.ruby-lang.org/

2 1

[ruby-core:117368] [Ruby master Bug#20401] Duplicated when clause warning line number
by kddnewton (Kevin Newton) 24 May '24

24 May '24

Issue #20401 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20401: Duplicated when clause warning line number https://bugs.ruby-lang.org/issues/20401 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When you have a duplicated when clause, you get a warning for it. For example: ```ruby case foo when :bar when :baz when :bar end ``` you get `warning: duplicated `when' clause with line 2 is ignored`. But the when clause that is ignored is the one on line 4, not line 2. It seems like it's warning for the wrong line. -- https://bugs.ruby-lang.org/

2 4

[ruby-core:115014] [Ruby master Bug#19920] Ruby 3.1 fails to build with --enable-shared on macos-arm64: is an incompatible architecture (have 'arm64', need '')
by Eregon (Benoit Daloze) 23 May '24

23 May '24

Issue #19920 has been reported by Eregon (Benoit Daloze). ---------------------------------------- Bug #19920: Ruby 3.1 fails to build with --enable-shared on macos-arm64: is an incompatible architecture (have 'arm64', need '') https://bugs.ruby-lang.org/issues/19920 * Author: Eregon (Benoit Daloze) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- See https://github.com/ruby/ruby-builder/actions/runs/6494296018/job/1763696879… ``` installing bundled gems: /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/ruby/gems/3.1.0 minitest 5.15.0 power_assert 2.0.1 rake 13.0.6 test-unit 3.5.3 rexml 3.2.5 rss 0.2.9 net-ftp 0.1.3 net-imap 0.2.3 net-pop 0.1.1 net-smtp 0.3.1 matrix 0.4.2 prime 0.1.2 rbs 2.7.0 Building native extensions. This could take a while... /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:102:in `run': ERROR: Failed to build gem native extension. (Gem::Ext::BuildError) current directory: /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/ruby/gems/3.1.0/gems/rbs-2.7.0/ext/rbs_extension /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/bin/ruby -I /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib extconf.rb checking for whether -std=c99 is accepted as CFLAGS... *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/bin/$(RUBY_BASE_NAME) /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:616:in `block in try_compile' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:563:in `with_werror' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:616:in `try_compile' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:680:in `try_cflags' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:1025:in `block (2 levels) in append_cflags' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:989:in `block in checking_for' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:354:in `block (2 levels) in postpone' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:324:in `open' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:354:in `block in postpone' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:324:in `open' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:350:in `postpone' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:988:in `checking_for' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:1024:in `block in append_cflags' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:1023:in `each' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/mkmf.rb:1023:in `append_cflags' from extconf.rb:3:in `<main>' To see why this extension failed to compile, please check the mkmf.log which can be found here: /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/ruby/gems/3.1.0/extensions/arm64-darwin-22/3.1.0/rbs-2.7.0/mkmf.log extconf failed, exit code 1 Gem files will remain installed in /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/ruby/gems/3.1.0/gems/rbs-2.7.0 for inspection. Results logged to /Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/ruby/gems/3.1.0/extensions/arm64-darwin-22/3.1.0/rbs-2.7.0/gem_make.out from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/ext_conf_builder.rb:28:in `build' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:171:in `build_extension' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:205:in `block in build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:202:in `each' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:202:in `build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/installer.rb:843:in `build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/installer.rb:326:in `install' from ./tool/rbinstall.rb:899:in `block in install' from ./tool/rbinstall.rb:713:in `no_write' from ./tool/rbinstall.rb:899:in `install' from ./tool/rbinstall.rb:1063:in `block (2 levels) in <main>' from ./tool/rbinstall.rb:1044:in `foreach' from ./tool/rbinstall.rb:1044:in `block in <main>' from ./tool/rbinstall.rb:1119:in `block in <main>' from ./tool/rbinstall.rb:1116:in `each' from ./tool/rbinstall.rb:1116:in `<main>' /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:102:in `run': extconf failed, exit code 1 (Gem::InstallError) from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/ext_conf_builder.rb:28:in `build' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:171:in `build_extension' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:205:in `block in build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:202:in `each' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/ext/builder.rb:202:in `build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/installer.rb:843:in `build_extensions' from /private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/lib/rubygems/installer.rb:326:in `install' from ./tool/rbinstall.rb:899:in `block in install' from ./tool/rbinstall.rb:713:in `no_write' from ./tool/rbinstall.rb:899:in `install' from ./tool/rbinstall.rb:1063:in `block (2 levels) in <main>' from ./tool/rbinstall.rb:1044:in `foreach' from ./tool/rbinstall.rb:1044:in `block in <main>' from ./tool/rbinstall.rb:1119:in `block in <main>' from ./tool/rbinstall.rb:1116:in `each' from ./tool/rbinstall.rb:1116:in `<main>' make: *** [do-install-nodoc] Error 1 BUILD FAILED (macOS 13.6 using ruby-build 20231012) ``` And the mkmf.log: ``` DYLD_FALLBACK_LIBRARY_PATH=.:/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib:/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4 "clang -o conftest -I/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/include/ruby-3.1.0/arm64-darwin22 -I/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/include/ruby-3.1.0/ruby/backward -I/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/include/ruby-3.1.0 -I. -I/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/include -DENABLE_PATH_CHECK=0 -I/opt/homebrew/opt/gmp/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef -fno-common -pipe conftest.c -L. -L/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib -L. -L/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib -fstack-protector-strong -L/opt/homebrew/opt/gmp/lib -lruby.3.1 " dyld[47952]: terminating because inserted dylib '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' could not be loaded: tried: '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (no such file), '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), './libruby.3.1.dylib' (no such file), '/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')) dyld[47952]: tried: '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (no such file), '/private/var/folders/df/1dm_t2rx0054k7bw1g_7nyl80000gn/T/ruby-build.20231012101401.55539.tTlMYN/ruby-3.1.4/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')), './libruby.3.1.dylib' (no such file), '/Users/runner/hostedtoolcache/Ruby/3.1.4/arm64/lib/libruby.3.1.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need '')) checked program was: /* begin */ 1: #include "ruby.h" 2: 3: int main(int argc, char **argv) 4: { 5: return !!argv[argc]; 6: } /* end */ ``` I believe this bug has nothing to do with ruby-build except that ruby-build does `--enable-shared` by default. A known workaround is to use `--disable-shared`: https://github.com/rbenv/ruby-build/discussions/1961#discussioncomment-4031… Lots of other people met the same or similar issue as discussed there. Also discussed in https://github.com/ruby/rbs/issues/877 It would be nice if this could be fixed on the 3.1 branch. -- https://bugs.ruby-lang.org/

3 4

[ruby-core:114062] [Ruby master Bug#19751] Ruby 3.2.2 Fails to Compile from Source
by martin_vahi (Martin Vahi) 23 May '24

23 May '24

Issue #19751 has been reported by martin_vahi (Martin Vahi). ---------------------------------------- Bug #19751: Ruby 3.2.2 Fails to Compile from Source https://bugs.ruby-lang.org/issues/19751 * Author: martin_vahi (Martin Vahi) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Details are at the attached file, but the build-crash-log seems to be: ``` compiling addr2line.c compiling dmyenc.c linking miniruby /bin/sh ./tool/ifchange "--timestamp=.rbconfig.time" rbconfig.rb rbconfig.tmp rbconfig.rb updated generating encdb.h encdb.h updated generating x86_64-linux-fake.rb ./template/fake.rb.in:19:in `eval': (eval):1: syntax error, unexpected backslash (SyntaxError) \\# 71 "./version.c" 3 ... ^ from ./template/fake.rb.in:19:in `value' from ./template/fake.rb.in:24:in `block (3 levels) in <main>' from ./template/fake.rb.in:23:in `scan' from ./template/fake.rb.in:23:in `block (2 levels) in <main>' from /home/mmmv/applications/Ruby/v_3_1_2/lib/ruby/3.1.0/erb.rb:905:in `eval' from /home/mmmv/applications/Ruby/v_3_1_2/lib/ruby/3.1.0/erb.rb:905:in `result' from ./tool/generic_erb.rb:36:in `block (2 levels) in <main>' from ./tool/generic_erb.rb:36:in `block in <main>' from ./tool/generic_erb.rb:29:in `map' from ./tool/generic_erb.rb:29:in `<main>' make: *** [uncommon.mk:791: x86_64-linux-fake.rb] Error 1 mmmv@hoidla01:~/tmp_/kompil/2023_03_30_v_3_2_2/ruby-3.2.2$ uname -a Linux hoidla01 4.19.0-22-amd64 #1 SMP Debian 4.19.260-1 (2022-09-29) x86_64 GNU/Linux mmmv@hoidla01:~/tmp_/kompil/2023_03_30_v_3_2_2/ruby-3.2.2$ ``` The source was the official one from the https://cache.ruby-lang.org/pub/ruby/3.2/ruby-3.2.2.tar.xz Thank You for reading this bug report :-) ---Files-------------------------------- Ruby_v_3_2_2_fails_to_compile.txt (31.6 KB) -- https://bugs.ruby-lang.org/

6 8

[ruby-core:116200] [Ruby master Bug#20183] `erb/escape.so` cannot be loaded when `--with-static-linked-ext`
by nobu (Nobuyoshi Nakada) 23 May '24

23 May '24

Issue #20183 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20183: `erb/escape.so` cannot be loaded when `--with-static-linked-ext` https://bugs.ruby-lang.org/issues/20183 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Since `cgi/escape.c` and `erb/escape.c` are both initialized by `Init_escape()` functions, both call the same function in `extinit.c`. -- https://bugs.ruby-lang.org/

4 3

[ruby-core:117658] [Ruby master Feature#20448] Make coverage event hooking C API public
by ms-tob (Matt S) 21 May '24

21 May '24

Issue #20448 has been reported by ms-tob (Matt S). ---------------------------------------- Feature #20448: Make coverage event hooking C API public https://bugs.ruby-lang.org/issues/20448 * Author: ms-tob (Matt S) * Status: Open ---------------------------------------- # Abstract Gathering code coverage information is a well-known goal within software engineering. It is most commonly used to assess code coverage during automated testing. A lesser known use-case is coverage-guided fuzz testing, which will be the primary use-case presented in this issue. This issue exists to request that Ruby coverage event hooking be made part of its official, public C API. # Background Ruby currently provides a number of avenues for hooking events *or* gathering coverage information: 1. The [Coverage](https://ruby-doc.org/3.3.0/exts/coverage/Coverage.html) module 2. The [TracePoint](https://ruby-doc.org/3.3.0/TracePoint.html) module 3. The [rb_add_event_hook](https://ruby-doc.org/3.3.0/extension_rdoc.html#label-Hoo… extension function Unfortunately, none of these pieces of functionality solve this issue's specific use-case. The `Coverage` module is not a great fit for real-time coverage analysis with an unknown start and stop point. Coverage-guided fuzz testing requires this. The `TracePoint` module and `rb_add_event_hook` are not able to hook branch and line coverage events. Coverage-guided fuzz testing typically tracks branch events. # Proposal The ultimate goal is to enable Ruby C extensions to process coverage events in real-time. I did some cursory investigation into the Ruby C internals to determine what it would take to achieve this, but I'm by no means an expert, so my list may be incomplete. The good news is that much of this functionality already exists, but it's part of the private, internal-only C API. 1. Make `RUBY_EVENT_COVERAGE_LINE` and `RUBY_EVENT_COVERAGE_BRANCH` public: https://github.com/ruby/ruby/blob/v3_3_0/vm_core.h#L2182-L2184 a. This would be an addition to the current public event types: https://github.com/ruby/ruby/blob/v3_3_0/include/ruby/internal/event.h#L32-… 2. Allow initializing global coverage state so that coverage tracking can be fully enabled a. Currently, if `Coverage.setup` or `Coverage.start` is not called, then coverage events cannot be hooked. I do not fully understand why this is, but I believe it has something to do with `rb_get_coverages` and `rb_set_coverages`. If calls to `rb_get_coverages` return `NULL` (https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L641-L647, https://github.com/ruby/ruby/blob/v3_3_0/iseq.c#L864-L868) then coverage hooking will not be enabled. I believe the `Coverage` module initializes that state via a `rb_set_coverages` call here: https://github.com/ruby/ruby/blob/v3_3_0/ext/coverage/coverage.c#L112-L120. b. So, to achieve this goal, a C extension would need to be able to call `rb_set_coverages` or somehow initialize the global coverage state. I've actually been able to achieve this functionality by calling undocumented features and defining `RUBY_EVENT_COVERAGE_BRANCH`: ```c #include <ruby.h> #include <ruby/debug.h> #define RUBY_EVENT_COVERAGE_BRANCH 0x020000 // ... rb_event_flag_t events = RUBY_EVENT_COVERAGE_BRANCH; rb_event_hook_flag_t flags = ( RUBY_EVENT_HOOK_FLAG_SAFE | RUBY_EVENT_HOOK_FLAG_RAW_ARG ); rb_add_event_hook2( (rb_event_hook_func_t) event_hook_branch, events, counter_hash, flags ); ``` If I call `Coverage.setup(branches: true)`, and add this event hook, then branch hooking works as expected. `rb_add_event_hook2` will still respect the `RUBY_EVENT_COVERAGE_BRANCH` value if its passed. But it would be better if I could rely on official functionality rather than undocumented features. The above two points would be requirements for this functionality, but there's an additional nice-to-have: 3. Extend the public `tracearg` functionality to include additional coverage information a. Currently, `tracearg` offers information like `rb_tracearg_lineno` and `rb_tracearg_path`. It would be helpful if it also provided additional coverage information like `coverage.c`'s column information and a unique identifier for each branch. Currently, I can only use `(path, lineno)` as a unique identifier for a branch because that's what's offered by the public API, but more information like column number would be helpful for uniquely identify branches. Since there can be multiple `if` statements on a single line, this can provide ambiguous identification for a branch event. # Use cases This use-case was born out of a new coverage-guided Ruby fuzzer: https://github.com/trailofbits/ruzzy. You can read more about its implementation details here: https://blog.trailofbits.com/2024/03/29/introducing-ruzzy-a-coverage-guided…. You can also find the Ruby C extension code behind its implementation here: https://github.com/trailofbits/ruzzy/blob/v0.7.0/ext/cruzzy/cruzzy.c#L220-L…. So, the primary use-case here is enabling real-time, coverage-guided fuzz testing of Ruby code. However, as mentioned in the abstract, gathering code coverage information is useful in many domains. For example, it could enable new workflows in standard unit/integration test coverage. It could also enable gathering coverage information in real-time as an application is running. I see this as the most generalized form of gathering code coverage information, and something like the `Coverage` module as a specialized implementation. Another example, https://bugs.ruby-lang.org/issues/20282 may be solved by this more generalized solution. We are tracking this request downstream here: https://github.com/trailofbits/ruzzy/issues/9 # Discussion Fuzz testing is another tool in a testers toolbelt. It is an increasingly common way to improve software's robustness. Go has it built in to the standard library, Python has Atheris, Java has Jazzer, JavaScript has Jazzer.js, etc. OSS-Fuzz has helped identify and fix over 10,000 vulnerabilities and 36,000 bugs [using fuzzing](https://google.github.io/oss-fuzz/#trophies). Ruby deserves a good fuzzer, and improving coverage gathering would help achieve that goal. The `Coverage` module, `TracePoint` module, and `rb_add_event_hook` function seem like they could fulfill this goal. However, after deeper investigation, none of them fit the exact requirements for this use-case. # See also - https://bugs.ruby-lang.org/issues/20282 - https://github.com/google/atheris - https://security.googleblog.com/2020/12/how-atheris-python-fuzzer-works.html - https://github.com/CodeIntelligenceTesting/jazzer/ - https://www.code-intelligence.com/blog/java-fuzzing-with-jazzer - https://go.dev/doc/security/fuzz/ -- https://bugs.ruby-lang.org/

2 8