May 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:117531] [Ruby master Bug#20431] Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1
by shubham_yadav (Shubham Yadav) 15 Jul '24

15 Jul '24

Issue #20431 has been reported by shubham_yadav (Shubham Yadav). ---------------------------------------- Bug #20431: Ruby 3.3.0 build fail with make: *** [io_buffer.o] Error 1 https://bugs.ruby-lang.org/issues/20431 * Author: shubham_yadav (Shubham Yadav) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I'm trying to build ruby from scratch with openss1.1.0. But it's failing with below error. In file included from ./include/ruby/internal/arithmetic/char.h:23, from ./include/ruby/internal/arithmetic.h:24, from ./include/ruby/ruby.h:28, from ./include/ruby/encoding.h:19, from ./include/ruby/io.h:16, from io_buffer.c:9: ./include/ruby/internal/arithmetic/int.h: In function ‘rb_int2num_inline’: ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h:241: warning: comparison is always true due to limited range of data type ./include/ruby/internal/arithmetic/int.h: In function ‘rb_uint2num_inline’: ./include/ruby/internal/arithmetic/int.h:256: warning: comparison is always true due to limited range of data type io_buffer.c: In function ‘io_buffer_read_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c:1872: error: (Each undeclared identifier is reported only once io_buffer.c:1872: error: for each function it appears in.) io_buffer.c: In function ‘io_buffer_write_U8’: io_buffer.c:1872: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S8’: io_buffer.c:1873: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u16’: io_buffer.c:1875: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U16’: io_buffer.c:1876: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s16’: io_buffer.c:1877: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S16’: io_buffer.c:1878: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u32’: io_buffer.c:1880: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U32’: io_buffer.c:1881: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s32’: io_buffer.c:1882: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S32’: io_buffer.c:1883: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_u64’: io_buffer.c:1885: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_U64’: io_buffer.c:1886: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_s64’: io_buffer.c:1887: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_S64’: io_buffer.c:1888: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f32’: io_buffer.c:1890: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F32’: io_buffer.c:1891: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_f64’: io_buffer.c:1892: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_read_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘io_buffer_write_F64’: io_buffer.c:1893: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) io_buffer.c: In function ‘Init_IO_Buffer’: io_buffer.c:3702: error: ‘RB_IO_BUFFER_HOST_ENDIAN’ undeclared (first use in this function) At top level: cc1: warning: unrecognized command line option "-Wno-maybe-uninitialized" cc1: warning: unrecognized command line option "-Wno-tautological-compare" cc1: warning: unrecognized command line option "-Wno-self-assign" cc1: warning: unrecognized command line option "-Wno-parentheses-equality" cc1: warning: unrecognized command line option "-Wno-constant-logical-operand" cc1: warning: unrecognized command line option "-Wno-cast-function-type" make: *** [io_buffer.o] Error 1 -- https://bugs.ruby-lang.org/

6 10

[ruby-core:116114] [Ruby master Bug#20169] `GC.compact` can raises `EFAULT` on IO
by ko1 (Koichi Sasada) 15 Jul '24

15 Jul '24

Issue #20169 has been reported by ko1 (Koichi Sasada). ---------------------------------------- Bug #20169: `GC.compact` can raises `EFAULT` on IO https://bugs.ruby-lang.org/issues/20169 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- 1. `GC.compact` introduces read barriers to detect read accesses to the pages. 2. I/O operations release GVL to pass the control while their execution, and another thread can call `GC.compact` (or auto compact feature I guess, but not checked yet). 3. Call `write(ptr)` can return `EFAULT` when `GC.compact` is running because `ptr` points read-barrier protected pages. Reproducible steps: Apply the following patch to increase possibility: ```patch diff --git a/io.c b/io.c index f6cd2c1a56..83d67ba2dc 100644 --- a/io.c +++ b/io.c @@ -1212,8 +1212,12 @@ internal_write_func(void *ptr) } } + int cnt = 0; retry: - do_write_retry(write(iis->fd, iis->buf, iis->capa)); + for (; cnt < 1000; cnt++) { + do_write_retry(write(iis->fd, iis->buf, iis->capa)); + if (result <= 0) break; + } if (result < 0 && !iis->nonblock) { int e = errno; ``` Run the following code: ```ruby t1 = Thread.new{ 10_000.times.map{"#{_1}"}; GC.compact while true } t2 = Thread.new{ i=0 $stdout.write "<#{i+=1}>" while true } t2.join ``` and ``` $ make run (snip) 4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4><4>#<Thread:0x00007fa61b4dd758 ../../src/trunk/test.rb:3 run> terminated with exception (report_on_exception is true): ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' ../../src/trunk/test.rb:5:in `write': Bad address @ io_write - <STDOUT> (Errno::EFAULT) from ../../src/trunk/test.rb:5:in `block in <main>' make: *** [uncommon.mk:1383: run] Error 1 ``` I think this is why we get `EFAULT` on CI. To increase possibilities running many busy processes (`ruby -e 'loop{}'` for example) will help (and on CI environment there are such busy processes accidentally). -- https://bugs.ruby-lang.org/

6 13

[ruby-core:116908] [Ruby master Bug#20292] Abort ruby by `String#initialize`
by ksss (Yuki Kurihara) 15 Jul '24

15 Jul '24

Issue #20292 has been reported by ksss (Yuki Kurihara). ---------------------------------------- Bug #20292: Abort ruby by `String#initialize` https://bugs.ruby-lang.org/issues/20292 * Author: ksss (Yuki Kurihara) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I found a code that causes an anomaly. ``` $ uname -a Darwin 20208671n 22.6.0 Darwin Kernel Version 22.6.0: Wed Oct 4 21:26:55 PDT 2023; root:xnu-8796.141.3.701.17~4/RELEASE_ARM64_T6020 arm64 $ ruby -v ruby 3.4.0dev (2024-02-22T06:43:46Z master e1c684e471) [arm64-darwin22] $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: -1) }' ruby(13817,0x1f22b2080) malloc: Heap corruption detected, free list is damaged at 0x600000e27bc0 *** Incorrect guard value: 29830901955328 ruby(13817,0x1f22b2080) malloc: *** set a breakpoint in malloc_error_break to debug [1] 13817 abort ruby -e $ ruby -e '100.times { "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".__send__(:initialize, capacity: 0) }' # never return .... ``` My research indicates the following conditions. - String must be at least 64 characters long - Call `#initialize` with capacity: 0 or less - Repeat this several times. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:116930] [Ruby master Bug#20296] Complex(:sym, exception: false) generate exception with weird timing
by MaxLap (Maxime Lapointe) 15 Jul '24

15 Jul '24

Issue #20296 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Bug #20296: Complex(:sym, exception: false) generate exception with weird timing https://bugs.ruby-lang.org/issues/20296 * Author: MaxLap (Maxime Lapointe) * Status: Open * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Happens in Ruby 3.2.2, 3.2.3 and 3.3.0 (didn't test others). Add this to a file and execute it as-is: ``` 3.times { Complex(:sym, exception: false) } puts "hi" ``` The output will be: ``` $ ruby local2.rb hi local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' local2.rb:1:in `Complex': can't convert Array into Complex (TypeError) 3.times { Complex([], exception: false) } ^^^^^^^^^^^^^^^^^^^^ from local2.rb:1:in `block in <main>' from local2.rb:1:in `times' from local2.rb:1:in `<main>' ``` So the exception is still reported (instead of being swallow), but the puts, which is after, had the time to execute first. I can even put a `sleep()` in between and the exceptions are still reported after the puts. In a IRB, there is no exception at all and everything seems to work as expected. Happens if I pass in an Array too. Doesn't seem to happen with bad strings. There is also a quadratic behavior somewhere in there. Running the code in an IRB (which doesn't show exceptions), if you double the number of `times` in the loop, it takes 4x as long. This is probably related... -- https://bugs.ruby-lang.org/

3 2

[ruby-core:116965] [Ruby master Bug#20307] `Hash#update` from compare_by_identity hash can have unfrozen string keys
by nobu (Nobuyoshi Nakada) 13 Jul '24

13 Jul '24

Issue #20307 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20307: `Hash#update` from compare_by_identity hash can have unfrozen string keys https://bugs.ruby-lang.org/issues/20307 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- I don't think this behavior is expected. ```ruby i = Hash.new.compare_by_identity k = "a" i[k] = 0 h = {}.update(i) p h["a"] # => 0 k.upcase! p h.fetch(k) # in 'Hash#fetch': key not found: "A" (KeyError) ``` -- https://bugs.ruby-lang.org/

5 6

[ruby-core:116643] [Ruby master Bug#20250] Crash with "Object ID seen, but not in mapping table: proc" error
by zetter (Chris Zetter) 13 Jul '24

13 Jul '24

Issue #20250 has been reported by zetter (Chris Zetter). ---------------------------------------- Bug #20250: Crash with "Object ID seen, but not in mapping table: proc" error https://bugs.ruby-lang.org/issues/20250 * Author: zetter (Chris Zetter) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hello, I experienced a crash which I was able to reliably reproduce with the following: ``` ruby require 'bundler/inline' gemfile(true) do source 'https://rubygems.org' gem 'activesupport', '7.1.3' end require 'active_support' logger = ActiveSupport::Logger.new('log/log.log', 1, 100 * 1024 * 1024) logger.formatter = proc {|_, _, _, message| "#{message}\n" } logger = ActiveSupport::TaggedLogging.new(logger) logger.tagged("TAG").info "hello" logger.tagged("TAG").info "hello" # usually crashes here GC.start # sometimes need to trigger crash with GC ``` It looks like this is caused by the interaction with the formatter proc and the tagged logging features of activesupport. Let me know if a more minimal example would be useful. I can reproduce this with: * ruby 3.3.0 on arm64-darwin23 * ruby 3.3.0 on x86_64-linux * latest ruby HEAD (5e12b75716) on arm64-darwin23 I cannot reproduce this on ruby 3.2.2. Thanks for any help ---Files-------------------------------- ruby-2024-02-09-092829.ips (13.2 KB) output.log (142 KB) crash_test_2.rb (465 Bytes) -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117644] [Ruby master Feature#20443] Allow Major GC's to be disabled
by eightbitraptor (Matthew Valentine-House) 11 Jul '24

11 Jul '24

Issue #20443 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Feature #20443: Allow Major GC's to be disabled https://bugs.ruby-lang.org/issues/20443 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open ---------------------------------------- [[Github PR #10598]](https://github.com/ruby/ruby/pull/10598) ## Background Ruby's GC running during Rails requests can have negative impacts on currently running requests, causing applications to have high tail-latency. A technique to mitigate this high tail-latency is Out-of-band GC (OOBGC). This is basically where the application is run with GC disabled, and then GC is explicitly started after each request, or when no requests are in progress. This can reduce the tail latency, but also introduces problems of its own. Long GC pauses after each request reduce throughput. This is more pronounced on threading servers like Puma because all the threads have to finish processing user requests and be "paused" before OOBGC can be triggered. This throughput decrease happens for a couple of reasons: 1. There are few heuristics available for users to determine when GC should run, this means that in OOBGC scenarios, it's possible that major GC's are being run more than necessary. 2. The lack of any GC during a request means that lots of garbage objects have been created and not cleaned up, so the process is using more memory than it should - requiring major GC's run as part of OOBGC to do more work and therefore take more time. This ticket attempts to address these issues by: 1. Provide `GC.disable_major` and its antonym `GC.enable_major` to disable and enable only major GC 2. Provide `GC.needs_major?` as a basic heuristic allowing users to tell when Ruby should run a Major GC. These ideas were originally proposed by @ko1 and @byroot in [this rails issue](https://github.com/rails/rails/issues/50449) Disabling GC major's would still allow minor GC's to run during the request, avoiding the ballooning memory usage caused by not running GC at all, and reducing the time that a major takes when we do run it, because the nursery objects have been cleaned up during the request already so there is less work for a major GC to do. This can be used in combination with `GC.needs_major?` to selectively run an OOBGC only when necessary ## Implementation This PR adds 3 new methods to the `GC` module - `GC.disable_major` This prevents major GC's from running automatically. It does not restrict minors. When `objspace->rgengc.need_major_gc` is set and a GC is run, instead of running a major, new heap pages will be allocated and a minor run instead. `objspace->rgengc.need_major_gc` will remain set until a major is manually run. If a major is not manually run then the process will eventually run out of memory. When major GC's are disabled, object promotion is disabled. That is, no objects will increment their ages during a minor GC. This is to attempt to minimise heap growth during the period between major GC's, by restricting the number of old-gen objects that will remain unconsidered by the GC until the next major. When `GC.start` is run, then major GC's will be enabled, a GC triggered with the options passed to `GC.start`, and then `disable_major` will be set to the state it was in before `GC.start` was called. - `GC.enable_major` This simply unsets the bit preventing major GC's. This will revert the GC to normal generational behaviour. Everything behaves as default again. - `GC.needs_major?` This exposes the value of `objspace->rgengc.need_major_gc` to the user level API. This is already exposed in `GC.latest_gc_info[:need_major_by]` but I felt that a simpler interface would make this easier to use and result in more readable code. eg. ```ruby out_of_band do GC.start if GC.needs_major? end ``` Because object aging is disabled when majors are disabled it is recommended to use this in conjunction with `Process.warmup`, which will prepare the heap by running a major GC, compacting the heap, and promoting every remaining object to old-gen. This ensures that minor GC's are running over the smallets possible set of young objects when `GC.disable_major` is true. ## Benchmarks We ran some tests in production on Shopify's core monolith over a weekend and found that: **Mean time spent in GC, as well as p99.9 and p99.99 GC times are all improved.** <img width="1000" alt="Screenshot 2024-04-22 at 16 41 49" src="https://github.com/ruby/ruby/assets/31869/6cff5b11-2e21-40c1-bb84-d994e0e17…"> **p99 GC time is slightly higher.** <img width="1000" alt="Screenshot 2024-04-22 at 16 44 55" src="https://github.com/ruby/ruby/assets/31869/dc645cbe-9495-46f0-8485-24e790c42…"> We're running far fewer OOBGC major GC's now that we have `GC.needs_major?` than we were before, and we believe that this is contributing to a slightly increased number of minor GC's. raising the p99 slightly. **App response times are all improved** We see a 9% reduction in average and p99 response times when compared against standard GC (4% p99.9 and p99.99). <img width="1000" alt="Screenshot 2024-04-22 at 16 55 53" src="https://github.com/ruby/ruby/assets/31869/8a80c102-1564-4bc9-ba44-6e9a8b85f…"> This drops slightly to an 8% reduction in average and p99 response times when compared against standard OOBGC (3.59 p99.9 and 4% p99.99) <img width="1000" alt="Screenshot 2024-04-22 at 16 56 10" src="https://github.com/ruby/ruby/assets/31869/1baef7ea-0155-4ff9-8ba4-a967b7574…"> -- https://bugs.ruby-lang.org/

7 24

[ruby-core:117431] [Ruby master Misc#20407] Question about applying encoding modifier to an interpolated Regexp
by andrykonchin (Andrew Konchin) 11 Jul '24

11 Jul '24

Issue #20407 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Misc #20407: Question about applying encoding modifier to an interpolated Regexp https://bugs.ruby-lang.org/issues/20407 * Author: andrykonchin (Andrew Konchin) * Status: Open ---------------------------------------- I am wondering how Regexp encoding modifiers (u, s, e, n) interfere in encoding negotiation of interpolated Regexp literal. Examples #1 ```ruby # encoding: us-ascii # Unicode: Ф - U+0424 # windows-1251: Ф - 0xD4 # without encoding modifier puts /a #{ "\xd4".force_encoding("windows-1251") } c/.encoding # Windows-1251 puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCII puts /a #{ "\u0424".force_encoding("UTF-8") } c/.encoding # UTF-8 puts /a #{ "\xc2\xa1".b } c/.encoding # ASCII-8BIT # with encoding modifier puts /a #{ "\xd4".force_encoding("windows-1251") } c/e.encoding # Windows-1251 puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP puts /a #{ "\u0424".force_encoding("UTF-8") } c/e.encoding # UTF-8 puts /a #{ "\xc2\xa1".b } c/e.encoding # ASCII-8BIT # string literals concatenation puts ("a" + "\xd4".force_encoding("windows-1251") + "c").encoding # Windows-1251 puts ("a" + "b".encode("windows-1251") + "c").encoding # US-ASCII puts ("a" + "\u0424".force_encoding("UTF-8") + "c").encoding # UTF-8 puts ("a" + "\xc2\xa1".b + "c").encoding # ASCII-8BIT ``` Example #2 ```ruby # encoding: utf-8 # windows-1251: Ф - 0xD4 # unicode: Ф - U+0424 # without encoding modifier puts /a #{ "\xd4".force_encoding("windows-1251") } c/.encoding # Windows-1251 puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCII puts /a #{ "\u0424".force_encoding("UTF-8") } c/.encoding # UTF-8 puts /a #{ "\xc2\xa1".b } c/.encoding # ASCII-8BIT # with encoding modifier puts /a #{ "\xd4".force_encoding("windows-1251") } c/e.encoding # Windows-1251 puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP puts /a #{ "\u0424".force_encoding("UTF-8") } c/e.encoding # UTF-8 puts /a #{ "\xc2\xa1".b } c/e.encoding # ASCII-8BIT # string literals concatenation puts ("a" + "\xd4".force_encoding("windows-1251") + "c").encoding # Windows-1251 puts ("a" + "b".encode("windows-1251") + "c").encoding # UTF-8 puts ("a" + "\u0424".force_encoding("UTF-8") + "c").encoding # UTF-8 puts ("a" + "\xc2\xa1".b + "c").encoding # ASCII-8BIT ``` In the examples above the `e` modifier changes Regexp's encoding only in one case when Regexp's encoding would be `US-ASCII` without the modifier: ```ruby # encoding: us-ascii puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCII puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP ``` ```ruby # encoding: utf-8 puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCII puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP ``` And the `e` modifier doesn't change Regexp's final encoding in all the other cases either Regexp's encoding without modifier is a file source encoding or `ASCII-8BIT`. Looking at the following example: ```ruby # encoding: us-ascii # without modifier p /\xc2\xa1 #{ "a" }\xc2\xa1/.encoding # ASCII-8BIT p /a #{ "\xc2\xa1".force_encoding("EUC-JP") } b/.encoding # EUC-JP p /a #{ "\xc2\xa1".b } b/.encoding # ASCII-8BIT # with modifier p /\xc2\xa1 #{ "a" }\xc2\xa1/e.encoding # EUC-JP p /a #{ "\xc2\xa1".force_encoding("EUC-JP") } b/e.encoding # EUC-JP p /a #{ "\xc2\xa1".b } b/e.encoding # ASCII-8BIT ``` we can notice that the `e` modifier change `ASCII-8BIT` to `EUC-JP` in the first case but doesn't in the third one. So I assume that the `e` modifier could be applied to the Regexp fragments (`\xc2\xa1` and `\xc2\xa1`) before encoding negotiation and not to the whole result after negotiation. Could you please clarify how it works? -- https://bugs.ruby-lang.org/

4 5

[ruby-core:117653] [Ruby master Bug#20446] OUtdated https://cache.ruby-lang.org/pub/ruby/index.txt
by vo.x (Vit Ondruch) 09 Jul '24

09 Jul '24

Issue #20446 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Bug #20446: OUtdated https://cache.ruby-lang.org/pub/ruby/index.txt https://bugs.ruby-lang.org/issues/20446 * Author: vo.x (Vit Ondruch) * Status: Open * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Is the https://cache.ruby-lang.org/pub/ruby/index.txt the place, where all Ruby releases can be found? If yes, then can this file be update with Ruby 3.3.1, 3.2.4 etc? If no, can this file be removed to avoid confusion? -- https://bugs.ruby-lang.org/

3 6

[ruby-core:116597] [Ruby master Bug#20239] Segmentation fault when using Regex on a large String
by martinsp (Martins Polakovs) 08 Jul '24

08 Jul '24

Issue #20239 has been reported by martinsp (Martins Polakovs). ---------------------------------------- Bug #20239: Segmentation fault when using Regex on a large String https://bugs.ruby-lang.org/issues/20239 * Author: martinsp (Martins Polakovs) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since v3.2.0 ruby crashes with segmentation fault on the following script with a `[BUG] Segmentation fault at ...` ``` ruby require "rbconfig/sizeof" ("\u{0101}" + "a" * RbConfig::LIMITS["INT_MAX"] + "b").match(/b/) ``` Crash can be reproduced on the following ruby versions: - ruby 3.2.0 (2022-12-25 revision a528908271) [aarch64-linux] - ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [aarch64-linux] - ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [aarch64-linux] works as expected It seems that call to `enclen` inside `str_lower_case_match` returns negative offset in this case https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/v3… -- https://bugs.ruby-lang.org/

3 2