September 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114070] [Ruby master Bug#19753] IO::Buffer#get_string can't handle negative offset

by noteflakes (Sharon Rosner)

Issue #19753 has been reported by noteflakes (Sharon Rosner). ---------------------------------------- Bug #19753: IO::Buffer#get_string can't handle negative offset https://bugs.ruby-lang.org/issues/19753 * Author: noteflakes (Sharon Rosner) * Status: Open * Priority: Normal * ruby -v: 3.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby irb(main):001:0> b = IO::Buffer.for('abc') => #<IO::Buffer 0x00007f858f5450c0+3 EXTERNAL READONLY SLICE> ... irb(main):002:0> b.get_string(-1) => "\x00abc" irb(main):003:0> b.get_string(-1000, 3) (irb):3:in `get_string': Specified offset+length exceeds data size! (ArgumentError) from (irb):3:in `<main>' from /home/sharon/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.7.1/exe/irb:9:in `<top (required)>' from /home/sharon/.rbenv/versions/3.2.0/bin/irb:25:in `load' from /home/sharon/.rbenv/versions/3.2.0/bin/irb:25:in `<main>' ``` Using a negative offset returns garbage in the string but it also might segfault: ```ruby irb(main):003:0> b = IO::Buffer.map(File.open('sgt-nodes.sql', 'r+')) => #<IO::Buffer 0x00007f189de14000+2008858 EXTERNAL MAPPED SHARED> irb(main):004:0> b.get_string(-1000) (irb):4: [BUG] Segmentation fault at 0x00007f189de13c18 ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0021 p:---- s:0109 e:000108 CFUNC :get_string ... ``` ## Expected behaviour I think it might be nice to have `#get_string` behave like other methods taking an offset, like `String#[]`. For example: ```ruby irb(main):001:0> b = IO::Buffer.for('abc') => #<IO::Buffer 0x00007f858f5450c0+3 EXTERNAL READONLY SLICE> ... irb(main):002:0> b.get_string(-1) => "c" irb(main):003:0> b.get_string(-2) => "bc" irb(main):003:0> b.get_string(-1000) => "abc" irb(main):003:0> b.get_string(-1000, 2) => "ab" ``` -- https://bugs.ruby-lang.org/

1 day, 15 hours

5
9
0 0

[ruby-core:112304] [Ruby master Bug#19427] Marshal.load(source, freeze: true) doesn't freeze in some cases

by andrykonchin (Andrew Konchin)

Issue #19427 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19427: Marshal.load(source, freeze: true) doesn't freeze in some cases https://bugs.ruby-lang.org/issues/19427 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * ruby -v: 3.1 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I've noticed that the `freeze` option doesn't work in the following cases: - when dumped object extends a module - when dumped object responds to `#marshal_dump` and `#marshal_load` methods - when dumped object responds to `#_dump` method Is it expected behaviour or a known issue? Examples: ```ruby module M end object = Object.new object.extend(M) object = Marshal.load(Marshal.dump(object), freeze: true) object.frozen? # => false ``` ```ruby class UserMarshal attr_accessor :data def initialize @data = 'stuff' end def marshal_dump() :data end def marshal_load(data) @data = data end end object = Marshal.load(Marshal.dump(UserMarshal.new), freeze: true) object.frozen? # => false ``` ```ruby class UserDefined attr_reader :a, :b def initialize @a = 'stuff' @b = @a end def _dump(depth) Marshal.dump [:stuff, :stuff] end def self._load(data) a, b = Marshal.load data obj = allocate obj.instance_variable_set :@a, a obj.instance_variable_set :@b, b obj end end ``` -- https://bugs.ruby-lang.org/

3 days, 19 hours

5
13
0 0

[ruby-core:112873] [Ruby master Bug#19530] `Array#sum` and `Enumerable#sum` sometimes show different behaviours

by dstosik (David Stosik)

Issue #19530 has been reported by dstosik (David Stosik). ---------------------------------------- Bug #19530: `Array#sum` and `Enumerable#sum` sometimes show different behaviours https://bugs.ruby-lang.org/issues/19530 * Author: dstosik (David Stosik) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.1 (2023-02-08 revision 31819e82c8) [arm64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi everyone. 👋🏻 We recently discovered that `Array#sum` and `Enumerable#sum` will output different results in some edge cases. Here's the smallest script I managed to write to reproduce the issue: ``` ruby class Money def initialize(amount) @amount = amount.to_f end def +(other) self.class.new(@amount + other.to_f) end def to_f @amount end end p [7.0].each.sum(Money.new(0)).class #=> Money p [7.0] .sum(Money.new(0)).class #=> Float 💥 ``` I understand that it is expected that `#sum` may not honor custom definitions of the `#+` method (particularly when the summed values are `Float`). However, I would like to bring your attention to the fact that, in the example above, calling `#sum` on an `Array` of `Float` values and calling `#sum` on an `Enumerable` that yields the same `Float` values will return results of different types. I've reproduced the same behaviour with multiple versions of Ruby going from 2.6.5 to 3.2.1. Ideally, I would expect `[7.0].sum(Money.new(0))` to return a `Money` object identical to the one returned by `[7.0].each.sum(Money.new(0))`. I think it would make sense if at least they returned an identical value (even if it is a `Float`). -- https://bugs.ruby-lang.org/

3 days, 22 hours

3
3
0 0

[ruby-core:114403] [Ruby master Feature#19840] [Proposal] Expand Find pattern to Multiple Find

by FlickGradley (Nick Bradley)

Issue #19840 has been reported by FlickGradley (Nick Bradley). ---------------------------------------- Feature #19840: [Proposal] Expand Find pattern to Multiple Find https://bugs.ruby-lang.org/issues/19840 * Author: FlickGradley (Nick Bradley) * Status: Open * Priority: Normal ---------------------------------------- Hello! I love Ruby's pattern matching features. I would like to propose an expansion of the Find pattern which allows the selection of multiple matching elements of an array. I often find myself dealing with data like this: ``` ruby { results: [{ id: 1, name: "foo" }, { id: 2, name: "bar" }, ... ] } ``` My problem is that I need to retrieve all the `id` values from the nested array of hashes, and I don't know how many there will be in advance. It seems that the Find pattern could be expanded from allowing `pattern` (matching a single element) to `*pattern`. Examples: ``` ruby # Base case case { results: [{ id: 1, name: "foo" }, { id: 2, name: "bar" }] } in results: [*{ id: ids }] "matched: #{ids}" else "not matched" end #=> matched: [1, 2] # With * at the end (rest of args) - same result case { results: [{ id: 1, name: "foo" }, { id: 2, name: "bar" }] } in results: [*{ id: ids }, *] "matched: #{ids}" else "not matched" end #=> matched: [1, 2] # When one element doesn't match and there is no *rest case { results: [{ name: "foo" }, { id: 2, name: "bar" }] } in results: [*{ id: ids }] "matched: #{ids}" else "not matched" end #=> not matched ``` Similarly, `*Constant` could work to pull out types with an As pattern: ``` ruby case [1, 2, 3, "string"] in *Integer => nums, * "matched: #{nums}" else "not matched" end #=> matched: [1, 2, 3] ``` Other patterns would work in the same way. Essentially, this expands the concept of `*` in pattern matching to mean "a variable number of `things matching subpattern`". Today, the only pattern supported by `*` is a variable binding - but it could be any of the other subpatterns as well. This proposal does imply that this would work: ``` ruby a = 2 [1, 2, 2, 3] in [*, *^a, *] #=> true ``` To me, the `*` represents the variable number of matches, so the syntax makes intuitive sense. But others may have different opinions about `*^` being adjacent. It may also imply this would work, though we could restrict the number of non-variable patterns (in other words, patterns that have the possibility of not matching) to 1 per Array so that this isn't possible.. I'm not sure something like this would be useful or clear. ``` ruby a = 2 [1, 2, "hello", "ruby"] in [*Integer, *String] #=> true ``` This feature feels like the missing piece of the Find pattern to me - I often want to "Find Multiple". If others agree, I would be happy to contribute by working on this feature and creating a pull request. -- https://bugs.ruby-lang.org/

4 days, 2 hours

3
2
0 0

[ruby-core:111448] [Ruby master Bug#19266] URI::Generic should use URI::RFC3986_PARSER instead of URI::DEFAULT_PARSER

by gareth (Gareth Adams)

Issue #19266 has been reported by gareth (Gareth Adams). ---------------------------------------- Bug #19266: URI::Generic should use URI::RFC3986_PARSER instead of URI::DEFAULT_PARSER https://bugs.ruby-lang.org/issues/19266 * Author: gareth (Gareth Adams) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [arm64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- In June 2014, [`uri/common` was updated][1] to introduce a RFC3986-compliant parser (`URI::RFC3986_PARSER`) as an alternative to the previous RFC2396 parser, and common methods like `URI()` were updated to use that new parser by default. The only methods in `common` not updated were [`URI.extract` and `URI.regexp`][2] which are marked as obsolete. (The old parser was kept in the `DEFAULT_PARSER` constant despite it not being the default for those methods, presumably for backward compatibility.) However, similar [methods called on `URI::Generic`][3] were never updated to use this new parser. This means that methods like `URI::Generic.build` fail when given input that succeeds normally, and this also affects subclasses like URI::HTTP: ``` $ pry -r uri -r uri/common -r uri/generic [1] pry(main)> URI::Generic.build(host: "underscore_host.example") URI::InvalidComponentError: bad component(expected host component): underscore_host.example from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host' [2] pry(main)> URI::HTTP.build(host: "underscore_host.example") URI::InvalidComponentError: bad component(expected host component): underscore_host.example from /Users/gareth/.asdf/installs/ruby/3.1.3/lib/ruby/3.1.0/uri/generic.rb:591:in `check_host' [3] pry(main)> URI("http://underscore_host.example") => #<URI::HTTP http://underscore_host.example> ``` `URI::Generic.new` allows a configurable `parser` positional argument to override the class' default parser, but other factory methods like `.build` don't allow this override. Arguably this doesn't cause problems because at least in the case above, the URI can be built with the polymorphic constructor, but having the option to build URIs from explicit named parts is useful, and leaving the outdated functionality in the `Generic` class is ambiguous. It's possible that the whole Generic class and its subclasses aren't intended to be used directly how I'm intending here, but there's nothing I could see that suggested this is the case. I'm not aware of the entire list of differences between RFC2396 and RFC3986. The relevant difference here is that in RFC2396 an individual segment of a host ([`domainlabel`s][4]) could only be `alphanum | alphanum *( alphanum | "-" ) alphanum`, whereas RFC3986 allows [hostnames][5] to include any of `ALPHA / DIGIT / "-" / "." / "_" / "~"`. It's possible that other differences might cause issues for developers, but since this has gone over 8 years without anyone else caring about this, this is definitely not especially urgent. [1]: https://github.com/ruby/ruby/commit/bb83f32dc3e0424d25fa4e55d8ff32b061320e41 [2]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/… [3]: https://github.com/ruby/ruby/blob/28a17436503c3c4cb7a35b423a894b697cd80da9/… [4]: https://www.rfc-editor.org/rfc/rfc2396#section-3.2.2 [5]: https://www.rfc-editor.org/rfc/rfc3986#page-13 -- https://bugs.ruby-lang.org/

1 month, 1 week

2
3
0 0

[ruby-core:113195] [Ruby master Bug#19592] Unable to statically link a single extension in 3.2.x and >3.1.4

by npic1 (Nat Pic1)

Issue #19592 has been reported by npic1 (Nat Pic1). ---------------------------------------- Bug #19592: Unable to statically link a single extension in 3.2.x and >3.1.4 https://bugs.ruby-lang.org/issues/19592 * Author: npic1 (Nat Pic1) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hi, I need to statically link a single extension (not all of them) by adding it to ext/Setup. This worked until version 3.1.4 and 3.2.x. It appears that this change broke it: [https://bugs.ruby-lang.org/projects/ruby-master/repository/git/revisions/79… [https://github.com/ruby/ruby/pull/6756](https://github.com/ruby/ruby/pull/6…. I have attached a script to reproduce the issue. When running the script, `ldd` fails with: ``` ext/extinit.o: in function `Init_ext': extinit.c:(.text+0x10): undefined reference to `ruby_init_ext' ``` ---Files-------------------------------- ruby_static_test.sh (1.57 KB) -- https://bugs.ruby-lang.org/

1 month, 1 week

2
2
0 0

[ruby-core:113782] [Ruby master Bug#19716] SystemStackError occurs too easily on Alpine Linux (due to small stack size reported by pthread_attr_getstacksize on musl libc)

by alexdowad (Alex Dowad)

Issue #19716 has been reported by alexdowad (Alex Dowad). ---------------------------------------- Bug #19716: SystemStackError occurs too easily on Alpine Linux (due to small stack size reported by pthread_attr_getstacksize on musl libc) https://bugs.ruby-lang.org/issues/19716 * Author: alexdowad (Alex Dowad) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux-musl] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- This is the same problem previously reported against Ruby 2.5 in https://bugs.ruby-lang.org/issues/14387. I just ran into the same problem on Ruby 3.1.4, built on Alpine Linux 3.16. @hsbt stated in the previous thread (https://bugs.ruby-lang.org/issues/14387#note-28): > If you have this issue with Ruby 3.2, please file it with another issue. I hacked `stack_check` in gc.c to print the values of `STACK_START` and `STACK_END` on stack overflow; on the Alpine 3.16 host where this problem just occurred, the values printed were: > Start=0x7ffd0bf4f000, End=0x7ffd0bf32530 ...which shows that Ruby thinks the stack size is only 131072 bytes. On the other hand, `ulimit -s` shows a stack size limit of 8192kb. This Ruby 3.1.4 was built from unmodified source code downloaded from https://cache.ruby-lang.org; the build was configured using `CFLAGS='-march=native' ./configure --disable-install-doc`. The invocation of Ruby which blew the stack was `bundle exec rake db:migrate`, on a mid-sized Rails project. Regarding @ncopa's patch from #14387, @wanabe listed some things which should be done before it is merged into mainline Ruby: > Okay, The patch needs one or more proofs of its behaviour, like that: > > Original issue [ruby-dev:50421] has gone away. > Standard test codes run well. > test-all > ruby/spec > getrlimit works on some situations like: > on single thread > with multiple threads > with RLIMIT_STACK environment variable > getrlimit code of musl is implemented correctly as expected. > (But It's doubtful whether it can be. I guess that a proof of code soundness is very difficult.) > Some "real world" applications can work. > I think it is better example that that application(s) can't work without the patch. I am happy to help cover some of these points if the Ruby development team is still interested in merging @ncopa's patch. -- https://bugs.ruby-lang.org/

1 month, 3 weeks

4
5
0 0

[ruby-core:111306] [Ruby master Feature#19236] Allow to create hashes with a specific capacity from Ruby

by byroot (Jean Boussier)

Issue #19236 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #19236: Allow to create hashes with a specific capacity from Ruby https://bugs.ruby-lang.org/issues/19236 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal * Target version: 3.3 ---------------------------------------- Followup on [Feature #18683] which added a C-API for this purpose. Various protocol parsers such as Redis `RESP3` or `msgpack`, have to create hashes, and they know the size in advance. For efficiency, it would be preferable if they could directly allocate a Hash of the necessary size, so that large hashes wouldn't cause many re-alloccations and re-hash. `String` and `Array` both already offer similar APIs: ```ruby String.new(capacity: XXX) Array.new(XX) / rb_ary_new_capa(long) ``` However there's no such public API for Hashes in Ruby land. ### Proposal I think `Hash` should have a way to create a new hash with a `capacity` parameter. The logical signature of `Hash.new(capacity: 1000)` was deemed too incompatible in [Feature #18683]. @Eregon proposed to add `Hash.create(capacity: 1000)`. -- https://bugs.ruby-lang.org/

1 month, 3 weeks

7
13
0 0

[ruby-core:111075] [Ruby master Bug#19161] Cannot compile 3.0.5 or 3.1.3 on Red Hat 7

by werebus (Matt Moretti)

Issue #19161 has been reported by werebus (Matt Moretti). ---------------------------------------- Bug #19161: Cannot compile 3.0.5 or 3.1.3 on Red Hat 7 https://bugs.ruby-lang.org/issues/19161 * Author: werebus (Matt Moretti) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- When attempting to run `make` on either the 3.0.5 or 3.1.3 release, I get the following error (I included the whole output as it's pretty short): ``` BASERUBY = /opt/ruby/bin/ruby --disable=gems CC = gcc -std=gnu11 LD = ld LDSHARED = gcc -std=gnu11 -shared CFLAGS = -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/13.0.0 CPPFLAGS = DLDFLAGS = -Wl,--compress-debug-sections=zlib -fstack-protector-strong -pie SOLIBS = -lz -lpthread -lrt -lrt -ldl -lcrypt -lm LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = MFLAGS = gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44) Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. compiling ./main.c compiling dmydln.c compiling miniinit.c In file included from vm_core.h:83:0, from iseq.h:14, from mini_builtin.c:3, from miniinit.c:51: thread_pthread.h:108:43: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘struct’ RUBY_EXTERN RB_THREAD_LOCAL_SPECIFIER struct rb_execution_context_struct *ruby_current_ec; ^ In file included from iseq.h:14:0, from mini_builtin.c:3, from miniinit.c:51: vm_core.h: In function ‘rb_current_execution_context’: vm_core.h:1870:34: error: ‘ruby_current_ec’ undeclared (first use in this function) rb_execution_context_t *ec = ruby_current_ec; ^ vm_core.h:1870:34: note: each undeclared identifier is reported only once for each function it appears in miniinit.c: At top level: cc1: warning: unrecognized command line option "-Wno-tautological-compare" [enabled by default] cc1: warning: unrecognized command line option "-Wno-self-assign" [enabled by default] cc1: warning: unrecognized command line option "-Wno-parentheses-equality" [enabled by default] cc1: warning: unrecognized command line option "-Wno-constant-logical-operand" [enabled by default] cc1: warning: unrecognized command line option "-Wno-cast-function-type" [enabled by default] make: *** [miniinit.o] Error 1 ``` Both Ruby 3.0.4 and 3.1.2 build without error and pass (all but one of) the `make check` tests. I know RHEL 7 is getting to be pretty old; I suspect a factor here are the ancient build tools available to me. But... EOL for Ruby 2.7 comes before the one for RHEL 7, so I'm trying to prioritize. -- https://bugs.ruby-lang.org/

1 month, 3 weeks

10
14
0 0

[ruby-core:114932] [Ruby master Bug#19907] Method calls with keyword arguments in eval leaks callcache and callinfo objects

by peterzhu2118 (Peter Zhu)

Issue #19907 has been reported by peterzhu2118 (Peter Zhu). ---------------------------------------- Bug #19907: Method calls with keyword arguments in eval leaks callcache and callinfo objects https://bugs.ruby-lang.org/issues/19907 * Author: peterzhu2118 (Peter Zhu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- The following script leaks callcache and callinfo objects: ```ruby def foo(a:); end 10.times do 10_000.times do eval(<<~RUBY) foo(a: 1) RUBY end puts "Number of live objects: #{GC.stat(:heap_live_slots)}" puts "Memory usage: #{`ps -o rss= -p #{$$}`}" puts end ``` Output is: ``` Number of live objects: 46248 Memory usage: 16160 Number of live objects: 65902 Memory usage: 19888 Number of live objects: 92656 Memory usage: 24032 Number of live objects: 126791 Memory usage: 28048 Number of live objects: 132919 Memory usage: 28816 Number of live objects: 180687 Memory usage: 32384 Number of live objects: 181957 Memory usage: 32464 Number of live objects: 227485 Memory usage: 34224 Number of live objects: 256101 Memory usage: 37200 Number of live objects: 274151 Memory usage: 38752 ``` After performing a `ObjectSpace.dump_all`, I found that it is leaking callcache and callinfo objects that is being held on by the `Object` class. ```json {"address":"0x102ecef70", "type":"CLASS", "shape_id":2, "slot_size":160, "class":"0x102ecf8d0", "variation_count":0, "superclass":"0x102ecfd30", "real_class_name":"Object", "singleton":true, "references":[ ... ]} {"address":"0x1030c90a0", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callinfo", "mid":"foo", "memsize":40, "flags":{"wb_protected":true}} {"address":"0x102fad568", "type":"IMEMO", "shape_id":0, "slot_size":40, "imemo_type":"callcache", "memsize":40, "flags":{"wb_protected":true, "old":true, "uncollectible":true, "marked":true}} ``` -- https://bugs.ruby-lang.org/

1 month, 4 weeks

4
3
0 0

2024

2023

2022

ruby-core September 2023