January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116015] [Ruby master Bug#20149] Fix memory leak in IPSocket rb_getaddrinfo
by HParker (Adam Hess) 01 Feb '24

01 Feb '24

Issue #20149 has been reported by HParker (Adam Hess). ---------------------------------------- Bug #20149: Fix memory leak in IPSocket rb_getaddrinfo https://bugs.ruby-lang.org/issues/20149 * Author: HParker (Adam Hess) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- PR: https://github.com/ruby/ruby/pull/9413 Fixes a memory leak pthread_attr is created, but never cleaned up ``` require 'socket' 10.times do 10_000.times do IPSocket.getaddress("localhost") end puts `ps -o rss= -p #{$$}` end ``` before: ``` 26744 33400 39928 46328 53240 59640 66168 72696 79352 85880 ``` after: ``` 20864 20864 20864 20864 20864 20992 20992 20992 20992 21120 ``` This likely needs a backport to Ruby 3.3 series. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116493] [Ruby master Bug#20226] Inconsistent Sort results on 3.3.0 compared to previous versions
by omerby (Omer Ben Yosef) 01 Feb '24

01 Feb '24

Issue #20226 has been reported by omerby (Omer Ben Yosef). ---------------------------------------- Bug #20226: Inconsistent Sort results on 3.3.0 compared to previous versions https://bugs.ruby-lang.org/issues/20226 * Author: omerby (Omer Ben Yosef) * Status: Open * Priority: Normal * ruby -v: 3.3.0 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Try this code block: ``` [-0.9, -0.88, -0.85, -0.83, -0.81, -0.79, -0.77, -0.75, -0.73, -0.71, -0.69, -0.67, -0.65, -0.63, -0.6, -0.58, -0.56, -0.54, -0.52, -0.5, -0.48, -0.46, -0.44, -0.42, -0.4, -0.38, -0.35, -0.33, -0.31, -0.29, -0.27, -0.25, -0.23, -0.21, -0.19, -0.17, -0.15, -0.13, -0.1, -0.08, -0.06, -0.04, -0.02, 0.0, 0.02, 0.0, 0.02].sort_by(&:abs) ``` The end result should be the numbers absolute sorted, look at the last 5 numbers of this, the end result of them should be `[0.0, 0.0, -0.02, 0.02, 0.02...]` maintaining the original order, and this behavior is what we see on ruby 3.2.0, however on ruby 3.3.0 the end result will be `[0.0, 0.0, 0.02, 0.02, -0.02...]` This is also inconsistent, as `[-0.02, 0.0, 0.02, 0.0, 0.02].sort_by(&:abs)` will actually provide the expected result. Again, the main issue for us is the difference between 3.3.0 and previous versions of ruby. -- https://bugs.ruby-lang.org/

7 13

[ruby-core:115978] [Ruby master Bug#20104] Regexp#match returns nil but allocates T_MATCH objects
by jeremyevans0 (Jeremy Evans) 01 Feb '24

01 Feb '24

Issue #20104 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20104: Regexp#match returns nil but allocates T_MATCH objects https://bugs.ruby-lang.org/issues/20104 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2023-12-30T03:14:38Z master 8e32c01742) [x86_64-openbsd7.4] * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- Between Ruby 3.2 and 3.3, behavior changed so that Regexp#match will allocate a T_MATCH object even when there is no match. Example code: ```ruby h = {} GC.start GC.disable ObjectSpace.count_objects(h) matches = h[:T_MATCH] || 0 md = /\A[A-Z]+\Z/.match('1') ObjectSpace.count_objects(h) new_matches = h[:T_MATCH] || 0 puts "/\\A[A-Z]+\\Z/.match('1') => #{md.inspect} generates #{new_matches - matches} T_MATCH objects" ``` Result with Ruby 1.9-3.2: ``` /\A[A-Z]+\Z/.match('1') => nil generates 0 T_MATCH objects ``` Results with Ruby 3.3.0 and current master branch: ``` /\A[A-Z]+\Z/.match('1') => nil generates 1 T_MATCH objects ``` This results in a measurable performance decrease for both Sinatra and Roda web applications, as reported at: https://old.reddit.com/r/ruby/comments/18sxtv9/ruby_330_performance_ups_and… Thanks to GitHub users kiskoza and tagliala for producing a minimal example showing this issue: https://github.com/caxlsx/caxlsx/issues/336 -- https://bugs.ruby-lang.org/

3 4

[ruby-core:115338] [Ruby master Bug#19999] Backport: .travis.yml and fixed commits
by jaruga (Jun Aruga) 31 Jan '24

31 Jan '24

Issue #19999 has been reported by jaruga (Jun Aruga). ---------------------------------------- Bug #19999: Backport: .travis.yml and fixed commits https://bugs.ruby-lang.org/issues/19999 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: REQUIRED ---------------------------------------- This is a backport suggestion to make Travis CI stable on the Ruby stable branches ruby_3_2 and etc. Travis CI for Ruby master branch became stable recently with arm64, ppc64le, s390x and arm32 cases. All the CI cases are running without `allow_failures` option. And more importantly, I simplified the `.travis.yml` to maintain it easily without sacrificing performance. So, I think it may be a good time to backport the Travis CI configuration file `.travis.yml` without some commits to fix some issues. https://app.travis-ci.com/github/ruby/ruby/builds/267166336 I can see ruby_3_2 and ruby_3_1 branches on the Travis CI page below. There are no ruby_3_0 and ruby_27 branches there. I am not sure how the branches are used. https://app.travis-ci.com/github/ruby/ruby/branches But it's beneficial to fix at least Travis CI for ruby_3_2 branch to save the Travis infra resource. Seeing the ruby_3_2 log, the ppc64le is already timeout after running maximum 50 minutes. https://app.travis-ci.com/github/ruby/ruby/builds/267156055 https://app.travis-ci.com/github/ruby/ruby/jobs/613043898#L2325 ``` [1/2] TestFiberQueue#test_pop_with_timeout_and_value = 0.00 s [2/2] TestFiberQueue#test_pop_with_timeout====[ 540 seconds still running ]==== ====[ 1080 seconds still running ]==== ====[ 1620 seconds still running ]==== ====[ 2160 seconds still running ]==== ``` First you can make the tests fail rather than stucking by the commit below. https://github.com/ruby/ruby/commit/3eaae72855b23158e2148566bb8a7667bfb395cb As this stucking issue happened with the combination of the `optflags=-O1` on ppc64le, I think you avoid the issue by porting the `.travis.yml` from the `master` branch where the `optflags=-O1` is not used in ppc64le. -- https://bugs.ruby-lang.org/

2 6

[ruby-core:116503] [Ruby master Bug#20227] Unexpected "<internal:array>" appears since c84237f9531aed3b204d3fdacc2dd9d2bd4c7d81
by yahonda (Yasuo Honda) 30 Jan '24

30 Jan '24

Issue #20227 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #20227: Unexpected "<internal:array>" appears since c84237f9531aed3b204d3fdacc2dd9d2bd4c7d81 https://bugs.ruby-lang.org/issues/20227 * Author: yahonda (Yasuo Honda) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2024-01-23T20:09:57Z master c84237f953) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Unexpected "<internal:array>" appears since c84237f9531aed3b204d3fdacc2dd9d2bd4c7d81 Rails CI against Ruby master branch gets failed https://buildkite.com/rails/rails-nightly/builds/85#018d3dec-4a38-41cb-9ebf… ``` Failure: ActionDispatch::Routing::RoutesInspectorTest#test_routes_when_expanded [test/dispatch/routing/inspector_test.rb:365]: --- expected +++ actual @@ -1 +1 @@ -["--[ Route 1 ]----------", "Prefix | custom_assets", "Verb | GET", "URI | /custom/assets(.:format)", "Controller#Action | custom_assets#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:333", "--[ Route 2 ]----------", "Prefix | custom_furnitures", "Verb | GET", "URI | /custom/furnitures(.:format)", "Controller#Action | custom_furnitures#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:334", "--[ Route 3 ]----------", "Prefix | blog", "Verb | ", "URI | /blog", "Controller#Action | Blog::Engine", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:335", "", "[ Routes for Blog::Engine ]", "--[ Route 1 ]----------", "Prefix | cart", "Verb | GET", "URI | /cart(.:format)", "Controller#Action | cart#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:329"] +["--[ Route 1 ]----------", "Prefix | custom_assets", "Verb | GET", "URI | /custom/assets(.:format)", "Controller#Action | custom_assets#show", "Source Location | <internal:array>:52", "--[ Route 2 ]----------", "Prefix | custom_furnitures", "Verb | GET", "URI | /custom/furnitures(.:format)", "Controller#Action | custom_furnitures#show", "Source Location | <internal:array>:52", "--[ Route 3 ]----------", "Prefix | blog", "Verb | ", "URI | /blog", "Controller#Action | Blog::Engine", "Source Location | <internal:array>:52", "", "[ Routes for Blog::Engine ]", "--[ Route 1 ]----------", "Prefix | cart", "Verb | GET", "URI | /cart(.:format)", "Controller#Action | cart#show", "Source Location | <internal:array>:52"] ``` This diff is longer, the difference is that the "Source Location" shows `<internal:array>` while the path to the route file like `/rails/actionpack/test/dispatch/routing/inspector_test.rb` expected. FYI, "Source Location" feature has been added to Rails via https://github.com/rails/rails/commit/a00e548da7a I can reproduce this failure locally as follows. - Steps to reproduce ```ruby git clone https://github.com/rails/rails cd rails rm Gemfile.lock bundle install cd actionpack bin/test test/dispatch/routing/inspector_test.rb -n test_routes_when_expanded ``` - Actual result ``` $ bin/test test/dispatch/routing/inspector_test.rb -n test_routes_when_expanded ... snip ... Run options: -n test_routes_when_expanded --seed 20764 # Running: F Failure: ActionDispatch::Routing::RoutesInspectorTest#test_routes_when_expanded [test/dispatch/routing/inspector_test.rb:365]: --- expected +++ actual @@ -1 +1 @@ -["--[ Route 1 ]----------", "Prefix | custom_assets", "Verb | GET", "URI | /custom/assets(.:format)", "Controller#Action | custom_assets#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:333", "--[ Route 2 ]----------", "Prefix | custom_furnitures", "Verb | GET", "URI | /custom/furnitures(.:format)", "Controller#Action | custom_furnitures#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:334", "--[ Route 3 ]----------", "Prefix | blog", "Verb | ", "URI | /blog", "Controller#Action | Blog::Engine", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:335", "", "[ Routes for Blog::Engine ]", "--[ Route 1 ]----------", "Prefix | cart", "Verb | GET", "URI | /cart(.:format)", "Controller#Action | cart#show", "Source Location | /home/yahonda/src/github.com/rails/rails/actionpack/test/dispatch/routing/inspector_test.rb:329"] +["--[ Route 1 ]----------", "Prefix | custom_assets", "Verb | GET", "URI | /custom/assets(.:format)", "Controller#Action | custom_assets#show", "Source Location | <internal:array>:52", "--[ Route 2 ]----------", "Prefix | custom_furnitures", "Verb | GET", "URI | /custom/furnitures(.:format)", "Controller#Action | custom_furnitures#show", "Source Location | <internal:array>:52", "--[ Route 3 ]----------", "Prefix | blog", "Verb | ", "URI | /blog", "Controller#Action | Blog::Engine", "Source Location | <internal:array>:52", "", "[ Routes for Blog::Engine ]", "--[ Route 1 ]----------", "Prefix | cart", "Verb | GET", "URI | /cart(.:format)", "Controller#Action | cart#show", "Source Location | <internal:array>:52"] bin/test test/dispatch/routing/inspector_test.rb:319 Finished in 0.051798s, 19.3056 runs/s, 19.3056 assertions/s. 1 runs, 1 assertions, 1 failures, 0 errors, 0 skips $ ``` -- https://bugs.ruby-lang.org/

4 4

[ruby-core:114089] [Ruby master Misc#19758] Statically link ext/json
by MyCo (Maik Menz) 30 Jan '24

30 Jan '24

Issue #19758 has been reported by MyCo (Maik Menz). ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/

4 6

[ruby-core:116494] Invitation to RubyKaigi 2024
by Akira Matsuda 30 Jan '24

30 Jan '24

Hello, ruby-core readers! Let me announce that registration for RubyKaigi 2024 is now open! And of course as usual, all Ruby core team members (who have direct commit access to the Git repo) are invited to the conference with free admission. Please proceed to our ticketing site from this URL: https://ti.to/rubykaigi/2024/with/nebxjc5wena For "Committer account name" field, please fill in your account name listed on the committer email list, https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml SVN account name, Redmine account name, or something equivalent that identifies you as a Ruby core committer. Also, RubyKaigi wants talk proposals from true Ruby hackers like you who are reading this message. Our CFP form is already open, and will be closed tomorrow! (in JST) So if you have anything in your mind that you want to share with the audience there, please submit it NOW! https://cfp.rubykaigi.org/events/2024 -- Akira Matsuda<ronnie(a)dio.jp>

1 0

[ruby-core:115888] [Ruby master Bug#20083] Regexp#match? behaving inconsistently with Ruby 3.3.0
by jussikos (Jussi Koljonen) 30 Jan '24

30 Jan '24

Issue #20083 has been reported by jussikos (Jussi Koljonen). ---------------------------------------- Bug #20083: Regexp#match? behaving inconsistently with Ruby 3.3.0 https://bugs.ruby-lang.org/issues/20083 * Author: jussikos (Jussi Koljonen) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- From irb, when calling String#match? ``` pattern = /([\s]*ABC)$/i p "1ABC".match?(pattern) # => true p "12ABC".match?(pattern) # => true p "123ABC".match?(pattern) # => true p "1231ABC".match?(pattern) # => true p "12312ABC".match?(pattern) # => false p "123123ABC".match?(pattern) # => false p "1231231ABC".match?(pattern) # => true p "12312312ABC".match?(pattern) # => true p "123123123ABC".match?(pattern) # => false p "1231231231ABC".match?(pattern) # => false p "12312312312ABC".match?(pattern) # => true p "123123123123ABC".match?(pattern) # => true p "1231231231231ABC".match?(pattern) # => false ``` With `ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22]` and earlier versions (2.7.8 to 3.2.2) return value is always `true` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115904] [Ruby master Bug#20086] Windows Memory Mapped Buffer is buggy.
by ioquatix (Samuel Williams) 30 Jan '24

30 Jan '24

Issue #20086 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20086: Windows Memory Mapped Buffer is buggy. https://bugs.ruby-lang.org/issues/20086 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED ---------------------------------------- We confirmed a bug in memory mapped files on Windows. See https://github.com/ruby/ruby/pull/9358 for the fix. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116345] [Ruby master Feature#20196] Proposal: Binary data literal
by ziggythehamster (Keith Gable) 29 Jan '24

29 Jan '24

Issue #20196 has been reported by ziggythehamster (Keith Gable). ---------------------------------------- Feature #20196: Proposal: Binary data literal https://bugs.ruby-lang.org/issues/20196 * Author: ziggythehamster (Keith Gable) * Status: Open * Priority: Normal ---------------------------------------- I sometimes find myself needing to write some bytes in a Ruby string literal, and this experience leaves a lot to be desired: * Bare strings don't work (potential for encoding corruption unless you remember to `.force_encoding` and never copy-paste just the literal into somewhere else) and are not particularly pleasant given all of the backslashes * Wrapping this in `String.new("\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00`\x00\x00\x00`\b\x06\x00\x00\x00\xE2\x98w8\x00\x000%IDAT", encoding: 'BINARY')` is better, but many tools explode with this because they expect all strings to be valid UTF-8 even if they're an argument to String.new, and it still doesn't have the "beauty" one might expect from Ruby (also it's not frozen unless you also freeze it) * `["9805e474d0a0a1a0000000d094844425000000060000000680600000002e8977830000035294441445"].pack("h*")` parses in all tools and is less harsh to look at, but if you're writing binary data, you probably want to annotate it Here's my basic syntax proposal: ``` %b[ 89504e470d0a1a0a # PNG header 0000000d # Length = 13 bytes 49484452 # IHDR chunk 00000060 # Width = 96px 00000060 # Height = 96px 08 06 # 8bpp RGBA 00 00 00 # deflate / no filter / non-interlaced ] # => "\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00`\x00\x00\x00`\b\x06\x00\x00\x00" ``` More formally: * To match the nibble ordering of a regular string escape, the hex characters are high nibble first (the same as the `H` unpack character). * It follows the same rules as other percent literals, and I am flexible on what character is used. I chose `b` because `h` could be confusing paired with the `h`/`H` unpack characters and the inverted meaning. * We could say that high-nibble-first is capitalized and the lower-case version is low-nibble-first, but I imagine most people will want high-nibble-first. We could also say that `%b[]` returns a `String` but `%B[]` returns an `IO::Buffer`, which has greater utility than having the capability of writing low-nibble-first literals * Whitespace is ignored * Comments are allowed * The encoding is always `Encoding::BINARY` * The resulting string is always frozen (and if `%B[]` means buffer then that is read-only as well) * a-f can also be written A-F Things to consider: * Interpolation could be allowed like normal strings * Embedding strings could be allowed (example below) * `?` literals (characters) should be handled identically to how other kinds of strings are embedded if that is allowed * If interpolation is allowed and you interpolate a number, this should either interpolate `.to_s` as you would expect in a string or raise an error, because there is no unsurprising way to take a number and convert it to one or more bytes * Strings encoded as `Encoding::BINARY` could have their `.inspect` output use this literal * When dealing with bitmasks, it's often convenient to write them out in binary instead of hex so the on bits are easier to identify, but there is no syntax for that here that I am fond of... but someone might have an idea. I thought about `.00001111` or `!00001111` with mandatory whitespace before resuming hex characters, but those didn't feel right to me Example with embedded strings: ``` %b[ 89 "PNG" 0d0a1a0a # PNG header 0000000d # Length = 13 bytes "IHDR" # IHDR chunk 00000060 # Width = 96px 00000060 # Height = 96px 08 06 # 8bpp RGBA 00 00 00 # deflate / no filter / non-interlaced ] ``` Example with interpolation: ``` %b[ #{png_header} #{ihdr = chunk(:ihdr, width: 96, height: 96, bpp: 8, format: :rgba)} #{png_crc(ihdr)} # I didn't include this in the other examples but I needed something to demonstrate here ] ``` Other possible alternatives: * A library (possibly standard library/gem) could have a function like `binary` take a string (potentially a heredoc) and parse it according to the same rules I wrote above. You would have to make the parser strip whitespace and comments, and only hex bytes could be interpolated. * A new pack/unpack symbol could be created that does the same thing as above, so you could `["hex #comments\netc"].pack("...")` * You could probably do a lot of this with an array of hex strings and `pack` but it doesn't allow for freeform whitespace and the way you do it is not obvious without reading the docs for `pack` ... and also you allocate a bunch of strings you don't need * A `Data`-like object more closely related to `IO::Buffer` could be defined that declares the size of things contained within a buffer and then a constant could be written to create an instance of the Data-subclass containing the actual data you want to write out ... but this is a lot of work Potential users: * People writing protocol code in Ruby * People who need to write out magic constants (in my case: the RDB encoding of a Redis value) * People using something like Metasploit Framework to reverse engineer something * Tools could e.g. disassemble x86 into a literal with comments showing the assembly mnemonics -- https://bugs.ruby-lang.org/

5 6