- ruby-core - ml.ruby-lang.org

[ruby-core:113288] [Ruby master Misc#19608] Being a co-maintainer of the ruby/openssl for the OpenSSL FIPS mode
by jaruga (Jun Aruga) 15 May '23

15 May '23

Issue #19608 has been reported by jaruga (Jun Aruga). ---------------------------------------- Misc #19608: Being a co-maintainer of the ruby/openssl for the OpenSSL FIPS mode https://bugs.ruby-lang.org/issues/19608 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal ---------------------------------------- ## Motivation and context Recently I have been working for the [ruby/openssl](https://github.com/ruby/openssl) to support OpenSSL 3 FIPS mode such as sending pull-requests and reporting issues to the [OpenSSL project](https://github.com/openssl/openssl). The related issue ticket is [here](https://github.com/ruby/openssl/issues/603). Currently a challenge of the ruby/openssl is that it doesn't work well on the OpenSSL FIPS mode, and I want ruby/openssl to work on it by adding the OpenSSL 3 FIPS mode case to the CI, and by adding more FIPS related unit tests and features. To solve this challenge, I would like to be a co-maintainer of the ruby/openssl for the FIPS mode related things. What do you think? ## What is FIPS mode? For someone who is interested in knowing the FIPS mode. Let me share the related documents below. In my understanding, FIPS mode is a security policy developed by US government. In some cases, the shipped Linux OS systems need to follow this policy. FIPS related documents: * [FIPS Wikipedia](https://en.wikipedia.org/wiki/Federal_Information_Processing_Sta… * [RHEL](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linu… * [Amazon Linux](https://aws.amazon.com/blogs/publicsector/enabling-fips-mode-amazon-… * [SUSE Linux](https://www.suse.com/support/kb/doc/?id=000019432) * [Ubuntu](https://ubuntu.com/security/certifications/docs/fips) And OpenSSL has a feature to enable the FIPS mode. The README is [here](https://github.com/openssl/openssl/blob/master/README-FIPS.md). And there can be FIPS specific issues in the ruby/openssl with the OpenSSL FIPS mode enabled. ## Past FIPS related issue tickets As a reference, I just found some old issue tickets below. It is about OpenSSL 1.0 and 1.1 FIPS mode. * https://bugs.ruby-lang.org/issues/6946 * https://bugs.ruby-lang.org/issues/19073 -- https://bugs.ruby-lang.org/

2 5

[ruby-core:113449] [Ruby master Bug#19635] errno may be overwritten by rb_sprintf if it triggers GC
by wks (Kunshan Wang) 15 May '23

15 May '23

Issue #19635 has been reported by wks (Kunshan Wang). ---------------------------------------- Bug #19635: errno may be overwritten by rb_sprintf if it triggers GC https://bugs.ruby-lang.org/issues/19635 * Author: wks (Kunshan Wang) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Here is an excerpt in the `trap` function in `signal.c` ``` c oldfunc = ruby_signal(sig, func); if (oldfunc == SIG_ERR) rb_sys_fail_str(rb_signo2signm(sig)); ``` `ruby_signal` tries to register a signal handling function. If it fails, it returns `SIG_ERR`, and `errno` is set to by `sigaction` to indicate the error. However, the snippet above calls `rb_signo2signm(sig)` before calling `rb_sys_fail_str`. `rb_signo2signm` allocates a Ruby heap object using `rb_sprintf`. The problem is, **if this `rb_sprintf` triggers GC**, the garbage collector may perform may complex operations, including executing `obj_free`, calling `mmap` to allocate more memory from the OS, etc. They **may overwrite the `errno`**. So if GC is triggered in the very unfortunate time, the caller of the `trap` function will receive an exception, but with the wrong reason. This may cause some tests, such as the `test_trap_uncatchable_#{sig}` test cases in `test_signal.rb`, to fail. There are similar use cases in `hash.c`, for example: ```c if (ret) rb_sys_fail_str(rb_sprintf("setenv(%s)", name)); ``` The good practice is always loading from `errno` immediately after calling functions that may set `errno`, such as `sigaction` and `setenv`. -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113486] [Ruby master Feature#19641] Allow setting OpenSSL::SSL::SSLContext in Net::HTTP
by shouichi (Shouichi KAMIYA) 15 May '23

15 May '23

Issue #19641 has been reported by shouichi (Shouichi KAMIYA). ---------------------------------------- Feature #19641: Allow setting OpenSSL::SSL::SSLContext in Net::HTTP https://bugs.ruby-lang.org/issues/19641 * Author: shouichi (Shouichi KAMIYA) * Status: Open * Priority: Normal ---------------------------------------- [Abstract] Allow setting OpenSSL::SSL::SSLContext in Net::HTTP. [Background] The current implementation of Net::HTTP in Ruby allows setting certain properties of SSLContext, but not SSLContext itself. This limits the flexibility when communicating with servers that are not under our control, as we need to keep adding options to Net::HTTP every time someone wants to set properties that are not already supported. - https://bugs.ruby-lang.org/issues/9450 - https://bugs.ruby-lang.org/issues/9758 - https://bugs.ruby-lang.org/issues/16555 - https://bugs.ruby-lang.org/issues/18418 [Proposal] This proposal suggests allowing setting SSLContext itself to Net::HTTP, in addition to the current options. This would increase the flexibility and allow users to perform more complex SSL configurations. [Use cases] A user wants to set OpenSSL::SSL::Context#security_level which is not currently supported by the existing SSL options in Net::HTTP. [Discussion] This feature is necessary because it gives more flexibility to users who need to communicate with servers that require custom SSL configurations. It is better than using existing features because it allows users to set up SSL configurations that are not currently supported by the existing options, and it can potentially simplify the API by allowing users to set SSLContext itself instead of having to provide multiple options for every possible SSL configuration. A potential drawback is conflict handling may not be obvious to users. For example, what happens if a user both sets Net::HTTP#verify_hostname and OpenSSL::SSL::Context#verify_hostname? [See also] Python allows users to pass context to HTTPSConnection and key_file and cert_file options are deprecated in favor of context. https://docs.python.org/3/library/http.client.html#http.client.HTTPSConnect… -- https://bugs.ruby-lang.org/

1 0

[ruby-core:111566] [Ruby master Bug#19294] Enumerator.product works incorrectly with consuming enumerators
by zverok (Victor Shepelev) 14 May '23

14 May '23

Issue #19294 has been reported by zverok (Victor Shepelev). ---------------------------------------- Bug #19294: Enumerator.product works incorrectly with consuming enumerators https://bugs.ruby-lang.org/issues/19294 * Author: zverok (Victor Shepelev) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ```ruby s = StringIO.new('abc') Enumerator.product([1, 2, 3], s.each_char).to_a # Expected: => [[1, "a"], [1, "b"], [1, "c"], [2, "a"], [2, "b"], [2, "c"], [3, "a"], [3, "b"], [3, "c"]] # Actual: => [[1, "a"], [1, "b"], [1, "c"]] ``` The implementation consumes the non-first enumerator to produce the first combination. Somewhat related to the dilemma of consuming and non-consuming enumerators (#19061). PS: I noticed I don't understand why it is `Enumerator.product` and not `Enumerable#product`, but probably it is too late to raise the questions :( -- https://bugs.ruby-lang.org/

3 2

[ruby-core:113420] [Ruby master Bug#19632] Disable external iterator for frozen enumerator
by make_now_just (Hiroya Fujinami) 14 May '23

14 May '23

Issue #19632 has been reported by make_now_just (Hiroya Fujinami). ---------------------------------------- Bug #19632: Disable external iterator for frozen enumerator https://bugs.ruby-lang.org/issues/19632 * Author: make_now_just (Hiroya Fujinami) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Currently, methods to manipulate an external iterator like `#next` and `#feed` can be called even if a receiver of an enumerator is frozen. However, these methods change the state of an external iterator in an enumerator. Therefore, it seems a BUG to me, and these methods should raise FrozenError if the receiver is frozen. ```ruby e = 3.times.freeze # Current e.next # => 1 e.next # => 2 # Expected e.next # raise FrozenError ``` Two years ago, this issue was mentioned in [a comment](https://bugs.ruby-lang.org/issues/17195#note-5). I suggest fixing the following methods to raise FrozenError against a frozen enumerator. - `Enumerator#next` - `Enumerator#next_values` - `Enumerator#peek` - `Enumerator#peek_values` - `Enumerator#feed` - `Enumerator#rewind` Also, even if an enumerator is frozen, it does not affect other methods for internal iterators. -- https://bugs.ruby-lang.org/

4 5

[ruby-core:113477] [Ruby master Bug#19639] Escaped newline literals in Regexp are ignored in extended / free-spacing mode
by janosch-x 14 May '23

14 May '23

Issue #19639 has been reported by janosch-x (Janosch Müller). ---------------------------------------- Bug #19639: Escaped newline literals in Regexp are ignored in extended / free-spacing mode https://bugs.ruby-lang.org/issues/19639 * Author: janosch-x (Janosch Müller) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271) [arm64-darwin21] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- When we want to match whitespace with a Regexp that uses the x-flag, we can do that with escaped literal whitespace: ```ruby ' '[/\ /x] # => " " ``` This works for all whitespace - except newlines: ```ruby "\n"[/\ /x] # => "" /\ /x.source # => "" ``` I guess another parsing step eliminates such escaped newlines before the regexp parsing happens? It is probably a rare issue because most people prefer "\n" in Regexps over literal newlines, but at least the relevant statement in the [documentation](https://docs.ruby-lang.org/en/master/regexp_rdoc.html#label-… is a bit too broad as it is: > Use escaped whitespace such as \ , i.e. a space preceded by a backslash. Maybe we want to add a caveat like `(Does not work for newlines.)`? -- https://bugs.ruby-lang.org/

2 1

[ruby-core:113475] [Ruby master Bug#18933] Dir.tmpdir implemented in non-Ractor-safe manner
by headius (Charles Nutter) 14 May '23

14 May '23

Issue #18933 has been updated by headius (Charles Nutter). Why not just put this in a private constant? I believe that would be Ractor-safe and it would make more sense than a lazy class variable or class instance variable (that's not actually lazy). ---------------------------------------- Bug #18933: Dir.tmpdir implemented in non-Ractor-safe manner https://bugs.ruby-lang.org/issues/18933#change-103057 * Author: kreynolds (Kelley Reynolds) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin21] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- # Background Inside a Ractor, it is expected that generating a temporary directory using `Dir.tmpdir` will work. The current implementation uses a number of things which are not considered Rector-safe and will require refactoring. # How to reproduce ```ruby Ractor.new { Dir.tmpdir } ``` # Expectation and result The result is expected to be something along the lines of `"/var/folders/xm/y4c00x0s26sgf_zlnqjh_7800000gn/T"` but instead a `Ractor::IsolationError` exception is raised: ```ruby 3.1.2/lib/ruby/3.1.0/tmpdir.rb:23:in `tmpdir': can not access class variables from non-main Ractors (Ractor::IsolationError) ``` # Suggested solutions * Make cached class variable a shareable constant * Modify how random values are generated to be Rector-safe ---Files-------------------------------- tmpdir.diff (1.77 KB) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113474] [Ruby master Feature#18258] Ractor.shareable? can be slow and mutates internal object flags.
by jeremyevans0 (Jeremy Evans) 14 May '23

14 May '23

Issue #18258 has been updated by jeremyevans0 (Jeremy Evans). Tracker changed from Bug to Feature Status changed from Open to Closed ruby -v deleted (3.0.2) Backport deleted (2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN) @headius and I discussed this and we agree it is not a bug, though potentially the situation could be improved. You could avoid the object flag mutation using a temporary hash per call to shareable, but that would likely make it even slower. It doesn't seem possible to cache that an object is not shareable without creating significant cache invalidation issues. Having String#freeze set the shareable flag if the string has no instance variables seems like a good idea, but that is more of a feature request than a bug fix. Changing `Ractor.shareable?` to only do a shallow check and not a full check seems like a large semantic change, not a bug fix, and would make Ractor.shareable? operate differently than Ractor.make_shareable. Maybe a new Ractor.flagged_shareable? method could be added to do a shallow check for the shareable flag, but I'm not sure how useful such a method would be. ---------------------------------------- Feature #18258: Ractor.shareable? can be slow and mutates internal object flags. https://bugs.ruby-lang.org/issues/18258#change-103055 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) ---------------------------------------- On my computer, even with a relatively small object graph,`Ractor.shareable?` can be quite slow (around 1-2ms). The following example creates an object graph with ~40k objects as an example, and on my computer takes around 20ms to execute `Ractor.shareable?`. Because the object cannot be marked as `RB_FL_SHAREABLE` because it contains mutable state, every time we check `Ractor.shareable?` it will perform the same object traversal which is the slow path. ``` ruby require 'benchmark' class Borked def freeze end end class Nested def initialize(count, top = true) if count > 0 @nested = count.times.map{Nested.new(count - 1, false).freeze}.freeze end if top @borked = Borked.new end end attr :nested attr :borked end def test(n) puts "Creating nested object of size N=#{n}" nested = Nested.new(n).freeze shareable = false result = Benchmark.measure do shareable = Ractor.shareable?(nested) end pp result: result, shareable: shareable end test(8) ``` I propose we change `Ractor.shareable?` to only check `RB_FL_SHAREABLE` which gives (1) predictable and fast performance in every case and (2) avoids mutating internal object flags when performing what looks like a read-only operation. I respect that one way of looking at `Ractor.shareable?` is as a cache for object state. But this kind of cache can lead to unpredictable performance. As a result, something like `String#freeze` would not create objects that can be shared with Ractor. However, I believe we can mitigate this by tweaking `String#freeze` to also set `RB_FL_SHAREABLE` if possible. I believe we should apply this to more objects. It will lead to more predictable performance for Ruby. Since there are few real-world examples of Ractor, it's hard to find real world example of the problem. However, I believe such an issue will prevent Ractor usage as even relatively small object graphs (~1000 objects) can cause 1-2ms of latency, and this particular operation does not release the GVL either which means it stalls the entire VM. This issue came from discussion regarding https://bugs.ruby-lang.org/issues/18035 where we are considering using `RB_FL_SHAREABLE` as a flag for immutability. By fixing this issue, we make it easier to implement model for immutability because we don't need to introduce new flags and can instead reuse existing flags. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113473] [Ruby master Bug#17680] tab completion no longer works on irb3.0
by jeremyevans0 (Jeremy Evans) 14 May '23

14 May '23

Issue #17680 has been updated by jeremyevans0 (Jeremy Evans). Status changed from Open to Closed This change from 2.5 is expected due to differences between reline and readline. As readline has been removed from stdlib, reline's behavior is now the only available behavior. Behavior is the same on JRuby. It's possible a pull request to reline to have tab completion behavior more compatible to readline may be considered, but that would be up to the reline maintainer. ---------------------------------------- Bug #17680: tab completion no longer works on irb3.0 https://bugs.ruby-lang.org/issues/17680#change-103054 * Author: simondedeo (Simon DeDeo) * Status: Closed * Priority: Normal * ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin16] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- in irb2.5: rb(main):001:0> source 'li [then I hit tab...] linking.rb list_load.rb list_load_OLD.rb irb(main):001:0> source 'lis [then I hit tab again...] list_load.rb list_load_OLD.rb irb(main):001:0> source 'list_load.rb' [then I hit tab again...] in irb3.0, hitting tab after typing "source 'li" gives. irb(main):001:0' source 'li' Here are my version numbers: ayerie:simon$ irb3.0 -v irb 1.3.0 (2020-12-25) ayerie:CHLOE simon$ ruby3.0 -v ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin16] ayerie:simon$ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113472] [Ruby master Bug#16959] Weakmap has specs and third-party usage despite being a private API
by headius (Charles Nutter) 14 May '23

14 May '23

Issue #16959 has been updated by headius (Charles Nutter). Status changed from Open to Rejected WeakMap has now been in the wild with documentation and specs for several years, so I think it has become a de-facto public API. I'm fine closing this, though I still would like to see the other features I have described added to Weakref. ---------------------------------------- Bug #16959: Weakmap has specs and third-party usage despite being a private API https://bugs.ruby-lang.org/issues/16959#change-103052 * Author: headius (Charles Nutter) * Status: Rejected * Priority: Normal * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- Weakmap is still described as an internal API, and the documentation points users at WeakRef as the official public API: https://github.com/ruby/ruby/blob/1fb16dbb6e28b9f32f92554d29e646e088b21a98/… However there are now specs for its current set of features, even though those features have never been discussed or approved as a public API: https://github.com/ruby/spec/tree/dd8437628a6f2de5b74b338d4960682bb1590a60/… And we are starting to see it being used by the community: * https://github.com/jruby/jruby/issues/6267 * https://github.com/rsim/oracle-enhanced/issues/2027 * https://github.com/rails/rails/pull/39121 One of two things needs to happen: * Weakmap is made a public API after some discussion. It would be an official public feature only in 2.8/3.0 or higher. * ~~The specs are be removed and Weakmap remains a private API not to be used by the community. I suspect the addition of the specs led to folks starting to use this private API.~~ (edit: The Rails PR was merged after the specs, but the change is actually a year old, as mentioned below. In any case there's plenty of in-the-wild uses of WeakMap that go back even further.) Personally, I'm in much more in favor of making WeakRef support all the features necessary to implement Weakmap in pure Ruby, rather than the other way around: https://bugs.ruby-lang.org/issues/6309 But whatever happens it needs to happen soon, since this use case is now a merged feature in Rails master. -- https://bugs.ruby-lang.org/

1 0