December 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:119801] [Ruby master Feature#20878] A new C API to create a String by adopting a pointer: `rb_enc_str_adopt(const char *ptr, long len, long capa, rb_encoding *enc)`
by byroot (Jean Boussier) 07 Jan '25

07 Jan '25

Issue #20878 has been reported by byroot (Jean Boussier). ---------------------------------------- Feature #20878: A new C API to create a String by adopting a pointer: `rb_enc_str_adopt(const char *ptr, long len, long capa, rb_encoding *enc)` https://bugs.ruby-lang.org/issues/20878 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Context A common use case when writing C extensions is to generate text or bytes into a buffer, and to return it back wrapped into a Ruby String. Examples are `JSON.generate(obj) -> String`, and all other format serializers, compression libraries such as `ZLib.deflate`, etc, but also methods such as `Time.strftime`, ### Current Solution #### Work in a buffer and copy the result The most often used solution is to work with a native buffer and to manage a native allocated buffer, and once the generation is done, call `rb_str_new*` to copy the result inside memory managed by Ruby. It works, but isn't very efficient because it cause an extra copy and an extra `free()`. On `ruby/json` macro-benchmarks, this represent around 5% of the time spent in `JSON.generate`. ```c static void fbuffer_free(FBuffer *fb) { if (fb->ptr && fb->type == FBUFFER_HEAP_ALLOCATED) { ruby_xfree(fb->ptr); } } static VALUE fbuffer_to_s(FBuffer *fb) { VALUE result = rb_utf8_str_new(FBUFFER_PTR(fb), FBUFFER_LEN(fb)); fbuffer_free(fb); return result; } ``` #### Work inside RString allocated memory Another way this is currently done, is to allocate an `RString` using `rb_str_buf_new`, and write into it with various functions such as `rb_str_catf`, or writing past `RString.len` through `RSTRING_PTR` and then resize it with `rb_str_set_len`. The downside with this approach is that it contains a lot of inefficiencies, as `rb_str_set_len` will perform numerous safety checks, compute coderange, and write the string terminator on every invocation. Another major inneficiency is that this API make it hard to be in control of the buffer growth, so it can result in a lot more `realloc()` calls than manually managing the buffer. This method is used by `Kernel#sprintf`, `Time#strftime` etc, and when I attempted to improve `Time#strftime` performance, this problem showed up as the biggest bottleneck: - https://github.com/ruby/ruby/pull/11547 - https://github.com/ruby/ruby/pull/11544 - https://github.com/ruby/ruby/pull/11542 ### Proposed API I think a more effcient way to do this would be to work with a native buffer, and then build a RString that "adopt" the memory region. Technically, you can currently do this by reaching directly into `RString` members, but I don't think it's clean, and a dedicated API would be preferable: ```c /** * Similar to rb_str_new(), but it adopts the pointer instead of copying. * * @param[in] ptr A memory region of `capa` bytes length. MUST have been allocated with `ruby_xmalloc` * @param[in] len Length of the string, in bytes, not including the * terminating NUL character, not including extra capacity. * @param[in] capa The usable length of `ptr`, in bytes, including the * terminating NUL character. * @param[in] enc Encoding of `ptr`. * @exception rb_eArgError `len` is negative. * @return An instance of ::rb_cString, of `len` bytes length, `capa - 1` bytes capacity, * and of `enc` encoding. * @pre At least `capa` bytes of continuous memory region shall be * accessible via `ptr`. * @pre `ptr` MUST have been allocated with `ruby_xmalloc`. * @pre `ptr` MUST not be manually freed after `rb_enc_str_adopt` has been called. * @note `enc` can be a null pointer. It can also be seen as a routine * identical to rb_usascii_str_new() then. */ rb_enc_str_adopt(const char *ptr, long len, long capa, rb_encoding *enc); ``` An alternative to the `adopt` term, could be `move`. -- https://bugs.ruby-lang.org/

8 31

[ruby-core:120281] [Ruby master Feature#20960] Support building OpenSSL for Ruby with AWS-LC
by sachiang (Samuel Chiang) 07 Jan '25

07 Jan '25

Issue #20960 has been reported by sachiang (Samuel Chiang). ---------------------------------------- Feature #20960: Support building OpenSSL for Ruby with AWS-LC https://bugs.ruby-lang.org/issues/20960 * Author: sachiang (Samuel Chiang) * Status: Open ---------------------------------------- Hello, I’m an engineer at AWS working on [AWS-LC](https://github.com/aws/aws-lc), AWS’s open-source cryptographic library maintained for AWS and their customers. AWS-LC supports CPU-specific performance optimizations for AWS Graviton 2, [AWS Graviton 3](https://github.com/aws/aws-lc/commit/ae87faf735c0241a115542b1c1022d12556…, and [Intel x86-64 with AVX-512 instructions](https://github.com/aws/aws-lc/commit/e22cf5065761bec8882c66ce…. We’ve formally verified [a subset of](https://github.com/awslabs/aws-lc-verification#verified-code) AWS-LC’s cryptographic primitives, and continue to invest in expanding this coverage. AWS-LC can be also built in FIPS mode to help consumers meet FIPS 140-3 compliance requirements. To give Ruby users a well-documented and supported way to take advantage of these investments, we would like to upstream build compatibility for AWS-LC into Ruby. We believe that this would provide the best experience for users wishing to build Ruby against AWS-LC. It would also allow users to circumvent maintaining and applying their own patch sets to build Ruby with AWS-LC. Earlier this year, we[engaged with the CPython maintainers](https://discuss.python.org/t/support-building-ssl-and-hashlib-… to successfully address similar needs for our respective users. We are working on patch sets to integrate Ruby‘s OpenSSL module with AWS-LC. AWS-LC is committed to backwards compatibility and we aim to keep our API stable. Our open source repository has CI jobs ([here](https://github.com/aws/aws-lc/tree/main/tests/ci/integration) and [here](https://github.com/aws/aws-lc/blob/main/.github/workflows/integration…) asserting every change’s compatibility with multiple different open-source projects. We’ve [recently added Ruby 3.1 and 3.2](https://github.com/aws/aws-lc/commit/57133c00c71f9a6790c7c9d9a63ab883a… to this test suite and we’re in the midst of incorporating Ruby’s main branch and 3.3. These tests are used to catch compatibility regressions against every change before they’re merged and to resolve potential build issues beforehand when upstream projects make relevant changes. Relevant unit tests from the upstream projects’ are also ran to confirm that the underlying `libcrypto` & `libssl` behave as expected. By expanding our regular testing processes to include Ruby, we proactively prevent any unanticipated breaks in the Ruby/AWS-LC build. The proposed integration supports all features of Ruby’s OpenSSL module, except for the use of DHE cipher suites in `libssl`. Excluding this, we have confirmed that all relevant unit tests for Ruby’s OpenSSL module perform as expected. If you folks agree that this integration would be beneficial for Ruby and its consumers, I’d be more than happy to put together a PR. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:117929] [Ruby master Feature#20498] Negated method calls
by MaxLap (Maxime Lapointe) 05 Jan '25

05 Jan '25

Issue #20498 has been reported by MaxLap (Maxime Lapointe). ---------------------------------------- Feature #20498: Negated method calls https://bugs.ruby-lang.org/issues/20498 * Author: MaxLap (Maxime Lapointe) * Status: Open ---------------------------------------- I want to propose the following syntax: `foo.!bar`. I know it's already valid syntax, but please read on for details. When someone write a somewhat long line of code that is negated, the main way I've seen of doing it is: ``` must_create_user = !User.where(somelong: :condition, even_more: "thing").exists? ``` I personally highly dislike it, as I must keep the "not" in the back of my mind as I read the line. When quickly reading a line like this, it's super easy to misread and understand the opposite result. The current ways around this I can think of are: * rename the variable (can be annoying) * Use `unless` (only possible when in a condition; some people, like me, have a hard time grapsping a `unless`) * use a `.!`in the end (`foo.exists?.!`), I've never seen that and it looks ugly to me (this is subjective). * create a new method name with the negated meaning (not always possible) My proposal would look like this: ``` must_create_user = User.where(somelong: :condition, even_more: "thing").!exists? ``` You cannot forget the bang that you saw 15 words ago, it's right there. It also basically reads as English: "user where ... doesn't exists". The main argument against this I can think of is that it's technically already a valid syntax. I believe it's frowned upon to override the bang operator and I'm not aware of places where it is overridden to with also having a parameter. I made a prototype in RubyNext, which you can try here: https://ruby-next.github.io/#gist:0e133bf6f27f2437193dc034d58083dc -- https://bugs.ruby-lang.org/

8 9

[ruby-core:119335] [Ruby master Bug#20770] A *new* pipe operator proposal
by AlexandreMagro (Alexandre Magro) 05 Jan '25

05 Jan '25

Issue #20770 has been reported by AlexandreMagro (Alexandre Magro). ---------------------------------------- Bug #20770: A *new* pipe operator proposal https://bugs.ruby-lang.org/issues/20770 * Author: AlexandreMagro (Alexandre Magro) * Status: Open * ruby -v: 3.3.5 * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Hello, This is my first contribution here. I have seen previous discussions around introducing a pipe operator, but it seems the community didn't reach a consensus. I would like to revisit this idea with a simpler approach, more of a syntactic sugar that aligns with how other languages implement the pipe operator, but without making significant changes to Ruby's syntax. Currently, we often write code like this: ```ruby value = half(square(add(value, 3))) ``` We can achieve the same result using the `then` method: ```ruby value = value.then { add(_1, 3) }.then { square(_1) }.then { half(_1) } ``` While `then` helps with readability, we can simplify it further using the proposed pipe operator: ```ruby value = add(value, 3) |> square(_1) |> half(_1) ``` Moreover, with the upcoming `it` feature in Ruby 3.4 (#18980), the code could look even cleaner: ```ruby value = add(value, 3) |> square(it) |> half(it) ``` This proposal uses the anonymous block argument `(_1)`, and with `it`, it simplifies the code without introducing complex syntax changes. It would allow us to achieve the same results as in other languages that support pipe operators, but in a way that feels natural to Ruby, using existing constructs like `then` underneath. I believe this operator would enhance code readability and maintainability, especially in cases where multiple operations are chained together. Thank you for considering this proposal! -- https://bugs.ruby-lang.org/

18 55

[ruby-core:120410] [Ruby master Bug#20983] New `Ruby` module is linked to from all kinds of documentation pages
by Earlopain (Earlopain _) 03 Jan '25

03 Jan '25

Issue #20983 has been reported by Earlopain (Earlopain _). ---------------------------------------- Bug #20983: New `Ruby` module is linked to from all kinds of documentation pages https://bugs.ruby-lang.org/issues/20983 * Author: Earlopain (Earlopain _) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- For example https://docs.ruby-lang.org/en/master/index.html * "Ruby" Documentation * New to "Ruby"? * "Ruby" interpreter * Get invovled with the "Ruby" community * Contribute to "Ruby" Many other pages have a link to it as well, especially in headers it doesn't look very good. The vast majority of cases will talk about Ruby the language, not Ruby the module. Is there maybe an option in rdoc to block autoreferences to a specific module? I think documentation will look much better without this link everywhere. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:120420] [Ruby master Bug#20985] Undefined constant in net/http
by BertramScharpf (Bertram Scharpf) 31 Dec '24

31 Dec '24

Issue #20985 has been reported by BertramScharpf (Bertram Scharpf). ---------------------------------------- Bug #20985: Undefined constant in net/http https://bugs.ruby-lang.org/issues/20985 * Author: BertramScharpf (Bertram Scharpf) * Status: Open * ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [amd64-freebsd14] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- $ grep -rnH HTTPAuthenticationError /usr/local/lib/ruby /usr/local/lib/ruby/site_ruby/3.3/rubygems/vendor/net-http/lib/net/http.rb:2455: raise HTTPAuthenticationError.new('HTTP authentication failed', err) /usr/local/lib/ruby/3.3/net/http.rb:2455: raise HTTPAuthenticationError.new('HTTP authentication failed', err) /usr/local/lib/ruby/3.3/rubygems/vendor/net-http/lib/net/http.rb:2455: raise HTTPAuthenticationError.new('HTTP authentication failed', err) $ ruby -r net/http -e 'HTTPAuthenticationError' -e:1:in `<main>': uninitialized constant HTTPAuthenticationError (NameError) HTTPAuthenticationError ^^^^^^^^^^^^^^^^^^^^^^^ The `raise` statement will just transform a meaningful error into a `NameError`. -- https://bugs.ruby-lang.org/

3 3

[ruby-core:111450] [Ruby master Bug#19268] Mingw64 Build Failure
by cfis (Charlie Savage) 31 Dec '24

31 Dec '24

Issue #19268 has been reported by cfis (Charlie Savage). ---------------------------------------- Bug #19268: Mingw64 Build Failure https://bugs.ruby-lang.org/issues/19268 * Author: cfis (Charlie Savage) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x64-mingw-ucrt] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- For both Ruby 3.1.3 and Ruby 3.2.0, building on msys2/ucrt64 fails: ``` c linking miniruby.exe /usr/bin/sh: -c: line 1: syntax error near unexpected token `(' /usr/bin/sh: -c: line 1: `/usr/local/ruby/bin/ruby --disable=gems -n -e BEGIN{version=ARGV.shift;mis=ARGV.dup} -e END{abort "UNICODE version mismatch: #{mis}" unless mis.empty?} -e (mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o 15.0.0 ./enc/unicode/15.0.0/casefold.h ./enc/unicode/15.0.0/name2ctype.h' make: *** [uncommon.mk:878: .rbconfig.time] Error 2 ``` The fix that works for me is changing: ``` c -e '(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +"#{Regexp.quote(version)}"/o' \ ``` To ``` c -e "(mis.delete(ARGF.path); ARGF.close) if /ONIG_UNICODE_VERSION_STRING +\"#{Regexp.quote(version)}\"/o" \ ``` ``` -- https://bugs.ruby-lang.org/

1 4

[ruby-core:120448] [Ruby master Bug#20994] C API - How to Create Classes with Non-ASCII characters in Name
by cfis (Charlie Savage) 31 Dec '24

31 Dec '24

Issue #20994 has been reported by cfis (Charlie Savage). ---------------------------------------- Bug #20994: C API - How to Create Classes with Non-ASCII characters in Name https://bugs.ruby-lang.org/issues/20994 * Author: cfis (Charlie Savage) * Status: Open * ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [x64-mswin64_140] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN ---------------------------------------- As part of Rice (Ruby C++ bindings), I am experimenting with using unicode characters to make more readable class names (see https://ruby-rice.github.io/4.x/stl/stl.html#automatically-generated-ruby-c…) I am experimenting with class names like this: ``` Map≺string≺char≻٬vector≺complex≺double≻≻≻` ``` Where < and > are actually Unicode characters precede (\u227A) and succeed (\u227B). In Ruby this works fine: ``` ruby irb(main):01> class Map≺string≺char≻٬vector≺complex≺double≻≻≻ irb(main):02> end => nil irb(main):013> Map≺string≺char≻٬vector≺complex≺double≻≻≻.new => #<Map≺string≺char≻٬vector≺complex≺double≻≻≻:0x0000021114674c98> ``` However, this fails using the Ruby C API `define_method`. Passing a `char*` that is utf8 encoded fails because `define_method` calls `rb_intern` which calls `rb_intern2` which forces the use of ascii encoding (see https://github.com/ruby/ruby/blob/5fec9308320e8b377681ef19b0cd46d53f94e8ac/…) I thought I might be able to define the class using ascii characters, and then call `rb_define_const` to add in a utf8 encoded name, but that also has the same problem. My question - how does one create class names that have non ascii characters in them via the C API? -- https://bugs.ruby-lang.org/

2 3

[ruby-core:116009] [Ruby master Bug#20147] FreeBSD libyaml
by kaiquekandykoga 31 Dec '24

31 Dec '24

Issue #20147 has been reported by kaiquekandykoga (Kaíque Koga). ---------------------------------------- Bug #20147: FreeBSD libyaml https://bugs.ruby-lang.org/issues/20147 * Author: kaiquekandykoga (Kaíque Koga) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I use ruby-install to manage multiple Ruby versions. Some time ago I had issues installing Ruby 3.2.2 https://github.com/postmodern/ruby-install/issues/463, and the same issue was open for ruby-build in https://github.com/rbenv/ruby-build/issues/2184. I am trying to install Ruby 3.3.0 using ruby-install, and the same behaviour is occurring. Please, take a look in the links I have provided for detailed information. ``` shell /usr/home/kandy/src/ruby-3.3.0/lib/yaml.rb:3: warning: It seems your ruby installation is missing psych (for YAML output). To eliminate this warning, please install libyaml and reinstall your ruby. uh-oh! RDoc had a problem: cannot load such file -- psych run with --debug for full backtrace *** Error code 1 Stop. make: stopped in /usr/home/kandy/src/ruby-3.3.0 !!! Compiling ruby 3.3.0 failed! ``` Thank you. -- https://bugs.ruby-lang.org/

3 6

[ruby-core:120174] [Ruby master Bug#20943] Constant defined in `Data` block
by nobu (Nobuyoshi Nakada) 30 Dec '24

30 Dec '24

Issue #20943 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20943: Constant defined in `Data` block https://bugs.ruby-lang.org/issues/20943 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- From https://github.com/ruby/ruby/pull/12274: > A couple times in code review I've seen constants inadvertently leak to top level from within a `Struct` or `Data` do block. I think it would be nice to show reopening the `Data` class when a constant is defined, so the constant is defined within the namespace. In this case, `Measure::NONE` instead of top level `Object::NONE`. It would also show readers that it's okay to reopen a `Data` class, which seems nice since some folk might not realize. Thanks for considering! However, I think that `NONE` probably might be intended to be defined under `Measure`. Current: ```ruby Measure = Data.define(:amount, :unit) do NONE = Data.define end p NONE #=> NONE ``` Another: ```ruby Measure = Data.define(:amount, :unit) do NONE = Data.define p NONE #=> Measure::NONE end p NONE # uninitialized constant NONE (NameError) ``` @zverok How do think? -- https://bugs.ruby-lang.org/

7 7