- ruby-core - ml.ruby-lang.org

[ruby-core:114106] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by nobu (Nobuyoshi Nakada) 06 Jul '23

06 Jul '23

Issue #19057 has been updated by nobu (Nobuyoshi Nakada). k0kubun (Takashi Kokubun) wrote in #note-20: > It looks like @byroot sent a patch for this at https://yhbt.net/kgio-public/B5843088-6EC1-4D1B-A45A-2699CA75DD7F@gmail.com…. `rb_convert_type(io, T_FILE, "IO", "to_io")` can be `rb_io_get_io(io)`. ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-103781 * Author: ioquatix (Samuel Williams) * Status: Assigned * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114104] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by k0kubun (Takashi Kokubun) 06 Jul '23

06 Jul '23

Issue #19057 has been updated by k0kubun (Takashi Kokubun). > I've noticed the same thing while trying to install kgio with the latest ruby HEAD. It looks like @byroot sent a patch for this at https://yhbt.net/kgio-public/B5843088-6EC1-4D1B-A45A-2699CA75DD7F@gmail.com…. Both patches are still not released. Unless @normalperson cuts a timely release for kgio and raindrops, we will not be able to use Unicorn on Ruby 3.3 because of this Feature. ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-103779 * Author: ioquatix (Samuel Williams) * Status: Assigned * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114080] [Ruby master Bug#19756] URI::HTTP.build does not accept a host of `_gateway`, but `URI.parse` will.
by postmodern (Hal Brodigan) 05 Jul '23

05 Jul '23

Issue #19756 has been reported by postmodern (Hal Brodigan). ---------------------------------------- Bug #19756: URI::HTTP.build does not accept a host of `_gateway`, but `URI.parse` will. https://bugs.ruby-lang.org/issues/19756 * Author: postmodern (Hal Brodigan) * Status: Open * Priority: Normal * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I noticed a difference in behavior between `URI::HTTP.build` and `URI.parse`. `URI::HTTP.build` will not accept `host:` value of `_gateway`, but `URI.parse` will. ## Steps To Reproduce ```ruby URI::HTTP.build(host: "_gateway") ``` vs. ```ruby URI.parse("http://_gateway") ``` ### Expected Results Both raise the same exception, or return the same URI object. ### Actual Results ``` URI::HTTP.build(host: "_gateway") /usr/share/ruby/uri/generic.rb:601:in `check_host': bad component(expected host component): _gateway (URI::InvalidComponentError) from /usr/share/ruby/uri/generic.rb:640:in `host=' from /usr/share/ruby/uri/generic.rb:673:in `hostname=' from /usr/share/ruby/uri/generic.rb:190:in `initialize' from /usr/share/ruby/uri/generic.rb:136:in `new' from /usr/share/ruby/uri/generic.rb:136:in `build' from /usr/share/ruby/uri/http.rb:61:in `build' from (irb):2:in `<main>' from /usr/local/share/gems/gems/irb-1.7.0/exe/irb:9:in `<top (required)>' from /usr/local/bin/irb:25:in `load' from /usr/local/bin/irb:25:in `<main>' ``` ``` URI.parse("https://_gateway") # => #<URI::HTTPS https://_gateway> ``` ## Additional Information ``` $ gem list uri uri (default: 0.12.1) ``` -- https://bugs.ruby-lang.org/

6 8

[ruby-core:114088] [Ruby master Feature#19757] Add new C API to create a subclass of `Data`
by nobu (Nobuyoshi Nakada) 05 Jul '23

05 Jul '23

Issue #19757 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Feature #19757: Add new C API to create a subclass of `Data` https://bugs.ruby-lang.org/issues/19757 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal ---------------------------------------- I propose a C API `rb_data_define` which crates a subclass of `Data`. ```C /** * Defines an anonymous data class. * * @endinternal * * @param[in] super Superclass of the defining class. Must be a * descendant of ::rb_cData, or 0 as ::rb_cData. * @param[in] ... Arbitrary number of `const char*`, terminated by * NULL. Each of which are the name of fields. * @exception rb_eArgError Duplicated field name. * @return The defined class. */ VALUE rb_data_define(VALUE super, ...); ``` -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114087] [Ruby master Feature#16657] Don't ship bundled gems as .gem files as well as in expanded form
by hsbt (Hiroshi SHIBATA) 05 Jul '23

05 Jul '23

Issue #16657 has been updated by hsbt (Hiroshi SHIBATA). Status changed from Open to Assigned Assignee set to hsbt (Hiroshi SHIBATA) ---------------------------------------- Feature #16657: Don't ship bundled gems as .gem files as well as in expanded form https://bugs.ruby-lang.org/issues/16657#change-103764 * Author: vo.x (Vit Ondruch) * Status: Assigned * Priority: Normal * Assignee: hsbt (Hiroshi SHIBATA) ---------------------------------------- Working at #16651, I wonder why the release tarball ships with the bundled gem in form of .gem packages as well as the expanded sources. It would be nice, if one option is chosen. Ideally just the .gem packages, because these are vanilla upstream packages installable via RubyGems without rbinstall magic. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:112773] [Ruby master Feature#19521] Support for `Module#name=` and `Class#name=`.
by ioquatix (Samuel Williams) 01 Jul '23

01 Jul '23

Issue #19521 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Feature #19521: Support for `Module#name=` and `Class#name=`. https://bugs.ruby-lang.org/issues/19521 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- See https://bugs.ruby-lang.org/issues/19450 for previous discussion and motivation. This proposal introduces `Class#name=` to set the temporary class name. This is an alternative or addition to <https://bugs.ruby-lang.org/issues/19520>. ```ruby c = Class.new do self.name = "fake name" end c = Class.new c.name = "fake name" ``` Alternatively, we could use `set_name`: ```ruby Class.new do set_name "fake_name" end ``` ## Example usage The current Ruby test suite has code which shows the usefulness of this new method: ```ruby def labeled_module(name, &block) Module.new do singleton_class.class_eval { define_method(:to_s) {name} alias inspect to_s alias name to_s } class_eval(&block) if block end end module_function :labeled_module def labeled_class(name, superclass = Object, &block) Class.new(superclass) do singleton_class.class_eval { define_method(:to_s) {name} alias inspect to_s alias name to_s } class_eval(&block) if block end end module_function :labeled_class ``` The updated code would look like this: ```ruby def labeled_module(name, &block) Module.new do self.name = name class_eval(&block) if block end end def labeled_class(name, superclass = Object, &block) Class.new(superclass) do self.name = name class_eval(&block) if block end end module_function :labeled_class ``` Because the name cannot be set as part of `.new`, we have to have a separate block to set the name, before calling `class_eval`. I think the ergonomics and performance of this are slightly worse than the [counter proposal](https://bugs.ruby-lang.org/issues/19520). -- https://bugs.ruby-lang.org/

10 30

[ruby-core:114065] [Ruby master Feature#17339] Semantic grouping with BigDecimal#to_s
by cryptogopher (crypto gopher) 01 Jul '23

01 Jul '23

Issue #17339 has been updated by cryptogopher (crypto gopher). I've sent pull request with solution as specified by author of issue: https://github.com/ruby/bigdecimal/pull/264 ---------------------------------------- Feature #17339: Semantic grouping with BigDecimal#to_s https://bugs.ruby-lang.org/issues/17339#change-103736 * Author: chumaltd (Takahiro Chuma) * Status: Assigned * Priority: Normal * Assignee: mrkn (Kenta Murata) ---------------------------------------- # Abstract Thousands, millions, ... should be expressible with `BigDecimal#to_s`. # Background `BigDecimal('1234567').to_s('3F')` returns "123 456 7.0". # Proposal - Have an option with which `BigDecimal('1234567').to_s('3F')` returns "_1 234 567_.0". - With decimal, `BigDecimal('1234567.8901234').to_s('3F')` should return "1 234 567.890 123 4". - Default behavior should be the above in long term. - And/Or, it would be nice to have a pretty method name. I think #to_s('3F') has universal use cases like money calculation. # Discussion - International System of Units aka SI defines 3-digit-grouping on long numeric sequence. https://www1.bipm.org/jsp/en/ViewCGPMResolution.jsp?CGPM=22&RES=10 - Original discussion in 1948 shows some example of 3-digit-grouping. https://www1.bipm.org/utils/common/pdf/CGPM/CGPM9.pdf#page=117 # Summary We want to have a natural format. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114032] [Ruby master Misc#19747] Propose Kevin Newton and Jemma Issroff as core committers
by k0kubun (Takashi Kokubun) 29 Jun '23

29 Jun '23

Issue #19747 has been reported by k0kubun (Takashi Kokubun). ---------------------------------------- Misc #19747: Propose Kevin Newton and Jemma Issroff as core committers https://bugs.ruby-lang.org/issues/19747 * Author: k0kubun (Takashi Kokubun) * Status: Open * Priority: Normal ---------------------------------------- # Proposal I'd like to propose Kevin Newton (@kddnewton) and Jemma Issroff (@jemmai) as core committers. Now that YARP [Feature #19741] is merged, I want them to work on YARP's CRuby integration without asking other committers to merge their patches. # Background They have not only authored YARP but also contributed various other things to CRuby. I'll include their message for this proposal. ## Kevin Newton I have been working on [ruby/ruby](https://github.com/ruby/ruby/commits?author=kddnewton) since early 2021. My main contributions so far have been: * **YJIT:** I helped design the IR and then implemented the backend for ARM64 support. * **Constants:** I designed and built a new system for constant caches to improve copy-on-write performance and help invalidate less JIT-ed code. * **YARP:** I designed and built YARP from the ground up, which was recently mirrored into CRuby. I want to continue to contribute to ruby/ruby to help maintain the work I've done so far and build more going forward. Being a committer would make it easier to maintain YARP and more easily respond to questions/concerns. ## Jemma Issroff I have been [an active contributor to ruby/ruby](https://github.com/ruby/ruby/pulls?page=1&q=author%3Ajemmaissrof…. Here are a few highlights of my contributions so far: * **Object Shapes:** I implemented the object shapes technique which fundamentally changed how instance variable access worked, to simplify cache checks, increase cache hits, and decrease runtime checks on instance variables. [I spoke about this work](https://www.youtube.com/watch?v=So-KvN3p-eE) at RubyKaigi 2022. https://bugs.ruby-lang.org/issues/18776 * **YARP:** I made [significant contributions to YARP](https://github.com/ruby/yarp/pulls?q=author%3Ajemmaissroff), including writing the code to mirror it into ruby/ruby. https://bugs.ruby-lang.org/issues/18776 * **Documentation:** I have worked on improving the documentation within ruby/ruby, and migrating the Wiki to make it easier to edit, avoid duplication, and stay up to date. [I spoke about contributing to CRuby](https://www.youtube.com/watch?v=7y6DhXQU4wU) at RubyKaigi 2023. https://bugs.ruby-lang.org/issues/19679 I would like to continue to contribute to Ruby. Specifically, being a Ruby committer will reduce friction as I work to compile YARP's AST into ISeqs and fully integrate YARP into ruby/ruby. # Onboarding Aaron and I have helped them work on the CRuby integration. We will continue to help them as they onboard as Ruby committers. -- https://bugs.ruby-lang.org/

7 7

[ruby-core:114029] [Ruby master Bug#19745] Confirm correct behaviour when attaching private method with `#define_method`/`#define_singleton_method`
by itarato (Peter Arato) 28 Jun '23

28 Jun '23

Issue #19745 has been reported by itarato (Peter Arato). ---------------------------------------- Bug #19745: Confirm correct behaviour when attaching private method with `#define_method`/`#define_singleton_method` https://bugs.ruby-lang.org/issues/19745 * Author: itarato (Peter Arato) * Status: Open * Priority: Normal * ruby -v: HEAD * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Should dynamically added private methods be accessible publicly? See the following example? ```ruby private def bar; end foo = Object.new foo.define_singleton_method(:bar, method(:bar)) foo.bar # No error. ``` The script above runs fine on latest Ruby HEAD. Is this correct to ignore the fact that the added method (`method(:bar)`) is private? This came up during a TruffleRuby investigation (https://github.com/oracle/truffleruby/issues/3134) where the result for the same script is: `private method 'bar' called for #<Object:0xc8> (NoMethodError)` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:114028] [Ruby master Bug#18464] RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors
by ivoanjo (Ivo Anjo) 27 Jun '23

27 Jun '23

Issue #18464 has been updated by ivoanjo (Ivo Anjo). Aaaahh. It's a shame, but I can understand 😓 . Thanks for the clarification :) ---------------------------------------- Bug #18464: RUBY_INTERNAL_EVENT_NEWOBJ tracepoint causes an interpreter crash when combined with Ractors https://bugs.ruby-lang.org/issues/18464#change-103691 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] * Backport: 2.6: DONTNEED, 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: DONE ---------------------------------------- When a Ractor is created whilst a tracepoint for `RUBY_INTERNAL_EVENT_NEWOBJ` is active (registered with `rb_tracepoint_new`/`rb_tracepoint_enabled`), the interpreter crashes with a null pointer dereference with the following backtrace: ``` [BUG] Segmentation fault at 0x0000000000000000 ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] ... -- C level backtrace information ------------------------------------------- /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_print_backtrace+0xf) [0x10a15fadd] vm_dump.c:759 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) vm_dump.c:1045 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_vm_bugreport) (null):0 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(bug_report_end+0x0) [0x109f96b81] error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_bug_for_fatal_signal) error.c:820 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(sigsegv+0x52) [0x10a0be3a2] signal.c:964 /usr/lib/system/libsystem_platform.dylib(_sigtramp+0x1d) [0x7fff20934d7d] /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(gc_event_hook_body+0x4) [0x109fb9d21] gc.c:2214 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath) gc.c:2486 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_slowpath_wb_unprotected) gc.c:2507 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_fill+0x0) [0x109fac92e] gc.c:2543 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of0) gc.c:2553 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(newobj_of) gc.c:2552 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_wb_unprotected_newobj_of) gc.c:2567 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(io_alloc+0x12) [0x109fd341c] io.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_io) io.c:8483 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(prep_stdio) io.c:8514 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_io_prep_stdin) io.c:8532 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_2+0xf7) [0x10a1058a7] thread.c:802 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(rb_native_cond_initialize+0x0) [0x10a1055fb] ./thread_pthread.c:1047 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(register_cached_thread_and_wait) ./thread_pthread.c:1099 /Users/ktsanaktsidis/Code/zendesk/ruby/ruby(thread_start_func_1) ./thread_pthread.c:1054 /usr/lib/system/libsystem_pthread.dylib(_pthread_start+0xe0) [0x7fff208ef8fc] ``` (full output is attached). This seems to be because the new Ractor sets up stdio objects (`rb_io_prep_stdin` et. al.), which in turn allocate Ruby objects, before `rb_ec_initialize_vm_stack` is called to set up the initial stack frame. I've attached a patch which works around this by not firing GC event hooks if there is no control frame on the execution context. The patch also includes a test which reproduces the issue using the `objspace` extension; creating a Ractor within an `ObjectSpace.trace_object_allocations` block is enough to trigger the crash. The patch seems to fix things, but if you folk prefer I can also try swapping around the order of `prep_stdio` and `rb_ec_initialize_vm_stack`. ---Files-------------------------------- 0001-Fix-interpreter-crash-caused-by-RUBY_INTERNAL_EVENT_.patch (1.91 KB) crash.log (26.1 KB) ruby_2022-01-08-151326_8927-ktsanaktsidis.crash (18.8 KB) -- https://bugs.ruby-lang.org/

1 0