May 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:113720] [Ruby master Bug#19704] Unable to install readline-ext since 18e55fc1e1ec20e8f3166e3059e76c885fc9f8f2
by yahonda (Yasuo Honda) 01 Jun '23

01 Jun '23

Issue #19704 has been reported by yahonda (Yasuo Honda). ---------------------------------------- Bug #19704: Unable to install readline-ext since 18e55fc1e1ec20e8f3166e3059e76c885fc9f8f2 https://bugs.ruby-lang.org/issues/19704 * Author: yahonda (Yasuo Honda) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-05-30T01:02:40Z master 18e55fc1e1) [x86_64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Rails CI against Ruby 3.3.0dev fails to install readline-ext recenty. https://buildkite.com/rails/rails/builds/96780#01886e3f-eaef-405c-b58a-e90c… According to git bisect, it is triggered by 18e55fc1e1ec20e8f3166e3059e76c885fc9f8f2 ### Steps to reproduce Install ruby 3.3.0dev gem install readline-ext ### Expected behavior It should install readline-ext as the previous commit of Ruby 7ddcd0622f does. ``` $ ruby -v ruby 3.3.0dev (2023-05-29T21:24:22Z master 7ddcd0622f) [x86_64-linux] $ gem install readline-ext Building native extensions. This could take a while... Successfully installed readline-ext-0.1.5 Ignoring rbs-3.0.4 because its extensions are not built. Try: gem pristine rbs --version 3.0.4 Parsing documentation for readline-ext-0.1.5 Installing ri documentation for readline-ext-0.1.5 Done installing documentation for readline-ext after 0 seconds 1 gem installed $ ``` ### Actual result ```ruby $ ruby -v ruby 3.3.0dev (2023-05-30T01:02:40Z master 18e55fc1e1) [x86_64-linux] $ gem install readline-ext Building native extensions. This could take a while... ERROR: Error installing readline-ext: ERROR: Failed to build gem native extension. current directory: /home/yahonda/.rbenv/versions/trunk/lib/ruby/gems/3.3.0+0/gems/readline-ext-0.1.5/ext/readline /home/yahonda/.rbenv/versions/trunk/bin/ruby extconf.rb checking for tgetnum() in -lncurses... yes checking for readline/readline.h... yes checking for readline/history.h... yes checking for readline() in -lreadline... yes checking for rl_getc() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_getc_function() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_filename_completion_function() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_username_completion_function() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_completion_matches() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_refresh_line() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_deprep_term_function in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_completion_append_character in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_completion_quote_character in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_basic_word_break_characters in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_completer_word_break_characters in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_basic_quote_characters in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_completer_quote_characters in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_filename_quote_characters in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_attempted_completion_over in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_library_version in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_editing_mode in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_line_buffer in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_point in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_char_is_quoted_p in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_event_hook in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_catch_sigwinch in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_catch_signals in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_pre_input_hook in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_special_prefixes in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_cleanup_after_signal() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_free_line_state() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_clear_signals() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_set_screen_size() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_get_screen_size() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_vi_editing_mode() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_emacs_editing_mode() in stdio.h,readline/readline.h,readline/history.h... yes checking for replace_history_entry() in stdio.h,readline/readline.h,readline/history.h... yes checking for remove_history() in stdio.h,readline/readline.h,readline/history.h... yes checking for clear_history() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_redisplay() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_insert_text() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_delete_text() in stdio.h,readline/readline.h,readline/history.h... yes checking for rl_hook_func_t* in stdio.h,readline/readline.h,readline/history.h... yes creating Makefile current directory: /home/yahonda/.rbenv/versions/trunk/lib/ruby/gems/3.3.0+0/gems/readline-ext-0.1.5/ext/readline make DESTDIR\= sitearchdir\=./.gem.20230601-64629-eyixt9 sitelibdir\=./.gem.20230601-64629-eyixt9 clean current directory: /home/yahonda/.rbenv/versions/trunk/lib/ruby/gems/3.3.0+0/gems/readline-ext-0.1.5/ext/readline make DESTDIR\= sitearchdir\=./.gem.20230601-64629-eyixt9 sitelibdir\=./.gem.20230601-64629-eyixt9 compiling readline.c readline.c: In function ‘prepare_readline’: readline.c:386:16: error: invalid use of incomplete typedef ‘rb_io_t’ {aka ‘struct rb_io’} 386 | if (ifp->fd < 0) { | ^~ readline.c:395:16: error: invalid use of incomplete typedef ‘rb_io_t’ {aka ‘struct rb_io’} 395 | if (ofp->fd < 0) { | ^~ readline.c: In function ‘readline_s_set_input’: readline.c:565:32: error: invalid use of incomplete typedef ‘rb_io_t’ {aka ‘struct rb_io’} 565 | fd = rb_cloexec_dup(ifp->fd); | ^~ readline.c: In function ‘readline_s_set_output’: readline.c:601:32: error: invalid use of incomplete typedef ‘rb_io_t’ {aka ‘struct rb_io’} 601 | fd = rb_cloexec_dup(ofp->fd); | ^~ readline.c: At top level: cc1: note: unrecognized command-line option ‘-Wno-self-assign’ may have been intended to silence earlier diagnostics cc1: note: unrecognized command-line option ‘-Wno-parentheses-equality’ may have been intended to silence earlier diagnostics cc1: note: unrecognized command-line option ‘-Wno-constant-logical-operand’ may have been intended to silence earlier diagnostics make: *** [Makefile:248: readline.o] Error 1 make failed, exit code 2 Gem files will remain installed in /home/yahonda/.rbenv/versions/trunk/lib/ruby/gems/3.3.0+0/gems/readline-ext-0.1.5 for inspection. Results logged to /home/yahonda/.rbenv/versions/trunk/lib/ruby/gems/3.3.0+0/extensions/x86_64-linux/3.3.0+0-static/readline-ext-0.1.5/gem_make.out ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113719] [Ruby master Bug#19703] text/binary mode of parent process fd is not properly inherited on windows
by YO4 (Yoshinao Muramatsu) 31 May '23

31 May '23

Issue #19703 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Bug #19703: text/binary mode of parent process fd is not properly inherited on windows https://bugs.ruby-lang.org/issues/19703 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Windows OS can't inherit parent process fds greater than 2, but C runtime library support this. https://learn.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functi… So I made test tool below. This tool opens NUL device several times and create child ruby process that dumps fds' translation mode. ``` c // fdinherit.c : test file descprotor inheritance for windows C runtime #include <stdio.h> #include <stdlib.h> #include <process.h> #include <sys/types.h> #include <sys/stat.h> #include <share.h> #include <fcntl.h> #include <io.h> int main(int argc, char* argv[]) { int fh, count = 0; errno_t err; // child process: read fds and check if it includes "\r" and prints // '.' : read as binary // 't' : read as text // 'F' : open error (no fd?) char rb1[] = "print '012'; 3.upto("; char rb2[] = ") { |n|" "print ((IO.for_fd(n, 'rb').read.include?(13.chr) ? '.' : 't') rescue 'F');" "puts if (n % 10) == 9" "}"; char upto[] = "-4294967295"; // fd max number to test if (argc >= 2) count = atoi(argv[1]); if (count < 3) count = 7; // create crlf file (current directory) system("echo;>crlf.txt"); // open fds to inherit // even fds are text mode, odd fds are binary mode for (int i = 3; i <= count; i++) { err = _sopen_s(&fh, "crlf.txt", _O_RDONLY | ((i & 1) ? _O_BINARY : _O_TEXT), _SH_DENYNO, _S_IREAD | _S_IWRITE); if (err > 0) printf("%d %d\n", err, fh); } // on child side, +1'd fd dump will fail (prints 'F') sprintf_s(upto, sizeof(upto), "%d", count + 1); // on windows, arguments are joined into a string. so below has bad manners, but let me take a few corners. _spawnlp(_P_WAIT, "ruby.exe", "ruby.exe", "-e", "\"", rb1, upto, rb2, "\"", NULL); _close(fh); } ``` As an example, this would display something like this ``` >fdinherit.exe 012.t.t.F >fdinherit.exe 16 012.t.t.t. t.t.t.tF > ``` 0,1,2 are stdin, stdout, stderr fds of 3,5,7,... are inherited as binary mode fds of 4,6,... are inherited as text mode And I found a few things to point out. When I modify ruby part like "IO.for_fd(n, 'r')" / "IO.for_fd(n, 'rt')" / "IO.for_fd(n)", It get result like ```012tttttF```, so translation mode are interprited wrong. The odd thing is that the conversion mode is interpreted as is when "IO.for_fd(n, 'rb')", which I think is the expected behavior when "IO.for_fd(n, 'r')". This tool and ruby.exe each work with a combination of msvcrt 32/64bit and ucrt64, so the C runtime interface seems stable. My understanding is that MRI does not state that it supports inheritance of fds larger than 3 in windows, but it seems to work partially. Are there any chances for improvement? For those who want to follow up: it seems that fds are not inherited via cmd.exe /c -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113710] [Ruby master Bug#9944] in ruby for windows in "system" a redirection to append a file works incorrectly
by jeremyevans0 (Jeremy Evans) 31 May '23

31 May '23

Issue #9944 has been updated by jeremyevans0 (Jeremy Evans). Status changed from Assigned to Rejected @headius and I discussed this on the train back from RubyKaigi and determined this is a Windows-specific limitation, not a Ruby bug, and nothing Ruby does can fix the issue in all cases. ---------------------------------------- Bug #9944: in ruby for windows in "system" a redirection to append a file works incorrectly https://bugs.ruby-lang.org/issues/9944#change-103361 * Author: bdimych (Dmitry Bolshakov) * Status: Rejected * Priority: Normal * Assignee: windows * ruby -v: ruby 2.0.0p481 (2014-05-08) [i386-mingw32] * Backport: 2.0.0: REQUIRED, 2.1: REQUIRED ---------------------------------------- ~~~ c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych>C:\Ruby200\bin\ruby.exe -v ruby 2.0.0p481 (2014-05-08) [i386-mingw32] c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych>( echo first line & echo second line & echo third line ) > test.txt c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych>type test.txt first line second line third line c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych>C:\Ruby200\bin\ruby.exe -e "system 'cmd', '/c', 'echo', 'blabla', :out => ['test.txt', 'a']" c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych>type test.txt blabla ne second line third line c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> c:\Users\bdimych> ~~~ --- ruby writes "blabla" in the beginning instead of appending windows 7 64 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113707] [Ruby master Feature#19699] Need a way to store values like dig
by edufcarvalho (Eduardo Carvalho) 31 May '23

31 May '23

Issue #19699 has been reported by edufcarvalho (Eduardo Carvalho). ---------------------------------------- Feature #19699: Need a way to store values like dig https://bugs.ruby-lang.org/issues/19699 * Author: edufcarvalho (Eduardo Carvalho) * Status: Open * Priority: Normal ---------------------------------------- Currently, there is no way to perform a `.store` operation on nested hashes in Ruby as there is for accessing them using `.dig`, so I would like to propose something like: Hash#dig_store(*args, **kwargs, arg = nil), which fills the value at *args with **kwargs if the value at *args is a hash, or with arg if the value is something else. Here are some examples: ```ruby a = {} a.dig_store(:a, :b, :c, :d, 10) => {:a=>{:b=>{:c=>{:d=>10}}}} ``` ```ruby a = {} a.dig_store(:a, :b, :c, :d, a: 10, b: 20, c: 30) => {:a=>{:b=>{:c=>{:d=>{:a=>10, :b=>20, :c=>30}}}}} ``` ```ruby a = {a: {b: {c: "test"}}} a.dig_store(:a, :b, :c, "123") => {:a=>{:b=>{:c=>"123"}}} ``` ```ruby a = {a: {b: {c: {}}} a.dig_store(:a, :b, :c, a: 10, b: 20) => {:a=>{:b=>{:c=>{:a=>10, :b=>20}}}} ``` I'm not that skilled on Ruby (actually a Junior developer), but i'm looking forward to implementing it if the core team approves it -- https://bugs.ruby-lang.org/

2 2

[ruby-core:113676] [Ruby master Bug#19697] Resolv::DNS resolution for international domains fails with "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT"
by clairity (claire c) 30 May '23

30 May '23

Issue #19697 has been reported by clairity (claire c). ---------------------------------------- Bug #19697: Resolv::DNS resolution for international domains fails with "Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT" https://bugs.ruby-lang.org/issues/19697 * Author: clairity (claire c) * Status: Open * Priority: Normal * ruby -v: 3.2.2 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- i try to resolve a registered international domain, and instead of receiving DNS records, get an `incompatible character encodings` error: ``` # irb 3.2.2 :001 > require 'resolv' 3.2.2 :002 > res = Resolv::DNS.open { |dns| dns.getresource '우편.닷컴', Resolv::DNS::Resource::IN::MX } ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `[]=': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError) from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:777:in `sender' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:527:in `block in fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1125:in `block (3 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1123:in `block (2 levels) in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1122:in `block in resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `each' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:1120:in `resolv' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:521:in `fetch_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:507:in `each_resource' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:488:in `getresource' from (irb):3:in `block in <top (required)>' from ~/.rvm/rubies/ruby-3.2.2/lib/ruby/3.2.0/resolv.rb:298:in `open' from (irb):3:in `<main>' from ~/.rvm/gems/ruby-3.2.2/gems/irb-1.6.4/exe/irb:9:in `<top (required)>' ... 4 levels... ``` i also tried this command in 3.3.0-preview1 and 2.1.4, receiving the same error. the expected response is to get DNS records for `우편.닷컴` (or any registered international domain). `dig 우편.닷컴` does indeed return a short but valid DNS record. -- https://bugs.ruby-lang.org/

3 5

[ruby-core:113702] [Ruby master Misc#19122] Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack
by smcgivern (Sean McGivern) 30 May '23

30 May '23

Issue #19122 has been updated by smcgivern (Sean McGivern). ioquatix (Samuel Williams) wrote in #note-5: > If you want to use a specific mode (OS specific), you can do this: > > On Linux, find the mode, e.g. `MADV_DONTNEED = 6` (https://github.com/torvalds/linux/blob/933174ae28ba72ab8de5b35cb7c98fc21123…) > > Shift it one bit to the left (i.e. multiply by 2) `= 12`. > > Then run Ruby like this: > > ``` > > RUBY_SHARED_FIBER_POOL_FREE_STACKS=12 ruby > ``` Thanks for adding this! > As for making this the default, I suppose we could consider it but we'd need to confirm the performance impact. I would prefer to have high performance by default. RSS is a poor measurement IMHO. What about USS and PSS? Are they impacted the same way? Can we subtract the usage that can be later dropped? I mean, as I said in the original description, this is the only part of Ruby that works this way, which makes it surprising. If usage of fibers becomes more common in the wild, or `MADV_FREE` gets used in other areas where Ruby frees memory, then I would predict more reports noting this behaviour. Other languages have reverted their usage of `MADV_FREE` for similar reasons. We can construct another metric that gives us exactly what we want (for cAdvisor, we'd need something like https://github.com/google/cadvisor/issues/3197) but not through PSS or USS; both of those track RSS closely here, at least in my tests. We need to look at either `active_anon + inactive_anon` (for a cgroups `memory.stat` file), or `LazyFree` plus some other things I'm not sure about (for a process's `smaps_rollup`). This won't be the default in most monitoring tools, or in `top` et al, so as I said above, that wouldn't necessarily stop issues coming in. ---------------------------------------- Misc #19122: Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack https://bugs.ruby-lang.org/issues/19122#change-103348 * Author: smcgivern (Sean McGivern) * Status: Open * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- I'd like to propose that Ruby stops using MADV_FREE when freeing a Fiber's stack, and switches to using MADV_DONTNEED even when MADV_FREE is supported. MADV_FREE is used in one place in the Ruby codebase, when freeing the stack of a freed Fiber: https://git.ruby-lang.org/ruby.git/tree/cont.c#n683 The comment for `fiber_pool_stack_free` says: ```c // We advise the operating system that the stack memory pages are no longer being used. // This introduce some performance overhead but allows system to relaim memory when there is pressure. ``` Where possible (i.e. on Linux 4.5 and later), `fiber_pool_stack_free` uses `MADV_FREE` over `MADV_DONTNEED`. This has the side effect that memory statistics such as RSS will not reduce until and unless the OS actually reclaims that memory. If that doesn't happen, then the reported memory usage via RSS will be much higher than the 'real' memory usage. If this was pervasive throughtout the Ruby codebase then that would be one thing, but currently this is just for Fiber. This means that: 1. A program that doesn't use Fiber will have somewhat reliable RSS statistics on recent Linux. 2. A program that heavily uses Fiber (such as something using Async::HTTP) will see an inflated RSS statistic. Go made a similar change to the one I'm proposing here for similar reasons: https://github.com/golang/go/issues/42330 > While `MADV_FREE` is somewhat faster than `MADV_DONTNEED`, it doesn't affect many of the statistics that `MADV_DONTNEED` does until the memory is actually reclaimed. This generally leads to poor user experience, like confusing stats in `top` and other monitoring tools; and bad integration with management systems that respond to memory usage. > [...] > I propose we change the default to prefer `MADV_DONTNEED` over `MADV_FREE`, to favor user-friendliness and minimal surprise over performance. I think it's become clear that Linux's implementation of `MADV_FREE` ultimately doesn't meet our needs. As an aside, MADV_FREE was not used in Ruby 3.1 (https://bugs.ruby-lang.org/issues/19101) and I haven't found any bugs filed about this behaviour other than that one. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113520] [Ruby master Bug#19680] test_process.rb tests fail sometimes on FreeBSD
by kjtsanaktsidis (KJ Tsanaktsidis) 30 May '23

30 May '23

Issue #19680 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #19680: test_process.rb tests fail sometimes on FreeBSD https://bugs.ruby-lang.org/issues/19680 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I've been investigating the repeated failures of test_process.rb on FreeBSD on rubyci. I'm still working on it but I wanted to open this ticket just to keep others in the loop and gather any pointers any of you might have! These are some of the failures I've been investigating - * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230516T063002Z.f… * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230515T103002Z.f… * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230512T083002Z.f… * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230506T023001Z.f… * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230505T103002Z.f… * http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20230517T003001Z.f… -- https://bugs.ruby-lang.org/

2 5

[ruby-core:113698] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by ioquatix (Samuel Williams) 30 May '23

30 May '23

Issue #19057 has been updated by ioquatix (Samuel Williams). Status changed from Open to Closed https://github.com/ruby/ruby/pull/6511 was merged. In addition, the following related PRs were merged: - https://github.com/ruby/etc/pull/26 - https://github.com/ruby/io-wait/pull/25 - https://github.com/ruby/io-console/pull/43 ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-103344 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113696] [Ruby master Feature#19057] Hide implementation of `rb_io_t`.
by Eregon (Benoit Daloze) 29 May '23

29 May '23

Issue #19057 has been updated by Eregon (Benoit Daloze). ianks (Ian Ker-Seymer) wrote in #note-8: > > With one tweak: not int rb_ioptr_descriptor(struct rb_io *ioptr); but int rb_io_descriptor(VALUE io);. > > The goal is to not expose the struct in the C API, so let's not expose it (even if it becomes just an opaque pointer). > > On the other, casting pointers to unsigned ints has implications for pointer provenance models. What do you mean? `struct RBasic*` to `VALUE` or vice-versa? Almost every C API function does that, so that seems a separate issue. Although maybe the above is not clear? `int rb_io_descriptor(VALUE io);` would be called with the Ruby (IO) object (like almost every other C API function), not with a pointer (opaque or not) to the `rb_io_t` struct. ---------------------------------------- Feature #19057: Hide implementation of `rb_io_t`. https://bugs.ruby-lang.org/issues/19057#change-103342 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal * Assignee: ioquatix (Samuel Williams) ---------------------------------------- In order to make improvements to the IO implementation like <https://bugs.ruby-lang.org/issues/18455>, we need to add new fields to `struct rb_io_t`. By the way, ending types in `_t` is not recommended by POSIX, so I'm also trying to rename the internal implementation to drop `_t` where possible during this conversion. Anyway, we should try to hide the implementation of `struct rb_io`. Ideally, we don't expose any of it, but the problem is backwards compatibility. So, in order to remain backwards compatibility, we should expose some fields of `struct rb_io`, the most commonly used one is `fd` and `mode`, but several others are commonly used. There are many fields which should not be exposed because they are implementation details. ## Current proposal The current proposed change <https://github.com/ruby/ruby/pull/6511> creates two structs: ```c // include/ruby/io.h #ifndef RB_IO_T struct rb_io { int fd; // ... public fields ... }; #else struct rb_io; #endif // internal/io.h #define RB_IO_T struct rb_io { int fd; // ... public fields ... // ... private fields ... }; ``` However, we are not 100% confident this is safe according to the C specification. My experience is not sufficiently wide to say this is safe in practice, but it does look okay to both myself, and @Eregon + @tenderlovemaking have both given some kind of approval. That being said, maybe it's not safe. There are two alternatives: ## Hide all details We can make public `struct rb_io` completely invisible. ```c // include/ruby/io.h #define RB_IO_HIDDEN struct rb_io; int rb_ioptr_descriptor(struct rb_io *ioptr); // accessor for previously visible state. // internal/io.h struct rb_io { // ... all fields ... }; ``` This would only be forwards compatible, and code would need to feature detect like this: ```c #ifdef RB_IO_HIDDEN #define RB_IOPTR_DESCRIPTOR rb_ioptr_descriptor #else #define RB_IOPTR_DESCRIPTOR(ioptr) rb_ioptr_descriptor(ioptr) #endif ``` ## Nested public interface Alternatively, we can nest the public fields into the private struct: ```c // include/ruby/io.h struct rb_io_public { int fd; // ... public fields ... }; // internal/io.h #define RB_IO_T struct rb_io { struct rb_io_public public; // ... private fields ... }; ``` ## Considerations I personally think the "Hide all details" implementation is the best, but it's also the lest compatible. This is also what we are ultimately aiming for, whether we decide to take an intermediate "compatibility step" is up to us. I think "Nested public interface" is messy and introduces more complexity, but it might be slightly better defined than the "Current proposal" which might create undefined behaviour. That being said, all the tests are passing. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:113662] [Ruby master Feature#19695] mkmf: Adding the extra option --with-verbose to enable verbose mode.
by jaruga (Jun Aruga) 29 May '23

29 May '23

Issue #19695 has been reported by jaruga (Jun Aruga). ---------------------------------------- Feature #19695: mkmf: Adding the extra option --with-verbose to enable verbose mode. https://bugs.ruby-lang.org/issues/19695 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal ---------------------------------------- I want the new feature to enable the verbose mode by setting the `V = 1` in the generated `Makefile` when building native extensions. Because the `V = 1` prints the compiler command lines. And it's useful to confirm and the compiler flags. It can be used like the commands below. ``` $ ruby /path/to/extconf.rb -- --with-verbose ``` ``` $ rake compiler -- --with-verbose ``` The pull-request is <https://github.com/ruby/ruby/pull/7863>. -- https://bugs.ruby-lang.org/

2 2