- ruby-core - ml.ruby-lang.org

[ruby-core:120147] [Ruby master Bug#20939] When using the parse.y parser, it occurs argument stack underflow (-1) if there is processing after the shareable_constant_value: experimental_everything comment and `C = { **{ k: v } }`
by ydah (Yudai Takada) 26 Dec '24

26 Dec '24

Issue #20939 has been reported by ydah (Yudai Takada). ---------------------------------------- Bug #20939: When using the parse.y parser, it occurs argument stack underflow (-1) if there is processing after the shareable_constant_value: experimental_everything comment and `C = { **{ k: v } }` https://bugs.ruby-lang.org/issues/20939 * Author: ydah (Yudai Takada) * Status: Open * ruby -v: ruby 3.4.0dev (2024-12-10T03:31:47Z master f12e2622c1) [arm64-darwin23] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- The following code will occurs argument stack underflow if it is a parse.y parser. ```ruby # shareable_constant_value: experimental_everything C = { **{ k: v } } p C ``` The following error message is output. ``` ruby 3.4.0dev (2024-12-06T03:32:20Z master 6ccaa37eb3) [arm64-darwin23] -- raw disasm-------- trace: 1 0000 putspecialobject 1 ( 2) 0002 putobject :k ( 2) 0004 putself ( 2) 0005 opt_send_without_block <calldata:v, 0> ( 2) 0007 newhash 2 ( 2) 0009 newhash 2 ( 2) 0011 opt_send_without_block <calldata:make_shareable, 1> ( 2) 0013 putspecialobject 3 ( 2) * 0015 setconstant :C ( 2) trace: 1 0017 putself ( 3) 0018 opt_getconstant_path [:C] ( 3) 0020 opt_send_without_block <calldata:p, 1> ( 3) 0022 leave ( 3) --------------------- -e:2: argument stack underflow (-1) -e: compile error (SyntaxError) ``` It does not occur in the following cases: ```ruby # shareable_constant_value: experimental_everything C = { **{ k: v } } ``` ```ruby C = { **{ k: v } } p C ``` ```ruby # shareable_constant_value: experimental_everything { **{ k: v } } p C ``` ```ruby # shareable_constant_value: experimental_everything C = { **{ k: 1 } } p C ``` This does not occur in Ruby 3.3: ``` ❯ ruby --parser=parse.y -v -e '# shareable_constant_value: experimental_everything C = { **{ foo: bar } } p C' ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [arm64-darwin23] -e:3:in `<main>': undefined local variable or method `bar' for main (NameError) C = { **{ foo: bar } } ^^^ ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:118182] [Ruby master Bug#20526] File.open(encoding: "bom|utf-8") converts "\r\n" to "\n" on Windows
by kou (Kouhei Sutou) 26 Dec '24

26 Dec '24

Issue #20526 has been reported by kou (Kouhei Sutou). ---------------------------------------- Bug #20526: File.open(encoding: "bom|utf-8") converts "\r\n" to "\n" on Windows https://bugs.ruby-lang.org/issues/20526 * Author: kou (Kouhei Sutou) * Status: Open * Target version: 3.2 * ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I'm not sure whether this is an intentional behavior but it seems that `encoding: "utf-8"` doesn't change newline conversion but `encoding: "bom|utf-8"` changes newline conversion: ```ruby File.write("a.txt", "a\r\n") File.read("a.txt").bytes # => [97, 13, 10] File.open("a.txt", encoding: "utf-8") {|f| f.read.bytes} # => [97, 10, 10] File.open("a.txt", encoding: "bom|utf-8") {|f| f.read.bytes} # => [97, 10] XXX: \r\n -> \n File.open("a.txt", encoding: "bom|utf-8", universal_newline: false) {|f| f.read.bytes} # => [97, 13, 10] ``` Note that the `XXX: ` line the above codes. Is this an intentional behavior? -- https://bugs.ruby-lang.org/

3 2

[ruby-core:120183] [Ruby master Misc#20946] Proposing tomoya ishida (@tompng) as a Ruby committer
by matsuda (Akira Matsuda) 26 Dec '24

26 Dec '24

Issue #20946 has been reported by matsuda (Akira Matsuda). ---------------------------------------- Misc #20946: Proposing tomoya ishida (@tompng) as a Ruby committer https://bugs.ruby-lang.org/issues/20946 * Author: matsuda (Akira Matsuda) * Status: Open * Assignee: matz (Yukihiro Matsumoto) ---------------------------------------- I'd like to propose tomoya ishida (@tompng) as a Ruby committer. * As a maintainer of IRB / Reline since 2023, he has been continuously improving the REPL's stability and performance. Also, he authored type-based completion mechanism which is going to be included by default in the upcoming Ruby 3.4. * For some reasons, he is extraordinarily good at finding trivial and tricky syntax bugs. He already has reported massive numbers of very weird bugs to the new parsers. https://github.com/ruby/prism/issues/created_by/tompng I believe, keeping having such unique inputs from him would help improving the parsers and the language syntax in a way that only he can do. * He triggered and implemented Ruby 3.4's Hash#inspect improvement https://bugs.ruby-lang.org/issues/20433 https://github.com/ruby/ruby/pull/10924 * In fact ruby's repo already includes some more pieces of code written by him that show how talented programmer he is. (e.g. https://github.com/ruby/ruby/blob/master/sample/trick2022/01-tompng/entry.rb * He won this year's "Ruby Prize", which means he is considered an individual who made the most notable achievement for the community this year. https://rubyprize.jp/ -- https://bugs.ruby-lang.org/

7 9

[ruby-core:120189] [Ruby master Misc#20947] Propose ydah (Yudai Takada) as a Ruby committer
by yui-knk (Kaneko Yuichiro) 26 Dec '24

26 Dec '24

Issue #20947 has been reported by yui-knk (Kaneko Yuichiro). ---------------------------------------- Misc #20947: Propose ydah (Yudai Takada) as a Ruby committer https://bugs.ruby-lang.org/issues/20947 * Author: yui-knk (Kaneko Yuichiro) * Status: Open ---------------------------------------- I’d like to propose ydah (https://bugs.ruby-lang.org/users/52290) as a Ruby committer. He has contributed to parse.y. His contribution includes applying Lrama’s new feature to parse.y (https://github.com/ruby/ruby/pull/10050) fixing invalid syntax error (https://github.com/ruby/ruby/pull/11850) modification of ripper related codes (https://github.com/ruby/ruby/pull/10602) and so on. He fixes not only parse.y but also compile.c (https://github.com/ruby/ruby/pull/12275) He is passionate about making Ruby’s parser better. Then I want to make his contribution easy. I will be a mentor for him. -- https://bugs.ruby-lang.org/

4 6

[ruby-core:119881] [Ruby master Feature#20884] reserve "Ruby" toplevel module for Ruby language
by Dan0042 (Daniel DeLorme) 25 Dec '24

25 Dec '24

Issue #20884 has been reported by Dan0042 (Daniel DeLorme). ---------------------------------------- Feature #20884: reserve "Ruby" toplevel module for Ruby language https://bugs.ruby-lang.org/issues/20884 * Author: Dan0042 (Daniel DeLorme) * Status: Open ---------------------------------------- `Ruby` would be a convenient namespace for many features of the Ruby language, in particular APIs related to the interpreter. All these constants: RUBY_VERSION RUBY_RELEASE_DATE RUBY_PLATFORM RUBY_PATCHLEVEL RUBY_REVISION RUBY_COPYRIGHT RUBY_ENGINE RUBY_ENGINE_VERSION RUBY_DESCRIPTION would have made a lot of sense as `Ruby::Version` etc. `Thread::Backtrace::Location` would have made a lot of sense as `Ruby::Backtrace::Location` `RubyVM` is considered specific to CRuby; so `RubyVM::AbstractSyntaxTree` should be `Ruby::AbstractSyntaxTree` if it is meant to be present in other implementations. In #6648 there's a bit of contention over where `ruby_args` should be. `RubyVM`, `RbConfig`, `Process` have all been proposed, but `Ruby` would be an excellent choice. `Process.argv0` was added in Ruby 2.1 but the `Process` namespace is really about OS-level process control (fork, signals, euid, limits) while this argv0 is not (in `ps` it's neither value of COMMAND nor CMD) so it would have made sense as `Ruby.argv0` The "ruby" gem name is reserved, so there's no conflict. https://rubygems.org/gems/ruby All in all, "Ruby" is an appropriate namespace for many Ruby things. We don't want to break compatibility over this, but we could at least start small by reserving the namespace, and see how it grows from there. module Ruby Version = ::RUBY_VERSION end -- https://bugs.ruby-lang.org/

7 7

[ruby-core:120403] Ruby 3.4.0 Released
by NARUSE, Yui 25 Dec '24

25 Dec '24

We are pleased to announce the release of Ruby 3.4.0. Ruby 3.4 adds `it` block parameter reference, change Prism as default parser, adds Happy Eyeballs Version 2 support to socket library, improves YJIT, adds Modular GC, and so on. ## `it` is introduced `it` is added to reference a block parameter with no variable name. [[Feature #18980]] ```ruby ary = ["foo", "bar", "baz"] p ary.map { it.upcase } #=> ["FOO", "BAR", "BAZ"] ``` `it` very much behaves the same as `_1`. When the intention is to only use `_1` in a block, the potential for other numbered parameters such as `_2` to also appear imposes an extra cognitive load onto readers. So `it` was introduced as a handy alias. Use `it` in simple cases where `it` speaks for itself, such as in one-line blocks. ## Prism is now the default parser Switch the default parser from parse.y to Prism. [[Feature #20564]] This is an internal improvement and there should be little change visible to the user. If you notice any compatibility issues, please report them to us. To use the conventional parser, use the command-line argument `--parser=parse.y`. ## The socket library now features Happy Eyeballs Version 2 (RFC 8305) The socket library now features [Happy Eyeballs Version 2 (RFC 8305)](https://datatracker.ietf.org/doc/html/rfc8305) the latest standardized version of a widely adopted approach for better connectivity in many programming languages, in `TCPSocket.new` (`TCPSocket.open`) and `Socket.tcp`. This improvement enables Ruby to provide efficient and reliable network connections, adapted to modern internet environments. Until Ruby 3.3, these methods performed name resolution and connection attempts serially. With this algorithm, they now operate as follows: 1. Performs IPv6 and IPv4 name resolution concurrently 2. Attempt connections to the resolved IP addresses, prioritizing IPv6, with parallel attempts staggered at 250ms intervals 3. Return the first successful connection while canceling any others This ensures minimized connection delays, even if a specific protocol or IP address is delayed or unavailable. This feature is enabled by default, so additional configuration is not required to use it. To disable it globally, set the environment variable `RUBY_TCP_NO_FAST_FALLBACK=1` or call `Socket.tcp_fast_fallback=false`. Or to disable it on a per-method basis, use the keyword argument `fast_fallback: false`. ## YJIT ### TL;DR * Better performance across most benchmarks on both x86-64 and arm64 platforms. * Reduced memory usage through compressed metadata and a unified memory limit. * Various bug fixes: YJIT is now more robust and thoroughly tested. ### New features * Command-line options * `--yjit-mem-size` introduces a unified memory limit (default 128MiB) to track total YJIT memory usage, providing a more intuitive alternative to the old `--yjit-exec-mem-size` option. * `--yjit-log` enables a compilation log to track what gets compiled. * Ruby API * `RubyVM::YJIT.log` provides access to the tail of the compilation log at run-time. * YJIT stats * `RubyVM::YJIT.runtime_stats` now always provides additional statistics on invalidation, inlining, and metadata encoding. ### New optimizations * Compressed context reduces memory needed to store YJIT metadata * Allocate registers for local variables and Ruby method arguments * When YJIT is enabled, use more Core primitives written in Ruby: * `Array#each`, `Array#select`, `Array#map` rewritten in Ruby for better performance [[Feature #20182]]. * Ability to inline small/trivial methods such as: * Empty methods * Methods returning a constant * Methods returning `self` * Methods directly returning an argument * Specialized codegen for many more runtime methods * Optimize `String#getbyte`, `String#setbyte` and other string methods * Optimize bitwise operations to speed up low-level bit/byte manipulation * Support shareable constants in multi-ractor mode * Various other incremental optimizations ## Modular GC * Alternative garbage collector (GC) implementations can be loaded dynamically through the modular garbage collector feature. To enable this feature, configure Ruby with `--with-modular-gc` at build time. GC libraries can be loaded at runtime using the environment variable `RUBY_GC_LIBRARY`. [[Feature #20351]] * Ruby's built-in garbage collector has been split into a separate file at `gc/default/default.c` and interacts with Ruby using an API defined in `gc/gc_impl.h`. The built-in garbage collector can now also be built as a library using `make modular-gc MODULAR_GC=default` and enabled using the environment variable `RUBY_GC_LIBRARY=default`. [[Feature #20470]] * An experimental GC library is provided based on [MMTk](https://www.mmtk.io/). This GC library can be built using `make modular-gc MODULAR_GC=mmtk` and enabled using the environment variable `RUBY_GC_LIBRARY=mmtk`. This requires the Rust toolchain on the build machine. [[Feature #20860]] ## Language changes * String literals in files without a `frozen_string_literal` comment now emit a deprecation warning when they are mutated. These warnings can be enabled with `-W:deprecated` or by setting `Warning[:deprecated] = true`. To disable this change, you can run Ruby with the `--disable-frozen-string-literal` command line argument. [[Feature #20205]] * Keyword splatting `nil` when calling methods is now supported. `**nil` is treated similarly to `**{}`, passing no keywords, and not calling any conversion methods. [[Bug #20064]] * Block passing is no longer allowed in index. [[Bug #19918]] * Keyword arguments are no longer allowed in index. [[Bug #20218]] * The toplevel name `::Ruby` is reserved now, and the definition will be warned when `Warning[:deprecated]`. [[Feature #20884]] ## Core classes updates Note: We're only listing notable updates of Core class. * Exception * `Exception#set_backtrace` now accepts an array of `Thread::Backtrace::Location`. `Kernel#raise`, `Thread#raise` and `Fiber#raise` also accept this new format. [[Feature #13557]] * GC * `GC.config` added to allow setting configuration variables on the Garbage Collector. [[Feature #20443]] * GC configuration parameter `rgengc_allow_full_mark` introduced. When `false` GC will only mark young objects. Default is `true`. [[Feature #20443]] * Ractor * `require` in Ractor is allowed. The requiring process will be run on the main Ractor. `Ractor._require(feature)` is added to run requiring process on the main Ractor. [[Feature #20627]] * `Ractor.main?` is added. [[Feature #20627]] * `Ractor.[]` and `Ractor.[]=` are added to access the ractor local storage of the current Ractor. [[Feature #20715]] * `Ractor.store_if_absent(key){ init }` is added to initialize ractor local variables in thread-safty. [[Feature #20875]] * Range * `Range#size` now raises `TypeError` if the range is not iterable. [[Misc #18984]] ## Standard Library updates Note: We're only listing notable updates of Standard librarires. * RubyGems * Add `--attestation` option to gem push. It enabled to store signature to [sigstore.dev] * Bundler * Add a `lockfile_checksums` configuration to include checksums in fresh lockfiles * Add bundle lock `--add-checksums` to add checksums to an existing lockfile * JSON * Performance improvements of `JSON.parse` about 1.5 times faster than json-2.7.x. * Tempfile * The keyword argument `anonymous: true` is implemented for Tempfile.create. `Tempfile.create(anonymous: true)` removes the created temporary file immediately. So applications don't need to remove the file. [[Feature #20497]] * win32/sspi.rb * This library is now extracted from the Ruby repository to [ruby/net-http-sspi]. [[Feature #20775]] ## Compatibility issues Note: Excluding feature bug fixes. * Error messages and backtrace displays have been changed. * Use a single quote instead of a backtick as a opening quote. [[Feature #16495]] * Display a class name before a method name (only when the class has a permanent name). [[Feature #19117]] * `Kernel#caller`, `Thread::Backtrace::Location`'s methods, etc. are also changed accordingly. ``` Old: test.rb:1:in `foo': undefined method `time' for an instance of Integer from test.rb:2:in `<main>' New: test.rb:1:in 'Object#foo': undefined method 'time' for an instance of Integer from test.rb:2:in '<main>' ``` * Hash#inspect rendering have been changed. [[Bug #20433]] * Symbol keys are displayed using the modern symbol key syntax: `"{user: 1}"` * Other keys now have spaces around `=>`: `'{"user" => 1}'`, while previously they didn't: `'{"user"=>1}'` * Kernel#Float() now accepts a decimal string with decimal part omitted. [[Feature #20705]] ```rb Float("1.") #=> 1.0 (previously, an ArgumentError was raised) Float("1.E-1") #=> 0.1 (previously, an ArgumentError was raised) ``` * String#to_f now accepts a decimal string with decimal part omitted. Note that the result changes when an exponent is specified. [[Feature #20705]] ```rb "1.".to_f #=> 1.0 "1.E-1".to_f #=> 0.1 (previously, 1.0 was returned) ``` * Refinement#refined_class has been removed. [[Feature #19714]] ## Standard library compatibility issues * DidYouMean * `DidYouMean::SPELL_CHECKERS[]=` and `DidYouMean::SPELL_CHECKERS.merge!` are removed. * Net::HTTP * Removed the following deprecated constants: * `Net::HTTP::ProxyMod` * `Net::NetPrivate::HTTPRequest` * `Net::HTTPInformationCode` * `Net::HTTPSuccessCode` * `Net::HTTPRedirectionCode` * `Net::HTTPRetriableCode` * `Net::HTTPClientErrorCode` * `Net::HTTPFatalErrorCode` * `Net::HTTPServerErrorCode` * `Net::HTTPResponseReceiver` * `Net::HTTPResponceReceiver` These constants were deprecated from 2012. * Timeout * Reject negative values for Timeout.timeout. [[Bug #20795]] * URI * Switched default parser to RFC 3986 compliant from RFC 2396 compliant. [[Bug #19266]] ## C API updates * `rb_newobj` and `rb_newobj_of` (and corresponding macros `RB_NEWOBJ`, `RB_NEWOBJ_OF`, `NEWOBJ`, `NEWOBJ_OF`) have been removed. [[Feature #20265]] * Removed deprecated function `rb_gc_force_recycle`. [[Feature #18290]] ## Miscellaneous changes * Passing a block to a method which doesn't use the passed block will show a warning on verbose mode (`-w`). [[Feature #15554]] * Redefining some core methods that are specially optimized by the interpeter and JIT like `String.freeze` or `Integer#+` now emits a performance class warning (`-W:performance` or `Warning[:performance] = true`). [[Feature #20429]] See [NEWS](https://github.com/ruby/ruby/blob/v3_4_0/NEWS.md) or [commit logs](https://github.com/ruby/ruby/compare/v3_3_0...v3_4_0) for more details. With those changes, [4942 files changed, 202244 insertions(+), 255528 deletions(-)](https://github.com/ruby/ruby/compare/v3_3_0...v3_4_0#file_buc… since Ruby 3.3.0! Merry Christmas, Happy Holidays, and enjoy programming with Ruby 3.4! ## Download * <https://cache.ruby-lang.org/pub/ruby/3.4/ruby-3.4.0.tar.gz> SIZE: 23153022 SHA1: 8ccb561848a7c460ae08e1a120a47c4a88a79335 SHA256: 068c8523442174bd3400e786f4a6952352c82b1b9f6210fd17fb4823086d3379 SHA512: bc70ecba27d1cdea00879f03487cad137a7d9ab2ad376cfb7a65780ad14da637fa3944eeeede2c04ab31eeafb970c64ccfeeb854c99c1093937ecc1165731562 * <https://cache.ruby-lang.org/pub/ruby/3.4/ruby-3.4.0.tar.xz> SIZE: 17215572 SHA1: eb25447cc404e8d2e177c62550d0224ebd410e68 SHA256: 0081930db22121eb997207f56c0e22720d4f5d21264b5907693f516c32f233ca SHA512: 776a2cf3e9ccc77c27500240f168aa3e996b0c7c1ee1ef5a7afc291a06c118444016fde38b5b139c0b800496b8eb1b5456562d833f0edc0658917164763b1af7 * <https://cache.ruby-lang.org/pub/ruby/3.4/ruby-3.4.0.zip> SIZE: 28310193 SHA1: 26254ca5d3decc28a4e5faec255995265e5270b5 SHA256: c120228038af04554f6363e716b0a32cbf53cf63c6adf9f2c22a24f43dc8b555 SHA512: 4d535ed10db76a6aa74f8a025df319deb28483a7a781c24045906ee7663f1cff9d9f9e71dbc993c9e050113a34b37c7fa2143c355a0a6e1e1029bf2c92213ecc ## What is Ruby Ruby was first developed by Matz (Yukihiro Matsumoto) in 1993, and is now developed as Open Source. It runs on multiple platforms and is used all over the world especially for web development. [Feature #13557]: https://bugs.ruby-lang.org/issues/13557 [Feature #15554]: https://bugs.ruby-lang.org/issues/15554 [Feature #16495]: https://bugs.ruby-lang.org/issues/16495 [Feature #18290]: https://bugs.ruby-lang.org/issues/18290 [Feature #18980]: https://bugs.ruby-lang.org/issues/18980 [Misc #18984]: https://bugs.ruby-lang.org/issues/18984 [Feature #19117]: https://bugs.ruby-lang.org/issues/19117 [Bug #19266]: https://bugs.ruby-lang.org/issues/19266 [Feature #19714]: https://bugs.ruby-lang.org/issues/19714 [Bug #19918]: https://bugs.ruby-lang.org/issues/19918 [Bug #20064]: https://bugs.ruby-lang.org/issues/20064 [Feature #20182]: https://bugs.ruby-lang.org/issues/20182 [Feature #20205]: https://bugs.ruby-lang.org/issues/20205 [Bug #20218]: https://bugs.ruby-lang.org/issues/20218 [Feature #20265]: https://bugs.ruby-lang.org/issues/20265 [Feature #20351]: https://bugs.ruby-lang.org/issues/20351 [Feature #20429]: https://bugs.ruby-lang.org/issues/20429 [Feature #20443]: https://bugs.ruby-lang.org/issues/20443 [Feature #20470]: https://bugs.ruby-lang.org/issues/20470 [Feature #20497]: https://bugs.ruby-lang.org/issues/20497 [Feature #20564]: https://bugs.ruby-lang.org/issues/20564 [Bug #20620]: https://bugs.ruby-lang.org/issues/20620 [Feature #20627]: https://bugs.ruby-lang.org/issues/20627 [Feature #20705]: https://bugs.ruby-lang.org/issues/20705 [Feature #20715]: https://bugs.ruby-lang.org/issues/20715 [Feature #20775]: https://bugs.ruby-lang.org/issues/20775 [Bug #20795]: https://bugs.ruby-lang.org/issues/20795 [Feature #20860]: https://bugs.ruby-lang.org/issues/20860 [Feature #20875]: https://bugs.ruby-lang.org/issues/20875 [Feature #20884]: https://bugs.ruby-lang.org/issues/20884 [sigstore.dev]: sigstore.dev [ruby/net-http-sspi]: https://github.com/ruby/net-http-sspi

1 0

[ruby-core:120279] [Ruby master Feature#20959] Add a way to get codepage of console.
by YO4 (Yoshinao Muramatsu) 25 Dec '24

25 Dec '24

Issue #20959 has been reported by YO4 (Yoshinao Muramatsu). ---------------------------------------- Feature #20959: Add a way to get codepage of console. https://bugs.ruby-lang.org/issues/20959 * Author: YO4 (Yoshinao Muramatsu) * Status: Open ---------------------------------------- # Abstract Add a way to retrieve code pages of console. On Windows, Encoding.find("locale") returns the console codepage. To prepare for future changes, specify ```console``` instead of ```locale``` to get the encoding when a console code page is needed. # Background On Windows, Encoding.find("locale") returns the console codepage. This is different from locale in other environments. Also the name and content do not seem to match. In the future, if we change the locale_encoding of the Windows port to the locale codepage of the C runtime library, we need to get the encoding for the console. Strings received via pipe from cmd.exe or powershell are encoded in the console codepage. This would be necessary when communicating with other programs via pipes. # Proposal Make Encoding.find("console") return the encoding that represents the console codepage. # Background(continued) Since Windows 10, UTF-8 support seems to be enhanced in the commandline environment. * build 17134 ucrt supports UTF-8 locale codepage ```setlocale(LC_CTYPE, ".utf8")``` Windows support UTF-8 ANSI Codepage(experimental) ```Beta: Use Unicode UTF-8 for worldwide language support``` * build 18362 support ```Set a process code page to UTF-8``` via manifest. * build 19041 Time zone name holds in wchar_t internaly, so proper tzname can be obtained regardless of the Windows language setting. This should not be all. Through these, I feel that the Microsoft team recommends the use of utf-8. It also seems that Microsoft, which has previously depricated the ANSI version of the API, is treating it as a valid alternative to use in UTF-8. If it becomes widespread on Windows for libraries to respect the C runtime library locale, ruby would do better to follow it. Of course, it is good to be able to get the encoding of the console by specifying a console. # Future plan By reducing differences from other platforms, bugs and extra code are hoped to be reduced. * Do ```setlocale(LC_CTYPE, ".utf8");``` in main.c (or refer LC_* environment variables). * ```Encoding.find("locale")``` returns C runtime library locale. Since strings obtained from the Windows system have a Unicode code range, the API to obtain fixed UTF-8 encoding remains unchanged. # Discussion The code page we can get from Windows also has ACP and OEMCP, but are these necessary? Is it reasonable to get locale_encoding if Encoding.find("console") is called on other platforms? -- https://bugs.ruby-lang.org/

2 1

[ruby-core:120325] [Ruby master Bug#20970] `it /1/i` raises undefined method 'it' for main (NoMethodError) even if binding.local_variables includes `it`
by tompng (tomoya ishida) 24 Dec '24

24 Dec '24

Issue #20970 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20970: `it /1/i` raises undefined method 'it' for main (NoMethodError) even if binding.local_variables includes `it` https://bugs.ruby-lang.org/issues/20970 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-12-19T07:16:12Z master 335bba0fde) +PRISM [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `it` parameter became a local variable with #20965, but it does not behave like local variable with `--parser=prism` ~~~ruby i=2 42.tap do p it # 42 p local_variables # [:it, :i] p it /1/i # should be 21, got NoMethodError end ~~~ It prints `42`, `[:it, :i], `21` with `--parser=parse.y` -- https://bugs.ruby-lang.org/

6 10

[ruby-core:118243] [Ruby master Feature#20564] Switch default parser to Prism
by kddnewton (Kevin Newton) 24 Dec '24

24 Dec '24

Issue #20564 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Feature #20564: Switch default parser to Prism https://bugs.ruby-lang.org/issues/20564 * Author: kddnewton (Kevin Newton) * Status: Open ---------------------------------------- This issue is to propose switching the default parser for Ruby to Prism on the master branch. The main goal is to get this default into both master and the next preview so that the community has plenty of time to test before the eventual 3.4 release. Matz has indicated that the final decision about the default parser for Ruby 3.4 will be made later in the year. Decisions about the default parser for Ruby 3.5+ and the relationship between Prism, lrama, parse.y, etc. will also be made at a later time. ## Background For some background on the Prism project itself, here are a couple of links in chronological order that you can read: * https://bugs.ruby-lang.org/issues/19741 * https://railsatscale.com/2023-06-12-rewriting-the-ruby-parser/ * https://kddnewton.com/2024/01/23/prism.html * https://railsatscale.com/2024-04-16-prism-in-2024/ ## Status Prism now passes all Ruby tests and specs. Internally at Shopify, it also passes our Core monolith's CI, which is a large and complex Rails application. We are confident that it is ready for production use. External to CRuby, Prism has already been adopted extensively around the ecosystem, including but not limited to: * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java) * [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java) * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby) * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM) * [repl_type_completor](https://github.com/ruby/repl_type_completor) * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-pa… (via parser translator) * [ruby-lsp](https://github.com/Shopify/ruby-lsp) * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator) Fortunately with the community adoption, we have been able to find many edge cases and bugs that have been fixed. Thank you so much to everyone who has tried, used, reported problems with, and improved Prism in the last couple of years. We would not have hit this milestone without this community involvement. ## Caveats While Prism correctly accepts and parses all valid Ruby syntax, there are still a number of cases where it does not reject invalid syntax as it should. We are currently working on these cases, and we are confident that we can eliminate them before the next preview release. You can find them listed here: https://github.com/ruby/prism/issues?q=is%3Aopen+is%3Aissue+label%3Ainvalid…. This is our top priority right now, and I anticipate these being resolved quickly. The other caveat is unknown unknowns: bugs or syntax issues that we have yet to discover. At Shopify we are currently taking numerous steps to mitigate this risk, including: * Prism currently passes all Ruby tests and specs, and will continue to going forward. The tests and specs are extensive, and we are confident this gets us almost all of the way there. * Grammar-based fuzzing to ensure we handle all possible syntax variations (both valid and invalid). As an example, that recently led to the discovery that the current compiler was segfaulting on safe navigation in for loops (https://bugs.ruby-lang.org/issues/20468) It has also found several memory leaks that have been fixed. * Mutation testing of existing source files (by adding and deleting random bytes) to find additional edge cases. Our goal is 100% compatibility, and we believe this is achievable through the above techniques and others. The goal of this proposal is to allow more companies to test their codebase against Prism, and to make this easier. We believe that we can sort out the remaining problems much more quickly in that manner. ## Implementation The implementation will be relatively straightforward, as there is already the `--parser` flag that can be used to switch between the current parser and Prism. We will change the default to be `"prism"` instead of `"parse.y"`. There is no plan to remove or change any code related to `parse.y`, `lrama`, or `compile.c` at this time, as we want to create the minimal amount of change possible to ensure stability. -- https://bugs.ruby-lang.org/

4 9

[ruby-core:120378] [Ruby master Bug#20979] +comment does not affect parsetree without `--parser=parse.y`
by znz (Kazuhiro NISHIYAMA) 24 Dec '24

24 Dec '24

Issue #20979 has been reported by znz (Kazuhiro NISHIYAMA). ---------------------------------------- Bug #20979: +comment does not affect parsetree without `--parser=parse.y` https://bugs.ruby-lang.org/issues/20979 * Author: znz (Kazuhiro NISHIYAMA) * Status: Open * Target version: 3.4 * ruby -v: ruby 3.4.0dev (2024-12-23T02:52:14Z master b19fcd02fa) +PRISM [arm64-darwin24] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- `ruby --help` says `+comment Add comments to AST (affects parsetree).`, but it does not affect without `--parser=parse.y`. If it is intentional, I think more help messages or more explanation somewhere. ``` Dump List: insns Instruction sequences. yydebug yydebug of yacc parser generator. parsetree Abstract syntax tree (AST). -optimize Disable optimization (affects insns). +error-tolerant Error-tolerant parsing (affects yydebug, parsetree). +comment Add comments to AST (affects parsetree). ``` ``` % git diff --no-index --stat =(ruby --dump=parsetree -e 0) =(ruby --dump=parsetree+comment -e 0) % git diff --no-index --stat =(ruby --parser=parse.y --dump=parsetree -e 0) =(ruby --parser=parse.y --dump=parsetree+comment -e 0) /tmp/{zshEGciGv => zshWgqfVT} | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) ``` -- https://bugs.ruby-lang.org/

2 1