June 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:118268] [Ruby master Bug#20569] Instruction sequence generated from `shareable_constant_value: literal` files may not be serialized to binary
by byroot (Jean Boussier) 15 Jun '24

15 Jun '24

Issue #20569 has been reported by byroot (Jean Boussier). ---------------------------------------- Bug #20569: Instruction sequence generated from `shareable_constant_value: literal` files may not be serialized to binary https://bugs.ruby-lang.org/issues/20569 * Author: byroot (Jean Boussier) * Status: Open * Assignee: ko1 (Koichi Sasada) * Backport: 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- ```ruby iseq = RubyVM::InstructionSequence.compile(<<~'RUBY') # shareable_constant_value: literal REGEXP = /#{}/ RUBY iseq.to_binary ``` ``` in `to_binary': ibf_dump_object_unsupported: 0x00000001026ac8b0 [3LM ] T_ICLASS (NotImplementedError) ``` The reason is that the ISeq contains `putobject RubyVM::FrozenCore` so `compile.c` need some special handling of that hidden class. Proposed patch: https://github.com/ruby/ruby/pull/10951 -- https://bugs.ruby-lang.org/

3 3

[ruby-core:116838] [Ruby master Bug#20280] Error at invalid encoding symbols
by nobu (Nobuyoshi Nakada) 15 Jun '24

15 Jun '24

Issue #20280 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20280: Error at invalid encoding symbols https://bugs.ruby-lang.org/issues/20280 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED ---------------------------------------- Parsing an invalid encoding symbol raises an `EncodingError`. It must be a `SyntaxError`. -- https://bugs.ruby-lang.org/

4 4

[ruby-core:118315] [Ruby master Bug#20579] `Ripper.tokenize("1\n ")` does not include trailing spaces
by tompng (tomoya ishida) 13 Jun '24

13 Jun '24

Issue #20579 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20579: `Ripper.tokenize("1\n ")` does not include trailing spaces https://bugs.ruby-lang.org/issues/20579 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-06-13T09:49:46Z master 8b843b0dc7) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ~~~ruby # trailing :on_sp, :on_ignored_nl are included in the result Ripper.tokenize("1\n\t \t\n\t \t") # => ["1", "\n", "\t \t", "\n", "\t \t"] # trailing :on_sp disappears Ripper.tokenize("1\n\t \t") # => ["1", "\n"] ~~~ Found with `Ripper.tokenize(random_string).join != random_string` while investigating https://bugs.ruby-lang.org/issues/20578 -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118314] [Ruby master Bug#20578] Tokenizing string literal that have newline and invalid escape is wrong
by tompng (tomoya ishida) 13 Jun '24

13 Jun '24

Issue #20578 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #20578: Tokenizing string literal that have newline and invalid escape is wrong https://bugs.ruby-lang.org/issues/20578 * Author: tompng (tomoya ishida) * Status: Open * ruby -v: ruby 3.4.0dev (2024-06-13T09:49:46Z master 8b843b0dc7) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Tokenizing string literal that have newline and invalid escape is wrong When a string literal includes `\n` and an invalid escape after it, tokenize result gets wrong. ~~~ruby Ripper.tokenize "\"hello\\x world" # => ["\"", "hello\\x", " world"] # looks good Ripper.tokenize "\"\nhello\\x world" # => ["\"", "\n world", "hello\\x"] # order is reversed ~~~ These invalid escapes also gets wrong ~~~ruby Ripper.tokenize("\"\n\\Cxx\"") #=> ["\"", "\nx", "\\Cx", "\""] Ripper.tokenize("\"\n\\Mxx\"") #=> ["\"", "\nx", "\\Mx", "\""] Ripper.tokenize("\"\n\\c\\cx\"") #=> ["\"", "\nx", "\\c\\c", "\""] Ripper.tokenize("\"\n\\ux\"") #=> ["\"", "\nx", "\""] Ripper.tokenize("\"\n\\xx\"") #=> ["\"", "\nx", "\\x", "\""] ~~~ And these literals also gets wrong ~~~ruby Ripper.tokenize("<<A\n\n\\xyz") #=> ["<<A", "\n", "\nyz", "\\x"] Ripper.tokenize("%(\n\\xyz)") #=> ["%(", "\nyz", "\\x", ")"] Ripper.tokenize("%Q(\n\\xyz)") #=> ["%Q(", "\nyz", "\\x", ")"] Ripper.tokenize(":\"\n\\xyz\"") #=> [":\"", "\nyz", "\\x", "\""] ~~~ I encountered this while typing a valid string literal into IRB ~~~ruby irb(main):001> " irb(main):002> \x█ ~~~ Other invalid escape sequence that disappears from tokenize result ~~~ruby Ripper.tokenize('"\u{123') # => ["\""] ~~~ -- https://bugs.ruby-lang.org/

1 0

[ruby-core:118267] [Ruby master Misc#20568] Look for test.rb in the current working directory
by eightbitraptor (Matthew Valentine-House) 12 Jun '24

12 Jun '24

Issue #20568 has been reported by eightbitraptor (Matthew Valentine-House). ---------------------------------------- Misc #20568: Look for test.rb in the current working directory https://bugs.ruby-lang.org/issues/20568 * Author: eightbitraptor (Matthew Valentine-House) * Status: Open ---------------------------------------- [Github PR link](https://github.com/ruby/ruby/pull/10950) The `test.rb` script used by `make run` etc is explicitly located in the `$srcdir` - the location of the Ruby source code. The [Building Ruby](https://docs.ruby-lang.org/en/master/contributing/building_ruby_md.ht… documentation recommends using out-of-source builds. Requiring `test.rb` to be located in the source directory prevents me from having multiple build directories, with a distinct `test.rb` per build, instead forcing pollution of the source directory. I propose looking for `test.rb` in the current working directory. This will maintain compatibility with in-source builds, and provide extra flexibility for those of us using out-of-source builds. There will be a small one time impact for Ruby committers, who will have to move their existing `test.rb` files into their build directories. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:118302] [Ruby master Bug#20577] FREE_AT_EXIT segfaulting after 32683aa1
by mdalessio (Mike Dalessio) 12 Jun '24

12 Jun '24

Issue #20577 has been reported by mdalessio (Mike Dalessio). ---------------------------------------- Bug #20577: FREE_AT_EXIT segfaulting after 32683aa1 https://bugs.ruby-lang.org/issues/20577 * Author: mdalessio (Mike Dalessio) * Status: Open * ruby -v: ruby 3.4.0dev (2024-06-11T14:33:51Z master 32683aa18d) [x86_64-linux] * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- After commit 32683aa1 on master, using `RUBY_FREE_AT_EXIT` results in a segfault. When run with valgrind, many invalid reads are reported. Repro with ``` RUBY_FREE_AT_EXIT=1 bundle exec /path/to/ruby -e 'puts RUBY_DESCRIPTION' ``` or for valgrind details ``` RUBY_FREE_AT_EXIT=1 bundle exec valgrind /path/to/ruby -e 'puts RUBY_DESCRIPTION' ``` Typical example valgrind error is: ``` ==441955== Invalid read of size 4 ==441955== at 0x2C6F97: hash_table_index (id_table.c:132) ==441955== by 0x2C6F97: rb_id_table_lookup (id_table.c:230) ==441955== by 0x31AB50: cached_callable_method_entry (vm_method.c:1415) ==441955== by 0x31AB50: callable_method_entry_or_negative (vm_method.c:1485) ==441955== by 0x31C4EE: callable_method_entry (vm_method.c:1522) ==441955== by 0x31C4EE: rb_callable_method_entry (vm_method.c:1529) ==441955== by 0x18F370: obj_any_hash (hash.c:222) ==441955== by 0x19F444: any_hash (hash.c:204) ==441955== by 0x19F444: rb_any_hash (hash.c:260) ==441955== by 0x19F444: ar_do_hash (hash.c:418) ==441955== by 0x19F444: ar_delete (hash.c:1093) ==441955== by 0x19F444: rb_hash_stlike_delete (hash.c:2385) ==441955== by 0x1A0CCB: rb_hash_delete_entry (hash.c:2402) ==441955== by 0x2C59B6: rb_gc_free_dsymbol (symbol.c:851) ==441955== by 0x17AC53: obj_free (gc.c:3460) ==441955== by 0x17D045: rb_objspace_call_finalizer_i (gc.c:4352) ==441955== by 0x17D045: gc_each_object (gc.c:4302) ==441955== by 0x17D045: rb_objspace_call_finalizer (gc.c:4401) ==441955== by 0x162719: rb_ec_finalize (eval.c:164) ==441955== by 0x162719: rb_ec_cleanup (eval.c:255) ==441955== by 0x15CE09: rb_main (main.c:40) ==441955== by 0x15CE09: main (main.c:59) ==441955== Address 0x4ec4610 is 0 bytes inside a block of size 24 free'd ==441955== at 0x484988F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==441955== by 0x175BD1: objspace_xfree (gc.c:12153) ==441955== by 0x175BD1: objspace_xfree (gc.c:12084) ==441955== by 0x175BD1: ruby_sized_xfree.constprop.0 (gc.c:12257) ==441955== by 0x17A9A0: obj_free (gc.c:3293) ==441955== by 0x17D045: rb_objspace_call_finalizer_i (gc.c:4352) ==441955== by 0x17D045: gc_each_object (gc.c:4302) ==441955== by 0x17D045: rb_objspace_call_finalizer (gc.c:4401) ==441955== by 0x162719: rb_ec_finalize (eval.c:164) ==441955== by 0x162719: rb_ec_cleanup (eval.c:255) ==441955== by 0x15CE09: rb_main (main.c:40) ==441955== by 0x15CE09: main (main.c:59) ==441955== Block was alloc'd at ==441955== at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==441955== by 0x189B0A: objspace_xmalloc0 (gc.c:11938) ==441955== by 0x2C6EA6: rb_id_table_create (id_table.c:98) ==441955== by 0x31AD99: cache_callable_method_entry (vm_method.c:1444) ==441955== by 0x31AD99: callable_method_entry_or_negative (vm_method.c:1502) ==441955== by 0x32A7DE: callable_method_entry (vm_method.c:1522) ==441955== by 0x32A7DE: vm_respond_to (vm_method.c:2878) ==441955== by 0x3305E3: check_funcall_respond_to (vm_eval.c:602) ==441955== by 0x3305E3: rb_check_funcall_default_kw (vm_eval.c:676) ==441955== by 0x20E896: convert_type_with_id (object.c:3131) ==441955== by 0x20E896: rb_check_convert_type_with_id (object.c:3223) ==441955== by 0x1672A7: rb_stat (file.c:1299) ==441955== by 0x16E15E: rb_file_exist_p (file.c:1805) ==441955== by 0x16E15E: rb_check_realpath_internal (file.c:4567) ==441955== by 0x1D6639: get_loaded_features_index (load.c:415) ==441955== by 0x1D6FB7: rb_provide_feature (load.c:703) ==441955== by 0x1A2629: rb_call_inits (inits.c:64) ==441955== by 0x162B67: ruby_setup (eval.c:86) ==441955== by 0x162C1C: ruby_init (eval.c:98) ==441955== by 0x15CDF7: rb_main (main.c:39) ==441955== by 0x15CDF7: main (main.c:59) ``` I'm attaching a full log from my system. cc @peterzhu2118 ---Files-------------------------------- bug.log (238 KB) -- https://bugs.ruby-lang.org/

2 5

[ruby-core:116790] [Ruby master Bug#20270] Options with `--parser=prism`
by nobu (Nobuyoshi Nakada) 11 Jun '24

11 Jun '24

Issue #20270 has been reported by nobu (Nobuyoshi Nakada). ---------------------------------------- Bug #20270: Options with `--parser=prism` https://bugs.ruby-lang.org/issues/20270 * Author: nobu (Nobuyoshi Nakada) * Status: Open * Priority: Normal * Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED ---------------------------------------- ### `--dump` option Currently `parsetree` and `prism_parsetree` bits are separated, but it seems meaningless as far as `--parser` option selects only one parser. Why doesn't simply `--dump=parsetree` dump AST by parse.y, or PRISM AST if `--parser=prism` is given? Actually, `ruby --dump=parsetree --parser=prism` just segfaults, because these bits and `--parser` option are separately handled. ### streaming code from stdin ```sh-session $ echo end | ruby --parser=prism - ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker. -: warning: Prism support for streaming code from stdin is not currently supported ``` Nothing reported is fine, since it is not currently supported. I was a bit curious that it looks like trying something after the warning. ```C if (strcmp(opt->script, "-") == 0) { int xflag = opt->xflag; VALUE rb_source = open_load_file(opt->script_name, &xflag); opt->xflag = xflag != 0; rb_warn("Prism support for streaming code from stdin is not currently supported"); error = pm_parse_string(&result, rb_source, opt->script_name); } ``` Note that `open_load_file` returns `rb_stdin` when `-` is given. This object is a `RFile`, not a `RString`, and `RSTRING_PTR(rb_source)` accesses after the object boundary. It should just bail out, not try obviously wrong thing. ### Fix https://github.com/nobu/ruby/tree/options-refactor -- https://bugs.ruby-lang.org/

2 3

[ruby-core:118154] [Ruby master Bug#20521] Memory leak in Ripper parsing
by kddnewton (Kevin Newton) 11 Jun '24

11 Jun '24

Issue #20521 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20521: Memory leak in Ripper parsing https://bugs.ruby-lang.org/issues/20521 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Currently, this code is leaking: ```ruby require "ripper" source = <<~'RUBY' <<-A; %w[j\ i A j] RUBY 10.times do 100_000.times do Ripper.sexp_raw(source) end puts `ps -o rss= -p #{$$}` end ``` This results in: ``` 31584 36304 41504 47008 51712 56384 61088 65792 70480 75200 ``` I haven't managed to track it down, but I can see that 3.2.2 does not leak but 3.3.0 does leak. -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118173] [Ruby master Misc#20524] mswin and gmp
by MSP-Greg (Greg L) 11 Jun '24

11 Jun '24

Issue #20524 has been reported by MSP-Greg (Greg L). ---------------------------------------- Misc #20524: mswin and gmp https://bugs.ruby-lang.org/issues/20524 * Author: MSP-Greg (Greg L) * Status: Open ---------------------------------------- Currently, the `vcpkg.json` file exists, and it can be used to specify what MSFT/vcpkg packages to install, and also a repo commit, which will determine the what package versions are used. The file lists the gmp package. 1. To use the gmp package in mswin, `win32/configure.bat` shows an option of `--with-gmp`. This option is not used in the `windows.yml` workflow file, nor is it mentioned in `doc/windows.md`. 2. When using the option, the build fails when compiling `bignum.c`, as it can’t find `gmp.h`. It appears the vcpkg include folder is not being passed to `cl`? So, it seems a few things are amiss. `gmp` was added a long time ago. With current mswin builds, will `gmp` improve `Bignum`’s performance? Is that unknown? If there are benchmarks to determine this, I could certainly run them… -- https://bugs.ruby-lang.org/

3 2

[ruby-core:118273] [Ruby master Bug#20572] Compilation hangs with nested until
by kddnewton (Kevin Newton) 11 Jun '24

11 Jun '24

Issue #20572 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20572: Compilation hangs with nested until https://bugs.ruby-lang.org/issues/20572 * Author: kddnewton (Kevin Newton) * Status: Open * Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby until while 1 end end ``` this code causes an infinite loop in the compiler. -- https://bugs.ruby-lang.org/

2 2