January 2024 - ruby-core - ml.ruby-lang.org

[ruby-core:116494] Invitation to RubyKaigi 2024
by Akira Matsuda 30 Jan '24

30 Jan '24

Hello, ruby-core readers! Let me announce that registration for RubyKaigi 2024 is now open! And of course as usual, all Ruby core team members (who have direct commit access to the Git repo) are invited to the conference with free admission. Please proceed to our ticketing site from this URL: https://ti.to/rubykaigi/2024/with/nebxjc5wena For "Committer account name" field, please fill in your account name listed on the committer email list, https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml SVN account name, Redmine account name, or something equivalent that identifies you as a Ruby core committer. Also, RubyKaigi wants talk proposals from true Ruby hackers like you who are reading this message. Our CFP form is already open, and will be closed tomorrow! (in JST) So if you have anything in your mind that you want to share with the audience there, please submit it NOW! https://cfp.rubykaigi.org/events/2024 -- Akira Matsuda<ronnie(a)dio.jp>

1 0

[ruby-core:115888] [Ruby master Bug#20083] Regexp#match? behaving inconsistently with Ruby 3.3.0
by jussikos (Jussi Koljonen) 30 Jan '24

30 Jan '24

Issue #20083 has been reported by jussikos (Jussi Koljonen). ---------------------------------------- Bug #20083: Regexp#match? behaving inconsistently with Ruby 3.3.0 https://bugs.ruby-lang.org/issues/20083 * Author: jussikos (Jussi Koljonen) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin23] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- From irb, when calling String#match? ``` pattern = /([\s]*ABC)$/i p "1ABC".match?(pattern) # => true p "12ABC".match?(pattern) # => true p "123ABC".match?(pattern) # => true p "1231ABC".match?(pattern) # => true p "12312ABC".match?(pattern) # => false p "123123ABC".match?(pattern) # => false p "1231231ABC".match?(pattern) # => true p "12312312ABC".match?(pattern) # => true p "123123123ABC".match?(pattern) # => false p "1231231231ABC".match?(pattern) # => false p "12312312312ABC".match?(pattern) # => true p "123123123123ABC".match?(pattern) # => true p "1231231231231ABC".match?(pattern) # => false ``` With `ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22]` and earlier versions (2.7.8 to 3.2.2) return value is always `true` -- https://bugs.ruby-lang.org/

3 2

[ruby-core:115904] [Ruby master Bug#20086] Windows Memory Mapped Buffer is buggy.
by ioquatix (Samuel Williams) 30 Jan '24

30 Jan '24

Issue #20086 has been reported by ioquatix (Samuel Williams). ---------------------------------------- Bug #20086: Windows Memory Mapped Buffer is buggy. https://bugs.ruby-lang.org/issues/20086 * Author: ioquatix (Samuel Williams) * Status: Closed * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED ---------------------------------------- We confirmed a bug in memory mapped files on Windows. See https://github.com/ruby/ruby/pull/9358 for the fix. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116345] [Ruby master Feature#20196] Proposal: Binary data literal
by ziggythehamster (Keith Gable) 29 Jan '24

29 Jan '24

Issue #20196 has been reported by ziggythehamster (Keith Gable). ---------------------------------------- Feature #20196: Proposal: Binary data literal https://bugs.ruby-lang.org/issues/20196 * Author: ziggythehamster (Keith Gable) * Status: Open * Priority: Normal ---------------------------------------- I sometimes find myself needing to write some bytes in a Ruby string literal, and this experience leaves a lot to be desired: * Bare strings don't work (potential for encoding corruption unless you remember to `.force_encoding` and never copy-paste just the literal into somewhere else) and are not particularly pleasant given all of the backslashes * Wrapping this in `String.new("\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00`\x00\x00\x00`\b\x06\x00\x00\x00\xE2\x98w8\x00\x000%IDAT", encoding: 'BINARY')` is better, but many tools explode with this because they expect all strings to be valid UTF-8 even if they're an argument to String.new, and it still doesn't have the "beauty" one might expect from Ruby (also it's not frozen unless you also freeze it) * `["9805e474d0a0a1a0000000d094844425000000060000000680600000002e8977830000035294441445"].pack("h*")` parses in all tools and is less harsh to look at, but if you're writing binary data, you probably want to annotate it Here's my basic syntax proposal: ``` %b[ 89504e470d0a1a0a # PNG header 0000000d # Length = 13 bytes 49484452 # IHDR chunk 00000060 # Width = 96px 00000060 # Height = 96px 08 06 # 8bpp RGBA 00 00 00 # deflate / no filter / non-interlaced ] # => "\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00`\x00\x00\x00`\b\x06\x00\x00\x00" ``` More formally: * To match the nibble ordering of a regular string escape, the hex characters are high nibble first (the same as the `H` unpack character). * It follows the same rules as other percent literals, and I am flexible on what character is used. I chose `b` because `h` could be confusing paired with the `h`/`H` unpack characters and the inverted meaning. * We could say that high-nibble-first is capitalized and the lower-case version is low-nibble-first, but I imagine most people will want high-nibble-first. We could also say that `%b[]` returns a `String` but `%B[]` returns an `IO::Buffer`, which has greater utility than having the capability of writing low-nibble-first literals * Whitespace is ignored * Comments are allowed * The encoding is always `Encoding::BINARY` * The resulting string is always frozen (and if `%B[]` means buffer then that is read-only as well) * a-f can also be written A-F Things to consider: * Interpolation could be allowed like normal strings * Embedding strings could be allowed (example below) * `?` literals (characters) should be handled identically to how other kinds of strings are embedded if that is allowed * If interpolation is allowed and you interpolate a number, this should either interpolate `.to_s` as you would expect in a string or raise an error, because there is no unsurprising way to take a number and convert it to one or more bytes * Strings encoded as `Encoding::BINARY` could have their `.inspect` output use this literal * When dealing with bitmasks, it's often convenient to write them out in binary instead of hex so the on bits are easier to identify, but there is no syntax for that here that I am fond of... but someone might have an idea. I thought about `.00001111` or `!00001111` with mandatory whitespace before resuming hex characters, but those didn't feel right to me Example with embedded strings: ``` %b[ 89 "PNG" 0d0a1a0a # PNG header 0000000d # Length = 13 bytes "IHDR" # IHDR chunk 00000060 # Width = 96px 00000060 # Height = 96px 08 06 # 8bpp RGBA 00 00 00 # deflate / no filter / non-interlaced ] ``` Example with interpolation: ``` %b[ #{png_header} #{ihdr = chunk(:ihdr, width: 96, height: 96, bpp: 8, format: :rgba)} #{png_crc(ihdr)} # I didn't include this in the other examples but I needed something to demonstrate here ] ``` Other possible alternatives: * A library (possibly standard library/gem) could have a function like `binary` take a string (potentially a heredoc) and parse it according to the same rules I wrote above. You would have to make the parser strip whitespace and comments, and only hex bytes could be interpolated. * A new pack/unpack symbol could be created that does the same thing as above, so you could `["hex #comments\netc"].pack("...")` * You could probably do a lot of this with an array of hex strings and `pack` but it doesn't allow for freeform whitespace and the way you do it is not obvious without reading the docs for `pack` ... and also you allocate a bunch of strings you don't need * A `Data`-like object more closely related to `IO::Buffer` could be defined that declares the size of things contained within a buffer and then a constant could be written to create an instance of the Data-subclass containing the actual data you want to write out ... but this is a lot of work Potential users: * People writing protocol code in Ruby * People who need to write out magic constants (in my case: the RDB encoding of a Redis value) * People using something like Metasploit Framework to reverse engineer something * Tools could e.g. disassemble x86 into a literal with comments showing the assembly mnemonics -- https://bugs.ruby-lang.org/

5 6

[ruby-core:116483] [Ruby master Bug#20223] For simple objects each_cons appears to work as I expect, but for other objects it seems to fail to terminate iterations when I would expect.
by mjflynt (Jeffrey Flynt) 29 Jan '24

29 Jan '24

Issue #20223 has been reported by mjflynt (Jeffrey Flynt). ---------------------------------------- Bug #20223: For simple objects each_cons appears to work as I expect, but for other objects it seems to fail to terminate iterations when I would expect. https://bugs.ruby-lang.org/issues/20223 * Author: mjflynt (Jeffrey Flynt) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- Reproduce process: ``` ruby p `ruby -v` class X include Enumerable attr_accessor :obj def initialize = @obj = (1..5).to_a def each(&block) = yield @obj.each(&block) end wtf = X.new (1..5).each_cons(3) { |g| p g } p '~' * 20 (1..5).to_a.each_cons(3) { |g| p g } p '~' * 20 ('a'..'e').each_cons(3) { |g| p g } p '~' * 20 wtf.each_cons(3) { |g| p g } ``` Result of reproduce process ``` "ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [aarch64-linux]\n" [1, 2, 3] [2, 3, 4] [3, 4, 5] "~~~~~~~~~~~~~~~~~~~~" [1, 2, 3] [2, 3, 4] [3, 4, 5] "~~~~~~~~~~~~~~~~~~~~" ["a", "b", "c"] ["b", "c", "d"] ["c", "d", "e"] "~~~~~~~~~~~~~~~~~~~~" [1, 2, 3] [2, 3, 4] [3, 4, 5] [4, 5, [1, 2, 3, 4, 5]] [Done] exited with code=0 in 0.173 seconds ``` Expected result and the reason why you expect I expect the results to not include the last line of output "[4, 5, [1, 2, 3, 4, 5]]" because my understanding is each_cons ends its iteration over the collection before it runs out of elements. In this case, this fourth row does not have enough elements to do three consecutive elements, but it is doing so anyway, appearing to use the original array as the third element before it completes. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116473] [Ruby master Bug#20221] ASAN: make test-basic: un-prefixed symbol leakage
by leeN (David Klein) 29 Jan '24

29 Jan '24

Issue #20221 has been reported by leeN (David Klein). ---------------------------------------- Bug #20221: ASAN: make test-basic: un-prefixed symbol leakage https://bugs.ruby-lang.org/issues/20221 * Author: leeN (David Klein) * Status: Open * Priority: Normal * ruby -v: ruby 3.4.0dev (2024-01-29T08:16:49Z master 8bff7e996c) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When building and running the tests (here, test-basic) with ASAN enabled, it fails with the following message: ``` Checking leaked global symbols...leaked __odr_asan_gen_rb_cArray __odr_asan_gen_ruby_digitmap __odr_asan_gen_rb_mComparable ...snip... __odr_asan_gen_OnigEncodingASCII __odr_asan_gen_OnigEncodingUS_ASCII __odr_asan_gen_OnigEncodingUTF_8 232 un-prefixed symbols leaked ``` To reproduce: ``` shell git checkout master mkdir build && cd build CXX=clang++ CC=clang ../configure cppflags="-fsanitize=address -fno-omit-frame-pointer" optflags=-O0 LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" ASAN_OPTIONS=use_sigaltstack=0:detect_leaks=0:abort_on_error=1 make ASAN_OPTIONS=use_sigaltstack=0:detect_leaks=0:abort_on_error=1 make test-basic ``` To fix: ``` diff --git a/tool/leaked-globals b/tool/leaked-globals index ee75f78d1d..9e78228274 100755 --- a/tool/leaked-globals +++ b/tool/leaked-globals @@ -79,6 +79,7 @@ next unless n.sub!(/^#{SYMBOL_PREFIX}/o, "") next if n.include?(".") next if !so and n.start_with?("___asan_") + next if n.start_with?("__odr_asan") case n when /\A(?:Init_|InitVM_|pm_|[Oo]nig|dln_|coroutine_)/ next ``` I'm not overly familiar with Ruby, so this might not be the preferred approach, but it made the issue go away for me. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116466] [Ruby master Bug#20220] M:N threading needs to tell ASAN about stack switches
by kjtsanaktsidis (KJ Tsanaktsidis) 28 Jan '24

28 Jan '24

Issue #20220 has been reported by kjtsanaktsidis (KJ Tsanaktsidis). ---------------------------------------- Bug #20220: M:N threading needs to tell ASAN about stack switches https://bugs.ruby-lang.org/issues/20220 * Author: kjtsanaktsidis (KJ Tsanaktsidis) * Status: Open * Priority: Normal * Assignee: kjtsanaktsidis (KJ Tsanaktsidis) * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- When switching stacks via `coroutine_transfer` in the M:N threading information, we need to tell ASAN about it in the same way that we do in cont.c. This is a matter of calling `__sanitizer_start_switch_fiber` in the transferring M:N thread, and then calling `__sanitizer_finish_switch_fiber` in the transferred-to M:N thread. -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116463] [Ruby master Bug#20219] Segfault with circular parameter
by kddnewton (Kevin Newton) 27 Jan '24

27 Jan '24

Issue #20219 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20219: Segfault with circular parameter https://bugs.ruby-lang.org/issues/20219 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby def foo(bar: (1 in ^bar)); end ``` Segfaults on master. (I don't think anyone would ever write this code.) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:116458] [Ruby master Bug#20216] Circular parameter reference not checked for assignment
by kddnewton (Kevin Newton) 27 Jan '24

27 Jan '24

Issue #20216 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20216: Circular parameter reference not checked for assignment https://bugs.ruby-lang.org/issues/20216 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- I wanted to check if this was a bug or desired. Usually when you reference a parameter within its own default value (optional positional or optional keyword) it raises a syntax error. This is unless you use it in an assignment, in which case it is not checked. Is this desired? Here's an example: ```ruby def foo(bar = bar = 1); end def foo(bar: bar = 1); end ``` I'm not sure what this code is supposed to be doing. I expected it to raise a syntax error. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:116459] [Ruby master Bug#20217] void value not checked for begin/ensure/rescue
by kddnewton (Kevin Newton) 26 Jan '24

26 Jan '24

Issue #20217 has been reported by kddnewton (Kevin Newton). ---------------------------------------- Bug #20217: void value not checked for begin/ensure/rescue https://bugs.ruby-lang.org/issues/20217 * Author: kddnewton (Kevin Newton) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN ---------------------------------------- ```ruby foo = begin return ensure end ``` I would expect this to raise a syntax error for a void value expression being assigned, but the `ensure` makes it so that this isn't checked. -- https://bugs.ruby-lang.org/

1 0