October 2023 - ruby-core - ml.ruby-lang.org

[ruby-core:114942] [Ruby master Bug#19909] s390x zlib different deflate algorithm producing a different compressed byte stream causing test failures
by jaruga (Jun Aruga) 04 Oct '23

04 Oct '23

Issue #19909 has been reported by jaruga (Jun Aruga). ---------------------------------------- Bug #19909: s390x zlib different deflate algorithm producing a different compressed byte stream causing test failures https://bugs.ruby-lang.org/issues/19909 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I faced the test failures ruby/zlib (zlib gem) in `make test-all` and ruby/spec (`make test-spec`) in Ubuntu version jammy (22.04) s390x. I intend to manage this issue on this ticket, and giving the space for users to comment about the issue. ## The test failures First, below are the test failures by `make test-all` and `make test-spec` on Ubuntu jammy s390x. Here is the used zlib version. ``` $ dpkg -s "$(dpkg -S /usr/include/zlib.h | cut -d ":" -f 1)" | grep ^Version Version: 1:1.2.11.dfsg-2ubuntu9.2 ``` ``` + make -s test-all -o exts TESTOPTS=-j3 -q --tty=no RUBYOPT=-w generating enc.mk making enc making srcs under enc generating transdb.h transdb.h unchanged making trans making encs Run options: --seed=42873 "--ruby=./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=../test/.excludes --name=!/memory_leak/ -j3 -q --tty=no # Running tests: Retrying... 1) Failure: TestZlibDeflate#test_deflate_chunked [/home/travis/build/junaruga/ruby/test/zlib/test_zlib.rb:66]: <7253> expected but was <8325>. 2) Failure: TestZlibDeflate#test_deflate_chunked_break [/home/travis/build/junaruga/ruby/test/zlib/test_zlib.rb:92]: <3632> expected but was <4702>. 3) Failure: TestZlibGzipReader#test_unused2 [/home/travis/build/junaruga/ruby/test/zlib/test_zlib.rb:968]: <24> expected but was <23>. 4) Failure: TestZlib#test_gzip [/home/travis/build/junaruga/ruby/test/zlib/test_zlib.rb:1419]: <"\x1F\x8B\b\x00\x00\x00\x00\x00\x00\xFFK\xCB\xCF\a\x00!es\x8C\x03\x00\x00\x00"> expected but was <"\x1F\x8B\b\x00\x00\x00\x00\x00\x00\xFFJ\xCB\xCF\a\f\x00!es\x8C\x03\x00\x00\x00">. 5) Failure: TestZlib#test_deflate_stream [/home/travis/build/junaruga/ruby/test/zlib/test_zlib.rb:1411]: <20016> expected but was <21085>. Finished tests in 290.303048s, 88.4145 tests/s, 21049.4999 assertions/s. 25667 tests, 6110734 assertions, 5 failures, 0 errors, 172 skips ``` ``` $SETARCH make -s test-spec ... 1) Zlib::Deflate.deflate deflates some data FAILED Expected "x\x9Cc\x80\x03\x00\x00 \x00\x01" == "x\x9Cc`\x80\x01\x00\x00 \x00\x01" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:10:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:4:in `<top (required)>' 2) Zlib::Deflate.deflate deflates lots of data FAILED Expected "x\x9Cc\x18\xE1`\xA4\x83\x91\x0Eh\rF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x91\x0E\x06;\x00\x00\x80\x00\x00\x01" == "x\x9C\xED\xC1\x01\x01\x00\x00\x00\x80\x90\xFE\xAF\xEE\b \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x18\x80\x00\x00\x01" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:18:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:4:in `<top (required)>' 3) Zlib::Deflate.deflate deflates chunked data FAILED Expected 21085 == 20016 to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:32:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:4:in `<top (required)>' 4) Zlib::Deflate#deflate deflates some data FAILED Expected "x\x9Cc\x80\x03\x00\x00 \x00\x01" == "x\x9Cc`\x80\x01\x00\x00 \x00\x01" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:47:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:36:in `<top (required)>' 5) Zlib::Deflate#deflate deflates lots of data FAILED Expected "x\x9Cc\x18\xE1`\xA4\x83\x91\x0EF:\x18\xE9`\xA4\x83\x81\x06#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\x8Ct0\xD2\xC1H\a#\x1D\f4\x00\x00\x80\x00\x00\x01" == "x\x9C\xED\xC1\x01\x01\x00\x00\x00\x80\x90\xFE\xAF\xEE\b \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x18\x80\x00\x00\x01" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:56:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:36:in `<top (required)>' 6) Zlib::Deflate#deflate without break deflates chunked data with final chunk FAILED Expected 8325 == 7253 to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:96:in `block (3 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:68:in `<top (required)>' 7) Zlib::Deflate#deflate with break deflates chunked data with final chunk FAILED Expected 4702 == 3632 to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:123:in `block (3 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/deflate_spec.rb:68:in `<top (required)>' 8) Zlib::Deflate#set_dictionary sets the dictionary FAILED Expected "x\xBB\x14\xE1\x03\xCBJLJNIMK\xCF\xC8\xCC\x02\f\x00\x15\x86\x03\xF8" == "x\xBB\x14\xE1\x03\xCBKLJNIMK\xCF\xC8\xCC\x02\x00\x15\x86\x03\xF8" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/set_dictionary_spec.rb:10:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate/set_dictionary_spec.rb:4:in `<top (required)>' 9) Zlib.deflate deflates some data FAILED Expected "x\x9C3\x84\x03\x00 \x91\x01\xEB" == "x\x9C34\x84\x01\x00 \x91\x01\xEB" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate_spec.rb:6:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/deflate_spec.rb:4:in `<top (required)>' 10) Zlib.gzip gzips the given string FAILED Expected "24261MLJNI\x05\f\x00\x9D\x05\x00$ \x00\x00\x00" == "34261MLJNI\x05\x00\x9D\x05\x00$ \x00\x00\x00" to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzip_spec.rb:13:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzip_spec.rb:4:in `<top (required)>' 11) Zlib::GzipWriter#write writes some compressed data FAILED Expected [50, 52, 50, 54, 49, 77, 76, 74, 78, 73, 5, 12, 0, 157, 5, 0, 36, 10, 0, 0, 0] == [51, 52, 50, 54, 49, 77, 76, 74, 78, 73, 5, 0, 157, 5, 0, 36, 10, 0, 0, 0] to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzipwriter/write_spec.rb:19:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzipwriter/write_spec.rb:5:in `<top (required)>' 12) Zlib::GzipWriter#write handles inputs of 2^23 bytes FAILED Expected 34263 == 8176 to be truthy but was false /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzipwriter/write_spec.rb:34:in `block (2 levels) in <top (required)>' /home/travis/build/junaruga/ruby/spec/ruby/library/zlib/gzipwriter/write_spec.rb:5:in `<top (required)>' Finished in 114.477929 seconds 3715 files, 32609 examples, 190709 expectations, 12 failures, 0 errors, 0 tagged ``` ## The root cause This issue can happen with zlib library applying [the patch zlib/pull#410](https://github.com/madler/zlib/pull/410) that is to enable a different deflate algorithm producing a different compressed byte stream causing the test failures. I am not sure how a Ubuntu zlib deb package is developed in Ubuntu. However I was able to download and check the source file `zlib_1.2.11.dfsg-2ubuntu9.2.debian.tar.xz` on [the Ubuntu jammy-updates zlib deb package page](https://packages.ubuntu.com/jammy-updates/zlib1g). debian/rules ``` ... 49 # s390x fails at compatibility. 50 ifneq (,$(findstring $(DEB_HOST_ARCH), s390x)) 51 m32=-m31 52 CFLAGS += -DDFLTCC_LEVEL_MASK=0x7e 53 CONFIGURE_COMMON += --dfltcc 54 CONFIGURE_HOST += --crc32-vx 55 else 56 m32=-m32 57 endif ... ``` debian/patches/series ``` ... 4 410.patch ... ``` debian/patches/410.patch ``` From 992a7afc3edfa511dff0650d1c545b11bf64e655 Mon Sep 17 00:00:00 2001 From: Ilya Leoshkevich <iii(a)linux.ibm.com> Date: Wed, 18 Jul 2018 13:14:07 +0200 Subject: [PATCH] Add support for IBM Z hardware-accelerated deflate ... ``` And the 410.patch is the [the patch zlib/pull#410](https://github.com/madler/zlib/pull/410) above. ## A reproducer For convenience, here is a relatively small reproducer. ``` $ cat test/zlib/_test_zlib_test_deflate_chunked.rb require 'zlib' z = Zlib::Deflate.new input = "\x01" z.deflate(input) do |chunk| end final = z.finish puts "final.length: #{final.length}" ``` Below is ok case. ``` $ ruby -I./lib test/zlib/_test_zlib_test_deflate_chunked.rb final.length: 9 ``` Below is the case with the zlib where the tests are failing in s390x. ``` $ ruby -I./lib test/zlib/_test_zlib_test_deflate_chunked.rb final.length: 10 ``` ## Affected distributions and versions Affected: at least Ubuntu jammy 22.04 I didn't see this issue in Ubuntu focal (20.04). The same `410.patch` is applied in both Ubuntu focal and jammy. But how to build is different. In Ubuntu jammy, the zlib is configured by `./configure --dfltcc`. ``` $ diff -u ubuntu_focal/zlib1g/debian/rules ubuntu_jammy/zlib1g/debian/rules --- ubuntu_focal/zlib1g/debian/rules 2020-08-20 01:52:59.000000000 +0200 +++ ubuntu_jammy/zlib1g/debian/rules 2021-08-12 05:28:03.000000000 +0200 @@ -21,6 +21,9 @@ LDFLAGS = `dpkg-buildflags --get LDFLAGS` EXTRA_MAKE = +CONFIGURE_COMMON=--shared --prefix=/usr +CONFIGURE_HOST=--libdir=\$${prefix}/lib/$(DEB_HOST_MULTIARCH) + # binutils doesn't supply the prefixed version normally like GCC does so # we can't just unconditionally use DEB_HOST_GNU_TYPE-ar ifeq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) @@ -46,8 +49,9 @@ # s390x fails at compatibility. ifneq (,$(findstring $(DEB_HOST_ARCH), s390x)) m32=-m31 -CFLAGS += -DDFLTCC -EXTRA_MAKE += OBJA=dfltcc.o PIC_OBJA=dfltcc.lo +CFLAGS += -DDFLTCC_LEVEL_MASK=0x7e +CONFIGURE_COMMON += --dfltcc +CONFIGURE_HOST += --crc32-vx else m32=-m32 endif @@ -95,7 +99,7 @@ if [ ! -f Makefile.stash ]; then cp Makefile Makefile.stash ; fi - AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc" CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" uname=GNU ./configure --shared --prefix=/usr --libdir=\$${prefix}/lib/$(DEB_HOST_MULTIARCH) + AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc" CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" uname=GNU ./configure $(CONFIGURE_COMMON) $(CONFIGURE_HOST) touch $@ @@ -106,7 +110,7 @@ cp -r $(COPYLIST) debian/64 cd debian/64 && AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc $(m64)" \ CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" \ - uname=GNU ./configure --shared --prefix=/usr --libdir=\$${prefix}/usr/lib64 + uname=GNU ./configure $(CONFIGURE_COMMON) --libdir=\$${prefix}/usr/lib64 touch $@ configure32-stamp: configure @@ -116,7 +120,7 @@ cp -r $(COPYLIST) debian/32 cd debian/32 && AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc $(m32)" \ CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" \ - uname=GNU ./configure --shared --prefix=/usr --libdir=\$${prefix}/usr/lib32 + uname=GNU ./configure $(CONFIGURE_COMMON) --libdir=\$${prefix}/usr/lib32 touch $@ configuren32-stamp: configure @@ -126,7 +130,7 @@ cp -r $(COPYLIST) debian/n32 cd debian/n32 && AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc $(mn32)" \ CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" \ - uname=GNU ./configure --shared --prefix=/usr --libdir=\$${prefix}/usr/lib32 + uname=GNU ./configure $(CONFIGURE_COMMON) --libdir=\$${prefix}/usr/lib32 touch $@ configurex32-stamp: configure @@ -136,7 +140,7 @@ cp -r $(COPYLIST) debian/x32 cd debian/x32 && AR=$(AR) CC="$(DEB_HOST_GNU_TYPE)-gcc $(mx32)" \ CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" \ - uname=GNU ./configure --shared --prefix=/usr --libdir=\$${prefix}/usr/libx32 + uname=GNU ./configure $(CONFIGURE_COMMON) --libdir=\$${prefix}/usr/libx32 touch $@ build: build-stamp $(EXTRA_BUILD) ``` I also didn't see this issue with [the zlib RPM package](https://src.fedoraproject.org/rpms/zlib) in Fedora rawhide (Fedora 40), though I see the zlib is configured by `./configure --dfltcc` applying the patch `zlib-*-IBM-Z-hw-accelerated-deflate.patch`. But when comparing the `zlib-1.2.13-IBM-Z-hw-accelerated-deflate.patch` in the zlib in Fedora rawhide (Fedora 40), and `410.patch` in Ubuntu jammy-updates. It is quite different. ## Workaround The patch author commented that a workaround is to set environment variable `DFLTCC=0` to disable the different deflate algorithm at [zlib/issues#60](https://github.com/ruby/zlib/issues/60#issuecomment-1733149…. ## How I fixed the issue with the workaround * ruby/zlib: https://github.com/ruby/zlib/pull/65 * ruby/spec: https://github.com/ruby/spec/pull/1088 * ruby/ruby: https://github.com/ruby/ruby/pull/8401 * RubyCI - s390x (Ubuntu), the server is actually Ubuntu jammy, and the cron job is running with the `DFLTCC=0` in the crontab. ## How can we do? The [upstream patch zlib/pull#410](https://github.com/madler/zlib/pull/410) is not merged to the zlib repository yet. And I am not sure if we want to do like this, modifying the `common.mk` (and `configure.ac`). ``` diff --git a/common.mk b/common.mk index b8ae911ef2..2605569443 100644 --- a/common.mk +++ b/common.mk @@ -982,6 +982,7 @@ test-spec: $(TEST_RUNNABLE)-test-spec yes-test-spec: yes-test-spec-precheck $(ACTIONS_GROUP) $(gnumake_recursive)$(Q) \ + DFLTCC=0 \ $(RUNRUBY) -r./$(arch)-fake -r$(tooldir)/rubyspec_temp \ $(srcdir)/spec/mspec/bin/mspec run -B $(srcdir)/spec/default.mspec $(MSPECOPT) $(SPECOPTS) $(ACTIONS_ENDGROUP) ``` So, it seems to me that just documenting this issue and workaround somewhere in ruby/ruby is a good fix for this issue. -- https://bugs.ruby-lang.org/

1 1

[ruby-core:114822] [Ruby master Bug#19893] OpenStruct#respond_to? when true fails with NameError
by msxavi (Emerson Xavier) 04 Oct '23

04 Oct '23

Issue #19893 has been reported by msxavi (Emerson Xavier). ---------------------------------------- Bug #19893: OpenStruct#respond_to? when true fails with NameError https://bugs.ruby-lang.org/issues/19893 * Author: msxavi (Emerson Xavier) * Status: Open * Priority: Normal * ruby -v: 3.1.4 * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- Hello, I have a case where I overrode respond_to? on a subclass of OpenStruct because I this class to work as fake object. ``` class Dep def a "Hello from Dep" end end class FakeDep < OpenStruct def respond_to?(*args) super || Dep.new.respond_to?(*args) end end FakeDep.new(a: []) NameError: undefined method `a' for class `FakeDep' owner = method!(name).owner ^^^^^^^ from /usr/local/lib/ruby/3.1.0/ostruct.rb:245:in `method' ``` Is this a valid issue? Also posted in https://github.com/ruby/ostruct/issues/55 -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114938] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by zdennis (Zach Dennis) 02 Oct '23

02 Oct '23

Issue #18915 has been updated by zdennis (Zach Dennis). File not-implemented-error-docs.patch added Whoops, last patch upload failed. Patch actually applied here. ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-104812 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (0 Bytes) not-implemented-error-docs.patch (1.57 KB) -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114935] [Ruby master Bug#18553] Memory leak on compiling method call with kwargs
by peterzhu2118 (Peter Zhu) 02 Oct '23

02 Oct '23

Issue #18553 has been updated by peterzhu2118 (Peter Zhu). Status changed from Open to Closed This was fixed in #19906, so I will close this issue. ---------------------------------------- Bug #18553: Memory leak on compiling method call with kwargs https://bugs.ruby-lang.org/issues/18553#change-104808 * Author: ibylich (Ilya Bylich) * Status: Closed * Priority: Normal * Assignee: ko1 (Koichi Sasada) * ruby -v: ruby 3.2.0dev * Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- The following code produces a memory leak: ```ruby p(foo: 1) ``` It comes from the allocation in `compile.c`: ```c struct rb_callinfo_kwarg *kw_arg = rb_xmalloc_mul_add(len, sizeof(VALUE), sizeof(struct rb_callinfo_kwarg)); ``` Later it's packed into `struct rb_callinfo`, but `imemo_callinfo` is a GC-managed object that has no `free` calls in `obj_free` function in gc.c. [I've tried to fix it](https://github.com/ruby/ruby/pull/5488/files#diff-d1cee85c3b0e24a64519c… by calling `free` on `callinfo->kwarg` and it fixed leak in `./miniruby -e 'p(foo: 1)'`, but it breaks `make install`. My addition causes a double-free error, looks like either `callinfo` or `callinfo->kwarg` is shared between multiple objects. `kwarg` field is a `const` pointer, so that's somewhat expected (I was under impression that `const` qualifier is redundant) :) I'm pretty sure old versions of Ruby are also affected by this memory leak. -- https://bugs.ruby-lang.org/

2 1

[ruby-core:114937] [Ruby master Feature#18915] New error class: NotImplementedYetError or scope change for NotImplementedError
by zdennis (Zach Dennis) 02 Oct '23

02 Oct '23

Issue #18915 has been updated by zdennis (Zach Dennis). File not-implemented-error-docs.patch added Attached is a patch to update the documentation for NotImplementedError to expand its scope beyond just missing features on the underlying platform. The current documentation has been in place since just before Ruby 1.9.2. Historically, it appears that Ruby hasn't limited its usage of NotImplementedError to errors that originate from features missing from the underlying platform (e.g. system calls) since at least 1.8.0. I believe the example provided in the current NotImplementedError documentation was meant as an example, not to suggest the only scope in which this error can be used. Ruby itself invalidates most of the interpretation provided in the Background section of this ticket. Here are just three examples: - [numeric.c](https://github.com/ruby/ruby/blob/50520cc1930331bccdb94730e17ddc… raises NotImplementedError when a class needs provide its own <=> method. - [delegate.rb](https://github.com/ruby/ruby/blob/c4d22d47f8bd019ae6df98fafb9d… raises NotImplementedError to indicate that a __getobj__ and __setobj__ need to provided by a subclass. - [tsort](https://github.com/ruby/ruby/blob/6dcd39997679914dad1c23f95adb3f7d84… uses this error and documents it expressly as: Should be implemented by a extended class. There are more examples to be found in Ruby as well. Outside of Ruby, this pattern is also commonly used as a way to communicate the intent to an application, framework, or library developer. Here are four examples from popular projects: - [rails uses this pattern](https://github.com/search?q=org%3Arails%20NotImplementedError&type… - [minitest uses this pattern](https://github.com/search?q=repo%3Aminitest%2Fminitest%20NotImplem… - [rspec uses this pattern](https://github.com/search?q=org%3Arspec%20NotImplementedError&type… - [dry-rb libraries use this pattern](https://github.com/search?q=org%3Adry-rb+NotImplementedError&type=… Additionally, even the latest Programming Ruby 3.2 book by PragPub uses an example of this in Chapter 6: Inheritance and Messages. ---------------------------------------- Feature #18915: New error class: NotImplementedYetError or scope change for NotImplementedError https://bugs.ruby-lang.org/issues/18915#change-104811 * Author: Quintasan (Michał Zając) * Status: Open * Priority: Normal ---------------------------------------- # Abstract Introduce `NotImplementedYetError` exception that should be used in case when the codepath has not been implemented by the developer for some reason (maybe they're designing an abstract class or are designing some sort of interface to reuse later on) OR extend the meaning of `NotImplementedError` to cover those usecases so we don't have to introduce another exception # Background `NotImplementedError` is supposed to be raised `if the underlying operating system or Ruby runtime does not support them` (https://ruby-doc.org/core-3.1.2/NotImplementedError.html) However it appears that many people are misusing this exception by raising this in a superclass from which they later inherit from. I do realize that Ruby promotes duck-typing (the default RuboCop style guide has a cop for this – https://github.com/rubocop/ruby-style-guide#duck-typing) However I have seen this being discussed numerous times: * https://github.com/rubocop/ruby-style-guide/issues/458 * http://chrisstump.online/2016/03/23/stop-abusing-notimplementederror/ * https://oleg0potapov.medium.com/ruby-notimplementederror-dont-use-it-dff1fd… * https://gitlab.com/gitlab-org/gitlab/-/issues/354314 (which I'm the author of) * https://github.com/rmosolgo/graphql-ruby/issues/2067 (here the author actually confused it with Python's `NotImplementedError`) * https://stackoverflow.com/questions/13668068/how-to-signal-not-implemented-… # Proposal Create `NotImplementedYetError` exception OR Allow raising `NotImplementedError` in cases other than OS or Ruby runtime incompatibilities # Evaluation ### Add `NotImplementedYetError` I think a new exception is a better idea than changing the usage of an existing one just because "everyone is using it". That said it would require people to refactor their code which might prevent wider adoption of the new exception. ### Change scope of `NotImplementedError` This would require the least amount of changes possible (only a documentation change) and I believe there would be no compatibility problems whatsoever. ---Files-------------------------------- not-implemented-error-docs.patch (0 Bytes) -- https://bugs.ruby-lang.org/

1 0

[ruby-core:114925] [Ruby master Bug#19906] fix kwarg memory leak
by HParker (Adam Hess) 01 Oct '23

01 Oct '23

Issue #19906 has been reported by HParker (Adam Hess). ---------------------------------------- Bug #19906: fix kwarg memory leak https://bugs.ruby-lang.org/issues/19906 * Author: HParker (Adam Hess) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- GitHub PR: https://github.com/ruby/ruby/pull/8556 ci info tracks kwargs and which are shared across aliases ```ruby 30.times do 100_000.times do def foo(k:); yield; end alias bar foo undef foo end puts `ps -o rss= -p #{$$}` end ``` before: ``` 30364 45120 59700 73932 88512 102680 116884 130884 145308 159688 174468 188680 203016 216908 231568 246096 260592 274900 289168 303420 317620 331956 346148 360708 374880 389172 403472 417824 432016 446040 460196 474604 489152 503248 517504 531944 546228 560520 574760 589108 ``` after: ``` 8760 8940 9156 9540 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9548 9556 9564 9564 9564 9564 9564 9564 9564 9564 9564 9564 9564 9564 9576 9744 9744 9744 9744 9744 9936 ``` -- https://bugs.ruby-lang.org/

2 2

[ruby-core:114933] [Ruby master Bug#10416] Create mechanism for updating of Unicode data files downstreams when we want
by nobu (Nobuyoshi Nakada) 01 Oct '23

01 Oct '23

Issue #10416 has been updated by nobu (Nobuyoshi Nakada). The current enc-unicode.rb seems to fail because of `Indic_Conjunct_break` properties with values. I'm not sure how these properties should be handled well. `/\p{InCB_Liner}/` or `/\p{InCB=Liner}/` as the comments in that file? https://github.com/nobu/ruby/tree/unicode-15.1 is the former. ---------------------------------------- Bug #10416: Create mechanism for updating of Unicode data files downstreams when we want https://bugs.ruby-lang.org/issues/10416#change-104804 * Author: duerst (Martin Dürst) * Status: Open * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) * ruby -v: ruby 2.2.0dev (2014-10-22 trunk 48092) [x86_64-cygwin] * Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN ---------------------------------------- The current mechanism for updating Unicode data files will create the following problem: Downstream compilers/packagers will download Unicode data files ONE time (they may already have done so). However, if they don't activate ALWAYS_UPDATE_UNICODE = yes, these files will never get updated, and they will stay on Unicode version 7.0 even if in five years Unicode is e.g. on version 12.0. On the other hand, if they activate ALWAYS_UPDATE_UNICODE = yes (and assuming issue #10415 gets fixed), they constantly update to the latest version of Unicode. That's good for those who actually want this, but now what our current policy is. What's missing is that we (Ruby core) can make sure downstream checkouts update to a new Unicode version when we want then to do so (as we e.g. can do for other parts that are based on Unicode data, see e.g. https://bugs.ruby-lang.org/issues/9092) without sending an email to everybody and hoping they read and follow it. [Currently, the only solution I know will work is the one pointed out by Yui Naruse in https://bugs.ruby-lang.org/issues/10084#note-17, but I'm okay with any other solution.] -- https://bugs.ruby-lang.org/

1 0