[ruby-core:126449] [Ruby Bug#22254] [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings
Issue #22254 has been reported by yqtian (Yongqiang Tian). ---------------------------------------- Bug #22254: [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings https://bugs.ruby-lang.org/issues/22254 * Author: yqtian (Yongqiang Tian) * Status: Open * ruby -v: ruby 4.1.0dev (2026-08-20T02:56:29Z master ed0b427b4b) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- I found a public `Prism.dump` path that passes a null source and zero length to `memcpy` in Prism's buffer helper. Clang's `nonnull-attribute` sanitizer reports the call because the effective glibc declaration marks the first two `memcpy` arguments as nonnull. I have not observed a crash, output corruption, or an ordinary non-sanitized failure. I am reporting this as a C correctness/toolchain-contract issue, not a security issue. #### Environment ```text x86-64 Ubuntu Clang 18.1.3 Ruby revision: ed0b427b4ba3c776c763d2673070e88372b2096a Relevant sanitizer: -fsanitize=nonnull-attribute Optimization: -O1 ``` I also checked Ruby HEAD `4012a96bf084d035bf582d58c7dafb31f4762ac6` and the corresponding Prism HEAD `3ea210b30108432b8b75b979872f4fa3c23acb7b`; the source site was unchanged. #### Reproduction From a Ruby checkout, create an out-of-tree build with the same flags used in the validation: ```sh ./autogen.sh mkdir build-ubsan cd build-ubsan ../configure \ CC=clang-18 \ CXX=clang++-18 \ CFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ CXXFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ LDFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all" \ optflags="-O1" \ debugflags="-g" \ --disable-install-doc \ --with-baseruby=/usr/bin/ruby make -j4 UBSAN_OPTIONS=print_stacktrace=0:halt_on_error=0:report_error_type=1 \ ruby ../tool/runruby.rb -- -rprism -e 'Prism.dump("//")' ``` Clang reports: ```text prism/buffer.c:86:40: runtime error: null pointer passed as argument 2, which is declared to never be null ``` Other reproducing inputs include `%r{}`, `//im`, and an empty quoted symbol. Nonempty regexps and symbols, an ordinary empty string, and `%q[]` were clean controls. #### Cause Some Prism node fields use a null-backed empty string representation. `pm_serialize_string` passes the source and zero length through `pm_buffer_append_bytes` to: ```c static PRISM_INLINE void pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) { size_t cursor = buffer->length; if (pm_buffer_append_length(buffer, length)) { memcpy(buffer->value + cursor, source, length); } } ``` A sink-side repair is: ```c if (pm_buffer_append_length(buffer, length)) { if (length > 0) { memcpy(buffer->value + cursor, source, length); } } ``` Another possibility is to skip `pm_buffer_append_bytes` when `length == 0` in the serialization template, preserving a stricter buffer-helper invariant. Should the buffer append helpers define a zero-length append as a no-op even when the source is null, or should the serialization caller filter null-backed empty strings? #### Validation I built unmodified and guarded trees with identical Docker/compiler/configure flags. The reproducing `Prism.dump` inputs report in the unmodified build and are clean in both recover and `halt_on_error=1` modes after adding the buffer guard. The unmodified and guarded builds produced byte-identical output for `Prism.dump("//")`: ```text SHA-256 eff5e01617aa527a49aa3b900589b163e4e396807ba252e8440c8a8fbfe047a5 ``` The same native test set passed in both builds, including 1,003 Prism dump tests with 403,474 assertions, plus Prism freeze/parse and Ruby eval/proc/syntax tests. The guarded build produced no `buffer.c` report in this test set. I would be happy to prepare a patch after confirming whether the preferred repair belongs in the common buffer helper or the serialization template. -- https://bugs.ruby-lang.org/
Issue #22254 has been updated by Earlopain (Earlopain _). https://github.com/ruby/prism/pull/4228 ---------------------------------------- Bug #22254: [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings https://bugs.ruby-lang.org/issues/22254#change-119022 * Author: yqtian (Yongqiang Tian) * Status: Assigned * Assignee: prism * ruby -v: ruby 4.1.0dev (2026-08-20T02:56:29Z master ed0b427b4b) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- I found a public `Prism.dump` path that passes a null source and zero length to `memcpy` in Prism's buffer helper. Clang's `nonnull-attribute` sanitizer reports the call because the effective glibc declaration marks the first two `memcpy` arguments as nonnull. I have not observed a crash, output corruption, or an ordinary non-sanitized failure. I am reporting this as a C correctness/toolchain-contract issue, not a security issue. #### Environment ```text x86-64 Ubuntu Clang 18.1.3 Ruby revision: ed0b427b4ba3c776c763d2673070e88372b2096a Relevant sanitizer: -fsanitize=nonnull-attribute Optimization: -O1 ``` I also checked Ruby HEAD `4012a96bf084d035bf582d58c7dafb31f4762ac6` and the corresponding Prism HEAD `3ea210b30108432b8b75b979872f4fa3c23acb7b`; the source site was unchanged. #### Reproduction From a Ruby checkout, create an out-of-tree build with the same flags used in the validation: ```sh ./autogen.sh mkdir build-ubsan cd build-ubsan ../configure \ CC=clang-18 \ CXX=clang++-18 \ CFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ CXXFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ LDFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all" \ optflags="-O1" \ debugflags="-g" \ --disable-install-doc \ --with-baseruby=/usr/bin/ruby make -j4 UBSAN_OPTIONS=print_stacktrace=0:halt_on_error=0:report_error_type=1 \ ruby ../tool/runruby.rb -- -rprism -e 'Prism.dump("//")' ``` Clang reports: ```text prism/buffer.c:86:40: runtime error: null pointer passed as argument 2, which is declared to never be null ``` Other reproducing inputs include `%r{}`, `//im`, and an empty quoted symbol. Nonempty regexps and symbols, an ordinary empty string, and `%q[]` were clean controls. #### Cause Some Prism node fields use a null-backed empty string representation. `pm_serialize_string` passes the source and zero length through `pm_buffer_append_bytes` to: ```c static PRISM_INLINE void pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) { size_t cursor = buffer->length; if (pm_buffer_append_length(buffer, length)) { memcpy(buffer->value + cursor, source, length); } } ``` A sink-side repair is: ```c if (pm_buffer_append_length(buffer, length)) { if (length > 0) { memcpy(buffer->value + cursor, source, length); } } ``` Another possibility is to skip `pm_buffer_append_bytes` when `length == 0` in the serialization template, preserving a stricter buffer-helper invariant. Should the buffer append helpers define a zero-length append as a no-op even when the source is null, or should the serialization caller filter null-backed empty strings? #### Validation I built unmodified and guarded trees with identical Docker/compiler/configure flags. The reproducing `Prism.dump` inputs report in the unmodified build and are clean in both recover and `halt_on_error=1` modes after adding the buffer guard. The unmodified and guarded builds produced byte-identical output for `Prism.dump("//")`: ```text SHA-256 eff5e01617aa527a49aa3b900589b163e4e396807ba252e8440c8a8fbfe047a5 ``` The same native test set passed in both builds, including 1,003 Prism dump tests with 403,474 assertions, plus Prism freeze/parse and Ruby eval/proc/syntax tests. The guarded build produced no `buffer.c` report in this test set. I would be happy to prepare a patch after confirming whether the preferred repair belongs in the common buffer helper or the serialization template. -- https://bugs.ruby-lang.org/
Issue #22254 has been updated by yqtian (Yongqiang Tian). Thanks a lot. ---------------------------------------- Bug #22254: [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings https://bugs.ruby-lang.org/issues/22254#change-119024 * Author: yqtian (Yongqiang Tian) * Status: Assigned * Assignee: prism * ruby -v: ruby 4.1.0dev (2026-08-20T02:56:29Z master ed0b427b4b) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- I found a public `Prism.dump` path that passes a null source and zero length to `memcpy` in Prism's buffer helper. Clang's `nonnull-attribute` sanitizer reports the call because the effective glibc declaration marks the first two `memcpy` arguments as nonnull. I have not observed a crash, output corruption, or an ordinary non-sanitized failure. I am reporting this as a C correctness/toolchain-contract issue, not a security issue. #### Environment ```text x86-64 Ubuntu Clang 18.1.3 Ruby revision: ed0b427b4ba3c776c763d2673070e88372b2096a Relevant sanitizer: -fsanitize=nonnull-attribute Optimization: -O1 ``` I also checked Ruby HEAD `4012a96bf084d035bf582d58c7dafb31f4762ac6` and the corresponding Prism HEAD `3ea210b30108432b8b75b979872f4fa3c23acb7b`; the source site was unchanged. #### Reproduction From a Ruby checkout, create an out-of-tree build with the same flags used in the validation: ```sh ./autogen.sh mkdir build-ubsan cd build-ubsan ../configure \ CC=clang-18 \ CXX=clang++-18 \ CFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ CXXFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ LDFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all" \ optflags="-O1" \ debugflags="-g" \ --disable-install-doc \ --with-baseruby=/usr/bin/ruby make -j4 UBSAN_OPTIONS=print_stacktrace=0:halt_on_error=0:report_error_type=1 \ ruby ../tool/runruby.rb -- -rprism -e 'Prism.dump("//")' ``` Clang reports: ```text prism/buffer.c:86:40: runtime error: null pointer passed as argument 2, which is declared to never be null ``` Other reproducing inputs include `%r{}`, `//im`, and an empty quoted symbol. Nonempty regexps and symbols, an ordinary empty string, and `%q[]` were clean controls. #### Cause Some Prism node fields use a null-backed empty string representation. `pm_serialize_string` passes the source and zero length through `pm_buffer_append_bytes` to: ```c static PRISM_INLINE void pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) { size_t cursor = buffer->length; if (pm_buffer_append_length(buffer, length)) { memcpy(buffer->value + cursor, source, length); } } ``` A sink-side repair is: ```c if (pm_buffer_append_length(buffer, length)) { if (length > 0) { memcpy(buffer->value + cursor, source, length); } } ``` Another possibility is to skip `pm_buffer_append_bytes` when `length == 0` in the serialization template, preserving a stricter buffer-helper invariant. Should the buffer append helpers define a zero-length append as a no-op even when the source is null, or should the serialization caller filter null-backed empty strings? #### Validation I built unmodified and guarded trees with identical Docker/compiler/configure flags. The reproducing `Prism.dump` inputs report in the unmodified build and are clean in both recover and `halt_on_error=1` modes after adding the buffer guard. The unmodified and guarded builds produced byte-identical output for `Prism.dump("//")`: ```text SHA-256 eff5e01617aa527a49aa3b900589b163e4e396807ba252e8440c8a8fbfe047a5 ``` The same native test set passed in both builds, including 1,003 Prism dump tests with 403,474 assertions, plus Prism freeze/parse and Ruby eval/proc/syntax tests. The guarded build produced no `buffer.c` report in this test set. I would be happy to prepare a patch after confirming whether the preferred repair belongs in the common buffer helper or the serialization template. -- https://bugs.ruby-lang.org/
Issue #22254 has been updated by Earlopain (Earlopain _). Status changed from Assigned to Closed Fixed by https://github.com/ruby/ruby/commit/8cdc7b59571dd5ea4fc08638e88255d3f43d2e60 ---------------------------------------- Bug #22254: [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings https://bugs.ruby-lang.org/issues/22254#change-119185 * Author: yqtian (Yongqiang Tian) * Status: Closed * Assignee: prism * ruby -v: ruby 4.1.0dev (2026-08-20T02:56:29Z master ed0b427b4b) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- I found a public `Prism.dump` path that passes a null source and zero length to `memcpy` in Prism's buffer helper. Clang's `nonnull-attribute` sanitizer reports the call because the effective glibc declaration marks the first two `memcpy` arguments as nonnull. I have not observed a crash, output corruption, or an ordinary non-sanitized failure. I am reporting this as a C correctness/toolchain-contract issue, not a security issue. #### Environment ```text x86-64 Ubuntu Clang 18.1.3 Ruby revision: ed0b427b4ba3c776c763d2673070e88372b2096a Relevant sanitizer: -fsanitize=nonnull-attribute Optimization: -O1 ``` I also checked Ruby HEAD `4012a96bf084d035bf582d58c7dafb31f4762ac6` and the corresponding Prism HEAD `3ea210b30108432b8b75b979872f4fa3c23acb7b`; the source site was unchanged. #### Reproduction From a Ruby checkout, create an out-of-tree build with the same flags used in the validation: ```sh ./autogen.sh mkdir build-ubsan cd build-ubsan ../configure \ CC=clang-18 \ CXX=clang++-18 \ CFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ CXXFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ LDFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all" \ optflags="-O1" \ debugflags="-g" \ --disable-install-doc \ --with-baseruby=/usr/bin/ruby make -j4 UBSAN_OPTIONS=print_stacktrace=0:halt_on_error=0:report_error_type=1 \ ruby ../tool/runruby.rb -- -rprism -e 'Prism.dump("//")' ``` Clang reports: ```text prism/buffer.c:86:40: runtime error: null pointer passed as argument 2, which is declared to never be null ``` Other reproducing inputs include `%r{}`, `//im`, and an empty quoted symbol. Nonempty regexps and symbols, an ordinary empty string, and `%q[]` were clean controls. #### Cause Some Prism node fields use a null-backed empty string representation. `pm_serialize_string` passes the source and zero length through `pm_buffer_append_bytes` to: ```c static PRISM_INLINE void pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) { size_t cursor = buffer->length; if (pm_buffer_append_length(buffer, length)) { memcpy(buffer->value + cursor, source, length); } } ``` A sink-side repair is: ```c if (pm_buffer_append_length(buffer, length)) { if (length > 0) { memcpy(buffer->value + cursor, source, length); } } ``` Another possibility is to skip `pm_buffer_append_bytes` when `length == 0` in the serialization template, preserving a stricter buffer-helper invariant. Should the buffer append helpers define a zero-length append as a no-op even when the source is null, or should the serialization caller filter null-backed empty strings? #### Validation I built unmodified and guarded trees with identical Docker/compiler/configure flags. The reproducing `Prism.dump` inputs report in the unmodified build and are clean in both recover and `halt_on_error=1` modes after adding the buffer guard. The unmodified and guarded builds produced byte-identical output for `Prism.dump("//")`: ```text SHA-256 eff5e01617aa527a49aa3b900589b163e4e396807ba252e8440c8a8fbfe047a5 ``` The same native test set passed in both builds, including 1,003 Prism dump tests with 403,474 assertions, plus Prism freeze/parse and Ruby eval/proc/syntax tests. The guarded build produced no `buffer.c` report in this test set. I would be happy to prepare a patch after confirming whether the preferred repair belongs in the common buffer helper or the serialization template. -- https://bugs.ruby-lang.org/
Issue #22254 has been updated by yqtian (Yongqiang Tian). Thank you very much for this. ---------------------------------------- Bug #22254: [Prism] Clang UBSan reports NULL/0 memcpy while serializing empty strings https://bugs.ruby-lang.org/issues/22254#change-119186 * Author: yqtian (Yongqiang Tian) * Status: Closed * Assignee: prism * ruby -v: ruby 4.1.0dev (2026-08-20T02:56:29Z master ed0b427b4b) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- I found a public `Prism.dump` path that passes a null source and zero length to `memcpy` in Prism's buffer helper. Clang's `nonnull-attribute` sanitizer reports the call because the effective glibc declaration marks the first two `memcpy` arguments as nonnull. I have not observed a crash, output corruption, or an ordinary non-sanitized failure. I am reporting this as a C correctness/toolchain-contract issue, not a security issue. #### Environment ```text x86-64 Ubuntu Clang 18.1.3 Ruby revision: ed0b427b4ba3c776c763d2673070e88372b2096a Relevant sanitizer: -fsanitize=nonnull-attribute Optimization: -O1 ``` I also checked Ruby HEAD `4012a96bf084d035bf582d58c7dafb31f4762ac6` and the corresponding Prism HEAD `3ea210b30108432b8b75b979872f4fa3c23acb7b`; the source site was unchanged. #### Reproduction From a Ruby checkout, create an out-of-tree build with the same flags used in the validation: ```sh ./autogen.sh mkdir build-ubsan cd build-ubsan ../configure \ CC=clang-18 \ CXX=clang++-18 \ CFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ CXXFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all -fno-omit-frame-pointer -g -O1" \ LDFLAGS="-fsanitize=nonnull-attribute,returns-nonnull-attribute,nullability-arg,nullability-return,nullability-assign -fsanitize-recover=all" \ optflags="-O1" \ debugflags="-g" \ --disable-install-doc \ --with-baseruby=/usr/bin/ruby make -j4 UBSAN_OPTIONS=print_stacktrace=0:halt_on_error=0:report_error_type=1 \ ruby ../tool/runruby.rb -- -rprism -e 'Prism.dump("//")' ``` Clang reports: ```text prism/buffer.c:86:40: runtime error: null pointer passed as argument 2, which is declared to never be null ``` Other reproducing inputs include `%r{}`, `//im`, and an empty quoted symbol. Nonempty regexps and symbols, an ordinary empty string, and `%q[]` were clean controls. #### Cause Some Prism node fields use a null-backed empty string representation. `pm_serialize_string` passes the source and zero length through `pm_buffer_append_bytes` to: ```c static PRISM_INLINE void pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) { size_t cursor = buffer->length; if (pm_buffer_append_length(buffer, length)) { memcpy(buffer->value + cursor, source, length); } } ``` A sink-side repair is: ```c if (pm_buffer_append_length(buffer, length)) { if (length > 0) { memcpy(buffer->value + cursor, source, length); } } ``` Another possibility is to skip `pm_buffer_append_bytes` when `length == 0` in the serialization template, preserving a stricter buffer-helper invariant. Should the buffer append helpers define a zero-length append as a no-op even when the source is null, or should the serialization caller filter null-backed empty strings? #### Validation I built unmodified and guarded trees with identical Docker/compiler/configure flags. The reproducing `Prism.dump` inputs report in the unmodified build and are clean in both recover and `halt_on_error=1` modes after adding the buffer guard. The unmodified and guarded builds produced byte-identical output for `Prism.dump("//")`: ```text SHA-256 eff5e01617aa527a49aa3b900589b163e4e396807ba252e8440c8a8fbfe047a5 ``` The same native test set passed in both builds, including 1,003 Prism dump tests with 403,474 assertions, plus Prism freeze/parse and Ruby eval/proc/syntax tests. The guarded build produced no `buffer.c` report in this test set. I would be happy to prepare a patch after confirming whether the preferred repair belongs in the common buffer helper or the serialization template. -- https://bugs.ruby-lang.org/
participants (2)
-
Earlopain (Earlopain _) -
yqtian (Yongqiang Tian)