[ruby-core:124292] [Ruby Bug#21792] 4.0.0-preview2: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions
Issue #21792 has been reported by mdalessio (Mike Dalessio). ---------------------------------------- Bug #21792: 4.0.0-preview2: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by mdalessio (Mike Dalessio). Note that this also reproduces in the source tree for 656de67d / 4.0.0-preview3. ---------------------------------------- Bug #21792: 4.0.0-preview2: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115790 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by mdalessio (Mike Dalessio). I've proposed a potential fix at https://github.com/ruby/ruby/pull/15617 ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115793 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by nobu (Nobuyoshi Nakada). I don't think the installation script works with no extensions because it loads rubygems that require `monitor`. ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115811 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by mdalessio (Mike Dalessio).
I don't think the installation script works with no extensions because it loads rubygems that require monitor
@nobu Is this something that changed since Ruby 3.4? `rake-compiler` has been using the `--with-ext=` (empty) option since 2015. I did a quick search but I could not find where this dependency was introduced. Turning off extensions is an optimization (see the original commit https://github.com/rake-compiler/rake-compiler/commit/8ed7c2a6 for more context) so we can remove it, but I would like to make sure that this is an intentional decision in Ruby before proposing its removal in `rake-compiler` for building Ruby >= 4.0. ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115829 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by mdalessio (Mike Dalessio).
I don't think the installation script works with no extensions because it loads rubygems that require monitor
Is this something that changed since Ruby 3.4?
Ah, I see, is this because logger relies on monitor, and logger is no longer part of the stdlib (it is a bundled gem)? ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115831 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by nobu (Nobuyoshi Nakada). mdalessio (Mike Dalessio) wrote in #note-6:
Ah, I see, is this because logger relies on monitor, and logger is no longer part of the stdlib (it is a bundled gem)?
Not `logger`, `lib/rubygems/core_ext/kernel_require.rb` requires `monitor`, to create `RUBYGEMS_ACTIVATION_MONITOR`. Also `date` via `time` is used in rdoc. ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115837 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
Issue #21792 has been updated by Eregon (Benoit Daloze). Do you then use this "minimal Ruby" to run the gem extconf.rb? I don't think that works in general without extensions, extconf.rb can assume the full stdlib to be available. As nobu said even RubyGems cannot load without `monitor`, a C extension (on CRuby). ---------------------------------------- Bug #21792: 4.0.0-preview3: Build fails with `--with-ext=` when ENABLE_SHARED=yes: ruby/digest.h not found for rubyspec CAPI extensions https://bugs.ruby-lang.org/issues/21792#change-115841 * Author: mdalessio (Mike Dalessio) * Status: Open * Target version: 4.0 ---------------------------------------- When building Ruby with `--enable-shared` and `--with-ext=` (empty, to disable all extensions), the build fails because spec/ruby/optional/capi/ext/digest_spec.c cannot find ruby/digest.h. This affects cross-compilation tooling like `rake-compiler` which uses these configure options to build a minimal Ruby for cross-compiling native gems. ## Reproducing Download and extract ruby-4.0.0-preview2 source code. ``` cd ruby-4.0.0-preview2 ./configure --enable-shared --disable-install-doc --with-ext= make ``` Or alternatively, use rake-compiler: ``` mkdir ~/.rake-compiler rake-compiler cross-ruby VERSION=4.0.0-preview2 HOST=x86_64-linux-gnu ``` Either way, you will see: ``` spec/ruby/optional/capi/ext/digest_spec.c:4:10: fatal error: ruby/digest.h: No such file or directory 4 | #include "ruby/digest.h" | ^~~~~~~~~~~~~~~ compilation terminated. make: *** [defs/gmake.mk:521: spec/ruby/optional/capi/ext/digest_spec.so] Error 1 ``` ## Root Cause I believe the root cause is: 1. defs/gmake.mk:531-532 unconditionally adds rubyspec-capiext to the exts target when ENABLE_SHARED=yes: ``` ifeq ($(ENABLE_SHARED),yes) exts: rubyspec-capiext endif ``` 2. rubyspec-capiext builds all *.c files in spec/ruby/optional/capi/ext/, including digest_spec.c 3. digest_spec.c (line 4) includes ruby/digest.h 4. ruby/digest.h is only installed when the digest extension is built. From ext/digest/extconf.rb:7-9: ``` $INSTALLFILES = { "digest.h" => "$(HDRDIR)" } if $extmk ``` 5. With --with-ext= (empty), the digest extension is not built, so digest.h is never copied to .ext/include/ruby/ 6. The build fails because digest_spec.c requires a header that was never installed Note that before 269ad29d, no rubyspec CAPI extension required extension-specific headers, so this configuration worked. -- https://bugs.ruby-lang.org/
participants (3)
-
Eregon (Benoit Daloze) -
mdalessio (Mike Dalessio)
-
nobu (Nobuyoshi Nakada)