[ruby-core:114089] [Ruby master Misc#19758] Statically link ext/json
Issue #19758 has been reported by MyCo (Maik Menz). ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by MyCo (Maik Menz). EDIT: I'm using `--enable-install-static-library --with-static-linked-ext` and it appears as if non of the ext's are actually linked into the static lib. ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-103766 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by nobu (Nobuyoshi Nakada). Does generated ext/extinit.c include `init(Init_parser, "json/ext/parser")` line? ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-103767 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by MyCo (Maik Menz). nobu (Nobuyoshi Nakada) wrote in #note-2:
Does generated ext/extinit.c include `init(Init_parser, "json/ext/parser")` line?
Yes, it also contains the other libraries that are missing,eg. Init_socket ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-103768 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by MyCo (Maik Menz). The problem appears to be here: ``` $(LIBRUBY_A): $(OBJS) $(INITOBJS) $(ECHO) linking static-library $(@:\=/) $(Q) $(AR) $(ARFLAGS)$@ $(OBJS) $(INITOBJS) ``` where: ``` INITOBJS = dmyext.$(OBJEXT) dmyenc.$(OBJEXT) ``` This should actually point to `ext/extinit.$(OBJEXT) enc/encinit.$(OBJEXT)` but no matter what I try, it's always creating build issues. For example `No such file or directory @ rb_sysopen - ext/extinit.c`, maybe someone can give me a hint what's going on. I don't understand how this file doesn't exist even if it's a dependency of a build step. ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-103775 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by mame (Yusuke Endoh). Assignee set to nobu (Nobuyoshi Nakada) ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-104524 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
Issue #19758 has been updated by jmarrec (Julien Marrec). Hello, I am facing the same issue today, though I am presently on Ubuntu, not windows. I realized that `Init_ext` was doing nothing (I couldn't then use `require 'bigdecimal'` without a crash) by using a debugger and I noticed that it was picking up the empty one from `dmyext.c` and not the auto-generated `ext/extinit.c` one. I have specifically uncommented all lines in `ext/Setup` (except the win32 ones) to have `option nodynamic` enabled in particular. I tried specifically not linking to INITOBJS by patching Makefile.in as follow, but I get a failure to build. At that point in time when the `libruby-static.a` target is called, the EXTOBJS / ENCOBJS do not exist on disk yet (EXTOBJS=ext/extinit.o, ENCOBJS=enc/encinit.o enc/libenc.a enc/libtrans.a). And `libruby-static.a` is needed for miniruby (I think), which is needed to build those EXT/ENC objects. So there needs to be some kind of final pass with AR overriding the `Init_ext`/`Init_enc` or something? And perhaps also merging the inidividual ext/ && enc `.a` libraries into the `libruby-static.a` itself so everything is self-contained? Do you have any tips please? ---- **(Failed) things I tried:** I'm not experienced enough with the ruby build system (I have trouble following how the extensions's target are called) to make that happen, so in the interim I tried to manually fix the libruby-static.a via new make target that calls `ar` to replace INITOBJS with EXTOBJS / ENCOBJS but that failed too, because it's missing the individual Init_<ext> (eg: Init_digest, Init_big5, etc). Truncated failure when `make && make final-libruby_a && verify-static-library`: ``` shell /usr/bin/ld: libruby-static.a(extinit.o): in function `Init_ext': /home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:12: undefined reference to `Init_bigdecimal' /usr/bin/ld: /home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:13: undefined reference to `Init_escape' /usr/bin/ld: /home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:14: undefined reference to `Init_continuation ``` Bad patch I tried to apply: ``` diff diff --git a/template/Makefile.in b/template/Makefile.in --- a/template/Makefile.in +++ b/template/Makefile.in @@ -333,7 +333,18 @@ $(LIBRUBY_A): @$(RM) $@ @-[ -z "$(EXTSTATIC)" ] || $(PRE_LIBRUBY_UPDATE) $(ECHO) linking static-library $@ - $(Q) $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS) + $(ECHO) LIBRUBY_A_OBJS=$(LIBRUBY_A_OBJS) + $(ECHO) INITOBJS=$(INITOBJS) + $(ECHO) EXTOBJS=$(EXTOBJS) + $(ECHO) ENCOBJS=$(ENCOBJS) + $(ECHO) ext/extinit.$(OBJEXT) enc/encinit.$(OBJEXT) + $(Q) if [ -z "$(EXTSTATIC)" ]; then \ + $(ECHO0) CMD=$(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \ + $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \ + else \ + $(ECHO0) CMD=$(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \ + $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \ + fi $(Q) if [ 'no' != '$(YJIT_SUPPORT)' ]; then \ set -eu$(V0:1=x) && \ $(ECHO0) 'merging $(YJIT_LIBS) into $@' && \ @@ -346,7 +357,18 @@ $(LIBRUBY_A): fi @-$(RANLIB) $@ 2> /dev/null || true -verify-static-library: $(LIBRUBY_A) +final-libruby_a: + @-[ -z "$(EXTSTATIC)" ] + $(ECHO) Replacing INCOBJS with EXTOBJS and ENCOBS in static-library $(LIBRUBY_A) + $(ECHO) LIBRUBY_A_OBJS=$(LIBRUBY_A_OBJS) + $(ECHO) INITOBJS=$(INITOBJS) + $(ECHO) EXTOBJS=$(EXTOBJS) + $(ECHO) ENCOBJS=$(ENCOBJS) + $(AR) dD $(LIBRUBY_A) $(INITOBJS) + $(AR) rD $(LIBRUBY_A) $(EXTOBJS) $(ENCOBJS) + @-$(RANLIB) $(LIBRUBY_A) 2> /dev/null || true + +verify-static-library: $(ECHO) verifying static-library $@ @$(PURIFY) $(CC) $(EXE_LDFLAGS) $(XLDFLAGS) $(MAINOBJ) $(LIBRUBY_A) $(MAINLIBS) $(EXTLIBS) $(LIBS) $(OUTFLAG)conftest$(EXEEXT) @$(RMALL) conftest$(EXEEXT) conftest.c conftest.dSYM ``` My configure call is as follows: ``` shell ../ruby-patching/configure --prefix=/ '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' --disable-shared --enable-static --disable-install-doc --with-static-linked-ext --enable-load-relative --with-zlib-dir=/home/julien/.conan2/p/b/zlib1b2fc466a3731/p --with-openssl-dir=/home/julien/.conan2/p/b/opensccd63d9154ad3/p --with-libffi-dir=/home/julien/.conan2/p/b/libffe070527d1c02e/p --with-libyaml-dir=/home/julien/.conan2/p/b/libyae7bed9b8d84ed/p --with-readline-dir=/home/julien/.conan2/p/b/readl6fb13f62b9c29/p --with-gmp-dir=/home/julien/.conan2/p/b/gmp598d198a78496/p --with-opt-dir=/home/julien/.conan2/p/b/zlib1b2fc466a3731/p:/home/julien/.conan2/p/b/opensccd63d9154ad3/p:/home/julien/.conan2/p/b/libffe070527d1c02e/p:/home/julien/.conan2/p/b/libyae7bed9b8d84ed/p:/home/julien/.conan2/p/b/readl6fb13f62b9c29/p:/home/julien/.conan2/p/b/gmp598d198a78496/p --disable-jit-support --enable-mkmf-verbose --enable-debug-env ``` ---------------------------------------- Misc #19758: Statically link ext/json https://bugs.ruby-lang.org/issues/19758#change-106527 * Author: MyCo (Maik Menz) * Status: Open * Priority: Normal * Assignee: nobu (Nobuyoshi Nakada) ---------------------------------------- Hi, I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version. In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place. I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib. With my limited knowledge of the building processes, my first attempt was to add both of those libs into `LOCAL_LIBS` and now they apear in the linking process. But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build. What am I missing? What do I have to do, to get those linked into the static lib? Regards Maik -- https://bugs.ruby-lang.org/
participants (4)
-
jmarrec (Julien Marrec) -
mame (Yusuke Endoh)
-
MyCo (Maik Menz)
-
nobu (Nobuyoshi Nakada)