Hello,
I noticed that the define "__LDSO_STANDALONE_SUPPORT__" does not just prevents you from calling a binary via ld e.g. '/lib/ld-uClibc.so my-binary', but also prevents programs from loading a binary via dlopen. Is this intended? If so the name is at least confusing.
Greetings
Luca Lindhorst
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
Just to clarify by "programs from loading a binary via dlopen" I meant loading an elf which is an executeable not a dynamic lib.
Am 31.05.19 um 14:15 schrieb Luca Lindhorst:
Hello,
I noticed that the define "__LDSO_STANDALONE_SUPPORT__" does not just prevents you from calling a binary via ld e.g. '/lib/ld-uClibc.so my-binary', but also prevents programs from loading a binary via dlopen. Is this intended? If so the name is at least confusing.
Greetings
Luca Lindhorst
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
Hello Luca,
I am not sure whether dlopen'ing an executable (and not a shared lib) is a supported feature of uClibc-ng...
Man page of dlopen says it is for opening "shared lib".
There has been a discussion on glibc bug tracker to widen the scope of dlopen and allow it to open executables ... but it seems Ulrich Drepper refused it, and then the discussion was revived and it still does not seem to hold a strong consensus. See https://sourceware.org/bugzilla/show_bug.cgi?id=11754
I didn't try, but I am guessing here that if it works, it is by accident.
opening and mapping an ET_EXEC is needed indeed in the case of "standalone" (dynamic loader run as an executable and loading another executable given as argument).
And the function used to do this, is also the one used by dlopen() to map the shared libs.
I guess you are refering to https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L596
_dl_load_elf_shared_library is indeed used to load the ET_EXEC in the "ld.so used as an executable" from there: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/ldso.c#L601
And it is also called from https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L200
Which is itself called from dlopen: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/libdl/libdl.c#L372
So it seems the same function is used in 2 different contexts and that leads to the support of ET_EXEC which is valid in one context ... but I'm not sure it is valid in the other one (directly loading an ET_EXEC with dlopen).
I hope this is clear.
Maybe someone else can give a hint whether my guess is correct or not? Rich ? Waldemar ?
Cheers,
Yann
Le 31/05/2019 à 14:15, Luca Lindhorst a écrit :
Hello,
I noticed that the define "__LDSO_STANDALONE_SUPPORT__" does not just prevents you from calling a binary via ld e.g. '/lib/ld-uClibc.so my-binary', but also prevents programs from loading a binary via dlopen. Is this intended? If so the name is at least confusing.
Greetings
Luca Lindhorst
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de _______________________________________________ devel mailing list devel@uclibc-ng.org https://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel
On Sat, Jun 01, 2019 at 06:35:23PM +0200, Yann Sionneau wrote:
Hello Luca,
I am not sure whether dlopen'ing an executable (and not a shared lib) is a supported feature of uClibc-ng...
Man page of dlopen says it is for opening "shared lib".
There has been a discussion on glibc bug tracker to widen the scope of dlopen and allow it to open executables ... but it seems Ulrich Drepper refused it, and then the discussion was revived and it still does not seem to hold a strong consensus. See https://sourceware.org/bugzilla/show_bug.cgi?id=11754
I didn't try, but I am guessing here that if it works, it is by accident.
opening and mapping an ET_EXEC is needed indeed in the case of "standalone" (dynamic loader run as an executable and loading another executable given as argument).
And the function used to do this, is also the one used by dlopen() to map the shared libs.
I guess you are refering to https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L596
_dl_load_elf_shared_library is indeed used to load the ET_EXEC in the "ld.so used as an executable" from there: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/ldso.c#L601
And it is also called from https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L200
Which is itself called from dlopen: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/libdl/libdl.c#L372
So it seems the same function is used in 2 different contexts and that leads to the support of ET_EXEC which is valid in one context ... but I'm not sure it is valid in the other one (directly loading an ET_EXEC with dlopen).
I hope this is clear.
Maybe someone else can give a hint whether my guess is correct or not? Rich ? Waldemar ?
I don't think dlopen of an ET_EXEC can be made safe/consistent. aside from the issue with whether the address space range is available, which could just be imposed as a constraint, you also have matters of interposition and TLS model.
When producing an executable, whether ET_EXEC or ET_DYN (PIE), the linker will bind address references to external function and data symbols to PLT slots of copy relocations in the main executable. But the rest of the program (into which it's loaded via dlopen) will be using different addresses for these functions and objects. That *might* not matter, if you don't care about function pointer equality and don't use global variables.
For TLS, it's even worse. All access to thread-local objects from an executable will use the local-exec model -- the offset of the object from the thread pointer is hard-coded at link time. The offsets used by the dlopened program will overlap with offsets used by the program it's loaded into, or its shared libraries, and major memory corruption will occur if they're ever accessed. There is no way to patch this up.
Rich
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
Am 01.06.19 um 18:55 schrieb Rich Felker:
On Sat, Jun 01, 2019 at 06:35:23PM +0200, Yann Sionneau wrote:
Hello Luca,
I am not sure whether dlopen'ing an executable (and not a shared lib) is a supported feature of uClibc-ng...
Man page of dlopen says it is for opening "shared lib".
There has been a discussion on glibc bug tracker to widen the scope of dlopen and allow it to open executables ... but it seems Ulrich Drepper refused it, and then the discussion was revived and it still does not seem to hold a strong consensus. See https://sourceware.org/bugzilla/show_bug.cgi?id=11754
I didn't try, but I am guessing here that if it works, it is by accident.
opening and mapping an ET_EXEC is needed indeed in the case of "standalone" (dynamic loader run as an executable and loading another executable given as argument).
And the function used to do this, is also the one used by dlopen() to map the shared libs.
I guess you are refering to https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L596
_dl_load_elf_shared_library is indeed used to load the ET_EXEC in the "ld.so used as an executable" from there: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/ldso.c#L601
And it is also called from https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/ldso/dl-elf.c#L200
Which is itself called from dlopen: https://elixir.bootlin.com/uclibc-ng/latest/source/ldso/libdl/libdl.c#L372
So it seems the same function is used in 2 different contexts and that leads to the support of ET_EXEC which is valid in one context ... but I'm not sure it is valid in the other one (directly loading an ET_EXEC with dlopen).
I hope this is clear.
Maybe someone else can give a hint whether my guess is correct or not? Rich ? Waldemar ?
I don't think dlopen of an ET_EXEC can be made safe/consistent. aside from the issue with whether the address space range is available, which could just be imposed as a constraint, you also have matters of interposition and TLS model.
When producing an executable, whether ET_EXEC or ET_DYN (PIE), the linker will bind address references to external function and data symbols to PLT slots of copy relocations in the main executable. But the rest of the program (into which it's loaded via dlopen) will be using different addresses for these functions and objects. That *might* not matter, if you don't care about function pointer equality and don't use global variables.
For TLS, it's even worse. All access to thread-local objects from an executable will use the local-exec model -- the offset of the object from the thread pointer is hard-coded at link time. The offsets used by the dlopened program will overlap with offsets used by the program it's loaded into, or its shared libraries, and major memory corruption will occur if they're ever accessed. There is no way to patch this up.
Rich
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
On Mon, Jun 03, 2019 at 03:47:48PM +0200, Luca Lindhorst wrote:
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
I don't understand why it would dlopen the openssl binary rather than libssl for this purpose. It should not be doing that, and there's probably some other bug in openssl or in your integration of it that's making it do that. It also might be something you could consider a bug in uclibc -- attempting to dlopen the main program itself, rather than some other executable, should probably give a handle to the existing instance, rather than attempting to load it again.
Rich
I meanwhile found the problem, it is already known, and they did something about in the newest OpenSSL version. OpenSSL wants for be sure, that their cleanup routines on atexit are executed. However if it was loaded by dload and later closed by dlclose neccessary code is not available anymore on exit. To achieve this they do a (presumably second) dload on their library to keep it loaded. This makes no sense if it is dynamically compile time linked, but does not really harm. But when linked statically like in my case it does something invalid, which doesn't really hurt either, but on uclibc apparently leads to a printed error message.
This might actually be a thing to consider for uclibc, not printing out an error, and only rely on the "normal" error handling mechanisms, like in this case dload returning NULL.
Am 03.06.19 um 17:06 schrieb Rich Felker:
On Mon, Jun 03, 2019 at 03:47:48PM +0200, Luca Lindhorst wrote:
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
I don't understand why it would dlopen the openssl binary rather than libssl for this purpose. It should not be doing that, and there's probably some other bug in openssl or in your integration of it that's making it do that. It also might be something you could consider a bug in uclibc -- attempting to dlopen the main program itself, rather than some other executable, should probably give a handle to the existing instance, rather than attempting to load it again.
Rich
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
For the record here are what I think are the corresponding threads:
https://github.com/openssl/openssl/issues/654
https://github.com/curl/curl/issues/1055
It seems those thread end up pointing that OpenSSL have invalid assumptions on atexit() in their code at least for Windows platform. (for the 2nd link)
This also seems to be of interest https://github.com/openssl/openssl/issues/653
In any case, I think it is hard to dig deeper in this topic without looking at the actual code and having more precise information about what is linked with what, and how (static? dynamic ?).
And see some crash dumps (stack trace).
On 6/3/19 5:16 PM, Luca Lindhorst wrote:
I meanwhile found the problem, it is already known, and they did something about in the newest OpenSSL version. OpenSSL wants for be sure, that their cleanup routines on atexit are executed. However if it was loaded by dload and later closed by dlclose neccessary code is not available anymore on exit. To achieve this they do a (presumably second) dload on their library to keep it loaded. This makes no sense if it is dynamically compile time linked, but does not really harm. But when linked statically like in my case it does something invalid, which doesn't really hurt either, but on uclibc apparently leads to a printed error message.
This might actually be a thing to consider for uclibc, not printing out an error, and only rely on the "normal" error handling mechanisms, like in this case dload returning NULL.
Am 03.06.19 um 17:06 schrieb Rich Felker:
On Mon, Jun 03, 2019 at 03:47:48PM +0200, Luca Lindhorst wrote:
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
I don't understand why it would dlopen the openssl binary rather than libssl for this purpose. It should not be doing that, and there's probably some other bug in openssl or in your integration of it that's making it do that. It also might be something you could consider a bug in uclibc -- attempting to dlopen the main program itself, rather than some other executable, should probably give a handle to the existing instance, rather than attempting to load it again.
Rich
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
And just out of curiosity, are you sure that you are following all requirements described there: https://github.com/openssl/openssl/blob/df1f538f28c10f2954757164b17781040d23... ?
Especially the "NOTES" part?
On 6/3/19 5:34 PM, Yann Sionneau wrote:
For the record here are what I think are the corresponding threads:
https://github.com/openssl/openssl/issues/654
https://github.com/curl/curl/issues/1055
It seems those thread end up pointing that OpenSSL have invalid assumptions on atexit() in their code at least for Windows platform. (for the 2nd link)
This also seems to be of interest https://github.com/openssl/openssl/issues/653
In any case, I think it is hard to dig deeper in this topic without looking at the actual code and having more precise information about what is linked with what, and how (static? dynamic ?).
And see some crash dumps (stack trace).
On 6/3/19 5:16 PM, Luca Lindhorst wrote:
I meanwhile found the problem, it is already known, and they did something about in the newest OpenSSL version. OpenSSL wants for be sure, that their cleanup routines on atexit are executed. However if it was loaded by dload and later closed by dlclose neccessary code is not available anymore on exit. To achieve this they do a (presumably second) dload on their library to keep it loaded. This makes no sense if it is dynamically compile time linked, but does not really harm. But when linked statically like in my case it does something invalid, which doesn't really hurt either, but on uclibc apparently leads to a printed error message.
This might actually be a thing to consider for uclibc, not printing out an error, and only rely on the "normal" error handling mechanisms, like in this case dload returning NULL.
Am 03.06.19 um 17:06 schrieb Rich Felker:
On Mon, Jun 03, 2019 at 03:47:48PM +0200, Luca Lindhorst wrote:
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
I don't understand why it would dlopen the openssl binary rather than libssl for this purpose. It should not be doing that, and there's probably some other bug in openssl or in your integration of it that's making it do that. It also might be something you could consider a bug in uclibc -- attempting to dlopen the main program itself, rather than some other executable, should probably give a handle to the existing instance, rather than attempting to load it again.
Rich
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
devel mailing list devel@uclibc-ng.org https://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel
Just to summarize and "finish" this: It is definitely not an uclibc bug, as I said I wrongly assumed that any executable can be dloaded. However the described behavior by openssl should only apply if compiled as dynamic lib, but not as a static one. A flag for this was added in 1.1.1b. However node.js did not use the flag yet, which I fixed with a patch.
Am 03.06.19 um 17:40 schrieb Yann Sionneau:
And just out of curiosity, are you sure that you are following all requirements described there: https://github.com/openssl/openssl/blob/df1f538f28c10f2954757164b17781040d23... ?
Especially the "NOTES" part?
On 6/3/19 5:34 PM, Yann Sionneau wrote:
For the record here are what I think are the corresponding threads:
https://github.com/openssl/openssl/issues/654
https://github.com/curl/curl/issues/1055
It seems those thread end up pointing that OpenSSL have invalid assumptions on atexit() in their code at least for Windows platform. (for the 2nd link)
This also seems to be of interest https://github.com/openssl/openssl/issues/653
In any case, I think it is hard to dig deeper in this topic without looking at the actual code and having more precise information about what is linked with what, and how (static? dynamic ?).
And see some crash dumps (stack trace).
On 6/3/19 5:16 PM, Luca Lindhorst wrote:
I meanwhile found the problem, it is already known, and they did something about in the newest OpenSSL version. OpenSSL wants for be sure, that their cleanup routines on atexit are executed. However if it was loaded by dload and later closed by dlclose neccessary code is not available anymore on exit. To achieve this they do a (presumably second) dload on their library to keep it loaded. This makes no sense if it is dynamically compile time linked, but does not really harm. But when linked statically like in my case it does something invalid, which doesn't really hurt either, but on uclibc apparently leads to a printed error message.
This might actually be a thing to consider for uclibc, not printing out an error, and only rely on the "normal" error handling mechanisms, like in this case dload returning NULL.
Am 03.06.19 um 17:06 schrieb Rich Felker:
On Mon, Jun 03, 2019 at 03:47:48PM +0200, Luca Lindhorst wrote:
Thanks for the help.
I guess there isn't a problem then. I didn't know that you can only load a "special" ET_DYN PIE elf, it makes sense though.
FYI: I am investigating a problem relating NodeJS on ARM. And after some more investigating I found out that openssl tries to dlload itself again, for a reason I don't fully understand yet ("|Deliberately leak a reference to ourselves. This will force the library to remain loaded until the atexit() handler is run at process exit.|"). But since openssl is statically linked in node, it loads the binary again.
Therefore this certainly isn't a uclibc problem.
I don't understand why it would dlopen the openssl binary rather than libssl for this purpose. It should not be doing that, and there's probably some other bug in openssl or in your integration of it that's making it do that. It also might be something you could consider a bug in uclibc -- attempting to dlopen the main program itself, rather than some other executable, should probably give a handle to the existing instance, rather than attempting to load it again.
Rich
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
devel mailing list devel@uclibc-ng.org https://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel
-- Unsere Aussagen koennen Irrtuemer und Missverstaendnisse enthalten. Bitte pruefen Sie die Aussagen fuer Ihren Fall, bevor Sie Entscheidungen auf Grundlage dieser Aussagen treffen. Wiesemann & Theis GmbH, Porschestr. 12, D-42279 Wuppertal Geschaeftsfuehrer: Dipl.-Ing. Ruediger Theis Registergericht: Amtsgericht Wuppertal, HRB 6377 Infos zum Datenschutz: http://www.wut.de/datenschutz Tel. +49-202/2680-0, Fax +49-202/2680-265, http://www.wut.de
-
Luca Lindhorst -
Rich Felker -
Yann Sionneau