Hello,
I am using openadk on the raspberry pi 3, and have an appliance where I have one main application that is running all the time. So I am using gcc 7.2.0 from openadk, and compilation starts to be *very slow* with gcc: 10 minutes for a full rebuild... So I am looking into having the clang toolchain available next to the gcc one so I can compile only my application with clang, which is faster. I have several shared libraries I also compile myself as part of my application build, all using cmake.
So, I used the llvm package under the toolchain dir and modified it so it could build in another output directory like this:
BUILD_DIR_INITIAL:= $(WRKBUILD)-initial BUILD_DIR_FINAL:= $(WRKBUILD)-final TC_INSTALL_DIR:=$(TOOLCHAIN_DIR)
ifeq ($(ADK_BUILD_USERLAND_COMPILER_LLVM),y) TC_INSTALL_DIR:=$(TOOLCHAIN_USERLAND_DIR) BUILD_DIR_INITIAL:= $(WRKBUILD)-userland-llvm-initial BUILD_DIR_FINAL:= $(WRKBUILD)-userland-llvn-final endif
Then as llvm triple I used this:
LLVM_TRIPLE:=arm-openadk-linux-uclibceabihf
And the version I used was 5.0.2 of llvm.
So that compiled for me and I got indeed a toolchain in the TOOLCHAIN_USERLAND_DIR location, which I added in vars.mk. So then I needed a toolchain-userland.cmake file to be able to use it, and this is what I came up with:
set(CMAKE_SYSTEM_NAME Linux) set(CMAKE_SYSTEM_PROCESSOR arm) set(MY_GCC_TOOLCHAIN /Volumes/CASE/my-os-pi3/toolchain_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf) set(MY_CLANG_TOOLCHAIN /Volumes/CASE/my-os-pi3/toolchain_userland_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf) SET(CROSS_MACHINE_FLAGS "-marm -mcpu=cortex-a53 -mfloat-abi=hard -mfpu=vfp") set(MY_COMMON_COMPILE_FLAGS "--target=arm-openadk-linux-uclibceabihf ${CROSS_MACHINE_FLAGS} --gcc-toolchain=${MY_GCC_TOOLCHAIN}/usr") set(MY_COMMON_LINKER_FLAGS "--target=arm-openadk-linux-uclibceabihf -B ${MY_GCC_TOOLCHAIN}/usr/arm-openadk-linux-uclibceabihf/bin -fuse-ld=bfd -L${MY_GCC_TOOLCHAIN}/usr/lib -L${MY_GCC_TOOLCHAIN}/usr/lib/gcc/arm-openadk-linux-uclibceabihf/7.2.0 --gcc-toolchain=${MY_GCC_TOOLCHAIN}/usr") set(CMAKE_C_COMPILER ${MY_CLANG_TOOLCHAIN}/usr/bin/clang) set(CMAKE_CXX_COMPILER ${MY_CLANG_TOOLCHAIN}/usr/bin/clang++) set(CMAKE_C_FLAGS "-fwrapv -fno-ident ${MY_COMMON_COMPILE_FLAGS} " CACHE STRING "OpenADK CFLAGS" FORCE) set(CMAKE_CXX_FLAGS "-fwrapv -fno-ident -stdlib=libstdc++ -I${MY_GCC_TOOLCHAIN}/usr/arm-openadk-linux-uclibceabihf/include/c++/7.2.0 -I${MY_GCC_TOOLCHAIN}/usr/arm-openadk-linux-uclibceabihf/include/c++/7.2.0/arm-openadk-linux-uclibceabihf ${MY_COMMON_COMPILE_FLAGS}" CACHE STRING "OpenADK CXXFLAGS" FORCE) set(CMAKE_EXE_LINKER_FLAGS ${MY_COMMON_LINKER_FLAGS} CACHE STRING "OpenADK LDFLAGS" FORCE) set(CMAKE_MODULE_LINKER_FLAGS ${MY_COMMON_LINKER_FLAGS} CACHE STRING "OpenADK LDFLAGS" FORCE) set(CMAKE_SHARED_LINKER_FLAGS ${MY_COMMON_LINKER_FLAGS} CACHE STRING "OpenADK LDFLAGS" FORCE) set(CMAKE_STATIC_LINKER_FLAGS ${MY_COMMON_LINKER_FLAGS} CACHE STRING "OpenADK LDFLAGS" FORCE) set(MY_SYSTEM_INCLUDE_DIRS ${MY_GCC_TOOLCHAIN}/usr/arm-openadk-linux-uclibceabihf/include/c++/7.2.0 ${MY_GCC_TOOLCHAIN}/usr/arm-openadk-linux-uclibceabihf/include/c++/7.2.0/arm-openadk-linux-uclibceabihf) set(CMAKE_INSTALL_SO_NO_EXE 0) set(CMAKE_PROGRAM_PATH "/Volumes/CASE/my-os-pi3/host_x86_64-apple-darwin16.7.0/usr/bin") set(CMAKE_SYSROOT "/Volumes/CASE/my-os-pi3/target_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf") set(CMAKE_FIND_ROOT_PATH "/Volumes/CASE/my-os-pi3/target_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf") set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) set(ENV{PKG_CONFIG_SYSROOT_DIR} "/Volumes/CASE/my-os-pi3/target_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf") set(ENV{PKG_CONFIG_LIBDIR} "/Volumes/CASE/my-os-pi3/target_raspberry-pi3-cm_uclibc-ng_cortex_a53_hard_eabihf/usr/lib/pkgconfig:@@STAGING_TARGET_DIR@@/usr/share/pkgconfig")
Note: First I tried using -fuse-ld=lld but it can not build the final app: it keeps searching in /lib/ on my host for libc.so.1 and 2 other libs(uclibc libs). If I then add -nostdlib then cmake can compile his testprograms for C and C++. And then all the shared libs of my app build fine, except when linking the final application with all of these libs, it does not find some cxx11 strings and sigint functions: it indicates several undefined references. Weird because I say -stdlib=libstdc++, and those undefined references look like libc++ ones.
So I switched to using the standard “bfd” linker and this compiles and links my libs+app fine. Compilation time is now down to 2 minutes, so I already like it ;)
But, running it failed, but the only thing that I had to do was add a symlink /lib/ld-linux-armhf.so.3 pointing to /lib/ld-uClibc-1.0.26.so and then it runs on my target system.
When I run it, it all seems to work, but an exception thrown from within one of the shared libraries is not caught in the main application. It just hangs there at that particular line. So I tried with the regular toolchain.cmake file (which uses gcc) and there that also didn’t work. So then I removed the "-fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables” options(hardcoded in vars.mk) and rebuild my entire os. Then using the gcc toolchain, it worked fine, and the exception coming from the shared lib was caught in the main application.
But trying that now with the clang toolchain, it still hangs in the shared lib… So try-catch and exceptions work inside these libs, but when one is uncaught, it just hangs there. So now I have the situation where it almost works, but these exceptions are now the only remaining problem I have. Any clues what could be wrong still?
Thanks Best regards, Tom,
Hi Tom, Tom Deblauwe wrote,
Hello,
I am using openadk on the raspberry pi 3, and have an appliance where I have one main application that is running all the time. So I am using gcc 7.2.0 from openadk, and compilation starts to be *very slow* with gcc: 10 minutes for a full rebuild... So I am looking into having the clang toolchain available next to the gcc one so I can compile only my application with clang, which is faster. I have several shared libraries I also compile myself as part of my application build, all using cmake.
Nice use case (llvm for userland).
Note: First I tried using -fuse-ld=lld but it can not build the final app: it keeps searching in /lib/ on my host for libc.so.1 and 2 other libs(uclibc libs). If I then add -nostdlib then cmake can compile his testprograms for C and C++. And then all the shared libs of my app build fine, except when linking the final application with all of these libs, it does not find some cxx11 strings and sigint functions: it indicates several undefined references. Weird because I say -stdlib=libstdc++, and those undefined references look like libc++ ones.
So I switched to using the standard “bfd” linker and this compiles and links my libs+app fine. Compilation time is now down to 2 minutes, so I already like it ;)
But, running it failed, but the only thing that I had to do was add a symlink / lib/ld-linux-armhf.so.3 pointing to /lib/ld-uClibc-1.0.26.so and then it runs on my target system.
I believe llvm/clang doesn't know anything about uClibc-ng and the shared library loader as gcc does.
When I run it, it all seems to work, but an exception thrown from within one of the shared libraries is not caught in the main application. It just hangs there at that particular line. So I tried with the regular toolchain.cmake file (which uses gcc) and there that also didn’t work. So then I removed the "-fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables” options(hardcoded in vars.mk) and rebuild my entire os. Then using the gcc toolchain, it worked fine, and the exception coming from the shared lib was caught in the main application.
I hope I fixed it in c0c6ccec7881c92e0a96fa9114af553c4434d83b. They are no longer hardcoded and must be enabled by the developer/user.
But trying that now with the clang toolchain, it still hangs in the shared lib… So try-catch and exceptions work inside these libs, but when one is uncaught, it just hangs there. So now I have the situation where it almost works, but these exceptions are now the only remaining problem I have. Any clues what could be wrong still?
Sounds like some llvm/clang bug not generating the right code? In both cases you have libgcc.so.1 on your target, which is used inside uClibc-ng/glibc. I am not sure if it is relevant to C++ exception handling. C++ is a mystique for me.
You could verify with glibc, if the remaining issue is llvm+uClibc-ng related. If it breaks for a gflibc system, it shouldn't be related.
Could you prepare a patch against master, so that others can benefit from your LLVM for userland system?
best regards Waldemar
Hello,
I changed to using glibc and it works now! Thanks for the tip…
I will look into giving my changes back, but now I am short on time unfortunately, but I will do it.
Best regards, Tom,
On 14 Jun 2018, at 06:08, Waldemar Brodkorb wbx@openadk.org wrote:
Hi Tom, Tom Deblauwe wrote,
Hello,
I am using openadk on the raspberry pi 3, and have an appliance where I have one main application that is running all the time. So I am using gcc 7.2.0 from openadk, and compilation starts to be *very slow* with gcc: 10 minutes for a full rebuild... So I am looking into having the clang toolchain available next to the gcc one so I can compile only my application with clang, which is faster. I have several shared libraries I also compile myself as part of my application build, all using cmake.
Nice use case (llvm for userland).
Note: First I tried using -fuse-ld=lld but it can not build the final app: it keeps searching in /lib/ on my host for libc.so.1 and 2 other libs(uclibc libs). If I then add -nostdlib then cmake can compile his testprograms for C and C++. And then all the shared libs of my app build fine, except when linking the final application with all of these libs, it does not find some cxx11 strings and sigint functions: it indicates several undefined references. Weird because I say -stdlib=libstdc++, and those undefined references look like libc++ ones.
So I switched to using the standard “bfd” linker and this compiles and links my libs+app fine. Compilation time is now down to 2 minutes, so I already like it ;)
But, running it failed, but the only thing that I had to do was add a symlink / lib/ld-linux-armhf.so.3 pointing to /lib/ld-uClibc-1.0.26.so and then it runs on my target system.
I believe llvm/clang doesn't know anything about uClibc-ng and the shared library loader as gcc does.
When I run it, it all seems to work, but an exception thrown from within one of the shared libraries is not caught in the main application. It just hangs there at that particular line. So I tried with the regular toolchain.cmake file (which uses gcc) and there that also didn’t work. So then I removed the "-fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables” options(hardcoded in vars.mk) and rebuild my entire os. Then using the gcc toolchain, it worked fine, and the exception coming from the shared lib was caught in the main application.
I hope I fixed it in c0c6ccec7881c92e0a96fa9114af553c4434d83b. They are no longer hardcoded and must be enabled by the developer/user.
But trying that now with the clang toolchain, it still hangs in the shared lib… So try-catch and exceptions work inside these libs, but when one is uncaught, it just hangs there. So now I have the situation where it almost works, but these exceptions are now the only remaining problem I have. Any clues what could be wrong still?
Sounds like some llvm/clang bug not generating the right code? In both cases you have libgcc.so.1 on your target, which is used inside uClibc-ng/glibc. I am not sure if it is relevant to C++ exception handling. C++ is a mystique for me.
You could verify with glibc, if the remaining issue is llvm+uClibc-ng related. If it breaks for a gflibc system, it shouldn't be related.
Could you prepare a patch against master, so that others can benefit from your LLVM for userland system?
best regards Waldemar
-
Tom Deblauwe -
Waldemar Brodkorb