Hello,
We have noticed random deadlocks in malloc/free when using uClibc-ng + Thumb1 + NPTL compiled for ARMv6 in multi-threaded environment.
Steps to reproduce
1. Build Buildroot 2019.02.9 SDK with the following configuration:
BR2_arm=y BR2_arm1176jz_s=y BR2_ARM_INSTRUCTIONS_THUMB=y BR2_SHARED_STATIC_LIBS=y BR2_TOOLCHAIN_BUILDROOT_WCHAR=y BR2_TOOLCHAIN_BUILDROOT_USE_SSP=y BR2_TOOLCHAIN_BUILDROOT_CXX=y
2. Cross compile the sample (see below) application with the SDK using static linking:
$ arm-buildroot-linux-uclibcgnueabi-gcc -static -g deadlock.c \ -lpthread -o deadlock
3. Run the application with at least two threads
3.1. Run the application using QEMU static emulation:
$ qemu-arm-static ./deadlock 4 1000000
3.2. Run the application on a device with ARMv6 support (ex. any RPi with Raspbian image):
$ ./deadlock 4 1000000
Expected results:
The application should always exit cleanly.
Actual results:
The application often deadlocks in both cases.
Notes:
1. The bug is also reproducible with latest uClibc-ng (v1.0.32)
2. According to our tests the endianness does not matter
3. The callstack when deadlock appears:
(gdb) info threads Id Target Id Frame * 1 LWP 11653 "mutex" 0x000165b2 in pthread_join () 2 LWP 11656 "mutex" 0x00014cac in __lll_lock_wait () 3 LWP 11657 "mutex" 0x00014ce2 in __lll_lock_wait () 4 LWP 11658 "mutex" 0x00014cac in __lll_lock_wait () 5 LWP 11659 "mutex" 0x00014cac in __lll_lock_wait ()
(gdb) t 1 [Switching to thread 1 (LWP 11653)] #0 0x000165b2 in pthread_join ()
(gdb) bt #0 0x000165b2 in pthread_join () #1 0x00010396 in main (argc=3, argv=0x7efff784) at deadlock.c:42
(gdb) t 2 [Switching to thread 2 (LWP 11656)] #0 0x00014cac in __lll_lock_wait ()
(gdb) bt #0 0x00014cac in __lll_lock_wait () #1 0x00016b7e in pthread_mutex_lock () #2 0x0001199e in __libc_malloc () #3 0x00010278 in do_request (data=0x1) at deadlock.c:13 #4 0x00015b36 in start_thread () #5 0x000173f4 in clone ()
(gdb) t 3 [Switching to thread 3 (LWP 11657)] #0 0x00014ce2 in __lll_lock_wait ()
(gdb) bt #0 0x00014ce2 in __lll_lock_wait () #1 0x00016b7e in pthread_mutex_lock () #2 0x000117c2 in __libc_free () #3 0x00010284 in do_request (data=0x2) at deadlock.c:15 #4 0x00015b36 in start_thread () #5 0x000173f4 in clone ()
(gdb) t 4 [Switching to thread 4 (LWP 11658)] #0 0x00014cac in __lll_lock_wait ()
(gdb) bt #0 0x00014cac in __lll_lock_wait () #1 0x00016b7e in pthread_mutex_lock () #2 0x0001199e in __libc_malloc () #3 0x00010278 in do_request (data=0x3) at deadlock.c:13 #4 0x00015b36 in start_thread () #5 0x000173f4 in clone ()
(gdb) t 5 [Switching to thread 5 (LWP 11659)] #0 0x00014cac in __lll_lock_wait ()
(gdb) bt #0 0x00014cac in __lll_lock_wait () #1 0x00016b7e in pthread_mutex_lock () #2 0x0001199e in __libc_malloc () #3 0x00010278 in do_request (data=0x4) at deadlock.c:13 #4 0x00015b36 in start_thread () #5 0x000173f4 in clone ()
Sample application:
#include <stdlib.h> #include <stdio.h> #include <pthread.h>
int count;
void *do_request(void *data) { for (int i = 0; i < count; i++) { if (i % 1000 == 0) printf("Alloc for %p count %d\n", data, i); char *buffer = malloc(32 * sizeof(char));
free(buffer); } return NULL; }
int main(int argc, char *argv[]) { int i, rc; int threads_count = atoi(argv[1]);
count = atoi(argv[2]);
pthread_t threads[threads_count]; printf("Creating %d threads\n", threads_count);
for (i = 0; i < threads_count; i++) { rc = pthread_create(&threads[i], NULL, do_request, (int*)(i+1));
if (rc) { printf("Error creating thread\n"); exit(1); } }
for (i = 0; i < threads_count; i++) if (pthread_join(threads[i], NULL)) { printf("Error joining thread\n"); exit(1); }
printf("Done. \n"); return 0; }
Best regards, Gyula Farkas