Crash when building multiple packages

[mssun]

To produce:

$ git clone git@github.com:mesalock-linux/mkpkg.git
$ cd mkpkg && cargo build --release
$ git clone -b yaml git@github.com:mesalock-linux/packages.git
$ docker run -v $(pwd):/mkpkg -w /mkpkg -it mesalocklinux/build-mesalock-linux /bin/bash -c "./target/release/mkpkg build packages/core/filesystem/build.yml \
packages/core/minit/build.yml \
packages/core/tzdata/build.yml \
packages/core/init/build.yml \
packages/core/gcc-libs/build.yml \
packages/core/giproute2/build.yml \
packages/core/ion-shell/build.yml \
packages/core/uutils-coreutils/build.yml \
packages/core/mgetty/build.yml \
packages/core/uutils-findutils/build.yml \
packages/core/linux/build.yml \
packages/core/syslinux/build.yml \
packages/core/ripgrep/build.yml \
packages/core/glibc/build.yml"

# after all package finished, it will report

*** longjmp causes uninitialized stack frame ***: ./target/release/mkpkg terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc0b2f1a7e5]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7fc0b2fbc15c]
/lib/x86_64-linux-gnu/libc.so.6(+0x11905d)[0x7fc0b2fbc05d]
/lib/x86_64-linux-gnu/libc.so.6(__longjmp_chk+0x29)[0x7fc0b2fbbfb9]
./target/release/mkpkg(+0x23ecd5)[0x55f5a9153cd5]
/lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7fc0b2ed84b0]
/lib/x86_64-linux-gnu/libpthread.so.0(pthread_cond_wait+0xc0)[0x7fc0b2a8f360]
./target/release/mkpkg(+0x3ca362)[0x55f5a92df362]
./target/release/mkpkg(+0x3ba590)[0x55f5a92cf590]
./target/release/mkpkg(+0x28af4a)[0x55f5a919ff4a]
./target/release/mkpkg(+0x283c51)[0x55f5a9198c51]
./target/release/mkpkg(+0x282802)[0x55f5a9197802]
./target/release/mkpkg(+0x28266b)[0x55f5a919766b]
./target/release/mkpkg(+0x8bc21)[0x55f5a8fa0c21]
./target/release/mkpkg(+0xa8a89)[0x55f5a8fbda89]
./target/release/mkpkg(+0xb4984)[0x55f5a8fc9984]
./target/release/mkpkg(+0xaa7c6)[0x55f5a8fbf7c6]
./target/release/mkpkg(+0xaec03)[0x55f5a8fc3c03]
./target/release/mkpkg(+0x3c92d8)[0x55f5a92de2d8]
./target/release/mkpkg(+0x3dcd1f)[0x55f5a92f1d1f]
./target/release/mkpkg(+0x3c6416)[0x55f5a92db416]
./target/release/mkpkg(+0xab194)[0x55f5a8fc0194]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fc0b2ec3830]
./target/release/mkpkg(+0x4a6d9)[0x55f5a8f5f6d9]

Attached is a full crash log:

dump.txt

[Arcterus]

After examining the implementation of channels in the standard library and how indicatif works, I am reasonably confident that the issue is this line. recv_timeout() seems to wait on a condvar from libpthread (using pthread_cond_wait(), as seen above), which internally uses setjmp()/longjmp(). setjmp()/longjmp() don't really play well with Rust, so I think when recv_timeout() is combined with crossbeam and/or indicatif things start falling apart.

The solution here would be to find something that offers similar functionality to our current usage of recv_timeout() (basically all we use it for is to exit the thread if it does not time out) without relying on something that uses setjmp()/longjmp() internally. AFAICT this rules out crossbeam-channel (as it uses thread::park_timeout(), which is where the condvar comes from in recv_timeout()).

I'm not sure, but indicatif may also be an issue as it uses channels as well. Internally, all the recv operations seem to use thread::park_timeout(), which is what uses the condvar.

[Arcterus]

Actually, from further examination, it seems that thread::park_timeout() calls pthread_cond_timedwait() rather than pthread_cond_wait(), so the issue seems to lie elsewhere. It seems like the issue might be related to a call to recv() rather than recv_timeout() as the former calls Condvar::wait() (which uses pthread_cond_wait() internally). MultiProgress::join() uses recv() internally, so that might explain why it only fails at the end.