[DocDB] flaky test: BlockingQueueDeathTest.TestPointerParamsMustBeEmptyOnDestruct #13895
Assignees
Labels
area/docdb
YugabyteDB core features
kind/bug
This issue is a bug
kind/failing-test
Tests and testing infra
priority/high
High Priority
Comments
bmatican
added
area/docdb
bmatican
added
kind/failing-test
ttyusupov
added a commit
that referenced
this issue
Sep 22, 2022
Summary:
BlockingQueueDeathTest.TestPointerParamsMustBeEmptyOnDestruct was failing under ASAN on Jenkins.
```
Result: died but not with expected error.
Expected: BlockingQueue holds bare pointers
Actual msg:
[ DEATH ]
```
It turned out that `gtest-death-test_test` gtest's own unit-tests are also failing on Jenkins with the common failure pattern:
```
[ DEATH ] AddressSanitizer:DEADLYSIGNAL
[ DEATH ] =================================================================
[ DEATH ] ==2262872==ERROR: AddressSanitizer: stack-overflow on address 0x7f823b324f80 (pc 0x7f823b2edc03 bp 0x7f823b325fb0 sp 0x7f823b324f80 T0)
[ DEATH ] #0 0x7f823b2edc03 (/lib64/ld-linux-x86-64.so.2+0x1c03)
[ DEATH ] #1 0x7f823a362684 (/nfusr/alma8-gcp-cloud/jenkins-worker-74vryc/jenkins/ty/yugabyte-db-thirdparty/build/asan/gmock-1.8.0/shared/googlemock/gtest/libgtest.so+0x134684)
[ DEATH ] #2 0x7f8239900dd2 (/lib64/libc.so.6+0x39dd2)
[ DEATH ]
[ DEATH ] SUMMARY: AddressSanitizer: stack-overflow (/lib64/ld-linux-x86-64.so.2+0x1c03)
[ DEATH ] ==2262872==ABORTING
[ DEATH ]
```
Which is related to https://chromium.googlesource.com/external/github.com/pwnall/googletest/+/681454dae48f109abf68c424c9d2e6db9a092238. With ASAN on x86_64, ExecDeathTestChildMain has frame size of 1728 bytes. Call to `chdir()` in `ExecDeathTestChildMain` ends up in `_dl_runtime_resolve_xsavec`, which attempts to save register state on the stack. And `XSAVE register save area size` is larger than 1728 bytes on Jenkins nodes:
```
$ cpuid -i | grep -i xsave
...
XSAVE features (0xd/0):
bytes required by XSAVE/XRSTOR area = 0x00000a80 (2688)
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) CPU @ 2.80GHz
```
GoogleTest 1.11.0 has a fix for this issue: google/googletest@681454d.
Decided to upgrade to latest GoogleTest 1.12.0 that also contains the fix.
Test Plan: - Jenkins
Reviewers: esheng, jhe, mbautin
Reviewed By: esheng, jhe, mbautin
Subscribers: bogdan, ybase
Differential Revision: https://phabricator.dev.yugabyte.com/D19390
spolitov
added a commit
that referenced
this issue
Sep 26, 2022
Summary: Previously we were building 3rd party using c++20 on linux and c++17 on mac. It was happening because used mac github actions image contains apple clang 12. And apple clang 12 does not support c++20, but it supports c++2a. So new thirdparty is built using c++2a on mac, except icu4c. Since icu4c cannot be built in this mode due to compiler bug, so we use c++17 for it. This uses new thirdparty built with c++2a on mac. Also fixed issue with build_postgres.py on some machines. Test Plan: Jenkins Reviewers: mbautin Reviewed By: mbautin Subscribers: bogdan, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D19783
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-3393
Description
https://detective-gcp.dev.yugabyte.com/stability/test?branch=master&build_type=all&class=BlockingQueueDeathTest&fail_tag=all&name=TestPointerParamsMustBeEmptyOnDestruct&platform=linux
The text was updated successfully, but these errors were encountered: