Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flaky test] org.yb.pgsql.TestPgIsolationRegress.isolationRegress #11662

Closed
bmatican opened this issue Mar 4, 2022 · 0 comments
Closed

[flaky test] org.yb.pgsql.TestPgIsolationRegress.isolationRegress #11662

bmatican opened this issue Mar 4, 2022 · 0 comments
Assignees
Labels
area/ysql Yugabyte SQL (YSQL) kind/failing-test Tests and testing infra

Comments

@bmatican
Copy link
Contributor

bmatican commented Mar 4, 2022

Description

high failure in ASAN

https://detective-gcp.dev.yugabyte.com/stability/test?branch=master&build_type=asan&class=org.yb.pgsql.TestPgIsolationRegress&name=isolationRegress&num_commits=50&platform=all

@bmatican bmatican added area/ysql Yugabyte SQL (YSQL) kind/failing-test Tests and testing infra labels Mar 4, 2022
@bmatican bmatican added this to YQL-beta Mar 4, 2022
pkj415 added a commit that referenced this issue Mar 10, 2022
Summary:
The asan issue occurs because of a memory leak in the isolation regress test
suite's spec parser if we have a comment block in setup with some SQL reserved
words. The error is as below -

```
+ =================================================================
+ ==3009==ERROR: LeakSanitizer: detected memory leaks
+
+ Direct leak of 2048 byte(s) in 1 object(s) allocated from:
+     #0 0x7f7df2412cd9 in realloc /opt/yb-build/llvm/yb-llvm-v12.0.1-yb-1-1633099823-bdb147e6-centos7-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:164:3
+     #1 0x40eed7 in pg_realloc /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/common/../../../../../src/postgres/src/common/fe_memutils.c:72:8
+     #2 0x405e31 in addlitchar /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l:116:12
+     #3 0x40542f in spec_yylex /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l
+     #4 0x402ae0 in spec_yyparse /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specparse.c:1190:16
+     #5 0x409349 in main /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/../../../../../../src/postgres/src/test/isolation/isolationtester.c:112:2
+     #6 0x7f7df12a3554 in __libc_start_main (/lib64/libc.so.6+0x22554)
+
+ Objects leaked above:
+ 0x61d000000080 (2048 bytes)
+
+ SUMMARY: AddressSanitizer: 2048 byte(s) leaked in 1 allocation(s).

======================================================================

```
I found which part of the spec caused the issue by just performing a binary
search and running different parts of the spec instead of reading the code
(since that would take more time and effort). And it is evident that the comment
block with reserved words was the issue. The fix is to just move those blocks
outside.

I have also increased the timeout since the test fails occsionaly with a timeout
error. This happened after I added more test schedules to TestPgIsolationRegress
as part of D15383.

Test Plan: ./yb_build.sh --java-test org.yb.pgsql.TestPgIsolationRegress

Reviewers: mtakahara

Reviewed By: mtakahara

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D15886
pkj415 added a commit that referenced this issue Mar 10, 2022
…ress

Summary:
The asan issue occurs because of a memory leak in the isolation regress test
suite's spec parser if we have a comment block in setup with some SQL reserved
words. The error is as below -

+ =================================================================
+ ==3009==ERROR: LeakSanitizer: detected memory leaks
+
+ Direct leak of 2048 byte(s) in 1 object(s) allocated from:
+     #0 0x7f7df2412cd9 in realloc /opt/yb-build/llvm/yb-llvm-v12.0.1-yb-1-1633099823-bdb147e6-centos7-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:164:3
+     #1 0x40eed7 in pg_realloc /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/common/../../../../../src/postgres/src/common/fe_memutils.c:72:8
+     #2 0x405e31 in addlitchar /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l:116:12
+     #3 0x40542f in spec_yylex /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l
+     #4 0x402ae0 in spec_yyparse /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specparse.c:1190:16
+     #5 0x409349 in main /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/../../../../../../src/postgres/src/test/isolation/isolationtester.c:112:2
+     #6 0x7f7df12a3554 in __libc_start_main (/lib64/libc.so.6+0x22554)
+
+ Objects leaked above:
+ 0x61d000000080 (2048 bytes)
+
+ SUMMARY: AddressSanitizer: 2048 byte(s) leaked in 1 allocation(s).

======================================================================

I found which part of the spec caused the issue by just performing a binary
search and running different parts of the spec instead of reading the code
(since that would take more time and effort). And it is evident that the comment
block with reserved words was the issue. The fix is to just move those blocks
outside.

I have also increased the timeout since the test fails occsionaly with a timeout
error. This happened after I added more test schedules to TestPgIsolationRegress
as part of D15383.

Original commit: https://phabricator.dev.yugabyte.com/D15886, 28902bf

Test Plan:
Jenkins: rebase: 2.12

./yb_build.sh --java-test org.yb.pgsql.TestPgIsolationRegress

Reviewers: mtakahara

Reviewed By: mtakahara

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D15893
@pkj415 pkj415 closed this as completed Mar 10, 2022
@pkj415 pkj415 moved this to Done in YQL-beta Mar 10, 2022
pkj415 added a commit to pkj415/yugabyte-db that referenced this issue Mar 22, 2022
Summary:
The asan issue occurs because of a memory leak in the isolation regress test
suite's spec parser if we have a comment block in setup with some SQL reserved
words. The error is as below -

```
+ =================================================================
+ ==3009==ERROR: LeakSanitizer: detected memory leaks
+
+ Direct leak of 2048 byte(s) in 1 object(s) allocated from:
+     #0 0x7f7df2412cd9 in realloc /opt/yb-build/llvm/yb-llvm-v12.0.1-yb-1-1633099823-bdb147e6-centos7-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:164:3
+     yugabyte#1 0x40eed7 in pg_realloc /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/common/../../../../../src/postgres/src/common/fe_memutils.c:72:8
+     yugabyte#2 0x405e31 in addlitchar /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l:116:12
+     yugabyte#3 0x40542f in spec_yylex /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specscanner.l
+     yugabyte#4 0x402ae0 in spec_yyparse /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/specparse.c:1190:16
+     yugabyte#5 0x409349 in main /nfusr/dev-server/pjain/code/yugabyte-db/src/postgres/src/test/isolation/../../../../../../src/postgres/src/test/isolation/isolationtester.c:112:2
+     yugabyte#6 0x7f7df12a3554 in __libc_start_main (/lib64/libc.so.6+0x22554)
+
+ Objects leaked above:
+ 0x61d000000080 (2048 bytes)
+
+ SUMMARY: AddressSanitizer: 2048 byte(s) leaked in 1 allocation(s).

======================================================================

```
I found which part of the spec caused the issue by just performing a binary
search and running different parts of the spec instead of reading the code
(since that would take more time and effort). And it is evident that the comment
block with reserved words was the issue. The fix is to just move those blocks
outside.

I have also increased the timeout since the test fails occsionaly with a timeout
error. This happened after I added more test schedules to TestPgIsolationRegress
as part of D15383.

Test Plan: ./yb_build.sh --java-test org.yb.pgsql.TestPgIsolationRegress

Reviewers: mtakahara

Reviewed By: mtakahara

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D15886
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ysql Yugabyte SQL (YSQL) kind/failing-test Tests and testing infra
Projects
Status: Done
Development

No branches or pull requests

2 participants