Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: acceptance/bank/zerosum-restart failed [skipped] #33683

Closed
cockroach-teamcity opened this issue Jan 12, 2019 · 5 comments · Fixed by #53693
Closed

roachtest: acceptance/bank/zerosum-restart failed [skipped] #33683

cockroach-teamcity opened this issue Jan 12, 2019 · 5 comments · Fixed by #53693
Assignees
@knz
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Milestone
19.1

Comments

@cockroach-teamcity
Copy link
Member

SHA: https://github.com/cockroachdb/cockroach/commits/19860c9f7b02a698109ff3905b93c384cdd8d6d5

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=acceptance/bank/zerosum-restart PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1090188&tab=buildLog

The test failed on master:
	test.go:696,bank.go:375,bank.go:569,acceptance.go:72: pq: cannot up-replicate to s3; missing gossiped StoreDescriptor
	test.go:696,cluster.go:1130,bank.go:247: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod stop teamcity-1090188-acceptance:1 returned:
		stderr:
		
		stdout:
		teamcity-1090188-acceptance: stopping and waiting: signal: killed
@cockroach-teamcity cockroach-teamcity added this to the 2.2 milestone Jan 12, 2019
@cockroach-teamcity cockroach-teamcity added C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Jan 12, 2019
@andreimatei
Copy link
Contributor

Also ran into this here: https://teamcity.cockroachdb.com/viewLog.html?buildId=1091735&tab=buildResultsDiv&buildTypeId=Cockroach_UnitTests_Roachtest

@andreimatei andreimatei mentioned this issue Jan 13, 2019
Merged
@andreimatei
Copy link
Contributor

I've ran into another error on the same test that seems related:

test.go:696,bank.go:375,bank.go:569,acceptance.go:72: pq: error looking up store 4: KeyNotPresentError: gossip key "store:4" does not exist or has expired

https://teamcity.cockroachdb.com/viewLog.html?buildId=1091750&buildTypeId=Cockroach_UnitTests

tbg added a commit to tbg/cockroach that referenced this issue Jan 14, 2019
@tbg
roachtest: add more exceptions to bank/zerosum-restart
29f70cc 
Fixes cockroachdb#33683.

Release note: None
@tbg tbg mentioned this issue Jan 14, 2019
Merged
@tbg tbg assigned petermattis and unassigned tbg Jan 15, 2019
craig bot pushed a commit that referenced this issue Jan 16, 2019
@tbg
Merge #33707
5b306cc 
33707: roachtest: add more exceptions to bank/zerosum-restart r=petermattis a=tbg

Fixes #33683.

Release note: None

Co-authored-by: Tobias Schottdorf <tobias.schottdorf@gmail.com>
@tbg tbg changed the title roachtest: acceptance/bank/zerosum-restart failed roachtest: acceptance/bank/zerosum-restart failed [skipped] Jan 16, 2019
craig bot pushed a commit that referenced this issue Jan 16, 2019
@tbg @yuzefovich
Merge #33707 #34018
7522b63 
33707: roachtest: add more exceptions to bank/zerosum-restart r=petermattis a=tbg

Fixes #33683.

Release note: None

34018: exec: sort indices of the selection vector in the testing framework r=yuzefovich a=yuzefovich

The selection vector is shuffled, so the indices corresponding to
the actual tuples would be in a random order; however, the columnar
operators process and output the vectors sequentially, so we should
sort the indices corresponding to the actual testing data.

Release note: None

Co-authored-by: Tobias Schottdorf <tobias.schottdorf@gmail.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
@craig craig bot closed this as completed in #33707 Jan 16, 2019
@andreimatei
Copy link
Contributor

Tobi, I don't think you wanted to close this, did you? Reopening.

running before the test was skipped, I also ran into this:

	test.go:727,bank.go:375,bank.go:569,acceptance.go:72: pq: unable to select removal target from [(n2,s2):?]; current replicas [(n2,s2):4 (n4,s4):2 (n1,s1):5 (n3,s3):?]: could not select an appropriate replica to be removed

@andreimatei andreimatei reopened this Jan 16, 2019
tbg added a commit to tbg/cockroach that referenced this issue Jan 17, 2019
@tbg
acceptance: properly skip acceptance/bank/zerosum-restart
e830bb6 
Touches cockroachdb#33683.

Release note: None
@tbg tbg mentioned this issue Jan 17, 2019
Merged
@tbg
Copy link
Member

tbg commented Jan 17, 2019

Thanks @andreimatei, added to the description within #34080.

craig bot pushed a commit that referenced this issue Jan 17, 2019
@tbg
Merge #34080
69ec40a 
34080: acceptance: properly skip acceptance/bank/zerosum-restart r=petermattis a=tbg

Touches #33683.

Release note: None

Co-authored-by: Tobias Schottdorf <tobias.schottdorf@gmail.com>
@tbg tbg assigned knz and unassigned petermattis Apr 25, 2019
@tbg tbg added the branch-master Failures and bugs on the master branch. label Jan 22, 2020
@knz
Copy link
Contributor

knz commented Aug 31, 2020

@tbg in #34080 I think you skipped the wrong test...

@knz knz mentioned this issue Aug 31, 2020
Merged
@craig craig bot closed this as completed in f2056ba Sep 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@knz knz

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Projects
None yet
Milestone
19.1
Development

Successfully merging a pull request may close this issue.

roachtest: re-enable acceptance/bank/zerosum-restart knz/cockroach
6 participants
@jordanlewis @andreimatei @knz @tbg @petermattis @cockroach-teamcity