Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: acceptance/version-upgrade failed #53812

Closed
cockroach-teamcity opened this issue Sep 2, 2020 · 2 comments
Closed

roachtest: acceptance/version-upgrade failed #53812

cockroach-teamcity opened this issue Sep 2, 2020 · 2 comments
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Milestone
20.2

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).acceptance/version-upgrade failed on release-19.2@ce736a5705d3a16e793cd4adc88cc87c15c97c9a:

		  |  1589.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1590.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1591.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1592.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1593.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1594.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1595.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1596.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1597.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1598.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1599.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1600.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  |  1601.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1602.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1603.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1604.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1605.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1606.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1607.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1608.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1609.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1610.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1611.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1612.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1613.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1614.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1615.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1616.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1617.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1618.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1619.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1620.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  |  1621.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1622.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1623.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1624.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1625.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1626.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1627.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1628.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1629.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1630.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		Wraps: (4) secondary error attachment
		  | signal: killed
		  | (1) signal: killed
		  | Error types: (1) *exec.ExitError
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString
More

Artifacts: /acceptance/version-upgrade

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-release-19.2 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Sep 2, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone Sep 2, 2020
@cockroach-teamcity cockroach-teamcity mentioned this issue Sep 4, 2020
Closed
@tbg
Copy link
Member

tbg commented Sep 8, 2020

Looks like the schema change step timed out

05:54:08 mixed_version_schemachange.go:54: Workload step run: 2
05:54:08 cluster.go:2261: [w13] > ./workload run schemachange --verbose=1 --tolerate-errors=true --max-ops 10 --concurrency 2 {pgurl:1-4}
06:21:19 test_runner.go:774: [w13] dumped stacks to __stacks
06:21:19 test.go:189: test status: stopping cluster
06:21:19 cluster.go:382: > /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=11 teamcity-2240609-1599025771-10-n4cpu4:1-4
teamcity-2240609-1599025771-10-n4cpu4: stopping
06:21:20 test.go:337: test failure: 	test_runner.go:801: test timed out (30m0s)
06:21:20 test.go:337: [not the first failure] test failure: 
	cluster.go:2197,mixed_version_schemachange.go:67,versionupgrade.go:190,versionupgrade.go:178,acceptance.go:54,acceptance.go:91,test_runner.go:754: output in run_055408.214_n3_workload_run_schemachange: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2240609-1599025771-10-n4cpu4:3 -- ./workload run schemachange --verbose=1 --tolerate-errors=true --max-ops 10 --concurrency 2 {pgurl:1-4} returned: context canceled
		(1) attached stack trace
		  -- stack trace:
		  | main.(*cluster).RunE

Its logs are not helpful: https://teamcity.cockroachdb.com/repository/download/Cockroach_Nightlies_WorkloadNightly/2240609:id/acceptance/version-upgrade/run_1/run_055408.214_n3_workload_run_schemachange.log

This was referenced Sep 9, 2020
Closed
Closed
@thoszhang thoszhang removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Sep 9, 2020
This was referenced Nov 17, 2020
Closed
Closed
@cockroach-teamcity cockroach-teamcity mentioned this issue Dec 21, 2020
Closed
@cockroach-teamcity cockroach-teamcity mentioned this issue Dec 29, 2020
Closed
This was referenced Jan 6, 2021
Closed
Closed
Closed
This was referenced Feb 9, 2021
Closed
Closed
@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 11, 2021
Closed
@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 19, 2021
Closed
@andreimatei
Copy link
Contributor

release-19.2 and old report. closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Projects
None yet
Milestone
20.2
Development

No branches or pull requests
4 participants
@andreimatei @thoszhang @tbg @cockroach-teamcity