roachtest: schemachange/indexrollback/tpcc-1000 failed #34159

cockroach-teamcity · 2019-01-21T22:26:54Z

SHA: https://github.com/cockroachdb/cockroach/commits/89ce71e9b733df3855b370aa20bb809db3d4362c

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1103150&tab=buildLog

The test failed on provisional_201901161553_v2.1.4:
	test.go:1133: test timed out (1h0m0s)
	test.go:727,cluster.go:1203,tpcc.go:118,cluster.go:1541,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1103150-schemachange-indexrollback-tpcc-1000:5 -- ./workload run tpcc --warehouses=1000 --histograms=logs/stats.json --wait=false --tolerate-errors --ramp=5m0s --duration=30m0s {pgurl:1-4} returned:
		stderr:
		
		stdout:
		       0            9.0            6.5    125.8  27917.3  27917.3  27917.3 stockLevel
		   2m58s        0            2.0            6.5     67.1    104.9    104.9    104.9 delivery
		   2m58s        0            6.0           57.4    453.0    570.4    570.4    570.4 newOrder
		   2m58s        0            2.0            6.3     37.7    503.3    503.3    503.3 orderStatus
		   2m58s        0            0.0           43.1      0.0      0.0      0.0      0.0 payment
		   2m58s        0            1.0            6.5    113.2    113.2    113.2    113.2 stockLevel
		   2m59s        0            0.0            6.5      0.0      0.0      0.0      0.0 delivery
		   2m59s        0           11.0           57.2    285.2  25769.8  40802.2  40802.2 newOrder
		   2m59s        0            2.0            6.2     28.3     31.5     31.5     31.5 orderStatus
		   2m59s        0            1.0           42.8  15569.3  15569.3  15569.3  15569.3 payment
		   2m59s        0            4.0            6.4     96.5    109.1    109.1    109.1 stockLevel
		: signal: killed
	test.go:727,cluster.go:1562,tpcc.go:128,schemachange.go:375: Goexit() was called
	asm_amd64.s:523,panic.go:513,log.go:219,test.go:1097,asm_amd64.s:522,panic.go:397,test.go:728,cluster.go:1562,tpcc.go:128,schemachange.go:375: write /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190121-1103150/schemachange/indexrollback/tpcc-1000/test.log: file already closed

The text was updated successfully, but these errors were encountered:

tbg · 2019-01-22T08:39:45Z

cc @petermattis about this error (which I believe is fallout, but not sure):

asm_amd64.s:523,panic.go:513,log.go:219,test.go:1097,asm_amd64.s:522,panic.go:397,test.go:728,cluster.go:1562,tpcc.go:128,schemachange.go:375: write /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190121-1103150/schemachange/indexrollback/tpcc-1000/test.log: file already closed

cockroach-teamcity · 2019-01-22T15:24:35Z

SHA: https://github.com/cockroachdb/cockroach/commits/798304879367166c8954825f40c404ba100cea0a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1103521&tab=buildLog

The test failed on master:
	test.go:727,cluster.go:1203,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1103521-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190122 15:21:20.407890 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 4.783165767s
		Error: check failed: 3.3.2.1: 76 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-24T15:38:52Z

SHA: https://github.com/cockroachdb/cockroach/commits/295b6ae3142518f04a5771c79ce171043697ee1f

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1107301&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1195,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1107301-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190124 15:36:41.601162 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 4.803121851s
		Error: check failed: 3.3.2.1: 61 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-25T15:54:17Z

SHA: https://github.com/cockroachdb/cockroach/commits/2952f08ba7260967d7dfd10addbfe80b51d2b8ed

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1109027&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1195,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1109027-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190125 15:51:57.036670 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 5.209553736s
		Error: check failed: 3.3.2.1: 23 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-26T16:28:40Z

SHA: https://github.com/cockroachdb/cockroach/commits/dc2fbcdc0dccb8cc676fc67370375bab36b3cff0

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1110068&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1195,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1110068-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190126 16:26:18.042480 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 3.569119751s
		Error: check failed: 3.3.2.1: 19 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-27T15:30:37Z

SHA: https://github.com/cockroachdb/cockroach/commits/8cbeb534432b81c57564956ed7d645b854b426be

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1111300&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1195,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1111300-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190127 15:27:12.403640 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 3.823369254s
		Error: check failed: 3.3.2.1: 8 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-28T15:58:30Z

SHA: https://github.com/cockroachdb/cockroach/commits/9f084ad576e85756c5c5a7e41335d9aa2d3eee30

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1112101&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1195,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1112101-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190128 15:56:12.541036 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 3.597113456s
		Error: check failed: 3.3.2.1: 5 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-29T15:35:56Z

SHA: https://github.com/cockroachdb/cockroach/commits/e10fb557b11b5ff1b8609aa963da23c37a1143c8

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1113854&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1226,tpcc.go:130,schemachange.go:375: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1113854-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190129 15:33:20.792931 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 3.824426349s
		Error: check failed: 3.3.2.1: 11 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

cockroach-teamcity · 2019-01-30T14:43:23Z

SHA: https://github.com/cockroachdb/cockroach/commits/395d842feb97c5bd8cad2b32b71a5156c03061eb

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1115923&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1226,tpcc.go:118,cluster.go:1564,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1115923-schemachange-indexrollback-tpcc-1000:5 -- ./workload run tpcc --warehouses=1000 --histograms=logs/stats.json --wait=false --tolerate-errors --ramp=5m0s --duration=1h30m0s {pgurl:1-4} returned:
		stderr:
		
		stdout:
		4  81604.4 newOrder
		   2m34s    73093            3.0            8.1   1140.9   2013.3   2013.3   2013.3 orderStatus
		   2m34s    73093            2.0           64.4  47244.6  81604.4  81604.4  81604.4 payment
		   2m34s    73093            2.0            8.2   1946.2   2550.1   2550.1   2550.1 stockLevel
		E190130 14:38:50.529576 1 workload/cli/run.go:402  error in payment: dial tcp 10.128.0.103:26257: connect: connection refused
		   2m35s   128158            3.0            8.0   9663.7  11274.3  11274.3  11274.3 delivery
		   2m35s   128158            2.0           76.2  15032.4  36507.2  36507.2  36507.2 newOrder
		   2m35s   128158            3.0            8.1   1073.7   2147.5   2147.5   2147.5 orderStatus
		   2m35s   128158           17.1           64.1  11274.3  40802.2  53687.1  53687.1 payment
		   2m35s   128158            4.0            8.1   1879.0   4563.4   4563.4   4563.4 stockLevel
		E190130 14:38:51.529701 1 workload/cli/run.go:402  error in payment: dial tcp 10.128.0.103:26257: connect: connection refused
		: signal: killed
	test.go:743,cluster.go:1585,tpcc.go:128,schemachange.go:398: unexpected node event: 3: dead

andreimatei · 2019-01-30T16:26:04Z

Last failure (dead node) fixed by #34399

cockroach-teamcity · 2019-02-01T15:06:09Z

SHA: https://github.com/cockroachdb/cockroach/commits/fc3ea118c87ae1a9d2ed6f4974f2296766607666

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1119860&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1226,tpcc.go:118,cluster.go:1564,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1119860-schemachange-indexrollback-tpcc-1000:5 -- ./workload run tpcc --warehouses=1000 --histograms=logs/stats.json --wait=false --tolerate-errors --ramp=5m0s --duration=1h30m0s {pgurl:1-4} returned:
		stderr:
		
		stdout:
		126.8 delivery
		  33m30s      231           15.0           23.7   4831.8   9126.8  10737.4  10737.4 newOrder
		  33m30s      231            0.0            2.5      0.0      0.0      0.0      0.0 orderStatus
		  33m30s      231           16.0           22.5   4563.4   6442.5 103079.2 103079.2 payment
		  33m30s      231            2.0            2.5    503.3   1040.2   1040.2   1040.2 stockLevel
		E190201 15:03:35.596487 1 workload/cli/run.go:402  error in payment: ERROR: TransactionStatusError: transaction deadline exceeded (REASON_UNKNOWN) (SQLSTATE XX000)
		  33m31s      232            4.0            2.5   2415.9   5368.7   5368.7   5368.7 delivery
		  33m31s      232           32.9           23.7   6442.5 103079.2 103079.2 103079.2 newOrder
		  33m31s      232            2.0            2.5     58.7    402.7    402.7    402.7 orderStatus
		  33m31s      232           20.0           22.5   4831.8 103079.2 103079.2 103079.2 payment
		  33m31s      232            2.0            2.5    142.6  15569.3  15569.3  15569.3 stockLevel
		: signal: killed
	test.go:743,cluster.go:1585,tpcc.go:128,schemachange.go:401: expected 'job canceled' error, but got <nil>

cockroach-teamcity · 2019-02-02T15:15:25Z

SHA: https://github.com/cockroachdb/cockroach/commits/1b8689c0b4df102e1bf4e271913c4bb096ca8ffe

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=schemachange/indexrollback/tpcc-1000 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1121356&tab=buildLog

The test failed on master:
	test.go:743,cluster.go:1226,tpcc.go:130,schemachange.go:401: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1121356-schemachange-indexrollback-tpcc-1000:5 -- ./workload check tpcc --warehouses=1000 {pgurl:1} returned:
		stderr:
		
		stdout:
		I190202 15:12:51.152589 1 workload/tpcc/tpcc.go:288  check 3.3.2.1 took 3.980522393s
		Error: check failed: 3.3.2.1: 8 rows returned, expected zero
		Error:  exit status 1
		: exit status 1

Fixes cockroachdb#34025. Fixes cockroachdb#33624. Fixes cockroachdb#33335. Fixes cockroachdb#33151. Fixes cockroachdb#33149. Fixes cockroachdb#34159. Fixes cockroachdb#34293. Fixes cockroachdb#32813. Fixes cockroachdb#30886. Fixes cockroachdb#34228. Fixes cockroachdb#34321. It is rare but possible for a replica to become a leaseholder but not learn about this until it applies a snapshot. Immediately upon the snapshot application's `ReplicaState` update, the replica will begin operating as a standard leaseholder. Before this change, leases acquired in this way would not trigger in-memory side-effects to be performed. This could result in a regression in the new leaseholder's timestamp cache compared to the previous leaseholder, allowing write-skew like we saw in cockroachdb#34025. This could presumably result in other anomalies as well, because all of the steps in `leasePostApply` were skipped. This PR fixes this bug by detecting lease updates when applying snapshots and making sure to react correctly to them. It also likely fixes the referenced issue. The new test demonstrated that without this fix, the serializable violation speculated about in the issue was possible. Release note (bug fix): Fix bug where lease transfers passed through Snapshots could forget to update in-memory state on the new leaseholder, allowing write-skew between read-modify-write operations.

34548: storage: apply lease change side-effects on snapshot recipients r=nvanbenschoten a=nvanbenschoten Fixes #34025. Fixes #33624. Fixes #33335. Fixes #33151. Fixes #33149. Fixes #34159. Fixes #34293. Fixes #32813. Fixes #30886. Fixes #34228. Fixes #34321. It is rare but possible for a replica to become a leaseholder but not learn about this until it applies a snapshot. Immediately upon the snapshot application's `ReplicaState` update, the replica will begin operating as a standard leaseholder. Before this change, leases acquired in this way would not trigger in-memory side-effects to be performed. This could result in a regression in the new leaseholder's timestamp cache compared to the previous leaseholder's cache, allowing write-skew like we saw in #34025. This could presumably result in other anomalies as well, because all of the steps in `leasePostApply` were skipped (as theorized by #34025 (comment)). This PR fixes this bug by detecting lease updates when applying snapshots and making sure to react correctly to them. It also likely fixes the referenced issue. The new test demonstrates that without this fix, the serializable violation speculated about in the issue was possible. Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>

cockroach-teamcity added this to the 2.2 milestone Jan 21, 2019

cockroach-teamcity assigned andreimatei Jan 21, 2019

cockroach-teamcity added C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Jan 21, 2019

tbg assigned vivekmenezes and unassigned andreimatei Jan 22, 2019

jordanlewis added the O-roachtest label Feb 5, 2019

nvanbenschoten mentioned this issue Feb 5, 2019

storage: apply lease change side-effects on snapshot recipients #34548

Merged

craig bot closed this as completed in #34548 Feb 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: schemachange/indexrollback/tpcc-1000 failed #34159

roachtest: schemachange/indexrollback/tpcc-1000 failed #34159

cockroach-teamcity commented Jan 21, 2019

tbg commented Jan 22, 2019

cockroach-teamcity commented Jan 22, 2019

cockroach-teamcity commented Jan 24, 2019

cockroach-teamcity commented Jan 25, 2019

cockroach-teamcity commented Jan 26, 2019

cockroach-teamcity commented Jan 27, 2019

cockroach-teamcity commented Jan 28, 2019

cockroach-teamcity commented Jan 29, 2019

cockroach-teamcity commented Jan 30, 2019

andreimatei commented Jan 30, 2019

cockroach-teamcity commented Feb 1, 2019

cockroach-teamcity commented Feb 2, 2019

roachtest: schemachange/indexrollback/tpcc-1000 failed #34159

roachtest: schemachange/indexrollback/tpcc-1000 failed #34159

Comments

cockroach-teamcity commented Jan 21, 2019

tbg commented Jan 22, 2019

cockroach-teamcity commented Jan 22, 2019

cockroach-teamcity commented Jan 24, 2019

cockroach-teamcity commented Jan 25, 2019

cockroach-teamcity commented Jan 26, 2019

cockroach-teamcity commented Jan 27, 2019

cockroach-teamcity commented Jan 28, 2019

cockroach-teamcity commented Jan 29, 2019

cockroach-teamcity commented Jan 30, 2019

andreimatei commented Jan 30, 2019

cockroach-teamcity commented Feb 1, 2019

cockroach-teamcity commented Feb 2, 2019