Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[exporter/clickhouse] Integration test hits a panic #32530

Open
crobert-1 opened this issue Apr 18, 2024 · 7 comments
Open

[exporter/clickhouse] Integration test hits a panic #32530

crobert-1 opened this issue Apr 18, 2024 · 7 comments
Labels
exporter/clickhouse needs triage New item requiring triage

Comments

@crobert-1
Copy link
Member

Component(s)

exporter/clickhouse

Describe the issue you're reporting

Failing CI/CD link

For context, some of these tests haven't been run for a while (if ever), so this may not be a recent break. See #32207.

Failure output:

Running target 'mod-integration-test' in module 'exporter/clickhouseexporter' as part of group 'exporter-0'
make --no-print-directory -C exporter/clickhouseexporter mod-integration-test
go: downloading github.com/klauspost/compress v1.17.7
running go integration test ./... in /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter
/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --packages="./..." -- -race -timeout 360s -parallel 4 -tags=integration,""
go: downloading github.com/jmoiron/sqlx v1.3.5
go: downloading github.com/testcontainers/testcontainers-go v0.30.0
go: downloading dario.cat/mergo v1.0.0
go: downloading github.com/containerd/containerd v1.7.12
go: downloading github.com/cpuguy83/dockercfg v0.3.1
go: downloading github.com/moby/patternmatcher v0.6.0
go: downloading github.com/moby/term v0.5.0
go: downloading golang.org/x/exp v0.0.0-20230711023510-fffb14384f22
go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0
go: downloading github.com/containerd/log v0.1.0
go: downloading github.com/moby/sys/sequential v0.5.0
go: downloading github.com/morikuni/aec v1.0.0
go: downloading github.com/moby/sys/user v0.1.0
∅  internal/metadata
✓  internal (1.02s)
✖  . (13.602s)

=== Failed
=== FAIL: . TestIntegration/test_clickhouse_24-alpine (13.53s)
2024/04/18 18:48:46 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 24.0.9
  API Version: 1.43
  Operating System: Ubuntu 22.04.4 LTS
  Total Memory: 15981 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: a51b8c75e41a407120f69d0b2cfc5ad0ce071bcc50ee005aac596474ef9bc907
  Test ProcessID: aa1bae38-c496-4218-9763-4306ba02d328
2024/04/18 18:48:48 🐳 Creating container for image testcontainers/ryuk:0.7.0
2024/04/18 18:48:48 ✅ Container created: 8f6246042821
2024/04/18 18:48:48 🐳 Starting container: 8f6246042821
2024/04/18 18:48:48 ✅ Container started: 8f6246042821
2024/04/18 18:48:48 🚧 Waiting for container id 8f6246042821 image: testcontainers/ryuk:0.7.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
2024/04/18 18:48:48 🔔 Container is ready: 8f6246042821
2024/04/18 18:48:58 🐳 Creating container for image clickhouse/clickhouse-server:24-alpine
2024/04/18 18:48:58 ✅ Container created: 5b2768928837
2024/04/18 18:48:58 🐳 Starting container: 5b2768928837
2024/04/18 18:48:58 ✅ Container started: 5b2768928837
2024/04/18 18:48:58 🚧 Waiting for container id 5b2768928837 image: clickhouse/clickhouse-server:24-alpine. Waiting for: &{Port:9000 timeout:0xc0004dc398 PollInterval:100ms}
2024/04/18 18:48:59 🔔 Container is ready: 5b2768928837
2024/04/18 18:48:59 🐳 Starting container: 5b2768928837
2024/04/18 18:48:59 ✅ Container started: 5b2768928837
2024/04/18 18:48:59 🚧 Waiting for container id 5b2768928837 image: clickhouse/clickhouse-server:24-alpine. Waiting for: &{Port:9000 timeout:0xc0004dc398 PollInterval:100ms}
2024/04/18 18:48:59 🔔 Container is ready: 5b2768928837
2024/04/18 18:48:59 🐳 Terminating container: 5b2768928837
2024/04/18 18:48:59 🚫 Container terminated: 5b2768928837

=== FAIL: . TestIntegration (13.53s)
panic: Fail in goroutine after TestTracesTableEngineConfig has completed [recovered]
	panic: Fail in goroutine after TestTracesTableEngineConfig has completed

goroutine 141 [running]:
testing.tRunner.func1.2({0x1626740, 0xc00011fe00})
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:1545 +0x3f7
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:1548 +0x716
panic({0x1626740?, 0xc00011fe00?})
	/opt/hostedtoolcache/go/1.21.9/x64/src/runtime/panic.go:920 +0x270
testing.(*common).Fail(0xc0003efa00)
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:952 +0x1b4
testing.(*common).Fail(0xc00049c680)
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:946 +0x85
testing.(*common).Errorf(0xc00049c680, {0x17fd5ea, 0x3}, {0xc00011fdb0, 0x1, 0x1})
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:1069 +0xaf
github.com/stretchr/testify/assert.Fail({0x1a894c0, 0xc00049c680}, {0x181da90, 0x21}, {0x0, 0x0, 0x0})
	/home/runner/go/pkg/mod/github.com/stretchr/testify@v1.9.0/assert/assertions.go:363 +0x436
github.com/stretchr/testify/assert.Error({0x1a894c0, 0xc00049c680}, {0x0, 0x0?}, {0x0, 0x0, 0x0})
	/home/runner/go/pkg/mod/github.com/stretchr/testify@v1.9.0/assert/assertions.go:1562 +0xa5
github.com/stretchr/testify/require.Error({0x1a8e260, 0xc00049c680}, {0x0, 0x0}, {0x0, 0x0, 0x0})
	/home/runner/go/pkg/mod/github.com/stretchr/testify@v1.9.0/require/require.go:290 +0x97
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.testTableEngineConfig.func1.1({0xc0004a8700, 0x68c}, {0xc0004e4bf8?, 0xc0004e4bf8?, 0x441c5e?})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/exporter_sql_test.go:170 +0x11d
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.(*testClickhouseDriverStmt).Exec(0xc0000d34a0, {0x2382d40, 0x0, 0x0})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/exporter_logs_test.go:245 +0x89
database/sql.ctxDriverStmtExec({0x1a95198, 0x2382d40}, {0x1a950f0, 0xc0000d34a0}, {0x2382d40?, 0x0, 0x0})
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/ctxutil.go:77 +0x34f
database/sql.resultFromStatement({0x1a95198, 0x2382d40}, {0x1a91558, 0xc0001202d0}, 0xc0004e4fa8, {0x0, 0x0, 0x0})
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:2642 +0x1e9
database/sql.(*DB).execDC(0x7f2562b82401?, {0x1a95198?, 0x2382d40}, 0xc00040e630, 0xc0004e5148, {0xc0004a8700, 0x68c}, {0x0, 0x0, 0x0})
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:1694 +0x6cd
database/sql.(*DB).exec(0x4adc39?, {0x1a95198, 0x2382d40}, {0xc0004a8700, 0x68c}, {0x0, 0x0, 0x0}, 0x0?)
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:1655 +0x15b
database/sql.(*DB).ExecContext.func1(0x0?)
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:1634 +0xd0
database/sql.(*DB).retry(0x4?, 0xc0004e52e8)
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:[153](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8743113381/job/23992945839?pr=32529#step:5:154)8 +0x4b
database/sql.(*DB).ExecContext(0xc0002f3e10, {0x1a95198?, 0x2382d40}, {0xc0004a8700, 0x68c}, {0x0, 0x0, 0x0})
	/opt/hostedtoolcache/go/1.21.9/x64/src/database/sql/sql.go:1633 +0x194
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.createLogsTable({0x1a95198, 0x2382d40}, 0xc0004baea0?, 0x1a894c0?)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/exporter_logs.go:230 +0x71
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.(*logsExporter).start(0xc000261b60, {0x1a95198, 0x2382d40}, {0x0?, 0x0?})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/exporter_logs.go:49 +0xc5
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.newTestLogsExporter(0xc000500680, {0xc000589608, 0x14}, {0x0, 0x0, 0x0})
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/exporter_logs_test.go:141 +0x12e
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhouseexporter.TestIntegration.func1(0xc000500680)
	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/clickhouseexporter/integration_test.go:62 +0x50f
ERROR rerun aborted because previous run had a suspected panic and some test may not have run
make[2]: *** [../../Makefile.Common:142: mod-integration-test] Error 3
make[1]: *** [Makefile:165: exporter/clickhouseexporter] Error 2
make: *** [Makefile:122: gointegration-test] Error 2
testing.tRunner(0xc000500680, 0xc0004d6f10)
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:[159](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8743113381/job/23992945839?pr=32529#step:5:160)5 +0x262
created by testing.(*T).Run in goroutine 140
	/opt/hostedtoolcache/go/1.21.9/x64/src/testing/testing.go:1648 +0x846

=== FAIL: . TestLoadConfig (unknown)

=== FAIL: . TestTableEngineConfigParsing (unknown)

=== FAIL: . TestClusterString (unknown)

=== FAIL: . TestExporter_pushMetricsData (unknown)

DONE 82 tests, 6 failures in 28.348s
make[1]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib'
@crobert-1 crobert-1 added the needs triage New item requiring triage label Apr 18, 2024
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot mentioned this issue May 28, 2024
Closed
This was referenced Jun 2, 2024
Closed
Closed
Closed
Closed
Closed
Closed
This was referenced Jun 10, 2024
Closed
Closed
Closed
Closed
@github-actions GitHub Actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Jun 18, 2024
@SpencerTorres
Copy link
Member

Not sure if we have any ideas to improve reliability for these tests, definitely want to keep this issue open though

@SpencerTorres
Copy link
Member

I would like to group this issue with #32275 since they both reference the integration tests, we need to revisit these and improve stability. I have it working locally, but the CI seems unreliable.

@github-actions github-actions bot removed the Stale label Jun 18, 2024
@crobert-1
Copy link
Member Author

I would like to group this issue

As far as grouping together, it's nice to have open issues for different test failures to track frequencies independently. It's totally fine to open a single PR that fixes both issues though for something like this, if that works for you.

This was referenced Jun 19, 2024
Closed
Closed
Closed
Closed
Closed
Closed
@SpencerTorres SpencerTorres mentioned this issue Jun 25, 2024
Open
@github-actions github-actions bot mentioned this issue Jul 2, 2024
Open
@github-actions github-actions bot mentioned this issue Jul 9, 2024
Open
@github-actions github-actions bot mentioned this issue Jul 16, 2024
Open
@github-actions github-actions bot mentioned this issue Jul 23, 2024
Open
This was referenced Jul 30, 2024
Open
Open
@github-actions github-actions bot mentioned this issue Aug 13, 2024
Open
@github-actions GitHub Actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Aug 19, 2024
@github-actions github-actions bot mentioned this issue Aug 20, 2024
Open
@SpencerTorres
Copy link
Member

Still need to investigate integration test stability further. For now these are disabled, and when I manually run them they work fine. Let me know if anyone else here has any ideas

@github-actions github-actions bot removed the Stale label Aug 21, 2024
This was referenced Aug 27, 2024
Open
Open
@github-actions github-actions bot mentioned this issue Sep 10, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
exporter/clickhouse needs triage New item requiring triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@SpencerTorres @crobert-1