Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix scheme data race #375

Open
kdorosh opened this issue Sep 21, 2022 · 7 comments · Fixed by solo-io/gloo#7207
Open

fix scheme data race #375

kdorosh opened this issue Sep 21, 2022 · 7 comments · Fixed by solo-io/gloo#7207

Comments

@kdorosh
Copy link
Contributor

kdorosh commented Sep 21, 2022

e.g.

WARNING: DATA RACE
Write at 0x00c0004c1c80 by goroutine 40:
  runtime.mapassign()
      /opt/hostedtoolcache/go/1.18.2/x64/src/runtime/map.go:578 +0x0
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme.go:196 +0x3a4
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme.go:170 +0x2e8
  sigs.k8s.io/controller-runtime/pkg/scheme.(*Builder).Register.func1()
      /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/scheme/scheme.go:72 +0x8b
  k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme_builder.go:29 +0xa1
  sigs.k8s.io/controller-runtime/pkg/scheme.(*Builder).AddToScheme()
      /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/scheme/scheme.go:87 +0x2c
  github.com/solo-io/solo-apis/pkg/api/ratelimit.solo.io/v1alpha1.AddToScheme()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-apis@v0.0.0-20210922150112-505473b2e66c/pkg/api/ratelimit.solo.io/v1alpha1/register.go:32 +0x5b
  github.com/solo-io/solo-apis/pkg/api/ratelimit.solo.io/v1alpha1.NewClientsetFromConfig()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-apis@v0.0.0-20210922150112-505473b2e66c/pkg/api/ratelimit.solo.io/v1alpha1/clients.go:52 +0x60
  github.com/solo-io/gloo/projects/gloo/pkg/api/external/solo/ratelimit.NewRateLimitClients()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/api/external/solo/ratelimit/extensions.go:54 +0xf9
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.RunGlooWithExtensions()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:504 +0xb5a
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.RunGloo()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:417 +0x24f
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.(*setupSyncer).Setup()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:391 +0x21f4
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.(*setupSyncer).Setup-fm()
      <autogenerated>:1 +0xdc
  github.com/solo-io/gloo/test/kube2e/gloo_test.glob..func4.1.1()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/setup_syncer_test.go:56 +0x16d
  github.com/solo-io/gloo/test/kube2e/gloo_test.glob..func4.2.3()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/setup_syncer_test.go:96 +0x2b0
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113 +0xf5
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64 +0x157
  github.com/onsi/ginkgo/internal/leafnodes.(*ItNode).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/it_node.go:26 +0x94
  github.com/onsi/ginkgo/internal/spec.(*Spec).runSample()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/spec/spec.go:215 +0x359
  github.com/onsi/ginkgo/internal/spec.(*Spec).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/spec/spec.go:138 +0x1ab
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:200 +0x15a
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:170 +0x255
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:66 +0x136
  github.com/onsi/ginkgo/internal/suite.(*Suite).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79 +0x6f5
  github.com/onsi/ginkgo.runSpecsWithCustomReporters()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245 +0x212
  github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:228 +0x257
  github.com/solo-io/gloo/test/kube2e/gloo_test.TestGloo()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/gloo_suite_test.go:38 +0x41d
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1486 +0x47

Previous read at 0x00c0004c1c80 by goroutine 989:
  runtime.mapaccess2()
      /opt/hostedtoolcache/go/1.18.2/x64/src/runtime/map.go:456 +0x0
  k8s.io/apimachinery/pkg/runtime.(*Scheme).New()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme.go:318 +0xc8
  k8s.io/apimachinery/pkg/runtime.UseOrCreateObject()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/codec.go:97 +0x149
  k8s.io/apimachinery/pkg/runtime/serializer/json.(*Serializer).Decode()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/serializer/json/json.go:259 +0xb2a
  k8s.io/apimachinery/pkg/runtime.WithoutVersionDecoder.Decode()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/helper.go:252 +0xb9
  k8s.io/apimachinery/pkg/runtime.(*WithoutVersionDecoder).Decode()
      <autogenerated>:1 +0xb5
  k8s.io/apimachinery/pkg/runtime.Decode()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/codec.go:58 +0x1f4
  k8s.io/client-go/rest/watch.(*Decoder).Decode()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/watch/decoder.go:62 +0x18c
  k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:105 +0x19a
  k8s.io/apimachinery/pkg/watch.NewStreamWatcher.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:76 +0x39

Goroutine 40 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:47 +0x2e4

Goroutine 989 (running) created at:
  k8s.io/apimachinery/pkg/watch.NewStreamWatcher()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:76 +0x216
  k8s.io/client-go/rest.(*Request).newStreamWatcher()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/request.go:775 +0x6e4
  k8s.io/client-go/rest.(*Request).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/request.go:720 +0x609
  k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/kubernetes/typed/core/v1/configmap.go:112 +0x2b8
  k8s.io/client-go/kubernetes/typed/core/v1.ConfigMapInterface.Watch-fm()
      <autogenerated>:1 +0xcc
  github.com/solo-io/solo-kit/pkg/api/v1/clients/kube.NewSharedInformer.func2()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-kit@v0.30.3/pkg/api/v1/clients/kube/resource_client_factory.go:157 +0x3e5
  k8s.io/client-go/tools/cache.(*ListWatch).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/listwatch.go:111 +0xd4
  k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:414 +0x7f4
  k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:221 +0x44
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:155 +0x48
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:156 +0xce
  k8s.io/client-go/tools/cache.(*Reflector).Run()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:220 +0x2de
  k8s.io/client-go/tools/cache.(*Reflector).Run-fm()
      <autogenerated>:1 +0x44
  k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:56 +0x3e
  k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:73 +0x73
==================
==================
WARNING: DATA RACE
Write at 0x00c0004c1ce0 by goroutine 40:
  runtime.mapassign()
      /opt/hostedtoolcache/go/1.18.2/x64/src/runtime/map.go:578 +0x0
  k8s.io/apimachinery/pkg/runtime.(*Scheme).AddUnversionedTypes()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme.go:150 +0x484
  k8s.io/apimachinery/pkg/apis/meta/v1.AddToGroupVersion()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/apis/meta/v1/register.go:75 +0x60a
  sigs.k8s.io/controller-runtime/pkg/scheme.(*Builder).Register.func1()
      /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/scheme/scheme.go:73 +0xbb
  k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme_builder.go:29 +0xa1
  sigs.k8s.io/controller-runtime/pkg/scheme.(*Builder).AddToScheme()
      /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/scheme/scheme.go:87 +0x2c
  github.com/solo-io/solo-apis/pkg/api/ratelimit.solo.io/v1alpha1.AddToScheme()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-apis@v0.0.0-20210922150112-505473b2e66c/pkg/api/ratelimit.solo.io/v1alpha1/register.go:32 +0x5b
  github.com/solo-io/solo-apis/pkg/api/ratelimit.solo.io/v1alpha1.NewClientsetFromConfig()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-apis@v0.0.0-20210922150112-505473b2e66c/pkg/api/ratelimit.solo.io/v1alpha1/clients.go:52 +0x60
  github.com/solo-io/gloo/projects/gloo/pkg/api/external/solo/ratelimit.NewRateLimitClients()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/api/external/solo/ratelimit/extensions.go:54 +0xf9
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.RunGlooWithExtensions()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:504 +0xb5a
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.RunGloo()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:417 +0x24f
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.(*setupSyncer).Setup()
      /home/runner/work/gloo/gloo/projects/gloo/pkg/syncer/setup/setup_syncer.go:391 +0x21f4
  github.com/solo-io/gloo/projects/gloo/pkg/syncer/setup.(*setupSyncer).Setup-fm()
      <autogenerated>:1 +0xdc
  github.com/solo-io/gloo/test/kube2e/gloo_test.glob..func4.1.1()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/setup_syncer_test.go:56 +0x16d
  github.com/solo-io/gloo/test/kube2e/gloo_test.glob..func4.2.3()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/setup_syncer_test.go:96 +0x2b0
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113 +0xf5
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64 +0x157
  github.com/onsi/ginkgo/internal/leafnodes.(*ItNode).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/it_node.go:26 +0x94
  github.com/onsi/ginkgo/internal/spec.(*Spec).runSample()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/spec/spec.go:215 +0x359
  github.com/onsi/ginkgo/internal/spec.(*Spec).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/spec/spec.go:138 +0x1ab
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:200 +0x15a
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:170 +0x255
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:66 +0x136
  github.com/onsi/ginkgo/internal/suite.(*Suite).Run()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79 +0x6f5
  github.com/onsi/ginkgo.runSpecsWithCustomReporters()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245 +0x212
  github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters()
      /home/runner/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:228 +0x257
  github.com/solo-io/gloo/test/kube2e/gloo_test.TestGloo()
      /home/runner/work/gloo/gloo/test/kube2e/gloo/gloo_suite_test.go:38 +0x41d
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1486 +0x47

Previous read at 0x00c0004c1ce0 by goroutine 783:
  runtime.mapaccess2()
      /opt/hostedtoolcache/go/1.18.2/x64/src/runtime/map.go:456 +0x0
  k8s.io/apimachinery/pkg/runtime.(*Scheme).ObjectKinds()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/scheme.go:289 +0x2f6
  k8s.io/apimachinery/pkg/runtime/serializer/json.(*Serializer).Decode()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/serializer/json/json.go:237 +0x54e
  k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/runtime/serializer/streaming/streaming.go:107 +0x569
  k8s.io/client-go/rest/watch.(*Decoder).Decode()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/watch/decoder.go:49 +0xbb
  k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:105 +0x19a
  k8s.io/apimachinery/pkg/watch.NewStreamWatcher.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:76 +0x39

Goroutine 40 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /opt/hostedtoolcache/go/1.18.2/x64/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:47 +0x2e4

Goroutine 783 (running) created at:
  k8s.io/apimachinery/pkg/watch.NewStreamWatcher()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/watch/streamwatcher.go:76 +0x216
  k8s.io/client-go/rest.(*Request).newStreamWatcher()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/request.go:775 +0x6e4
  k8s.io/client-go/rest.(*Request).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/rest/request.go:720 +0x609
  k8s.io/client-go/kubernetes/typed/core/v1.(*configMaps).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/kubernetes/typed/core/v1/configmap.go:112 +0x2b8
  k8s.io/client-go/kubernetes/typed/core/v1.ConfigMapInterface.Watch-fm()
      <autogenerated>:1 +0xcc
  github.com/solo-io/solo-kit/pkg/api/v1/clients/kube.NewSharedInformer.func2()
      /home/runner/go/pkg/mod/github.com/solo-io/solo-kit@v0.30.3/pkg/api/v1/clients/kube/resource_client_factory.go:157 +0x3e5
  k8s.io/client-go/tools/cache.(*ListWatch).Watch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/listwatch.go:111 +0xd4
  k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:414 +0x7f4
  k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:221 +0x44
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:155 +0x48
  k8s.io/apimachinery/pkg/util/wait.BackoffUntil()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:156 +0xce
  k8s.io/client-go/tools/cache.(*Reflector).Run()
      /home/runner/go/pkg/mod/k8s.io/client-go@v0.22.4/tools/cache/reflector.go:220 +0x2de
  k8s.io/client-go/tools/cache.(*Reflector).Run-fm()
      <autogenerated>:1 +0x44
  k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:56 +0x3e
  k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
      /home/runner/go/pkg/mod/k8s.io/apimachinery@v0.22.4/pkg/util/wait/wait.go:73 +0x73
@kdorosh kdorosh self-assigned this Sep 21, 2022
@kdorosh
Copy link
Contributor Author

kdorosh commented Sep 21, 2022

race comes from

skv2/codegen/templates/code/types/clients.gotmpl

Line 57 in 0e3000f

scheme := scheme.Scheme

@solo-changelog-bot solo-changelog-bot bot mentioned this issue Sep 21, 2022
Closed
@sam-heilbron
Copy link
Contributor

I previously thought I had fixed this with solo-io/gloo#6282, and then some of the HA work may have changed the tests to bring it back in.

@kdorosh
Copy link
Contributor Author

kdorosh commented Sep 22, 2022

@sam-heilbron this was actually seen in the live race image deployed in internal testing environment; not just a testing bug. It's rare but separate; the proposed fix is to add to the global scheme during init() before the k8s watcher has a chance to read using the global scheme. you can see the proposed fix as part of solo-io/gloo#7207

@solo-changelog-bot solo-changelog-bot bot mentioned this issue Sep 23, 2022
Merged
8 tasks
@solo-changelog-bot solo-changelog-bot bot mentioned this issue Sep 23, 2022
Merged
8 tasks
@kdorosh
Copy link
Contributor Author

kdorosh commented Sep 23, 2022

fix for gloo was merged. reopening until skv2 is updated with the function removed, so people don't use this possibly racy function by accident

@kdorosh kdorosh reopened this Sep 23, 2022
@kdorosh kdorosh removed their assignment Sep 27, 2022
@kdorosh
Copy link
Contributor Author

kdorosh commented Sep 29, 2022

reopening to ensure we fix and gloo mesh does not get hit by this

@kdorosh kdorosh reopened this Sep 29, 2022
@kdorosh
Copy link
Contributor Author

kdorosh commented Sep 29, 2022

we should not close this ticket until the code has been removed from skv2 so no one else introduces this by accident

@marcogschmidt marcogschmidt mentioned this issue Jul 7, 2023
Open
@fernandosanchezs
Copy link

Seen again recently: solo-io/external-apis#34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@sam-heilbron @chrisgaun @kdorosh @fernandosanchezs