Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when replace pod by label encountering recreate update #212

Closed
ColdsteelRail opened this issue May 29, 2024 · 0 comments
Closed

Panic when replace pod by label encountering recreate update #212

ColdsteelRail opened this issue May 29, 2024 · 0 comments
Assignees
Labels
kind/bug Something isn't working

Comments

@ColdsteelRail
Copy link
Collaborator

ColdsteelRail commented May 29, 2024

Minimal reproduce step

  1. prepare a collaset with 1 replicas
  2. replace the pod using podopslifecycle.kusionstack.io/to-replace label
  3. at the same time, update collaset with recreate update

collaset pod

replace-recreate-bug2

controller pod

rplace-recreate-update-bug

panic log

anic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xfe1e3c]

goroutine 286 [running]:
kusionstack.io/operating/apis/apps/v1alpha1.(*ContextDetail).Contains(...)
	/workspace/apis/apps/v1alpha1/resourcecontext_types.go:61
kusionstack.io/operating/pkg/controllers/collaset/synccontrol.(*GenericPodUpdater).FilterAllowOpsPods(0x4000d2d478, {0x40002ce588, 0x1, 0x40007d8240?}, 0x4000725b60, 0x4000ae7da0, 0x400000c918?)
	/workspace/pkg/controllers/collaset/synccontrol/update.go:336 +0x23c
kusionstack.io/operating/pkg/controllers/collaset/synccontrol.(*RealSyncControl).Update(0x40003bf270, {0x16b4638, 0x4000573470}, 0x40007e4580?, 0x4000ae7da0, {0x40002ce4c0, 0x1, 0x1}, 0x4000725b60?)
	/workspace/pkg/controllers/collaset/synccontrol/sync_control.go:780 +0x488
kusionstack.io/operating/pkg/controllers/collaset.(*CollaSetReconciler).doSync(0x40003fef40, {0x16b4638, 0x4000573470}, 0x4000ae7da0?, 0x0?)
	/workspace/pkg/controllers/collaset/collaset_controller.go:236 +0x118
kusionstack.io/operating/pkg/controllers/collaset.(*CollaSetReconciler).DoReconcile(0x16bae30?, {0x16b4638?, 0x4000573470?}, 0x7?, 0x4?)
	/workspace/pkg/controllers/collaset/collaset_controller.go:212 +0x2c
kusionstack.io/operating/pkg/controllers/collaset.(*CollaSetReconciler).Reconcile(0x40003fef40, {0x16b4638, 0x4000573470}, {{{0x4000dd5dd0?, 0x139fa40?}, {0x4000dd5dc0?, 0x30?}}})
	/workspace/pkg/controllers/collaset/collaset_controller.go:198 +0x6fc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x40000bc0a0, {0x16b4638, 0x4000573440}, {{{0x4000dd5dd0?, 0x139fa40?}, {0x4000dd5dc0?, 0x0?}}})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/internal/controller/controller.go:114 +0x1f4
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0x40000bc0a0, {0x16b4590, 0x40003c1dc0}, {0x12746e0?, 0x40002c8f60?})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/internal/controller/controller.go:311 +0x25c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0x40000bc0a0, {0x16b4590, 0x40003c1dc0})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/internal/controller/controller.go:266 +0x1b0
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/internal/controller/controller.go:227 +0x74
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/internal/controller/controller.go:223 +0x280

What did you expect to see?

  1. replace pod should work well
  2. after pod is replaced, do the recreate update

What did you see instead

  1. replace pod do not work well: origin pod is deleted before new replace pod service available
  2. controller panic

What is your KusionStack components and its version?

0.4.0

@ColdsteelRail ColdsteelRail added the kind/bug Something isn't working label May 29, 2024
@ColdsteelRail ColdsteelRail assigned wu8685 and ColdsteelRail and unassigned wu8685 May 29, 2024
@ColdsteelRail ColdsteelRail changed the title Bug: panic when replace pod by label encountering recreate update Panic when replace pod by label encountering recreate update May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants