Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: panic in TestRandomResignLeader #7280

Closed
lhy1024 opened this issue Oct 27, 2023 · 1 comment · Fixed by #7472
Closed

tests: panic in TestRandomResignLeader #7280

lhy1024 opened this issue Oct 27, 2023 · 1 comment · Fixed by #7472
Assignees
Labels
type/ci The issue is related to CI.

Comments

@lhy1024
Copy link
Contributor

lhy1024 commented Oct 27, 2023

Flaky Test

Which jobs are failing

2023-10-26T16:54:16.7734625Z === RUN   TestMicroserviceTSOClient/TestRandomResignLeader
2023-10-26T16:54:20.7752051Z [2023/10/26 16:54:20.774 +00:00] [INFO] [lease.go:187] ["stop lease keep alive worker"] [purpose="keyspace group primary election"]
2023-10-26T16:54:20.7766489Z [2023/10/26 16:54:20.776 +00:00] [INFO] [lease.go:187] ["stop lease keep alive worker"] [purpose="keyspace group primary election"]
2023-10-26T16:54:20.7772898Z [2023/10/26 16:54:20.777 +00:00] [INFO] [lease.go:187] ["stop lease keep alive worker"] [purpose="keyspace group primary election"]
2023-10-26T16:54:20.7791238Z [2023/10/26 16:54:20.778 +00:00] [ERROR] [tso_dispatcher.go:493] ["[tso] getTS error"] [dc-location=global] [stream-addr=http://127.0.0.1:42227] [error="[PD:client:ErrClientGetTSO]get TSO failed, after processing requests"]
2023-10-26T16:54:20.7848658Z [2023/10/26 16:54:20.784 +00:00] [ERROR] [tso_dispatcher.go:493] ["[tso] getTS error"] [dc-location=global] [stream-addr=http://127.0.0.1:42227] [error="[PD:client:ErrClientGetTSO]get TSO failed, after processing requests"]
2023-10-26T16:54:20.7855432Z [2023/10/26 16:54:20.785 +00:00] [ERROR] [tso_dispatcher.go:493] ["[tso] getTS error"] [dc-location=global] [stream-addr=http://127.0.0.1:42227] [error="[PD:client:ErrClientGetTSO]get TSO failed, after processing requests"]
2023-10-26T16:54:20.7902963Z [2023/10/26 16:54:20.789 +00:00] [INFO] [leadership.go:320] ["current leadership is deleted"] [revision=140] [leader-key=/ms/7294311158318438319/tso/keyspace_groups/election/00001/primary] [purpose="keyspace group primary election"]
2023-10-26T16:54:20.7912563Z [2023/10/26 16:54:20.790 +00:00] [INFO] [leadership.go:320] ["current leadership is deleted"] [revision=140] [leader-key=/ms/7294311158318438319/tso/keyspace_groups/election/00001/primary] [purpose="keyspace group primary election"]
2023-10-26T16:54:20.7920644Z [2023/10/26 16:54:20.789 +00:00] [ERROR] [tso_dispatcher.go:493] ["[tso] getTS error"] [dc-location=global] [stream-addr=http://127.0.0.1:42227] [error="[PD:client:ErrClientGetTSO]get TSO failed, after processing requests"]
2023-10-26T16:54:20.7933581Z [2023/10/26 16:54:20.792 +00:00] [ERROR] [tso_dispatcher.go:493] ["[tso] getTS error"] [dc-location=global] [stream-addr=http://127.0.0.1:42835] [error="[PD:client:ErrClientGetTSO]get TSO failed, after processing requests"]
2023-10-26T16:54:20.7951348Z [2023/10/26 16:54:20.794 +00:00] [INFO] [global_allocator.go:645] ["no longer a primary because lease has expired, the tso primary will step down"] []
2023-10-26T16:54:20.7990599Z panic: runtime error: invalid memory address or nil pointer dereference
2023-10-26T16:54:20.7993916Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x2eaf654]
2023-10-26T16:54:20.7994884Z 
2023-10-26T16:54:20.7995534Z goroutine 561154 [running]:
2023-10-26T16:54:20.7996421Z github.com/tikv/pd/pkg/member.(*Participant).GetLeaderListenUrls(0xc009a53320?)
2023-10-26T16:54:20.7997532Z 	/home/runner/work/pd/pd/pkg/member/participant.go:140 +0x94
2023-10-26T16:54:20.7998544Z github.com/tikv/pd/pkg/tso.(*AllocatorManager).GetLeaderAddr(0xc006130000)
2023-10-26T16:54:20.7999586Z 	/home/runner/work/pd/pd/pkg/tso/allocator_manager.go:1415 +0xe2
2023-10-26T16:54:20.8001350Z github.com/tikv/pd/pkg/mcs/tso/server.(*Service).FindGroupByKeyspaceID(0xc0014c8b68, {0xc00e7ddd10?, 0x1041f45?}, 0xc00e7ddd10)
2023-10-26T16:54:20.8003159Z 	/home/runner/work/pd/pd/pkg/mcs/tso/server/grpc_service.go:201 +0xd16
2023-10-26T16:54:20.8004693Z github.com/pingcap/kvproto/pkg/tsopb._TSO_FindGroupByKeyspaceID_Handler({0x436da60?, 0xc0014c8b68}, {0x5146170, 0xc00e7ddce0}, 0xc0167831a0, 0x0)
2023-10-26T16:54:20.8007689Z 	/home/runner/go/pkg/mod/github.com/pingcap/kvproto@v0.0.0-20231018065736-c0689aded40c/pkg/tsopb/tsopb.pb.go:1097 +0x22e
2023-10-26T16:54:20.8009574Z google.golang.org/grpc.(*Server).processUnaryRPC(0xc0052e9980, {0x5154d00, 0xc00c7c5980}, 0xc009f0ae00, 0xc0088e6bd0, 0x6530540, 0x0)
2023-10-26T16:54:20.8011456Z 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1024 +0x1495
2023-10-26T16:54:20.8012734Z google.golang.org/grpc.(*Server).handleStream(0xc0052e9980, {0x5154d00, 0xc00c7c5980}, 0xc009f0ae00, 0x0)
2023-10-26T16:54:20.8014520Z 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1313 +0xf73
2023-10-26T16:54:20.8015525Z google.golang.org/grpc.(*Server).serveStreams.func1.1()
2023-10-26T16:54:20.8016507Z 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xe7
2023-10-26T16:54:20.8017613Z created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 80923
2023-10-26T16:54:20.8018777Z 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0x186
2023-10-26T16:54:20.8609486Z FAIL	github.com/tikv/pd/tests/integrations/tso	178.303s
2023-10-26T16:54:20.8769148Z FAIL

CI link

https://github.com/tikv/pd/actions/runs/6657235162/job/18091509886

Reason for failure (if possible)

Anything else

@lhy1024 lhy1024 added the type/ci The issue is related to CI. label Oct 27, 2023
@lhy1024
Copy link
Contributor Author

lhy1024 commented Oct 27, 2023

func (m *Participant) GetLeaderListenUrls() []string {
if m.GetLeader() == nil {
return nil
}
return m.GetLeader().GetListenUrls()
}

@lhy1024 lhy1024 changed the title tests: panic in integrations/tso tests: panic in TestRandomResignLeader Oct 27, 2023
@lhy1024 lhy1024 self-assigned this Nov 24, 2023
@ti-chi-bot ti-chi-bot bot closed this as completed in #7472 Dec 7, 2023
ti-chi-bot bot pushed a commit that referenced this issue Dec 7, 2023
close #7280

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/ci The issue is related to CI.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant