Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lightning might report "verify allocator base failed. local..." using parallel import and they have same task id #55384

Closed
D3Hunter opened this issue Aug 13, 2024 · 3 comments · Fixed by #55505
Closed

lightning might report "verify allocator base failed. local..." using parallel import and they have same task id #55384

D3Hunter opened this issue Aug 13, 2024 · 3 comments · Fixed by #55505
Labels
affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. severity/moderate type/bug The issue is confirmed as a bug.

Comments

@D3Hunter
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

suppose we are using 2 instances to do parallel import, if they start at same time, their task ID will be the same as we use current time as the id, but we expect 2, it might fail when allocating row ids.

tidb/lightning/pkg/server/lightning.go

Line 359 in 7e9f853

taskCfg.TaskID = time.Now().UnixNano()

2. What did you expect to see? (Required)

success

3. What did you see instead (Required)

fail

4. What is your TiDB version? (Required)

master
@D3Hunter D3Hunter added type/bug The issue is confirmed as a bug. severity/moderate component/lightning This issue is related to Lightning of TiDB. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. labels Aug 13, 2024
@D3Hunter
Copy link
Contributor Author

start those instances one by one with some small interval can workaround this

@ei-sugimoto
Copy link
Contributor

I would like to tackle this program. How about using uuid?

@D3Hunter
Copy link
Contributor Author

I would like to tackle this program. How about using uuid?

lgtm

@D3Hunter D3Hunter added the good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. label Aug 19, 2024
@ei-sugimoto ei-sugimoto mentioned this issue Aug 19, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 26, 2024
@ei-sugimoto
lightning: Change how the ID of taskCfg.TaskID is generated (#55505)
ac9916a 
close #55384
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Sep 24, 2024
@ei-sugimoto @ti-chi-bot
lightning: Change how the ID of taskCfg.TaskID is generated (pingca…
5ac6a69 
…p#55505)

close pingcap#55384
@ti-chi-bot ti-chi-bot mentioned this issue Sep 24, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 24, 2024
@ti-chi-bot
lightning: Change how the ID of taskCfg.TaskID is generated (#55505) (
ed63cd2 
#56256)

close #55384
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Oct 31, 2024
@ei-sugimoto @ti-chi-bot
lightning: Change how the ID of taskCfg.TaskID is generated (pingca…
161c7d1 
…p#55505)

close pingcap#55384
@ti-chi-bot ti-chi-bot mentioned this issue Oct 31, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 4, 2024
@ti-chi-bot
lightning: Change how the ID of taskCfg.TaskID is generated (#55505) (
3378d7a 
#57018)

close #55384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/lightning This issue is related to Lightning of TiDB. good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. severity/moderate type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

lightning: Change how the ID of taskCfg.TaskID is generated ei-sugimoto/tidb
2 participants
@D3Hunter @ei-sugimoto