ddl: Support drop column on composite index

[blacktear23]

What problem does this PR solve?

Issue Number: close #19196

Problem Summary:

For now TiDB can drop column with single column index covered. This PR will let TiDB can drop column with composite index covered. But if the column is covered by cluster index this PR still not support this case.

What is changed and how it works?

On drop column with composite index, the step is like below:

1. Create new composite index without dropped column with temp name, and not set the index's state to StatePublic after write reorganization.
2. Drop column and column covered single column indexes.
3. At column's StateDeleteReorganization state rename temp composite index to origin name and set those indexes to StatePublic
4. Finish the job.

For tracking the new composite indexes reorg progress, there has 2 fields add in job.Model:

• NeedReorgIndexIDs string: The composite index's IDs in string separated by '|' (For example: "12|13|15")
• CurrentReorgIndexIdx int: The index for NeedReorgIndexIDs array

After introduced those 2 fields, we can reuse the create index reorg code. Just check if all indexes is reorg finished then fall-through to do the rest works. And if we reuse the create index reorg logic we can get the benefit of TxnMerge and Ingest support.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Support drop column with composite index covered.

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[ti-chi-bot]

Hi @blacktear23. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tangenta]

Just thinking, is it possible to use multi-schema change to implement this feature? Thus, we can introduce less changes:

create table t (a int, b int, index idx(a, b));
-- Query OK, 0 rows affected (0.17 sec)

alter table t drop index idx, drop column a, add index idx_1(b), rename index idx_1 to idx;

What we need to change is to lift the restriction for renaming index specifically.

[blacktear23]

For current implements I used to write it before, and the design is more friendly for cancelling and rollback. And drop column with composite index may cause create and remove more than 1 index at once. And I am not familiar with multi-schema change logic so I don't know whether it can handle thise logic.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign windtalker for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ti-chi-bot]

@blacktear23: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/mysql-test	f9d16b8	link	true	/test mysql-test
idc-jenkins-ci-tidb/build	f9d16b8	link	true	/test build
pull-mysql-client-test	f9d16b8	link	true	/test pull-mysql-client-test
idc-jenkins-ci-tidb/check_dev	f9d16b8	link	true	/test check-dev
idc-jenkins-ci-tidb/check_dev_2	f9d16b8	link	true	/test check-dev2
idc-jenkins-ci-tidb/unit-test	f9d16b8	link	true	/test unit-test
pull-integration-ddl-test	f9d16b8	link	true	/test pull-integration-ddl-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.