domain: load all non-public tables into infoschema

[joechenrh]

What problem does this PR solve?

Issue Number: close #54906

What changed and how does it work?

Currently, fetchSchemasWithTables will load non-public schema. This can cause inconsistence between v1 and v2 after drop database and V1V2 consistency check has found several failed cases related to this.

Consider the following sequence of operations:

drop database --> Change to WriteOnly --> Full Load --> Change to DeleteOnly --> ApplyDiff

applyDropSchemaV2 will return all tableIDs in this deleted schema while applyDropSchema will return nothing.

tidb/pkg/infoschema/infoschema_v2.go

Lines 1246 to 1264 in ebbe53c

	func (b *Builder) applyDropSchemaV2(diff *model.SchemaDiff) []int64 {
	di, ok := b.infoschemaV2.SchemaByID(diff.SchemaID)
	if !ok {
	return nil
	}
	tableIDs := make([]int64, 0, len(di.Deprecated.Tables))
	tables, err := b.infoschemaV2.SchemaTableInfos(context.Background(), di.Name)
	terror.Log(err)
	for _, tbl := range tables {
	tableIDs = appendAffectedIDs(tableIDs, tbl)
	}
	for _, id := range tableIDs {
	b.deleteBundle(b.infoSchema, id)
	b.applyDropTableV2(diff, di, id, nil)
	}
	b.infoData.deleteDB(di, diff.Version)
	return tableIDs

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

You can test it using unit test in this PR: #55030

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[ti-chi-bot]

Hi @joechenrh. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tiprow]

Hi @joechenrh. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[wjhuang2016]

/ok-to-test

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.7243%. Comparing base (487ef06) to head (e138e53).
Report is 5 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #55142         +/-   ##
=================================================
- Coverage   72.8514%   56.7243%   -16.1271%     
=================================================
  Files          1598       1748        +150     
  Lines        444454     627838     +183384     
=================================================
+ Hits         323791     356137      +32346     
- Misses       100734     247142     +146408     
- Partials      19929      24559       +4630     

Flag	Coverage Δ
integration	39.6214% <ø> (?)
unit	72.0018% <ø> (+0.0311%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	52.9567% <ø> (ø)
parser	∅ <ø> (∅)
br	63.1563% <ø> (+17.3830%)	⬆️

[tiancaiamao]

Does it means something wrong if we see the non-public tables?

Do you have any case to cover it? or this is just 'defensive programming'

[joechenrh]

Maybe the only case is: user would see table being deleted from information_schema.tables. I'll try to construct this case.

[joechenrh]

/ok-to-test

[joechenrh]

/retest

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tiancaiamao, wjhuang2016

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/domain/OWNERS [wjhuang2016]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-08-02 02:48:32.484474875 +0000 UTC m=+57709.703236479: ☑️ agreed by wjhuang2016.
• 2024-09-04 02:48:55.436093034 +0000 UTC m=+412659.954145959: ☑️ agreed by tiancaiamao.

[tiancaiamao]

/retest

[joechenrh]

/retest

[joechenrh]

/retest

[ti-chi-bot]

@joechenrh: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/mysql-test	e138e53	link	unknown	/test mysql-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[joechenrh]

/retest