collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface #20504

xiongjiwei · 2020-10-19T02:41:05Z

What problem does this PR solve?

Problem Summary:

we are going to add a new collation named utf8mb4_zh_pinyin_tidb_as_cs for pinyin order, this PR is aim to add the interface for it.

What is changed and how it works?

Proposal: #19984

What's Changed:

move some constant and function to common file
a new function to avoid access collation id from mysql.collation directly

Related changes

PR in parser collation: add new pinyin collation utf8mb4_zh_pinyin_tidb_as_cs parser#1058

Check List

Tests

Unit test
Integration test

Release note

No release note

sre-bot · 2020-10-19T02:41:17Z

No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md

ti-challenge-bot · 2020-10-19T03:23:44Z

Usage:

/command [argument]

The commands are:

      /ping            ping the bot
      /pick-up      pick up the issue
      /give-up      give up the issue
      /reward score      reward the score to PR

More Details

You can find more details on ti-challenge-bot-docs.

If you think there is a problem, you can report bug on report-a-bug.

If you think the bot is dead, please contact Rustin-Liu to fix it.

xiongjiwei · 2020-10-19T04:36:51Z

/run-all-tests

xiongjiwei · 2020-10-19T07:34:56Z

/run-all-tests

# Conflicts: # go.mod # go.sum

wjhuang2016 · 2020-10-28T03:49:55Z

expression/expr_to_pb.go

-	if ok {
-		return v
+	if coll, err := charset.GetCollationByID(int(collate.RestoreCollationIDIfNeeded(c))); err == nil {
+		return coll.Name
 	}
 	logutil.BgLogger().Warn(


log the err

You should log it as a warning.

The err and the original log say the same thing and the original log have more detail. It's better to use the original code.

wjhuang2016 · 2020-10-28T03:50:41Z

expression/expr_to_pb.go

 	}
 	logutil.BgLogger().Warn(
 		"Unable to get collation name from ID, use name of the default collation instead",
 		zap.Int32("id", c),
 		zap.Int("default collation ID", mysql.DefaultCollationID),
 		zap.String("default collation", mysql.DefaultCollationName),
 	)
+


this line can be removed.

bb7133 · 2020-10-29T03:08:31Z

util/collate/pinyin_tidb_as_cs.go

+
+package collate
+
+type zhPinyinTiDBASCS struct {


What does 'ASCS' mean exactly?

I rename it to zhPinyinTiDBASCSCollator and add comments. 'ASCS' for accent-sensitive and case-sensitive

wjhuang2016

LGTM

bb7133

LGTM

bb7133 · 2020-11-03T02:24:03Z

/merge

ti-srebot · 2020-11-03T02:26:26Z

/run-all-tests

ti-srebot · 2020-11-03T02:31:24Z

@xiongjiwei merge failed.

bb7133 · 2020-11-03T02:49:58Z

/run-check_dev
/run-unit-test

wjhuang2016 · 2020-11-03T03:10:34Z

/merge

ti-srebot · 2020-11-03T03:13:11Z

/run-all-tests

ti-srebot · 2020-11-03T03:24:08Z

@xiongjiwei merge failed.

wjhuang2016 · 2020-11-03T03:24:13Z

/run-unit-test

github-actions bot added component/expression sig/sql-infra SIG: SQL Infra labels Oct 19, 2020

add utf8mb4_general_zh_ci interface

ebcccb6

xiongjiwei force-pushed the pinyin-order branch from 324046c to ebcccb6 Compare October 19, 2020 02:44

xiongjiwei added 2 commits October 19, 2020 10:47

fmt

b60a3fd

use parser method instead access collation directly

d50b27b

update parser

64059f8

xiongjiwei force-pushed the pinyin-order branch from 9a8bfca to 64059f8 Compare October 19, 2020 04:18

xiongjiwei added 5 commits October 19, 2020 15:53

Merge remote-tracking branch 'upstream/master' into pinyin-order

971101e

update parser

fbe58f6

use collate to get collation id

61b834b

change test

cecafd0

update test

bd4a76d

xiongjiwei mentioned this pull request Oct 20, 2020

collation: add new pinyin collation utf8mb4_zh_pinyin_tidb_as_cs pingcap/parser#1058

Merged

xiongjiwei added 4 commits October 22, 2020 19:03

update go.mod

ac5b989

Merge remote-tracking branch 'upstream/master' into pinyin-order

2be2d9d

# Conflicts: # go.mod # go.sum

Merge remote-tracking branch 'upstream/master' into pinyin-order

6e5972f

# Conflicts: # go.mod # go.sum

change code style

961a626

xiongjiwei changed the title ~~collation: add utf8mb4_general_zh_ci pinyin-order interface~~ collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface Oct 22, 2020

xiongjiwei marked this pull request as ready for review October 22, 2020 12:44

xiongjiwei requested a review from a team as a code owner October 22, 2020 12:44

xiongjiwei requested review from XuHuaiyu and removed request for a team October 22, 2020 12:44

xiongjiwei added 3 commits October 23, 2020 10:52

rename file name

09fe5af

move const number

4f634ea

update license

91db9f5

wjhuang2016 reviewed Oct 28, 2020

View reviewed changes

xiongjiwei added 5 commits October 28, 2020 11:58

address comments

a705702

Merge branch 'master' into pinyin-order

31c2182

rename file

7cdd8ee

remove dup log

7cb682c

Merge branch 'master' into pinyin-order

bc80c35

bb7133 reviewed Oct 29, 2020

View reviewed changes

rename collator

d32f607

wjhuang2016 reviewed Nov 2, 2020

View reviewed changes

ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 2, 2020

ti-srebot approved these changes Nov 2, 2020

View reviewed changes

bb7133 reviewed Nov 3, 2020

View reviewed changes

ti-srebot added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT2 Indicates that a PR has LGTM 2. labels Nov 3, 2020

ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 3, 2020

Merge branch 'master' into pinyin-order

46a7270

Merge branch 'master' into pinyin-order

dc9c281

wjhuang2016 merged commit f1c464a into pingcap:master Nov 3, 2020

xiongjiwei deleted the pinyin-order branch November 3, 2020 03:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface #20504

collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface #20504

xiongjiwei commented Oct 19, 2020 •

edited

Loading

sre-bot commented Oct 19, 2020

ti-challenge-bot bot commented Oct 19, 2020

xiongjiwei commented Oct 19, 2020

xiongjiwei commented Oct 19, 2020

wjhuang2016 Oct 28, 2020

xiongjiwei Oct 28, 2020

wjhuang2016 Oct 28, 2020

xiongjiwei Oct 28, 2020

wjhuang2016 Oct 28, 2020

bb7133 Oct 29, 2020

xiongjiwei Oct 29, 2020

wjhuang2016 left a comment

bb7133 left a comment

bb7133 commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

bb7133 commented Nov 3, 2020

wjhuang2016 commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

wjhuang2016 commented Nov 3, 2020

collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface #20504

collation: add utf8mb4_zh_pinyin_tidb_as_cs collation interface #20504

Conversation

xiongjiwei commented Oct 19, 2020 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Related changes

Check List

Release note

sre-bot commented Oct 19, 2020

ti-challenge-bot bot commented Oct 19, 2020

xiongjiwei commented Oct 19, 2020

xiongjiwei commented Oct 19, 2020

wjhuang2016 Oct 28, 2020

Choose a reason for hiding this comment

xiongjiwei Oct 28, 2020

Choose a reason for hiding this comment

wjhuang2016 Oct 28, 2020

Choose a reason for hiding this comment

xiongjiwei Oct 28, 2020

Choose a reason for hiding this comment

wjhuang2016 Oct 28, 2020

Choose a reason for hiding this comment

bb7133 Oct 29, 2020

Choose a reason for hiding this comment

xiongjiwei Oct 29, 2020

Choose a reason for hiding this comment

wjhuang2016 left a comment

Choose a reason for hiding this comment

bb7133 left a comment

Choose a reason for hiding this comment

bb7133 commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

bb7133 commented Nov 3, 2020

wjhuang2016 commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

ti-srebot commented Nov 3, 2020

wjhuang2016 commented Nov 3, 2020

xiongjiwei commented Oct 19, 2020 •

edited

Loading